Avamar Data Migration Enabler Adam Kirkpatrick – Last Update March
19 Slides3.67 MB
Avamar Data Migration Enabler Adam Kirkpatrick – Last Update March 1, 2022
Overview this content taken from ADMe Supplemental Material ADMe - Avamar Data Migration enabler is a utility providing automation for the migration of Avamar backup data to a different storage medium such as tape, disk, cloud based storage, or to another Avamar/DD. Backup data is first rehydrated to a staging disk area into a structured top-level folder layout for organizational purposes. From there, it can automatically be backed up to a tape backup application as a standard file system backup or to another Avamar for purposes of placing the backup data into its Data Domain. When staging to cloud enabled storage including ECS or ATMOS or any third-party S3 object storage system using an appropriate gateway such as Geo Drive or TNT Drive, files are automatically uploaded to the cloud storage provider directly rather than passing it through a tape backup application. The ADMe migration process is comprised of three distinct phases, client backup selection to identify the backups to act on, backup recovery or staging the backup data to disk followed by an optional export to an external tape backup application or cloud synchronization phase. The end-to-end workflow is fully automated by ADMe Job Policies eliminating the need for manual intervention or localized scripting. ADMe Job policies can be scheduled to run automatically or run on demand as needed. ADMe is compatible with all Avamar systems including single or multi-node GRID systems, source or replication target systems, IDPA and those using Data Domain storage. Several tape backup applications refer to compatibility guide for details, are supported and leveraged in a consistent manner for purposes of automation. Performance varies based on the infrastructure involved and the migration strategy used incremental or non-incremental. Nonincremental typically averages 50-150 GB/Hr per migration thread during the rehydration phase with slightly higher rates when file counts are low or larger files are involved. Incremental staging is effective against large file system or NDMP backups only and can provide effective staging rates up to 500 GB/Hr with a single staging thread. Multiple staging servers can be leveraged where each can support multiple staging threads all of which can be leveraged concurrently to increase the aggregate throughput. The tape backup phase performs at the tape applications normal rate for a file system type backup. ADMe – Avamar Data Migration Enabler is a utility providing a framework for migrating Avamar backup data to a different storage medium such as tape, disk a target Avamar or cloud based storage of2Y Copyright 2021 Dell Inc.
Features Easy to use CLI interactive Menu driven UI Single point of management No dependency on Avamar when recovering from tape or cloud (Web based UI is almost here) (exceptions Oracle or PAX archives) Batch driven migration process Seamless support for different staging platform OS types Supports migrating Avamar backup contents in Supports all Avamar system types Automated export backup initiation bulk to Hybrid cloud storage via various gateways Email notifications & job logs capture all migration activity Incremental & non-incremental migration Cataloged tape backups strategies Supports all popular Avamar plug-in types Indexed and searchable cloud copy File level recoveries directly from tape or cloud Supports multiple staging servers and Extensive backup selection filters optionally multiple migration threads per server all of which can be used concurrently Recoveries visible in Avamar Activity Monitor of3Y Copyright 2021 Dell Inc.
Export to Tape Workflow (Networker Shown) Avamar Server (ADS or AV/DD) ROBO Migration Workflow Using Networker Networker / Staging Server Networker / Tape App FS Client Agent Avamar FS CLient Desktops Laptops Local ADMe staging disk file systems Additional Staging Servers Avamar FS Client Networker / Tape FS Client Agent Local ADMe staging disk file systems Files NAS/NDMP Networker 64 Bit Linux FS Client DB VMware & Hyper-V Databases Email Applications Automated Tape Library Tape/VTL/DD/Cloud/Disk of4Y Copyright 2021 Dell Inc. Pre-defined Networker policy points backup to designated target storage using appropriate retention. Technologies Legend EMC Networker Avamar
ADMe Avamar GSAN/DD - Avamar/DD Workflow Staging server activated to source Avamar only Staging Server Desktops Laptops Files Avamar Client Local NAS/NDMP Storage Volumes DB ROBO VMware & Hyper-V Databases Email Applications Source Avamar Server (ADS or AV/DD or IDPA) Recovery Experience Almost business as usual for fiel system backups Uses target Avamar Recovery UI File level granularity Each migrated backup browse-able at the client level and its original backup date displayed as a folder Files retain their original timestamps Recoveries can be to any available activated Avamar client Plug-IN’s recoveries require using ADMe documented procedural recovery process of5Y Copyright 2021 Dell Inc. Migration Workflow Migrated backups will be stored on target Avamar Data Domain as a net new backup Expiry times passed as part of the ADMe migration job policy criteria Incremental staging can be leveraged in suitable scenarios DD-Boost leveraged during Target Avamar Server ingest to target Data Domain With Data Domain Supports multiple concurrent Recovery UI migration threads Automated end-to-end workflow Migrations can be performed before/during/after activating production clients to new target system
Staging Method - Non-Incremental CONCEPTS Referred to as BYDATE as it establishes a unique top-level folder representing the point-in-time of the original Avamar backup formatted as yyyy-mm-dd-hh-mm-buid# Performs a full recovery of all files contained in the backup Provides separate point-in-time copy for each backup migrated Effective with both Tape, Cloud targets and ADS migrations Can be used with all backup types Staging disk is automatically erased between migration sessions REQUIREMENTS Minimizes staging disk space required Must be large enough to accommodate size of largest individual backup From a practical sense, at least 4 TB’s should be available per staging thread For a Windows staging server, multiple staging drive letters are required in order to leverage concurrent migration threads PERFORMANCE Expect throughput of 50-150 GB’s/Hr per thread No dependency on file change rates between migration sessions Throughput impacted by high file counts and/or slow infrastructure Tape media consumption based on data size and its ability to compress and model of tape drive used of6Y Copyright 2021 Dell Inc.
Staging Folder Hierarchy - NonIncremental Destination staging path automatically includes Avamar source client names and point-in-time folders Point-in-time folders ADMe establishes the user defined top level folder structure along with the source client names involved Insert keyword BYDATE prior to the source client name Insert point-in-time folder YYYY-MM-DD-HH-MM-buid# represents the migrated backup original Avamar timestamp and associated BUID# User data will be present under the point-in-time folder of7Y Copyright 2021 Dell Inc.
Staging Method - Incremental CONCEPTS Overwrites previously staged path name Point in time folder is removed with incremental staging Performs timestamp comparison prior to moving the file across the wire from current backup with the same previously staged file name Only net new or updated files are moved Deleted files remain on staging disk until it is manually formatted Provide single instance at the file level on the staging disk Effective for Tape and Cloud targets Benefits File System or NDMP Plug-IN types only REQUIREMENTS Incremental staging is relative to previously staged data, 1 st stage is always a full Requires one copy of staged data remain staged between migration sessions Therefore, staging disk must be dedicated Minimum staging disk space required equivalent to one copy of client BU data involved For an incremental tape-backup, incremental staging is a prerequisite PERFORMANCE Effective throughput gains versus non-incremental usually 3 to 6 times Expect 250-600 GB’s/Hr effective staging throughput Highly dependent on % incremental change rate & total number of files involved Comparable performance gains achieved for incremental tape backup phase (optional) Usually 70% reduction in tape media usage of8Y Copyright 2021 Dell Inc.
Staging Folder Hierarchy - Incremental Incremental destination staging path automatically includes Avamar source client names but NO point-in-time folder A prereq of incremental is the need for a consistent path name. As a substitute for the lack of a point-in-time folder a txt file is provided containing the details of a given migration including that of date/time/buid# etc. ADMe establishes the user defined top level folder along with the source client name involved ADMe Incremental relies on a consistent path name to a client's data therefore a point-in-time folder cannot be used To replace the point-in-time folder, a text file named ADMe-Incinfo is provided containing details of the migration including metrics such as date/time/BUID# of backups migrated An effective incremental tape backup can be accomplished from incrementally staged BU’s of9Y Copyright 2021 Dell Inc.
Tape Application Recovery UI View (Networker shown) BYDATE Non-Incremental Its point-in-time folder represents its original Avamar backup timestamp INCREMENTAL Incremental A common destination folder is used, tape export application provides the effective point-in-time reference File Recovery From Tape Choose desired backup by date from applicable staging server Navigate to staging path and source client folder name Mark files/folders to be recovered as a directed recovery from tape of 10Y Copyright 2021 Dell Inc.
Questions? of 11Y Copyright 2021 Dell Inc.
User Interfaces CLI Interactive Menu UI Interactive Menu Driven All Functionality Available Color Coded Displays Easy Access to Logs Online Help Automated Upgrade Process ATO Rocks! New Web based UI Most Common Functionality Available Job Policy Setup, Scheduling & Monitoring Easy Access to Job Logs Login via MCS authentication of 12Y Copyright 2021 Dell Inc.
Step-1 ADMe Architectural Overview Establish environment files which define the staging server names or staging thread name where you intended to stage your backup data to. These can be shared by different client groups and job policies or they can be dedicated if appropriate. Environment File Definitions Staging Server Name Staging Server Type Export Application Name Export End-of-Year Month-ID’s Mail-Notification Address’s ADMe Control variables Avamar & Tape Binary paths Step-3 A Job policy definition references an environment file by its numeric number, a client group name, backup selection and migration criteria customizing the migration process based on user requirements. Job Policy Definitions Environment-File-Number Client-Group-Name Backup Selection Criteria Backup Migration Criteria Client Group Definitions Group-Name Client domain/name Staging Path Tape-Export-Policy Staging Server OS type Step-2 Establish client group(s) by giving it a name and adding the desired clients to it. A client group can contain any number of clients, but their aggregate size must not exceed its defined target staging path. Client groups are assigned to a job policy definition to be acted on. of 13Y Copyright 2021 Dell Inc. Execute the Job
Environment Files Tool Bar Functions Environment File Purpose Defines the physical infrastructure available to ADMe including staging server names & OS type, export tape application, binary paths, temporary workspace Contains several customizable operational variables Defines the mail notification recipients of 14Y Establish a new Environment file Update an Existing Environment file Remove an Existing Environment file Save an inline edit change to an Existing Environment file Display and Email an Environment Summary report Refresh filters and Environment display contents Help information Copyright 2021 Dell Inc.
Environment File Creation Click on Environment Tab Click on tool to create a new Environment file Click on Staging Server button Choose the desired staging server from the client tree displayed of 15Y Copyright 2021 Dell Inc.
WEB-UI – Client Configuration Manager Create a Client Group Tape Policy fields Browse-Able Domain/Client selection tree Add/Remove multiple clients at a time View a client group contents are field values Support disable/enable/modify existing client group entries Server initiated Tape policy syntax is automatically inserted for Fld-12 month to month and Fld-14 and Yearly Fld-13 exports keying of your entered client group name of 19Y Copyright 2021 Dell Inc.
WEB-UI – Standard Job Policy Creation Job Policy Setup Wizard is split into two panels of 20Y Copyright 2021 Dell Inc.
WEB-UI – Batch Job Policy Creation Batch Policies are used to schedule a series of Standard policies sequentially to a given thread number Simplifies scheduling when multiple jobs are required to share the same resources, staging path, thread# etc of 21Y Copyright 2021 Dell Inc.
Job Policies Connect the Dots To Perform the Migration process! of 22Y Copyright 2021 Dell Inc.