Accelerator VMware backup best practices 1

Accelerator VMware backup best practices 1

Today’s Challenges Volume of data to be backed up from VMware environments is generally high and backup windows are small Traditional full backups of virtual machines on VMware platform are slow as all data blocks need to be processed and backed up over the network High network bandwidth utilization during backups High I/O activity and CPU overhead within the virtual machine Accelerator could be used in NetBackup 7.5 to back up VMware virtual machine file system data using MS-Windows or standard policy type provided that the NetBackup client software was installed on the virtual machine. vStorage API or VADP based backup could not use the accelerator feature. 2

NetBackup Solution Accelerator for VMware introduced in NetBackup 7.6 uses VMware Changed Block Tracking (CBT) and sends only changed data blocks for backup. The NetBackup media server then synthesizes a full virtual machine image using data from previous backups plus data from changed blocks A full backup image of the VM is completed in approximately the time it takes to run an incremental backup. Reduces storage requirements for backup with content-aware deduplication Reduces the I/O activity and CPU load within the virtual machine. Uses less network bandwidth for backup data transfer Supports GRT for MSExchange, MSSQL and MS SharePoint 3

High Level Architectural Overview 6 7 3 1 4 2 5 4

NetBackup Process Flow and Log details Windows & Java GUI Policy / Schedule Enable / Disable CBT? CLIs bppllist, bpadm,bpschedule Changed Dependency BPDBM BPBRM Base Image Query Client Side – VM Proxy Host VMWARE bpbkar bpfis 1 VxMS 2 BPTM 3 BPTM Core xlate stat STS-H Interface 4 sts include sts write, etc STS-I Interface STSH-LIB STSI-LIB Plug-ins Plug-ins TAR Stream Handler Plugin Any OST plugin with OptSynth bpcd 1. 2. 3. 4. VMware proxy host (backup host) requests changed blocks from VM server. Merge new changed blocks with previous extents from state file. Generate Accelerator tar stream, send to bptm. Generate a full image on the storage server, using blocks from previous image and the inline data received from the client (proxy host). OST 5

Accelerator backup process flow: STEPS 1, 2, and 3 1. bpbrm starts bpfis on client (proxy host). 2. bpfis verifies that VMware server version greater than 4.0.3. If VMware version is less than 4.0.3: – This message is logged in the job entry visible in the Activity Monitor: Accelerator works with VMware Server version 4.0.3 or higher, and current version of 4.0.0 is not supported – The snapshot job fails. 3. bpfis takes snapshot, creates entries in the NBU DATA.xml file – NBU DATA.xml file is pre-existing file used during all VMware backups. – New information was added to the NBU DATA.xml file to identify the state file from the previous backup. – More information on virtualization state files on an upcoming slide. 6

Accelerator backup process flow: STEP 4a 4a. bpbrm checks with bpcd on client (proxy host) for last backup id. – For the first Accelerator backup, a previous state file will not exist: 16:50:03.567 [1879] 4 nb accl get last backup byname: get backup ids from client failed. 16:50:03.567 [1879] 4 nb accl get last backup byname: END: client param egyptvm8%20-%20w2k3,client hostname param egyptvm8%20-%20w2k3 , last backup ids . 16:50:03.567 [1879] 4 nb accl get last backup: last backup ids is empty 16:50:03.567 [1879] 2 bpbrm main: last backup ids ‘’ 16:50:03.567 [1879] 2 set job details: Tfile (27): LOG 1343685003 4 bpbrm 1879 There is no complete backup image match with track journal, a regular full backup will be performed. – In subsequent Accelerator backups the previous state file is found: 12:13:50.253 [21953] 4 nb accl get last backup byname: END: client param egyptvm8%20-%20w2k3,client hostname param egyptvm8%20-%20w2k3 , last backup ids egyptvm8%20-%20w2k3 1343684998,. 12:13:50.253 [21953] 4 nb accl get last backup: client egyptvm8%20%20w2k3, last backup ids is 'egyptvm8%20-%20w2k3 1343684998’ 12:13:50.253 [21953] 2 bpbrm main: last backup ids 'egyptvm8%20%20w2k3 1343684998' 7

Accelerator backup process flow: STEP 4b 4b. bpcd replies to BACKUP ID RQST. – For first Accelerator backup, a previous state file will not exist: 16:50:03.570 [19344] 2 process requests: BPCD GET VM CLIENT ACCL BACKUP ID RQST 16:50:03.570 [19344] 16 read vm header by name: could not open statefile /usr/openv/netbackup/online util/fi cntl/bpfis.fim.egyptvm8@20@20w2k3 0.1.0 egyptvm11.extent (2) 16:50:03.570 [19344] 16 get vm backupid by backup time: could not read header of file /usr/openv/netbackup/online util/fi cntl/bpfis.fim.egyptvm8@20@20w2k3 0.1.0 egyptvm11.extent 16:50:03.570 [19344] 2 process requests: could not get any backup id from extent file – If subsequent Accelerator backup, backupid is read from the state file: 12:13:43.799 [15979] 2 process requests: BPCD GET VM CLIENT ACCL BACKUP ID RQST 12:13:43.799 [15979] 2 process requests: last vm backup id (egyptvm8%20%20w2k3 1343684998)(egyptvm8%20-%20w2k3 1343684998) 8

Accelerator backup process flow: STEP 5,6 5. bpbrm starts bpbkar on proxy host and bptm on media server. 6. bpbkar looks for previous state file. – For first Accelerator backup, a previous state file will not exist, and the backup continues down the normal Flashbackup VMware code path and will look like a normal full backup: 16:50:04.348 [19352] 2 stage one virtual: could not find previous extent file (./flashmap.cpp:3753) – Subsequent Accelerator backups will find the state file: 12:13:51.576 [15988] 2 stage one virtual: previous extent file /usr/openv/netbackup/online util/fi cntl/bpfis.fim.egyptvm8@20@20w2k3 1343684973.1.0 egyptvm11.extent (./flashmap.cpp:3751) The state file is found in the following path: [install path]/netbackup/online util/fi cntl/bpfis.fim. [client name] [backup time].1.0 [storage server].extent 9

Accelerator backup process flow: STEP 7 7. During subsequent Accelerator backups, bpbkar reads data from state file (reads a list of extents that composed the last backup). 12:13:51.725 [15988] 2 stage one virtual: Read accl extent state file pathname /usr/openv/netbackup/online util/fi cntl/bpfis.fim.egyptvm8@20@20w2k3 1343684973.1.0 egyptvm11.extent (./flashmap.cpp:3976) 12:13:51.725 [15988] 2 read state extent file: read State Extent File backupid egyptvm8%20-%20w2k3 1343684998 (./flashmap.cpp:5052) – fiml set fvv table reads list of all extents and only changed blocked extents from VxMS representing the VM as it exists currently, and merges extents with changed data into a full list of extents that shows the entire VMDK, highlighting those extents that have been changed. – fiml set fvv table uses the previous image’s extents (read from the state file) to update the extents that haven’t been changed – and identifies where the extents can be found in the previous image. 1 0

Accelerator backup process flow: STEP 8 8. As the Accelerator backup continues. – write vm files extents ACCL writes a tar stream to bptm using the extents formed during fiml set fvv table. – Previous image extents: 12:14:38.028 [15988] 2 write vm files extents ACCL: Fast Backup processing extent 0 VMDK (./flashmap.cpp:4756) 12:14:38.079 [15988] 2 write accl hdr ex: PREVIOUS IMAGE DATA without include checksum (./flashmap.cpp:1207) 12:14:38.079 [15988] 2 put acclhdr: PREVIOUS IMAGE DATA header offset: 36928000, num bytes: 524288, vmdk off: 0, verify len:131072 (./flashmap.cpp:1268) – New (inline) data: 12:14:38.079 [15988] 2 write vm files extents ACCL: Fast Backup processing extent 1 VMDK (./flashmap.cpp:4756) 12:14:38.079 [15988] 2 put acclhdr: Fast Backup put IN LINE DATA header num bytes: 327680 (./flashmap.cpp:1291) 12:14:38.079 [15988] 2 write vm files extents ACCL: Accelerator writing inline data (./flashmap.cpp:4779) 12:14:38.079 [15988] 2 write extent data: Accelerator write incr data, offset 524288 length 327680 (./flashmap.cpp:5344 1 1

Accelerator backup process flow: STEP 9,10 and 11 9. For all Accelerator backups (first and subsequent): – bpbkar writes the state file for the backup. 16:56:31.088 [19352] 2 write state extent file: write State Extent File create file name /usr/openv/netbackup/online util/fi cntl/bpfis.fim.egyptvm8@20@20w2k3 1343684973.1.0 egyptvm11.extent (./flashmap.cpp:4988) 10. bptm writes inline data to storage server. Uses Accelerator technology to create a traditional full NetBackup image. 11. bpbkar reports optimization percentage to Job Details. 12:16:46.983 [15988] 4 bpbkar main: JBD - accelerator sent 581953536 bytes out of 7567738880 bytes to server, optimization 92.3%

Policy Configuration Set the Policy type to VMware. The Accelerator for VMware feature is configured using the Use Accelerator checkbox in the policy Attributes tab. The storage unit must support optimized synthetics, limiting the storage unit options to MSDP, PDDO, and certain 3rd-party OST devices. If Use Accelerator check-box is selected, NetBackup automatically verifies that the selected storage unit is supported for Accelerator backups. If the Use Accelerator check-box is greyed out and unselectable, verify that the NetBackup Data Protection Optimization Option license is installed. 1 3

VWware Policy BLI Setting When Accelerator is enabled, the Enable block-level incremental backup parameter setting is automatically selected. Automatically selected when Use Accelerator is selected. Cannot be de-selected. 1 4

Best Practices and known issues Symantec recommends not to enable Expire after copy retention for any storage units that are used with storage lifecycle policies (SLP) in combination with Accelerator Update the NetBackup device mapping files if needed Storage unit groups are supported only if the storage unit selection in the group is Failover. Accelerator requires the storage to have the OptimizedImage attribute enabled. If a backup of the virtual machine does not exist, NetBackup performs a full backup (non accelerator) for the first time and accelerator enabled backups subsequently. To protect against any potential omissions by underlying VMware CBT, run Accelerator forced rescan option enabled backups from time to time to establish a new baseline 1 5

Troubleshooting Issue: “Use Accelerator” checkbox not selectable If “Use Accelerator” policy checkbox is grayed out, and not selectable – This is caused by lack of a proper license. – Add the NetBackup Data Protection Optimization Option. – You can obtain a list of the currently active features using the command: bpminlicense -list keys -verbose “output file name” (This command reads a verbose list of NetBackup licensed features and writes the output to a specified file name.) 1 6

Troubleshooting Issue: Policy creation failure Creation of a policy with “Use Accelerator” may fail due to the selection of an invalid storage unit. NetBackup validates the selected storage unit, and posts the message (above) if an invalid storage unit is selected. Valid types are MSDP, PureDisk, Symantec cloud, and various third party OST storage servers. If using a third-party storage server that has been validated for use with Accelerator, verify device mappings are up-to-date. 1 7

Troubleshooting Issue: Low or no optimization for Accelerator backups Possible causes for 0% optimization – First backup of VM using Accelerator. From Job Details: 08/03/2012 09:26:29 - Info bpbrm (pid 17147) There is no complete backup image match with track journal, a regular full backup will be performed. – Accelerator forced rescan is enabled in the policy schedule. From Job Details: 08/03/2012 09:41:30 - Info bpbrm (pid 18875) Accelerator enabled backup with "Accelerator forced rescan", each file will be read in order to validate checksums. Backup time will be longer than a normal Accelerator enabled backup. – Storage server was changed after previous backup. This situation looks the same as a first backup in the Job Details. Possible cause for low optimization High rate of change on the virtual machine. 18

Troubleshooting Issue: Snapshot or backup failures Accelerator snapshot failures – Unsupported VM server. In Job Details look for: 13:51:11.973 [30188] 2 onlfi vfms logf: INF - VMware freeze: Accelerator works with VMware Server version 4.0.3 or higher, and current version of 4.0.0 is not supported. – The VMware server must be upgraded to version 4.0.3, or higher. – For other snapshot errors, look in bpfis log. Accelerator backup errors. Look in bpbrm, bpcd, bpbkar, bptm and VxMS logs. More information on configuring VxMS logging for NetBackup 7.6 is available in the following technote: http://www.symantec.com/docs/HOWTO70905 19

NetBackup logs for Accelerator for VMware Accelerator does not require its own log directory. For log messages about Accelerator, see the following standard NetBackup log directories. LOG Directory Location UNIX: /usr/openv/netbackup/logs/bpbrm Windows: install path\NetBackup\logs\bpbrm NetBackup master or media server UNIX: /usr/openv/netbackup/logs/bptm Windows: install path\NetBackup\logs\bptm NetBackup media server UNIX: /usr/openv/netbackup/logs/bpbkar Windows: install path\NetBackup\logs\bpbkar VMware Backup host UNIX: usr/openv/netbackup/logs/bpfis Windows: install path\NetBackup\logs\bpfis VMware Backup host Linux: /usr/openv/netbackup/logs/vxms Windows: install path\NetBackup\logs\vxms VMware Backup host 2 0

Top Technotes Symantec NetBackup 7.6 for VMware Administrator's Guide http:// www.symantec.com/docs/DOC6461 Accelerator notes and requirements for virtual machines http ://www.symantec.com/docs/HOWTO92075 About the NetBackup Accelerator for virtual machines http ://www.symantec.com/docs/HOWTO92076 Accelerator messages in the backup job details log http ://www.symantec.com/docs/HOWTO92077 Accelerator forced rescan for virtual machines http:// www.symantec.com/docs/HOWTO92079 Support for NetBackup 7.x in virtual environments http:// www.symantec.com/docs/TECH127089 2 1

Thank You! 2 2