EMC Avamar Backup Solutions for VMware ESX … Avamar Backup Solutions for VMware ESX Server on...

EMC Avamar Backup Solutions for VMware ESX Server on Celerra NS Series

Applied Technology

Abstract

This white paper discusses various backup options for VMware ESX Server deployed on Celerra® NS Series storage using EMC® Avamar Virtual Edition for VMware. The paper highlights how each option works and the benefits of utilizing EMC Avamar to perform backup in a VMware environment.

November 2008

Copyright © 2008 EMC Corporation. All rights reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com

All other trademarks used herein are the property of their respective owners.

Part Number h5897

EMC Avamar Backup Solutions for VMware ESX Server on Celerra NS Series 2 Applied Technology

Table of Contents

Executive summary.............................................................................................4 Introduction .........................................................................................................4

Audience.......................................................................................................................................4 Terminology ..................................................................................................................................4

Technology overview..........................................................................................5 EMC Avamar ................................................................................................................................5 VMware Consolidated Backup .....................................................................................................5 Avamar VCB Interoperability Module ...........................................................................................6 Test environment ..........................................................................................................................6

Reference architecture..............................................................................................................6 ESX Servers..............................................................................................................................7 Storage......................................................................................................................................8 Storage layout ...........................................................................................................................8 Network architecture ...............................................................................................................10 Avamar Virtual Edition configuration.......................................................................................10 EMC NetWorker configuration ................................................................................................11

Test methodology .......................................................................................................................11 Backup source ............................................................................................................................12 Tested scenarios ........................................................................................................................12

VCB image-level backup.........................................................................................................13 VCB file-level backup..............................................................................................................14 Guest-based backup...............................................................................................................15 Guest-based incremental backup ...........................................................................................17 Service Console-based backup ..............................................................................................17 Restore....................................................................................................................................18 Summary.................................................................................................................................19 Comparison of backup methods .............................................................................................21

Avamar requirements and best practices .......................................................21 Conclusion.........................................................................................................22 References.........................................................................................................22 Appendix A: Avamar configuration for VCB backup .....................................23


Executive summary VMware Infrastructure is the industry’s most widely deployed virtualization solution. Virtual machines deployed in the data center must be protected against failure. Extending data protection to virtual machines is thus an important function. In the virtualized environment provided by VMware Infrastructure, there are many ways to improve the convenience and reliability of data protection, each with its particular advantages and challenges.

EMC® Avamar software is a backup and recovery software with an important advantage. Avamar’s source-based data de-duplication technology eliminates the unnecessary transmission over the network and storage of redundant backup data. This de-duplication slows the pace of data growth both in core data centers and in remote offices. Avamar is especially good in areas where traditional backup solutions such as virtual machines, remote offices, and large LAN-attached file servers struggle. EMC Avamar Virtual Edition for VMware is specifically designed to accelerate and simplify virtual machine backup and eliminate the need for a dedicated backup server infrastructure. This paper discusses details of the Avamar solution and various ways of providing data protection.

Introduction Avamar software quickly and efficiently protects VMware Infrastructure environments by reducing the size of the backup data within and across virtual machines using agents in the virtual machines, on the VMware Consolidated Backup Server, or in the ESX Server service console. For virtual machine backups, Avamar eliminates traditional backup bottlenecks caused by the large amount of data that must pass through the same set of shared resources — the physical server’s CPU, Ethernet adapter, memory, and disk storage.

Avamar brings benefits whether an organization performs backups using Avamar in the virtual machines directly, or using ESX Server backup methodology. Unlike traditional backup solutions, Avamar can de-duplicate the data stored in virtual disks (.vmdk files). This paper describes the benefits of the Avamar de-duplication technology in VMware environment deployed on EMC Celerra® NS Series storage. The solutions listed in this paper are generic and apply to almost all configurations regardless of the storage layout, RAID type, and volume and file system management.

Audience This white paper is intended for EMC customers, partners, and service personnel involved in planning, architecting, or administering a VMware environment with EMC Celerra as the storage and looking forward to implement backup and restore solutions with EMC Avamar software. It is assumed that readers have working knowledge of VMware ESX Server and Celerra.

Terminology Avamar Server: An Avamar server is a logical grouping of one or more nodes that is used to store and manage client snapups.

Avamar Virtual Edition (AVE): AVE is a single-node Avamar server that runs as a virtual machine in a VMware ESX Server. It integrates the Avamar software with Red Hat Enterprise Linux as a VMware virtual machine.

Avamar Client: An Avamar client is a computer or workstation running Avamar software and accessing the Avamar server through a network connection. Avamar client software comprises a client agent and one or many plug-ins.

Dataset: Dataset is a policy that defines a set of files, directories and file systems for each supported platform that will be included or excluded in snapups across a group of clients. A dataset is a persistent and reusable Avamar policy that can be named and attached to multiple groups.


Datastore: Datastore is a either a VMFS or NFS file system, that serves as a virtual representation of an underlying pool of physical storage resources. These physical storage resources can consist of SCSI disks from a local server, Fibre Channel SAN disk arrays, iSCSI SAN disk arrays, or NAS storage arrays.

Group: Group is a level of organization in Avamar Administrator for one or more Avamar clients. All clients in an Avamar group use the same group policies (dataset, snapup schedule, and retention policy).

Node: An Avamar node is a self-contained rack-mountable network-addressable computer that runs Avamar server software on the Linux operating system.

Raw Device Mapping (RDM): RDM includes a combination of a pointer, which is a .vmdk file that resides on a VMFS volume, and a physical raw device that the .vmdk file points to. RDM can either be in a physical compatibility mode or virtual compatibility mode.

Snapup: Snapup is a point-in-time copy of client data that can be restored as individual files, selected as directories or as entire file systems. Although more efficient than a conventional incremental backup, a Snapup is always a full, stand-alone copy of client data that can be restored immediately from an Avamar server.

VMware Snapshot: VMware Snapshot is a function provided by VMware to capture the entire state of a virtual machine at the time the snapshot is taken.

Technology overview This section describes EMC Avamar, VMware Consolidated Backup, and Avamar VCB Interoperability Module.

EMC Avamar EMC Avamar is backup and recovery software. Avamar’s global data de-duplication technology eliminates the unnecessary transmission over the network and storage of redundant backup data. This de-duplication slows the pace of data growth both in core data centers and at remote offices. Avamar is especially good in areas where traditional backup solutions such as virtual machines, remote offices, and large LAN-attached file servers struggle.

Avamar solves traditional backup challenges by reducing the size of backup data at the source. Avamar does this using patented global data de-duplication technology that stores only a single copy of sub-file data segments across all sites and servers. Avamar de-duplicated backups function like full backups and can be recovered in just one step. In addition, Avamar verifies backup data recoverability.

VMware Consolidated Backup VMware Consolidated Backup (VCB) allows a virtual machine backup at any time by providing a centralized backup facility that leverages a centralized proxy server and reduces the load on production ESX Server hosts. VCB can perform full image-level backup for virtual machines running any operating system and file-level backups for virtual machines running Microsoft Windows.

VMware ESX Server 3.5 supports two modes of VCB. The two modes are SAN Mode and LAN Mode. SAN Mode is used when the ESX Server uses Fibre Channel SAN or iSCSI SAN to store its virtual machine disks. LAN Mode is selected when the ESX Server uses NAS and local storage devices such as direct-attached storage (DAS).


Avamar VCB Interoperability Module This interoperability module enables VMware Consolidated Backup (VCB) integration with EMC Avamar software. Integration is achieved by using pre- and post-scripts for snapping virtual machines, mounting and unmounting virtual disk images on a Windows 2003 proxy. Scripts support full VMDK image level backups for all virtual machine types and file level backup of Windows virtual machines.

Test environment This section describes the test environment where the various Avamar backup solutions were implemented.

Reference architecture Figure 1 shows the reference architecture of the overall environment.

Figure 1 Reference architecture


VCB Proxy Server

The VCB Proxy Server used was a Dell PowerEdge 1750 (1).

Table 1 lists the hardware and software components in the VCB Proxy Server.

Table 1 VCB Proxy Server

Dell PowerEdge 1750 (1)

Hardware

Component Quantity and description

2.8 GHz EM64T Single-core Intel Xeon processors

1

Memory 4 GB Intel PRO/1000 GbE NICs 2 (1 used) 73 GB 10k rpm internal SCSI disks 2 (1 used)

Software


Dell firmware A06 Microsoft Windows Server 2003 Enterprise Edition

Service Pack 2 for 32-bit x86

VCB Module Version 1.1 Avamar client Version 3.7.1 EMC NetWorker® client Version 7.4

ESX Servers Two VMware ESX Server hosts were configured. At the backup source, a VMware ESX Server running three guest Windows 2003 Server operating systems was installed. Hereafter, this ESX Server is referred to as the Production ESX Server in this document. Another VMware ESX Server was configured to host two virtual machines. One virtual machine was running the RHEL3 U8 operating system and Avamar Virtual Edition 1.0. The other virtual machine was running Windows 2003 Server and EMC NetWorker 7.4. Hereafter, this ESX Server is referred to as the Backup ESX Server in this document. Table 2 summarizes the hardware and software configuration of the ESX Servers.

Table 2 ESX Servers

Dell PowerEdge 2950s (2)

Hardware


3 GHz EM64T dual-core Intel Xeon processors supporting Virtual Technology

2

Memory 16 GB Intel PRO/1000 GbE NICs Two onboard 10/100/1000 MB Ethernet NICs (used for virtual machine

public network) One onboard 10/100/1000 MB Ethernet NIC (used for Service console) Two additional dual port 10/100/1000 MB Ethernet NICs (Used for VMkernel network)


Dell PowerEdge 2950s (2)

Hardware


73 GB 10k rpm internal SATA disk 1 (Production and Backup ESX Server OS and Backup ESX Server guest OS)

Software


Dell firmware A07 Windows 2003 Enterprise Edition with SP2

x86-32-bit

Red Hat Enterprise Linux RHEL3 U8 Avamar Virtual Edition for VMware Release 1.0 EMC NetWorker Server 7.4 VMware Source and Backup ESX Server 3.5.0 build 64607

Storage A single Celerra NS40 storage array was used in the test configuration with an Active-Active configuration. Data Mover 2 was used to manage storage for the guest virtual machines that needed to be backed up using Avamar, and also storage for the VCB Proxy Server to hold backup data temporarily. Data Mover 3 was used to manage storage as a backup-to-disk destination. Table 3 lists the components in the storage array.

Table 3 Storage array

EMC Celerra NS40

Base system


Data Movers 2 (active-active) GbE NICs 4 per Data Mover

Control Station 1 DART Version 5.6.36-2

FC shelf


FC shelf 2 FC disks 20 disks of 15k rpm and 146 GB SATA shelf 1 SATA disks 15 disks of 10k rpm and 500 GB FC RAID groups 1 RAID 5 (4+1) for Celerra system LUNs

3 RAID 5 (4+1) hosting guest VMs

SATA RAID groups 2 RAID 5 (6+1) as backup-to-disk storage

Storage layout Figure 2 illustrates storage configuration for the testing.


Figure 2 Storage layout

RAID 5 storage groups were striped together using Celerra Automatic Volume Management (AVM) to provide a sufficient number of disk spindles for both performance and capacity purposes.

Table 4 lists the file systems and iSCSI LUNs created for testing.

Table 4 File systems

File System

Connection Disk type Disk volumes Use

/vmdatastore NFS FC d13, d20, d15 To back up Guest VMs and files /proxyfs iSCSI FC d13, d20, d15 Temporary storage on Proxy to hold snaps of

the data to be backed up

/backupfs iSCSI SATA d16, d22 Backup-to-disk destination


Network architecture Network configuration for the solution stack was structured as shown in Table 5.

Table 5 lists the details of both physical and virtual networks.

Table 5 Network configuration

Component NIC Use

VCB Proxy Server (Dell PowerEdge 1850)

One on-board GbE Client/driver VMware ESX Servers (Dell PowerEdge 2950)

ESX Physical Networking

vmnic0 ESX Server Service Console network vmnic4 vmnic5

Teamed NICs connecting to storage network

vmnic2,Vmnic3 Virtual machine public network

ESX Virtual Networking

vSwitch0 Service console port (using vmnic0) vSwitch1 VMkernel port (using vmnic4 and vmnic5) vSwitch2 Virtual machine port (using vmnic2) vSwitch3 Virtual machine port (using vmnic3)

Celerra NS40 Data Movers

eth0 eth1 eth2

Not Used

eth3 Storage network

The following VLANs were configured in the test environment:

• Client VLAN: Client VLAN controls and manages the ESX Servers • Storage VLAN: Storage connects the VCB Proxy Server, ESX Server hosts, and Celerra using either

the NFS protocol or iSCSI protocol Each ESX Server connected to the storage VLANs has two physical NICs dedicated to the storage VLAN. The two NICs were teamed together to provide load balancing and failover functionality.

Avamar Virtual Edition configuration Avamar Virtual Edition (AVE) is a single-node Avamar server that runs as a virtual machine in a VMware ESX Server environment. It integrates the Avamar software (3.7.1) with Red Hat Enterprise Linux RHEL3u8 in a virtual machine. AVE is available in two configurations: 0.5 TB and 1 TB licensed capacity. The testing has been done on VMware ESX Server version 3.5 with an AVE 0.5 TB license.

The 0.5 TB AVE was installed into four partitions: one operating system partition of 35 GB and three storage partitions of 250 GB each. Before installation of Avamar Virtual Edition (AVE), a benchmark test must be run to ensure that AVE runs in an environment with acceptable I/O performance and also to stress virtual machine resources to determine whether the impact of running AVE is acceptable to other applications on the Backup ESX Server.


Table 6 shows the values achieved after conducting the standard benchmark tests.

Table 6 Avamar benchmark results

Conditions File Server data value

Mixed data value

Tested data value

Total Read throughput 60 MB/s 75 MB/s 86 MB/s Total Write throughput 30 MB/s 60 MB/s 122.67 MB/s Total Seek throughput for 4 threads 400 seeks/sec 400 seeks/sec 2380 seeks/sec

For more information on the benchmark, refer to Avamar Virtual Edition Performance Assurance Tool available on Powerlink®.

EMC NetWorker configuration The EMC NetWorker server was configured as a virtual machine in the Backup ESX Server. It contained one operating system partition of 35 GB and one storage partition of 250 GB.

Test methodology Tests were developed to: • Validate that Avamar performs source-based de-duplication and understand the benefits resulted from

such de-duplication in a backup process • Examine the different options of performing Avamar backup in a VMware environment and their

respective impact on backup • Conduct traditional backup using EMC NetWorker without Avamar and compare it with Avamar

backup to identify key differences Data de-duplication ratios depend on the type of data being de-duplicated. For our testing, the backup source contained cloned virtual machines with regular text files within the virtual machines. The primary goal was to illustrate the impact of source-based de-duplication during backup, rather than finding out the level of de-duplication ratio that can be minimally or maximally achieved by Avamar. To understand the impact and benefits of using Avamar, a set of performance matrices was defined and performance data was collected accordingly for all test cases during testing. Most of the monitoring and operational tasks in the testing were carried out through the VMware VirtualCenter Management Server. ESX Server performance statistics were obtained through esxtop, Proxy Server performance statistics were monitored using perfmon, and Data Mover CPU utilization was collected on Celerra Control Station. These performance counters included: • Storage usage – the actual amount of storage in GB used for the backup job • Backup time – the actual amount of time in seconds taken to complete a backup job • On Production ESX Server, Backup ESX Server and VCB Proxy Server:

Average network throughput – the average throughput in MB/s achieved during the backup Max network throughput – the maximum throughput in MB/s achieved during the backup Average CPU utilization – the average CPU utilization on ESX Servers or Proxy Server during the

backup


Average memory utilization – the average memory utilization on ESX Servers or Proxy Server during the backup

Average Data Mover (DM) CPU – the average CPU utilization on the Data Mover during the backup

Backup source A common virtual machine environment and data set was created for each option of Avamar backup. The common environment is shown in Figure 3.

Figure 3 Backup source The three guest virtual machines that needed to be backed up were created using a common template. Each virtual machine had two virtual disks, one 5 GB disk for guest OS residing on NFS Datastore, the other 20 GB disk containing data also on NFS Datastore. Both disks were filled up with files that were created using a utility tool that populated a directory tree with unique files. In our testing, the data disk contained 1,000 unique 20 MB files.

As unique files were being created, we expected a minimal data de-duplication ratio when a virtual machine was backed up individually using Avamar. However, as the three virtual machines were created using the same template, we expected to observe a higher data de-duplication ratio when the three virtual machines are backed up together. Again, our testing intended to identify the impact of data de-duplication on backup, rather than the level of de-duplication ratio that can be achieved.

Tested scenarios A total of six test scenarios were conducted, which include VCB image-level backup, VCB file-level backup, guest-based backup, guest-based incremental backup, Service Console-based backup, and restore.


VCB image-level backup To perform a VCB image-level backup, the Avamar agent, Avamar VCB Interoperability Module (AVIM), and the VCB software itself have to be installed on the VCB Proxy Server. VCB software is downloadable from the VMware website (www.VMware.com) at no cost. However, the VCB license needs to be purchased and enabled on the Production ESX Server or VirtualCenter (VC) Server.

After all three software components are installed, the VCB configuration file config.js located in the Proxy Server directory <VCB installed path\config> needs to be modified before the first backup can be taken. This file contains the comments that define each parameter for Avamar backup. After initiating the backup job from Avamar, VCB first retrieves configuration files and virtual disks to its local directory before Avamar copies them to the backup destination. Once the job is successful, Avamar removes the duplicate copy on the VCB Proxy Server. This type of backup can be performed on any guest OS and de-duplication happens at the .vmdk level. Appendix A: Avamar configuration for VCB backup presents a procedure on how to configure Avamar to perform image-level backup.

Figure 4 VCB backup with Avamar


http://www.vmware.com/

Table 7 shows the results of VCB image-level backup of three virtual machines with and without Avamar. In this case, VCB Proxy Server was considered the backup source.

Table 7 VCB image-level backup results

Backup Source Server Backup Target Server VM Size (GB)

Back- up

Stor-age

(GB)

De-dup ratio

Back up

Time (min)

Avg Network (MB/s)

Max Network (MB/s)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU

(%)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU

(%)

Avamar VM1 25 24.52 1.01:1 31 14.08 18.74 31 19.2 2 21 27.58 3 VM2 25 0.056 446:1 15 0.38 2.732 25 17.2 1 1 27.22 1 VM3 25 0.023 1086:1 14 0.33 0.85 24 17.1 1 1 27.19 1 NetWorker VM1 25 25 N/A 16 26.82 31.07 9.43 28.96 4 18 61.13

4 ALL 75 75 N/A 34 30.41 50.36 22 21.4 2 27 76.56 6

Note: The time taken for transmitting the snap of a VM from the Production ESX Server to the VCB Proxy Server was not included in Table 7 for comparison because it is a required step regardless of whether Avamar is used.

The following are the observations: • With our designed backup source, little de-duplication was achieved on the first virtual machine

containing mostly unique text files. However, as the second and third virtual machines were processed, huge de-duplication ratios were realized. Consequently, a lot less data was transmitted on the network, and little storage was needed for backing up the additional virtual machines. Avamar can greatly benefit an environment such as VMware Virtual Desktop Infrastructure (VDI) as VDI is typically deployed with a large number of virtual desktops generated from a golden image.

• Avamar backed up one job at a time from a single Avamar client when scripts were provided to Avamar for automated backup. In contrast, NetWorker backed up multiple jobs simultaneously from the same server. Moreover, Avamar always consumed some time to analyze data and perform de-duplication upfront. Altogether, Avamar required a longer backup window than a traditional backup when more than one virtual machine is backed up.

• Avamar was more CPU intensive due to its data analysis and de-duplication activities, especially at the source. Without Avamar, NetWorker consumed more memory during the backup as it streamlined the backup data stream faster.

• Regardless of whether Avamar was used, backup of virtual machines that resided on Celerra through NFS generated very light workload against the storage array. Very little CPU cycle was spent on Celerra Data Mover to process the virtual machine I/O and handle the backup process.

VCB file-level backup VCB file-level backup is very similar to VCB image-level backup. For more information, refer to Figure 4 on page 13 and Appendix A: Avamar configuration for VCB backup. When a backup is initiated through Avamar, it triggers the scripts provided in the integration module, which in turn starts the Command Line Interface (CLI) command vcbMounter.exe (included in the VCB software) to contact the VirtualCenter Server or the ESX Server directly to locate the virtual machine to be backed up. The arguments passed to vcbMounter.exe come from config.js and the Dataset syntax in Avamar. VCB then uses mountvm.exe to mount the virtual disk on the VCB Proxy for Avamar to perform backup.


Data de-duplication happens at file level in this case. However, presently VCB file-level backup works only for virtual machines that run the Windows operation system.

Table 8 on page 15 shows the results of VCB file-level backup of three virtual machines with and without Avamar. The backup data set includes all files, including OS system files on disks. In this case, VCB Proxy Server was again considered the backup source.

Table 8 VCB file-level backup results


Backup Storage

(GB)

De-dup ratio

Backup Time (min)

Avg Network (MB/s)

Max Network (MB/s)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU

(%)

Avg CPU (%)

Avg Memory

Avg DM CPU

(%) (%)

Avamar VM1 25 21.8 1.146:1 41 10.22 15.68 21 21.5 2 16 26.5 2 VM2 25 0.238 105:1 33 0.43 1.958 21 20.94 2 2 27.6 1 VM3 25 0.047 531:1 32 0.305 1.91 24 20.09 2 2 27.62 1 NetWorker VM1 25 25 N/A 32 13.31 21.19 10 21.1 2 9 76.4 2 ALL 75 75 N/A 57 23.4 40.67 22 28.05 4 16 75.6 5

The following are the observations: • For the first virtual machine, a slightly better data de-duplication ratio was achieved in VCB

file-level backup. For additional virtual machines, image-level backup appeared to have better de-duplication ratios. In aggregation, the de-duplication ratio for all three machines was 3.05 in case of image-level backup and 3.39 in case of file-level backup. The more virtual machines that are backed up, the higher the de-duplication ratio would be. Overall, Avamar’s de-duplication mechanism seems to be very efficient in both .vmdk level and file level VCB backup.

• Similarly as in image-level backup, use of Avamar puts more demand on CPU while memory

utilization is higher without Avamar.

Guest-based backup Guest-based backup involves installing Avamar Agent inside each virtual machine. Backup configuration for this method is no different from that for a physical server. Usually no scripts are required for this type of backup. No Proxy Server is required to do the backup; only the Avamar agent has to be installed on all the virtual machines that need to be backed up.


Figure 5 shows how the Guest-based Avamar backup works.

Figure 5 Guest-based Avamar backup Table 9 shows results of guest-based backup of three virtual machines with and without Avamar. The backup data set included all files, including OS system files on disks. In this case, the Production ESX Server was considered the backup source. As an Avamar client was installed inside each virtual machine, backup jobs of the virtual machines were done concurrently. Therefore, no data is presented in the Table 9 for individual virtual machines.

Table 9 Guest-based backup results


Backup Storage

(GB)

De-dup ratio

Backup Time

The following are the observations: • Guest-based backup achieved slightly lower de-duplication ratio than VCB backup. Nevertheless, the

benefits of significantly smaller backup storage being used and less data being transmitted on the network remain the same when Avamar is used.

• With or without Avamar, guest-based backup consumed less time than VCB-based backup. This indicates the advantage of a shorter backup window when guest-based backup is performed. On the other hand, VCB backup brings the advantage of centralized backup management and can be done while the virtual machine is online or offline.

• Without a VCB Proxy Server, guest-based backup consumed sizeable CPU and memory resources on the Production ESX Server.

(min) Avg

Network (MB/s)

Max Network (MB/s)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU

(%)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU (%)

Avamar ALL 75 25.8 2.9:1 37 19.12 43.87 33 33.2 4 26 26.46 6 NetWorker ALL 75 75 N/A 24 48 108 22 15.85 5 33 76.31 8


Guest-based incremental backup After the previous guest-based backup was done, 10 GB of additional files were created in each of the virtual machine. These files were again unique text files. Incremental backup was then performed on each virtual machine using the guest-based backup method.

Table 10 shows the results of guest-based incremental backup of three virtual machines with and without Avamar. The backup data set included only the additional files created, which is 30 GB total. In this case, the Production ESX Server was considered the backup source. As an Avamar client was installed inside each virtual machine, backup jobs of the virtual machines were done concurrently. Therefore, no data is presented in Table 10 for individual virtual machines.

Table 10 Guest-based incremental backup results


Backup Storage

(GB)

De-dup

Backup Time (mins)

Avg Network (MB/s)

Max Network (MB/s)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU

(%)

Avg Avg ratio

CPU (%)

Memory (%)

Avg DM CPU

(%)

Avamar ALL 30 10.5 2.86:1 16 19.5 32.5 31

25 4 26 26.87 5 NetWorker ALL 30 30 N/A 10 45.37 73.87 22 13.66 5 32 76.52 8

The observation is:

Using the guest-based backup method, you can perform incremental backup instead of having to back up the entire virtual machine again. This significantly reduces the backup window, and also eliminates unnecessary storage consumption and data transmission on the network, which are further helped by use of Avamar.

Service Console-based backup Service Console-based backup involves installing Avamar Agent on the Service Console of each ESX Server host that needs protection. This backup method combines the advantages of efficient management and image-level de-duplication. Using the Service Console-based backup method, virtual machine backup can be done either online or offline.

Figure 6 Service Console-based Avamar backup

Full image-level backups can be done using this method, which utilizes the VMware command line interface vcbMounter.exe. A snapshot of the virtual machine is taken first. The snapshot is then copied to


a temporary location on the ESX Server specified in the vcbMounter.exe command. After this the Avamar Agent initiates backup of the virtual machine image

Table 11 shows results of Service Console-based backup of three virtual machines with and without Avamar. In this case, the Production ESX Server was considered the backup source. As Avamar scripts could not be used for automated backup from the Service Console, backup jobs of the virtual machines were manually scheduled to start concurrently. Therefore, no data is presented in the table for individual virtual machines.

Table 11 Service Console-based backup results

Note: The time taken for the snap to be copied to the temporary location of the Production ESX Server is not considered for comparison.

The following are the observations:

• Service Console-based backup resulted in the smallest data de-duplication ratio among all the backup methods discussed in this paper. However, the benefits of using Avamar are still very significant compared with Avamar not being used.

• Service Console-based backup is very similar to VCB image-level backup. The major difference is that no VCB Proxy Server is used to offload backup jobs from the Production ESX Server, and hence no VCB scripts were used to automate the backup jobs.

Restore VCB image-level backup with Avamar offers two choices when it comes to restoring the virtual machines. You can restore the entire virtual machine directly to the Production ESX Server host or to the Proxy Server. In either case, Avamar Agent must be present.

The following details provide information about the restore methods: • Restore directly to the Production ESX Server host

Run the VCB vcbRestore command with proper options in the service console. • Restore to the VCB proxy

Manually copy the virtual machine files to the Production ESX Server host, and run the VCB restore command with proper options in the service console.

(or) Use VMware Converter to restore the virtual machine. With VCB file-level backup, individual files can be restored directly to the virtual machines running Windows operating systems. This operation requires Avamar Agent to be installed inside the virtual machine.

Guest-based file level restore involves Avamar Agent inside each virtual machine. This type of restoration is no different from that of a physical server. No proxy server is required either.


Back up

Storage

De-dup ratio

Backup Time (min)

Avg Network (MB/s)

Max Network (MB/s)

Avg CPU (%)

Avg Memory

(%)

Avg DM CPU

Avg CPU

(GB) (%) (%)

Avg Memory

(%)

Avg DM CPU

(%)

Avamar ALL 75 28.46 2.64:1 58 8.875 19.87 12 6.72 3 22 27.61 6 NetWorker ALL 75 75 N/A 87 15.12 22.25 10 13.7 3 17 71.3 4


Restoring a virtual machine can be done with Avamar Agent when using the Service Console-based backup method.

To restore a virtual machine:

1. Restore the virtual machine files to the Production ESX Server host’s VMFS file system.

2. If the backup was done by copying the virtual machine files while the machine was offline, register the restored virtual machine directly with VirtualCenter Management Server.

3. If the backup was performed while a virtual machine was online, run the vcbRestore command to restore.

Summary In this paper, we described and verified the different backup options with Avamar in a VMware Infrastructure environment deployed on EMC Celerra. Test data from the various test cases is illustrated in Figure 7.

Figure 7 Comparison of backup options

The following are the key findings from our testing:

• Avamar’s data de-duplication capability reduces backup size significantly. Although the de-duplication ratio on an individual virtual machine depends on the types of files in the virtual machine, de-duplication can benefit an environment greatly where redundant data across virtual machines is rich. For instance, using Avamar to back up virtual desktops in a VDI deployment is ideal as most of the virtual machines are created from a golden image; the larger the VDI deployment, the higher the de-duplication ratio and hence less storage being used for backup.

• Since Avamar data de-duplication happens at the source side, it brings an additional benefit of transmitting less data on the network. This saves network bandwidth for other critical tasks such as VMotion that may happen concurrently.

• Avamar needs to analyze data and then perform de-duplication, which means Avamar will likely take more time to finish a backup job than a traditional way of backup. As a result, a longer backup window should be planned if Avamar is used.


• EMC Celerra is utilized a lot less during backup in all cases. In our testing, we placed the virtual machines on NFS Datastore. All the backup methods described in this paper would work in the same way if Celerra storage is presented to the virtual machines in a different format such as VMFS over iSCSI. In addition, we expect similar testing data to be achieved regardless of the storage connection method.

• EMC Celerra can be very flexibly deployed to provide storage to VMware Infrastructure as the source of Avamar backup, and also host storage for the backup target. This allows Avamar backup to be managed centrally, and performed easily and cost-effectively.

To perform Avamar backup in a virtual environment, you can perform VCB backup, guest-based backup, and Service Console-based backup. Each method has its pros and cons. High-level comparisons of the methods are listed in Table 12.

Table 12 Comparison of backup methods

VCB Pros: • Backups are performed on VCB proxy and offload processing load on the ESX Server host • Backup can be performed with virtual machines online or offline, and there is no need for

scheduled downtime • Full image backup or file-level backup provide flexibility options • Managed centrally without install and manage backup agents inside each guest

Cons: • Additional hardware for VCB proxy is required • File-level backup is supported only for virtual machines running Windows • Virtual machine snaps need to be created and copied to the proxy, which can be time-consuming

and requires additional storage for temporary use

Guest-based Pros: • Same file-level backup methods and procedures as on a physical machine • Support for backup of applications inside the virtual machine • No requirements for advanced scripting in VMware environment • No need for additional hardware to be used as VCB proxy

Cons: • Resource intensive on the ESX Server host • No image-level backup to take advantage of virtual machine encapsulation, and hence difficult

for bare-metal restore • Distributed backup management with the need to install backup software on every individual

virtual machine

Service Console-based

Pros: • Image-level backup can be done for all guest operating systems using standard VMware

command line utilities • Backup can be performed with virtual machines online or offline and there is no need for

scheduled downtime • Simplified management with no need to install and manage backup agents inside each guest • Additional hardware for VCB proxy is not required

Cons: • Backups are performed on the ESX Server host by increasing the workload • Virtual machine snaps need to be created, which requires additional storage for temporary use • Security concerns arise as ESX Server’s built-in firewall may need to be compromised to make

use of third-party backup agent


Comparison of backup methods With regard to Avamar being used, comparative data de-duplication ratios were achieved in all backup methods using the specific data set we designed for our testing. With this data set, Avamar appeared to be the most efficient in VCB file-level backup, while at the cost of being the most time-consuming.

Avamar requirements and best practices Based on Avamar and VCB documentation and our testing, we recommend users of Avamar Virtual Edition for VMware to adhere to the following requirements and best practices:

• Avamar VCB Integration Module (AVIM) scripts assume that VCB is installed in its default location c:\Programfiles\VMware\VMware consolidated backup. Scripts are present in the appropriate directory, and the VCB backup root exists. Failing to meet the above conditions will lead to Avamar job failure for the image-level backup and file-level backup using VCB.

• The default VCB backup mode is ‘SAN’. To perform LAN based VCB backup, modify parameter TRANSPORT_MODE to “nbd” or “nbdssl” in the config.js file.

• If you use a virtual machine hostname in the Avamar data set, AVIM searches for the target virtual machine by its name. The lookup method in config.js file needs to be changed to “name” from the default “ipaddres”. If you do not change the default lookup method, ensure that the virtual machine IP address is specified in the Avamar data set. Ensure that the virtual machine is powered on the first time it is backed up. This information is then cached locally on the VCB proxy for later use.

• RDM physical compatibility mode is not supported by VCB backup. Virtual machine backup can be done either online or offline except when using the guest-based backup method, in which case the virtual machine to be backed up must be online.

• Avamar does not support multiple concurrent backup jobs on a single Avamar client even when the scripts are provided by Avamar for automated backup. However, multiple jobs can be run using multiple Avamar clients. Concurrent backup can be achieved on a single Avamar client if you choose to manually mount the virtual machine snaps and start backup jobs manually.

• When restoring data for a virtual machine that was backed up using VCB, ensure that the backup was associated with the VCB Proxy and not with the virtual machine's DNS name.

• VcbMounter and vcbRestore commands can be executed directly on the Production ESX Server • The Avamar client process (avtar) loads two cache files into memory when performing a backup.

These client caches are used to reduce the amount of time required to perform a backup, reduce the load on the Avamar client, and reduce the load on the Avamar server. Deleting these files can increase the backup time significantly.

• Deleting an Avamar client permanently deletes all snapups stored for that client. Therefore, you should only delete an Avamar client when there is no reason to retain the snapups. You should retire the Avamar client instead. Retiring a client is less destructive than deleting it. Retired clients do not actively participate in the Avamar system with regular snapups. However, old snapups associated with a retired client are maintained in the system (subject to snapup retention settings) and files can still be restored using Avamar.

• When redirecting a restore to a new location (a target directory other than the original directory that was snapped up), restoring a single directory to the new location only restores the contents of the directory; the original parent directory is not restored as part of this operation. However, if you restore two or more directories to a new location, the original parent directories do get restored along with the contents of those directories.

• By default, all new clients that have been activated with the Avamar server are automatically added to the ‘Default Group’ and will automatically be scheduled to be backed up per the default group policy. To help manage the capacity on the Avamar server, you should leave the 'Default Group’ disabled. Any clients that need to be backed up to the Avamar server should be added to an enabled group.


• Configure and Install the Avamar File System to have read-only accessibility to all snapups stored on an Avamar server down to the individual file level.

Conclusion Avamar software quickly and efficiently protects VMware Infrastructure environments by reducing the size of backup data within and across virtual machines — using agents in the virtual machines, on the VMware Consolidated Backup proxy server, or in the ESX Server Service Console. For virtual machine backups, Avamar eliminates traditional backup bottlenecks caused by the large amount of data that must pass through the same set of shared resources — the physical server’s CPU, Ethernet adapter, memory, and disk storage. Avamar’s source-based de-duplication also significantly reduces network traffic between the backup source and target.

EMC Celerra is the ideal platform for providing NAS and IP SAN storage to VMware Infrastructure. Avamar Virtual Edition for VMware integrates nicely in such an environment to help protect user data and at the same time address the challenge of ever-growing storage needs. It is a great addition to the various backup options that EMC Celerra provides to VMware Infrastructure.

References The following documents, located on EMC Powerlink, provide additional relevant information:

• Using EMC Celerra IP Storage with VMware Infrastructure 3 over iSCSI and NFS – Best Practices Planning white paper

• Avamar Virtual Edition 1.0 Installation Guide • Avamar Virtual Edition 1.0 Reference Manual • Using the Avamar Virtual Edition Performance Assurance Tool – Technical Notes

• VMware ESX Server Backup and Replication on EMC Celerra NS Series – Best Practices Planning white paper

Refer to the following documents on the VMware website (http://www.VMware.com/) for information about VMware Virtual Infrastructure:

• VMware Virtual Infrastructure 3 Documentation • VMware Consolidated Backup • Optimized Backup and Recovery for VMware Infrastructure with EMC Avamar white paper 1


Appendix A: Avamar configuration for VCB backup This appendix explains the procedure to configure Avamar for VCB backup. To configure Avamar for VCB backup:

1. Extract the AVIM zip file to the etc\scripts folder in the Avamar client directory.

2. From the Tools menu, select Manage Data Set and then click New. The New Dataset window appears.

3. Under the Source Data tab select the following options:

a. Select Enter Explicitly.

b. Select Windows File System from the Select Plug-In Type list.

c. Click adjacent to the Select Files and/or Folder box and specify the VCB backup root directory configured in the config.js file.

Name it differently to “VCB-filelevel” for file-level backup

Figure 8 Avamar configuration for image-level backup

4. Under the Options tab, click More and add the appropriate options (refer to Table 13 on page 24)


Figure 9 Avamar configuration for file-level backup

Replace the Avamar scripts in case of image-level backup

Table 13 Attributes

Attribute Attribute Value (image-level)

Attribute Value (file-level)

run-at-start pre_avamar-fullVM.bat

pre_avamar-file.bat

run-at-end post_avamar-fullVM.bat

post_avamar_file.bat

cacheprefix Name of the virtual machine

Name of the virtual machine

5. Click OK to complete the creation of the data set.

6. Create a group using this new data set to complete the process.


EMC Avamar Backup Solutions for VMware ESX … Avamar Backup Solutions for VMware ESX Server on...

Documents

Transcript of EMC Avamar Backup Solutions for VMware ESX … Avamar Backup Solutions for VMware ESX Server on...