virtualizing microsoft exchange...
Transcript of virtualizing microsoft exchange...
VIRTEXCPOWSTO
A DELL
End-t
Dell
Septe
RTUALICHANGWEREDORAGE
L TECHNICAL
to-End So
Product G
ember 20
ZING MGE 20DGE™
E
L WHITE PA
lutions En
Group
10
MICRO10 ON SERVE
APER
ngineering
OSOFTN DELLERS AN
g
T L ND
THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND.
© 2010 Dell Inc. All rights reserved. Reproduction of this material in any manner whatsoever without the express written permission of Dell Inc. is strictly forbidden. For more information, contact Dell.
Dell, the DELL logo, and the DELL badge, PowerConnect, and PowerVault are trademarks of Dell Inc. Symantec and the SYMANTEC logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the US and other countries. Microsoft, Windows, Windows Server, and Active Directory are either trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Other trademarks and trade names may be used in this document to refer to either the entities claiming the marks and names or their products. Dell Inc. disclaims any proprietary interest in trademarks and trade names other than its own.
September 2010
Contents A DELL TECHNICAL WHITE PAPER ................................................................................. 1
1. Introduction ................................................................................................... 3
2. Overview....................................................................................................... 4
2.1 Hardware Components .............................................................................. 4
3. Deployment Best Practices for Virtualized Exchange Server 2010 ................................... 7
4. Reference Architecture for 4000 Mailbox Virtualized deployments .................................. 9
4.1 Performance Study ................................................................................... 14
5. Summary ..................................................................................................... 17
6. Reference and Additional Information .................................................................. 18
1. Introduction Enterprises are seeking ways to streamline IT budgets without affecting operations and limiting
features to end users. Microsoft Exchange® 2010 introduces new performance enhancements,
which can significantly reduce storage requirements, and extends high-availability features.
Exchange Server 2010 is officially supported by Microsoft on virtualized infrastructures and if
designed correctly, can further reduce the total cost of ownership and maximize return on
investment. In this white paper, we design and evaluate a 4000 mailbox Exchange 2010
deployment on Dell’s Virtualization Business Ready Configuration (BRC).
Virtualization Business Ready Configuration’s (BRC) from Dell are tools that help customers to
minimize complexity during planning, ordering, deployment and installation of their virtual
infrastructure. Dell offers multiple BRC’s that provides the customer a choice between multiple
server and storage platforms, and for their virtualization infrastructure software. The BRC is
optimized for running virtualized workloads and provide optimal redundancy, scalability and
manageability. This document covers an approach to virtualize Exchange 2010 on the Dell
Virtualization BRC for a Small Medium Business. The approach used in this document is hypervisor
agnostic, and therefore the configuration can be used for hypervisors from Microsoft, VMware or
any third party vendor that is validated by Microsoft to be used for Exchange Virtualization.
This white paper describes using the Business Ready Configuration for Virtualization to implement
an Exchange Server 2010 messaging infrastructure on a VMware® vSphere™ environment. It
documents the Exchange 2010 architecture, server and storage configurations, virtual machine
sizing and best practices to enable an efficient virtualized Exchange 2010 messaging
infrastructure. The best practices presented are the result of installing, sizing and evaluating
Exchange Server 2010 components on a Virtualization BRC at the End-to-End Solutions Engineering
Lab. Experimental results from the testing form the basis for sizing and performance guidance
presented in this white paper to architect an optimal Exchange 2010 solution
2. Overview Dell offers multiple BRC’s that provide the customer a choice between multiple server, storage
and virtualization platforms. Customers use the BRC’s from Dell for quick and efficient ordering,
deployment and installation of their virtual infrastructure. The BRC used to host the virtualized
Exchange 2010 building block presented in this document is available at
http://content.dell.com/us/en/enterprise/d/virtualization/BusinessReadyDataCenter.aspx. This
BRC combines the Dell PowerEdge™ M-Series blade servers and EqualLogic™ PS6000 storage area
network (SAN) with Cisco® networking switches and VMware® vSphere™ 4 technology to help
achieve an automated and efficient data center that can be used to host Exchange 2010 Server
roles along with other virtualized workloads.
2.1 Hardware Components
Figure 1: Typical vSphere 4 Architecture with PS series SAN
The PowerEdge M610 is a two-socket blade server designed with energy-optimized options
supporting the new Intel® Xeon® 5600 series processors, DDR3 memory and advanced embedded
management capability.
P o we r E d g eM1 0 0 0 e
9 1 01 2
1 1 1 23 4
1 3 1 45 6
1 5 1 67 8
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
0
1
00
1
0 0
1
00
1
0 0
1
00
1
0 0
1
00
1
0
VMware vSphere 4
Storage Fabric
Network
VM VM VM VM
Pow
erEdge M
1000e M
610 blade servers
EqualLog
ic PS
6000 S
eries SA
N
The Dell EqualLogic PS6000 is an iSCSI storage device from Dell designed for deployment in
enterprise business environments. It consists of four 1Gb Ethernet network ports per controller for
data, faster processors, 2 GB of cache per controller, support for RAID 6, increased drive capacity,
and a new monitoring application, SAN HQ, at no additional cost. For more information on Dell
EqualLogic storage, see www.dell.com/equallogic.
In the following table, we provide a summary specification of the pre-configured solution. In the
rest of the document, we will discuss reference architectures and best practices for deploying and
using these solutions. The solution ID can be used to order or view the complete virtualization
solution to deploy the reference architecture.
Table 1: Solution Specification
Hardware
Solution ID 1131816.1
Blade Chassis (1) M1000e
ESX Blade Server (6) x M610
Storage Device (4) PS6000XV
Management Blade Server (1) x M610
Chassis Configuration
I/O module for A1, C1, A2, C2 Cisco M 3130G
I/O module for B1, B2 Cisco M 3130G with (2) 1000BAS-T SFP modules
Stacking cables 6
Management Redundant Chassis Management Controllers (CMC)
KVM Integrated Avocent keyboard, video, and mouse (iKVM) switch
ESX Blade Server Configuration
Blade Server Model M610
Processor 2 x Intel Xeon X5660, 2.8Ghz, 12M Cache,Turbo, HT, 1333MHz Memory
Memory 96 GB (12 x 8 GB, DDR3)
Add-in Controllers (2) Broadcom 5709 Dual Port GbE I/O Card for M-Series Blades
Local storage and controller Diskless configuration
Hypervisor VMware ESXi 4.0
Storage Configuration
Storage Device (4) PS6000XV
Drives 16 X 600GB, 15K SAS
Storage Capacity 9.6 Terabyte capacity per Device
Management Blade Server
Blade Server Model (1) M610
Processor (2) x Intel Xeon (Nehalem) E5520, 2.26Ghz, 8M Cache
Memory 8 GB
Network Controller (2) Broadcom 5709 Dual Port GbE I/O Card for M-Series Blades
Virtualization Management S/W VMware vCenter 4.0, 3Yr Upgrade Subscription
Software and Services
Services 3 Year ProSupport for IT and Mission Critical 4HR 7x24 Onsite Pack
EqualLogic PS Array Installation M1000e Blade Chassis On-site installation service
3. Deployment Best Practices for Virtualized Exchange Server 2010
Microsoft started supporting Exchange Server on Virtualized hardware with Exchange Server 2007.
Exchange Server 2010 is supported as well, assuming certain conditions are fulfilled. In terms of
hypervisors, Windows 2008 with Hyper-V Technology, Microsoft Hyper-V Server and any third party
hypervisor (such as VMware and Citrix offerings) that are validated under Window’s SVVP (Server
Virtualization Validation Program) are supported. The guest VM running the Exchange Server 2010
roles has to be deployed on Windows Server 2008 SP2 or R2 to be supported. All server roles are
supported in a virtualized environment, except for Unified Messaging role, which has to be on a
physical server.
Other important configuration or deployment scenarios that to keep in mind –
Dynamic VHD disks are not supported; neither are snapshots, differencing or delta
mechanisms
Exchange 2010 HA features like Database Availability Groups (DAGs) cannot be combined
with hypervisor based clustering solutions.
Overprovisioning of virtual processor is not supported for a ratio greater than 2:1
Minimum size of the fixed disk for the Exchange 2010 operating system virtual disk has to
be 15Gb in addition to page file size (equal to the memory allocated for the VM).
In a supported configuration, server based applications cannot be run on the physical root
machine of a server that is hosting the Exchange 2010 roles inside virtual machines. It is
okay to run management software like anti-virus, backup software etc on the root
partition.
Operating System disk images for the virtual machines have to be stored on serparate
storage array or disks from the Exchange database or log storage.
Sizing the Exchange Server Mailbox Roles/VMs Exchange Server 2010 is not virtualization aware and therefore the majority of the design
principles used for a physical deployment will translate to the virtual deployment design. Virtual
machines are sized similar to physical servers, depending on the Exchange server role that is being
hosted on the virtual machine. The number of cores and memory per virtual machine will depend
on the message profiles and mailbox sizes. The following guidelines are recommended by
Microsoft when deploying Exchange 2010 on a Hyper-V environment to size virtual machines
hosting Exchange roles–
The Hub and CAS server roles are combined in a single VM. This VM is assigned 4 virtual
CPUs and 8GB of memory. The number of multi-role Hub/CAS deployed is the same as the
number of Mailbox VMs deployed, i.e. the ratio of Hub/CAS VMs to Mailbox VMs is 1:1.
The Mailbox server role is assigned four virtual CPUs. The approximate maximum number
of mailboxes recommended per Mailbox VM is 3000 to 4000 depending on the user profile
and concurrency. The memory assigned to each Mailbox VM is 4GB (for OS) + memory
needed for mailbox database cache. The requirements per user for mailbox database
cache size ranges from 3MB-9MB (light profile to very heavy profile).
In the design presented in this paper, we use a multi-role Exchange VM which combines the
HUB/CAS/Mailbox roles in a single machine. The goal of this design is to use a part of the
resources available in the Virtualization infrastructure, and share the hardware with other
virtualized applications and workloads.
The number of drives required for database and logs is calculated using the same
principles as a physical deployment. The Microsoft storage calculator is used to determine
the storage design in terms of the number of drives and RAID type for the solution. For
internal or direct attach storage configurations SCSI pass-thru is recommended where the
volumes are presented directly to the Mailbox VMs for database and log volumes. For iSCSI
SAN, the initiator is configured on the host server disk is again presented to the guest VM
as a pass-thru disk to database and log storage.
4. Reference Architecture for 4000 Mailbox Virtualized deployments
The Agile Consolidated model from Dell for Exchange Server 2010 deployments are targeted for
the virtualized datacenters that have already invested heavily in virtualization1. These
datacenters are already running their Tier-2 and Tier-3 applications on VMs and looking at moving
their Tier-1 apps on to VMs as well. The storage infrastructure for this class of deployment is on
Dell SAN products such as Dell EqualLogic™ PS arrays and Dell/EMC CX4 and AX4 arrays.
A sample reference architecture for the Agile Consolidated Dell model using internal storage is
shown in Error! Reference source not found.. This architecture is designed for a 4000 mailbox
with an I/O profile of 150 messages per day and mailbox sizes of up to 2GB. Table 2 lists the input
variables that were used for this configuration.
Table 2 - Input Requirements for 4000 User Configuration
Input Detail
Number of mailboxes 4,000
I/O Profile 150 Messages/Day
Size of Mailbox Up to 2GB
Average Message Size 75KB
Number of Copies 2-copy DAG (1 active, 1 passive)
The high-availability for mailboxes is provided using built-in Exchange resiliency features thru the
Database Availabiliy Group (DAG). The 4000 mailboxes are evenly distributed between four
Exchange VMs, hosted on two separate physical servers. Two Database Availability Groups (DAGs)
are created across the 4 VMs so that the mailbox copy is hosted on a separate physical server. The
hypervisors are not part of a cluster since virtualization features are not used in this design to
provide high-availability. The hypervisors not hosting Exchange, can be part of a cluster and
applications running on them can be load balanced thru the virtualization software.
Each physical server hosts a multi-role Mailbox/Hub/CAS server VM. Error! Reference source not
found. details the host and VM configurations to support 4000 users. Full provisioning of resources
is configured so that any server can handle a scenario where all the passive copies become active
in the event of failure happening on a server or individual databases. High Availbility in the local
1 http://www.dell.com/downloads/global/solutions/security/exchange_2010.pdf
site is pr
applicatio
Figure 3
deployme
Virtual M
copies, l
performa
used for
images an
ovided by Ex
on and serve
shows the c
ent is install
Machine. Dell
logs, and th
ance and pro
storing the E
nd storage fo
xchange Data
er layers as w
Figure
consolidated
led. Two Pow
EqualLogic
he VM OS im
vide resilien
Exchange da
or other VMs
abase Availa
well as the st
2 - Agile Con
hardware in
werEdge M6
iSCSI SAN st
mages. The
cy against d
atabase and
.
ability Group
torage and da
nsolidated Mo
nfrastructure
10 servers a
torage is use
arrays are
isk failures a
log files in a
ps (DAGs) wh
atabase laye
odel on VMw
e on which t
are used to
ed to host th
configured
at the storag
addition to o
hich provide
ers.
ware
the 4,000 use
host the mu
e Exchange
using RAID
ge level. Mul
other volume
protection a
er Exchange
ulti-role Exch
Mailbox data
D-10 to max
ltiple volume
es to hold th
at the
2010
hange
abase
ximize
es are
he VM
Figure 3 shows the layout of the Exchange 2010 server roles and distribution of virtual machines
on the servers. The 4,000 users are distributed across four Exchange VMs, each host running two
Mailbox VMs. In the event of a server failure, Exchange DAG will activate the passive copy on
working server to restore service to the failed mailboxes.
Configuration details for the server, storage and virtual machines are shown in
Table 3. Each PowerEdge M610 is configured with two quad-core CPUs and 96 GB of memory so it
can handle additional load in case of server failure on the other host. Each Mailbox VM instance is
configured with 4v CPUs and 48GB of memory and handles 1000 active mailbox users and 1000
passive users. The hosts are connected to the iSCSI SAN using the software iSCSI initiator in the
host hypervisor.
Figure 3 - Exchange 2010 Reference Architecture for 4000 Users
Table 3 – Server and Storage Configurations
Solution Output Detail
Host Server Configuration 2 PowerEdge M610
2 x Quad-Core CPUs 6.4 GT/s
96 GB DDR4 Memory
Mailbox/HUB/CAS VM 2VM’s per Host
4 vCPU, Memory per VM: 48GB
1000 Mailboxes Active and 1000 Mailboxes Passive per Virtual Machine
Storage Configuration 4 x EqualLogic PS6000XV iSCSI Enclosures
16 x 3.5” 600 GB 15K RPM SAS disks per enclosure
RAID-10 across the entire array
Active Copy (first copy)
Passive Copy (second copy)
Exchange VM1
Exchange VM2
Exchange VM3
Exchange VM4
DB3 DB4
DB8 DB7
DB1 DB2
DB6 DB5
DB1 DB2
DB5 DB6
DB3 DB4
DB7 DB8
MB VM1 MB VM2 MB VM3 MB VM4
DAG2
DAG1
Database Configuration
Databases 4 database per Mailbox VM (data and logs combined per database)
Total 16 databases (8 active and 8 passive databases)
Mailboxes 500 Mailboxes per DB
Up to 2GB capacity per Mailbox
4.1 Performance Study A smaller configuration of the reference architecture presented was tested and evaluated in the
Dell enterprise solutions lab to validate the architecture and gather experimental data. Figure 4
illustrates the configuration that was tested - VMware ESX Server v4.1 is installed on both Dell
PowerEdge M610 servers. Each host is running a single Guest VM installed with Microsoft Windows
Server 2008 R2 Enterprise Edition. Exchange 2010 roles are installed on the guest virtual machines
and host 1000 active and 1000 passive mailboxes each. The mailbox VM is assigned four virtual
CPUs and 24GB of memory. It was configured with 2 vNICs, one connected to the public server
network and the other was a private network configured for the DAG (Database Availability
Group). The Exchange 2010 database and logs are hosted on volumes created on the externally
connected EqualLogic PS6000XV array.
Figure 4 - Performance Study Configuration and Setup
Figure 5 shows the average SendMail and 95th Percentile Send Mail latencies measured when
simulating a 1000 user mailbox deployment on each virtual machine. The 95th percentile latencies
are well within the 1-second threshold recommended by Microsoft for adequate end user
performance. The SendMail latencies are still within the threshold when a Database Availability
Group (DAG) is created across the two Exchange VMs and they host the passive copy of the
databases.
Figure 5 - Microsoft LoadGen Send Mail Latencies
Figure 6 illustrates the CPU and Memory usage for the ESX host and Virtual Machines when running
2000 mailbox LoadGen simulation on the standalone configuration and DAG configuration. As
expected there is a minor increase in both CPU and memory usage for the DAG configuration as
the Exchange server is hosting the passive copy in addition to the active mailboxes.
Figure 6 - CPU and Memory Usage for ESX Server and Exchange VM
The database and log performance metrics are shown in the following figures. In the stand alone
configuration there are no log reads being performed. When a DAG is established, we see log
reads on both Exchange VMs as an artifact of the DAG functionality. As expected the DB and log
I/O’s are significantly higher in a DAG configuration due to the maintaining a database copy. The
DB read latencies are observed to increase in the DAG configuration as well, but are within the
recommended thresholds by Microsoft for acceptable performance. The latencies presented in the
70 69 71 77
223 218
245 249
0
50
100
150
200
250
300
VM1 1K Mailboxes
VM2 1K Mailboxes
VM1 2K Mailboxes
(1K Active + 1K Passive)
VM2 2K Mailboxes
(1K Active + 1K Passive)
Late
ncy
(m
s)
Send Mail Latency 95th Percentile SendMail Latency
3 2 3 4
119
1315
25537 25376 25950 25922
18948 1877721029 20658
0
5000
10000
15000
20000
25000
30000
0
2
4
6
8
10
12
14
16
VM1 1K Mailboxes
VM2 1K Mailboxes
VM1 2K Mailboxes
(1K Active + 1K Passive)
VM2 2K Mailboxes
(1K Active + 1K Passive)
Me
mo
ry C
on
sum
pti
on
(M
byt
es)
CP
U U
tiliz
atio
n (
%)
Physical Server CPU Utilization VM CPU Utilization Memory Usage ‐ Physical Server Memory Usage ‐ VM
DAG
DAG
DAG configuration is a sum of the attached and recovery latencies – the attached read latency is
still within 10ms which is half of the recommended threshold by Microsoft for optimal end user
Exchange performance.
Figure 7 - Database Throughput Performance
Figure 8 - Database and Log Latency
Figure 9 shows how the ESX host and Exchange VM are affected when a DB failiure occurs on one of the Mailbox VMs. We simulate a failure on VM2 and this makes the copy on VM1 active, and it is therefore running 2000 active mailboxes. This was done to validate that the VM resources were adequate to handle a failure scenario at the database or host level. The chart plots the impact on resource usage over normal operation when the VM is running 1K active and 1K passive mailboxes.
11
4
42
14
47 47
87 86
0 02.9 2.7
67 6973
68
0
10
20
30
40
50
60
70
80
90
100
VM1 1K Mailboxes
VM2 1K Mailboxes
VM1 2K Mailboxes
(1K Active + 1K Passive)
VM2 2K Mailboxes
(1K Active + 1K Passive)
DB Reads/Sec DB Writes/SecLog Reads/Sec Log Writes/Sec
8.5 9.0
13.3
21.1
2.11.2
2.0 1.50 0
2.6 2.41.01 0.71 0.78 0.82
0.0
5.0
10.0
15.0
20.0
25.0
VM1 1K Mailboxes
VM2 1K Mailboxes
VM1 2K Mailboxes
(1K Active + 1K Passive)
VM2 2K Mailboxes
(1K Active + 1K Passive)
Late
ncy
(ms)
DB Read Latency DB Write Latency
Log Read Latency Log Write Latency
DAG
DAG
Figure 9 - Performance Impact when simulating a database faliure
At the Exchange application level we observe an increase in the RPC latency, operations/sec and messages being queued. This is expected behavior since all mailbox operations that was being handled by the two VMs are now being handled by a single VM. One VM is now hosting 2K active mailboxes, as compared to 1K active and 1K passive mailboxes. Hosting passive mailboxes does not incur RPC or Exchange related load on the VM, but has in impact on the storage subsystem. Since the VM was sized adequately to handle this addition load of 1K active mailboxes, there is only a minor increase in the resource utilization and end user performance is not impacted as verified by the SendMail latencies in the two scenarios. The DB read and write latencies were verified to be less than 10 ms which is well within the recommended threshold by Microsoft, proving that we have enough resources allocated to the VM to handle a database failure.
5. Summary This paper presented a reference architecture for deploying Exchange Server 2010 on virtualized Dell infrastructure. A four thousand user architecture was presented using the agile consolidated model from Dell. The high-availability for mailboxes was provided using built-in Exchange resiliency features thru the Database Availabiliy Group (DAG). The performance of the proposed architecture was evaluated using a scaled-down version and test results showed that the configuration was able to handle a 2000 mailbox deployment efficiently. The test setup can be scaled up to handle 4000 users without adding more blades since the CPU and memory usage was very minimal. Adding additional EqualLogic arrays result in a linear performance increase and will handle the I/O requirements for 4000 mailboxes. This setup can be shared with other virtualized applications, and scaled to accommodate future growth for the organization.
2.3% 1.8% 0.0%5.4%
26.1%
63.4%
23.4%
91.7% 92.1%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
ESX Host CPU Utilization
VM CPU Utilization
Memory Usage ESX Host
Memory Usage Virtual Machine
MS ExchIS RPC Avg Latency
MS ExchIS RPC Operations/sec
MS ExchIS Mailbox Messages
Queued
MS ExchIS Mailbox Messages
Delivered/Sec
MS ExchIS Mailbox Messages
Submitted/Sec
Pe
rfo
rman
ce I
mp
act
Relative Performance of VM with 2K active Mailboxes vs. VM with 1K Active + 1K Passive Mailboxes
6. Reference and Additional Information
Dell Exchange 2010 Sizing Guide http://www.dell.com/downloads/global/solutions/Introduction_to_Exchange_2010_Sizing.pdf Microsoft’s Exchange 2010 Mailbox Server Role Requirements Calculator http://msexchangeteam.com/archive/2009/11/09/453117.aspx
Dell Architecture Models for Exchange 2010 http://www.dell.com/downloads/global/solutions/security/exchange_2010.pdf VMware vSphere™ Reference Architecture for Small Medium Business http://i.dell.com/sites/content/business/solutions/engineering-docs/en/Documents/VMware-vSphere-Reference-Architecture-SMB.pdf
Documentation for VMware vSphere 4 for Dell PowerEdge Servers http://support.dell.com/support/edocs/software/eslvmwre/VS/VS.htm VMware vSphere 4 iSCSI SAN Configuration Guide http://www.vmware.com/pdf/vsphere4/r40/vsp_40_iscsi_san_cfg.pdf VMware Network Concepts http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf Configuring VMware vSphere Software iSCSI with Dell EqualLogic PS Series Storage http://www.equallogic.com/resourcecenter/assetview.aspx?id=8453