1© Copyright 2013 EMC Corporation. All rights reserved. Microsoft Exchange Best Practices and...
-
Upload
brook-long -
Category
Documents
-
view
217 -
download
1
Transcript of 1© Copyright 2013 EMC Corporation. All rights reserved. Microsoft Exchange Best Practices and...
1© Copyright 2013 EMC Corporation. All rights reserved.
Microsoft ExchangeBest Practices and Design Guidelines on EMC Storage• Exchange 2010 and Exchange 2013• VNX and VMAX Storage Systems
Strategic Solutions Engineering
Updated – October 2013
2© Copyright 2013 EMC Corporation. All rights reserved.
Topics
Exchange – What has changed
Exchange virtualization
Exchange storage design and best practices
Exchange backup best practices
Additional References
3© Copyright 2013 EMC Corporation. All rights reserved.
Exchange – What has changed?• Exchange User Profile changes
• Exchange I/O characteristics
• Background Database Maintenance
• Database Availability Group (DAG)
• Storage choices
4© Copyright 2013 EMC Corporation. All rights reserved.
Exchange…What has changed
Exchange 2007
◊ 64-bit Windows◊ 32+ GB database cache◊ 8Kb block size◊ 1:1 DB read/write ratio◊ 70% reduction in IOPS from Exchange 2003
Exchange 2010
◊ 64-bit Windows◊ 100GB database cache (DAG)◊ 32Kb block size◊ 3:2 DB read/write ratio◊ 70% reduction in IOPS from Exchange 2007
Exchange 2013
◊ 64-bit Windows◊ 100GB database cache (DAG)◊ 32Kb block size◊ 3:2 DB read/write ratio◊ 33% reduction in IOPS from Exchange 2010
5© Copyright 2013 EMC Corporation. All rights reserved.
Exchange User Profile ChangesMessages sent/
received per mailbox per day
Exchange 2010 Estimated IOPS per mailbox (Active or Passive)
Exchange 2013 Estimated IOPS per mailbox(Active or Passive)
Mailbox resiliency Stand-alone
50 0.05 0.06 0.034
100 0.100 0.120 0.067
150 0.150 0.180 0.101
200 0.200 0.240 0.134
250 0.250 0.300 0.168
300 0.300 0.360 0.201
350 0.350 0.420 0.235
400 0.400 0.480 0.268
450 0.450 0.540 0.302
500 0.500 0.600 0.335
6© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Processor Requirements ChangesMessages sent or
received per mailbox per day
Megacycles per UserActive DB Copy or Standalone (MBX
only)
Megacycles per UserActive DB Copy or Standalone (Multi-
Role)
Megacycles per UserPassive DB Copy
Exchange 2010
Exchange 2013
Exchange 2010
Exchange 2013
Exchange 2010
Exchange 2013
50 1 2.13 N/A 2.66 0.15 0.69
100 2 4.25 N/A 5.31 0.3 1.37
150 3 6.38 N/A 7.97 0.45 2.06
200 4 8.50 N/A 10.63 0.6 2.74
250 5 10.63 N/A 13.28 0.75 3.43
300 6 12.75 N/A 15.94 0.9 4.11
350 7 14.88 N/A 18.59 1.05 4.80
400 8 17.00 N/A 21.25 1.2 5.48
450 9 19.13 N/A 23.91 1.35 6.17
500 10 21.25 N/A 26.56 1.5 6.85
7© Copyright 2013 EMC Corporation. All rights reserved.
Exchange I/O Characteristics
• User IOPS decreased for Exchange Server 2010/2013 but the size of the I/O has significantly increased
I/O Type Exchange 2007 Exchange 2010 Exchange 2013
Database I/O 8 KB random write I/O 32 KB random I/O 32 KB random I/O
Background Database Maintenance (BDM) I/O
N/A 256KB Sequential Read I/O 256KB Sequential Read I/O
Log I/OVaries in size from 512 bytes to the log buffer size (1 MB)
Varies in size from 4 KB to the log buffer size (1 MB)
Varies in size from 4 KB to the log buffer size (1 MB)
8© Copyright 2013 EMC Corporation. All rights reserved.
Exchange 2010/2013 mailbox database I/O read/write ratiosMessages sent/received per mailbox per day
Stand-alone databasesDatabases participating in mailbox resiliency
50 1:1 3:2
100 1:1 3:2
150 1:1 3:2
200 1:1 3:2
250 1:1 3:2
300 2:3 1:1
350 2:3 1:1
400 2:3 1:1
450 2:3 1:1
500 2:3 1:1
9© Copyright 2013 EMC Corporation. All rights reserved.
Understanding Exchange I/O
• Exchange 2010/2013 I/O to the database (.edb) are divided into two types: – Transactional I/O (aka user I/O)
• Database volume I/O (database reads and writes)• Log volume I/O (logs reads and writes)• Only database I/O’s are measured when sizing storage and during
Jetstress validation
– Non Transactional I/O• Background Database Maintenance (Checksum) (BDM)
For more details see “Understanding Database and Log Performance Factors” at http://technet.microsoft.com/en-us/library/ee832791(v=exchg.141).aspx
10© Copyright 2013 EMC Corporation. All rights reserved.
Background Database Maintenance (BDM)
• What is BDM and what does it do?– It is the process of Exchange Server 2010/2013 database
maintenance that includes online defragmentation and online database scanning
– Both active and passive database copies are scanned
Exchange 2010 Exchange 2013Read I/O size 256 KB 256 KB
Database scan completion 1 week every 4 weeks
IOPS per database 30 9
Bandwidth 7.5 MB/s* 2.25 MB/s*
* Based on EMC testing with JetStress 2010/2013
11© Copyright 2013 EMC Corporation. All rights reserved.
Background Database Maintenance (BDM)
• Both active and passive database copies are scanned– On active copy can be scheduled to run during the online
maintenance window (default is 24 x 7)– Passive copy is ”hardcoded” to 24 x 7 scan– Jetstress has no concept of passive copy, all are active
• Possible BDM related issues (mostly for Exchange 2010): • Bandwidth/throughput required for BDM and BDM IOPS• Not enough FE ports, not enough BE ports, improper RAID
configuration
12© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Content Index Considerations
• Content Indexing space considerations:– In Exchange 2010 content index space is estimated at about
10% of the database size.– In Exchange 2013 content index space is estimated at about
20% of the database size. • Additional 20% must be added for content indexing maintenance tasks (such
as the master merge process) to complete.
13© Copyright 2013 EMC Corporation. All rights reserved.
DB1
DB3
DB2Copy
Copy
Copy
Copy
Copy
Copy
Database Availability Group
A = Active
P = Passive
A
A
A
P
P
P
P
P
P
Exchange High Availability
Base component of the high availability and site resilience framework built into Exchange 2010/2013
A DAG is a group of servers participating within a Windows failover cluster with a limit of 16 servers and 100 databases.
All servers participating within a DAG can have a copy of any database within the DAG
– Each DAG member server can house one copy of each database, up to 16 copies, with only one being active, passive, or lagged
No configuration of cluster services are required, Exchange 2010/2013 handles the entire installation
– During site DR manual work, scripts must be run
A DAG does not provide recovery for logical database corruption
Database Availability Group (DAG)
MBX1 MBX3MBX2
14© Copyright 2013 EMC Corporation. All rights reserved.
Exchange High Availability
Ensure all elements of the design have resilient components – Storage processors– Connectivity to the servers– Storage spindles – Multiple arrays in DR scenarios
DAG copies should be stored on separate physical spindles– Provided all resiliency is reached at the source site
On SANs, consider performance of the passive and active copies within one array
Guidance for deploying DAGs
15© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage Options
• Understand which storage type best meets design requirements− Physical or virtual?− Dedicated for Exchange or shared with other applications?
• Follow EMC proven guidance for each platform− Proven Solutions Whitepapers
http://www.emc.com/solutions/application-environment/microsoft/solutions-for-microsoft-exchange-unified-communications.htm
− EMC storage solution submissions to Microsoft ESRP program http://technet.microsoft.com/en-us/exchange/ff182054.aspx
DAS or SAN?EMC offers both options
• For small, low-cost = DAS• For large-scale efficiency = SAN• Best long-term TCO = SAN• Virtualization ready = SAN
Iomega VNXe VNX VMAX
16© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Server IOPS Per Disks• Use the following table for IOPS per drive values when
calculating disk requirements for Exchange 2010/2013• Recommendations may change based on future test results
Disk typeExchange 2010/2013
database IOPS per disk
(random workload)
Exchange Server 2010/2013 database logs IOPS per disk
(Sequential workload)
VNX/VMAX VNX VMAX VNX and VMAX
7.2 K rpm NL-SAS/SATA 65 60 18010 K rpm SAS/FC 135 130 27015 K rpm SAS/FC 180 180 450
Flash 1250 1250 2000
17© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Virtualization• Recommendations and supportably
• Design best practices references
• Supported deployment configurations
• Configuration best practices
18© Copyright 2013 EMC Corporation. All rights reserved.
Exchange 2010/2013 virtualization
• Virtualization is a proven technology and cloud ready
• Exchange is not virtualization aware but virtualization friendly
• EMC recommends virtualizing Exchange for most deployments based on user requirements
• Supported on Hyper-V and VMware, and other hypervisors
• Hypervisor vendors must participate in the Windows Server Virtualization Validation Program (SVVP)
Recommendations and supportably
19© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
DAG
Exchange DAG on stand-alone hypervisor
hosts
Exchange stand-alone Mailbox servers on a
hypervisor host clusters
VMware DRS/HAor Windows Cluster
DAG
Exchange DAG with hypervisor host clusters
Exchange 2010/2013 supported deployment configurations (example)
VMware DRS/HAor Windows Cluster
20© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
Know your hypervisors limits:– 256 SCSI disks per host (or cluster)– Processor limits (vCPUs per virtual machine)– Memory limits
Be aware of the hypervisor CPU overhead:– Microsoft Hyper-V: ~10-12%– VMware vSphere: ~5-7%
Configuration best practices - General
21© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
Core Exchange design principles still apply– Design for performance and high availability– Design for user workloads
Size virtual machines specific to the Exchange role
Physical sizing still applies– Physical hypervisor server must accommodate the guests they will
support– DAG copies must be spread out across physical hosts to minimize outage
in case of physical server issues
Configuration best practices
22© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
Use common sense when placing virtual machines– Deploy VMs with the same role across multiple hosts
– Do not deploy MBX virtual machines from the same DAG on the same host server
Configuration best practices - VM placement recommendations
23© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
Disable migration technologies that save state and migrate– Always migrate live or completely shut down virtual machines
Dedicate/reserve CPU and memory to the Mailbox virtual machines and do not over commit
– pCPU to vCPU ratios: 2:1 is OK, 1:1 is a best practice
Disable hypervisor-based auto tuning features– No dynamic memory
Hypervisor server should have at least 4 paths (HBA/CNA/iSCSI) to the storage – 4 total ports
Install EMC PowerPath on Hypervisor server for maximum throughput, load balancing, path management, and I/O path failure detection
Configuration best practices
24© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
Exchange storage should be on spindles separate from guest OS physical storage
Exchange storage must be block-level– Network attached storage (NAS) volumes are not supported– No NFS, SMB (other than SMB 3.0), or any other NAS technology– Storage must be fixed VHD/VHDX/VMDK, SCSI pass-through/RDM
or iSCSI▪ Hyper-V Live Migration suggests Cluster Shared Volumes with fixed VHD (faster
“black-out” period)
Configuration best practices - Storage
25© Copyright 2013 EMC Corporation. All rights reserved.
SMB 3.0 Support
Only in virtualized configurations
VHDs can reside on SMB 3.0 shares presented to Hyper-V host
No support for UNC path for Exchange db and log volumes (\\server\share\db1\db1.edb)
26© Copyright 2013 EMC Corporation. All rights reserved.
Supported SMB 3.0 Configuration Example
27© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
Virtual SCSI (pass-through or fixed disk)• VHD on host – recommended for OS, program files• Pass-through disk on host - recommended for Exchange database and log
volumes
iSCSI• iSCSI direct from a guest virtual machine• iSCSI initiator on host and disk presented to guest as pass-through• ISCSI initiator from guest performs well and is easier to configure• MPIO or EMC PowerPath – PowerPath recommended
Configuration best practices – Hyper-V Storage
28© Copyright 2013 EMC Corporation. All rights reserved.
Exchange virtualization
VMFS RDMVolume can host many virtual machines (or can be dedicated to one virtual machine)
Maps a single LUN to one virtual machine; isolated I/O
Increases storage utilization, provides better flexibility, easier administration, and management
More LUNs = easier to hit the LUN limit of 256 that can be presented to ESX Server
Can’t have hardware enabled VSS backups Required for hardware VSS and replication tools that integrate with Exchange databases
Large third-party ecosystem with V2P products to aid in certain support situations
Can help reduce physical to virtual migration time
Not supported for shared-disk clustering Required for shared-disk clustering
Full support for VMware Site Recovery Manager
Full support for VMware Site Recovery Manager
Configuration best practices – VMware VMFS and RDM Trade-offs
29© Copyright 2013 EMC Corporation. All rights reserved.
Virtualized Exchange Best Practices
For Hyper-V:– Best Practices for Virtualizing Exchange Server 2010 with Windows Server 2008 R2
Hyper-V– Best Practices for Virtualizing and Managing Exchange 2013
For VMware:– Microsoft Exchange 2010 on VMware Best Practices Guide– Microsoft Exchange 2010 on VMware Design and Sizing Examples– Microsoft Exchange 2013 on VMware Best Practices Guide– Microsoft Exchange 2013 on VMware Availability and Recovery Options– Microsoft Exchange 2013 on VMware Design and Sizing Guide
References
30© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage Design and Best Practices• Mailbox Server storage design methodology
• Recommendations and supportably
• Design best practices references
• Supported deployment configurations
• Configuration best practices
31© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Mailbox Server Storage Design Methodology
Phase 1:Gather requirements
•Total number of users•Number of users per server
•User profile and mailbox size
•User concurrency•High availability requirements (DAG configuration)
•Backup and restore SLAs
•Third party software in use (archiving, Blackberry, etc.)
Phase 2:Design the building block and storage architecture
•Design the building-block using Microsoft and EMC best practices
•Design the storage architecture using EMC best practices
•Use EMC Proven Solutions whitepapers
•Use Exchange Solution Review Program (ESRP) documentation
Phase 3:Validate the design
•Use Microsoft Exchange validation tools:•Jetstress - for storage validation
•LoadGen - for user workload validation and end-to-end solution validation
32© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage DesignExchange building-block design methodology
• What is a building-block?• A building-block represents the required amount of resources needed to
support a specific number of Exchange users on a single server or VM• Building blocks are based on requirements and include:
• Compute requirements (CPU, memory, and network)• Disk requirements (database, log, and OS)
• Why use the building-block approach?• Can be easily reproduced to support all users with similar user profile
characteristics• Makes Exchange environment additions much easier and straightforward,
helpful for future environment growth• Has been very successful for many real-world customer implementations
See Appendix for building-block design process
33© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage Design
• Do not solely rely on automated tools when sizing your Exchange environment
• Place time and effort into your calculations, and provide supporting factual evidence on your designs rather than providing fictional calculations
• Size Exchange based on I/O, mailbox capacity and bandwidth requirements
• Factor in other overhead variables such as archiving, snapshots protection, virus protection, mobile devices, and risk factor
• Confirm Exchange storage requirements with specific array sizing tools
Exchange storage sizing
34© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage Design
• EMC Exchange 2010-2013 Designer with DIVA: https://
community.emc.com/docs/DOC-13037?et=watches.email.document
• Microsoft Exchange Server Role Requirements Calculator• Exchange 2010:
http://gallery.technet.microsoft.com/Exchange-2010-Mailbox-Server-Role-/• Exchange 2013: http://
gallery.technet.microsoft.com/Exchange-2013-Server-Role-f8a61780
• VSPEX Sizing Tool • Specific array sizing tools, i.e. VNX Disksizer
• Manual calculation for advanced administrators
Exchange Sizing Tools options
35© Copyright 2013 EMC Corporation. All rights reserved.
General Storage Design Guidance
• Isolate the Exchange server workload to its own set of spindles from other workloads to guarantee performance
• When sizing, always calculate I/O requirements and capacity requirements
• Separate the database and logs onto different volumes• Deploy DAG copies on separate physical spindles• Databases up to 2 TB in size are acceptable when DAG is
being used:− The exact size should be based or customer requirements− Ensure that your solution can support LUNs larger than 2 TB
Best Practices
36© Copyright 2013 EMC Corporation. All rights reserved.
General Storage Design Guidance
• Consider backup and restore times when calculating the database size
• Spread the load as evenly as possible across array resources, V-Max Engines, VNX SPs, back-end buses, etc.
• Always format Windows NTFS volumes for databases and logs with an allocation unit size of 64K
• Use an Exchange building block design approach whenever possible
Best Practices - continued
37© Copyright 2013 EMC Corporation. All rights reserved.
• Either method works well and provides the same performance (thick pools versus RGs)
• RAID groups are limited to 16 disks per RG, pools can support many more disks
• Pools are more efficient and easier to manage
• Use pools if planning to use advanced features such as:− FAST VP, VNX Snapshots
• Storage pools can support a single building block or multiple building blocks based on customer requirements
• Design and expand pools using the correct multiplier for best performance (R1/0 4+4, R5 4+1, R6 6+2)
Pools or RAID Groups?
General Storage Design Guidance - VNX
38© Copyright 2013 EMC Corporation. All rights reserved.
General Storage Design Guidance - VNX
Both Thick and Thin LUNs can be used for Exchange storage (database and logs)
Thick LUNs are recommended for heavy workloads with high IOPS user profiles
Thin LUNs are recommended for light to medium workloads with low IOPS user profiles
– Benefits: significantly reduces initial storage requirements– Use VNX Pool Optimizer before formatting volumes– Can enable FAST Cache or FAST VP for fast metadata promotions
Thick or Thin LUNs?
39© Copyright 2013 EMC Corporation. All rights reserved.
Configuration recommendations (VNX and VMAX)Exchange Storage Design with FAST VP
Separate data from logs, due to different workloads– Data – random workload with skew; high FAST VP benefit– Logs – sequential data without skew; no FAST VP benefit
Use dedicated pools – Provides a better SLA guarantee– Provides fault domains– Microsoft recommends for most deterministic behavior
Use Thick Pool LUNs for highest performance (on VNX)– Thin Pool LUNs are acceptable with Flash in FAST Cache or Pool
Use Thin LUNs on VMAX
40© Copyright 2013 EMC Corporation. All rights reserved.
FAST Cache (VNX only)Exchange Storage Design
• FAST Cache allows the storage system to provide Flash drive class performance to the most heavily accessed chunks of data across the entire system
– Absorbs I/O bursts from applications, thus reducing the load on back-end hard disks.
– Automatically absorbs pools metadata– Improves performance of the storage solution– Can be enabled/disabled on per storage pool bases
41© Copyright 2013 EMC Corporation. All rights reserved.
FAST Cache (VNX only)Exchange Storage Design
FAST Cache usage– Pools with Thin LUNs for metadata tracking– Pools with Thin and Thick LUNs when VNX Snapshots are used– Pools with Thick LUNs
▪ Not required but not restricted either▪ Required with VNX Snapshots
FAST Cache Sizing guidance– Rule of thumb: for every 1 TB of Exchange dataset, provision 1 GB
of FAST Cache▪ Monitor and adjust the FAST Cache size, your mileage may vary▪ Enable FAST Cache on pools with database LUNs only
42© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage design - VMAX Ensure that the initial disk configurations can support the I/O
requirements
Can configure a thin pool to support a single Exchange building block or multiple building blocks, depending on customer requirements
Use Unisphere for VMAX to monitor the thin pool utilization and prevent the thin pools from running out of space
Install the Microsoft hotfix KB2870270 on the Windows Server 2012 hosts in your environment.
43© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Storage design - VMAX
Use Symmetrix Virtual Provisioning
Can share database and log volumes across the same disks, but separate them into different LUNs on the same hosts
For optimal Exchange performance, use striped meta volumes
Design Best Practices
44© Copyright 2013 EMC Corporation. All rights reserved.
VMAX FAST VP with Exchange
When designing FAST VP for Exchange 2010/2013 on VMAX follow these guidelines:
– Separate databases and logs onto their own volumes▪ Can share database and log volumes across the same disks.
– Exclude transaction log volumes from the FAST VP policy or pin all the log volumes into the tier on which they are created
– Select Allocate by FAST Policy to allow FAST VP to use all tiers for new allocations based on the performance and capacity restrictions
▪ New feature introduced in the Enginuity™ 5876 code
– When using FAST VP with Exchange DAG, do not place DAG copies of the same database in the same pool on the same disks
Design Best Practices
45© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Validation Tools
Exchange JetStress and Load Generator Tools - http://technet.microsoft.com/en-us/library/dd335108
Exchange 2010 Mailbox Server Role Requirements Calculator - http://blogs.technet.com/b/exchange/archive/2010/01/22/updates-to-the-exchange-2010-mailbox-server-role-requirements-calculator.aspx
46© Copyright 2013 EMC Corporation. All rights reserved.
XtremCache with Exchange
XtremCache is a server Flash caching solution that reduces latency and increases throughput to improve application performance by leveraging intelligent software and PCIe Flash technology. XtremCache accelerates block I/O reads for those applications that require the highest IOPS
and/or the lowest response time.
XtremCache accelerates reads and protects data using a write-through cache to the networked storage to deliver persistent high availability and disaster recovery.
Works with array-based FAST and FAST Cache software
Optimized for both physical and virtual environments
What is XtremCache?
47© Copyright 2013 EMC Corporation. All rights reserved.
Why XtremCache for Exchange?
Consider XtremCache for Exchange if:– You have an I/O bound Exchange solution– You are not sure about your anticipated workload– You need to guarantee high performance and low latency for specific users (VIP servers,
databases, and so on)
XtremCache proven to improve Exchange performance by:– Reducing read latencies– Increasing I/O throughput– Eliminating almost all high latency spikes– Providing more improvements as workload increases– Reducing RPC latencies– Reducing writes to the back-end storage with XtremCache deduplication
48© Copyright 2013 EMC Corporation. All rights reserved.
XtremCache with Exchange
XtremCache PCI Flash card can be installed on– A physical Exchange Mailbox server– Hypervisor server hosting Exchange Mailbox virtual machines (VMware or
Hyper-V)
Enable XtremCache acceleration only on database volumes
XtremCache sizing guidance:For a 1,000 GB working dataset, configure 10 GB of XtremCache device
Configuration recommendations
49© Copyright 2013 EMC Corporation. All rights reserved.
XtremCache with Exchange
• When implementing XtremCache with VMware vSphere, consider the following:
– Size of the PCI cache card to deploy– Number of Exchange virtual machines deployed on each vSphere
host that will be using XtremCache– Exchange workload characteristics (read:write ratio, user profile
type)– Exchange dataset size
▪ The most benefits will be achieved when all reads from a working dataset are cached
Configuration recommendations
50© Copyright 2013 EMC Corporation. All rights reserved.
XtremCache with Exchange
• When adding a XtremCache device to an Exchange virtual machine:
– Set cache page to 64KB and Max IO size to 64 (BDM I/O will not be cached)
– Can use VSI Plug-in or XtemCache CLI command to set the cache page size to 64 KB and max I/O size to 64 KB when adding the cache device to a virtual machine:vfcmt add -cache <cache_device> -set_page_size 64 -set_max_io_size 64)
Configuration recommendations
51© Copyright 2013 EMC Corporation. All rights reserved.
XtremCache with Exchange
Evaluate your workload before considering enabling deduplication for accelerated Exchange LUNs
Consider CPU overhead when enabling deduplication
Set the deduplication ratio based on workload characteristics:– If the observed deduplication ratio is less than 10%, EMC recommends that
you turn it off (or set it to 0%), which enables you to benefit from extended cache device life.
– If the observed ratio is over 35%, EMC recommends that you raise the deduplication gain to match the observed deduplication.
– If the observed ratio is between 10% and 35%, EMC recommends that you leave the deduplication gain as it is.
Configuration recommendations (deduplication)
52© Copyright 2013 EMC Corporation. All rights reserved.
Backup Best Practices• Backup options
• Hardware and software VSS providers
• Backup best practices
53© Copyright 2013 EMC Corporation. All rights reserved.
Make sure you understand the importance of backups and ask if there are long term retention requirementsUsing DAG HA copy/lagged copy for point-in-time backups is an option, however consider the following:
Backups are often subject to regulatory requirements Extended time retentionThere are scenarios that DAG HA/lagged copy will not addressIf lagged copy is activated, new copy must created again
Exchange Backup Best Practices
54© Copyright 2013 EMC Corporation. All rights reserved.
Streaming backups are no longer allowed or supported in Exchange 2010/2013EMC recommends leveraging VSS technology for consistent replication and backup of Exchange databases and log files because it:
Provides rapid recovery through hardware VSS providersDecreases the hardware requirements for backupsIntegrates data deduplication mechanisms Alleviates hardware and resource contention at the server level
Exchange Backup Best Practices
55© Copyright 2013 EMC Corporation. All rights reserved.
There are two VSS providers: hardware-based and software-based. Know the difference:Hardware-based providers:
Act as an interface between Volume Shadow Copy Service and the hardware level by working with a hardware storage adapter or controllerPerform a shadow copy by the storage appliance outside of the OSDepend on hardware to take the clone/snapshotInclude VNX SnapView, VNX Snapshots, and Symmetrix TimeFinder
Exchange Backup Best Practices
• Software-based providers:– Intercept and process I/O requests in a software layer between the
file system and volume manager – Are implemented as user-mode DLL and a kernel mode device
drivers. The shadow copy is performed by the software– Are not dependent on hardware, so there is a wider range of platform
support– Include Avamar, Networker NMM, and NetBackup
56© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Backup Best Practices• Snaps are preferred with VSS
− Consistency checks are no longer required with two or more DAG copies − Less storage space required
• Protected clones are possible, but consider the following: – Larger database sizes – Perform backup from a passive DAG copy– Activity during consistency check, backup
• Backup and restore have the same granularity with hardware VSS:– Recovery is performed at the LUN level. Consider this during design and layout– Consider leveraging AppSync with ItemPoint for single item recovery
57© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Validated Solutions
58© Copyright 2013 EMC Corporation. All rights reserved.
Exchange 2013 ESRP on VNX5700
59© Copyright 2013 EMC Corporation. All rights reserved.
Exchange with XtremSW Cache
60© Copyright 2013 EMC Corporation. All rights reserved.
VSPEX Solutions for Exchange
61© Copyright 2013 EMC Corporation. All rights reserved.
ACCELERATING MICROSOFT EXCHANGE PERFORMANCE WITH EMC XtremCache
EMC VNX STORAGE AND VMWARE VSPHERE
Solution Completed in March 2013
62© Copyright 2013 EMC Corporation. All rights reserved.
Solution Architecture
63© Copyright 2013 EMC Corporation. All rights reserved.
Exchange 2010 Building block detailsTotal number of mailboxes per server 5,000 mailboxes/serverMailbox size 1.5 GB per userUser profile 150 messages/user/day (0.150 IOPS)Target average message size 75 KB
Database Design
• 6 Databases/server
• 833 user per DB
• DB size ~1300 GB (LUN size 1650 GB)Log Design 6 Log LUNs (90 GB LUN size)
Number of Exchange Mailbox VMs per ESX 3
Disk configuration per Server 18 (16 DB+2 Logs), 2TB NL-SAS Drives
Memory/CPU per VM Recommended 32 GB RAM, 29040 CPU Megacycles
Deleted items retention window (“dumpster”) 14 days
Logs protection buffer 3 days
24 x 7 BDM configuration Enabled
Database read/write ratio 3:2 (in Mailbox Resiliency configuration)
64© Copyright 2013 EMC Corporation. All rights reserved.
Storage design with XtremSW Cache Two storage pools created for
databases– 48 x 2 TB 7.2k rpm NL-SAS drives per pool, RAID 1/0
Each pool contains multiple copies from different VM’s
– 3 Building blocks (3 VM’s)– 18 x 1.6 TB LUNs (6 LUNs per VM)
326 GB VFMS datastore is created from the XtremCache PCI card on each vSphere server
– 50 GB cache devices created for each Exchange VM from the VMFS cache datastore
– Remaining capacity reserved for VM’s that can be migrated from the other vSphere server
65© Copyright 2013 EMC Corporation. All rights reserved.
Additional References
66© Copyright 2013 EMC Corporation. All rights reserved.
Additional References• Exchange Storage Best Practices and Design Guidelines for EMC Storage
whitepaper:http://www.emc.com/collateral/hardware/white-papers/h8888-exch-2010-storage-best-pract-design-guid-emc-storage.pdf
• EMC Community Network− https://community.emc.com/community/connect/everything_microsoft
• EMC and Partner Exchange 2010 Tested Virtualized Solutions– http://technet.microsoft.com/en-us/library/gg598215.aspx– http://www.emc.com/collateral/hardware/white-papers/h7337-exchange-unified-cisco-hyper-v-wp.pdf – http://www.emc.com/collateral/software/white-papers/h7410-zero-data-loss-exchange-wp.pdf
• Exchange Solution Reviewed Program Submissions (ESRP)– http://technet.microsoft.com/en-us/exchange/ff182054.aspx
Exchange Mailbox Server Storage Design (Microsoft TechNet)– http://technet.microsoft.com/en-us/library/dd346703.aspx
67© Copyright 2013 EMC Corporation. All rights reserved.
Additional References
• Exchange virtualization supportability guidance - http://technet.microsoft.com/en-us/library/jj126252.aspx
• Understanding Exchange Performance - http://technet.microsoft.com/en-us/library/dd351192
• Server Virtualization Validation Program - http://www.windowsservercatalog.com/svvp/
• Exchange 2010 EMC-tested OEM solutions (on Hyper-V)– 20,000 users on EMC storage with virtual provisioning http
://technet.microsoft.com/en-us/library/gg598215(v=exchg.141).aspx– 32,400 users on EMC storage with EMC REE http
://technet.microsoft.com/en-us/library/hh145600(v=exchg.141).aspx
68© Copyright 2013 EMC Corporation. All rights reserved.
Appendix• Building block design process• ESI for VNX Pool Optimization
69© Copyright 2013 EMC Corporation. All rights reserved.
Building block design process
70© Copyright 2013 EMC Corporation. All rights reserved.
Exchange Mailbox Server Storage Design Methodology
Phase 1:Gather requirements
•Total number of users•Number of users per server
•User profile and mailbox size
•User concurrency•High availability requirements (DAG configuration)
•Backup and restore SLAs•Third party software in use (archiving, blackberry, etc.)
Phase 2:Design the building block and storage architecture
•Design the building block using Microsoft and EMC best practices
•Design storage architecture using EMC best practices
•Leverage EMC Proven Solutions whitepapers
•Leverage Exchange Solution Review Program (ESRP) documentation
Phase 3:Validate the design
•Use Microsoft Exchange validation tools
• Jetstress - for storage validation
•LoadGen - for user workloads validation and end-to-end solution validation
71© Copyright 2013 EMC Corporation. All rights reserved.
Requirements Gathering ExampleItem Value Exchange version. Total number of active users (mailboxes) in the environment
Exchange 2013, 20,000
Site resiliency requirements Single siteStorage Infrastructure SANType of deployment (Physical or Virtual) Virtual (VMware vSphere)HA requirements One DAG with two database copiesMailbox size limit 2 GB max quota
User profile200 messages per user per day (0.134 IOPS)
Target average message size 75 KBOutlook mode Cached mode, 100 percent MAPINumber of mailbox servers 8Number of mailboxes per server 5,000 (2,500 active/2,500 passive)Number of databases per server 10Number of users per database 500Deleted items retention (DIR) period 14 daysLog protection buffer (to protect against log truncation failure) 3 daysBDM configuration Enabled 24 x7Database read/write ratio 3:2 (60/40 percent) in a DAG configurationUser concurrency requirements 100 percentThird-party software that affects space or I/O (for example, Blackberry, snapshots)
Storage snapshots for data protection
Disk Type 3 TB NL-SAS (7,200 rpm)Storage platform VNX
72© Copyright 2013 EMC Corporation. All rights reserved.
Building Block design
In our example we are defining a building block as:– A mailbox server that will support 5,000 users
▪ 2,500 users will be active during normal runtime and the other 2,500 users will be passive until a switchover from another mailbox server occurs.
Each building block will support two database copies.
Define and design Building Block
73© Copyright 2013 EMC Corporation. All rights reserved.
Building block sizing and scaling process
Perform calculations for IOPS requirements
Perform calculations for capacity requirements based on different RAID types
Determine the best option
Scale building block– Multiple building blocks may be combined together to
create the final configuration and storage layout (pools or RGs)
74© Copyright 2013 EMC Corporation. All rights reserved.
Building block sizing and scaling process
Front-end IOPS ≠ Back-end IOPS – Front-end IOPS = Total Exchange Mailbox server IOPS– Back-end IOPS = Storage array IOPS (including RAID penalty)
Understand disk IOPS by RAID type– Block front-end Exchange application workload is translated into a different back-end
disk workload based on the RAID type in use.– For reads there is no impact of RAID type:
▪ 1 application read I/O = 1 back-end read I/O
– For random writes like Exchange:▪ RAID 1/0: 1 application write I/O = 2 back-end write I/O▪ RAID 5: 1 application write I/O = 4 back-end disk I/O (2 read I/O + 2 write I/O)▪ RAID 6: 1 application write I/O = 6 back-end write I/O (3 read I/O + 3 write I/O)
75© Copyright 2013 EMC Corporation. All rights reserved.
Formula & Calculations
Database IOPS requirements
Total transactional IOPS = IOPS per mailbox * mailboxes per server + (Microsoft recommended overhead %)
Total transactional IOPS = 5,000 users * 0.134 IOPS per user + 20% Microsoft recommended overhead = 670 + 134 = 804 IOPS
Total front-end IOPS = (Total transactional IOPS) + (EMC required overhead %)
Total Front-end IOPS = 804 + 20% EMC required overhead = 965 IOPS (rounded-up from 964.8)
76© Copyright 2013 EMC Corporation. All rights reserved.
Database Disks requirements for Performance (IOPS)
Disks required for Exchange database IOPS = (Total backend database Read IOPS) + (Total backend database Write IOPS)/ Exchange random IOPS per disk
Where:
Total back-end database read IOPS = (Total Front-end IOPS) * (% of Reads IOPS)
Total back-end database write IOPS = RAID Write Penalty *(Total Front-end IOPS * (% of Write IOPS)
Formula
77© Copyright 2013 EMC Corporation. All rights reserved.
Database Disks requirements for Performance (IOPS)
RAID Option RAID Penalty Disks required
RAID 1/0 (4+4) 2(965 x 0.60) + 2(965 x 0.40) = 579 + 772 = 1351 / 65 = 21 (round-up to 24 disks)
RAID 5 (4+1) 4(965 x 0.60) + 4(965 x 0.40) = 579 + 1544 = 2123 / 65 = 33 (round-up to 35 disks)
RAID 6 (6+2) 6(965 x 0.60) + 6(965 x 0.40) = 579 + 2316 = 2895 / 65 = 45 disks (round-up to 48 disks)
Calculations
78© Copyright 2013 EMC Corporation. All rights reserved.
Formula & Calculations
Transactional logs IOPS requirements
Disks required for Exchange log IOPS = (Total backend database Write IOPS * 50%) + (Total backend database Write IOPS * 10%)/ Exchange sequential IOPS per disk
Disks required for Exchange log IOPS = (772 back-end write IOPS * 50%) + (772 *10%))/ 180 sequential Exchange IOPS per disk = (386 + 77.2)/180 = 2.57(round-up to 4 disks)
79© Copyright 2013 EMC Corporation. All rights reserved.
Formula
Storage capacity calculations
Calculate User Mailbox on Disk
Calculate Database Size on Disk
Calculate Database LUN Size
Mailbox size on disk = Maximum mailbox size + White space + Dumpster
Database size on disk = number of mailboxes per database * mailbox size on disk
Database LUN size = Number of mailboxes * Mailbox size on disk * (1 + Index space + additional Index space for maintenance) / (1 + LUN free space)
80© Copyright 2013 EMC Corporation. All rights reserved.
FormulaMailbox size on disk
Mailbox size on disk = Maximum mailbox size + White space + Dumpster
Where:Estimated Database Whitespace per Mailbox = per-user daily message profile * average message size
Where:Dumpster = (per-user daily message profile * average message size * deleted item retention window) + (mailbox quota size * 0.012) + (mailbox quota size * 0.03)
81© Copyright 2013 EMC Corporation. All rights reserved.
CalculationsMailbox size on disk
White space = 200 messages /day * 75KB = 14.65MB
Dumpster = (200 messages/day * 75KB * 14 days) + (2GB * 0.012) + (2GB x 0.03) = 205.1 + 24.58 + 61.44= 291.12MB
Mailbox size on disk = 2GB mailbox quota + 14.65MB database whitespace + 291.12MB Dumpster = 2,354 MB (2.3GB)
82© Copyright 2013 EMC Corporation. All rights reserved.
CalculationsDatabase Size On Disk & LUN size
Database size on disk = 500 users per database * 2,354 MB mailbox on disk = 1,177 GB (1.15 TB)
Database LUN size = 1,177 GB * (1 + 0.2 + 0.2) / (1 - 0.2) = 2,060 (2 TB)
In our example:– 20% added for the Index– 20% added for the Index maintenance task– 20% added for LUN-free space protection
83© Copyright 2013 EMC Corporation. All rights reserved.
Formula & CalculationsLogs space calculations
Log LUN size = (Log size)*(Number of mailboxes per database)*(Backup/truncation failure tolerance days)+ (Space to support mailbox moves)/(1 + LUN free space)
Log Capacity to Support 3 Days of Truncation Failure = (500 mailboxes/database x 40 logs/day x 1MB log size) x 3 days = 58.59GB
Log Capacity to Support 1% mailbox moves per week = 500 mailboxes/database x 0.01 x 2.3GB mailbox size = 11.5GB
Log LUN size = 58.59GB + 11.5 GB /(1 - 0.2) = 87.61 GB (round-up to 88 GB)
84© Copyright 2013 EMC Corporation. All rights reserved.
Total Capacity per Building Block
LUN Capacity Type LUN Capacity Required per server
Database LUN capacity 2,060 GB per LUN * 10 LUNs per server = 20,600 GB
Log LUN capacity 88 GB per LUN * 10 LUNs per server = 880 GBTotal LUN capacity per server
20,600 + 880 = 21,480 GB
Total LUN size capacity required per server = (Database LUN size per server) + (Log LUN size per server) * (Number of databases per server)
85© Copyright 2013 EMC Corporation. All rights reserved.
Total number of disks required
Database disks
Logs disks
Disks required for Exchange database capacity = Total database LUN size / Physical Disk Capacity * RAID Multiplication Factor
Disks required for Exchange log capacity = Total log LUN size) / Physical Disk Capacity * RAID Multiplication Factor
86© Copyright 2013 EMC Corporation. All rights reserved.
Disk requirements based on capacity
RAID Option Database Disks requiredRAID 1/0 (4+4) 20,600/2794.5 * 2 = 7.37 * 2 = 14.74 (round-up to 16 disks)
RAID 5 (4+1) 20,600/2794.5 * 1.25 = 7.37 * 1.25 = 9.2 (round-up to 10 disks)
RAID 6 (6+2) 20,600/2794.5 * 1.33 = 7.37 * 1.33 = 9.8 (round-up to 16 disks)
RAID Option Logs Disks requiredRAID 1/0 (1+1) 880 / 2,794.5 * 2 = 0.63 (round up to 2 disks)
87© Copyright 2013 EMC Corporation. All rights reserved.
Building Block SummaryFinal storage calculation results
Volume Type RAID Option
Disks Required for Performance (IOPS)
Disks Required for Capacity
Best Option
Exchange Databases
RAID 1/0 (4+4) 24 disks 16 disks 24 disks
RAID 5 (4+1) 35 disks 10 disks
RAID 6 (6+2) 48 disks 16 disks
Exchange Logs RAID 1/0 (1+1) 4 disks 2 disks 4 disks
Total disks per building block 28 disks
88© Copyright 2013 EMC Corporation. All rights reserved.
Building Block Scalability
Final storage calculation results
Total number of disks required for entire 20,000 users solution in a DAG with two copies = 28 disk per building block * 8 building blocks = 224 disks total
89© Copyright 2013 EMC Corporation. All rights reserved.
Bandwidth calculations
Building block sizing and scaling process
• Array throughput MB/s validation for Exchange involves:• Determining how many databases the customer will require• Confirming the database LUNs are evenly distributed among the backend busses and
storage processors.• Determine if each bus can accommodate the peak Exchange database throughput
• Use this calculation to calculate the throughput required
(DB throughput * number of DBs per bus) = Exchange DB throughput
• Compare that number with array bus throughputDB throughput = Total transactional (user) IOPS per DB * 32K + (BDM throughput per DB in MB/s)Number of DBs per bus = the total number of active and passive databases per bus
90© Copyright 2013 EMC Corporation. All rights reserved.
The process
Storage Bandwidth Requirements
The bandwidth validation process involves the following steps:1. Determine how many databases in the Exchange environment2. Determine the bandwidth requirements per database3. Determine the required bandwidth requirements per array bus4. Determine whether each bus can accommodate the peak Exchange
database bandwidth• Use DiskSizer for VNX or contact your local storage specialist to get the array and bus throughput numbers
• DiskSizer is available through your local USPEED contact
5. Evenly distribute database LUNs among the back-end buses and storage processors• Uniformed distribution is key for best performance
• FE/BE/RAID Group/POOLs & DAEs• DBs uniformly distributed onto the pools• Use even numbers on the SAS loops (0 & 2) for max performance
91© Copyright 2013 EMC Corporation. All rights reserved.
Calculations
Storage Bandwidth Requirements
Bandwidth per database (MB/s) = Total transactional IOPS per database * 32 KB + Estimated BDM Throughput per database (MB/s)Where:• 32 KB is an Exchange page size• Estimated BDM throughput per database is 7.5 MB/s for Exchange 2010 and 2.25 MB/s for Exchange 2013
Required throughput MB/s per bus = (throughput MB/s per database) * (total number of active and passive databases per bus)
92© Copyright 2013 EMC Corporation. All rights reserved.
Calculations
Storage Bandwidth Requirements
Total transactional IOPS per database = (500 * 0.134 * 32 KB = 2.1 MB/s Throughput per database = 2.1 MB/s + 2.25 MB/s = 4.35 MB/s Required throughput per bus = 4.35 MB/s * 200 databases per bus = 870 MB/s
Example assumptions:• 500 users at 0.134 IOPS per database• 200 databases per bus
If the array supports a maximum throughput of 3,200 MB/s per bus, 200 databases can be supported from a throughput perspective.
93© Copyright 2013 EMC Corporation. All rights reserved.
• Configured dedicated storage pools for each mailbox server with 24 x 3 TB NL-SAS drives.
• Each storage pool holds two copies from different mailbox servers.
• Separated Exchange log files into different storage pools
• For better storage utilization created one storage pool with 16 x 3 TB NL-SAS drives for logs per four mailbox server building blocks.
Final Design
94© Copyright 2013 EMC Corporation. All rights reserved.
Storage Design validation
• Exchange Server Jetstress − Uses Exchange executables to simulate I/O load (use same version)− Initialized and executed during pre-production before Exchange Server is installed− Throughput and mailbox profile tests – Pass gives confidence that storage design will
perform as designed
• Exchange Server Load Generator (LoadGen) (optional)
− Validation must be performed in isolated lab− Produces a simulated client workload against a test Exchange deployment− Estimate number of users per server and validate Exchange deployment− LoadGen testing can take many weeks to configure and populate DBs
95© Copyright 2013 EMC Corporation. All rights reserved.
Storage Design validation• Exchange Solution Reviewed Program (ESRP)
Results• Microsoft program for validation of Storage vendor designs with Exchange
• Vendor runs multiple JetStress tests based on requirements for performance, stress, backup to disk, and log file replay
• Reviewed and approved by Microsoft
http://technet.microsoft.com/en-us/exchange/ff182054.aspx
96© Copyright 2013 EMC Corporation. All rights reserved.
ESI for VNX Pool Optimization Utility (a.k.a. SOAP Tool)
97© Copyright 2013 EMC Corporation. All rights reserved.
VNX Storage Pool Optimizer Utility
What is it?– A utility for optimization of pool-based LUNs (thin or thick)
that allows to achieve the maximum performance– Provides the best option for Exchange or any application
requiring deterministic high performance across all LUNs in the pool equally
– Pre-allocates slices in a pool evenly across all disks and private raid groups
98© Copyright 2013 EMC Corporation. All rights reserved.
VNX Storage Pool Optimizer Utility
Why do I need to use it?– To achieve the best performance for pool-based LUNs
(primarily thin) and to pass Jetstress during pre-deployment storage validation
– To mitigate “Jetstress effect”
99© Copyright 2013 EMC Corporation. All rights reserved.
When to Use Optimizer Utility
• Use Cases• CX4/VNX (OE for Block prior to 5.32 ) - Thick LUNs only
• With VNX OE 5.32 for block thick pool LUNs are pre-allocated on creation
• VNX Rockies (OE for Block 5.33 ) – Thin and Thick LUNs (primarily Thin)
• Use SOAP tool utility with CX4 and VNX OE for Block Release 32 (Inyo)
• Use new ESI for VNX Pool Optimization utility with VNX OE for Block Release 33 (Rockies)• The short-term plan is to merge both tools into one• The long-term plan is to implement the functionality into native VNX code.
100© Copyright 2013 EMC Corporation. All rights reserved.
What is the problem? How is the issue surfaced?
– With Jetstress testing first database on the Exchange server will:
▪ Experience higher latencies then the others when LUN is Thick
▪ Experience lower latencies then the others when LUN is Thin
“JetStress effect”– Jetstress data population results in
imbalances of underlying virtual disks in a pool
101© Copyright 2013 EMC Corporation. All rights reserved.
Jetstress Initialization Process
How Jetstress initialization phase works:
• Jetstress creates first database• It then creates other databases
by copying the first database to other databases concurrently
102© Copyright 2013 EMC Corporation. All rights reserved.
Looking under the covers… Slice Maps
• Without Optimization • With Optimization
103© Copyright 2013 EMC Corporation. All rights reserved.
VMDK Optimization
• Without Optimization • With Optimization
105© Copyright 2013 EMC Corporation. All rights reserved.
Old SOAP Utility – Where and how?• Old SOAP Utility is available on EMC Online Support site
– Enter “soap” in the search and select “Support Tools”
• Must be used with CX4/VNX Inyo (OE 5.32) only– Support Thick LUN optimization only
• Zip file contains the tool, step-by-step documentation, and demo video
106© Copyright 2013 EMC Corporation. All rights reserved.
ESI for VNX Pool Optimization Utility
• Available for download from EMC Online Support site in November 2013
• Can be used with Next Gen VNX Series(OE 5.33) only (VNX5200, VNX5400, VNX5600,VNX5800, VNX7600, VNX8000)
• Supports Thick and Thin LUN optimization