Microsoft Exchange Best Practices and Design Guidelines on EMC Storage

1© Copyright 2014 EMC Corporation. All rights reserved.© Copyright 2014 EMC Corporation. All rights reserved.

Microsoft ExchangeBest Practices and Design Guidelines on EMC Storage• Exchange 2010 and Exchange 2013• VNX and VMAX Storage Systems

EMC Global Solutions | Global Solutions Engineering

September 2014 Update


Exchange – What has changed

Exchange Virtualization

Exchange Storage Design and Best Practices

Exchange DR options

Exchange Validated Solutions

VNX Pool Optimization Tool (aka SOAP)

Building Block Design Example

Topics





Exchange DR options





Exchange…What has changed

Exchange 2007

◊ 64-bit Windows◊ 32+ GB database cache◊ 8Kb block size◊ 1:1 DB read/write ratio◊ 70% reduction in IOPS from Exchange 2003

Exchange 2010

◊ 64-bit Windows◊ 100GB database cache (DAG)◊ 32Kb block size◊ 3:2 DB read/write ratio◊ 70% reduction in IOPS from Exchange 2007

Exchange 2013

◊ 64-bit Windows◊ 100GB database cache (DAG)◊ 32Kb block size◊ 3:2 DB read/write ratio◊ 33% reduction in IOPS from Exchange 2010


Exchange User Profile ChangesMessages sent/

received per mailbox per day

Exchange 2010 Estimated IOPS per mailbox (Active or Passive)

Exchange 2013 Estimated IOPS per mailbox(Active or Passive)

Mailbox resiliency Stand-alone

50 0.05 0.06 0.034

100 0.100 0.120 0.067

150 0.150 0.180 0.101

200 0.200 0.240 0.134

250 0.250 0.300 0.168

300 0.300 0.360 0.201

350 0.350 0.420 0.235

400 0.400 0.480 0.268

450 0.450 0.540 0.302

500 0.500 0.600 0.335


Exchange Processor Requirements ChangesMessages sent or

received per mailbox per day

Megacycles per UserActive DB Copy or

Standalone (MBX only)

Megacycles per UserActive DB Copy or

Standalone (Multi-Role)

Megacycles per UserPassive DB Copy

Exchange 2010

Exchange 2013

Exchange 2010

Exchange 2013

Exchange 2010

Exchange 2013

50 1 2.13 N/A 2.66 0.15 0.69

100 2 4.25 N/A 5.31 0.3 1.37

150 3 6.38 N/A 7.97 0.45 2.06

200 4 8.50 N/A 10.63 0.6 2.74

250 5 10.63 N/A 13.28 0.75 3.43

300 6 12.75 N/A 15.94 0.9 4.11

350 7 14.88 N/A 18.59 1.05 4.80

400 8 17.00 N/A 21.25 1.2 5.48

450 9 19.13 N/A 23.91 1.35 6.17

500 10 21.25 N/A 26.56 1.5 6.85


Exchange I/O Characteristics

I/O Type Exchange 2007 Exchange 2010 Exchange 2013

Database I/O 8 KB random write I/O 32 KB random I/O 32 KB random I/O

Background Database Maintenance (BDM) I/O

N/A256 KB Sequential Read I/O

256 KB Sequential Read I/O

Log I/OVaries in size from 512 bytes to the log buffer size (1 MB)

Varies in size from 4 KB to the log buffer size (1 MB)

Varies in size from 4 KB to the log buffer size (1 MB)


Exchange 2010/2013 mailbox database I/O read/write ratios

Messages sent/received per mailbox per day

Stand-alone databasesDatabases participating in mailbox resiliency

50 1:1 3:2

100 1:1 3:2

150 1:1 3:2

200 1:1 3:2

250 1:1 3:2

300 2:3 1:1

350 2:3 1:1

400 2:3 1:1

450 2:3 1:1

500 2:3 1:1


Understanding Exchange I/O

• Exchange 2010/2013 I/O’s to the database (.edb) are divided into two types: – Transactional I/O (aka user I/O)

• Database volume I/O (database reads and writes)• Log volume I/O (logs reads and writes)

– Non Transactional I/O• Background Database Maintenance (Checksum) (BDM)

NOTE: Only database I/O’s are measured when sizing storage and during Jetstress validation

For more details see “Understanding Database and Log Performance Factors” at http://technet.microsoft.com/en-us/library/ee832791(v=exchg.141).aspx

http://technet.microsoft.com/en-us/library/ee832791(v=exchg.141).aspx


Background Database Maintenance (BDM) BDM is the process of Exchange Server 2010/2013 database maintenance that includes

online defragmentation and online database scanning

Both active and passive database copies are scanned– On active copy can be scheduled to run during the online maintenance window (default is 24 x 7)– Passive copy is ”hardcoded” to 24 x 7 scan– Jetstress has no concept of passive copy, all are active

Possible BDM related issues (mostly for Exchange 2010): • Bandwidth/throughput required for BDM and BDM IOPS• Not enough FE ports, not enough BE ports, non-optimal RAID configuration

Exchange 2010 Exchange 2013

Read I/O size 256 KB 256 KB

Database scan completion 1 week every 4 weeks

IOPS per database 30 9

Bandwidth 7.5 MB/s* 2.25 MB/s*


Exchange Content Index Considerations• Content Indexing space considerations:

– In Exchange 2010 content index space is estimated at about 10% of the database size.

– In Exchange 2013 content index space is estimated at about 20% of the database size. • An additional 20% must be added for content indexing maintenance tasks (such

as the master merge process) to complete.

• Calculations References http://blogs.technet.com/b/exchange/archive/2013/05/06/ask-the-perf-guy-sizing-exchange-2013-deployments.aspx

http://blogs.technet.com/b/exchange/archive/2013/05/06/ask-the-perf-guy-sizing-exchange-2013-deployments.aspx




Exchange High AvailabilityKey Terminology

Term DescriptionActive Manager An internal Exchange component which runs inside the Microsoft Exchange Replication

service that's responsible for failure monitoring and corrective action through failover within a database availability group (DAG).

AutoDatabaseMountDial A property setting of a Mailbox server that determines whether a passive database copy will automatically mount as the new active copy, based on the number of log files missing by the copy being mounted.

Continuous replication - block mode In block mode, as each update is written to the active database copy's active log buffer, it's also shipped to a log buffer on each of the passive mailbox copies in block mode. When the log buffer is full, each database copy builds, inspects, and creates the next log file in the generation sequence.

Continuous replication - file mode In file mode, closed transaction log files are pushed from the active database copy to one or more passive database copies.

Database availability group (DAG) A group of up to 16 Exchange 2013 Mailbox servers that hosts a set of replicated databases.

Database mobility The ability of an Exchange 2013 mailbox database to be replicated to and mounted on other Exchange 2013 Mailbox servers.

Datacenter Activation Coordination mode

A property of the DAG setting that, when enabled, forces the Microsoft Exchange Replication service to acquire permission to mount databases at startup.



Term DescriptionDisaster recovery Any process used to manually recover from a failure. This can be a failure that

affects a single item, or it can be a failure that affects an entire physical location.

Exchange third-party replication API An Exchange-provided API that enables use of third-party synchronous replication for a DAG instead of continuous replication.

High availability A solution that provides service availability, data availability, and automatic recovery from failures that affect the service or data (such as a network, storage, or server failure).

Lagged mailbox database copy A passive mailbox database copy that has a log replay lag time greater than zero.

Mailbox database copy A mailbox database (.edb file and logs), which is either active or passive.

Mailbox resiliency The name of a unified high availability and site resilience solution in Exchange 2013.

Managed availability A set of internal processes made up of probes, monitors, and responders that incorporate monitoring and high availability across all server roles and all protocols.



Term DescriptionA switchover Is a manual activation of one or more database copies

A failover Is an automatic activation of one or more database copies after a failure.

Safety Net Formerly known as transport dumpster, this is a feature of the transport service that stores a copy of all messages for X days. The default setting is 2 days.

Shadow redundancy A transport server feature that provides redundancy for messages for the entire time they're in transit.

Site resilience A configuration that extends the messaging infrastructure to multiple Active Directory sites to provide operational continuity for the messaging system in the event of a failure affecting one of the sites.


DB1

DB3

DB2Copy

Copy

Copy

Copy

Copy

Copy

Database Availability Group

A = Active

P = Passive

A

A

A

P

P

P

P

P

P

Exchange High Availability

Base component of the high availability and site resilience framework built into Exchange 2010/2013

A group of servers participating within a Windows failover cluster with a limit of 16 servers and 100 databases.

All servers participating within a DAG can have a copy of any database within the DAG

– Each DAG member server can house one copy of each database, up to 16 copies, with only one being active, passive, or lagged

No configuration of cluster services are required– Exchange 2010/2013 handles the entire installation– During site DR - manual work, scripts must be run

A DAG does not provide recovery for logical database corruption

Database Availability Group (DAG)

MBX1 MBX3MBX2


Exchange High Availability

Ensure all elements of the design have resilient components – Storage processors– Connectivity to the servers– Storage spindles – Multiple arrays in DR scenarios

DAG copies should be stored on separate physical spindles– Provided all resiliency is reached at the source site

On SANs, consider performance of the passive and active copies within one array

Guidance for deploying DAGs








References


Exchange 2010/2013 virtualization

• Virtualizing Exchange is supported on Hyper-V, VMware, and other hypervisors

• Hypervisor vendors must participate in the Windows Server Virtualization Validation Program (SVVP)

• EMC recommends virtualizing Exchange for most deployments based on customer requirements

http://go.microsoft.com/fwlink/?LinkId=125375


Exchange virtualization

Do not deploy VMs from the same DAG on the same host

VM placement considerations

Deploy VMs with the same role across multiple hosts

DAG1 DAG2

DAG1

DAG2



• Physical sizing still applies– Hypervisor server must accommodate the guests they will support– DAG copies must be spread out across physical hosts to minimize outage in case of

physical server issues

• Know your hypervisors limits– 256 SCSI disks per host (or cluster)– Processor limits (vCPUs per virtual machine) and Memory limits

• Be aware of the hypervisor CPU overhead– Microsoft Hyper-V: ~10-12%– VMware vSphere: ~5-7%

• Core Exchange design principles still apply– Design for performance and high availability– Design for user workloads

Configuration best practices



• Hypervisor server– Have at least 4 paths (HBA/CNA/iSCSI) to the storage

– Install EMC PowerPath for maximum throughput, load balancing, path management, and I/O path failure detection

– Multiple NICs - Segregate management and clients traffic from Exchange replication traffic

– Disable hypervisor-based auto tuning features - No dynamic memory

• CPU & Memory– Dedicate/reserve CPU and memory to the Mailbox virtual machines and do not over

commit

– pCPU to vCPU ratios: 2:1 is OK, 1:1 is a best practice

• VM migrations– Always migrate live or completely shut down virtual machines

Configuration best practices – Hypervisor and VMs



• Exchange storage should be on spindles separate from guest OS physical storage

• Exchange storage must be block-level– Network attached storage (NAS) volumes are not supported– No NFS, SMB (other than SMB 3.0), or any other NAS technology– Storage must be fixed VHD/VHDX/VMDK, SCSI pass-through/RDM

or iSCSI

Configuration best practices - Storage



• Only in virtualized configurations

• VHDs can reside on SMB 3.0 shares presented to Hyper-V host

• No support for UNC path for Exchange db and log volumes (\\server\share\db1\db1.edb)

Configuration best practices - SMB 3.0 Support


Exchange virtualizationSupported SMB 3.0 Configuration Example



Virtual SCSI (pass-through or fixed disk)• VHD on host – recommended for OS, program files• Pass-through disk on host - recommended for Exchange database and log

volumes

iSCSI• iSCSI direct from a guest virtual machine• iSCSI initiator on host and disk presented to guest as pass-through• ISCSI initiator from guest performs well and is easier to configure• MPIO or EMC PowerPath – PowerPath recommended

Configuration best practices – Hyper-V Storage



VMFS RDMVolume can host many virtual machines (or can be dedicated to one virtual machine)

Maps a single LUN to one virtual machine; isolated I/O

Increases storage utilization, provides better flexibility, easier administration, and management

More LUNs = easier to hit the LUN limit of 256 that can be presented to ESX Server

Can’t have hardware enabled VSS backups Required for hardware VSS and replication tools that integrate with Exchange databases

Large third-party ecosystem with V2P products to aid in certain support situations

Can help reduce physical to virtual migration time

Not supported for shared-disk clustering Required for shared-disk clustering

Full support for VMware Site Recovery Manager

Full support for VMware Site Recovery Manager

Configuration best practices – VMware VMFS or RDM Trade-offs



• For Hyper-V:– Best Practices for Virtualizing Exchange Server 2010 with Windows Server 2008 R2

Hyper-V– Best Practices for Virtualizing and Managing Exchange 2013

• For VMware:– Microsoft Exchange 2010 on VMware Best Practices Guide– Microsoft Exchange 2010 on VMware Design and Sizing Examples– Microsoft Exchange 2013 on VMware Best Practices Guide– Microsoft Exchange 2013 on VMware Availability and Recovery Options– Microsoft Exchange 2013 on VMware Design and Sizing Guide

References

http://www.microsoft.com/en-us/download/details.aspx?id=2428





http://download.microsoft.com/download/4/A/C/4AC32FD3-220E-45DC-AA97-DBDBE19C15B2/Best_Practices_for_Virtualizing_and_Managing_Exchange_2013.pdf

http://download.microsoft.com/download/4/A/C/4AC32FD3-220E-45DC-AA97-DBDBE19C15B2/Best_Practices_for_Virtualizing_and_Managing_Exchange_2013.pdf

http://www.vmware.com/files/pdf/Exchange_2010_on_VMware_-_Best_Practices_Guide.pdf





http://www.vmware.com/files/pdf/exchange-2010-on-vmware-design-and-sizing-examples.pdf



http://www.vmware.com/files/pdf/Exchange_2013_on_VMware_Best_Practices_Guide.pdf




http://www.vmware.com/files/pdf/Exchange_2013_on_VMware_Availability_and_Recovery_Options.pdf





http://www.vmware.com/files/pdf/Exchange_2013_on_VMware_Design_and_Sizing_Guide.pdf









Exchange DR options





Exchange Storage Options

• Understand which storage type best meets design requirements

− Physical or virtual?− Dedicated for Exchange or shared with other applications?

• Follow EMC proven guidance for each platform

DAS or SAN?EMC offers both options

• For small, low-cost = DAS• For large-scale efficiency = SAN• Best long-term TCO = SAN• Virtualization ready = SAN

VNXe VNX VMAX XtremIO


Exchange Server IOPS Per Disks• Use the following table for IOPS per drive values when

calculating disk requirements for Exchange 2010/2013*

Disk typeExchange 2010/2013

database IOPS per disk

(random workload)

Exchange Server 2010/2013 database logs IOPS per disk

(Sequential workload)

VNX/VMAX VNX VMAX VNX and VMAX

7.2 K rpm NL-SAS/SATA 65 60 18010 K rpm SAS/FC 135 130 27015 K rpm SAS/FC 180 180 450

Flash 1250 1250 2000

*Recommendations may change based on future test results


I/O Characteristics for Various RAID Types

I/O characteristics RAID 1/0 RAID 5 RAID 6

Random I/ORead Excellent Excellent Excellent

Write Excellent Moderate Poor

Sequential I/ORead Excellent Good Good

Write Excellent Moderate Moderate

RAID write overhead 2 4 6

Disk capacity utilization1 1/2 4/5 (in 4+1 R5) 4/6 (in 4+2 R6)

Minimal drives required2 4 3 6

1 Depends on the size of RAID group2 Depends on the array


Exchange Server Design Methodology

Phase 1:Gather requirements

•Total number of users•Number of users per server

•User profile and mailbox size

•User concurrency•High availability requirements (DAG configuration)

•Backup and restore SLAs

•Third party software in use (archiving, Blackberry, etc.)

Phase 2:Design the building block and storage architecture

•Design the building-block using Microsoft and EMC best practices

•Design the storage architecture using EMC best practices

•Use EMC Proven Solutions whitepapers

•Use Exchange Solution Review Program (ESRP) documentation

Phase 3:Validate the design

•Use Microsoft Exchange validation tools:•Jetstress - for storage validation

•LoadGen - for user workload validation and end-to-end solution validation


Exchange Storage DesignExchange building-block design methodology

• What is a building-block?• A building-block represents the required amount of resources needed to

support a specific number of Exchange users on a single server or VM• Building blocks are based on requirements and include:

• Compute requirements (CPU, memory, and network)• Disk requirements (database, log, and OS)

• Why use the building-block approach?• Can be easily reproduced to support all users with similar user profile

characteristics• Makes Exchange environment additions much easier and straightforward,

helpful for future environment growth• Has been very successful for many real-world customer implementations

See Appendix for building-block design process


Exchange Storage Design

• Microsoft Exchange Server Role Requirements Calculator• Exchange 2010: http://gallery.technet.microsoft.com/Exchange-2010-Mailbox-Server-Role-/

• Exchange 2013: http://gallery.technet.microsoft.com/Exchange-2013-Server-Role-f8a61780

• EMC Exchange 2010-2013 Designer: https://community.emc.com/docs/DOC-13037?et=watches.email.document

• VSPEX Sizing Tool: http://express.salire.com/go/emc

• Specific array sizing tools, i.e. VNX Disksizer

• Perform manual calculations (for advanced administrators)

Exchange Sizing Tools options

http://gallery.technet.microsoft.com/Exchange-2010-Mailbox-Server-Role-/

http://gallery.technet.microsoft.com/Exchange-2010-Mailbox-Server-Role-/

http://gallery.technet.microsoft.com/Exchange-2013-Server-Role-f8a61780



https://community.emc.com/docs/DOC-13037?et=watches.email.document

http://express.salire.com/go/emc

http://express.salire.com/go/emc



• Exchange JetStress for Storage Validation• Exchange 2010 - http://www.microsoft.com/en-us/download/details.aspx?id=4167

• Exchange 2013 - http://www.microsoft.com/en-us/download/details.aspx?id=36849

• Exchange Load Generator (Loadgen) for end-to-end environment validation (must be used in isolated lab only)• Exchange 2010 - http://www.microsoft.com/en-us/download/details.aspx?id=20322

• Exchange 2013 - http://www.microsoft.com/en-us/download/details.aspx?id=40726

Tools for Performance and Scalability Evaluation











Storage Design validation• Exchange Solution Reviewed Program (ESRP)

Results• Microsoft program for validation of Storage vendor designs with

Exchange

• Vendor runs multiple JetStress tests based on requirements for performance, stress, backup to disk, and log file replay

• Reviewed and approved by Microsoft

http://technet.microsoft.com/en-us/exchange/ff182054.aspx





• Exchange Jetstress − Uses Exchange executables to simulate I/O load (use same version)− Initialized and executed during pre-production before Exchange Server is installed− Throughput and mailbox profile tests – Pass gives confidence that storage design will

perform as designed

• Exchange Load Generator (Loadgen) (optional)− Validation must be performed in isolated lab− Produces a simulated client workload against a test Exchange deployment− Estimate number of users per server and validate Exchange deployment− LoadGen testing can take many weeks to configure and populate DBs

Tools for Performance and Scalability Evaluation



• Do not solely rely on automated tools when sizing your Exchange environment

• Place time and effort into your calculations, and provide supporting factual evidence on your designs rather than providing fictional calculations

• Size Exchange based on I/O, mailbox capacity and bandwidth requirements

• Factor in other overhead variables such as archiving, snapshots protection, virus protection, mobile devices, and risk factor

• Confirm Exchange storage requirements with specific array sizing tools

Storage sizing guidance


Storage Design Guidance

• Isolate the Exchange server workload to its own set of spindles from other workloads to guarantee performance

• When sizing, always calculate I/O requirements and capacity requirements

• Separate the database and logs onto different volumes

• Deploy DAG copies on separate physical spindles

• Databases up to 2 TB in size are acceptable when DAG is being used:

− The exact size should be based or customer requirements− Ensure that your solution can support LUNs larger than 2 TB

Best Practices


Storage Design Guidance

• Consider backup and restore times when calculating the database size

• Spread the load as evenly as possible across array resources, V-Max Engines, VNX SPs, back-end buses, etc.

• Always format Windows NTFS volumes for databases and logs with an allocation unit size of 64K

• Use an Exchange building block design approach whenever possible

Best Practices - continued


Storage Design Guidance - VNX

• Either method works well and provides the same performance (thick pools versus RGs)

• RAID groups are limited to 16 disks per RG, pools can support many more disks

• Pools are more efficient and easier to manage

• Use pools if planning to use advanced features such as:− FAST VP, VNX Snapshots

• Storage pools can support a single building block or multiple building blocks based on customer requirements

• Design and expand pools using the correct multiplier for best performance (R1/0 4+4, R5 4+1, R6 6+2)

Pools and RAID Groups


Storage Design Guidance - VNX

• Both Thick and Thin LUNs can be used for Exchange storage (database and logs)

• Thick LUNs are recommended for heavy workloads with high IOPS user profiles

• Thin LUNs are recommended for light to medium workloads with low IOPS user profiles– Benefits: significantly reduces initial storage requirements– Use VNX Pool Optimizer before formatting volumes– Can enable FAST Cache or FAST VP for fast metadata promotions

Thick or Thin LUNs?


FAST VP and FAST Cache

FAST VP FAST Cache

A VMAX and VNX feature that automates the identification of data for allocating or reallocating across various performance and capacity tiers within the storage array

A VNX performance optimization feature that provides performance boost to frequently accessed data by leveraging the use of flash drives to extend cache capacities

FC

Exchange SQL Server SharePoint

Virtual ServerVirtual ServerVirtual Server

SAS

NLSAS

Flash

FAST CacheFAST VP

Exchange SQL Server SharePoint

Virtual ServerVirtual ServerVirtual Server


FAST VP vs. FAST Cache on VNX StorageFAST VP FAST CacheLeverages pools to provide sub-LUN tiering,enabling the utilization of multiple tiers of storageSimultaneously.

Enables flash drives to extend theexisting caching capacity of the storage system.

Uses 1 GB chunks (256 MB in Next-Gen VNX). Uses 64 KB chunks.

Local feature – per storage pool. Assured performance per pool.

Global feature – per storage array. Shared resource, performance for one pool is not guaranteed under FAST Cache performance contention.

Moves data between different storage tiers basedon a weighted average of access statistics collected over a period of time.

Copies data from hard disks to flash disks when accessed frequently.

Uses a relocation window to periodically makestorage tiering adjustments. Default setting is an 8-hour relocation window each day.

Adapts continuously to changes in workload.

While it can improve performance, it is primarilydesigned to improve usability and reduce TCO.

Designed primarily to improve performance.


VNX Considerations for FAST VP and FAST Cache When the number of flash drives is limited, use flash drives to

create FAST Cache first– FAST Cache can benefit multiple pools in the storage system.– FAST Cache uses 64 KB chunks, smaller than 1 GB or 256 MB chunks in FAST VP,

which results in higher performance benefits and faster reaction time for changing usage patterns.

Use flash drives to create the FAST VP performance tier for a specific pool

– This ensures the performance of certain mission-critical data– The FAST VP tier is dedicated to a storage pool and cannot be shared with other

storage pools in the same storage array.


Configuration recommendations (VNX and VMAX)Exchange Design with FAST VP

Separate databases from logs, due to different workloads– Data – random workload with skew; high FAST VP benefit– Logs – sequential data without skew; no FAST VP benefit

Use dedicated pools – Provides a better SLA guarantee– Provides fault domains– Recommended for most deterministic behavior

Use Thick Pool LUNs for highest performance (on VNX)– Thin Pool LUNs are acceptable with optimization

Use Thin LUNs on VMAX


Tools for FAST VP

Tier Advisor for sizing– Historical performance data is needed from storage arrays.

Workload Performance Assessment Tool – It shows FAST VP heat map. For more information, refer to https://

emc.mitrend.com.

VSPEX Sizing Tool (for VNX) – For more information, refer to

http://www.emc.com/microsites/vspex-ebook/vspex-solutions.htm.

EMC Professional Services and qualified partners can assist in properly sizing tiers and pools to maximize investment.

https://emc.mitrend.com/

https://emc.mitrend.com/

http://www.emc.com/microsites/vspex-ebook/vspex-solutions.htm


VNX onlyExchange Design with FAST Cache

FAST Cache usage– Pools with Thin LUNs for metadata tracking– Pools with Thin and Thick LUNs when VNX Snapshots are used– Pools with Thick LUNs

▪ Not required but not restricted either▪ Required with VNX Snapshots

FAST Cache Sizing guidance– Rule of thumb: for every 1 TB of Exchange dataset, provision 1 GB

of FAST Cache▪ Monitor and adjust the FAST Cache size, your mileage may vary▪ Enable FAST Cache on pools with database LUNs only


Exchange Storage design - VMAX

• Ensure that the initial disk configurations can support the I/O requirements

• Can configure a thin pool to support a single Exchange building block or multiple building blocks, depending on customer requirements

• Use Unisphere for VMAX to monitor the thin pool utilization and prevent the thin pools from running out of space

• Install the Microsoft hotfix KB2870270 on the Windows Server 2012 hosts in your environment.

Design Best Practices

http://support.microsoft.com/kb/2870270


Exchange Storage design - VMAX

• Use Symmetrix Virtual Provisioning

• Can share database and log volumes across the same disks, but separate them into different LUNs on the same hosts

• For optimal Exchange performance, use striped meta volumes



VMAX FAST VP with Exchange

• When designing FAST VP for Exchange 2010/2013 on VMAX follow these guidelines:– Separate databases and logs onto their own volumes

• Can share database and log volumes across the same disks

– Exclude transaction log volumes from the FAST VP policy or pin all the log volumes into the tier on which they are created

– Select Allocate by FAST Policy to allow FAST VP to use all tiers for new allocations based on the performance and capacity restrictions• New feature introduced in the Enginuity™ 5876 code

– When using FAST VP with Exchange DAG, do not place DAG copies of the same database in the same pool on the same disks



XtremCache with Exchange

• Consider XtremCache for Exchange if:– You have an I/O bound Exchange solution– You are not sure about anticipated workload– You need to guarantee high performance and low latency for specific users (VIP servers,

databases, and so on)

• XtremCache is proven to improve Exchange 2010 performance by:– Reducing read latencies– Increasing I/O throughput– Eliminating almost all high latency spikes– Providing more improvements as workload increases– Reducing RPC latencies– Reducing writes to the back-end storage with XtremCache deduplication



• XtremCache PCI Flash card can be installed on– A physical Exchange Mailbox server– Hypervisor server hosting Exchange Mailbox virtual machines (VMware or

Hyper-V)

• Enable XtremCache acceleration only on database volumes only– Do not enable on log volumes – sequential workload=no performance

benefits

• XtremCache sizing guidance:For a 1,000 GB working dataset, configure 10 GB of XtremCache device

Configuration recommendations



• When implementing XtremCache with VMware vSphere, consider the following:– Size of the PCI cache card to deploy– Number of Exchange virtual machines deployed on each vSphere

host that will be using XtremCache– Exchange workload characteristics (read:write ratio, user profile

type)– Exchange dataset size

• The most benefits will be achieved when all reads from a working dataset are cached




• When adding a XtremCache device to an Exchange virtual machine:– Set cache page to 64KB and Max IO size to 64 (BDM I/O will not be

cached)– Can use VSI Plug-in or XtemCache CLI command to set the cache

page size to 64 KB and max I/O size to 64 KB when adding the cache device to a virtual machine:vfcmt add -cache <cache_device> -set_page_size 64 -set_max_io_size 64)




• Evaluate your workload before considering enabling deduplication for accelerated Exchange LUNs

• Consider CPU overhead when enabling deduplication

• Set the deduplication ratio based on workload characteristics:– If the observed deduplication ratio is less than 10%, EMC recommends that

you turn it off (or set it to 0%), which enables you to benefit from extended cache device life.

– If the observed ratio is over 35%, EMC recommends that you raise the deduplication gain to match the observed deduplication.

– If the observed ratio is between 10% and 35%, EMC recommends that you leave the deduplication gain as it is.

Configuration recommendations (deduplication)





Exchange DR options





EMC’s Common Solutions for Exchange DRExchange DR Options

Exchange DR Options Replication Method Best for

Stretched (cross-site) DAG

Native Exchange host-based replication

Small environments, local replication

Third Party Replication with Manual Exchange Recovery (database portability)

EMC RecoverPoint or SRDF

Mixed environments with large workloads

and strict SLAs

Third Party Replication with Automated Exchange Recovery

EMC RecoverPoint or SRDF with VMware SRM

VMware shops with large workloads and

strict SLAs


Replication MatrixFeatures SRDF/S SRDF/A MV/S MV/A RecoverPoint

SyncRecoverPoint

Async SYNC-API DAG

Replication Technology Array Array Array Array SAN SAN Host Host

Heterogeneous storage support No No No No Yes Yes Yes Yes

Heterogeneous applications support Yes Yes Yes Yes Yes Yes Yes No

Active Bi-Directional Yes Yes Yes Yes Yes Yes Yes Yes

Latency Impact to applications Medium Minimum Medium Minimum Minimum Minimum High High

Data replication Type Synch Async Sync Async Sync Async Sync Asynch

Automated Restart Yes Yes Yes Yes Yes Yes Yes Yes

Coexistence with VSS replication Yes Yes Yes Yes Yes Yes No Yes

Zero data loss RPO = 0 Yes No Yes No Yes No Yes No

Time to recover RTO Minutes Minutes Minutes Minutes Minutes Minutes Minutes Seconds

Minutes

Replication interval Continuous Continuous IO Pools Continuous Continuous IO

Pools Continuous IO

PoolsContinuous IO

Pools Continuous Log Shipping

Replication distance 200km unlimited 200km unlimited 200km unlimited 200km Data centerUnlimited

Ability to resynchronize data incrementally Yes Yes Yes Yes Yes Yes Yes No

Link Cost Requirement Medium Low Medium Low Low Low Medium High





Exchange DR options





Exchange 2013 ESRP on VNX5700


Exchange with XtremCache Solution


Cross-site DAG vs. RecoverPoint


Automated DR with RecoverPoint and SRM


VSPEX Solutions for Exchange


Exchange Resources• Exchange Storage Best Practices and Design Guidelines for EMC Storage

whitepaper:http://www.emc.com/collateral/hardware/white-papers/h8888-exch-2010-storage-best-pract-design-guid-emc-storage.pdf

• EMC Community Network− https://community.emc.com/community/connect/everything_microsoft

• EMC and Partner Exchange 2010 Tested Virtualized Solutions– http://technet.microsoft.com/en-us/library/gg598215.aspx– http://www.emc.com/collateral/hardware/white-papers/h7337-exchange-unified-cisco-hyper-v-wp.pdf – http://www.emc.com/collateral/software/white-papers/h7410-zero-data-loss-exchange-wp.pdf

• Exchange Solution Reviewed Program Submissions (ESRP)– http://technet.microsoft.com/en-us/exchange/ff182054.aspx

Exchange Mailbox Server Storage Design (Microsoft TechNet)– http://technet.microsoft.com/en-us/library/dd346703.aspx

http://www.emc.com/collateral/hardware/white-papers/h8888-exch-2010-storage-best-pract-design-guid-emc-storage.pdf



https://community.emc.com/community/connect/everything_microsoft

https://community.emc.com/community/connect/everything_microsoft

http://technet.microsoft.com/en-us/library/gg598215.aspx

http://www.emc.com/collateral/hardware/white-papers/h7337-exchange-unified-cisco-hyper-v-wp.pdf

http://www.emc.com/collateral/hardware/white-papers/h7337-exchange-unified-cisco-hyper-v-wp.pdf

http://www.emc.com/collateral/software/white-papers/h7410-zero-data-loss-exchange-wp.pdf



http://technet.microsoft.com/en-us/library/dd346703.aspx

http://technet.microsoft.com/en-us/library/dd346703.aspx


Additional References

• Exchange virtualization supportability guidance - http://technet.microsoft.com/en-us/library/jj126252.aspx

• Understanding Exchange Performance - http://technet.microsoft.com/en-us/library/dd351192

• Server Virtualization Validation Program - http://www.windowsservercatalog.com/svvp/

• Exchange 2010 EMC-tested OEM solutions (on Hyper-V)– 20,000 users on EMC storage with virtual provisioning http

://technet.microsoft.com/en-us/library/gg598215(v=exchg.141).aspx– 32,400 users on EMC storage with EMC REE http

://technet.microsoft.com/en-us/library/hh145600(v=exchg.141).aspx

http://technet.microsoft.com/en-us/library/jj126252.aspx

http://technet.microsoft.com/en-us/library/dd351192

http://www.windowsservercatalog.com/svvp/

http://technet.microsoft.com/en-us/library/gg598215(v=exchg.141).aspx



http://technet.microsoft.com/en-us/library/hh145600(v=exchg.141).aspx







Exchange DR options





ESI For VNX Pool Optimization Tool

• A utility for optimization of VNX pool-based LUNs (thick or thin) for maximum performance

What is it?



Included in the ESI 3.0.1 Package

Is a stand-alone utility– ESI PowerShell Toolkit is required for LUNs provisioning

Current version works only with VNX OE for Block Release 33 or later (VNX5200, VNX5400, VNX5600, VNX5800, VNX7600 and VNX8000)

For VNX OE for Block Release 32 (VNX5100,VNX5300, VNX5500, VNX5700 and VNX7500), must use older SOAP utility available on EMC Online support (thick LUNs only)

Availability and supportability


What Does This Tool Do?

• The tool optimizes pool-based LUNs by pre-allocating slices in the pool evenly across all disks, private raid groups and LUNs

• Provides the best option for any application requiring deterministic high performance across all LUNs in the pool equally


When To Use The Optimization Tool

• When maximum application performance is required

• Applications to benefit from using the tool:• Microsoft Exchange

• Microsoft SQL Server

• Oracle



• Why do I need to use it?– To achieve the best performance for pool-based LUNs

(primarily thin)– To eliminate performance issues and successfully pass

Exchange Jetstress during Exchange pre-deployment storage validation

– To mitigate “Jetstress effect”


Jetstress Initialization Process

How Jetstress initialization phase works:

• Jetstress creates first database• It then creates other databases

by copying the first database to other databases concurrently


What is the “JetStress effect”

• Jetstress initialization process (data population) results in imbalances of underlying virtual disks in a pool

• How is the issue surfaced?– With Jetstress testing first database on

the Exchange server will:• Experience higher latencies then the others

when the LUN is Thick• Experience lower latencies then the others

when the LUN is Thin


Looking under the covers… Slice Maps

• Without Optimization • With Optimization


Prerequisites

• VNX OE for Block Release 33 SP1 or later

• Before using the tool, LUNs must be created using ESI PowerShell with specific parameters in order for the tool to optimize the LUNs*

• All LUNs in the pool must be the same size

• All LUNs in the pool must be optimized at the same time*It is also possible to use NaviSecCLI to create LUNs using undocumented black content switches


Options for Thick pool LUNs provisioning

Two options for thick pool LUNs provisioning 1. For good and optimal performance – No SOAP is

necessary. Use default pool LUNs provisioning via Unisphere, NaviSecCLI, EMC ESI, or EMC VSI

2. For best performance (max IOPS) – Use NaviSecCLI to disable pre-provisioning and then run SOAP• Turn off pre-provisioning via the CLI• Run SOAP• Re-enable pre-provisioning

VNX OE for Block Release 32 (Inyo) and later


SOAP Utility for VNX R32– Where and how?• Old SOAP Utility is available on EMC Online Support site

– Enter “soap” in the search and select “Support Tools”

• Must be used with CX4/VNX Inyo (OE 5.32) only– Support Thick LUN optimization only

• Zip file contains the tool, step-by-step documentation, and demo video





Exchange DR options





Exchange Mailbox Server Storage Design Methodology

Phase 1:Gather requirements

•Total number of users•Number of users per server

•User profile and mailbox size

•User concurrency•High availability requirements (DAG configuration)

•Backup and restore SLAs•Third party software in use (archiving, blackberry, etc.)

Phase 2:Design the building block and storage architecture

•Design the building block using Microsoft and EMC best practices

•Design storage architecture using EMC best practices

•Leverage EMC Proven Solutions whitepapers

•Leverage Exchange Solution Review Program (ESRP) documentation

Phase 3:Validate the design

•Use Microsoft Exchange validation tools

• Jetstress - for storage validation

•LoadGen - for user workloads validation and end-to-end solution validation


Requirements ExampleItem Value Exchange version. Total number of active users (mailboxes) in Exchange environment

Exchange 2013, 20,000

Site resiliency requirements Single siteStorage Infrastructure SANType of deployment (Physical or Virtual) Virtual (VMware vSphere)HA requirements One DAG with two database copiesMailbox size limit 2 GB max quota

User profile200 messages per user per day (0.134 IOPS)

Target average message size 75 KBOutlook mode Cached mode, 100 percent MAPINumber of mailbox servers 8Number of mailboxes per server 5,000 (2,500 active/2,500 passive)Number of databases per server 10Number of users per database 500Deleted items retention (DIR) period 14 daysLog protection buffer (to protect against log truncation failure) 3 daysBDM configuration Enabled 24 x7Database read/write ratio 3:2 (60/40 percent) in a DAG configurationUser concurrency requirements 100 percentThird-party software that affects space or I/O (for example, Blackberry, snapshots)

Storage snapshots for data protection

Disk Type 3 TB NL-SAS (7,200 rpm)Storage platform VNX


Building Block design

• In our example we are defining a building block as:– A mailbox server that will support 5,000 users

• 2,500 users will be active during normal runtime and the other 2,500 users will be passive until a switchover from another mailbox server occurs.

• Each building block will support two database copies.

Define and design Building Block


Building block sizing and scaling process

• Perform calculations for IOPS requirements

• Perform calculations for capacity requirements based on different RAID types

• Determine the best option

• Scale building block– Multiple building blocks may be combined together to

create the final configuration and storage layout (pools or RGs)


Building block sizing and scaling process

Front-end IOPS ≠ Back-end IOPS – Front-end IOPS = Total Exchange Mailbox server IOPS– Back-end IOPS = Storage array IOPS (including RAID penalty)

Understand disk IOPS by RAID type– Block front-end Exchange application workload is translated into a different back-end

disk workload based on the RAID type in use.– For reads there is no impact of RAID type:

▪ 1 application read I/O = 1 back-end read I/O

– For random writes like Exchange:▪ RAID 1/0: 1 application write I/O = 2 back-end write I/O▪ RAID 5: 1 application write I/O = 4 back-end disk I/O (2 read I/O + 2 write I/O)▪ RAID 6: 1 application write I/O = 6 back-end write I/O (3 read I/O + 3 write I/O)


Formula & CalculationsDatabase IOPS requirements

Total transactional IOPS = IOPS per mailbox * mailboxes per server + (Microsoft recommended overhead %)

Total transactional IOPS = 5,000 users * 0.134 IOPS per user + 20% Microsoft recommended overhead = 670 + 134 = 804 IOPS

Total front-end IOPS = (Total transactional IOPS) + (EMC required overhead %)

Total Front-end IOPS = 804 + 20% EMC required overhead = 965 IOPS (rounded-up from 964.8)


Formula

Database Disks requirements for Performance (IOPS)

Disks required for Exchange database IOPS = (Total backend database Read IOPS) + (Total backend database Write IOPS)/ Exchange random IOPS per disk

Where:

Total back-end database read IOPS = (Total Front-end IOPS) * (% of Reads IOPS)

Total back-end database write IOPS = RAID Write Penalty *(Total Front-end IOPS * (% of Write IOPS)


Calculations

Database Disks requirements for Performance (IOPS)

RAID Option RAID Penalty Disks required

RAID 1/0 (4+4) 2(965 x 0.60) + 2(965 x 0.40) = 579 + 772 = 1351 / 65 = 21 (round-up to 24 disks)

RAID 5 (4+1) 4(965 x 0.60) + 4(965 x 0.40) = 579 + 1544 = 2123 / 65 = 33 (round-up to 35 disks)

RAID 6 (6+2) 6(965 x 0.60) + 6(965 x 0.40) = 579 + 2316 = 2895 / 65 = 45 disks (round-up to 48 disks)


Formula & CalculationsTransactional logs IOPS requirements

Disks required for Exchange log IOPS = (Total backend database Write IOPS * 50%) + (Total backend database Write IOPS * 10%)/ Exchange sequential IOPS per disk

Disks required for Exchange log IOPS = (772 back-end write IOPS * 50%) + (772 *10%))/ 180 sequential Exchange IOPS per disk = (386 + 77.2)/180 = 2.57(round-up to 4 disks)


FormulaStorage capacity calculations

Calculate User Mailbox on Disk

Calculate Database Size on Disk

Calculate Database LUN Size

Mailbox size on disk = Maximum mailbox size + White space + Dumpster

Database size on disk = number of mailboxes per database * mailbox size on disk

Database LUN size = Number of mailboxes * Mailbox size on disk * (1 + Index space + additional Index space for maintenance) / (1 + LUN free space)


FormulaMailbox size on disk

Mailbox size on disk = Maximum mailbox size + White space + Dumpster

Where:Estimated Database Whitespace per Mailbox = per-user daily message profile * average message size

Where:Dumpster = (per-user daily message profile * average message size * deleted item retention window) + (mailbox quota size * 0.012) + (mailbox quota size * 0.03)


CalculationsMailbox size on disk

White space = 200 messages /day * 75KB = 14.65MB

Dumpster = (200 messages/day * 75KB * 14 days) + (2GB * 0.012) + (2GB x 0.03) = 205.1 + 24.58 + 61.44= 291.12MB

Mailbox size on disk = 2GB mailbox quota + 14.65MB database whitespace + 291.12MB Dumpster = 2,354 MB (2.3GB)


CalculationsDatabase Size On Disk & LUN size

Database size on disk = 500 users per database * 2,354 MB mailbox on disk = 1,177 GB (1.15 TB)

Database LUN size = 1,177 GB * (1 + 0.2 + 0.2) / (1 - 0.2) = 2,060 (2 TB)

In our example:– 20% added for the Index– 20% added for the Index maintenance task– 20% added for LUN-free space protection


Formula & CalculationsLogs space calculations

Log LUN size = (Log size)*(Number of mailboxes per database)*(Backup/truncation failure tolerance days)+ (Space to support mailbox moves)/(1 + LUN free space)

Log Capacity to Support 3 Days of Truncation Failure = (500 mailboxes/database x 40 logs/day x 1MB log size) x 3 days = 58.59GB

Log Capacity to Support 1% mailbox moves per week = 500 mailboxes/database x 0.01 x 2.3GB mailbox size = 11.5GB

Log LUN size = 58.59GB + 11.5 GB /(1 - 0.2) = 87.61 GB (round-up to 88 GB)


Total Capacity per Building Block

LUN Capacity Type LUN Capacity Required per server

Database LUN capacity 2,060 GB per LUN * 10 LUNs per server = 20,600 GB

Log LUN capacity 88 GB per LUN * 10 LUNs per server = 880 GBTotal LUN capacity per server

20,600 + 880 = 21,480 GB

Total LUN size capacity required per server = (Database LUN size per server) + (Log LUN size per server) * (Number of databases per server)


Total number of disks required

Database disks

Logs disks

Disks required for Exchange database capacity = Total database LUN size / Physical Disk Capacity * RAID Multiplication Factor

Disks required for Exchange log capacity = Total log LUN size) / Physical Disk Capacity * RAID Multiplication Factor


Disk requirements based on capacity

RAID Option Database Disks required

RAID 1/0 (4+4) 20,600/2794.5 * 2 = 7.37 * 2 = 14.74 (round-up to 16 disks)

RAID 5 (4+1) 20,600/2794.5 * 1.25 = 7.37 * 1.25 = 9.2 (round-up to 10 disks)

RAID 6 (6+2)20,600/2794.5 * 1.33 = 7.37 * 1.33 = 9.8 (round-up to 16 disks)

RAID Option Logs Disks required

RAID 1/0 (1+1) 880 / 2,794.5 * 2 = 0.63 (round up to 2 disks)


Building Block SummaryFinal storage calculation results

Volume Type RAID Option

Disks Required for Performance (IOPS)

Disks Required for Capacity

Best Option

Exchange Databases

RAID 1/0 (4+4) 24 disks 16 disks 24 disks

RAID 5 (4+1) 35 disks 10 disks

RAID 6 (6+2) 48 disks 16 disks

Exchange Logs RAID 1/0 (1+1) 4 disks 2 disks 4 disks

Total disks per building block 28 disks


Building Block ScalabilityFinal storage calculation results

Total number of disks required for entire 20,000 users solution in a DAG with two copies = 28 disk per building block * 8 building blocks = 224 disks total


Bandwidth calculationsBuilding block sizing and scaling process

• Array throughput MB/s validation for Exchange involves:• Determining how many databases the customer will require• Confirming the database LUNs are evenly distributed among the backend busses and

storage processors.• Determine if each bus can accommodate the peak Exchange database throughput

• Use this calculation to calculate the throughput required

(DB throughput * number of DBs per bus) = Exchange DB throughput

• Compare that number with array bus throughputDB throughput = Total transactional (user) IOPS per DB * 32K + (BDM throughput per DB in MB/s)Number of DBs per bus = the total number of active and passive databases per bus


The processStorage Bandwidth Requirements

The bandwidth validation process involves the following steps:1. Determine how many databases in the Exchange environment2. Determine the bandwidth requirements per database3. Determine the required bandwidth requirements per array bus4. Determine whether each bus can accommodate the peak Exchange

database bandwidth• Use DiskSizer for VNX or contact your local storage specialist to get the array and bus throughput numbers

• DiskSizer is available through your local USPEED contact

5. Evenly distribute database LUNs among the back-end buses and storage processors• Uniformed distribution is key for best performance

• FE/BE/RAID Group/POOLs & DAEs• DBs uniformly distributed onto the pools• Use even numbers on the SAS loops (0 & 2) for max performance


CalculationsStorage Bandwidth Requirements

Bandwidth per database (MB/s) = Total transactional IOPS per database * 32 KB + Estimated BDM Throughput per database (MB/s)Where:• 32 KB is an Exchange page size• Estimated BDM throughput per database is 7.5 MB/s for Exchange 2010 and 2.25 MB/s for Exchange 2013

Required throughput MB/s per bus = (throughput MB/s per database) * (total number of active and passive databases per bus)


CalculationsStorage Bandwidth Requirements

Total transactional IOPS per database = (500 * 0.134 * 32 KB = 2.1 MB/s Throughput per database = 2.1 MB/s + 2.25 MB/s = 4.35 MB/s Required throughput per bus = 4.35 MB/s * 200 databases per bus = 870 MB/s

Example assumptions:• 500 users at 0.134 IOPS per database• 200 databases per bus

If the array supports a maximum throughput of 3,200 MB/s per bus, 200 databases can be supported from a throughput perspective.


• Configured dedicated storage pools for each mailbox server with 24 x 3 TB NL-SAS drives.

• Each storage pool holds two copies from different mailbox servers.

• Separated Exchange log files into different storage pools

• For better storage utilization created one storage pool with 16 x 3 TB NL-SAS drives for logs per four mailbox server building blocks.

Final Design Example


AppendixExchange Archiving


Exchange Archiving


Archiving – Native vs. EMC SourceOneArchiving Capabilities Exchange 2013 EMC SourceOne

Name In-Place Archiving Comprehensive email management capabilities

Consistent & automated policy-based with proper retention and disposition û ü

Full-text indexing of attachments Limited to few file types Support many file types

Single-instance, de-duplication and compression capabilities û ü

Multiple content types (email, SharePoint, files) û ü

Offline Access Limitation In-place archiving is not accessible in offline mode ü

Access from non-Microsoft Device Not available ü

Doesn’t Required Exchange enterprise CAL license û ü

Doesn’t required In-place Exchange Migration û ü

Search results between online and offline Inconsistent as archived emails are not searchable offline consistent

Archiving policy Basic Advanced - based on many other parameters like for example based word in subject field, specific outlook folder, with attachment or no, attachment format, read/unread message and etc….


Archiving – Native vs. EMC SourceOneeDiscovery Capabilities Exchange 2013 EMC SourceOne

Comprehensive eDiscovery Capabilities Limited Comprehensive eDiscovery

Legal hold Limited Up to lower level (single email level)

Assign Custodians û üLegal hold based on time frame of content Limited ü

Supported File types for Indexing 54 file types – limits the keyword search result 400 file types – accurate keyword search result

Proper eDiscovery Workflow (assessment, culling, review and tagging then holding)

û ü

Message Tagging û ü

Message distribution among reviewers û ü

Run Complex Search Query û ü

BCC Field Search û ü

Microsoft Exchange Best Practices and Design Guidelines on EMC Storage

Documents

Transcript of Microsoft Exchange Best Practices and Design Guidelines on EMC Storage