Exchange Server 2013 Database and Store Changes
-
Upload
microsoft-technet-belgium-and-luxembourg -
Category
Documents
-
view
927 -
download
3
description
Transcript of Exchange Server 2013 Database and Store Changes
Exchange Server 2013Database and Store ChangesScott SchnollPrincipal Technical WriterMicrosoft Corporation
Agenda
Mailbox Server Platform EvolutionExchange 2013 GoalsDatabase and Store Changes
Mailbox Server Platform Evolution
Exchange 2010 Exchange 2013 Challenges Solutions
Processor
8 core(2 socket)
20+ cores(2 socket)
• Store scales poorly past 12 cores
• Eggs/basket increase
• Multi-role servers• Multi-process Store
Memory32GB(8x4GB DIMM)
96GB(12x8GB DIMM)
• n/a • n/a
Disk
3.5” 7.2K 2TB (35/server)
3.5” 7.2K 6TB(12/server)
• Disks getting larger but not faster
• DB Maintenance times• Seed reliability/duration• Eggs/basket increase
• Reduce IOPS by +50%• Multiple Databases/disk• Tune DB maintenance
Network
4 x 1GBit Nics 2 x 10Gbit Nics • How do you leverage the bandwidth for seeding
• Faster seeding with multiple databases/disk
Exchange Server 2013 Goals
Decrease hardware costsIncrease reliability and availabilityProvide data protection enhancementsEnable faster root cause analysis through better diagnosticsDeliver core platform investments for future innovations
Decrease Hardware Costs
Decrease Hardware Costs
Reduce IOPS by 50% compared with Exchange 2010Disk sizes increasing (8TB) with no corresponding increase in IOPSLarger mailboxes (100GB)Mailbox schema and ESE pre-read optimizations
Support multiple databases per volumeMaximize disk space utilization without increased reseed timesDistribute active users across available database volumes
Drive higher adoption of JBOD deploymentTake advantage of low-cost locally attached storage
IOPS Reductions
Improvements to logical contiguity of store schemaProperty blobs are used to store actual message propertiesSeveral messages per page means fewer large IOs to retrieve message propertiesUse of long-value storage is reduced, though when accessed, large sequential IOs are used
Reduction in passive copy IO100MB checkpoint depth reduces write IOTransaction log code refactored for faster failover
Tables Optimized for Sequential IOGlobal Tables Catalog – registry of tables existing in database Globals – database version, etc Mailbox – MailboxNumber, Owner Info, Locale, LastLogonTime, etc DeliveredTo – duplicate delivery information Events – reliable events for assistants
Tables partitioned by MailboxNumber Folder - FolderId, Item Count, Size, PropertyBlob Message – DocumentId, MessageId, FolderId, PropertyBlob,
OffPagePropertyBlob, MessageClass ordered by DateReceived Attachment – AttachmentId, Name, Size, CreationTime, etc PhysicalIndexes (partitioned by LogicalIndex)
Message Table Property Storage
Blobs used to store collection of MAPI properties Referred to as On-page and Off-page property blobs ESE compression optimizes physical storage of blob data Compression more efficient when input contains more properties
PropertyBlob Contains properties previously stored in Header table in message table
column Property promotion OffPagePropertyBlob PropertyBlob possible Blob size limited to eliminate LV tree access for core message properties
OffPagePropertyBlob ESE LV Hints push storage of this blob into separate LV tree Reading LV tree involves large sequential I/O
Higher Reliability and Availability
Higher Reliability and Availability
Improved isolation from hardware/software failuresStore process per database, faster failover and disk failure handlingBuilt-in monitoring and availability managementBest copy and server selection includes health of entire protocol stackService recovery through failover and/or restartNon-stop operationsNo scheduled mailbox database maintenance Autoreseed automatically restores redundancy on disk failureMaintain data protection without manual interventionDynamically uses spare disks to restore database copy health
Exchange Information Store
Previously a single monolithic process
Unmanaged code
Very nested code that made it difficult to debug
Managed Store
Store service/process (Microsoft.Exchange.Store.Service.exe)Microsoft Information Store serviceManages worker process lifetime based on mount/dismountLogs failure item when store worker process problems detectedTerminates store worker process in response to “dirty” dismount during failover
Store worker process (Microsoft.Exchange.Store.Worker.exe)One process per database, RPC endpoint instance is database GUIDResponsible for block-mode replication for passive databasesFast transition to active when mountedTransition from passive to active increases ESE cache size 5X
Microsoft Exchange Replication serviceReplication service process (MSExchangeRepl.exe)Detecting unexpected database failuresIssues mount/dismount operations to StoreProvides administrative interface for management tasksInitiates failovers on failures reported by ESE, Store and Responders
ESE Cache Management
Allocates 25% of memory for store worker process ESE cacheThis is referred to as the max cache targetAmount allocated to each store worker process based on number of hosted database copies and value of MaximumActiveDatabasesStatic amount of cache allocated to passive and active copies
Store worker process will only use max cache target when copy is activePassive database copies allocate 20% of max cache target
Max cache target computed at service process startupRestart Store service process when adding/removing copies or changing value of MaximumActiveDatabases
Store Worker Process Example
MountedOnServer: EXHV-1125.EXHV-44159dom.extest.microsoft.com
Name WorkerProcessId Mounted ActivationPreference EdbFilePath
---- --------------- ------- -------------------- -----------
MDB01C 11328 True {[EXHV-1125, 1], [EXHV-94234, 2], [EXHV-96496, 3]} C:\Databases\MDB01\MDB01C\MDB01C.edb
MDB02C 12504 True {[EXHV-1125, 1], [EXHV-94234, 2], [EXHV-96496, 3]} C:\Databases\MDB02\MDB02C\MDB02C.edb
MDB03C 11564 True {[EXHV-1125, 1], [EXHV-94234, 2], [EXHV-96496, 3]} C:\Databases\MDB03\MDB03C\MDB03C.edb
MountedOnServer: EXHV-94234.EXHV-44159dom.extest.microsoft.com
Name WorkerProcessId Mounted ActivationPreference EdbFilePath
---- --------------- ------- -------------------- -----------
MDB01A 2920 True {[EXHV-94234, 1], [EXHV-96496, 2], [EXHV-1125, 3]} C:\Databases\MDB01\MDB01A\MDB01A.edb
MDB02A 17480 True {[EXHV-94234, 1], [EXHV-96496, 2], [EXHV-1125, 3]} C:\Databases\MDB02\MDB02A\MDB02A.edb
MDB03A 22372 True {[EXHV-94234, 1], [EXHV-96496, 2], [EXHV-1125, 3]} C:\Databases\MDB03\MDB03A\MDB03A.edb
MountedOnServer: EXHV-96496.EXHV-44159dom.extest.microsoft.com
Name WorkerProcessId Mounted ActivationPreference EdbFilePath
---- --------------- ------- -------------------- -----------
MDB01B 16692 True {[EXHV-96496, 1], [EXHV-1125, 2], [EXHV-94234, 3]} C:\Databases\MDB01\MDB01B\MDB01B.edb
MDB02B 3336 True {[EXHV-96496, 1], [EXHV-1125, 2], [EXHV-94234, 3]} C:\Databases\MDB02\MDB02B\MDB02B.edb
MDB03B 17912 True {[EXHV-96496, 1], [EXHV-1125, 2], [EXHV-94234, 3]} C:\Databases\MDB03\MDB03B\MDB03B.edb
Recurring Maintenance
Scheduled maintenance is eliminated in Exchange 2013Recurring maintenance now part of time-based assistant (TBA) infrastructureStoreMaintenance: lazy index maintenance, isintegStoreDirectoryServiceMaintenance: disconnected mailbox expiration
Workload Management monitors CPU, RPC latency, and replication health Task execution throttled/deferred when resource pressure exists
Background ESE database scanning further throttledBased on datacenter disk failure analysis, target to complete background database scan within 4 weeks (using multiple databases on 8 TB disks)
Managed Availability
Tests determine viability of various components on Mailbox serverDatabase connectivity and replicationProtocol services (Outlook, OWA, EAS, IMAP, POP)
Recommend HA actions when service-impacting condition foundDatabase failoverRestart serviceRestart computer
Escalate when auto recovery unsuccessful and service not restoredIntegration with System Center to raise awareness of service-impacting conditions that cannot be automatically resolved
Managed Availability
Name Trigger/Recovery sequence
Database Availability 12 logon failures in 16 minutes Escalate
Store service not running Restart service Bugcheck Escalate
Database Free space Free disk space drops below 10% Escalate
Store service process repeatedly crashing
3 crashes for store service in 1 hour Escalate
Store worker process repeatedly crashing
3 crashes for store work (across all workers) in 1 hour Escalate
Percent RPC requests 90% of available threads per database Database Failover Escalate
70ms RPC latency 70ms RPC Avg latency Determine impact scope Id/quarantine mailbox Escalate
150ms RPC latency 150ms RPC Avg latency Determine impact scope Id/quarantine mailbox Escalate
Mailbox quarantined More than 1 mailbox quarantined on database Escalate
Assistants service not running Restart service Escalate
Event assistants behind watermarks* Assistant watermark age exceeds threshold Escalate
Number of search tasks* Count of search tasks exceeds threshold Escalate
Mailbox Quota Management
Reduction in overhead to generate over-quota notificationAt logon time, system evaluates mailbox quota against policySends over-quota notification message once per notification interval, notifications are NOT sent to inactive mailboxes
Mailbox size calculation is more accurate measurement of mailbox database storage usedIncludes both internal and end-user items/properties
Mailbox size will likely increase when moved to Exchange 2013Search metadata stored on items increases overall mailbox sizeNo increase in database footprintShould plan to increase quota per mailbox
Data Protection Improvements
Data Protection Improvements
Autoreseed automatically restores redundancy on disk failure
Lag copies can “care for themselves”Play down when low on space, during page patching, and when required for availability (no other copies available)Lagged copy activation can be simplified with transport enhancements
Lagged Copy Example
MailboxServer: EXHV-1125
Name Status ReplayLagStatus---- ------ ---------------MDB01A\EXHV-1125 Healthy Enabled:True; PlayDownReason:None; Percentage:47; Configured:1.00:00:00; Actual:11:20:49MDB02A\EXHV-1125 Healthy Enabled:True; PlayDownReason:None; Percentage:47; Configured:1.00:00:00; Actual:11:20:14MDB03A\EXHV-1125 Healthy Enabled:True; PlayDownReason:None; Percentage:47; Configured:1.00:00:00; Actual:11:19:53
MailboxServer: EXHV-94234
Name Status ReplayLagStatus---- ------ ---------------MDB01B\EXHV-94234 Healthy Enabled:True; PlayDownReason:None; Percentage:48; Configured:1.00:00:00; Actual:11:36:23MDB02B\EXHV-94234 Healthy Enabled:True; PlayDownReason:None; Percentage:48; Configured:1.00:00:00; Actual:11:36:24MDB03B\EXHV-94234 Healthy Enabled:True; PlayDownReason:None; Percentage:48; Configured:1.00:00:00; Actual:11:36:18
Data Protection Improvements
Continued support of VSS backup APIWindows Server Backup support, supports both active and passive database copies3rd party VSS applications should be compatible with Exchange 2013 without major changes1
VSS full, copy, incremental, and differential backup/restore supported
Windows Server Backup supports backup of both active and passive databasesScheduled backup succeeds regardless of mount state
1 Backup vendors responsible for integration and supportability statements
Diagnostic Improvements
Diagnostic Improvements
Insight into runtime without dumps or external clientsPowerShell access to mailbox database internals and in-memory state
Eliminate need for end-user reproAlways on tracing to capture “outlier” operational behavior
Get-MailboxStatistics Example
Get-MailboxStatistics –Database DB1 sort ItemCount -Descending | select -first 10 | FT -a MailboxGuid,DisplayName,IsArchiveMailbox,TotalItemSize,ItemCount
MailboxGuid DisplayName IsArchiveMailbox TotalItemSize ItemCount----------- ----------- ---------------- ------------- ---------ac2dcfd6-555b-460e-85bf-1c656367dc2c Todd Luttinen False 13.24 GB (14,219,331,170 bytes) 616162d6928c49-19d8-481d-acbf-139e86cbbfa7 Ananth Ramanathan False 12.59 GB (13,514,775,720 bytes) 337668c672c655-cbb2-4684-a366-35dc9c2e95e1 Online Archive - Ananth Ramanathan True 7.717 GB (8,285,648,981 bytes) 27588112c16313-df0f-43e5-abb7-06d420f922f5 Archive - Danilo Benzatti True 7.788 GB (8,362,091,456 bytes) 244379b037b150-9abe-46ba-83f8-c65cc4973f0d Paul Bowden False 7.474 GB (8,025,245,880 bytes) 22672167894514-a60a-4050-94f7-566c196154fa Danilo Benzatti False 5.565 GB (5,975,363,784 bytes) 206090a32d480c-6aac-41f5-9915-df1cdd70c045 William Rall False 9.239 GB (9,920,247,961 bytes) 2032793b85bb49-936e-4f9c-83d1-7c3ee4e4b1ee Ashish Mishra False 4.495 GB (4,826,655,382 bytes) 1990345796fb94-482f-4f54-ba15-c516c718afcf Liang Mou False 8.151 GB (8,751,575,868 bytes) 1977678b493423-a362-4a0e-b97c-0ec6bfc483cf Mihai Jalobeanu False 8.989 GB (9,652,362,586 bytes) 180590
Get-StoreUsageStatistics Example[PS] C:\>$mailbox = get-mailbox toddlutt[PS] C:\>$statistics = Get-StoreUsageStatistics -Database $mailbox.database[PS] C:\>$statistics | ? {$_.DigestCategory -eq 'timeInServer'} | sort TimeInServer -descending | select -first 10 | FT -a DisplayName,SampleTime,TimeIn*,LogRecord*,RopCount,LDAP*
DisplayName SampleTime TimeInServer TimeInCPU LogRecordCount LogRecordBytes ROPCount LdapReads LdapSearches----------- ---------- ------------ --------- -------------- -------------- -------- --------- ------------Jack Xiao 9/4/2012 3:11:53 PM 13905 0 0 0 1514 0 0Ananth Ramanathan 9/4/2012 3:16:53 PM 7074 0 290 38893 1620 0 0Liang Mou 9/4/2012 3:16:53 PM 5884 0 89 4623 983 0 0Ananth Ramanathan 9/4/2012 3:10:53 PM 5651 0 142 209590 1520 0 0Ananth Ramanathan 9/4/2012 3:13:53 PM 5058 0 155 15729 1580 0 0Ananth Ramanathan 9/4/2012 3:12:53 PM 4643 0 0 0 1131 0 0Ananth Ramanathan 9/4/2012 3:15:53 PM 4575 0 146 12036 1306 0 0Ananth Ramanathan 9/4/2012 3:19:53 PM 4528 0 9 299 1163 0 0Ananth Ramanathan 9/4/2012 3:18:53 PM 4306 0 216 163766 1380 0 0Ananth Ramanathan 9/4/2012 3:11:53 PM 3999 0 0 0 1092 0 0
[PS] C:\>$statistics | ? {$_.DigestCategory -eq 'LogBytes'} | sort LogRecordBytes -descending | select -first 10 | FT -a DisplayName,SampleTime,TimeIn*,LogRecord*,RopCount,LDAP*
DisplayName SampleTime TimeInServer TimeInCPU LogRecordCount LogRecordBytes ROPCount LdapReads LdapSearches----------- ---------- ------------ --------- -------------- -------------- -------- --------- ------------Ben Rinaca 9/4/2012 1:43:51 PM 205649 0 1150007 413624055 68559 0 9Neil Joyer 9/4/2012 2:33:52 PM 67961 0 379201 121880991 27142 0 0Neil Joyer 9/4/2012 2:43:52 PM 90086 0 248803 83060705 18797 0 6William Rall 9/4/2012 1:53:51 PM 15971 0 14232 35585703 4301 0 0Syed Ali (Sapphire Technologies Canada) 9/4/2012 3:02:53 PM 16822 0 340227 10458217 5224 0 0Paul Bowden 9/4/2012 2:12:52 PM 24531 0 8885 7390547 21883 0 4Todd Luttinen 9/4/2012 2:03:51 PM 133244 0 28227 6846890 23852 0 2Paul Bowden 9/4/2012 2:53:53 PM 12208 0 5539 5198231 15663 0 4Daniel Popper 9/4/2012 2:43:52 PM 9472 0 5621 4396546 9600 0 5Paul Bowden 9/4/2012 2:33:52 PM 11543 0 4504 3754540 15831 0 0
Core Investments for Future Innovations
Core Investments
Integration of new search engine – Search FoundationSame search engine used by SharePoint 2013
C# Development PlatformImproved developer productivity through better toolsLarger community of developers within team available to contribute
Better layering of implementationImplementation of physical layer isolates underlying database engine from upper (logical) layers and MAPI implementation
Virus Scanning API (VSAPI)
Exchange 2013 does not support VSAPIIt does include transport extensibility to scan messages in-flight3rd party A/V extensions no longer run in Store process
On-demand scanning not considered an effective solution with clients that cache data (Outlook, OWA, EAS, POP, IMAP)EWS is available for scheduled and on-demand scan scenarios
Results
E14 vs. E15: DITL Performance Comparison
• 48 | 76% reduction in disk IOPS
• 18 | 41% reduction in Average RPC Latency
• 17 | 34% increase in CPU per RPC processed
• ~4x increase in Store memory overhead
DB IOPS/Mailbox0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.65
0.16
RPC Average La-tency
Mcycles per RPC packet
Store Memory per Mailbox (MB)
0
0.5
1
1.5
2
2.5
3
3.5
4
Online Mode | Cached Mode
E14 vs. E15: DITL Performance Comparison
• 48 | 76% reduction in disk IOPS• 18 | 41% reduction in Average RPC Latency• 17 | 34% increase in CPU per RPC
processed• ~4x increase in Store memory overhead
DB IOPS/Mailbox0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.65
0.16
RPC A
vera
ge Lat
ency
Mcy
cles
per
RPC
pac
ket
Stor
e Mem
ory
per M
ailb
ox (M
B)0
0.51
1.52
2.53
3.54
Online Mode | Cached Mode
LoadGen Simulation – 10 DBs/1000 usersTwo profiles: Online and Cached (Default/Optimized)
Perf gains are not free – increase in CPU and memory
CPU increase is factor of optimizing for two-socket servers and moving to multi-process architecture
Enables us to scale out using multi-core processors without having to cross processor bridge to access shared L2 cacheSome CPU overhead comes from using managed code
Memory increase is also factor of multi-processor architecture
Most of the memory is in small and large object heaps in .NET primarily used for object allocation and cleanup
IOPS Reductions
Exchange 2003 Exchange 2007 Exchange 2010 Exchange 20130
0.2
0.4
0.6
0.8
1
DB IOPS/Mailbox
IOPS/Mailbox
~95.5% Reduction!
Summary
Mailbox storage has…Reduced IOPS by 50-70%...again!Optimized for large disks (8TB) and larger mailboxes (+100GB) Better isolation leading to higher reliabilityBuilt-in monitoring and recovery to drive higher availabilityImproved data protection to reduce risk of data loss