Developing a SQL Server 2008 Fast Track Data Warehouse

Ross LoForteTechnology ArchitectMicrosoft Corporation

SESSION CODE: BIE07-INT

Eric KraemerSenior Program ManagerMicrosoft Corporation

Agenda

● SQL Fast Track DW Overview● Fast Track DW Implementation Key Principles (Server)● Fast Track DW Implementation Key Principles (Storage Layer)● Fast Track DW Implementation Key Principles (Data Loading)● Q&A

Fast Track DW Overview

Challenges of Traditional Data Warehouse

Key to Fast Track Data Warehouse Architecture

Data Warehouse appliances for SQL Server

The Alternative: A Balanced System

● Balance storage IO capability to the Server/CPU capability to process data● Ensure consistent, predictable IO throughput● Match the storage IO optimization to the RDBMS workload pattern

• Scan or Seek? IOPs or MB/s?● Configure the database software to take advantage of the optimized

server, network, and storage “Stack”

Fast Track SQL DW Architecture vs. Traditional DW

SQL 2008 Data Warehouse4 Processor 16 Core Server

Shared Network Bandwidth

Enterprise Shared SAN Storage

Dedicated Network Bandwidth

Traditional SQL DWArchitectureShared Infrastructure

Fast Track SQL DW ArchitectureDedicated DW InfrastructureArchitecture modeled after DW Appliances 1TB – 48TB Pre-Tested

Dedicated Low Cost SAN Arrays 1 for every 4 CPU Cores EMC AX4 – HP MSA2312

OLTP Applications

Benefits:-More System Predictability Thus User Experience-Pretested Configurations Lowers TCO-Balanced CPU to I/O Channel Optimized for DW-Modular Building Block Approach-Scale Out or Up within limits of Server and SAN

What is Fast Track Data Warehouse?

A method for designing a cost-effective, balanced system for Data Warehouse workloads Reference hardware configurations developed in conjunction with hardware partners using this methodBest practices for data layout, loading and management

Relational Database Only – Not SSAS, IS, RS

Fast Track Component Architecture

Fast Track focuses on balancing the major hardware components“Major” is defined as the components relevant to overall I/O throughput and data processing capability“Balance” for Fast Track is defined in relation to• SQL Server data processing capability• Storage system I/O capability• Total hardware component cost

Fast Track Data Warehouse Components

Software:•SQL Server 2008 Enterprise•Windows Server 2008

Configuration guidelines:• Physical table structures• Indexes• Compression• SQL Server settings• Windows Server settings• Loading

Hardware:•Tight specifications for servers, storage and networking•‘Per core’ building block

Lower TCOMinimizes risk of overspending on un-balanced hardware configurationsCommodity Hardware

ChoiceHW platformImplementation vendor

Reduced RiskValidated by MicrosoftEncapsulates best practicesKnown performance & scalability

Fast Track Data Warehouse Benefits

SI Solution Templates

Twelve SMP Reference Architectures

Sequential I/O

Sequential I/O

Scans on large data stores are usually read with sequential read patterns and not random read patternsScalable, predictable performanceRequires 1/3 or fewer drives to match server I/O consumption capability.

Random I/O

OLTP usually random-read centric. Discrete lookups benefit from index optimization and random read capability.Not as predictable & scalable for data warehousingRequires large number of drives to match server I/O consumption capability.

All databases contain both scans and seeks among with other types of reads and writes, DW workload indicate that the vast

majority of reads are sequential – not all

Fast Track DW Implementation Key Principles

Server Layer

Fast Track Data Warehouse Components Balanced across all components

FCHBA

AB

AB

FCHBA

AB

AB FC

SW

ITCH

STORAGECONTROLLER

AB

ABCA

CHE

SERV

ER

CACH

ESQ

L SE

RVER

WIN

DO

WS

CPU

CO

RES

CPU Feed Rate HBA Port Rate Switch Port Rate SP Port Rate

A

BDISK DISK

LUN

DISK DISK

LUN

SQL Server Read Ahead Rate

LUN Read Rate Disk Feed Rate

SQL Server 2008 Potential Performance Bottlenecks

Current Fast Track Architectures are rated at 200 MB/s per CPU core

System Validation

Validation is intended to confirm the proper installation and configuration of a Fast Track RAValidation is achieved in two phases

Synthetic IO testingValidates storage, network, and operating systemSQLIO can be used to generate IOPerfmon can be used to monitor results

SQL Server testingValidates performance across SQL Server stackFinal step of deployment process

Core Fast Track Metrics

These metrics are use to both validate and position Fast Track RA’sMaximum Consumption Rate – Ability of SQL Server to process data for a specific CPU and Server combination and a standard SQL query.

Benchmark Consumption Rate – Ability of SQL Server to process data for a specific CPU and Server combination and a user workload or query.

User Data Capacity – Maximum available SQL Server storage for a specific Fast Track RA assuming 2.5:1 page compression factor and 300 GB 15K SAS. 30% of this storage should be reserved for DBA operations

MCRSimilar in concept to Miles Per Gallon rating for a new car

Not necessarily what you will see when you drive the car, but a good starting point

Provides a standard reference point forSimple evaluationsRelative comparison between different Fast Track configurationsSystem validation and benchmarking

Current value for published Fast Track RA’s200MB/s per core

Fast Track Benchmark Results• Actual results from Fast Track validation

• HP 2 socket, 8 core Configuration

Server

Windows Server OS

MCR 1.6 GB/s

Storage Enclosure

Storage Enclosure

Fib

er

Sw

itch

500 MB/s

500 MB/s

500 MB/s

500 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

HBA

HBA

Min2

GB/s Min 2 GB/s

BCR

Similar to actual MPG you get with your current driving habitsProvides a workload specific reference point

Defines the ideal outcome of the Full Evaluation scenarioCan be compared to MCR to choose an appropriate FT configurationProvides a framework for validating Fast Track data warehouse configurations.

Fast Track Benchmark Results

Server

SQL Server OS

BCR 1.2 GB/s

HBA

HBA

Storage Enclosure

Storage Enclosure

Fib

er

Sw

itch

1.2 GB/s

1.2 GB/s

300 MB/s

300 MB/s

300 MB/s

300 MB/s

150 MB/s

150 MB/s

150 MB/s

150 MB/s

150 MB/s

150 MB/s

150 MB/s

150 MB/s

• Actual results from Fast Track validation• HP 2 socket, 8 core Configuration

UDCUDC is customer supplied and is the data capacity required

Plan for projected growthBased on customer projectionsNeeds to be allocated up-front

Allocate for data management needsStaging database requirementsTemporary objects

Allocate for TempDBTypically 20-30% of primary data spaceTempdb is not compressed

Fast Track DW Implementation Key Principles

Storage Layer

Storage – Disk Configuration

Fast Track is very disk efficientFT uses RAID-1 to enable sequential I/O

2 disk RAID-1 array per CPU coreDepending on which FT system is selected, system will have at least 16 RAID-1 arraysCreates virtual affinity between a RAID-1 array and a CPU coreData is evenly split across RAID-1 arrays using partitioning

Enables ability to load data sequentiallySequential I/O uses about 1/3 of the number of disks versus random I/O to get same level of performance

Storage – Disk Configuration

Creating RAID GroupsHP MSA, EMC AX, IBM DS

11 disks per enclosure10 dedicated to user data1 hot spare

1 Storage Enclosure per (4) physical Cores2 socket quad core server

2 Storage Enclosures – 22 total disksRaid Configuration

Primary data: (4) 2 disk RAID-1 arraysLog: (1) 2 disk RAID-1 array

SWIT

CH

SP

A

SP B

SQL Server 2008 Minimum Server Configuration SMP Core-Balanced Architecture using Dual Read on HP MSA 2312

Per MSA2312 Drive Details• Each MSA can hold 12 drives, this configuration requires 11• MSA is 2U in total (capacitor eliminates need for battery)• Each MSA SP port controls 4 LUNs, SP-A also controls LOG LUN• Each pair of LUNs consists of (2) 300GB 15k FC drives RAID1

Each SP rated at 500MB/s or 1000MB/sfor both SP’s

Using 300GB 15k FC driveseach LUN rated at 125MB/seach SP controls 4 LUN’s at 500MB/s or 1000MB/s per MSA DAE

Each SP port rated at 4Gb/sor 400MB/s and 1600MB/s for all 4 SP ports.

Each HBA port rated at 4Gb/sor 400MB/s and 1600MB/s for all 4 HBA ports.

03 04

RAID GP02

LUN3

LUN4

01 02

RAID GP01

LUN1

LUN2

05 06

RAID GP03

LUN5

LUN6

07 08

RAID GP04

LUN7

LUN8

09 10

RAID GP05

LUN0(Logs)

HS

Quad Core CPU* Compressed Data

200MB/s per Core*

200MB/s per Core*

HBA FC 1 4Gb/s or 400MB/s x 2

200MB/s per Core*

200MB/s per Core*

HBA FC 24Gb/s or 400MB/s x 2

DAE = Disk Array EnclosureHBA = Host Bus AdapterSP = Storage ProcessorFC = Fibre ChannelPorts = 4Gbs FC

SQL Server File Layout

LUN16 LUN 2 LUN 3

Local Drive 1

Log LUN 1

Permanent DB Log

LUN 1

Tem

pD

B

TempDB.mdf (25GB) TempDB_02.ndf (25GB) TempDB_03ndf (25GB) TempDB_16.ndf (25GB)

Permanent FG

Permanent_1.ndf

Per

ma

na

nt_

DB

Sta

ge

D

ata

ba

se Stage FG

Stage_1.ndf Stage_2.ndf Stage_3.ndf Stage_16.ndf

Stage DB Log

Permanent_2.ndf Permanent_3.ndf Permanent_16.ndf

Log LUN 2

Permanent DB Log

Stage DB Log

SQL Server Files

User DatabasesCreate at least one Filegroup containing one data file per LUN

FT targets 1:1 LUN to CPU core affinityMake all files the same sizeEffectively stripes database files across data LUNs

Multiple file groups may be advantageousDisable Auto-Grow for the databaseTransaction Log is allocated to a Log LUN.

SQL Server Files

Transaction LogCreate a single transaction log file per database and place on a dedicated Log LUN.Enable auto-grow for log filesThe transaction log size for each database should be at least twice the size of the largest operation

SQL Server Files

TempdbCreate one Tempdb data file per LUN

Make all files the same sizeFollow standard tempdb best practices

Auto-Grow should be enabled for tempdbUse large growth increment (10% of initial size)

Fast Track DW Implementation Key Principles

Data Loading

Conventional data loads lead to fragmentation

Bulk Inserts into Clustered Index using a moderate ‘batchsize’ parameter• Each ‘batch’ is sorted independentlyOverlapping batches lead to page splits

1:321:31 1:351:341:331:36 1:381:37 1:401:391:321:31 1:351:341:33

Key Order of Index

Minimizing File fragmentation

Pre-allocate database files• Size files correctly to prevent growth• Do not shrink filesDo not use NTFS file fragmentation tools: Rebuild table to ensure disk block level optimal organizationWriting data

Concurrent load operations to the same file will induce fragmentationDML change operations (Update/Delete) may induce fragmentation

Consider Filegroups and Partitioning to manage concurrent writes for large tables

Writing Sequential Data

• Sequential scan performance starts with database creation and extent allocation

• Recall that the –E startup option is used• Allocate up to 64 extents contiguously (4MB)

• Pre-allocation of user databases is strongly recommended• Autogrow should be avoided if possible

• If used, always use T1117

Loading DataGoal: Minimize logical fragmentation, maximize read performance

Minimizes Disk head movementMaintains high average request size (Think ~400k not 8k)

Implies lower operation (IOP) counts to achieve high scan rates than traditionally seen with SQL Server.

Sustain high average scan rates (up to 240 MB/s per RAID1 LUN)Key considerations for a Fast Track data load

Data Architecture: Destination table, partitioning, and filegroupSource Data: Format & sizeSystem Resources: CPU & Memory

Data Architecture

Starting point for building a Fast Track Load methodIdentify target table type

Structure (Heap, Cluster Index)Define data volatility for Cluster Index targets

Choose table architecture based on data volatilityPartition geometryFilegroup geometry

SQL Filegroups enable concurrent Load/DML operations with minimal logical fragmentation. Partitioning allows a single table to live in multiple Filegroups.

Source Data and ResourcesConsider source data architecture

Type: File or StreamTransaction: Bulk or RowFormat: Ordered, unordered, multi-file, single-fileFlexibility: Split to multiple files, key order

System resourcesCPU Cores & Memory

Review existing FT best practices for loadsmsdn.microsoft.com/en-us/library/dd459178.aspx

Building a Fast Track Bulk Load ProcessScenario: Single table migration (320GB)

Destination Table: Page Compressed, Partitioned Cluster IndexSource data considerations

Location: Legacy DBFrequency: One-time extractFormat: 8 Flat files, Unordered data

System Information: 8 CPU Cores, 192GB memorySQL file structure

8 years data, 8 total partitions: 40GB per partition4 Filegroups, 2 partitions per filegroup

Filegroup“Stage A”

Filegroup “A”

Partition 1,2

Filegroup “B”

Partition 3,4

Filegroup “C”

Partition 5,6

Filegroup “D”

Partition 7,8

Filegroup“Stage B”

Example: Fast Track Migration Load to Partitioned CI

8 Source Data Files

8 Core Server Base Heap StageTable

No Compression

Target Database

Partition 2 Destination CI

Partition 1 Destination CI

Partition 4 Destination CI

Partition 3 Destination CI

Partition 6 Destination CI

Partition 5 Destination CI

Partition 8 Destination CI

Partition 7 Destination CI

8 Concurrent BCP Loads

Step 1“Base Load”

Step 2“Stage Insert”

Step 3“Transform” Step 4

“Final Append”

8 Heap Stage TableConstraint on CI Part

Key

8 Concurrent Inserts

2 CI Stage Tables

2 CI Stage Tables

2 CI Stage Tables

2 CI Stage Tables

2 sets, 4 concurrent Create Cluster Index with Compression

INTO “Final Destination”

Create CI

8 Concurrent Partition Switch

Part Switch

Part Switch

Part Switch

Part Switch

Destination Partitioned CI Table

Fast Track Partition CI Load (Migration): PrinciplesThe following determines Filegroup Architecture

VolatilityPartition SizeAvailable MemoryTotal Physical CPU Cores

Maximize efficiency of the initial Bulk LoadAvoid sortsAvoid page lock contentionAvoid compression prior to creation of index

Maximize the Create Index operationStage by partition to keep sorts in memoryParallelize across filegroups to minimize fragmentationCompress with the Create IndexPartition switch into destination

Applying Concepts from the Partitioned CI Example

Scenario: 1GB Daily incremental load to Partitioned CISource Data

Single text file, unorderedCurrent data may touch two most recent partitions

Source Table: 48 monthly partitionsSystem: 8 Core, 192GB RAMSQL file structure

Filegroup “Historical”: Partitions 1..46Filegroup “Current”: Partitions 47,48

Filegroup“Stage A”

Filegroup “Current” Partition

47,48

Filegroup “Historical”

Partition1-46

Example: Fast Track Incremental Load to Partitioned CI

8 Core Server

Target Database

Partition 48 Destination CI

Partition 47 Destination CI

BCP Load

Step 1“Base Load”

Destination Partitioned CI Table

1 Source Data File

Partition 1.. Destination CI

ResourcesSQL Server Fast Track DW Home Page

http://www.microsoft.com/sqlserver/2008/en/us/fasttrack.aspx

Fast Track DW 2.0 Architecture Whitepaperhttp://msdn.microsoft.com/en-us/library/dd459178.aspx

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

INFRASTRUCTURE PLANNING AND DESIGN (IPD) GUIDEMicrosoft SQL Server 2008 and SQL Server 2008 R2What are IPD Guides?

Guidance & best practices for infrastructure planning of Microsoft technologies

SQL Server Guide BenefitsHelps organizations confidently plan a Microsoft SQL Server 2008 and SQL Server 2008 R2 implementation.

Assists database administrators and technical decision makers identify appropriate server roles

Guides architects and administrators in determining the infrastructure components, server placement, and fault-tolerance configuration

It’s a free download!Go to www.microsoft.com/ipd

Check out the entire IPD series for streamlined IT infrastructure planning

“At the end of the day, IT operations is really about running your business as

efficiently as you can so you have more dollars left for innovation. IPD guides help

us achieve this.” Peter Zerger, Consulting Practice Lead for Management Solutions, AKOS Technology Services

Complete an evaluation on CommNet and enter to win!

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Fast Track DW Implementation Key Principles

High Availability

High Availability

Fans 6 hot plug redundant fans, 3 shown

Core I/O -2 USB, 1 serial, 1 video port,3 RJ-45 PS2 keyboard/mouse support

I/O slots11 PCIe slots std.,Option to upgrade to 2 HTx and 7 PCIe

Power Supplies -3+3 redundant power supplies

Clustering With Server 2008

No single points of failure in Failover Clustering!

Make Clustering SimpleEasy to create, use, and manageEnabling the IT Generalist

Reduce Total Cost of OwnershipMaking Clusters a smart business choice for the enterprise

Support for 16 node clusters

Fast Track DW Case Study

Fast Track Case Study - #1

Current EnvironmentTeradata 4-node (5450 model) with 6TB of user dataBI: Business ObjectsETL: Informatica and BTEQ scripts

Proposed Microsoft PlatformSQL Server Fast Track Data WarehouseHP DL580 Server - 4 Quadcore Processors (16 core total)256 GB MemorySAN Storage: MSA 2000 (Qty 4) – 8TB User Data CapacityBI: Business ObjectsETL: SQL Server and SSIS

Fast Track Case Study – #1 Results

Teradata SQL Server Fast Track DW Comparison

Loading Subject Area 1 5:10:21 total time 0:51:31 total time R

6x faster

Loading Subject Area 2 4:36:08 total time 1:50.01 total time R

2.5x faster

Query times Subject Area 1

3:03 avg query time(using 9 benchmark queries)

0:15 avg query time(using 9 benchmark queries)

R 12x faster

Query times Subject Area 2

56:44 avg query time(using 4 benchmark queries)

8:09 avg query time(using 4 benchmark queries)

R 7x faster

Large Retailer with limited capabilities because of their legacy based business intelligence solution. The solution has capacity for 212 users at the cost of ~1 million in annual maintenance. Competition – Netezza & Oracle

1) Lower their maintenance cost2) They wanted to address the business needs (POS data, etc)3) They also wanted to proliferate the advantages of Business Intelligence across their

enterprise.

Business Needs

Solution

Situation

Fast Track Case Study #2 - Retailer

Full MS BI stack Fast Track , SSRS, Excel Services , PPS & Office 2007Our solution will replace and extend the existing DB2 AS400 systemSSIS will replace existing COBOL ETL (including ODI)

Benefit1) Improvements in market analysis and business forecasting 2) Reduce maintenance costs by $50K per month

Sequential Scan Components

• Contiguous allocation, data striping, pre-fetch, and read-ahead work to create efficient Sequential IO• Data stripe width is balanced against read-ahead “Depth”• Combined, these elements provide effective access to the full data stripe from a single thread

• Each element is necessary to maximize efficiency

ARY01D1v01

ARY01D2v02

ARY02D1v03

ARY02D2v04

ARY03D1v05

ARY03D2v06

ARY04D1v07

ARY04D2v08

DB1-1.ndf DB1-7.ndfDB1-5.ndfDB1-3.ndf

DB1-2.ndf DB1-4.ndf DB1-6.ndf DB1-8.ndf

4MB

4MB4MB

4MB4MB4MB

4MB4MB

Fast Track Data Stripe

512k read*2MB Pre-Fetch

*8MB Read-Ahead

Achieving Sequential Scan

Fast Track GoalsMaintain sequential data layoutIncrease effective maximum request size beyond the 512k limitMinimize disk head movementLeverage

RAID-1Storage Enclosure pre-fetchSQL Server Read-Ahead

These elements combine to create optimized Sequential Scan performance

57©2009 Microsoft Corporation

Fast Track SMP RA for SQL Server 2008 CPU Core Calculator v2.4Updated 10/09/2009 - uw

This spreadsheet can be used to estimate the number of cores required to support a user workload and workload mix.Enter your factors into the green fields and the results will be calculated in the pink cells.The spreadsheet uses a weighted average to determine the number of cores required based on your inputs.User Variable Input

Anticipated total number of users expected on the system 3,000 users

Adjust for workload mix

Estimated % of workload

Estimated % data found in

SQL Server cache

Estimated Query Data

Scan Volume MB (Uncompressed)

Desired Query Response Time

(seconds)(under load)

Estimated Disk Scan volume MB (Uncompressed)

Estimated percent of actual query concurrency 1% concurrency Simple 70% 10% 8,000 25 7,200Fast Track DW CPU max core consumption rate

(MCR) in MB/s of page compressed data per core 200 MB/s Average 20% 0% 75,000 180 75,000

Estimated compression ratio (default = 2.5:1) 2.5 :1 Complex 10% 0% 450,000 1,200 450,000Estimated drive serial throughput speed in

compressed MB/s 100 MB/s 100%Number of data drives in single storage array 8 drives

Usable capacity per drive 272 GB

Space Reserved for TempDB 26%

Calculations and Results

% of core consumption rate achieved

Expected per CPU core

consumption rate (MB/s)

Calculated Single Query Scan

Volume in MB (compressed)

Calculated Target

Concurrent Queries

Estimated Target Queries

per Hour

Required IO Throughput in

MB/s

Estimated Number of Cores

Required

Estimated Single Query Run Time

(seconds)

Simple 100% 200 2,880 21 3,024 2,419 12.10 0.5Average 50% 100 30,000 6 120 1,000 10.00 9.4Complex 25% 50 180,000 3 9 450 9.00 112.5

30 3,153 3,869 32.00

Arrays Required based on throughput

Single Array Throughput in

MB/s

Throughput in MB/s for All

Required Arrays5 800 4,000

Suggested Fast Track RA Server Requirements No of CPU

coresNumber of

arrays

Total Compressed Data Capacity

(TB)

Max achievable IO Throughput

in MB/s

Max achievable CPU consumption in

MB/s

Required IO Throughput in

MB/s

32 8 16 6,400 6,400 3,869