James Serra

Implement SQL Server on an Azure VM

About Me Microsoft, Big Data Evangelist In IT for 30 years, worked on many BI and DW projects Worked as desktop/web/database developer, DBA, BI and DW architect and

developer, MDM architect, PDW/APS developer Been perm employee, contractor, consultant, business owner Presenter at PASS Business Analytics Conference, PASS Summit, Enterprise Data

World conference Certifications: MCSE: Data Platform, Business Intelligence; MS: Architecting

Microsoft Azure Solutions, Design and Implement Big Data Analytics Solutions, Design and Implement Cloud Data Platform Solutions

Blog at JamesSerra.com Former SQL Server MVP Author of book “Reporting with Microsoft SQL Server 2012”

Agenda Azure VMs Migrating data Scaling VMs SQL Server VM features VM storage HA/DR architectures Best practices

Who manages what?Infrastructure

as a Service

Storage

Servers

Networking

O/S

Middleware

Virtualization

Data

Applications

Runtime

Managed by M

icrosoft

You

scal

e, m

ake

resil

ient

& m

anag

e

Platformas a Service

Scale, Resilience and m

anagement by M

icrosoft

You

man

age

Storage

Servers

Networking

O/S

Middleware

Virtualization

Applications

Runtime

Data

On PremisesPhysical / Virtual

You

scal

e, m

ake

resil

ient

and

man

age

Storage

Servers

Networking

O/S

Middleware

Virtualization

Data

Applications

Runtime

Softwareas a Service

Storage

Servers

Networking

O/S

Middleware

Virtualization

Applications

Runtime

Data

Scale, Resilience and m

anagement by M

icrosoft

Windows AzureVirtual Machines

Windows AzureCloud Services

Virtual Machine components

Compute Storage Networking

Flexible, Scalable CPU and memory

Standard and Premium Storage tiers

Internal/External Networking

Azure VM VM hosted on Microsoft Azure Infrastructure (“IaaS”)

• From Microsoft images (gallery) or your own images (custom) SQL 2008R2 / 2012 / 2014 / 2016 Web / Standard / Enterprise

Images refreshed with latest version, SP, CU• Fast provisioning (~10 minutes). Provision groups of servers with resource templates• Accessible via RDP and Powershell• Full compatability with SQL Server “Box” software

Pay per use• Per minute (only when running)• Cost depends on size and licensing• EA customers can use existing SQL licenses (BYOL)

• Network: only outgoing (not incoming)• Storage: only used (not allocated)

Elasticity• 1 core / 2 GB mem / 1 TB 32 cores / 448 GB mem / 64 TB

Windows Azure virtual machine tiers

Basic Standard

A0 – A41 – 8 CPU cores768 MB – 14 GB RAMMax 16 datadisks w/300 IOPS per disk

For dev/test workloads or applications that don’t require load-balancing, auto-scaling, or memory-intensive VM’s.

G series (G1 – G5)2 – 32 CPU cores28 GB – 448 GB RAMUp to 64 datadisks with 500 IOPS/disk

GS Series (GS1 – GS5)2 – 32 CPU cores28 GB – 448 GB RAMUp to 64 datadisks with 5000 – 80000 IOPS/disk

F SeriesWeb/Application servers

H SeriesModeling/Simulation servers

N SeriesGraphics workloads

https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-size-specs/

A0 – A111 – 16 CPU cores768 MB – 112 GB RAMMax 16 datadisks with 500 IOPS/disk

D1 – D14 & D1_v2 – D15_v21 – 20 CPU cores768 MB – 140 GB RAMMax 16 datadisks with 500 IOPS/disk

DS1 – DS14Up to 50,000 IOPS32 – 512 MB second

Compare compute performance

ACU = Azure Compute Unit

Microsoft Azure VMs - Performance Different VM Options

A-Series: Slowest CPU, least memory (A8-A11 compute intensive) D-Series: Faster CPU, more memory. Non-persistent SSD drive (good for TempDB) Dv2-Series: 35% faster CPU than D-Series DS-Series: Same CPU and memory as D-Series. Support Premium Storage (good for Data, Log,

and TempDB!!) G-Series: Fastest CPU, most memory. GS-Series: Fastest CPU, most memory. Support Premium Storage

G5: 32 cores, 448 GB mem, 6 TB SSD, 64 1TB disks - Biggest VM in the market

Different Storage Options Standard Storage: Low throughput (max 500 IOPs p/disk), High latency (avg 40ms), pay for

used space Premium Storage: High throughput (max 5000 IOPs p/disk), Low latency (avg 4ms), pay for

allocated spaceAzure calculator: https://azure.microsoft.com/en-us/pricing/calculator/

SQL Server in Azure VMs – Performance• Premium Storage

High throughput (max 5000 IOPs p/disk) Low latency (avg 4ms) SSD disks in the backend Uses local SSD disk as read cache

Avg 1ms read latency Cache hits not part of max IOPs/bandwidth

VM Size

CPU cores

Memory

Cache size Max. IOPS

Max. Storage Bandwidth

DS1 1 3.5 GB 43 GB 3,200 32 MB/sDS2 2 7 GB 86 GB 6,400 64 MB/sDS3 4 14 GB 172 GB 12,800 128 MB/sDS4 8 28 GB 344 GB 25,600 256 MB/sDS11 2 14 GB 72 GB 6,400 64 MB/sDS12 4 28 GB 144 GB 12,800 128 MB/sDS13 8 56 GB 288 GB 25,600 256 MB/sDS14 16 112 GB 576 GB 50,000 512 MB/s

Disk Type

Disk Size

Storage IOPS

Storage Bandwidth (MB/s)

P10 128 GB 500 100

P20 512 GB 2300 150

P30 1024 GB 5000 200

GS5 32 448 GB 4,608 GB 100,000 2048 MB/s

VM Gallery Images via Azure MarketplaceCertified pre-configured software images (1087)

Azure Quickstart TemplatesFree community contributed templates (417)

Storage configuration

Automatically creates one Windows storage space (virtual drive) across all disks

Services by region

Hyper scale Infrastructure is the enabler34 Regions Worldwide, 30 Generally Available…

100+ datacenters Top 3 networks in the world 2.5x AWS, 7x Google DC Regions G Series – Largest VM in World, 32 cores, 448GB Ram, SSD…

OperationalAnnounced/Not Operational

Central US

Iowa

West USCaliforni

a

East USVirginia

US GovVirginia

North Central US

Illinois

US GovIowa

South Central US

Texas

Brazil SouthSao Paulo

State

West Europe

Netherlands

China North *

BeijingChina

South *Shanghai

Japan EastTokyo,

Saitama

Japan West

OsakaIndia South

Chennai East AsiaHong Kong

SE AsiaSingapo

re

Australia South East

Victoria

Australia EastNew South

Wales

India CentralPune

Canada EastQuebec City

Canada CentralToronto

India West

Mumbai

Germany North East **

Magdeburg

Germany Central **Frankfurt

North EuropeIreland

East US 2

Virginia

United KingdomRegions (south &

west)

US DoD EastTBD

US DoD WestTBD

* Operated by 21Vianet ** Data Stewardship by Deutsche Telekom

Korea (Seoul

& South)

West US 2

West Central

US

Migrating DataMigrate from on-prem SQL server to Azure VM IaaS: • Use the Deploy a SQL Server Database to a Microsoft Azure VM wizard. Recommended method for migrating an on-

premises user database when the compressed database backup file is less than 1 TB. Use on SQL Server 2005 or greater to SQL Server 2014 or greater

• Perform on-premises backup using compression and manually copy the backup file into the Azure virtual machine and then do a restore (only if you cannot use the above wizard or the database backup size is larger than 1 TB). Use on SQL Server 2005 or greater to SQL Server 2005 or greater

• Perform a backup to URL and restore into the Azure virtual machine from the URL. Use on SQL Server 2012 SP1 CU2 or greater to SQL Server 2012 SP1 CU2 or greater

• Detach and then copy the data and log files to Azure blob storage and then attach to SQL Server in Azure VM from URL. Use on SQL Server 2005 or greater to SQL Server 2014 or greater

• Convert on-premises physical machine to Hyper-V VHD, upload to Azure Blob storage, and then deploy as new VM using uploaded VHD. Use when bringing your own SQL Server license, when migrating a database that you will run on an older version of SQL Server, or when migrating system and user databases together as part of the migration of database dependent on other user databases and/or system databases. Use on SQL Server 2005 or greater to SQL Server 2005 or greater

• Ship hard drive using Windows Import/Export Service. Use when manual copy method is too slow, such as with very large databases. Use on SQL Server 2005 or greater to SQL Server 2005 or greater

• If you have an AlwaysOn deployment on-premises and want to minimize downtime, use the Add Azure Replica Wizard to create a replica in Azure and then failover, pointing users to the Azure database instance. Use on SQL Server 2012 or greater to SQL Server 2012 or greater

• If you do not have an AlwaysOn deployment on-premises and want to minimize downtime, use SQL Server transactional replication to configure the Azure SQL Server instance as a subscriber and then disable replication, pointing users to the Azure database instance. Use on SQL Server 2005 or greater to SQL Server 2005 or greater

• Others: data-tier application, transact-SQL scripts, sql server import and export wizard, SSIS, copy database wizard

Scale VMs

Scale VMs• Be aware a region may not support the VM

size• Resize requires just a VM reboot if in same

family or if Azure hardware cluster supports new VM size

• If hardware cluster does not support new VM size: • If using Resource Manager (ARM)

deployment model you can resize VMs by first stopping your VM, selecting a new VM size and then restarting the VM

• If using Classic (ASM) deployment model, VMs must be deleted and then recreated using the same OS and data disks. See PowerShell script

Microsoft Azure VMs – ManagementAutomated Patching• Predictable solution for patching (Windows & SQL)

• Simple: just specify a time window• Uses SQL Agent Extension and MS Update• Portal and Powershell• It relies on the Windows Update and the

Microsoft Update infrastructure and installs any update that matches the ‘Important’ category for the machine

Microsoft Azure VMs – ManagementAutomated Backup• For all DBs in the SQL instance• Simple: just specify a retention period• Supports Compression and Encryption• Portal and Powershell• Full database and transaction log backups• Configure at database level or SQL Server instance

level

In SQL Server 2016:• Full, bulk-logged and simple recovery models are

all supported• System databases can be configured for backups• Backup striping can be used to support backup

sizes of up to 12 TB• Customer backup schedules can be specified to

ensure your backups are created when it is best for your workload

Licensing/bring your own license

Obtain SQL image from Azure VM

gallery

pay per use

Install or upload your

own SQL Server Image

Connecting to Azure VMs

Virtual Network

VPN GW

Frontend10.1/16

Mid-tier10.2/16

Backend10.3/16

Internet

On Premises10.0/16

VPN &ExpressRoute

AD / DNS

Azure

Direct InternetConnectivity

Virtual Machine networking• Create subnets with private or public IP addresses• Bring your own DNS or use Azure-provided DNS• Secure with Network Security Groups ACLs• Control traffic with user-defined routes

Connectivity options and hybrid offeringsCloud Customer Segment and

workloads

Secure site-to-site VPN connectivity

• SMB, Enterprises• Connect to Azure

compute

Secure point-to-site connectivity

• Developers• POC Efforts• Small scale

deployments• Connect from

anywhere

ExpressRoute private connectivity

• SMB, Enterprises• Mission critical workloads• Backup/DR, media, HPC• Connect to Microsoft

services

Internet connectivity

• Consumers• Access over public IP• DNS resolution• Connect from anywhere

Virtual Machine storage architecture

C:\OS disk (127 GB)

Usually 115 GB free

E:\, F:\, etc.Data disks (1 TB)Attach SSD/HDD up to 1TB.

These are .vhd files

D:\Temporary disk

(Contents can be lost)

SSD/HDD and size depends on VM chosenDisk Cache

Azure Blob storage

Azure Default Blob Storage Azure Storage Page Blobs, 3 copies Storage high durability built-in

(like have RAID) VHD disks, up to 1 TB per disk (64 TB

total)

Geo-storage replication 3 copies locally, another 3 copies in

different region Disable for SQL Server VM disk

(consistent write order across multiple disks is not guaranteed). Instead use DR techniques in this deck

Defend against regional disasters

Geo replication

Blog Storage Replication Options

LRS = Locally Redundant Storage (synchronous, same facility)ZRS = Zone-Redundant Storage (synchronous,, multiple facilities)GRS = Geo-Redundant Storage (asynchronous, multiple regions – secondary region chosen for you)RA-GRS = Read-Access Geo-Redundant Storage (readable)

HA/DR pieces• Virtual Machine• Application• Storage• Network• SQL Server (main focus of this deck)

HA/DR deployment architecturesAlwaysOn Failover Cluster

Instances (FCI)

AlwaysOn Availability

Groups

Database Mirroring

Log Shipping Backup to Azure (blob

storage)

Azure Site Recovery

Azure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery.Cross-region solution protects against complete site outage.Hybrid Some availability replicas running in Azure VMs and other replicas running on-premises for cross-site disaster recovery.

HA only, not DR FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute.

Azure OnlyPrincipal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA.HybridOne partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates.

For DR only / Hybrid onlyOne server running in an Azure VM and the other running on-premises for cross-site disaster recovery.Log shipping depends on Windows file sharing, so a VPN connection between the Azure virtual network and the on-premises network is required.Requires AD deployment on DR site.

On-prem or Azure production databases backed up directly to Azure blob storage for disaster recovery.

SQL 2016: Backup to Azure with file snapshots

Simpler BCDR storySite Recovery makes it easy to handle replication, failover and recovery for your on-premises workloads and applications (not data!).Flexible replication You can replicate on-premises servers, Hyper-V virtual machines, and VMware virtual machines. Eliminate the need for secondary

SQL Server data files in

AzureNative support for SQL Server data files stored as Azure blobs

RPO/RTORTO – Recover Time Objective. How much time after a failure until we have to be up and running again?RPO – Recover Point Objective. How much data can we lose?• HA – High Availability• RTO: seconds to minutes• RPO: Zero to seconds• Automatic failover• Well tested (maybe with each patch or

release)• DR – Disaster Recovery• RTO: minutes to hours• RPO: seconds to minutes • Manual failover into prepared

environment• Tested from time to time

How long does it take to fail over:• Backup-Restore: Hours• Log Shipping: Minutes• AlwaysOn FCI: Seconds to

minutes• AlwaysOn AG/Mirroring:

Seconds

AlwaysOn Availability GroupsAlwaysOn Availability

GroupsAzure Only Availability replicas running across multiple datacenters in Azure VMs for disaster recovery.Cross-region solution protects against complete site outage.

HybridSome availability replicas running in Azure VMs and other replicas running on-premises for cross-site disaster recovery.

Availability replicas running across multiple datacenters in Azure VMs for disaster recovery. This cross-region solution protects against complete site outage. Within a region, all replicas should be within the same cloud service and the same VNet. Because each region will have a separate VNet, these solutions require VNet to VNet connectivity. For more information, see Configure a Site-to-Site VPN in the Azure classic portal.

All availability replicas running in Azure VMs for high availability within the same region. You need to configure a domain controller VM, because Windows Server Failover Clustering (WSFC) requires an Active Directory domain.For more information, see Configure AlwaysOn Availability Groups in Azure (GUI).

Microsoft Azure VMs – HA & DRAlwaysOn between Azure Regions• Configure AlwaysOn between VMs in different geographic regions (asynchronous)• Over secure tunnel

• Manual Failover (~15 seconds) in case of a regional failure• Test it at any time

• Use closest secondary for read workloads• Region 1: AG used instead

of FCI (synchronous)

AlwaysOn Failover Cluster Instances (FCI)

An FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute. For example, NetApp Private Storage (NPS) exposes an iSCSI target via ExpressRoute with Equinix to Azure VMs.

For third-party shared storage and data replication solutions, you should contact the vendor for any issues related to accessing data on failover.

Note that using FCI on top of Azure File storage is not supported yet, because this solution does not utilize Premium Storage. We are working to support this soon.

Always On Failover Cluster

Instances (FCI)HA only, not DR

FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.

FCI on a two-node WSFC running in Azure VMs with remote iSCSI Target shared block storage via ExpressRoute.

You can use FCI to host an availability replica for an availability group

FCI on a two-node WSFC running in Azure VMs with storage supported by a third-party clustering solution.

AlwaysOn FCI vs AlwaysOn AG

Database MirroringDatabase Mirroring

Azure OnlyPrincipal and mirror and servers running in different datacenters for disaster recovery. Principal, Mirror, and Witness run within same Azure data center, deployed using a DC or server certificates for HA.HybridOne partner running in an Azure VM and the other running on-premises for cross-site disaster recovery using server certificates.

Principal and mirror and servers running in different datacenters for disaster recovery. You must deploy using server certificates because an Active Directory domain cannot span multiple datacenters.

Principal, mirror, and witness servers all running in the same Azure datacenter for high availability. You can deploy using a domain controller.

You can also deploy the same database mirroring configuration without a domain controller by using server certificates instead.

Block blobsReduced storage costsSignificantly improved restore performanceMore granular control over Azure Storage

Azure Storage snapshot backupFastest method for creating backups and running restoresSupport of SQL Server database files on Azure Blob Storage

Backup to AzureBackup to

Azure (blob storage)

Managed backupOn-prem to AzureGranular control of the backup scheduleLocal staging for faster recovery and greater network resiliencySystem database supportSimple recovery mode support

On-prem or Azure production databases backed up directly to Azure blob storage for disaster recovery.

SQL 2016: Backup to Azure with file snapshots

Production databases backed up directly to blob storage in a different datacenter for disaster recovery

On-premises production databases backed up directly to Azure blob storage for disaster recovery.

SQL Server in Azure VM Best Practices

https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-sql-server-performance-best-practices/

VM size• DS3 or higher for SQL Enterprise edition• DS2 or higher for SQL Standard and Web editionStorage• Use Premium Storage• Keep the storage account and SQL Server VM in the same region• Disable Azure geo-redundant storage (geo-replication) on the storage account (consistent write order across multiple disks is not guaranteed)Disks• Use a minimum of 2 P30 disks (1 for log files; 1 for data files and TempDB)• Avoid using operating system or temporary disks for database storage or logging• Enable read caching on the disk(s) hosting the data files and TempDB• Do not enable caching on disk(s) hosting the log file• Stripe multiple Azure data disks to get increased IO throughput• Format with documented allocation sizesI/O• Enable database page compression• Enable instant file initialization for data files• Limit or disable autogrow on the database• Disable autoshrink on the database• Move all databases to data disks, including system databases• Move SQL Server error log and trace file directories to data disks• Setup default backup and database file locations• Enable locked pages• Apply SQL Server performance fixes

© 2016 Microsoft Corporation. All rights reserved.