Sq l 2014 New Features
-
Upload
franklin-bustinza-macedo -
Category
Documents
-
view
2 -
download
0
description
Transcript of Sq l 2014 New Features
Kevin G. Boles
SQL Server Consultant Extraordinaire“The Wolf” for the Relational EngineIndicium Resources, Inc.AIM/Twitter: TheSQLGuruGTalk/GMail: [email protected] 2007-2012, MCT, MCITP, yada-yada-yada
Who Am I??
World-class relational engine expert Almost 45,000 hours invested in SQL Server Absolutely LOVE doing targeted performance
analysis and tuning work, and designing and building high-scale, highly-performing database applications
VERY good at making others better at interacting with SQL Server
Couldn’t SSAS my way out of a paper bag ◦ … with both ends open!
What Do I Do?
What’s being delivered◦ New cardinality estimator◦ Incremental statistics for partition◦ Parallel SELECT INTO
Main benefits◦ Better query performance:
Better choice of query plans Faster/more frequent statistics refresh at partition level
◦ Consistent query performance ◦ Better supportability using two steps (decision making
and execution) to enable better query plan troubleshooting
◦ Loading speed into table improved significantly using parallel operation
Query Processing Enhancements
SQL Express+ Feature SQL Server transaction commits can now be
either fully durable or delayed durable (aka Lazy Commit)
Delayed transaction durability reduces the latency due to log I/O by keeping the transaction log records in memory and writing to the transaction log in batches, thus requiring fewer I/O operations
When to use delayed transaction durability ◦ You can tolerate some data loss◦ You are experiencing a bottleneck on transaction
log writes.◦ Your workloads have a high contention rate.
Delayed Durability
The in-memory transaction log is flushed to disk when:◦ A fully durable transaction in the same database makes a
change in the database and successfully commits (KEY GOTCHA HERE!)
◦ The user executes the system stored procedure sp_flush_log successfully
◦ The in-memory transaction log buffer fills up and automatically flushes to disk
Delayed Durability
ALTER DATABASE … SET DELAYED_DURABILITY = { DISABLED | ALLOWED | FORCED }
DELAYED_DURABILITY = { OFF | ON } (with NATIVE_COMPILED sprocs ONLY
COMMIT [ { TRAN | TRANSACTION } ] [ transaction_name | @tran_name_variable ] ] [ WITH ( DELAYED_DURABILITY = { OFF | ON } ) ]
In-Memory Built-In
New In-Memory OLTP
Enhanced In-Memory ColumnStore for DW
In-Memory BI with PowerPivot (tabular model is sits in memory too!)
Buffer Pool Extension to SSDs & Enhanced Query Processing
Before
15,000 requests/s
ec
16x FasterWith In-Memory
250,000 requests/s
ec
SQL Server 2014 with In-Memory
Bwin Game Application
Bwin Game Application
Select only highly utilized tables to be in-memory
Optimize in-memory to fit existing hardware
In-memory performance across OLTP, DW and BI
All in a single SKU
On average 10x faster, without having rewrite entire app
Leverage full SQL Server capabilities
2012 Column Store IndexesIndex Storage Design• Column-Orientation
– Store data vertically instead of per-row– String Dictionaries for variable-length data
• Segment data into groups (1 million rows/group)• Benefits
– DW queries usually pick only a subset of columns– You can do the IO only for those columns– We can also compress that data effectively since it often has lots of duplicates– Space savings of 1.5x-2x vs. a row-based page-compressed equivalent
IO Patterns for:CI Scan
Column-based scan 3 colsColumn-based
w/Compression
Column-Oriented Sized to fit within L2 cache
(1000 rows at a time) Multiple Operators work on a
batch sequentially Goal: Reduce avg. per-tuple
cost ◦ Reducing L2 data and instruction
cache misses
2012 Batch-Mode Query Execution
Off The CPU Die SUCKS … a LOT!!Latency Comparison Numbers
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1
Compress 1K bytes with Zippy 3,000 ns
Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms
Read 4K randomly from SSD* 150,000 ns 0.15 ms
Read 1 MB sequentially from memory 250,000 ns 0.25 ms
Round trip within same datacenter 500,000 ns 0.5 ms
Read 1 MB sequentially from SSD* 1,000,000 ns 1 ms 4X memory
Disk seek 10,000,000 ns 10 ms 20x datacenter roundtrip
Read 1 MB sequentially from disk 20,000,000 ns 20 ms 80x memory, 20X SSD
Send packet CA->Netherlands->CA 150,000,000 ns 150 ms
Credit
Peter Norvig: http://norvig.com/21-days.html#answers
Off The CPU Die SUCKS … a LOT!!
Another Way to Look At It
L1 cache reference : 0:00:01 (one second)Branch mispredict : 0:00:10L2 cache reference : 0:00:14Mutex lock/unlock : 0:00:50Main memory reference : 0:03:20 !!!Compress 1K bytes with Zippy : 1:40:00Send 1K bytes over 1 Gbps network : 5:33:20Read 4K randomly from SSD : 3 days, 11:20:00Read 1 MB sequentially from memory : 5 days, 18:53:20 !!!Round trip within same datacenter : 11 days, 13:46:40Read 1 MB sequentially from SSD : 23 days, 3:33:20 Disk seek : 231 days, 11:33:20 !!!Read 1 MB sequentially from disk : 462 days, 23:06:40 !!!Send packet CA->Netherlands->CA : 3472 days, 5:20:00
Updatable Columnstore Index
Only One of these
Hopefully few of these
Hopefully lots of these, vast majority with ~1M row segments!
The Column Store Index IS the data now!
Table consists of column store and row store
DML (update, delete, insert) operations leverage delta store
INSERT Values◦ Always lands into delta store
DELETE◦ Logical operation◦ Data physically remove after REBUILD
operation is performed. UPDATE
◦ DELETE followed by INSERT. BULK INSERT
◦ if batch < 100k, inserts go into delta store, otherwise columnstore
SELECT ◦ Unifies data from Column and Row stores -
internal UNION operation. “Tuple mover” converts data into
columnar format once segment is full (1M of rows)
Updatable Columnstore Index
What’s being delivered◦ Clustered and updateable columnstore index◦ Columnstore archive option for data compression◦ Global batchaggregation◦ MUCH better Batch Mode operations, fewer limitations
Main benefits◦ Real-time super fast data warehouse engine◦ Ability to continue queries while updating without the
need to drop and recreate index or partition switching◦ Huge disk space saving due to compression◦ Better performance and more efficient (less memory)
batch query processing using batch mode rather than row mode
Columnstore Enhancements Summary
In-Memory OLTPSQL Server engine
Low latency ,Up to 10 times the improvement in performance
2 to 5 times the improvement in scalability
Takes advantage of investments in SQL Server
New high-performance, memory-optimized online transaction processing (OLTP) engine integrated into SQL Server and architected for modern hardware trends
• Integrated into SQL Server relational database• Full ACID support• Memory-optimized (no B-Tree indexes or buffer pools)• Non blocking multi-version optimistic concurrency
control (no locks or latches)• T-SQL compiled to native code
Memory-optimized table
file group
In-Memory OLTP engine: Memory-optimized tables & indexes
Native compiled SPs & schema
In-Memory OLTP compiler
TDS handler and session management
Transaction log Data file group
Buffer pool for tables & indexes
Poc/plan cache for adhoc, T-SQL; interpreter, plans, etc.
Parser, catalog,
optimizer
* All performance critical data already fits in memory* Transaction locking or physical latching causing stalls and blocking* Relatively short-lived transactions* High degree of concurrent transactions from many connections Performance problems isolated to subset of tables and SPs
bit All integer types: tinyint, smallint, int, bigint All money types: money, smallmoney All floating types: float, real date/time types: datetime, smalldatetime,
datetime2, date, time numeric and decimal types All non-LOB string types: char(n), varchar(n),
nchar(n), nvarchar(n), sysname Non-LOB binary types: binary(n), varbinary(n) Uniqueidentifier
In-Memory OLTP Limitations
No DML triggers No FOREIGN KEY or CHECK constraints No IDENTITY columns (added in CTP 2) 8060 byte hard limit for row length No UNIQUE indexes other than for the PRIMARY
KEY A maximum of 8 indexes, including the index
supporting the PRIMARY KEY No schema changes are allowed once a table is
created (drop/recreate) Indexes are always and ONLY created at as part
of the table creation
In-Memory OLTP Limitations
What’s being delivered◦ Use of non-volatile drives (SSD) to extend buffer pool◦ NUMA-Aware large page and BUF array allocation
Main benefits◦ BP extension for SSDs
Improve OLTP query performance with no application changes No risk of data loss (using clean pages only) Easy configuration optimized for OLTP workloads on commodity
servers (32 GB RAM) Scalability improvements for systems with more than
eight socketsExample:
ALTER SERVER CONFIGURATION
SET BUFFER POOL EXTENSION ON(FILENAME = 'F:\SSDCACHE\EXAMPLE.BPE‘, SIZE = 50 GB)
SSD Buffer Pool Extension
Four new permissions◦ CONNECT ANY DATABASE(server scope)◦ IMPERSONATE ANY LOGIN (server scope)◦ SELECT ALL USER SECURABLES (server scope)◦ ALTER ANY DATABASE EVEN SESSION (database scope)
Main benefit◦ Greater role separation to restrict multiple DBA roles◦ Ability to create new roles for database administrators who are not sysadmin◦ Ability to create new roles for users or apps with specific purposes
Examples:
1.Database administrator but cannot see user data2.Database administrator but cannot modify logins3.Database administrator but cannot impersonate any logins4.New roles (e.g. auditors) to read all data but not admin rights5.New roles to read all metadata for monitoring purposes (e.g. SCOM) but cannot see
user data6.New roles for middle tier app to impersonate any login except sa, sysadmin, or other
high privilege logins
Separation of Duties Enhancement
SQL 2014 Scalability
Up 640 logical processors
64 vCPUs/VM
1 TB memory/VM
64 nodes/cluster
Network virtualization provides flexibility and isolation
Assign minimum and maximum bandwidth
Storage virtualization
Enterprise-class network storage on standard hardware
Storage tiering for higher performance
SQL Server 2014• Resource Governor adds
IO governance
• SysPrep at cluster level (prepare even CLUSTER!!)
Windows Server 2012 / R2• Hyper-V
• Storage Spaces
• NIC Teaming
• Online VHDX Resize
• Live migration
• Network & Storage QoS
Pool 1
Pool 2
Pool 3
Ability to differentiate workloads Ability to monitor resource usage per group Limit controls to enable throttled execution
or prevent/minimize probability of “run-aways”
Prioritize workloads Provide predictable execution of workloads Specify resource boundaries between
workloads
Resource Governor Goals
Represents physical resources of server Can have one or more workloads
assigned to pool Pools divided into shared and non-shared Pools control min/max for CPU/memory
and now IOPS
Resource Pools
CREATE RESOURCE POOL pool_nameWITH( [ MIN_CPU_PERCENT = value ][ [ , ] MAX_CPU_PERCENT = value ] [ [ , ] CAP_CPU_PERCENT = value ] [ [ , ] AFFINITY {SCHEDULER = AUTO | (Scheduler_range_spec) | NUMANODE = (NUMA_node_range_spec)} ] [ [ , ] MIN_MEMORY_PERCENT = value ][ [ , ] MAX_MEMORY_PERCENT = value ] [ [ , ] MIN_IOPS_PER_VOLUME = value ][ [ , ] MAX_IOPS_PER_VOLUME = value ])
What’s being delivered◦ Add max/min IOPS per volume to Resource Governor
pools◦ Add DMVs and perfcounters for IO statistics per pool
per volume◦ Update SSMS Intellisense for new T-SQL◦ Update SMO and DOM for new T-SQL and objects
Main benefits◦ Better isolation (CPU, memory, and IO) for
multitenant workloads◦ Guarantee performance in private cloud and hosters
scenario
IO Resource Governance
SysPrep: Why Invest?Support SQL Server images in Azure Gallery
-Provide quick and flexible SQL Server provisioning for IaaS scenarios-Support SQL Server configuration as part of the provisioning process-Need to be faster than full installationRemove limitations that currently exist
This has been long requested by customers
VM OS Selection
CREATE VIRTUAL MACHINE
VM OS Selection
SQL Server 2012 Web Edition Service Pack 1 (64-bit) on Windows Server 2008 R2 Service Pack 2. This image contains the full version of SQL Server, including all components except Distributed Replay, Always On, and Clustering capabilities. Some SQL Server components require additional setup and configuration before use. Medium is the minimum recommended size for this image. To evaluate advanced SQL Server 2012 capabilities, Large or Extra-Large sizes are recommended.
Microsoft SQL Server 2012 Standard ...
Microsoft SQL Server 2012 Evaluatio ...
Microsoft SQL Server 2008 R2 Web E...
Microsoft SQL Server 2012 Web Editi...
Microsoft SQL Server 2008 R2 Standa...
1 2 3
SysPrep Recent Enhancements Sysprep support for
◦ Database engine◦ Reporting Services◦ Analysis Services◦ Integration Services◦ Management Tools
(SSMS)◦ Other Shared Features◦ Performance
Improvements
Delivered in SQL Server 2012 SP1 CU2
Sysprep for SQL Server Cluster What’s being delivered:
◦ Extensions to SQL Server SysPrep functionality in order to support image based deployment of clustered SQL Server instances
Main benefit:◦ Supports full automation of SQL Server Failover Cluster
deployment scenarios◦ Reduces deployment times for SQL Server Failover Clusters◦ Combined together, these features allow customers to
automate the provisioning of SQL Server Failover Clusters both on-premises and through Infrastructure as a Service (IaaS)
◦ Built on top of SQL Server 2012 SP1 CU2 Sysprep enhancements
High Availability
Multi-DB failover, manual or automatic
Up to 8 secondary replicas
100% availability during all online operations
HA without expensive SANs
Offload reads & backups to secondaries
Deployment & management wizards
Monitoring dashboard within SSMS & System Center
SQL Server 2014• Enhanced AlwaysOn
• Replica Wizard (create
replika on Windows Azure!)
• Online Ops
Windows Server 2012/R2• Clustered Shared
Volume support
• VHDX support
• RefS support
System Center 2012 R2• Manage on-premises
and cloud apps
Primary Secondary 2Running BI Workloads
Secondary 1Running Back-
ups
Synchronous data
movement
Asynch
ronous
data
movementAG_Listener
What’s being delivered:◦ Increase number of secondaries to 8 (from 4), 2
SyncReplicas◦ Increased availability of Readable Secondaries◦ Support for Windows Server 2012 Cluster Shared Volumes
(CSV)◦ Enhanced Diagnostics
Main benefits:◦ Further scale-out read workloads across, possibly geo-
distributed, replicas ◦ Use readable secondaries despite network failures
(important in geo-distributed environments)◦ Improve SAN storage utilization
Avoid drive letter limitation (max 24 drives) via CSV paths Increase resiliency of storage failover
◦ Ease troubleshooting
AlwaysOn in SQL Server 2014
What’s being delivered:◦ Single partition online index rebuild(!!)◦ Managing Lock Priority for table SWITCH and Online
Index Rebuild◦ Built-in diagnostics
Main benefits:◦ Increased concurrency and application availability◦ New partition-level granularity for online index rebuild
allows lower resource (CPU/memory) usage for customers who need 24/7 access
◦ Provide customers greater control over impacting running transactions if using switch partition or online index rebuild commands
Online Ops Enhancements
On-Prem and/or Cloud (Hybrid)
Platform for Hybrid Cloud
Hybrid Cloud SolutionsEnable new hybrid scenarios like cloud backup and cloud DR to reduce costs and improve on-premises DR
Easy On-Ramp to CloudEasily migrate and run SQL Server in the cloud to gain benefits of cloud computing
Complete & ConsistentDeploy a complete and consistent data platform from on-premises to cloud
Hybrid Cloud Solutions
Manual or Automatic
At an instance level with point in time restore
Measures DB usage patterns to set backup frequency
Fast disaster recovery (low RTO)
Easy to deploy & manage
Cloud bursting
Greater global reach
Better isolation of internal assets
On-Premise Network
Domain Controller
Periodic Snapshot (automatic/manual)
SQL Server
Restore to VM
Geo Replication
*CAPEX –(Capital Expendures -investments for IT Solutions)**OPEX -(Operational Expendures - cost for operate the IT solutions
Hybrid Cloud Solutions
Fast disaster recovery (low RTO)
Easy to deploy & manage
Cloud bursting
Greater global reach
Better isolation of internal assets
On-Premise Network
Domain Controller
VPN Tunnel (Windows Azure Virtual Network)Asynchronous Commit
Primary Replica Secondary Replica
Synchronous Commit
Run backups
Run BI reports
Manual or Automatic
At an instance level with point in time restore
Measures DB usage patterns to set backup frequency
Hybrid Cloud Solutions
Fast disaster recovery (low RTO)
Easy to deploy & manage
Cloud bursting
Greater global reach
Better isolation of internal assets
Manual or Automatic
At an instance level with point in time restore
Measures DB usage patterns to set backup frequency
On-Premise Network
Domain Controller(authenticating all users on-prem & cloud)SQL Server
Lin
ked
Serv
er
Web Server
Complete & Consistent Data Platform
Microsoft Cloud
SQL Server in a
Windows Azure VM
SQL Database
service
HDInsightTablesBlobs
SQL Server on Windows
Server
PDW Appliance
SQL Server in a
Windows Server VM
BI features in O365
SQL Server in a
Windows Server VM
3rd Party
HD Insight Server
OTHER (not a complete list!) Backup Encryption
◦ AES 128/192/256, Triple DES◦ DO NOT LOSE YOUR CERTIFICATE!◦ No appending to backup◦ This IS Standard Edition Feature
TempDB write ‘deferal’ (Bob Dorr blog post)
SQL Server Data & Log Files in Windows Azure Storage (silly?)
OTHER - BI Cra … er, Stuff :-)Self-Serve BI
◦PowerQuery, PowerPivot, PowerView, PowerMap, PowerPotty
Tabular/BISM in SSASHDInsightPDW V2, PolyBase
Kevin G. Boles◦ @TheSQLGuru◦ [email protected]
Don’t forget about the #sqlhelp hash tag Don’t forget about free downloads on
SQLSaturday.com Schedule Pages
QUESTIONS??