Ms Tech Series - SQL Server 2014 Nuevas Capacidades
-
Upload
christian-marcelo-astete-zuleta -
Category
Documents
-
view
44 -
download
13
Transcript of Ms Tech Series - SQL Server 2014 Nuevas Capacidades
Nuevas Capacidades SQL Server 2014 Álvaro OlivaresConsultorMicrosoft Consulting Services
SQL Server 2014 and the Data Platform
Mission Critical Performance
In-Memory Built-InAverage 10x faster for new and existing SQL Server apps
Secure & ScalableMost secure with enterprise scale using Windows Server
High AvailabilityThe 9’s you need, with AlwaysOn
Mission Critical SupportLive support designed for mission critical solutions
In-Memory Built-In
In-Memory Built-In
New In-Memory OLTP
Enhanced In-Memory ColumnStore for DW
In-Memory BI with PowerPivot
Buffer Pool Extension to SSDs & Enhanced Query Processing
Before
15,000 requests/se
c
16x FasterWith In-Memory
250,000 requests/se
c
SQL Server 2014 with In-Memory
Bwin Game Application
Bwin Game Application
Select only highly utilized tables to be in-memory
Optimize in-memory to fit existing hardware
In-memory performance across OLTP, DW and BI
All in a single SKU
On average 10x faster, without having rewrite entire app
Leverage full SQL Server capabilities
Microsoft’s In-Memory TechnologiesAnalytics OLTPStreaming Data
• In-Memory Analytics in PowerPivot for Excel
• In-Memory Analytics in SQL Server Analysis Services
• In-Memory ColumnStore in SQL Server 2012, Fast Track Data Warehouse, and Parallel Data Warehouse
• StreamInsight • In-Memory OLTP• In-Memory ColumnStore
(in SQL Server 2014)
2014
2012
2014
2012
2014
Main-Memory Optimized
• Optimized for in-memory data
• Indexes (hash and range) exist only in memory
• No buffer pool• Stream-based
storage for durability
Steadily declining memory price,
NVRAM
Hardware trends
Table ConstructsFixed schema – no ALTER TABLE, must drop/recreate/reloadNo LOB datatypes; row size limited 8060No constraints support (PK only)No Identity or Calculated columns, CLR etc…
Data and table size considerations
Size of tables = (row size * # of rows) Size of hash index = (bucket_count * 8-bytes) Max size SCHEMA_AND_DATA = 512 GB
IO for Durability SCHEMA_ONLY vs. SCHEMA_AND_DATA
Memory Optimized Filegroup Data and Delta files Transaction Log Database Recovery
Memory Optimized Tables – Design Considerations
7
High performance
data operationsBen
efi
tsIn
-Mem
ory
OLT
P
Tech
Pilla
rsD
rivers
Memory Optimized Table - Migration
Create Table DDLCREATE TABLE [Customer]( [CustomerID] INT NOT NULL
PRIMARY KEY NONCLUSTERED HASH WITH (BUCKET_COUNT = 1000000), [Name] NVARCHAR(250) NOT NULL
INDEX [IName] HASH WITH (BUCKET_COUNT = 1000000), [CustomerSince] DATETIME NULL)WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA);
This table is memory
optimized
This table is durable
Secondary Indexes are
specified inline
Hash Index
Table Creation ProcessCREATE TABLE DDL
Table code generated
Compiler invoked
Table DLL produced
Table DLL loaded
High Concurrency• Multi-version
optimistic concurrency control with full ACID support
• Core engine uses lock-free algorithms
• No lock manager, latches or spinlocksMany-core
processors
Hardware Trends
11
Frictionless scale-up
Ben
efi
tsIn
-Mem
ory
OLT
P Te
ch
P
illa
rsD
rivers
Impact of no locks or latchesWrite-Write Conflict: Design Application for condition with try.catchApplications dependent on locking: May not be a good candidate
Can take an application lock
Multiple versions of recordsIncreases the memory needed by memory optimized tablesGarbage Collection used to reclaim old versions
Transaction scopeSupport for Isolation Levels: Snapshot, Repeatable Read, SerializableCommit time validation – again have retry logic to deal with failure
High Concurrency – Design Considerations
In-Memory OLTP Concurrency Control
Multi-version data store
Snapshot-based transaction isolation
No TempDB
Conflict detection to ensure isolation
No deadlocks
No locks, no latches, minimal context switches
No blocking
Multi-version
Optimistic
Example: Write Conflict
Time Transaction T1 (SNAPSHOT) Transaction T2 (SNAPSHOT)
1 BEGIN
2 BEGIN
3 UPDATE t SET c1=‘bla’ WHERE c2=123
4 UPDATE t SET c1=‘bla’ WHERE c2=123 (write conflict)
First writer wins
T-SQL compiled to machine code
• T-SQL compiled to machine code via C code generator and VC
• Invoking a procedure is just a DLL entry-point
• Aggressive optimizations @ compile-timeStalling CPU clock
rate
Hardware trends
Native Compiled Stored Procedures – Design Considerations
14
Efficient, business-logic processingB
en
efi
tsIn
-Mem
ory
OLT
P
Tech
Pilla
rsD
rivers
Native compiled stored procedures
Non-native compilation
Performance High. Significantly less instructions to go through
No different than T-SQL calls in SQL Server today
Migration strategy Application changes – development overhead
Easier app migration as can still access Memory Optimized (MO) tables
Access to objects Can only interact with Memory Optimized tables
All Objects. Access for transactions across MO and b-tree tables
Support for T-SQL constructs
Limited. T-SQL surface area (limit on MO interaction)
Optimization/stats and query plan
Statistics utilized at CREATE -> Compile time
Statistics updates can be utilized to modify plan at runtime
Flexibility Limited (e.g., no ALTER procedure, compile-time isolation level)
Ad-hoc query patterns
Native Compiled Stored Proc - Migration
Accessing Memory Optimized Tables • Natively compiled stored
procedures• Access only memory optimized
tables• Maximum performance• Limited T-SQL surface area
• When to use• OLTP-style operations• Optimize performance critical
business logic
• Interpreted T-SQL access• Access both memory- and
disk-based tables • Less performant• Virtually full T-SQL surface
area• When to use• Ad hoc queries• Reporting-style queries• Speeding up app migration
In-Memory
OLTP Compiler In-Memory
OLTP Component
Memory-optimized Table Filegroup
Data Filegroup
SQL Server.exe
In-Memory OLTP Engine for Memory_optimized Tables & Indexes
TDS Handler and Session Management
Performance Gains
Natively Compiled SPs and Schema
Buffer Pool for Tables & Indexes
Proc/Plan cache for ad-hoc T-SQL and SPs
Client App
Transaction Log
Query Interop
Interpreter for TSQL, query plans, expressions
Access Methods
Parser, Catalog,
Algebrizer, Optimizer
10-30x more efficient
Reduced log bandwidth &
contention. Log latency remains
Checkpoints are background
sequential IO
No improvements in communication
stack, parameter passing, result set
generation
Key
Existing SQL Component
Generated .dll
SQL Server engine
In-Memory OLTP
• Low latency
• Up to 10 times the improvement in performance
• 2 to 5 times the improvement in scalability
• Takes advantage of investments in Microsoft SQL Server
New high-performance, memory-optimized online transaction processing (OLTP) engine integrated into SQL Server and architected for modern hardware trends
• Integrated into SQL Server relational database• Full ACID support• Memory-optimized (no B-Tree indexes or buffer pools)• Non blocking multi-version optimistic concurrency
control (no locks or latches)• T-SQL compiled to native code
Memory-optimized table
file group
In-Memory OLTP engine: Memory-optimized tables & indexes
Native compiled SPs & schema
In-Memory OLTP compiler
TDS handler and session management
Transaction log Data file group
Buffer pool for tables & indexes
Poc/plan cache for adhoc, T-SQL; interpreter, plans, etc.
Parser, catalog, optimize
r
• What’s being delivered:• High performance, memory-optimized OLTP engine integrated into
SQL Server and architected for modern hardware trends • Main benefits• Optimized for in-memory data up to 20-30X throughput• Indexes (hash and range) exist only in memory. No buffer pool, B-
trees• T-SQL compiled to machine code via C code generator and VC• Core engine uses lock-free algorithms. No lock manager, latches
or spinlocks• Multi-version optimistic concurrency control with full ACID support• On ramp existing applications• Integrated experience• Same manageability, administration & development experience
In-Memory OLTP Summary
In-Memory ColumnStore
• In-Memory ColumnStore
• Both memory and disk
• Built-in to core RDBMS engine
Customer Benefits:
• 10-100x faster
• Reduced design effort
• Work on customers’ existing hardware
• Easy upgrade; Easy deployment
Data Stored Row-Wise: Heaps, b-Trees, Key-Value
C1
C2
C3
C5
C6C4
Existing Tables (Partitions)
Needed columns
Columnstore Index Representation
In-memory In the Data Warehouse
21
“By using SQL Server 2012 In-Memory ColumnStore, we were able to extract about 100 million records in 2 or 3 seconds versus the 30 minutes required previously. “- Atsuo Nakajima Asst Director, Bank of
Nagoya
Traditional Storage ModelsData stored row-wise: Heaps, b-trees, key-value
• Relational, dimensional, map reduce
…
In-Memory ColumnStore Storage ModelData stored Column-wise
• Each page stores data from a single column
• Highly compressed• More data fits in memory
• Each column can be accessed independently• Fetch only columns needed• Can dramatically decrease I/O
C1 C2 C3 C5 C6C4
In-Memory ColumnStore index structureRow Groups & Segments
• A segment contains values for one column for a set of rows
• Segments for the same set of rows comprise a row group
• Segments are compressed• Each segment stored in a
separate LOB• Segment is unit of transfer
between disk and memory
Segments
C1 C2
C3 C5 C6C4
Row group
In-Memory ColumnStore IndexProcessing an Example
Horizontally PartitionRow Groups
Vertical PartitionSegments
Compress Each Segment*Some Compress More than Others
*Encoding and reordering not shown
Fetch Only Needed ColumnsSegment Elimination
Fetch Only Needed SegmentsSegment Elimination
Batch ModeImproving CPU Utilization
• Biggest advancement in query processing in years!
• Data moves in batch through query plan operators• Minimizes instructions per row• Takes advantage of cache structures
• Highly efficient algorithms• Better parallelism
• Process ~1000 rows at a time• Batch stored in vector form• Optimized to fit in cache• Vector operators implemented• Filter, hash join, hash aggregation, …• Greatly reduced CPU time (7 to 40X)
Batch Mode ProcessingQP Vector Operators
bit
map o
f qu
alif
yin
g r
ow
s
Column vectors
Batch object
• Fast execution for data warehouse queries• Speedups of 10x and more
• No need for separate base table• Save space
• Data can be inserted, updated or deleted• Simpler management
• Eliminate need for other indexes• Save space and simpler management
• More data types supported• Removes many limitations from non-clustered Columnstores in SQL Server 2012
In-Memory ColumnStores: Clustered & Updatable
Comparing Space Savings101 Million Row Table + Index Space
19.7GB
10.9GB
5.0GB4.0GB
6.9GB
1.8GB
Structure of In-Memory ColumnStoreHow It Works• CREATE CLUSTERED COLUMNSTORE
• Organizes and compresses data into ColumnStore
• BULK INSERT• Creates new ColumnStore Row Groups
• INSERT• Rows are placed in the Row Store (heap)• When Row Store is big enough, a new ColumnStore Row Group is created
• DELETE• Rows are marked in the Deleted Bitmap
• UPDATE• Delete plus insert
• Most data is in ColumnStore format
ColumnStore
DeletedBitmap
Row Store
Partition
Batch Mode ProcessingWhat’s New?
• SQL Server 2014• Support for all flavors of JOINs• OUTER JOIN• Semi-join: IN, NOT IN
• UNION ALL• Scalar aggregates• Mixed mode plans• Improvements in bitmaps, spill support, …
Archival CompressionWhat’s New?
• Adds an additional layer of compression on top of the inherent compression used by columnstore• Shrink on-disk database sizes by up to 27%• Compression applies per partition and can be set either during index
creation or during rebuild
Insert & Updating DataLoad Sizes
• Bulk insert• Creates row groups of 1Million rows, last row group is probably not full• But if <100K rows, will be left in Row Store
• Insert/Update• Collects rows in Row Store
• Tuple Mover• When Row Store reaches 1Million rows, convert to a ColumnStore Row
Group• Runs every 5 minutes by default• Started explicitly by ALTER INDEX <name> ON <table> REORGANIZE
ColumnStore Enhancements Summary• What’s being delivered:
• Clustered & Updateable In-Memory Columnstore• ColumnStore archive option for data compression• Global batch aggregation
• Main benefits:• Real-time super-fast data warehouse engine• Ability to continue queries while updating without the need to
drop and re-create index or partition switching• Huge disk space saving due to compression• Ability to compress data 5X – 15X using archival per-partition
compression• Better performance and more efficient (less memory) batch query
processing using batch mode rather than row mode
SSD Bufferpool Extension
SSD Buffer Pool Extension and Scale up• What’s being delivered:• Usage of non-volatile drives (SSD) to extend buffer pool• NUMA aware Large page and BUF array allocation
• Main benefits:• BP Extension for SSDs
• Improve OLTP query performance with no application changes• No risk of data loss (using clean pages only)• Easy configuration optimized for OLTP workloads on commodity
servers (32GB RAM)
• Scalability improvements for systems with >8 sockets
Example:ALTER SERVER CONFIGURATIONSET BUFFER POOL EXTENSION ON(FILENAME = 'F:\SSDCACHE\EXAMPLE.BPE‘, SIZE = 50 GB)
Buffer Pool Manager
DataFiles
Relational Engine
ProtocolLayer
Storage Engine Buffer Pool
Optimizer
CmdParser
QueryExecu
torSNI
Transaction
Manager
AccessMethods
BufferManage
r
Transaction Log
Plan Cache
Data Cache
TDS
Query Plan Result Sets
Resu
ltsData
GetPage D
TDS
ReadI/O
WriteI/O
CommandQuery Tree
Cached Pages
IOPS Offload to Storage Class Memory (SCM) in Memory Hierarchy
RANDOM IOPS to Mechanical Drives IOPS Offload to SCM
Consultas
Complete and Consistent Data Platform
1CONSISTENTPLATFORM
SQL Server 2014Mission Critical Performance
Faster Insights from Any Data
Platform for Hybrid Cloud
•www.yammer.com/serversmschile
• Nuestra comunidad de It Pros & Desarrolladores Microsoft
Chile
Fecha Hora TemasJueves 8 Agosto 18:30-20:30 Nuevas capacidades de SQL Server 2014
Jueves 22 Agosoto 18:30-20:30Azure SQL Reporting: Escenarios de uso para ambientes de BI 100% Cloud.
Martes 27 Agosto 18:30-20:30Arquitectura de integración de aplicaciones hibridas con Azure Biztalk Services
Martes 3 Sept. 18:30-20:30Windows Server 2012 R2: Revisión técnica del nuevo release.
Jueves 5 Sept. 18:30-20:30Performance, xVelocity, Indexing - SQL Server 2012 - Best Practices
Martes 10 Sept. 18:30-20:30Monitoreo de Servicios en la nube con System Center Operations Manager
Jueves 26 Sept. 18:30-20:30Redes en Windows Azure: VPN, Traffic Manager, etc... para qué sirve cada uno y cómo se consume.
Marte 1 Octubre. 18:30-20:30Remote Desktop Services: Escenarios de uso para ambientes de escritorio centralizados/virtualizados.
Tech Series