Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2...

27
Mark Broadbent Principal Consultant SQLCloud SQLCLOUD.CO.UK l o c k, l o c k b & t w os m o k i ng barrel s

Transcript of Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2...

Page 1: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Mark Broadbent

Principal Consultant

SQLCloud

SQLCLOUD.CO.UK

lock, lockb& two smoking barrels

Page 2: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

About

Mark Broadbent.“30 billion times more intelligent than a live mattress”Microsoft Certified Master/ Certified Solutions Master: SQL Server

Microsoft Data Platform MVP

Owner/ Principal at SQLCloud

Email: [email protected]

Twitter: https://twitter.com/retracement

Blog: http://tenbulls.co.uk

Event Lead to the UK’s first ever SQLSaturday (Cambridge)

Event #4 in Sept 2016 (pre-cons & Community day)

http://sqlcambs.org.uk

Page 3: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

X X

Locking

Agenda

Theory

1

Isolation2

Optimistic Concurrency

4

3

The best part of the presentation… Gin O‘Clock

5

Page 4: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Theory(aka whatcha talkin’ ‘bout Willis)

Page 5: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

AC

ID

Page 6: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Transaction Log –the D in ACID

Log file (.ldf)

Version

FseqNo

Size

Virtual Log Files (VLFs)

Header

Record 1

Slot Array

Log fileHeader

Log Block

Record n

Record 2

Log growth size <=64MB then 4 VLFs> 64MB AND <= 1GB 8 VLFs > 1GB 16 VLFSThink about how big your log needs to be from the beginning

Log Buffer

1

Log Buffer

n

Log Writer

Signal to flush

Log writes are first written to log buffers. Flush on fill or on commit.

Delayed Durability is a compromise.

Inactive VLFS

async

Control returned to spid on commit.

Record has unique LSN LSN has 3 part name consisting of:

VLF Number + Log Block Offset + Slot Number

VLF header

512 bytes to 60KB

Page 7: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Concurrency-versus-Correctness

“ I s o l a t i o n i s a b a l a n c e b e t w e e n c o n c u r r e n c y a n d c o r r e c t n e s s ” – Me *1

1st law of Concurrency Control: Concurrent

execution should not cause application

programs to malfunction.

2nd law of Concurrency Control: Concurrent

execution should not have lower throughput or

much higher response times than serial

execution.

– Jim Gray and Andreas Reuter

*1 derived from the writings of Gray, Reuter, Bernstein et.al

Page 8: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Serial Processing

Read R1Read R2Write R3

T1

Read R1Read R2Write R3

T2

Read R4Write R4

T3

Read R4Write R4

T4

Page 9: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Transaction Interleaving

T1 Read R1T1 Read R2T1 Write R3

T2 Read R1T2 Read R2T2 Write R3

T3 Read R4T3 Write R4T4 Read R4T4 Write R3

Order 1

T2 Read R1T2 Read R2T2 Write R3

T1 Read R1T1 Read R2T1 Write R3

T4 Read R4T4 Write R3T3 Read R4T3 Write R4

Order 2

T1 Read R1T1 Read R2T2 Read R1T2 Read R2T1 Write R3T2 Write R3T3 Read R4T3 Write R4T4 Read R4T4 Write R3

Order 3

T1 Read R1T1 Read R2

T2 Read R1T2 Read R2T2 Write R3T1 Write R3T3 Read R4T3 Write R4T4 Read R4T4 Write R4

Order 4

T1 Read R1T1 Read R2T2 Read R1T2 Read R2T1 Write R3T2 Write R3T3 Read R4T4 Read R4T3 Write R4T4 Write R4

Order 5

Page 10: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Transactional Histories

T1 Read R1T1 Read R2T2 Read R1T2 Read R2T1 Write R3T2 Write R3T3 Read R4T4 Read R4T3 Write R4T4 Write R4

T1 SLOCK R1T1 Read R1T1 UNLOCK R1T1 SLOCK R2

T2 SLOCK R1T2 Read R1T1 Read R2T2 UNLOCK R1T2 SLOCK R2T2 Read R2T1 UNLOCK R2T2 UNLOCK R2T1 XLOCK R3T1 Write R3T1 UNLOCK R3

T2 XLOCK R3T2 Write R3T2 UNLOCK R3

A transaction is

considered Two-Phased

if all LOCK actions

precede an UNLOCK

action

A transaction is

considered to be Well-

Formed if all READ,

WRITE and UNLOCK

operations are covered

by a preceding LOCK.

Isolation mechanism has “serialized these transactional writes to same resource

Page 11: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Locking(aka what the blockin’ hell!)

Page 12: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Locks

• Are compatible OR incompatible with each other

• Are ONLY memory structures (lock blocks) and long blocking chains can consume a lot of memory

• Compared by lock manager across same resource or lock partition

• They can be converted or escalated

• Poor queries cause less granular locks or escalation of locks• For performance and memory savings

• Increases chances of blocking and deadlocks which cause perceived OR actual poor performance!

• Is why developers love NOLOCK

Page 13: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

orders

id status

1 0

2 2

3 1

4 0

(U)pdate lock …a special kind of lock

UPDATE ordersSET status = 1 WHERE id > 1AND id < 4

Lock Conversion

spid 115

UPDATE ordersSET status = 1 WHERE id > 1AND id < 3

spid 162

Lock will be blocked!

SEARCHPHASE

UPDATEPHASE

X

U

S

S U X

X

X

Page 14: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Isolation(aka nobody loves me)

Page 15: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

SERIALIZABLE4

NoneWRITE WRITEWRITE READREAD WRITE

LOWEST: read and write locks held to EOT

Isolation Levels

READ UNCOMMITTED1

“Bad Dependencies”Isolation Level

Dirty Read,Non-Repeatable Read, Phantoms, Lost Updates

X Dependencies

WRITE WRITE

Concurrency

GOOD: only wait on write-write locks

READ COMMITTED2

Non-Repeatable Read,Phantoms , Lost Updates

WRITE WRITEREAD WRITEWRITE READ

OK: wait on both but only writes held to EOT

REPEATABLE READ3

PhantomsWRITE WRITEREAD WRITEWRITE READ

LOW: read and write locks held to EOT

READ COMMITTED(with Snapshot Isolation)

2Non-Repeatable Read,

Phantoms , Lost UpdatesWRITE WRITE GOOD: only wait on writes held to EOT

SNAPSHOT5

None (though Causal consistency concerns, lost update prevention and other behaviours)

WRITE WRITE GOOD: only wait on writes held to EOT

Page 16: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Isolation Levels…

• Attempt to solve reduce interleaving dependency problems

• 4 Levels are defined standard by ANSI, SQL 2005+ introduced 5th

• Not all Isolation Levels can be used everywhere – FILESTREAM enabled databases support only RC, RCS and SI

• Set at session level, transaction level and statement

• Sometimes used behind the covers (such as Readable Secondaries and In-Memory OLTP)

Page 17: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Read Committed Lost Updates?

DECLARE @basketcount INT = 1DECLARE @newquantity INTBEGIN TRANSACTION

SELECT @newquantity = quantity FROM GunInventoryWITH (NOLOCK) WHERE id = 1SET @newquantity = @newquantity - @basketcount

UPDATE GunInventory SET quantity = @newquantityWHERE id = 1

COMMIT

Page 18: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

DemoPessimistic isolation

Page 19: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Optimistic Concurrency(aka sharks with frikken lasers!)

Page 20: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Row Versioning

1 2

1 4 B 9910

1 6 A 9925UPDATE… x=4COMMIT

UPDATE… x=2COMMIT

Version created on

statement execution

not transaction commit.

Pointer XSN

14 Bytes

Row

Version

Version

Row data

T1

T2

Page 21: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

• SI provides isolation at the transaction level, RCS provides isolation at the statement level

• SI implements automatic update conflict detection –don’t forget to use error handling!

• SI must be explicitly SET in each connection, for RCS it becomes the new default

• Enabling SI level requires no active transactions in order to transition. Enabling RCS requires Exclusive Transaction Workspace lock (and therefore no other connections to DB)

• RCS not allowed on master, tempdb and msdb, SI is allowed

Read Committed Snapshot vs Snapshot Isolation

READPAST & Furious: In defence of being Pessimistic · Mark Broadbent · sqlcloud.co.uk

Page 22: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Disk Based Table Optimistic Concurrency #FAIL

• Snapshot Isolation introduces • Update conflicts (not quite true – in reality it doesn’t cause and instead provides detection!)

• Causal*1 consistency behaviours and no serializable guarantees

• TempDB version-store overhead• Increased IOPS for version creation

• Tree Traversal

• Long running transactions can result in large version chains

• Page Fragmentation and 14 byte version pointer

• Writers use e(X)clusive locks• Which still block other writers

• And could still escalate

*1 At least that is what I believe it is called (i.e. causal consistency).

See “A Read-Only Transaction Anomaly Under Snapshot IsolationBy Alan Fekete, Elizabeth O'Neil, and Patrick O'Neil”

And Craig Freadman’s blog http://bit.ly/SerialVsSnapshot

Page 23: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Are you Pessimistic about being Optimistic?

• Being pessimistic means that “I expect lots of contention, I must prevent concurrent access.”

• Being optimistic means: “I do not expect resource contention, so lets not overburden the system with concurrency controls.”

Consider:

• Proper use of indexing strategy

• Keep Transactions short lived

• Avoid escalation (in the wrong situation)

• Avoid higher ISOLATION LEVELS

• Make an intelligent use of SQL Server Scalability

Page 24: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

DemoOptimistic isolation

Page 25: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Further Reading

Page 26: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

In Summary (what we have learnt today)…

• Interleaving and concurrent execution is why isolation is required• Isolation and bad dependencies determined by the ISOLATION Level

• But static read-only data does not require isolation

• RCSI is perhaps the best optimistic compromise but has overheads

• Snapshot Isolation is almost conceptually a perfect Isolation level but practically dangerous because It does not offer SERIALIZABLE protection

• Optimistic isolation is not a silver bullet for Concurrency• But in the right situation In-Memory OLTP sure can help

Page 27: Title Slide No more than 2 lines...T 1 Write R 3 T 1 UNLOCK R 3 T 2 XLOCK R 3 T 2 Write R 3 T 2 UNLOCK R 3 A transaction is considered Two-Phased if all LOCK actions precede an UNLOCK

Copyright © 2014, SQLCloud Limited. Please do not redistribute, republish in whole or in part without prior permission of content owner. www.sqlcloud.co.uk

Thank you for listening!

Email: [email protected]

Twitter: retracement

Blog: http://tenbulls.co.uk

Slideshare: http://www.slideshare.net/retracement