The SNOW Theorem and Latency-Optimal Read-Only Transactions · The SNOW Theorem ! and...
49
The SNOW Theorem and Latency-Optimal Read-Only Transactions Haonan Lu ❄ , Christopher Hodsdon ❄ , Khiem Ngo ❄ , Shuai Mu † , Wyatt Lloyd ❄ ❄ University of Southern California, † New York University 1
Transcript of The SNOW Theorem and Latency-Optimal Read-Only Transactions · The SNOW Theorem ! and...
The SNOW Theorem !and Latency-Optimal !
Read-Only Transactions!Haonan Lu❄, !
Christopher Hodsdon❄, Khiem Ngo❄, !Shuai Mu†, Wyatt Lloyd❄ !
!❄University of Southern California, †New York University!
1!
Huge Web Services Shard Data!
2!
Massive amount of data!
! must be distributed across servers!
Reads dominate the workloads !– need to be as fast as possible!!
Simple Reads Are Insufficient!
3!
Storage Tier!Web!Load!Page!
ACL=public!
Photo=A!
Read !
Public!
Clients! Servers!
Datacenter!
Simple Reads Are Insufficient!
4!
Storage Tier!Web!Load!Page!
Photo=A!
Read !
Public!
Private!Photo B!
Set !“Private”!
Done!
Clients! Servers!
ACL=private!
Datacenter!
Simple Reads Are Insufficient!
5!
Storage Tier!Web!Load!Page! Read !
Public!
Private!Photo B!
Set !“Private”!
Photo B!
Done!
Done!Done!
Clients! Servers!
ACL=private!
Photo=B!
Datacenter!
Simple Reads Are Insufficient!
6!
Storage Tier!Web!Load!Page! Read !
Read!
Public!
Photo B!
Photo B!
Private!Photo B!
Done!
Clients! Servers!
ACL=private!
Photo=B!
Datacenter!
Simple Reads Are Insufficient!
7!
Storage Tier!Web!Load!Page! Read !
Read!
Public!
Photo B!
Photo B!
Private!Photo B!
Done!
Not Acceptable!!
Clients! Servers!
ACL=private!
Photo=B!
Datacenter!
Read-Only Transactions !• Transactions that do not modify data!!• Consistently read data across servers!
8!
The Power of Read-Only Txn!• Consistency restricts what can be read!– Eliminates unacceptable combinations!!
• Compatibility enables write transactions!– Write transactions atomically update data!
• Higher power ! more useful!– Stronger consistency ! higher power!– Compatibility ! higher power!
!9!
Intuitive Tension!
10!
High Power! Low Latency!
• Reduces anomalies (the ACL – Photo example)!
• Better user experience!
Our study proves:!highest power + lowest latency is
impossible!
• Easier to reason about! • Higher revenue!
Intuitive Tension!
11!
High Power! Low Latency!
• Reduces anomalies (the ACL – Photo example)!
• Better user experience!
Our study proves:!highest power + lowest latency is
impossible!
• Easier to reason about! • Higher revenue!
Intuitive Tension!
12!
High Power! Low Latency!
• Reduces anomalies (the ACL – Photo example)!
• Better user experience!
Our study proves:!highest power + lowest latency is
impossible!
• Easier to reason about! • Higher revenue!
Fundamental Tradeoff!
The SNOW Properties !
[S]trict serializability!! [N]on-blocking operations!
[O]ne response per read!! [W]rite transactions that conflict!!
13!
The SNOW Properties !
[S]trict serializability!! [W]rite transactions that conflict!
[O]ne response per read!! [N]on-blocking operations!!
14!
Highest !Power!
Lowest!Latency!
[S]trict Serializability!• Strongest model: real-time + total order!!
15!
CR!!
SACL!!
SPhoto!!
CW!!
[S]trict Serializability!• Strongest model: real-time + total order!!
16!
CR!!
SACL!!
SPhoto!!
CW!!
ACL := Private!Upload Photo B!
W starts!
W finishes!
Private
Photo B
[S]trict Serializability!• Strongest model: real-time + total order!!
17!
CR!!
SACL!!
SPhoto!!
CW!!
ACL := Private!Upload Photo B!
W starts!
W finishes!
Private
Photo B
R starts!
R finishes!
[S]trict Serializability!• Strongest model: real-time + total order!!
18!
CR!!
SACL!!
SPhoto!!
CW!!
ACL := Private!Upload Photo B!
W starts!
W finishes!
Private
Photo B
R starts!
R finishes!
“Photo B is private!”!
[S]trict Serializability!• Strongest model: real-time + total order!!!
19!
CR!!
SACL!!
SPhoto!!
CW!!
ACL := Private!Upload Photo B!
W starts!
W finishes!
Private
Photo BR starts!
R finishes!
[S]trict Serializability!• Strongest model: real-time + total order!!!
20!
CR!!
SACL!!
SPhoto!!
CW!!
ACL := Private!Upload Photo B!
W starts!
W finishes!
Private
Photo BR starts!
R finishes!
“Public + Photo A” !“Photo B is private!”!
!“Public + Photo B” !
“Photo A is private!”!!
[N]on-blocking Operations!• Do not wait on external events!– Locks, timeouts, messages, etc.!
• Lower latency!– Save the time spent blocking!
21!
[O]ne Response!• One round-trip!– No message redirection!
• Centralized components: coordinator, etc.!– No retries!– Save the time for extra round-trips!
• One value per response!– Less time for transmitting, marshaling, etc.!
22!
[W]rite Transactions That Conflict!• Compatible with write transactions !– Richer system model!– Easier to program!
23!
24!
The SNOW Theorem: !!
Impossible for read-only transaction !algorithms to have all SNOW properties !
Why SNOW Is Impossible!
25!
CW!SA! SB!CR!
Why SNOW Is Impossible!
26!
CW!SA! SB!CR!
W!visible!
RA = new!RB = old! W finishes!
W!invisible!
W starts ! A := new! B := new!!
R!
T!
Assume !SNOW! !
Why SNOW Is Impossible!
27!
CW!SA! SB!CR!
W!visible!
RA = new!RB = old! W finishes!
W!invisible!
W starts ! A := new! B := new!!
R!
T!
Assume !SNOW! !
Violates!property S
A Deeper Look at SNOW!• Complete proof in the paper!!• SNOW is tight!– Any combination of 3 properties is possible!
!• Optimality!– SNOW-optimal: have any 3 properties!– Latency-optimal: have property N and O!!
• Spectrums of property S and O!– Show what is possible to achieve!!
!!
28!
Study Existing Systems with SNOW!SNOW-optimal and latency-optimal!
29!
System! S! N! O! W!Spanner-Snap!
[OSDI’12]!✖! ✔! ✔! ✔!
Yesquel![SOSP’15]!
✖! ✔! ✔! ✔!
MySQL Cluster! ✖! ✔! ✔! ✔!
Spanner-Snap![OSDI ’12]!
Yesquel![SOSP ’15]!
MySQL Cluster!
✔!✖!
✖!
✖!
✔!
✔!
✔!
✔!
✔!
✔!
✔!
✔!
Study Existing Systems with SNOW!SNOW-optimal!
30!
System! S! N! O! W!Eiger [NSDI ’13]! ✔! ✔! ≤ 3! ✔!DrTM [SOSP ’15]! ✔! ✔! ≥ 1! ✔!RIFL [SOSP ’15]! ✔! ✔! ≥ 2! ✔!
Sinfonia [SOSP ’07]! ✔! ✔! ≥ 2! ✔!Spanner-RO
[OSDI ’12]!✔! ✖! ✔! ✔!✔! ✔! ✔!✖!
Study Existing Systems with SNOW!Candidates for Improvement!
31!
System! S! N! O! W!COPS! ✖! ✔! ≤ 2! ✖!
Rococo! ✔! ✖! > 1! ✔!!
Many more!
Improve Existing Systems!with the SNOW Theorem!
• COPS [SOSP ’11]!– Geo-replicated!– Causally consistent!– Read-only txn!!
• Rococo [OSDI ’14]!– Supports general transactions!– Strictly serializable!– Read-only txn!
!!
32!
: S N O W!
: S N O W!
New Algorithm Designs!• COPS-SNOW!– Latency-optimal (N + O)!!
• Rococo-SNOW!– SNOW-optimal (S + O + W) !
!!
33!
Design insight for optimizing reads: !shift the overhead to writes!
Rococo’s Read-Only Txn (S + W)!
34!
CR!!
SA!!
SB!!
CW!!
Rococo’s Read-Only Txn (S + W)!
35!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
Gather !conflict info!
Blocks!
R: 1st round!
R: 2nd round!
EQUAL ?!A=“old”!
B=“new”!
R: Nth round!
Rococo-SNOW (S+O+W)!
36!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
R!
Rococo-SNOW (S+O+W)!
37!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
A=“old”!
R!
Rococo-SNOW (S+O+W)!
38!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
A=“old”!
TS!R!
Rococo-SNOW (S+O+W)!
39!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
A=“old”!
TS!
TS!
R!
Rococo-SNOW (S+O+W)!
40!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
Forward TS!A=“old”!
TS!R!
TS!
Rococo-SNOW (S+O+W)!
41!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
Forward TS!A=“old”!
TS!
TS!
R!
TS!
Rococo-SNOW (S+O+W)!
42!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
Forward TS!A=“old”!
TS!
TS!
R!
TS!
Blocks!
B=“old”!
Rococo-SNOW (S+O+W)!
43!
CR!!
SA!!
SB!!
CW!! W starts!
A := “new”!B := “new”!
W commits!
W finishes!
Forward TS!
Blocks!
A=“old”!
B=“old”!
TS!
TS!
TS!
R!
A=old!B=old!
Strictly Serializable!
Evaluation of Rococo-SNOW!• To understand!– Latency of read-only transactions!– Throughput of other types of transactions!!
• Experiment configuration!– Identical to Rococo’s!– TPC-C workloads!!
• https://github.com/USC-NSL/Rococo-SNOW!!!!
44!
Significantly Lower Latency!for Read-Only Txn!
45!
0
200
400
600
800
1000
1200
0 20 40 60 80 100
Late
ncy
(ms)!
Concurrent requests/server!
OCC!
Rococo!
2PL!Rococo!-SNOW!
Significantly Lower Latency!for Read-Only Txn!
46!
0
200
400
600
800
1000
1200
0 20 40 60 80 100
Late
ncy
(ms)!
Concurrent requests/server!
Retries!
Lock Wait!
Always 1 round!
OCC!
Rococo!
2PL!Rococo!-SNOW!
Higher Throughput!under High Contention!
47!
0
1000
2000
3000
4000
5000
6000
0 20 40 60 80 100
Thro
ughp
ut (n
ew-o
rder
/s)!
Concurrent requests/server!
OCC!
Rococo!
2PL!
Rococo!-SNOW!
Higher Throughput!under High Contention!
48!
0
1000
2000
3000
4000
5000
6000
0 20 40 60 80 100
Thro
ughp
ut (n
ew-o
rder
/s)!
Concurrent requests/server!
-14% throughput!(Low Contention)!
2X throughput!(High Contention)!
OCC!
Rococo!
2PL!
Rococo!-SNOW!
Conclusion!
49!
• The SNOW Theorem for read-only txns!– Impossible to have all of the SNOW properties!
• SNOW helps understand existing systems!– Many are not yet optimal!!
• Rococo-SNOW!– SNOW Theorem guided SNOW-optimal design!– Significantly higher throughput and lower
latency under high contention !!
!
