What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative...
Transcript of What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative...
![Page 1: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/1.jpg)
What Came First? The Ordering of Events
in Systems
@kavya719
![Page 2: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/2.jpg)
kavya
![Page 3: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/3.jpg)
the design of concurrent systems
![Page 4: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/4.jpg)
![Page 5: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/5.jpg)
Slack architecture on AWS
![Page 6: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/6.jpg)
systems with multiple independent actors.
nodes in a distributed system.
threads in a multithreaded program.
concurrent actors
![Page 7: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/7.jpg)
user-space or system threadsthreads
![Page 8: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/8.jpg)
R
W
R
W
func main() { for {
if len(tasks) > 0 { task := dequeue(tasks)process(task)
} }
}
user-space or system threadsthreadsvar tasks []Task
![Page 9: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/9.jpg)
multiple threads:// Shared variable var tasks []Task
func worker() { for len(tasks) > 0 {
task := dequeue(tasks) process(task)
} }
func main() { // Spawn fixed-pool of worker threads.startWorkers(3, worker)
// Populate task queue. for _, t := range hellaTasks { tasks = append(tasks, t) }
}
R
WR
W
g2g1
“when two+ threads concurrently access a shared memory location, at least one access is a write.”
data race
![Page 10: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/10.jpg)
…many threads provides concurrency, may introduce data races.
![Page 11: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/11.jpg)
nodes processes i.e. logical nodes(but term can also refer to machines i.e.physical nodes).
communicate by message-passing i.e.connected by unreliable network, no shared memory.
are sequential.
no global clock.
![Page 12: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/12.jpg)
distributed key-value store.three nodes with master and two replicas.
M
R R
cart: [ apple crepe, blueberry crepe ]
cart: [ ]
ADD apple crepe userX
ADD blueberry crepe userY
![Page 13: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/13.jpg)
distributed key-value store.three nodes with three equal replicas. read_quorum = write_quorum = 1. eventually consistent.
cart: [ ]
N2 N3
N1cart: [ apple crepe ]
ADD apple crepe userX
cart: [ blueberry crepe ]
ADD blueberry crepe userY
![Page 14: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/14.jpg)
…multiple nodes accepting writes provides availability,
may introduce conflicts.
![Page 15: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/15.jpg)
given we want concurrent systems, we need to deal with
data races,conflict resolution.
![Page 16: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/16.jpg)
riak: distributed
key-value store
channels: Go concurrency primitive
stepping back: similarity,
meta-lessons
![Page 17: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/17.jpg)
riak a distributed datastore
![Page 18: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/18.jpg)
riak• Distributed key-value database:// A data item = <key: blob>{“uuid1234”: {“name”:”ada”}}
• v1.0 released in 2011.Based on Amazon’s Dynamo.
• Eventually consistent:uses optimistic replication i.e. replicas can temporarily diverge, will eventually converge.
• Highly available:data partitioned and replicated, decentralized,sloppy quorum.
] AP system (CAP theorem)
![Page 19: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/19.jpg)
cart: [ ]
N2 N3
N1cart: [ apple crepe ]
cart: [ blueberry crepe ]
ADD apple crepe ADD blueberry crepe
cart: [ apple crepe ]
N2 N3
N1
cart: [ date crepe ]
UPDATE to date crepe
conflict resolution
causal updates
![Page 20: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/20.jpg)
how do we determine causal vs. concurrent
updates?
![Page 21: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/21.jpg)
{ cart : [ A ] }
N1
N2
N3
userY
{ cart : [ B ] } userX
{ cart : [ A ]} userX
{ cart : [ D ]}
A BC D
concurrent events?
A: apple B: blueberry D: date
![Page 22: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/22.jpg)
N1
N2
N3
A BC D
concurrent events?
![Page 23: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/23.jpg)
A BC D
N1
N2
N3
A, C: not concurrent — same sequential actor
![Page 24: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/24.jpg)
A BC D
N1
N2
N3
A, C: not concurrent — same sequential actor
C, D: not concurrent — fetch/ update pair
![Page 25: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/25.jpg)
happens-beforeX ≺ Y IF one of:
— same actor — are a synchronization pair — X ≺ E ≺ Y
across actors.
IF X not ≺ Y and Y not ≺ X , concurrent!
orders events
Formulated in Lamport’s Time, Clocks, and the Ordering of Events paper in 1978.
establishes causality and concurrency.
(threads or nodes)
![Page 26: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/26.jpg)
A ≺ C (same actor) C ≺ D (synchronization pair) So, A ≺ D (transitivity)
causality and concurrency
A BC D
N1
N2
N3
![Page 27: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/27.jpg)
…but B ? D D ? B So, B, D concurrent!
A BC D
N1
N2
N3
causality and concurrency
![Page 28: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/28.jpg)
A BC D
N1
N2
N3
{ cart : [ A ] } { cart : [ B ] }
{ cart : [ A ]} { cart : [ D ]}
A ≺ DD should update A
B, D concurrent
B, D need resolution
![Page 29: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/29.jpg)
how do we implement happens-before?
![Page 30: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/30.jpg)
0 0 1
0 0 0
n1 n2 n3
0 0 0 0 0 0
n1 n2 n3 n1 n2 n3
n1 n2 n3
vector clocksmeans to establish happens-before edges.
1 0 0
![Page 31: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/31.jpg)
0 0 0
n1 n2 n3
0 0 0
1 0 0
2 0 0
0 0 0
0 0 1
n1 n2 n3 n1 n2 n3
n1 n2 n3
vector clocksmeans to establish happens-before edges.
![Page 32: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/32.jpg)
0 0 0
n1 n2 n3
0 0 0
1 0 0
2 0 0
0 0 0
0 0 1
0 1 0
n1 n2 n3 n1 n2 n3
n1 n2 n3
vector clocksmeans to establish happens-before edges.
![Page 33: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/33.jpg)
0 0 0
2 1 0
n1 n2 n3
0 0 0
1 0 0
2 0 0
0 0 0
0 0 1
n1 n2 n3 n1 n2 n3
n1 n2 n3
vector clocksmeans to establish happens-before edges.
max ((2, 0, 0), (0, 1, 0))
![Page 34: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/34.jpg)
0 0 0
2 1 0
n1 n2 n3
0 0 0
1 0 0
2 0 0
0 0 0
0 0 1
n1 n2 n3 n1 n2 n3
n1 n2 n3
vector clocksmeans to establish happens-before edges.
max ((2, 0, 0), (0, 1, 0))
happens-before comparison: X ≺ Y iff VCx < VCy
![Page 35: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/35.jpg)
A BC D
N1
N2
N3
1 0 0
0 0 1
2 0 0
2 0 0
2 1 0
1 0 0
1 0 0
2 1 0
So, A ≺ D
VC at D:VC at A:
![Page 36: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/36.jpg)
A BC D
N1
N2
N3
1 0 0
0 0 1
2 0 0
2 0 0
2 1 0
1 0 0
0 0 1
2 1 0VC at D:VC at B:
So, B, D concurrent
![Page 37: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/37.jpg)
causality tracking in riak
GET, PUT operations on a key pass around a casual context object, that contains the vector clocks.
Therefore, able to detect conflicts.
a more precise form,“dotted version vector”
Riak stores a vector clock with each version of the data.
2 1 0
2 0 0
n1 n2
max ((2, 0, 0), (0, 1, 0))
![Page 38: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/38.jpg)
…what about resolving those conflicts?
causality tracking in riak
GET, PUT operations on a key pass around a casual context object, that contains the vector clocks.
a more precise form,“dotted version vector”
Riak stores a vector clock with each version of the data.
Therefore, able to detect conflicts.
![Page 39: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/39.jpg)
conflict resolution in riakBehavior is configurable.Assuming vector clock analysis enabled:
• last-write-wins i.e. version with higher timestamp picked.
• merge, iff the underlying data type is a CRDT
• return conflicting versions to application riak stores “siblings” or conflicting versions,returned to application for resolution.
![Page 40: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/40.jpg)
return conflicting versions to application:
0 0 1
2 1 0D: { cart: [ “date crepe” ] }B: { cart: [ “blueberry crepe” ] }Riak stores both versions
next op returns both to application
application must resolve conflict
{ cart: [ “blueberry crepe”, “date crepe” ] }
2 1 1
which creates a causal update { cart: [ “blueberry crepe”, “date crepe” ] }
![Page 41: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/41.jpg)
…what about resolving those conflicts?
doesn’t (default behavior).
instead, exposes happens-before graph to the application for conflict resolution.
![Page 42: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/42.jpg)
riak:
uses vector clocks
to track causality and conflicts.
exposes happens-before graph
to the user for conflict resolution.
![Page 43: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/43.jpg)
channels Go concurrency primitive
![Page 44: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/44.jpg)
R
WR
W
g2g1multiple threads:// Shared variable var tasks []Task
func worker() { for len(tasks) > 0 {
task := dequeue(tasks) process(task)
} }
func main() { // Spawn fixed-pool of worker threads.startWorkers(3, worker)
// Populate task queue. for _, t := range hellaTasks { tasks = append(tasks, t) }
}
“when two+ threads concurrently access a shared memory location, at least one access is a write.”
data race
![Page 45: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/45.jpg)
specifies when an event happens before another.
memory model
X ≺ Y IF one of:
— same thread — are a synchronization pair — X ≺ E ≺ Y
IF X not ≺ Y and Y not ≺ X , concurrent!
x = 1 print(x)
XY
unlock/ lock on a mutex, send / recv on a channel,
spawn/ first event of a thread. etc.
![Page 46: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/46.jpg)
The unit of concurrent execution: goroutines
user-space threads
use as you would threads > go handle_request(r)
Go memory model specified in terms of goroutines within a goroutine: reads + writes are ordered with multiple goroutines: shared data must be synchronized…else data races!
goroutines
![Page 47: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/47.jpg)
The synchronization primitives are:
mutexes, conditional vars, … > import “sync” > mu.Lock()
atomics > import “sync/ atomic" > atomic.AddUint64(&myInt, 1)
channels
synchronization
![Page 48: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/48.jpg)
“Do not communicate by sharing memory; instead, share memory by communicating.”
standard type in Go — chan safe for concurrent use.
mechanism for goroutines to communicate, and synchronize.
Conceptually similar to Unix pipes: > ch := make(chan int) // Initialize > go func() { ch <- 1 } () // Send > <-ch // Receive, blocks until sent.
channels
![Page 49: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/49.jpg)
// Shared variable var tasks []Task
func worker() { for len(tasks) > 0 {
task := dequeue(tasks) process(task)
} }
func main() { // Spawn fixed-pool of workers.startWorkers(3, worker)
// Populate task queue. for _, t := range hellaTasks { tasks = append(tasks, t) }
}
want:
main: * give tasks to workers.
worker: * get a task. * process it. * repeat.
![Page 50: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/50.jpg)
var taskCh = make(chan Task, n) var resultCh = make(chan Result)
func worker() { for {
// Get a task. t := <-taskCh process(t)// Send the result. resultCh <- r
} }
func main() { // Spawn fixed-pool of workers.startWorkers(3, worker)
// Populate task queue. for _, t := range hellaTasks { taskCh <- t }
// Wait for and amalgamate results. var results []Result for r := range resultCh {
results = append(results, r) }
}
![Page 51: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/51.jpg)
// Shared variable var tasks []Task
func worker() { for len(tasks) > 0 {
task := dequeue(tasks) process(task)
} }
func main() { // Spawn fixed-pool of workers.startWorkers(3, worker)
// Populate task queue. for _, t := range hellaTasks { tasks = append(tasks, t) }
}
]
]
mu
mu
] mu
…but workers can exit early.
mutex?
![Page 52: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/52.jpg)
want:
worker: * wait for task * process it * repeat
main: * send tasks
mainworker
send task wait for task
process
recv task
channel semantics(as used):
send task to happen before worker runs.
…channels allow us to express happens-before constraints.
![Page 53: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/53.jpg)
channels:
allow, and force, the user to express
happens-before constraints.
![Page 54: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/54.jpg)
stepping back…
![Page 55: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/55.jpg)
first principle:happens-before
riak: distributed
key-value store
channels: Go
concurrency primitive
surface happens-before to the user
similarities
![Page 56: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/56.jpg)
meta-lessons
![Page 57: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/57.jpg)
new technologies cleverly decompose
into old ideas
![Page 58: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/58.jpg)
the “right” boundaries for abstractions
are flexible.
![Page 59: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/59.jpg)
@kavya719
≺happens-before
riak channels
https://speakerdeck.com/kavya719/what-came-first
![Page 60: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/60.jpg)
nodes in Riak: > virtual nodes (“vnodes”) > key-space partitioning by consistent hashing,1 vnode per partition.> sequential because Erlang processes, use message queues.
replicas:> N, R, W, etc. configurable by key. > on network partition, defaults to sloppy quorum w/ hinted-handoff.
conflict-resolution: > by read-repair, active anti-entropy.
riak: a note (or two)…
![Page 61: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/61.jpg)
riak: dotted version vectorsproblem with standard vector clocks: false concurrency.userX: PUT “cart”:”A”, {} —> (1, 0); “A” userY: PUT “cart”:”B”, {} —> (2, 0); [“A”, “B”] userX: PUT “cart”:”C”, {(1, 0); “A”} —> (1, 0) !< (2, 0) —> (3, 0); [“A”, “B”, “C”]This is false concurrency; leads to “sibling explosion”.dotted version vectors
fine-grained mechanism to detect causal updates.decompose each vector clock into its set of discrete events, so: userX: PUT “cart”:”A”, {} —> (1, 0); “A” userY: PUT “cart”:”B”, {} —> (2, 0); [(1, 0)->”A”, (2, 0)->”B”] userX: PUT “cart”:”C”, {} —> (3, 0); [(2, 0)->”B”, (3, 0)->”C”]
![Page 62: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/62.jpg)
riak: CRDTsConflict-free / Convergent / Commutative Replicated Data Type
> data structure with property: replicas can be updated concurrently without coordination, and it’s mathematically possible to always resolve conflicts. > two types: op-based (commutative) and state-based (convergent). > examples: G-Set (Grow-Only Set), G-Counter, PN-Counter > Riak DT is state-based CRDTs.
![Page 63: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/63.jpg)
ch := make(chan int, 3)
channels: implementation
nil
nil
buf
sendq
recvq
lock
...
waiting senders
waiting receivers
ring buffer
mutex
hchan
![Page 64: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/64.jpg)
ch <- t1
g1
ch <- t4
ch <- t2
ch <- t3
nilnil
nil
buf
sendq
recvq
lock
g1
buf
sendq
recvq
lock
![Page 65: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/65.jpg)
ch <- t1
g1
buf
sendq
recvq
lock
g1nil
<-ch
g2
![Page 66: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/66.jpg)
buf
sendq
recvq
lock
nilnil
<-ch
g2
g1
![Page 67: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/67.jpg)
buf
sendq
recvq
lock
nilnil
<-ch
g2g1
ch <- t4 buf
sendq
recvq
lock
nilnil
![Page 68: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/68.jpg)
A
B
C
D
W
send
R
g1 g2
recv
// Shared variable var count = 0 var ch = make(chan bool, 1)
func setCount() { count++ ch <- true
}
func printCount() { <- chprint(count)
}
go setCount()go printCount()
B ≺ CSo, A ≺ D
1. send happens-before corresponding receive
![Page 69: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/69.jpg)
2. nth receive on a channel of size C happens-before n+Cth send completes.
var maxOutstanding = 3 var taskCh = make(chan int, maxOutstanding)
func worker() { for {
t := <-taskCh processAndStore(t)
} }
func main() { go worker()
tasks := generateHellaTasks() for _, t := range tasks {
taskCh <- t }
}
![Page 70: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/70.jpg)
If channel empty:receiver goroutine paused;resumed after a channel send occurs. If channel not empty:receiver gets first unreceived elementi.e. buffer is a FIFO queue.
Sends must have completed due to mutex.
1. send happens-before corresponding receive.
![Page 71: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/71.jpg)
“2nd receive happens-before 5th send.”
2. nth receive on a channel of size C happens-before n+Cth send completes.
send #3 can occur.send #4 can occur after receive #1. send #5 can occur after receive #2.
Fixed-size, circular buffer.
![Page 72: What Came First? - QCon New York · riak: CRDTs Conflict-free / Convergent / Commutative Replicated Data Type > data structure with property: replicas can be updated concurrently](https://reader034.fdocuments.net/reader034/viewer/2022042316/5f0563c97e708231d412b8b7/html5/thumbnails/72.jpg)
2. nth receive on a channel of size C happens-before n+Cth send completes.
If channel full:sender goroutine paused;resumed after a channel recv occurs. If channel not empty:receiver gets first unreceived elementi.e. buffer is a FIFO queue.
Send of that element must have completed due to channel mutex