Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here...
Transcript of Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here...
![Page 1: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/1.jpg)
Infinite Parallel Universes:State at the EdgePeter Bourgon · Fastly
![Page 2: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/2.jpg)
Infinite Parallel Universes:State at the EdgePeter Bourgon · Fastly
![Page 3: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/3.jpg)
Infinite Parallel Universes
Context Architecture Protocol Complications Conclusions
@peterbourgon · Fastly
![Page 4: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/4.jpg)
@peterbourgon
![Page 5: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/5.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
https://www.fastly.com/press/press-releases/fastly-expands-serverless-capabilities-launch-compute-edge
![Page 6: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/6.jpg)
Infinite Parallel Universes · Context
If Fastly provides compute at the edge, what about state?
@peterbourgon · Fastly
![Page 7: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/7.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
as of 31 Dec 2019 · https://www.fastly.com/network-map
![Page 8: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/8.jpg)
Infinite Parallel Universes · Context
133ms
@peterbourgon · Fastly
as of 31 Dec 2019 · https://www.fastly.com/network-map
![Page 9: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/9.jpg)
Infinite Parallel Universes · Context
What are we doing here - Not a general purpose database - Operate in the request lifecycle - Local (POP) reads and writes - Eventually consistent - State conflicts are normal
⇒ Sort of a writeable cache
@peterbourgon · Fastly
![Page 10: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/10.jpg)
Infinite Parallel Universes · Context
Data model - Lean in to the physical constraints - A single, global truth? — No, a fiction! - Multiple, simultaneous truths — Reality! - Converge toward a stable global state
@peterbourgon · Fastly
![Page 11: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/11.jpg)
Infinite Parallel Universes · Context
State primitive - CRDT - Type-specific methods + Merge - Associative, commutative, idempotent - Tolerates out-of-order, duplicate merges ⇒ Reduces higher-order complexity
@peterbourgon · Fastly
![Page 12: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/12.jpg)
Infinite Parallel Universes · Context
Integer addition - Associative? (a + b) + c = a + (b + c) ✔ - Commutative? a + b = b + a ✔ - Idempotent? a + a = a ✘
⇒ Not a CRDT
@peterbourgon · Fastly
![Page 13: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/13.jpg)
Infinite Parallel Universes · Context
Set union - Associative? ({a} ∪ {b}) ∪ {c} = {a} ∪ ({b} ∪ {c}) — Yes - Commutative? {a} ∪ {b} = {b} ∪ {a} — Yes - Idempotent? {a} ∪ {a} = {a} — Yes
⇒ A CRDT
@peterbourgon · Fastly
![Page 14: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/14.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
![Page 15: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/15.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}
![Page 16: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/16.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
![Page 17: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/17.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
![Page 18: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/18.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4}
![Page 19: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/19.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4 5 6}
![Page 20: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/20.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 3 4 5 6}
![Page 21: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/21.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4} {1 3 4 5 6}{1 2 3 4 5 6}
![Page 22: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/22.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4} {1 3 4 5 6}
{1 2 3 4 5 6}
{1 2 3 4 5 6}
![Page 23: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/23.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4}
{1 2 3 4 5 6}
![Page 24: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/24.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4}
{1 2 3 4 5 6}
![Page 25: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/25.jpg)
Infinite Parallel Universes · Context @peterbourgon · Fastly
{1}
{1} {1} {1}2
3 34
456
{1 2 3} {1 3 4} {1 4 5 6}
{1 2 3 4} {1 3 4 5 6}
{1 2 3 4 5 6}
{1 2 3 4 5 6}
Join semilattice
![Page 26: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/26.jpg)
Infinite Parallel Universes
Context Architecture Protocol Complications Conclusions
@peterbourgon · Fastly
![Page 27: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/27.jpg)
Infinite Parallel Universes · Architecture @peterbourgon · Fastly
AKL
AMSYVR
JHB
HND
EZE
as of 31 Dec 2019 · https://www.fastly.com/network-map
![Page 28: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/28.jpg)
Write primary
AKL AMS YVR···Read replica Read replica Read replica
Write
Familiar. Consistent. Slow. Doesn't satisfy local read/write criteria.
1. Single write primary
Infinite Parallel Universes · Architecture @peterbourgon · Fastly
![Page 29: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/29.jpg)
AKL AMS YVR···Replica Replica Replica
Write
2. Gossip network
Lots of connections. Where do objects live? How do objects propagate? etc.
Infinite Parallel Universes · Architecture @peterbourgon · Fastly
![Page 30: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/30.jpg)
AKL AMS YVR···Replica Replica Replica
Upstream
Write
3. Hub-and-spoke
Fewer connections. Objects live in root. Replica like LRU cache.
Infinite Parallel Universes · Architecture @peterbourgon · Fastly
![Page 31: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/31.jpg)
Infinite Parallel Universes
Context Architecture Protocol Complications Conclusions
@peterbourgon · Fastly
![Page 32: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/32.jpg)
A
B
C
Infinite Parallel Universes · Protocol
Protocol (i) - How do objects get from A to B? - Track objects that receive any request - Batch them into epochs - Regularly emit batches
@peterbourgon · Fastly
![Page 33: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/33.jpg)
A
B
C
Infinite Parallel Universes · Protocol
Protocol (ii) - How do objects get from B to C? - Bad answer: push from B to C - Key insight: sites know which objects have been requested - Better answer: pull relevant objects to C from B — sync
@peterbourgon · Fastly
![Page 34: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/34.jpg)
Site 1. Write K1=V1 2. Read K2: miss 3. Write K3=V3
Site
Interest {K1 K2 K3}
State {K1:V1
K3:V3}
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 35: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/35.jpg)
Site
Interest {K1 K2 K3}
State {K1:V1
K3:V3}
SyncRequest {K1:V1 K2:-- K3:V3}
Sync step 1 After some time, make a SyncRequest with all interesting keys and values.
+ =
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 36: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/36.jpg)
Site
SyncRequest {K1:V1 K2:-- K3:V3}
Upstream
Sync step 2 Send the SyncRequest to the upstream.
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 37: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/37.jpg)
SyncRequest {K1:V1 K2:-- K3:V3}
Upstream
State {K1:V5 K2:V6
K4:V7}
Sync step 3 Upstream merges incoming data to its own data.
New state {K1:V5⋃V1=V8 K2:V6 K3:V3 K4:V7}
⋃ =
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 38: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/38.jpg)
Site
SyncRequest {K1:V1 K2:-- K3:V3}
Upstream
SyncResponse {K1:V8 K2:V6 K3:V3}
Sync step 4 Upstream returns a SyncResponse to the site, with latest value for each requested key.
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 39: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/39.jpg)
Sync step 5 Site merges the SyncResponse to its own state, and resets the interest set.
New state {K1:V1⋃V8=V8 K2:V6 K3:V3⋃V3=V3}
⋃ =State {K1:V1 K3:V3}
SyncResponse {K1:V8 K2:V6 K3:V3}
Site
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 40: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/40.jpg)
Site Upstream
State {K1:V8 K2:V6 K3:V3 K4:V7}
Properties of sync (i) - Synchronous - The only way data moves - Bandwidth minimized - Data sets minimized - Authoritative upstream
State {K1:V8 K2:V6 K3:V3}
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 41: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/41.jpg)
Site Upstream
State {K1:V8 K2:V6 K3:V3 K4:V7}
Properties of sync (ii) - Schedule is flexible - Missed syncs impact liveness,
not correctness - Eventually consistent - Quiet by default
State {K1:V8 K2:V6 K3:V3}
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 42: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/42.jpg)
Upstream
SiteSite Site
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 43: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/43.jpg)
Upstream
Site Site
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 44: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/44.jpg)
Incoming user requests - Requests are hashed and proxied to N replicas - Response is the union of all returned CRDTs - Compare union with individual responses - Discrepant replicas easily identified, fixed
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 45: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/45.jpg)
Upstream
Site Site
Infinite Parallel Universes · Protocol
?
@peterbourgon · Fastly
![Page 46: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/46.jpg)
Outgoing sync requests - Each instance has different set of objects - But CRDTs tolerate over-merging - Instances may sync totally independently - Upstream state converges to stability
Upstream
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 47: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/47.jpg)
Infinite Parallel Universes · Protocol
? ?
??
@peterbourgon · Fastly
![Page 48: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/48.jpg)
Infinite Parallel Universes · Protocol
Incoming sync requests - User request = 1 key · Op - Sync = N keys · Merge - Otherwise identical - Choose any upstream replica
@peterbourgon · Fastly
![Page 49: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/49.jpg)
Fractal design - N-ary tree - Can be hierarchical - May insulate against
regional connectivity issues - Tradeoff between liveness
and capacity
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 50: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/50.jpg)
Smart primitives, simple systems - One operation for moving data - No coördination between - Faults need no handling besides retry
⇒ All from CRDT properties
Infinite Parallel Universes · Protocol @peterbourgon · Fastly
![Page 51: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/51.jpg)
Infinite Parallel Universes
Context Architecture Protocol Complications Conclusions
@peterbourgon · Fastly
![Page 52: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/52.jpg)
Infinite Parallel Universes · Complications @peterbourgon · Fastly
Complication: Read miss - First read of an object will likely miss - Object may "appear" later - Approach: blocking sync request to upstream - Better UX, worse latency
![Page 53: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/53.jpg)
Infinite Parallel Universes · Complications @peterbourgon · Fastly
Complication: API design - CRDTs can be nonintuitive to program with - Applications really like a single global truth - General-purpose CRDT-based state layer not obvious - Approach: narrowly-scoped APIs × time
![Page 54: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/54.jpg)
Infinite Parallel Universes · Complications @peterbourgon · Fastly
Complication: Resources - Reliability via duplication of effort — cost to bandwidth - CRDTs larger than normal data types — cost to storage - Approach: careful attention to cost-per-byte
![Page 55: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/55.jpg)
Infinite Parallel Universes · Complications @peterbourgon · Fastly
Complication: Quantum entanglement - Quantum entanglement renders all of this irrelevant - Pro: easy, instantaneous communication invariant to distance - Con: I'm probably out of a job - Approach: 🤷
![Page 56: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/56.jpg)
Infinite Parallel Universes
Context Architecture Protocol Complications Conclusions
@peterbourgon · Fastly
![Page 57: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/57.jpg)
Infinite Parallel Universes · Conclusions @peterbourgon · Fastly
State at the edge — IMO - There are inescapable constraints at large physical scale - Old abstractions (single global truth) break down - New abstractions (multiple parallel truths) necessary - Reliable systems require more robust primitives (CRDTs)
![Page 58: Infinite Parallel Universes€¦ · Infinite Parallel Universes · Context What are we doing here -Not a general purpose database -Operate in the request lifecycle -Local (POP)](https://reader036.fdocuments.net/reader036/viewer/2022071101/5fda49b9462f033a181167dd/html5/thumbnails/58.jpg)
Infinite Parallel Universes:State at the EdgePeter Bourgon · Fastly