Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace...
Transcript of Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace...
![Page 1: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/1.jpg)
Debugging Distributed Systems with
Why-Across-Time Provenance
Michael Whittaker, Cristina Teodoropol, Peter Alvaro, Joseph M. Hellerstein
![Page 2: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/2.jpg)
Reasoning about the causes of events in a
distributed system is hard
![Page 3: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/3.jpg)
Causality
![Page 4: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/4.jpg)
![Page 5: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/5.jpg)
![Page 6: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/6.jpg)
![Page 7: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/7.jpg)
![Page 8: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/8.jpg)
![Page 9: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/9.jpg)
![Page 10: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/10.jpg)
![Page 11: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/11.jpg)
Causality is general-purpose but too coarse-grained
![Page 12: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/12.jpg)
![Page 13: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/13.jpg)
Why-Provenance
![Page 14: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/14.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Instance I
r1
r2
r3
s1
s2
s3
![Page 15: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/15.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Instance I
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
r1
r2
r3
s1
s2
s3
![Page 16: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/16.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Instance I
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
x
10
600
Output Q(I)
r1
r2
r3
s1
s2
s3
![Page 17: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/17.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Instance I
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
x
10
600 Output Tuple tOutput Q(I)
r1
r2
r3
s1
s2
s3
![Page 18: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/18.jpg)
A subinstance I’ of I is a witness of t if t is in Q(I’)
![Page 19: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/19.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Subinstance I’
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
r1
r2
r3
s1
s2
s3
![Page 20: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/20.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Subinstance I’
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
x
10
Output Tuple tOutput Q(I’)
r1
r2
r3
s1
s2
s3
![Page 21: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/21.jpg)
A witness I’ of t is a minimal witness of t if no
proper subinstance of I’ is also a witness of t
![Page 22: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/22.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Subinstance I’
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
x
10
Output Tuple tOutput Q(I’)
r1
r2
r3
s1
s2
s3
![Page 23: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/23.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Subinstance I’
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
x
10
Output Tuple tOutput Q(I’)
r1
r2
r3
s1
s2
s3
![Page 24: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/24.jpg)
Why-ProvenanceR
x y
10 20
10 40
600 700
S
y z
20 30
40 50
700 800
Database Subinstance I’
SELECT x FROM R, S WHERE R.y = S.y
SQL Query Q
Output Tuple t
x
10
Output Q(I’)
r1
r2
r3
s1
s2
s3
![Page 25: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/25.jpg)
The why-provenance of t is the set of minimal
witnesses of t
![Page 26: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/26.jpg)
Why-provenance is fine-grained but not generally
applicable
![Page 27: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/27.jpg)
Causality is general-purpose but too coarse-
grained
Why-provenance is fine-grained but not generally
applicable
![Page 28: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/28.jpg)
Wat-provenance is general-purpose and fine-
grained
![Page 29: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/29.jpg)
Wat-provenance generalizes why-provenance from static
relational databases to arbitrary state machines
![Page 30: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/30.jpg)
Wat-provenance is to state machines what why-provenance is to relational databases
![Page 31: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/31.jpg)
Wat-Provenance
![Page 32: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/32.jpg)
Wat-Provenance
Database Instance I
Rx y
10 2010 40700 800
![Page 33: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/33.jpg)
Wat-Provenance
Database Instance I Query Q
Rx y
10 2010 40700 800
SELECT x FROM R, S WHERE R.y = S.y
![Page 34: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/34.jpg)
Wat-Provenance
Database Instance I Query Q Output Tuple t
Rx y
10 2010 40700 800
SELECT x FROM R, S WHERE R.y = S.y
x10700
![Page 35: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/35.jpg)
Wat-Provenance
Database Instance I Query Q Output Tuple t
State Machine M
Rx y
10 2010 40700 800
SELECT x FROM R, S WHERE R.y = S.y
x10700
![Page 36: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/36.jpg)
Wat-Provenance
Database Instance I Query Q Output Tuple t
State Machine M Input Trace T
Rx y
10 2010 40700 800
SELECT x FROM R, S WHERE R.y = S.y
x10700
a b c d
![Page 37: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/37.jpg)
Wat-Provenance
Database Instance I Query Q Output Tuple t
State Machine M Input Trace T Input i, Output o
Rx y
10 2010 40700 800
SELECT x FROM R, S WHERE R.y = S.y
x10700
a b c d get(x); 1
![Page 38: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/38.jpg)
Example 1: Key-Value Store
![Page 39: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/39.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
![Page 40: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/40.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
![Page 41: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/41.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
Output o 1
![Page 42: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/42.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
Output o 1
![Page 43: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/43.jpg)
Lessons
1.The “cause” of an output o should be a subtrace of the input that suffices to generate o. We call such a subtrace a witness of o.
![Page 44: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/44.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
Output o 1
![Page 45: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/45.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
Output o 1
![Page 46: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/46.jpg)
Lessons
1.The “cause” of an output o is a subtrace of the input that suffices to generate o. We call such a subtrace a witness of o.
2.The cause of an output o should be a “minimal” witness of o.
![Page 47: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/47.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
Output o 1
![Page 48: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/48.jpg)
Example 1: Key-Value Store
set(x,1) set(y,2)Trace T
Input i get(x)
Output o 1
![Page 49: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/49.jpg)
Example 2: Boolean Formulas
![Page 50: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/50.jpg)
Example 2: Boolean Formulas
set(a) set(b) set(c) set(d)Trace T
![Page 51: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/51.jpg)
Example 2: Boolean Formulas
set(a) set(b) set(c) set(d)Trace T
Input i eval((a and d) or (b and c))
![Page 52: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/52.jpg)
Example 2: Boolean Formulas
set(a) set(b) set(c) set(d)Trace T
Input i eval((a and d) or (b and c))
Output o true
![Page 53: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/53.jpg)
set(a) set(b) set(c) set(d)Trace T
Input i eval((a and d) or (b and c))
Output o true
Example 2: Boolean Formulas
![Page 54: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/54.jpg)
set(a) set(b) set(c) set(d)Trace T
Input i eval((a and d) or (b and c))
Output o true
Example 2: Boolean Formulas
![Page 55: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/55.jpg)
Lessons
1.The “cause” of an output o is a subtrace of the input that suffices to generate o. We call such a subtrace a witness of o.
2.The cause of an output o should be a “minimal” witness of o.
3.An output o could have multiple “minimal” witnesses.
![Page 56: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/56.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
![Page 57: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/57.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
![Page 58: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/58.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
![Page 59: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/59.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
![Page 60: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/60.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
![Page 61: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/61.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 62: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/62.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 63: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/63.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 64: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/64.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 65: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/65.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 66: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/66.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 67: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/67.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 68: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/68.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 69: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/69.jpg)
Lessons1.The “cause” of an output o is a subtrace of
the input that suffices to generate o. We call such a subtrace a witness of o.
2.The cause of an output o should be a “minimal” witness of o.
3.An output o could have multiple “minimal” witnesses.
4.If a witness is a “cause” of an output o, then all supertraces of the witness should be too.
![Page 70: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/70.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 71: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/71.jpg)
Example 3: Negation
set(a) set(b) set(c)Trace T
Input i eval((a and not b) or c)
Output o true
![Page 72: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/72.jpg)
Wat-Provenance
![Page 73: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/73.jpg)
Wat-Provenance• Given state machine M, trace T, input i,
and output o.
![Page 74: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/74.jpg)
Wat-Provenance• Given state machine M, trace T, input i,
and output o. • A witness of o is a subtrace of T that
suffices to produce o.
![Page 75: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/75.jpg)
Wat-Provenance• Given state machine M, trace T, input i,
and output o. • A witness of o is a subtrace of T that
suffices to produce o. • A witness T’ of o is closed under
supertrace in T if every supertrace of T’ in T is also a witness of o.
![Page 76: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/76.jpg)
Wat-Provenance• Given state machine M, trace T, input i,
and output o. • A witness of o is a subtrace of T that
suffices to produce o. • A witness T’ of o is closed under
supertrace in T if every supertrace of T’ in T is also a witness of o.
• The wat-provenance of o is the set of minimal witnesses of o that are closed under supertrace in T.
![Page 77: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/77.jpg)
Wat-provenance is general-purpose and fine-grained
Causality is general-purpose but too coarse-grained
Why-provenance is fine-grained but not generally applicable
![Page 78: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/78.jpg)
Computing wat-provenance
![Page 79: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/79.jpg)
Wat-provenance is general-purpose and fine-grained
Causality is general-purpose but too coarse-grained
Why-provenance is fine-grained but not generally applicable
![Page 80: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/80.jpg)
Wat-provenance is general-purpose and fine-grained but hard to compute
Causality is general-purpose but too coarse-grained and easy to compute
Why-provenance is fine-grained but not generally applicable and easy to compute
![Page 81: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/81.jpg)
Computing Causal History
![Page 82: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/82.jpg)
Computing Causal HistoryUse vector clocks
![Page 83: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/83.jpg)
Computing Why-Provenance
![Page 84: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/84.jpg)
Computing Why-ProvenanceCompute it explicitly
![Page 85: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/85.jpg)
Computing Wat-Provenance
State Machine Implementation
![Page 86: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/86.jpg)
Computing Wat-Provenance
State Machine Implementation Wat-Provenance
Infer
![Page 87: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/87.jpg)
Computing Wat-Provenance
State Machine Implementation Wat-Provenance
InferCode Analysis
![Page 88: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/88.jpg)
Computing Wat-Provenance
State Machine Implementation Wat-Provenance
InferCode Analysis
State Machine Interface
![Page 89: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/89.jpg)
Computing Wat-Provenance
State Machine Implementation Wat-Provenance
InferCode Analysis
State Machine Interface Wat-Provenance
Specify
![Page 90: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/90.jpg)
Computing Wat-Provenance
State Machine Implementation Wat-Provenance
InferCode Analysis
State Machine Interface Wat-ProvenanceWat-Provenance Spec
Specify
![Page 91: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/91.jpg)
Example: Key-Value Store
set(d,1) set(a,3) set(e,1) set(f,4)
Input i = get(b)
Output o = 1
Trace T
![Page 92: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/92.jpg)
Example: Key-Value Store
set(b,2) set(a,1) set(c,2) set(b,1) set(a,3) set(d,1) set(a,0)
Input i = get(b)
Output o = 1
Trace T
![Page 93: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/93.jpg)
Example: Key-Value Store
set(b,2) set(a,1) set(c,2) set(b,1) set(a,3) set(d,1) set(a,0)
Input i = get(b)
Output o = 1
Trace T
English wat-provenance specification: The wat-provenance of a get of key k is
the most recent set to k.
![Page 94: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/94.jpg)
Example: Key-Value Store
def get_prov(T: List[Request], i: GetRequest): for a in reversed(T): if (isinstance(a, SetRequest) and a.key == i.key): return {[a]} return {[]}
Python wat-provenance specification:
English wat-provenance specification: The wat-provenance of a get of key k is
the most recent set to k.
![Page 95: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/95.jpg)
Wat-Provenance SpecificationsSimple wat-provenance specifications are
not uncommon:
•Key-Value Stores•Object Stores•Distributed File Systems•Coordination Services•Load Balancers•Stateless Services
![Page 96: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/96.jpg)
What is wat-provenance?
How do you compute wat-provenance?
How do you use wat-provenance?
What are the limitations of wat-provenace?
Come to my poster!
![Page 97: Debugging Distributed Systems with Why-Across-Time Provenance · 2018. 11. 13. · under supertrace in T. Wat-provenance is general-purpose and fine-grained Causality is general-purpose](https://reader036.fdocuments.net/reader036/viewer/2022071403/60f73de1a1adc460886c0c95/html5/thumbnails/97.jpg)
Thank you!