Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · •...
Transcript of Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · •...
![Page 1: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/1.jpg)
Running ProbabilisticRunning ProbabilisticRunning Probabilistic
Programs BackwardsPrograms BackwardsPrograms Backwards
Neil Toronto * Jay McCarthy † David Van Horn *
* University of Maryland † Vassar College
ESOP 2015
2015/04/14
![Page 2: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/2.jpg)
RoadmapRoadmapRoadmap
• Probabilistic inference, and why it’s hard
1111111111111111
![Page 3: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/3.jpg)
RoadmapRoadmapRoadmap
• Probabilistic inference, and why it’s hard
• Limitations of current probabilistic programming languages(PPLs)
1111111111111111
![Page 4: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/4.jpg)
RoadmapRoadmapRoadmap
• Probabilistic inference, and why it’s hard
• Limitations of current probabilistic programming languages(PPLs)
• Contributions
1111111111111111
![Page 5: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/5.jpg)
RoadmapRoadmapRoadmap
• Probabilistic inference, and why it’s hard
• Limitations of current probabilistic programming languages(PPLs)
• Contributions
Uncomputable, compositional ways to not limit language
1111111111111111
![Page 6: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/6.jpg)
RoadmapRoadmapRoadmap
• Probabilistic inference, and why it’s hard
• Limitations of current probabilistic programming languages(PPLs)
• Contributions
Uncomputable, compositional ways to not limit language
Computable, compositional ways to not limit language
1111111111111111
![Page 7: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/7.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let ([x (flip 0.5)]) x)
2222222222222222
![Page 8: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/8.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let ([x (flip 0.5)]) x)
0.5
2222222222222222
![Page 9: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/9.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let ([x (flip 0.5)]) x)
0.5
0.5
2222222222222222
![Page 10: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/10.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let ([x (flip 0.5)][y (flip 0.5)])
(cons x y))
0.5
0.5 ⟨ , ⟩0.5 ⟨ , ⟩
2222222222222222
![Page 11: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/11.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let ([x (flip 0.5)][y (flip 0.5)])
(cons x y))
0.5 0.5
0.5 ⟨ , ⟩ ⟨ , ⟩0.5 ⟨ , ⟩ ⟨ , ⟩
2222222222222222
![Page 12: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/12.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let ([x (flip 0.5)][y (flip 0.5)])
(cons x y))
0.5 0.5
0.5 ⟨ , ⟩ ⟨ , ⟩0.5 ⟨ , ⟩ ⟨ , ⟩
2222222222222222
![Page 13: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/13.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let* ([x (flip 0.5)][y (flip (if (equal? x heads) 0.5 0.3))])
(cons x y))
0.5 0.5
0.5 ⟨ , ⟩ ⟨ , ⟩0.5 ⟨ , ⟩ ⟨ , ⟩
2222222222222222
![Page 14: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/14.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
(let* ([x (flip 0.5)][y (flip (if (equal? x heads) 0.5 0.3))])
(cons x y))
0.5 0.5
0.5 ⟨ , ⟩ ⟨ , ⟩0.5 ⟨ , ⟩ ⟨ , ⟩
0.3 0.72222222222222222
![Page 15: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/15.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
0.5 0.5
0.5 ⟨ , ⟩ ⟨ , ⟩0.5 ⟨ , ⟩ ⟨ , ⟩
0.3 0.72222222222222222
![Page 16: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/16.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
0.5 0.5
0.5 ⟨ , ⟩ ⟨ , ⟩0.5 ⟨ , ⟩ ⟨ , ⟩
0.3 0.72222222222222222
![Page 17: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/17.jpg)
Programming Coin FlipsProgramming Coin FlipsProgramming Coin Flips
0.5
0.5 ⟨ , ⟩0.5 ⟨ , ⟩
0.32222222222222222
![Page 18: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/18.jpg)
Stochastic Ray TracingStochastic Ray TracingStochastic Ray Tracing
sto·cha·stic /stō-ˈkas-tik/ adj. fancy word for "randomized"
2222222222222222
![Page 19: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/19.jpg)
Stochastic Ray TracingStochastic Ray TracingStochastic Ray Tracing
sto·cha·stic /stō-ˈkas-tik/ adj. fancy word for "randomized"
2222222222222222
![Page 20: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/20.jpg)
Stochastic Ray TracingStochastic Ray TracingStochastic Ray Tracing
ap·er·ture /ˈap-ə(r)-chər/ n. fancy word for "opening"
2222222222222222
![Page 21: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/21.jpg)
Stochastic Ray TracingStochastic Ray TracingStochastic Ray Tracing
ap·er·ture /ˈap-ə(r)-chər/ n. fancy word for "opening"
2222222222222222
![Page 22: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/22.jpg)
Stochastic Ray TracingStochastic Ray TracingStochastic Ray Tracing
Simulate projecting rays onto a sensor...
2222222222222222
![Page 23: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/23.jpg)
Stochastic Ray TracingStochastic Ray TracingStochastic Ray Tracing
... and collect them to form an image
2222222222222222
![Page 24: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/24.jpg)
Programming Stochastic Ray TracingProgramming Stochastic Ray TracingProgramming Stochastic Ray Tracing
• Normally thousands of lines of code
2222222222222222
![Page 25: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/25.jpg)
Programming Stochastic Ray TracingProgramming Stochastic Ray TracingProgramming Stochastic Ray Tracing
• Normally thousands of lines of code
• Bears little resemblance to the physical process
2222222222222222
![Page 26: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/26.jpg)
Programming Stochastic Ray TracingProgramming Stochastic Ray TracingProgramming Stochastic Ray Tracing
• Normally thousands of lines of code
• Bears little resemblance to the physical process
• In DrBayes, it’s simple physics simulation:
(define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (dot v n))])
(if (> denom 0)(let ([t (/ (+ d (dot p0 n)) denom)]) (if (> t 0)
(collision t (vec+ p0 (vec* v t)) n)#f))
#f)))
2222222222222222
![Page 27: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/27.jpg)
Programming Stochastic Ray TracingProgramming Stochastic Ray TracingProgramming Stochastic Ray Tracing
• Normally thousands of lines of code
• Bears little resemblance to the physical process
• In DrBayes, it’s simple physics simulation:
(define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (dot v n))])
(if (> denom 0)(let ([t (/ (+ d (dot p0 n)) denom)]) (if (> t 0)
(collision t (vec+ p0 (vec* v t)) n)#f))
#f)))
• Other PPLs really aren’t up to this yet
2222222222222222
![Page 28: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/28.jpg)
Programming Stochastic Ray TracingProgramming Stochastic Ray TracingProgramming Stochastic Ray Tracing
• Normally thousands of lines of code
• Bears little resemblance to the physical process
• In DrBayes, it’s simple physics simulation:
(define/drbayes (ray-plane-intersect p0 v n d) (let ([denom (- (dot v n))])
(if (> denom 0)(let ([t (/ (+ d (dot p0 n)) denom)]) (if (> t 0)
(collision t (vec+ p0 (vec* v t)) n)#f))
#f)))
• Other PPLs really aren’t up to this yet
• The issue is one of theory, not engineering effort
2222222222222222
![Page 29: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/29.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
3333333333333333
![Page 30: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/30.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (random):
xxxxxxxxx
p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
3333333333333333
![Page 31: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/31.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (random):
xxxxxxxxx
p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
3333333333333333
![Page 32: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/32.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (random):
xxxxxxxxx
p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)p(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
3333333333333333
![Page 33: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/33.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (max 0.5 (random)):
xxxxxxxxx
pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ???
3333333333333333
![Page 34: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/34.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (max 0.5 (random)):
xxxxxxxxx
pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ???
3333333333333333
![Page 35: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/35.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (max 0.5 (random)):
xxxxxxxxx
pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ??? pₘ(x) = ???
3333333333333333
![Page 36: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/36.jpg)
Simpler ExampleSimpler ExampleSimpler Example
• Assume (random) returns a value uniformly in
Density function for value of (max 0.5 (random)):
xxxxxxxxx
pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)pₘ(x
)
000000000 .25.25.25.25.25.25.25.25.25 .5.5.5.5.5.5.5.5.5 .75.75.75.75.75.75.75.75.75 111111111 1.251.251.251.251.251.251.251.251.25000000000
.25.25.25.25.25.25.25.25.25
.5.5.5.5.5.5.5.5.5
.75.75.75.75.75.75.75.75.75
111111111
1.251.251.251.251.251.251.251.251.25
pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist pₘ(x) doesn't exist
3333333333333333
![Page 37: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/37.jpg)
What Can’t Densities Model?What Can’t Densities Model?What Can’t Densities Model?
4444444444444444
![Page 38: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/38.jpg)
What Can’t Densities Model?What Can’t Densities Model?What Can’t Densities Model?
• Results of discontinuous functions (bounded measuring devices)
(let ([temperature (normal 99 1)]) (min 100 temperature))
4444444444444444
![Page 39: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/39.jpg)
What Can’t Densities Model?What Can’t Densities Model?What Can’t Densities Model?
• Results of discontinuous functions (bounded measuring devices)
(let ([temperature (normal 99 1)]) (min 100 temperature))
• Variable-dimensional things (union types)
(if test? none (just x))
4444444444444444
![Page 40: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/40.jpg)
What Can’t Densities Model?What Can’t Densities Model?What Can’t Densities Model?
• Results of discontinuous functions (bounded measuring devices)
(let ([temperature (normal 99 1)]) (min 100 temperature))
• Variable-dimensional things (union types)
(if test? none (just x))
• Infinite-dimensional things (recursion)
4444444444444444
![Page 41: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/41.jpg)
What Can’t Densities Model?What Can’t Densities Model?What Can’t Densities Model?
• Results of discontinuous functions (bounded measuring devices)
(let ([temperature (normal 99 1)]) (min 100 temperature))
• Variable-dimensional things (union types)
(if test? none (just x))
• Infinite-dimensional things (recursion)
• In general: the distributions of program values
4444444444444444
![Page 42: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/42.jpg)
Probability MeasuresProbability MeasuresProbability Measures
• Like already-integrated densities, but a primitive concept
5555555555555555
![Page 43: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/43.jpg)
Probability MeasuresProbability MeasuresProbability Measures
• Like already-integrated densities, but a primitive concept
• Measure of (random) is , defined by
5555555555555555
![Page 44: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/44.jpg)
Probability MeasuresProbability MeasuresProbability Measures
• Like already-integrated densities, but a primitive concept
• Measure of (random) is , defined by
5555555555555555
![Page 45: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/45.jpg)
Probability MeasuresProbability MeasuresProbability Measures
• Like already-integrated densities, but a primitive concept
• Measure of (random) is , defined by
• Measure of (max 0.5 (random)) defined by
5555555555555555
![Page 46: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/46.jpg)
Probability MeasuresProbability MeasuresProbability Measures
• Like already-integrated densities, but a primitive concept
• Measure of (random) is , defined by
• Measure of (max 0.5 (random)) defined by
This term assigns probability
5555555555555555
![Page 47: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/47.jpg)
Probability MeasuresProbability MeasuresProbability Measures
• Like already-integrated densities, but a primitive concept
• Measure of (random) is , defined by
• Measure of (max 0.5 (random)) defined by
This term assigns probability
• Need a way to derive measures from code
5555555555555555
![Page 48: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/48.jpg)
Probability Measures Via PreimagesProbability Measures Via PreimagesProbability Measures Via Preimages
• Interpret (max 0.5 (random)) as , defined
6666666666666666
![Page 49: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/49.jpg)
Probability Measures Via PreimagesProbability Measures Via PreimagesProbability Measures Via Preimages
• Interpret (max 0.5 (random)) as , defined
• Derive measure of (max 0.5 (random)) as
6666666666666666
![Page 50: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/50.jpg)
Probability Measures Via PreimagesProbability Measures Via PreimagesProbability Measures Via Preimages
• Interpret (max 0.5 (random)) as , defined
• Derive measure of (max 0.5 (random)) as
where
6666666666666666
![Page 51: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/51.jpg)
Probability Measures Via PreimagesProbability Measures Via PreimagesProbability Measures Via Preimages
• Interpret (max 0.5 (random)) as , defined
• Derive measure of (max 0.5 (random)) as
where
• Factored into random and deterministic parts:
6666666666666666
![Page 52: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/52.jpg)
Probability Measures Via PreimagesProbability Measures Via PreimagesProbability Measures Via Preimages
• Interpret (max 0.5 (random)) as , defined
• Derive measure of (max 0.5 (random)) as
where
• Factored into random and deterministic parts:
• In other words, compute measures of expressions by runningthem backwards
6666666666666666
![Page 53: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/53.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
7777777777777777
![Page 54: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/54.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
Standard interpretation of programs as pure functions from arandom source
7777777777777777
![Page 55: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/55.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
Standard interpretation of programs as pure functions from arandom source
Efficient way to compute preimage sets
7777777777777777
![Page 56: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/56.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
Standard interpretation of programs as pure functions from arandom source
Efficient way to compute preimage sets
Efficient representation of arbitrary sets
7777777777777777
![Page 57: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/57.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
Standard interpretation of programs as pure functions from arandom source
Efficient way to compute preimage sets
Efficient representation of arbitrary sets
Efficient way to compute areas of preimage sets
7777777777777777
![Page 58: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/58.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
Standard interpretation of programs as pure functions from arandom source
Efficient way to compute preimage sets
Efficient representation of arbitrary sets
Efficient way to compute areas of preimage sets
Proof of correctness w.r.t. standard interpretation
7777777777777777
![Page 59: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/59.jpg)
Crazy Idea is Feasible If...Crazy Idea is Feasible If...Crazy Idea is Feasible If...
• Seems like we need:
Standard interpretation of programs as pure functions from arandom source
Efficient way to compute preimage sets
Efficient representation of arbitrary sets
Efficient way to compute areas of preimage sets
Proof of correctness w.r.t. standard interpretation
• WAT
7777777777777777
![Page 60: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/60.jpg)
What About Approximating?What About Approximating?What About Approximating?
Conservative approximation with rectangles:
8888888888888888
![Page 61: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/61.jpg)
What About Approximating?What About Approximating?What About Approximating?
Conservative approximation with rectangles:
8888888888888888
![Page 62: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/62.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 63: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/63.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 64: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/64.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 65: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/65.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 66: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/66.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 67: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/67.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 68: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/68.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 69: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/69.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 70: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/70.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 71: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/71.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 72: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/72.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 73: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/73.jpg)
What About Approximating?What About Approximating?What About Approximating?
Restricting preimages to rectangular subdomains:
9999999999999999
![Page 74: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/74.jpg)
What About Approximating?What About Approximating?What About Approximating?
Sampling: exponential to quadratic (e.g. days to minutes)
10101010101010101010101010101010
![Page 75: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/75.jpg)
What About Approximating?What About Approximating?What About Approximating?
Sampling: exponential to quadratic (e.g. days to minutes)
10101010101010101010101010101010
![Page 76: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/76.jpg)
Contribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea Feasible
• Standard interpretation of programs as pure functions from arandom source
• Efficient way to compute preimage sets
• Efficient representation of arbitrary sets
• Efficient way to compute volumes of preimage sets
• Proof of correctness w.r.t. standard interpretation
11111111111111111111111111111111
![Page 77: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/77.jpg)
Contribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea Feasible
• Standard interpretation of programs as pure functions from arandom source
• Efficient way to compute abstract preimage subsets
• Efficient representation of arbitrary sets
• Efficient way to compute volumes of preimage sets
• Proof of correctness w.r.t. standard interpretation
11111111111111111111111111111111
![Page 78: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/78.jpg)
Contribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea Feasible
• Standard interpretation of programs as pure functions from arandom source
• Efficient way to compute abstract preimage subsets
• Efficient representation of abstract sets
• Efficient way to compute volumes of preimage sets
• Proof of correctness w.r.t. standard interpretation
11111111111111111111111111111111
![Page 79: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/79.jpg)
Contribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea Feasible
• Standard interpretation of programs as pure functions from arandom source
• Efficient way to compute abstract preimage subsets
• Efficient representation of abstract sets
• Efficient way to sample uniformly in preimage sets
• Proof of correctness w.r.t. standard interpretation
11111111111111111111111111111111
![Page 80: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/80.jpg)
Contribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea Feasible
• Standard interpretation of programs as pure functions from arandom source
• Efficient way to compute abstract preimage subsets
• Efficient representation of abstract sets
• Efficient way to sample uniformly in preimage sets
Efficient domain partition sampling
• Proof of correctness w.r.t. standard interpretation
11111111111111111111111111111111
![Page 81: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/81.jpg)
Contribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea FeasibleContribution: Making This Crazy Idea Feasible
• Standard interpretation of programs as pure functions from arandom source
• Efficient way to compute abstract preimage subsets
• Efficient representation of abstract sets
• Efficient way to sample uniformly in preimage sets
Efficient domain partition sampling
Efficient way to determine whether a domain sample isactually in the preimage (just use standard interpretation)
• Proof of correctness w.r.t. standard interpretation
11111111111111111111111111111111
![Page 82: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/82.jpg)
How Many Meanings?How Many Meanings?How Many Meanings?
• Start with pure programs, then lift by threading a random store
11111111111111111111111111111111
![Page 83: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/83.jpg)
How Many Meanings?How Many Meanings?How Many Meanings?
• Start with pure programs, then lift by threading a random store
• Nonrecursive, nonprobabilistic programs: , ,
11111111111111111111111111111111
![Page 84: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/84.jpg)
How Many Meanings?How Many Meanings?How Many Meanings?
• Start with pure programs, then lift by threading a random store
• Nonrecursive, nonprobabilistic programs: , ,
• Add 3 semantic functions for recursion and probabilistic choice
11111111111111111111111111111111
![Page 85: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/85.jpg)
How Many Meanings?How Many Meanings?How Many Meanings?
• Start with pure programs, then lift by threading a random store
• Nonrecursive, nonprobabilistic programs: , ,
• Add 3 semantic functions for recursion and probabilistic choice
• Full development needs 2 more to transfer theorems frommeasure theory...
11111111111111111111111111111111
![Page 86: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/86.jpg)
How Many Meanings?How Many Meanings?How Many Meanings?
• Start with pure programs, then lift by threading a random store
• Nonrecursive, nonprobabilistic programs: , ,
• Add 3 semantic functions for recursion and probabilistic choice
• Full development needs 2 more to transfer theorems frommeasure theory...
• ... oh, and 1 more to collect information for Monte Carlointegration
11111111111111111111111111111111
![Page 87: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/87.jpg)
How Many Meanings?How Many Meanings?How Many Meanings?
• Start with pure programs, then lift by threading a random store
• Nonrecursive, nonprobabilistic programs: , ,
• Add 3 semantic functions for recursion and probabilistic choice
• Full development needs 2 more to transfer theorems frommeasure theory...
• ... oh, and 1 more to collect information for Monte Carlointegration
Tally: 3+3+2+1 = 9 semantic functions, 11 or 12 rules each
11111111111111111111111111111111
![Page 88: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/88.jpg)
Enter Category TheoryEnter Category TheoryEnter Category Theory
• Moggi (1989): Introduces monads for interpreting effects
11111111111111111111111111111111
![Page 89: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/89.jpg)
Enter Category TheoryEnter Category TheoryEnter Category Theory
• Moggi (1989): Introduces monads for interpreting effects
• Other kinds of categories: idioms, arrows
11111111111111111111111111111111
![Page 90: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/90.jpg)
Enter Category TheoryEnter Category TheoryEnter Category Theory
• Moggi (1989): Introduces monads for interpreting effects
• Other kinds of categories: idioms, arrows
• Arrow defined by type constructor and thesecombinators:
11111111111111111111111111111111
![Page 91: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/91.jpg)
Enter Category TheoryEnter Category TheoryEnter Category Theory
• Moggi (1989): Introduces monads for interpreting effects
• Other kinds of categories: idioms, arrows
• Arrow defined by type constructor and thesecombinators:
• Arrows are always function-like
11111111111111111111111111111111
![Page 92: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/92.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 93: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/93.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 94: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/94.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 95: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/95.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 96: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/96.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 97: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/97.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 98: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/98.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 99: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/99.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 100: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/100.jpg)
Reducing ComplexityReducing ComplexityReducing Complexity
Function arrow: is just
11111111111111111111111111111111
![Page 101: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/101.jpg)
Pair PreimagesPair PreimagesPair Preimages
12121212121212121212121212121212
![Page 102: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/102.jpg)
Pair PreimagesPair PreimagesPair Preimages
12121212121212121212121212121212
![Page 103: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/103.jpg)
Pair PreimagesPair PreimagesPair Preimages
:
12121212121212121212121212121212
![Page 104: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/104.jpg)
Pair PreimagesPair PreimagesPair Preimages
and :
12121212121212121212121212121212
![Page 105: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/105.jpg)
Pair PreimagesPair PreimagesPair Preimages
:
12121212121212121212121212121212
![Page 106: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/106.jpg)
Correctness Theorems For Low, Low PricesCorrectness Theorems For Low, Low PricesCorrectness Theorems For Low, Low Prices
• Define
12121212121212121212121212121212
![Page 107: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/107.jpg)
Correctness Theorems For Low, Low PricesCorrectness Theorems For Low, Low PricesCorrectness Theorems For Low, Low Prices
• Define
• Derive and others so that distributes; e.g.
12121212121212121212121212121212
![Page 108: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/108.jpg)
Correctness Theorems For Low, Low PricesCorrectness Theorems For Low, Low PricesCorrectness Theorems For Low, Low Prices
• Define
• Derive and others so that distributes; e.g.
• Distributive properties makes proving this very easy:
Theorem (correctness). For all , .
12121212121212121212121212121212
![Page 109: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/109.jpg)
Correctness Theorems For Low, Low PricesCorrectness Theorems For Low, Low PricesCorrectness Theorems For Low, Low Prices
• Define
• Derive and others so that distributes; e.g.
• Distributive properties makes proving this very easy:
Theorem (correctness). For all , .
In English: computes preimages under .
12121212121212121212121212121212
![Page 110: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/110.jpg)
Correctness Theorems For Low, Low PricesCorrectness Theorems For Low, Low PricesCorrectness Theorems For Low, Low Prices
• Define
• Derive and others so that distributes; e.g.
• Distributive properties makes proving this very easy:
Theorem (correctness). For all , .
In English: computes preimages under .
• Other correctness proofs are similarly easy: prove 5 distributiveproperties
12121212121212121212121212121212
![Page 111: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/111.jpg)
Correctness Theorems For Low, Low PricesCorrectness Theorems For Low, Low PricesCorrectness Theorems For Low, Low Prices
• Define
• Derive and others so that distributes; e.g.
• Distributive properties makes proving this very easy:
Theorem (correctness). For all , .
In English: computes preimages under .
• Other correctness proofs are similarly easy: prove 5 distributiveproperties
• Can add (random) and recursion to all semantics in one shot
12121212121212121212121212121212
![Page 112: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/112.jpg)
AbstractionAbstractionAbstraction
Rectangle: An interval or union of intervals, a subset of , or for rectangles and
13131313131313131313131313131313
![Page 113: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/113.jpg)
AbstractionAbstractionAbstraction
Rectangle: An interval or union of intervals, a subset of , or for rectangles and
• Easy representation; easy intersection, join (whichoverapproximates union), empty test, etc.
13131313131313131313131313131313
![Page 114: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/114.jpg)
AbstractionAbstractionAbstraction
Rectangle: An interval or union of intervals, a subset of , or for rectangles and
• Easy representation; easy intersection, join (whichoverapproximates union), empty test, etc.
• Define (and therefore ) by replacing sets and setoperations with rectangles and rectangle operations
13131313131313131313131313131313
![Page 115: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/115.jpg)
AbstractionAbstractionAbstraction
Rectangle: An interval or union of intervals, a subset of , or for rectangles and
• Easy representation; easy intersection, join (whichoverapproximates union), empty test, etc.
• Define (and therefore ) by replacing sets and setoperations with rectangles and rectangle operations
• Recursion is somewhat tricky—requires fine control overrecursion depth or if choices
13131313131313131313131313131313
![Page 116: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/116.jpg)
In Theory...In Theory...In Theory...
Theorem (sound). computes overapproximations of thepreimages computed by .
• Consequence: Sampling in abstract preimages doesn’t leaveanything out
14141414141414141414141414141414
![Page 117: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/117.jpg)
In Theory...In Theory...In Theory...
Theorem (sound). computes overapproximations of thepreimages computed by .
• Consequence: Sampling in abstract preimages doesn’t leaveanything out
Theorem (decreasing). never returns preimages larger thanthe given subdomain.
• Consequence: Refining abstract preimage sets never results in aworse approximation
14141414141414141414141414141414
![Page 118: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/118.jpg)
In Theory...In Theory...In Theory...
Theorem (sound). computes overapproximations of thepreimages computed by .
• Consequence: Sampling in abstract preimages doesn’t leaveanything out
Theorem (decreasing). never returns preimages larger thanthe given subdomain.
• Consequence: Refining abstract preimage sets never results in aworse approximation
Theorem (monotone). is monotone.
• Consequence: Partitioning and then refining never results in aworse approximation 14141414141414141414141414141414
![Page 119: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/119.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 120: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/120.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 121: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/121.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 122: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/122.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 123: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/123.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 124: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/124.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 125: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/125.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 126: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/126.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 127: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/127.jpg)
In Practice...In Practice...In Practice...
Theorems prove this always works:
15151515151515151515151515151515
![Page 128: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/128.jpg)
Program Domain ValuesProgram Domain ValuesProgram Domain Values
16161616161616161616161616161616
![Page 129: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/129.jpg)
Program Domain ValuesProgram Domain ValuesProgram Domain Values
• Program inputs are infinite binary trees:
16161616161616161616161616161616
![Page 130: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/130.jpg)
Program Domain ValuesProgram Domain ValuesProgram Domain Values
• Program inputs are infinite binary trees:
• Every expression in a program is assigned a node
16161616161616161616161616161616
![Page 131: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/131.jpg)
Program Domain ValuesProgram Domain ValuesProgram Domain Values
• Program inputs are infinite binary trees:
• Every expression in a program is assigned a node
• Implemented using lazy trees of random values
16161616161616161616161616161616
![Page 132: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/132.jpg)
Program Domain ValuesProgram Domain ValuesProgram Domain Values
• Program inputs are infinite binary trees:
• Every expression in a program is assigned a node
• Implemented using lazy trees of random values
• No probability density for domain, but there is a measure 16161616161616161616161616161616
![Page 133: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/133.jpg)
Example: Stochastic Ray TracingExample: Stochastic Ray TracingExample: Stochastic Ray Tracing
17171717171717171717171717171717
![Page 134: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/134.jpg)
Example: Probabilistic VerificationExample: Probabilistic VerificationExample: Probabilistic Verification
(struct/drbayes float-any ())(struct/drbayes float (value error))
18181818181818181818181818181818
![Page 135: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/135.jpg)
Example: Probabilistic VerificationExample: Probabilistic VerificationExample: Probabilistic Verification
(struct/drbayes float-any ())(struct/drbayes float (value error))
(define/drbayes (flsqrt x) (if (float-any? x)
x(let ([v (float-value x)]
[e (float-error x)]) (cond [(negative? v) (float-any)]
[(zero? v) (float 0 0)][else (float (sqrt v)
(+ (- 1 (sqrt (- 1 e)))(* 1/2 epsilon)))]))))
18181818181818181818181818181818
![Page 136: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/136.jpg)
Example: Probabilistic VerificationExample: Probabilistic VerificationExample: Probabilistic Verification
(struct/drbayes float-any ())(struct/drbayes float (value error))
(define/drbayes (flsqrt x) (if (float-any? x)
x(let ([v (float-value x)]
[e (float-error x)]) (cond [(negative? v) (float-any)]
[(zero? v) (float 0 0)][else (float (sqrt v)
(+ (- 1 (sqrt (- 1 e)))(* 1/2 epsilon)))]))))
• Idea: sample e where (> (float-error e) threshold)
18181818181818181818181818181818
![Page 137: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/137.jpg)
Example: Probabilistic VerificationExample: Probabilistic VerificationExample: Probabilistic Verification
(struct/drbayes float-any ())(struct/drbayes float (value error))
(define/drbayes (flsqrt x) (if (float-any? x)
x(let ([v (float-value x)]
[e (float-error x)]) (cond [(negative? v) (float-any)]
[(zero? v) (float 0 0)][else (float (sqrt v)
(+ (- 1 (sqrt (- 1 e)))(* 1/2 epsilon)))]))))
• Idea: sample e where (> (float-error e) threshold)
• Verified flhypot, flsqrt1pm1, flsinh in Racket’s mathlibrary, as well as others
18181818181818181818181818181818
![Page 138: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/138.jpg)
Examples: Other Inference TasksExamples: Other Inference TasksExamples: Other Inference Tasks
• Typical Bayesian inference
Hierarchical models
Bayesian regression
Model selection
19191919191919191919191919191919
![Page 139: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/139.jpg)
Examples: Other Inference TasksExamples: Other Inference TasksExamples: Other Inference Tasks
• Typical Bayesian inference
Hierarchical models
Bayesian regression
Model selection
• Atypical
Programs that halt with probability < 1, or never halt
Probabilistic context-free grammars with context-sensitiveconstraints
19191919191919191919191919191919
![Page 140: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/140.jpg)
SummarySummarySummary
• Probabilistic inference is hard, so PPLs have been popping up
20202020202020202020202020202020
![Page 141: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/141.jpg)
SummarySummarySummary
• Probabilistic inference is hard, so PPLs have been popping up
• Interpreting every program requires measure theory
20202020202020202020202020202020
![Page 142: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/142.jpg)
SummarySummarySummary
• Probabilistic inference is hard, so PPLs have been popping up
• Interpreting every program requires measure theory
• Defined a semantics that computes preimages
20202020202020202020202020202020
![Page 143: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/143.jpg)
SummarySummarySummary
• Probabilistic inference is hard, so PPLs have been popping up
• Interpreting every program requires measure theory
• Defined a semantics that computes preimages
• Measuring abstract preimages or sampling in them carries outinference
20202020202020202020202020202020
![Page 144: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/144.jpg)
SummarySummarySummary
• Probabilistic inference is hard, so PPLs have been popping up
• Interpreting every program requires measure theory
• Defined a semantics that computes preimages
• Measuring abstract preimages or sampling in them carries outinference
• Can do a lot of cool stuff that’s normally inaccessible
20202020202020202020202020202020
![Page 145: Running Probabilistic Programs Backwardsntoronto/papers/toronto-2015esop-slides.p… · • Probabilistic inference, and why it’s hard • Limitations of current probabilistic programming](https://reader034.fdocuments.net/reader034/viewer/2022050313/5f75c4505e2133045c75b272/html5/thumbnails/145.jpg)
https://github.com/ntoronto/drbayes
21212121212121212121212121212121