Temporal Probabilistic Models
description
Transcript of Temporal Probabilistic Models
![Page 1: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/1.jpg)
Temporal Probabilistic Models
![Page 2: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/2.jpg)
Temporal (Sequential) Process• A temporal process is the evolution of
system state over time• Often the system state is hidden, and we
need to reconstruct the state from the observations
• Relation to Planning:– When you are observing a temporal process,
you are observing the execution trace of someone else’s plan…
![Page 3: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/3.jpg)
State Estimation…
In non-deterministic scenarios, observation from a state had to be certain I.e., P(o|s) must be either 0 or 1. Saying “state s may give observation o doesn’t make sense since there is no way of exploiting that information.
In stochastic scenarios, P(o|s) can be a number between 0 and 1, and this makes for pretty interesting reasoning opportunities..
![Page 4: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/4.jpg)
![Page 5: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/5.jpg)
Stationarity assumption over time is akin to “relational” assumption over objects
![Page 6: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/6.jpg)
![Page 7: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/7.jpg)
Dynamic Bayes Networks are “templates” for specifying the relation between the values of a random variable across time-slices e.g. How is Rain at time t related to Rain at time t+1? We call them templates because they need to be expanded (unfolded) to the required number of time steps to reason about the connection between variables at different time points
![Page 8: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/8.jpg)
While DBNs are special cases of B.N.’s there are a certain inference tasks that are particularly frequently useful for them (Notice that all of them involve estimating posterior probability distributions—as is done in any B.N. inference)
Most inference tasks in DBNs will be about posterior distributions over single or multiple state variables.
![Page 9: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/9.jpg)
Can do much better if we exploit the repetitive structure
Both Exact and Approximate B.N. Inference methods can be made to take the temporal structure into account. Specialized variable-elimination method Unfold t+1th level, and roll-up tth level by variable elimination
Specialized Likelihood-weighting methods that take evidence into account Particle Filtering Techniques
![Page 10: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/10.jpg)
![Page 11: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/11.jpg)
![Page 12: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/12.jpg)
Can do much better if we exploit the repetitive structure
Both Exact and Approximate B.N. Inference methods can be made to take the temporal structure into account. Specialized variable-elimination method Unfold t+1th level, and roll-up tth level by variable elimination
Specialized Likelihood-weighting methods that take evidence into account Particle Filtering Techniques
![Page 13: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/13.jpg)
![Page 14: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/14.jpg)
![Page 15: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/15.jpg)
![Page 16: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/16.jpg)
![Page 17: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/17.jpg)
![Page 18: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/18.jpg)
![Page 19: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/19.jpg)
Notice that to attach the likelihood to the evidence, we are using the CPTs in the bayes net. (Model-free empirical observation, in contrast, either gives you a sample or not; we can’t get fractional samples)
![Page 20: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/20.jpg)
Normal LW takes each sample through the network one by one
Idea 1: Take then all from t to t+1 lock-step the samples are the distribution
Normal LW doesn’t do well when the evidence is downstream (the sample weight will be too small)
In DBN, none of the evidence is affecting the sampling! EVEN MORE of an issue
![Page 21: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/21.jpg)
Idea: Put samples back into high probability regions
(particle ~ sample)
![Page 22: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/22.jpg)
![Page 23: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/23.jpg)
![Page 24: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/24.jpg)
A Robot localizing itself using particle filters
![Page 25: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/25.jpg)
Factored action representations for MDPs
Based on Section 4.2of Boutilier, Hanks & Dean
![Page 26: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/26.jpg)
![Page 27: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/27.jpg)
What does IPPC use?
![Page 28: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/28.jpg)
2-TBNs model effect on each each variable separately Need “persistence” links over unaffected variables
PSO’s (probabilistic STRIPS operators)Explicitly represent complete outcomes bad when an action has multiple independent stochastic effects
Synchronic vs. diachronic dependencies Simple 2-TBN have only diachronic dependencies
![Page 29: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/29.jpg)
Special Cases of DBNs are well known in the literature
• Restrict number of variables per state– Markov Chain: DBN with one variable that is
fully observable– Hidden Markov Model: DBN with only one
state variable that is hidden and can be estimated through evidence variable(s)
• Restrict the type of CPD– Kalman Filters: DBN where the system
transition function as well as the observation variable are linear gaussian• The advantage of Gaussians is that the posterior
distribution remains Gaussian
![Page 30: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/30.jpg)
![Page 31: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/31.jpg)
![Page 32: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/32.jpg)
![Page 33: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/33.jpg)
![Page 34: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/34.jpg)
![Page 35: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/35.jpg)
![Page 36: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/36.jpg)
Class Ended here..Slides beyond this not discussed
![Page 37: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/37.jpg)
![Page 38: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/38.jpg)
![Page 39: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/39.jpg)
![Page 40: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/40.jpg)
![Page 41: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/41.jpg)
![Page 42: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/42.jpg)
![Page 43: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/43.jpg)
![Page 44: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/44.jpg)
![Page 45: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/45.jpg)
![Page 46: Temporal Probabilistic Models](https://reader035.fdocuments.net/reader035/viewer/2022062501/56816359550346895dd41375/html5/thumbnails/46.jpg)