19 cv mil_temporal_models
-
Upload
zukun -
Category
Technology
-
view
259 -
download
0
Transcript of 19 cv mil_temporal_models
![Page 1: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/1.jpg)
Computer vision: models, learning and inference
Chapter 19 Temporal models
Please send errata to [email protected]
![Page 2: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/2.jpg)
Goal
To track object state from frame to frame in a video
Difficulties:
• Clutter (data association)• One image may not be enough to fully define state• Relationship between frames may be complicated
![Page 3: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/3.jpg)
3
Structure
3Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Temporal models• Kalman filter• Extended Kalman filter• Unscented Kalman filter• Particle filters• Applications
![Page 4: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/4.jpg)
4
Temporal Models
• Consider an evolving system• Represented by an unknown vector, w• This is termed the state• Examples:– 2D Position of tracked object in image– 3D Pose of tracked object in world– Joint positions of articulated model
• OUR GOAL: To compute the marginal posterior distribution over w at time t.
4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 5: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/5.jpg)
5
Estimating State
Two contributions to estimating the state:1. A set of measurements xt, which provide
information about the state wt at time t. This is a generative model: the measurements are derived from the state using a known probability relation Pr(xt|w1…wT)
2. A time series model, which says something about the expected way that the system will evolve e.g., Pr(wt|w1...wt-1,wt+1…wT)
5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 6: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/6.jpg)
• Only the immediate past matters (Markov)
– the probability of the state at time t is conditionally independent of states at times 1...t-2 given the state at time t-1.
• Measurements depend on only the current state
– the likelihood of the measurements at time t is conditionally independent of all of the other measurements and the states at times 1...t-1, t+1..t given the state at time t.
Assumptions
6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 7: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/7.jpg)
Graphical Model
World states
Measurements
7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 8: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/8.jpg)
Recursive EstimationTime 1
Time 2
Time tfrom
temporal model
8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 9: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/9.jpg)
Computing the prior (time evolution)
Each time, the prior is based on the Chapman-Kolmogorov equation
Prior at time t Temporal model Posterior at time t-1
9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 10: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/10.jpg)
Summary
Temporal Evolution
Measurement Update
Alternate between:
Temporal model
Measurement model
10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 11: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/11.jpg)
11
Structure
11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Temporal models• Kalman filter• Extended Kalman filter• Unscented Kalman filter• Particle filters• Applications
![Page 12: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/12.jpg)
Kalman FilterThe Kalman filter is just a special case of this type of recursive estimation procedure.
Temporal model and measurement model carefully chosen so that if the posterior at time t-1 was Gaussian then the
• prior at time t will be Gaussian• posterior at time t will be Gaussian
The Kalman filter equations are rules for updating the means and covariances of these Gaussians
12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 13: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/13.jpg)
The Kalman Filter
Previous time step Prediction
Measurement likelihood Combination13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 14: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/14.jpg)
Kalman Filter Definition
Time evolution equation
Measurement equationState transition matrix Additive Gaussian noise
Additive Gaussian noiseRelates state and measurement
14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 15: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/15.jpg)
Kalman Filter Definition
Time evolution equation
Measurment equationState transition matrix Additive Gaussian noise
Additive Gaussian noiseRelates state and measurement
15Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 16: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/16.jpg)
Temporal evolution
16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 17: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/17.jpg)
Measurement incorporation
17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 18: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/18.jpg)
Kalman Filter
This is not the usual way these equations are presented.
Part of the reason for this is the size of the inverses: f is usually landscape and so fTf is inefficient
Define Kalman gain:
18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 19: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/19.jpg)
Mean Term
Using Matrix inversion relations:
19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 20: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/20.jpg)
Covariance TermKalman Filter
Using Matrix inversion relations:
20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 21: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/21.jpg)
Final Kalman Filter Equation
Innovation (difference betweenactual and predicted measurements
Prior variance minus a term due to information from measurement
21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 22: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/22.jpg)
Kalman Filter SummaryTime evolution equation
Measurement equation
Inference
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 23: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/23.jpg)
Kalman Filter Example 1
23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 24: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/24.jpg)
Kalman Filter Example 2
Alternates:
24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 25: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/25.jpg)
25
Smoothing• Estimates depend only on measurements up to the current point in time.• Sometimes want to estimate state based on future measurements as well
Fixed Lag Smoother:
This is an on-line scheme in which the optimal estimate for a state at time t -t is calculated based on measurements up to time t, where t is the time lag. i.e. we wish to calculate Pr(wt-t |x1 . . .xt ).
Fixed Interval Smoother:
We have a fixed time interval of measurements and want to calculate the optimal state estimate based on all of these measurements. In other words, instead of calculating Pr(wt |x1 . . .xt ) we now estimate Pr(wt |x1 . . .xT) where T is the total length of the interval.
25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 26: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/26.jpg)
26
Fixed lag smoother
26Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
State evolution equation
Measurement equation
Estimate delayed by t
![Page 27: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/27.jpg)
Fixed-lag Kalman Smoothing
27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 28: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/28.jpg)
Fixed interval smoothing
28Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Backward set of recursions
where
Equivalent to belief propagation / forward-backward algorithm
![Page 29: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/29.jpg)
Temporal Models
29Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 30: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/30.jpg)
Problems with the Kalman filter
• Requires linear temporal and measurement equations
• Represents result as a normal distribution: what if the posterior is genuinely multi-modal?
30Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 31: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/31.jpg)
31
Structure
31Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Temporal models• Kalman filter• Extended Kalman filter• Unscented Kalman filter• Particle filters• Applications
![Page 32: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/32.jpg)
Roadmap
32Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 33: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/33.jpg)
Extended Kalman FilterAllows non-linear measurement and temporal equations
Key idea: take Taylor expansion and treat as locally linear
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 34: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/34.jpg)
JacobiansBased on Jacobians matrices of derivatives
34Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 35: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/35.jpg)
Extended Kalman Filter Equations
35Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 36: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/36.jpg)
Extended Kalman Filter
36Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 37: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/37.jpg)
Problems with EKF
37Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 38: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/38.jpg)
38
Structure
38Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Temporal models• Kalman filter• Extended Kalman filter• Unscented Kalman filter• Particle filters• Applications
![Page 39: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/39.jpg)
Unscented Kalman Filter
Key ideas:
• Approximate distribution as a sum of weighted particles with correct mean and covariance
• Pass particles through non-linear function of the form
• Compute mean and covariance of transformed variables
39Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 40: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/40.jpg)
Unscented Kalman Filter
Choose so that
40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Approximate with particles:
![Page 41: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/41.jpg)
One possible scheme
With:
41Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 42: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/42.jpg)
Reconstitution
42Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 43: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/43.jpg)
Unscented Kalman Filter
43Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 44: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/44.jpg)
Measurement incorportation
44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Measurement incorporation works in a similar way:Approximate predicted distribution by set of particles
Particles chosen so that mean and covariance the same
![Page 45: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/45.jpg)
Measurement incorportation
45Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Measurement update equations:
Kalman gain now computed from particles:
Pass particles through measurement equationand recompute mean and variance:
![Page 46: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/46.jpg)
Problems with UKF
46Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 47: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/47.jpg)
47
Structure
47Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Temporal models• Kalman filter• Extended Kalman filter• Unscented Kalman filter• Particle filters• Applications
![Page 48: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/48.jpg)
Particle filters
48Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Key idea:
• Represent probability distribution as a set of weighted particles
Advantages and disadvantages:
+ Can represent non-Gaussian multimodal densities+ No need for data association- Expensive
![Page 49: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/49.jpg)
Condensation Algorithm
49Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Stage 1: Resample from weighted particles according to their weight to get unweighted particles
![Page 50: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/50.jpg)
Condensation Algorithm
50Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Stage 2: Pass unweighted samples through temporal model and add noise
![Page 51: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/51.jpg)
Condensation Algorithm
51Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Stage 3: Weight samples by measurement density
![Page 52: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/52.jpg)
Data Association
52Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 53: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/53.jpg)
53
Structure
53Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Temporal models• Kalman filter• Extended Kalman filter• Unscented Kalman filter• Particle filters• Applications
![Page 54: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/54.jpg)
54
Tracking pedestrians
54Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
![Page 55: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/55.jpg)
5555Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Tracking contour in clutter
![Page 56: 19 cv mil_temporal_models](https://reader036.fdocuments.net/reader036/viewer/2022081518/554a4fe2b4c905572f8b47ca/html5/thumbnails/56.jpg)
56
Simultaneous localization and mapping
56Computer vision: models, learning and inference. ©2011 Simon J.D. Prince