Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D....
-
date post
20-Jan-2016 -
Category
Documents
-
view
218 -
download
0
Transcript of Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D....
![Page 1: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/1.jpg)
ComputerVision
EM and model selection
Marc PollefeysCOMP 256
Some slides and illustrations from D. Forsyth, …
![Page 2: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/2.jpg)
ComputerVision
2
Jan 16/18 - Introduction
Jan 23/25 Cameras Radiometry
Jan 30/Feb1 Sources & Shadows Color
Feb 6/8 Linear filters & edges Texture
Feb 13/15 Multi-View Geometry Stereo
Feb 20/22 Optical flow Project proposals
Feb27/Mar1 Affine SfM Projective SfM
Mar 6/8 Camera Calibration Segmentation
Mar 13/15 Springbreak Springbreak
Mar 20/22 Fitting Prob. Segmentation
Mar 27/29 Silhouettes and Photoconsistency
Linear tracking
Apr 3/5 Project Update Non-linear Tracking
Apr 10/12 Object Recognition Object Recognition
Apr 17/19 Range data Range data
Apr 24/26 Final project Final project
Tentative class schedule
![Page 3: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/3.jpg)
ComputerVision
3
Project Update and Final project presentation
Project update: 4 minute presentation (send to Talha by April 2 for compilation)
Final presentation: • Presentation and/or Demo
(your choice, but let me know)
• Short paper (Due April 22 by 23:59)(preferably Latex IEEE proc. style)
• Final presentation/demoApril 24 and 26
![Page 4: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/4.jpg)
ComputerVision
4
• Hough transform
• K-means fitting
• M-estimators
Last tuesday: Fitting
• RANSACDo– generate hypothesis
(from minimal data)– verify hypothesis
(on all the rest of data)Until good enough
influence function
![Page 5: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/5.jpg)
ComputerVision
5
Missing variable problems
• In many vision problems, if some variables were known the maximum likelihood inference problem would be easy– fitting; if we knew which line each token came
from, it would be easy to determine line parameters
– segmentation; if we knew the segment each pixel came from, it would be easy to determine the segment parameters
– fundamental matrix estimation; if we knew which feature corresponded to which, it would be easy to determine the fundamental matrix
– etc.
• This sort of thing happens in statistics, too
![Page 6: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/6.jpg)
ComputerVision
6
Missing variable problems
• Strategy– estimate appropriate values for the missing
variables– plug these in, now estimate parameters– re-estimate appropriate values for missing
variables, continue
• e.g.– guess which line gets which point– now fit the lines– now reallocate points to lines, using our knowledge
of the lines– now refit, etc.
• We’ve seen this line of thought before (k-means)
![Page 7: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/7.jpg)
ComputerVision
7
Missing variables - strategy
• We have a problem with parameters, missing variables
• This suggests:• Iterate until
convergence– replace missing
variable with expected values, given fixed values of parameters
– fix missing variables, choose parameters to maximise likelihood given fixed values of missing variables
• e.g., iterate till convergence– allocate each point to
a line with a weight, which is the probability of the point given the line
– refit lines to the weighted set of points
• Converges to local extremum
• Somewhat more general form is available
![Page 8: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/8.jpg)
ComputerVision
8
Lines and robustness
• We have one line, and n points
• Some come from the line, some from “noise”
• This is a mixture model:
• We wish to determine– line parameters– p(comes from line)
P point | line and noise params P point | line P comes from line P point | noise P comes from noise P point | line P point | noise (1 )
P point | line and noise params P point | line P comes from line P point | noise P comes from noise P point | line P point | noise (1 )
![Page 9: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/9.jpg)
ComputerVision
9
Estimating the mixture model
• Introduce a set of hidden variables, , one for each point. They are one when the point is on the line, and zero when off.
• If these are known, the negative log-likelihood becomes (the line’s parameters are c):
• Here K is a normalising constant, kn is the noise intensity (we’ll choose this later).
Qc x; i
xi cos yi sin c 2
2 2
1 i kn
i
K
![Page 10: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/10.jpg)
ComputerVision
10
Substituting for delta
• We shall substitute the expected value of , for a given
• recall =(, c, )• E(_i)=1. P(_i=1|)
+0....
• Notice that if kn is small and positive, then if distance is small, this value is close to 1 and if it is large, close to zero
P i 1|,xi P xi | i 1, P i 1 P xi | i 1, P i 1 P xi | i 0, P i 0
exp 1
2 2 xi cos yi sin c 2 exp 1
2 2 xi cos yi sin c 2 exp kn 1
![Page 11: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/11.jpg)
ComputerVision
11
Algorithm for line fitting
• Obtain some start point
• Now compute’s using formula above
• Now compute maximum likelihood estimate of
– , c come from fitting to weighted points
– comes by counting
• Iterate to convergence
0 0 ,c 0 , 0
1
![Page 12: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/12.jpg)
ComputerVision
12
![Page 13: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/13.jpg)
ComputerVision
13
The expected values of the deltas at the maximum(notice the one value close to zero).
![Page 14: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/14.jpg)
ComputerVision
14
Closeup of the fit
![Page 15: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/15.jpg)
ComputerVision
15
Choosing parameters
• What about the noise parameter, and the sigma for the line?– several methods
• from first principles knowledge of the problem (seldom really possible)
• play around with a few examples and choose (usually quite effective, as precise choice doesn’t matter much)
– notice that if kn is large, this says that points very seldom come from noise, however far from the line they lie
• usually biases the fit, by pushing outliers into the line
• rule of thumb; its better to fit to the better fitting points, within reason; if this is hard to do, then the model could be a problem
![Page 16: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/16.jpg)
ComputerVision
16
Other examples
• Segmentation– a segment is a
gaussian that emits feature vectors (which could contain colour; or colour and position; or colour, texture and position).
– segment parameters are mean and (perhaps) covariance
– if we knew which segment each point belonged to, estimating these parameters would be easy
– rest is on same lines as fitting line
• Fitting multiple lines– rather like fitting one
line, except there are more hidden variables
– easiest is to encode as an array of hidden variables, which represent a table with a one where the i’th point comes from the j’th line, zeros otherwise
– rest is on same lines as above
![Page 17: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/17.jpg)
ComputerVision
17
Issues with EM
• Local maxima– can be a serious nuisance in some problems– no guarantee that we have reached the “right”
maximum
• Starting– k means to cluster the points is often a good idea
![Page 18: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/18.jpg)
ComputerVision
18
Local maximum
![Page 19: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/19.jpg)
ComputerVision
19
which is an excellent fit to some points
![Page 20: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/20.jpg)
ComputerVision
20
and the deltas for this maximum
![Page 21: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/21.jpg)
ComputerVision
21
A dataset that is well fitted by four lines
![Page 22: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/22.jpg)
ComputerVision
22
Result of EM fitting, with one line (or at least, one available local maximum).
![Page 23: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/23.jpg)
ComputerVision
23
Result of EM fitting, with two lines (or at least, one available local maximum).
![Page 24: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/24.jpg)
ComputerVision
24
Seven lines can produce a rather logical answer
![Page 25: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/25.jpg)
ComputerVision
25
Figure from “Color and Texture Based Image Segmentation Using EM and Its Application to Content Based Image Retrieval”,S.J. Belongie et al., Proc. Int. Conf. Computer Vision, 1998, c1998, IEEE
Segmentation with EM
![Page 26: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/26.jpg)
ComputerVision
26
Motion segmentation with EM
• Model image pair (or video sequence) as consisting of regions of parametric motion– affine motion is
popular
• Now we need to– determine which
pixels belong to which region
– estimate parameters
• Likelihood– assume
• Straightforward missing variable problem, rest is calculation
vxvy
a b
c d
x
y
txty
I x, y, t I x vx , y vy, t 1 noise
![Page 27: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/27.jpg)
ComputerVision
27
Three frames from the MPEG “flower garden” sequence
Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE
![Page 28: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/28.jpg)
ComputerVision
28
Grey level shows region no. with highest probability
Segments and motion fields associated with them
Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE
![Page 29: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/29.jpg)
ComputerVision
29
If we use multiple frames to estimate the appearanceof a segment, we can fill in occlusions; so we canre-render the sequence with some segments removed.
Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE
![Page 30: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/30.jpg)
ComputerVision
30
Structure from motion without correspondences
Dellaert et al.00
EMMCMCChain Flipping
![Page 31: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/31.jpg)
ComputerVision
31
Some generalities
• Many, but not all problems that can be attacked with EM can also be attacked with RANSAC– need to be able to get
a parameter estimate with a manageably small number of random choices.
– RANSAC is usually better
• Didn’t present in the most general form– in the general form,
the likelihood may not be a linear function of the missing variables
– in this case, one takes an expectation of the likelihood, rather than substituting expected values of missing variables
– Issue doesn’t seem to arise in vision applications.
![Page 32: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/32.jpg)
ComputerVision
32
Model Selection
• We wish to choose a model to fit to data– e.g. is it a line or a
circle?– e.g is this a
perspective or orthographic camera?
– e.g. is there an aeroplane there or is it noise?
• Issue– In general, models
with more parameters will fit a dataset better, but are poorer at prediction
– This means we can’t simply look at the negative log-likelihood (or fitting error)
![Page 33: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/33.jpg)
ComputerVision
33
Top is not necessarily a betterfit than bottom(actually, almost always worse)
![Page 34: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/34.jpg)
ComputerVision
34
![Page 35: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/35.jpg)
ComputerVision
35
We can discount the fitting error with some term in the number of parameters in the model.
![Page 36: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/36.jpg)
ComputerVision
36
Discounts
• AIC (an information criterion)– choose model with
smallest value of
– p is the number of parameters
• BIC (Bayes information criterion)– choose model with
smallest value of
– N is the number of data points
• Minimum description length– same criterion as
BIC, but derived in a completely different way
2L D;* p logN
2L D;* 2 p
![Page 37: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/37.jpg)
ComputerVision
37
Example: Dominant planes in USfM
(Pollefeys et al., ECCV’2002)
Two or three views only sharing points in a plane cause problem for uncalibrated structure from motion
Solution: – Detect problem through model selection– Delay computation of ambiguous parameters until more information is
available
![Page 38: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/38.jpg)
ComputerVision
38
GRIC: Geometric Robust Information Criterion
(Torr et al., ICCV’98)
robust! (otherwise outliers will make you get the most expensive model every time,i.e. the model is never good enough for them)
(2D or 3D)
(6 for points in 3 views)
(11x3-15 (PPP) or 8x2(HH))
(inspired from Kanatani’s GIC)
structure complexity
model complexity
co-dimension
![Page 39: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/39.jpg)
ComputerVision
39
F vs. H PPP vs. HH
![Page 40: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/40.jpg)
ComputerVision
40
HHgric
PPPgric
![Page 41: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/41.jpg)
ComputerVision
41
Farmhouse 3D models
Strategy:– Detect problem using model selection– Reconstruct planes and 3D separately– Perform self-calibration– Assemble reconstruction – compute poses (for planar parts)– Refine complete reconstruction
(bundle adjustment)
![Page 42: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/42.jpg)
ComputerVision
42
Key-frame selection in video
• Algorithm– Compare models for F and H using GRIC– Continue tracking as long as
#inliers>90%
H-GRIC
F-GRIC
100% 90%
(Repko and Pollefeys, 3DIM05)
![Page 43: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/43.jpg)
ComputerVision
43
Cross-validation
• Split data set into two pieces, fit to one, and compute negative log-likelihood on the other
• Average over multiple different splits
• Choose the model with the smallest value of this average
• The difference in averages for two different models is an estimate of the difference in KL divergence of the models from the source of the data
![Page 44: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/44.jpg)
ComputerVision
44
Model averaging
• Very often, it is smarter to use multiple models for prediction than just one
• e.g. motion capture data– there are a small
number of schemes that are used to put markers on the body
– given we know the scheme S and the measurements D, we can estimate the configuration of the body X
• We want
• If it is obvious what the scheme is from the data, then averaging makes little difference
• If it isn’t, then not averaging underestimates the variance of X --- we think we have a more precise estimate than we do.
P X | D P X | S1,D P S1 | D P X | S2,D P S2 | D P X | S3,D P S3 | D
![Page 45: Computer Vision EM and model selection Marc Pollefeys COMP 256 Some slides and illustrations from D. Forsyth, …](https://reader031.fdocuments.net/reader031/viewer/2022020308/56649d625503460f94a43d4a/html5/thumbnails/45.jpg)
ComputerVision
45
Next class: Shape from silhouettes and photo-consistency