Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.
-
Upload
martin-patrick -
Category
Documents
-
view
215 -
download
0
Transcript of Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.
![Page 1: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/1.jpg)
Latent Boosting for Action RecognitionZhi Feng Huang et al.
BMVC 2011
2014. 6. 12. Jeany Son
![Page 2: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/2.jpg)
Background – learning with latent variables
• Multiple Instance Learning (MI-SVM, mi-SVM)• (-) Single plain latent variable
• Latent SVM • (+) Structured latent variable• (-) Control parameters / Normalize different features
• MILboost• (+) Not require to normalize different features• (-) Single latent variable / Not structured
• hCRF• Learning parameters and weights for features
Latent Boosting: structured latent variable, not require to normalize different features, feature selection
![Page 3: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/3.jpg)
Boosting
• Combining many weak predictors to produce an ensemble predictor• training examples with high error are weighted higher than those with lower
error• Difficult instances get more attention
• AdaBoost : “shortcoming” are identified by high-weight data points
• Gradient Boosting : “shortcomings” are identified by gradients
![Page 4: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/4.jpg)
Gradient Boost
• Gradient Boosting = Gradient Descent + Boosting
• Analogous to line search in steepest descent • Construct the new base-learners to be maximally correlated with the negative
gradient of the loss function, associated with the whole ensemble.• Arbitrary loss functions can be applied
![Page 5: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/5.jpg)
• Function estimate (parametric)• Change the function optimization problem into the parameter estimation one
Function estimation
given
![Page 6: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/6.jpg)
Steepest descent optimization
• “greedy stage-wise” approach of function incrementing with the base-learners
• The optimal step-size rho should be specified at each iteration
• The optimization rule is defined as:
![Page 7: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/7.jpg)
Gradient Boost
• Solution to the parameter estimates can be difficult to obtain
• Choose new function h to be most correlated with –g(x)
• Classic least-squares minimization problem
![Page 8: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/8.jpg)
: Line search by Newton’s method
![Page 9: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/9.jpg)
K-class Gradient Boost
• Goal : learn a set of scoring function
• by minimizing negative log-loss of the training data
• Probability of an example x being class k :
Weak classifier
![Page 10: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/10.jpg)
• Solve the optimization problem
• Select h to the most parallel with the –g(X) by following minimization problem
• Scoring function is updated as
![Page 11: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/11.jpg)
-1
![Page 12: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/12.jpg)
LatentBoost for Human Action Recognition
• A tracklet is denoted by 5 tuples • : image feature• : position of person in the t-th frame of the tracklet
• : latent variables
l1 l2 l3 l4 l5
x1 x2 x3 x4 x5
![Page 13: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/13.jpg)
Features
• Optical flow features (unary)• Split into 4 scalar fields channels & motion magnitude
• Color histogram features (pairwise)• difference between color histograms in rectangular sub-windows taken from
adjacent frames
![Page 14: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/14.jpg)
Positive optical flow features
(a) Bend (b) Jack (c) Jump (d) pJump (e) run (f) side (g) walk (h) wave1 (i) wave2
![Page 15: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/15.jpg)
Latent Boosting
• Assume that • an example (x,y) is associated with a set of latent variables L={l1, l2, …, lT}• These latent variables are constrained by an undirected graph structure
G=(V,E)
• Scoring function of (x,L) pair for the k-th class
where
l1 l2 l3 l4 l5
x1 x2 x3 x4 x5
![Page 16: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/16.jpg)
Weak learners of the unary & pairwise potential
: gradient of loss function w.r.t. unary potential
: gradient of loss function w.r.t. pairwise potential
![Page 17: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/17.jpg)
Marginal distributions
These can be computed efficiently by using Belief Propagation
l1 l2 l3 l4 l5
x1 x2 x3 x4 x5
y
![Page 18: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/18.jpg)
F
![Page 19: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/19.jpg)
Weizmann dataset (83 videos, 9 events)
![Page 20: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/20.jpg)
Typical tracklets (29x60) from the Weizmann dataset
Jacking
Running
Jumping
Waving
![Page 21: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/21.jpg)
TRECVID dataset (5 cameras, 10 videos, 7 events)
• Typical tracklets (29x60) from the TRECVID dataset
![Page 22: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/22.jpg)
![Page 23: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/23.jpg)
![Page 24: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/24.jpg)
Limitations
• Not guaranteed to find the global optimum in a non-convex problem
• Performance of the final classifier is very sensitive to the initialization
• If the latent structure is not a tree, LatentBoost can perform inference with LBP : slow and not exact than BP
• Summation over all the possible latent variable may cause problems
![Page 25: Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son.](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649dc75503460f94abbe0e/html5/thumbnails/25.jpg)
Summary
• Novel boosting algorithm with latent variables
• Applying to the task of human action recognition
• New way to solve problems with a structure of latent variables