Contextual Classification with Functional Max-Margin Markov Networks
description
Transcript of Contextual Classification with Functional Max-Margin Markov Networks
![Page 1: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/1.jpg)
Contextual Classification withFunctional Max-Margin Markov Networks
Dan Munoz Drew BagnellNicolas Vandapel Martial Hebert
![Page 2: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/2.jpg)
2
Problem
Wall
Vegetation
Ground
Tree trunk
Sky
Support
Vertical
Geometry Estimation(Hoiem et al.)
3-D Point Cloud Classification
Our classifications
![Page 3: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/3.jpg)
3
Room For Improvement
![Page 4: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/4.jpg)
4
Approach: Improving CRF Learning
Gradient descent (w) “Boosting” (h)
+ Better learn models with high-order interactions+ Efficiently handle large data & feature sets+ Enable non-linear clique potentials
• Friedman et al. 2001, Ratliff et al. 2007
![Page 5: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/5.jpg)
5
Conditional Random FieldsLafferty et al. 2001
Pairwise model
MAP Inference
yi
xLabels
![Page 6: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/6.jpg)
6
Parametric Linear Model
Weights
Local features that describe label
![Page 7: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/7.jpg)
7
Associative/Potts Potentials
Labels Disagree
![Page 8: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/8.jpg)
8
Overall Score
Overall Score for a labeling y to all nodes
![Page 9: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/9.jpg)
9
Learning Intuition Iterate
• Classify with current CRF model
• If (misclassified)
• (Same update with edges)
φ( ) increase score
φ( ) decrease score
![Page 10: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/10.jpg)
10
Max-Margin Structured Prediction
min Best score fromall labelings (+M)
Score withground truth labelingw
Ground truth labels
Taskar et al. 2003
Convex
![Page 11: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/11.jpg)
11
Descending† Direction
Labels from MAP inference
Ground truth labels
(Objective)
![Page 12: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/12.jpg)
12
Learned Model
![Page 13: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/13.jpg)
13
Unit step-size
Update Rule
Ground truth Inferred
wt+1+= - + -
, and λ = 0
![Page 14: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/14.jpg)
14
Iterate
• If (misclassified)
Verify Learning Intuition
wt+1 += - + -
φ( ) increase score
φ( ) decrease score
![Page 15: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/15.jpg)
15
Alternative Update1. Create training set: D
• From the misclassified nodes & edges
, +1
, -1
, +1
, -1D =
![Page 16: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/16.jpg)
16
Alternative Update1. Create training set: D2. Train regressor: ht
ht(∙)D
![Page 17: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/17.jpg)
17
Alternative Update1. Create training set: D2. Train regressor: h3. Augment model:
(Before)
![Page 18: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/18.jpg)
18
Functional M3N Summary Given features and labels for T iterations
• Classification with current model
• Create training set from misclassified cliques
• Train regressor/classifier ht
• Augment model
D
![Page 19: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/19.jpg)
19
Illustration Create training set
D
+1
-1
+1
-1
![Page 20: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/20.jpg)
20
Illustration Train regressor ht
h1(∙)
ϕ(∙) = α1 h1(∙)
![Page 21: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/21.jpg)
21
Illustration Classification with current CRF model
ϕ(∙) = α1 h1(∙)
![Page 22: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/22.jpg)
22
Illustration Create training set
ϕ(∙) = α1 h1(∙)
D
-1
+1
![Page 23: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/23.jpg)
23
Illustration Train regressor ht
h2(∙)
ϕ(∙) = α1 h1(∙)+α2 h2(∙)
![Page 24: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/24.jpg)
24
Illustration Stop
ϕ(∙) = α1 h1(∙)+α2 h2(∙)
![Page 25: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/25.jpg)
25
Boosted CRF Related Work Gradient Tree Boosting for CRFs
• Dietterich et al. 2004 Boosted Random Fields
• Torralba et al. 2004 Virtual Evidence Boosting for CRFs
• Liao et al. 2007
Benefits of Max-Margin objective• Do not need marginal probabilities• (Robust) High-order interactions
Kohli et al. 2007, 2008
![Page 26: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/26.jpg)
26
Using Higher Order Information
Colored by elevation
![Page 27: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/27.jpg)
27
Region Based Model
![Page 28: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/28.jpg)
28
Region Based Model
![Page 29: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/29.jpg)
29
Region Based Model
Inference: graph-cut procedure• Pn Potts model (Kohli et al. 2007)
![Page 30: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/30.jpg)
30
How To Train The Model
1Learning
![Page 31: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/31.jpg)
31
How To Train The Model
Learning(ignores features from clique c)
![Page 32: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/32.jpg)
32
How To Train The Model
β
Learning
β1
Robust Pn PottsKohli et al. 2008
![Page 33: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/33.jpg)
33
Experimental Analysis 3-D Point Cloud Classification Geometry Surface Estimation
![Page 34: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/34.jpg)
34
Random Field Description Nodes: 3-D points Edges: 5-Nearest Neighbors Cliques: Two K-means segmentations
Features[0,0,1]
normalθ
Local shape Orientation Elevation
![Page 35: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/35.jpg)
35
Qualitative Comparisons
Parametric Functional (this work)
![Page 36: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/36.jpg)
36
Qualitative Comparisons
Parametric Functional (this work)
![Page 37: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/37.jpg)
37
Qualitative Comparisons
Parametric Functional (this work)
![Page 38: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/38.jpg)
38
Quantitative Results (1.2 M pts) Macro* AP: Parametric 64.3% Functional 71.5%
Wire Pole/Trunk Façade Vegetation0.75
0.85
0.95
0.90
0.80
0.89 0.880.90
0.81
0.88
0.93
Recall0.00
0.20
0.40
0.60
0.80
1.00
0.26 0.22
0.880.99
0.50
0.26
0.910.99
Precision
+24%
+5%
+4%
+3%
-1%
![Page 39: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/39.jpg)
39
Experimental Analysis 3-D Point Cloud Classification Geometry Surface Estimation
![Page 40: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/40.jpg)
40
Random Field Description Nodes: Superpixels (Hoiem et al. 2007) Edges: (none) Cliques: 15 segmentations (Hoiem et al. 2007)
Features (Hoiem et al. 2007)• Perspective, color, texture, etc.
1,000 dimensional space
More Robust
![Page 41: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/41.jpg)
41
Quantitative Comparisons
Parametric (Potts)
Functional (Potts)
Hoiem et al. 2007
Functional (Robust Potts)
![Page 42: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/42.jpg)
42
Qualitative Comparisons
Parametric (Potts)
Functional (Potts)
![Page 43: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/43.jpg)
43
Qualitative Comparisons
Parametric (Potts)
Functional (Potts) Functional (Robust Potts)
![Page 44: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/44.jpg)
44
Qualitative Comparisons
Parametric (Potts)
Functional (Potts)
Hoiem et al. 2007
Functional (Robust Potts)
![Page 45: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/45.jpg)
45
Qualitative Comparisons
Parametric (Potts)
Functional (Potts)
Hoiem et al. 2007
Functional (Robust Potts)
![Page 46: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/46.jpg)
46
Qualitative Comparisons
Parametric (Potts)
Functional (Potts)
Hoiem et al. 2007
Functional (Robust Potts)
![Page 47: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/47.jpg)
47
Conclusion Effective max-margin learning of high-order CRFs
• Especially for large dimensional spaces• Robust Potts interactions• Easy to implement
Future work• Non-linear potentials (decision tree/random forest)• New inference procedures:
Komodakisand Paragios 2009Ishikawa 2009Gould et al. 2009Rother et al. 2009
![Page 48: Contextual Classification with Functional Max-Margin Markov Networks](https://reader034.fdocuments.net/reader034/viewer/2022051402/5681632e550346895dd3a983/html5/thumbnails/48.jpg)
48
Thank you Acknowledgements
• U. S. Army Research Laboratory• Siebel Scholars Foundation• S. K. Divalla, N. Ratliff, B. Becker
Questions?