CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications
description
Transcript of CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications
![Page 1: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/1.jpg)
Semi-Supervised Learning in Computer VisionPart II
Amir Saffari,Christian Leistner,Horst Bischof
Institute for Computer Graphics and Vision, Graz University of Technology
June 18th, 2010
![Page 2: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/2.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised BoostingTracking
3 Semi-Supervised Random ForestsMILForestsOn-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 3: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/3.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost[Mallapragada et al.,PAMI’09] [Leistner et al.,CVPR’08]
Loss function ∑(x,y)∈XL e−yF(x)+∑
x∈XU
(λu
∑x ′∈XU
s(x, x ′) cosh(F (x) − F (x ′)) + λl
∑(x ′,y ′)∈XL
s(x, x ′)e−2y ′F(x))
Optimization Problem
arg minf (x),α
=∑
x ′∈XU
( ∑(x,y)∈XL
s(x, x ′)e−2y(F(x ′)+αf (x ′))
+λu
∑x ′∈XU
s(x, x ′)e((F(x ′)−F(x))eα(f (x)−f (x ′)))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 4: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/4.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost
px = λl
∑(x ′,y ′)∈XL
I(y ′ = 1)s(x, x ′)e−2F(x ′)+λu
2
∑x∈XU
s(x, x ′)eF(x ′)−F(x)
and
qx = λl
∑(x ′,y ′)∈XL
I(y ′ = −1)s(x, x ′)e−2F(x ′)+λu
2
∑x∈XU
s(x, x ′)eF(x)−F(x ′)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 5: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/5.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost
Pseudo Labels and Weights
yx = sign(px − qx)
wx = |px − qx|
Optimal α
α =14
ln
∑x∈XU
(piI(f (x) = 1) + qiI(f (x) = −1)
)∑x∈XU
(piI(f (x) = −1) + qiI(f (x) = 1)
)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 6: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/6.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost
I labeled training data (x, y) ∈ XL and unlabeled data x ′ ∈ XU
I Similarity measure s(x, x ′)I Weak learners fi
I weight parameters λu, λl
I max iterations T
1 For t = 1, 2, . . . , T2 Compute pi and qi for every given sample3 yx = sign(px − qx)
4 wx = |px − qx|
5 Train weak classifier ft (x)
6 Compute αt
7 F (x)← F (x) + αt ft (x)
8 EndFor
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 7: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/7.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost with learned Similarities[Hertz et al.,CVPR’04]
Radial Basis Function [Zhu et al.,ICML’03]
s(x, x ′) = e
(−
d(x,x ′)2
σ2
)
d(x, x ′) . . . distance between points
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 8: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/8.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Learning Distance Functions
Idea
Learn distance or metric function on labeled data which thencan discriminatively support task-specific classification.
Distance Function
F d : X× X→ Y = [−1 1]
Training Pairs of “same” or “different” [Hertz et al.,CVPR’04]
Dd = {(x, x ′, +1)|y = y ′, x, x ′ ∈ DL} ∪∪{(x, x ′, −1)|y , y ′, x, x ′ ∈ DL}
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 9: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/9.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost with learned Distance Functions
Number of Training Pairs (Symmetric case)n·(n−1)
2
SemiBoost+
-
?-
-++
+ ??
?
?
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 10: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/10.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Using Arbitrary Classifiers
Approximate pair-wise classifier
|F (x, x ′)| ≈ |F (x) − F (x ′)|
+
-
?
?
+
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 11: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/11.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Reusing Prior Classifiers[Schapire et al,ML’02]
Classifier Combination
F C(x) = α0F P(x) + F (x)
SemiBoost+
-
?
??
?
?
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 12: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/12.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost Applications
Car Detection
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 13: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/13.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Similarity Performance
Accuracy depending on the number of samples
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 14: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/14.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost Applications
Car Detection
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 15: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/15.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost Applications
Face Detection
(a) prior (b) trained (c) combined
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 16: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/16.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Simple Data mining method[Levin et al.,ICCV’03][Rosenberg et al.,2005]
1 Labeled training data (x, y) ∈ XL
2 Train cascaded detector F P(x) on XL using [Viola & Jones,2001]
3 Use a web image search engine in order to collect hugeamounts of possibly useful images XU ; pass phrases that aremuch likely related to your target object
4 Apply F P(x) in a sliding window manner on XU and copy alldetections to XU∗
5 Train a SemiBoost classifier F (x) on XL and XU∗ using F P(x)
as prior
6 Output the final classifier F (x)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 17: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/17.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost Applications
Transfer Learning
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 18: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/18.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SemiBoost Applications
Transfer Learning
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 19: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/19.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised BoostingTracking
3 Semi-Supervised Random ForestsMILForestsOn-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 20: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/20.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Boosting
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 21: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/21.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
[Oza,PhD-Thesis’01], [Grabner & Bischof,CVPR’06]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 22: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/22.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 23: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/23.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 24: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/24.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 25: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/25.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Tracking is an One-Shot Semi-supervised Learning Problem
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 26: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/26.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line SemiBoost
px ≈ e−Fn−1(x)∑
xi∈X+
S(x, xi) ≈ e−Fn−1(x)F +(x) ≈ e−Fn−1(x)eF P(x)
eF P(x) + e−F P(x)
qx ≈ eFn−1(x)∑
xi∈X−
S(x, xi) ≈ eFn−1(x)F −(x) ≈ eFn−1(x)e−F P(x)
eF P(x) + e−F P(x)
pn(x)−qn(x) =sinh(F P(x) − Fn−1)
cosh(F P(x))= tanh(F P(x))−tanh(Fn−1(x))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 27: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/27.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 28: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/28.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 29: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/29.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Problem: Rapid Appearance Changes
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 30: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/30.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 31: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/31.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Exploration-Exploitation Dilemma
Convex Trade-off
`(F (x)) = (1 − α)`l(F (x)) + α`u(F (x))
We need more Robustness when minimizing the labeled loss!
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 32: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/32.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Loss Functions
Random classification noise defeats all convex potential boosters[Long and Servidio,ICML’08]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 33: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/33.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Gradient Boost
Gradient Descent Functional Gradient Descent
GradientBoost [Friedman et al.,Annals of Statistics’01]
ft (x) = arg maxf (x)
−∇LT f (x)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 34: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/34.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Gradient Boost
I A training sample: (xn, yn), A differentiable loss function `(·)I Number of selectors M , Number of weak learners per selector K
1 Set F0(xn) = 0.
2 Set the initial weight wn = −`′(0).
3 For m = 1 to M
4 For k = 1 to K
5 Train kth weak learner f km(x) with sample (xn, yn) and weight wn.
6 ekm ← ek
m + wnI(sign(f km(xn)) , yn) //Compute the error
7 EndFor
8 Find the best weak learner with the least total weighted error:j = arg min
kek
m.
9 Set fm(xn) = f jm(xn).
10 Set Fm(xn) = Fm−1(xn) + fm(xn).
11 Set the weight wn = −`′(ynFm(xn)).
12 EndFor
13
→Output the final model: F (x)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 35: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/35.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Weight Updates
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 36: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/36.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Co-Training of Pedestrian Detectors
Exponential Loss Logit Loss
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 37: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/37.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
SERBoost
Expectation Regularization [Mann and MacCallum,ICML’07]
Penalize model predictions on unlabeled data that deviate fromcertain expectation.
SERBoost [Saffari et al.,ECCV’08]
L(H (x), X) = Ll(H (x), Xl) + βLu(H (x), Xu)
L(H (x), X) =∑
x∈XL
e−yH(x) +∑
x∈XU
e−ypH(x)cosh(H (x))
Pseudo Label
yp = 2P+p (x) − 1
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 38: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/38.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line SERBoost with logistic loss
Supervised Loss
Ll(XL) =
∑(x,y)∈Xl
log(
1 + e−2yF(x))
=∑
(x,y)∈XL
log(
e−yF(x)(eyF(x) + e−yF(x)))
=∑
(x,y)∈XL
−yF (x) + log(
eF(x) + e−F(x))
.
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 39: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/39.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line SERBoost with logistic loss
Minimize the cross entropy
H (Pp, P) = −∑
z∈{−1,1}
Pp(y = z|x) log P(y = z|x)
= −(
2Pp(y = 1|x) − 1)︸ ︷︷ ︸
yp(x)
F (x) + log(
eF(x) + e−F(x))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 40: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/40.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line SERBoost with logistic loss
Unsupervised Loss
Lu(XU ) =∑
x∈XU
H (Pp, P) =∑
x∈XU
−yp(x)F (x) + log(
eF(x) + e−F(x))
Unlabeled Update
∀ x ∈ XU :wx =∣∣yp(x) − tanh(F (x))
∣∣yx = sign
(yp(x) − tanh(F (x))
)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 41: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/41.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line SERBoost with logistic loss
Unsupervised Loss
Lu(XU ) =∑
x∈XU
H (Pp, P) =∑
x∈XU
−yp(x)F (x) + log(
eF(x) + e−F(x))
Unlabeled Update
∀ x ∈ XU :wx =∣∣yp(x) − tanh(F (x))
∣∣yx = sign
(yp(x) − tanh(F (x))
)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 42: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/42.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
OSER Tracking
λ = 0.5
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 43: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/43.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Influence of convex combination
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 44: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/44.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Multiple Instance Boosting[Viola et al.,NIPS’05][Babenko et al.,CVPR’09]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 45: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/45.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Multiple Instance Boosting[Viola et al.,NIPS’05][Babenko et al.,CVPR’09]
Bags
{(B1, y1), . . . , (Bn, yn)}
Bi = {x1i , x2
i , . . . , xnii }
Minimize binary log-likelihood
log L =∑
i
(yi log p(yi) + (1 − yi) log p(yi))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 46: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/46.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Semi-Supervised Multiple Instance Boosting[Zeisl et al.,CVPR’10]
Combine benefits of MIL and SSL
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 47: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/47.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Semi-Supervised Multiple Instance Boosting[Zeisl et al.,CVPR’10]
Unlabeled Loss of the Bags
Lu(XBu ) = −
Nu∑i=1
∑z∈Y
Pp(z|Bui ) log(P(z|Bu
i ))
Approximate max with geometric mean
P(y = 1|Bi) = 1 −[NBi∏
j=1
(1 − P(y = 1|xij)
)]1/NBi
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 48: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/48.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Semi-Supervised Multiple Instance Boosting[Zeisl et al.,CVPR’10]
Gradient for NOR and geometric mean
aij(z) =2
NBi
z − P(y = 1|Bi)
P(y = 1|Bi)P(y = 1|xij)
Pseudo Labels and Weights
wij =β∣∣∣∑
z∈Y
Pp(z|Bui )aij(z)
∣∣∣yij =I
(β
∑z∈Y
Pp(z|Bui )aij(z) > 0
)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 49: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/49.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Semi-Supervised Multiple Instance Boosting[Zeisl et al.,CVPR’10]
Experimental Results
Sequence MILSER MIL OSB OAB
sylv 0.64 0.61 0.46 0.50david 0.71 0.54 0.31 0.32faceocc2 0.78 0.65 0.63 0.64coke11 0.18 0.29 0.12 0.20tiger1 0.60 0.51 0.17 0.27tiger2 0.46 0.50 0.08 0.25faceocc1 0.68 0.63 0.71 0.47girl 0.64 0.53 0.69 0.38
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 50: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/50.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Co-Training[Liu et al.,ICCV’09][Saffari et al.,ECCV’10]
Performance measured in average location center errors in pixels
Approach sylv david faceocc2 tiger1 tiger2 coke faceocc1 girl
MV-GPBoost 17 20 10 15 16 20 12 15CoBoost 15 33 11 22 19 14 13 17SemiBoost 22 59 43 46 53 85 41 52MILBoost 11 23 20 15 17 21 27 32
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 51: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/51.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
End Part I
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 52: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/52.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Random Forests
[Breiman,ML’01]
Ensemble of n decision trees
F (x) =∑N
n=1 f (x)
Information Gain
∆H = −|Il |
|Il |+|Ir |H (Il) −
|Ir ||Il |+|Ir |
H (Ir)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 53: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/53.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Random Forests
Advantages:I speedI parallelismI noise robustI inherently multi-class
Applications:I Object Detection, Semantic Segmentation, Categorization,
Tracking, etc.
Disadvantage:I RFs demand a huge amount of data in order to leverage their
full potential [Caruana et al.,ICML’08]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 54: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/54.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Random Forests
Advantages:I speedI parallelismI noise robustI inherently multi-class
Applications:I Object Detection, Semantic Segmentation, Categorization,
Tracking, etc.
Disadvantage:I RFs demand a huge amount of data in order to leverage their
full potential [Caruana et al.,ICML’08]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 55: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/55.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Random Forests
Advantages:I speedI parallelismI noise robustI inherently multi-class
Applications:I Object Detection, Semantic Segmentation, Categorization,
Tracking, etc.
Disadvantage:I RFs demand a huge amount of data in order to leverage their
full potential [Caruana et al.,ICML’08]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 56: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/56.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Semi-Supervised Random Forests
Random Forests maximize the margin
ml(x, y) = p(y |x) − maxk∈Yk,y
p(k|x)
Unlabeled Margin
mu(xu) = maxi∈Y
fi(xu)
Semi-supervised Loss
L(f) =1
|Xl |
∑(x,y)∈Xl
`(fy(x)) +λ
|Xu |
∑x∈Xu
`(mu(x))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 57: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/57.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Optimization
Incorporate labels for the unlabeled data as additionaloptimization variables!
Deterministic Annealing [Rose,IJCNN’98]
p∗ = arg minp∈P
Ep(F(y)) − TH(p)
T0 > T1 > . . . > T∞ = 0
p∗ . . . distributions over the label predictions
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 58: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/58.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Optimization
Incorporate labels for the unlabeled data as additionaloptimization variables!
Deterministic Annealing [Rose,IJCNN’98]
p∗ = arg minp∈P
Ep(F(y)) − TH(p)
T0 > T1 > . . . > T∞ = 0
p∗ . . . distributions over the label predictions
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 59: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/59.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Optimization
DA-Loss for Semi-supervised Random Forests
LDA(f, p) =1
|Xl |
∑(x,y)∈Xl
`(fy(x))+
+α
|Xu |
∑x∈Xu
K∑i=1
p(i|x)`(fi(x))+
+T
|Xu |
∑x∈Xu
K∑i=1
H (p)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 60: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/60.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Two Step Optimization
First Stage
f∗n = arg minf
1|Xl |
∑(x,y)∈Xl
`(fy(x))+
+α
|Xu |
∑x∈Xu
`(fyu(x))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 61: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/61.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Two Step Optimization
Second Stage
p∗ =arg minp
α
|Xu |
∑x∈Xu
K∑i=1
p(i|x)`(fi(x))+
+T
|Xu |
∑x∈Xu
K∑i=1
p(i|x) log(p(i|x))
p∗(i|x) = exp(−α`(fi(x))+T
T )/Z(x)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 62: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/62.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Finding the optimal Distributions
Take the derivate w.r.t. each class
hi(p, x) = p(i|x)(α`(gi(x)) + T log(p(i|x))) (1)
dhi
dpi= α`(gi(x)) + T log(p(i|x)) + T (2)
Optimal Distribution
p∗(i|x) = exp(−α`(fi(x))+T
T )/Z(x)
Z(x) =∑K
i=1 p∗(i|x) is the partition function
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 63: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/63.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Experiments
Classification Accuracy in %
Method SVM TSVM SER RMSB RF DAS-RF
g50c 91.7 93.1 91.9 94.2 89.1 93.3Letter 70.3 65.9 76.5 79.9 76.4 79.7SensIt 80.2 79.9 81.9 83.7 76.5 84.3
Train and Test time in Seconds
Method SVM TSVM SER RMSB RF DAS-RF GPU
Letter 25 74 3124 125 35 72 29SensIt 195 687 1158 514 125 410 137
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 64: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/64.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Caltech-101
binary classification error
Class RF DAS-RF ImprovementC4 0.0081 0.0033 58%C5 0.0078 0.002 65%C20 0.011 0.0013 87.5%C33 0.007 0.003 52%C81 0.0027 0.001 62.5%
classification error over different numbers of labeled samples
Algorithm l = 15 l = 30
RF 0.72 0.64DAS-RF 0.70 0.60LinSVM 0.74 0.65improvement 2% 4%
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 65: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/65.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Prior Regularization
Potential Information Gain
∆H = −|Il |
|Il |+|Ir |H (Il) −
|Ir ||Il |+|Ir |
H (Ir)
Kullback-Leibler Divergence
DKL(q‖p) = H (q, p) − H (q)
DSKL(q‖p) = 12(DKL(q‖p) + DKL(p‖q))
Prior-regularized node score
∆H∗ = ∆H + λ∆DSKL(q‖p)
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 66: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/66.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Airbag
OOBE : em−1F − em
F
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 67: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/67.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Airbag
OOBE : em−1F − em
F
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 68: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/68.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised BoostingTracking
3 Semi-Supervised Random ForestsMILForestsOn-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 69: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/69.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Multiple Instance Forests[Leistner et al.,ECCV’10]
--
-
-
-
-
+
-
---
-
+
+
[Dietterich,AI’97]
I Content-based Image RetrievalI Object Detection and CategorizationI TrackingI Action Recognition
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 70: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/70.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Multiple Instance Forests
Multiple Instance Learning is a special case of semi-supervisedLearning!
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 71: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/71.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Multiple Instance Forests
Multi-class Instance Classifier
F (x) : X→ Y = {1, . . . , K }
{(B1, y1), . . . , (Bn, yn)}, where yi ∈ {1, . . . , K }
Objective Function
({y ji }∗, F∗) =arg min
{y ji },F(·)
n∑i=1
ni∑j=1
`(Fy j
i(xj
i))
s.t. ∀i :
ni∑j=1
I(yi = arg maxk∈Y
Fk(xji)) > 1.
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 72: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/72.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Multiple Instance Forests
DA Loss Function
LDA(F , p) =
n∑i=1
ni∑j=1
K∑k=1
p(k|xji)`(Fk(xj
i)) + Tn∑
i=1
H (pi)
Entropy of the distribution inside a bag
H (pi) = −
ni∑j=1
K∑k=1
p(k|xji) log(p(k|xj
i))
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 73: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/73.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Evaluation
Method Elephant Fox Tiger Musk1 Musk2
RandomForest[Breiman,2001] 74 60 77 85 78
MILForest 84 64 82 85 82
MI-Kernel[Andrews,2003] 84 60 84 88 89
MI-SVM[Zhou,2009] 81 59 84 78 84
mi-SVM[Zhou,2009] 82 58 79 87 84
MILES[Chen,2006] 81 62 80 88 83
SIL-SVM[Bunescu,2007] 85 53 77 88 87
AW-SVM[Gehler,2007] 82 64 83 86 84
AL-SVM[Gehler,2007] 79 63 78 86 83
EM-DD[Zhang,2001] 78 56 72 85 85
MILBoost-NOR[Viola,2006] 73 58 56 71 61
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 74: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/74.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Corel Data Set
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 75: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/75.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Corel Data Set
Results for the COREL image categorization benchmark
Method Corel-1000 Corel-2000 Testing[sec.] Training[sec.]
MILForest 59 66 4.6 22.0
MILES 58 67 180 960
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 76: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/76.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Semantic Segmentation
[Vezhnevets & Buhmann,CVPR’10]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 77: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/77.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Outline
1 SemiBoost & Visual Similarity Learning
2 On-line Semi-supervised BoostingTracking
3 Semi-Supervised Random ForestsMILForestsOn-line Random Forests
4 On-line Manifold Regularization
5 Conclusion & Outlook
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 78: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/78.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Random Forests
I On-line Bagging [Oza,PhD-Thesis’01]→ Poisson(λ)
I On-line recursive splitting is hard→ Tree Growing
Info Gain
∆L(Rj , s) = L(Rj) −|Rjls |
|Rj |L(Rjls) −
|Rjrs |
|Rj |L(Rjrs)
Splitting Rules
|Rj | > α and ∃s ∈ S : ∆L(Rj , s) > β
I On-line DA→ Annealing Schedule for each sample xi
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 79: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/79.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Random Forests
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 80: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/80.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Interactive Segmentation
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 81: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/81.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking with On-line RF
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 82: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/82.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 83: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/83.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Tracking
RT ∩ RGT/RT ∪ RGT
Sequence OSERB MILBoost OSB OAB ORF MILForest RF
sylv 0.64 0.61 0.46 0.50 0.53 0.59 0.50david 0.69 0.54 0.31 0.32 0.69 0.72 0.32faceocc2 0.77 0.65 0.63 0.64 0.72 0.77 0.79tiger1 0.65 0.51 0.17 0.27 0.38 0.55 0.34tiger2 0.42 0.50 0.08 0.25 0.43 0.53 0.32coke 0.2 0.33 0.08 0.25 0.35 0.35 0.15faceocc1 0.77 0.63 0.71 0.47 0.71 0.77 0.77girl 0.77 0.53 0.69 0.38 0.70 0.71 0.74
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 84: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/84.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Manifold Regularization[Goldberg et al.,ECML’08]
I Based on Convex Programming in kernel space usingstochastic gradient descent
I Random Projection Trees [Dasgupta & Freund, TR, 2007]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 85: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/85.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Manifold Regularization[Goldberg et al.,ECML’08]
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 86: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/86.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Graph-based SSL[Kveton et al.,OLCV’10]
Harmonic Function Solution
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 87: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/87.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Graph-based SSL[Kveton et al.,OLCV’10]
Merge the two most similar vertices and add the new vertex
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 88: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/88.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
On-line Graph-based SSL[Kveton et al.,OLCV’10]
Face recognition of 8 people
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 89: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/89.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Conclusion
Semi-supervised Learning is a powerful learning paradigm withmany potential applications in Computer Vision
I It is often also the way how learning is done in natureI It can be applied virtually everywhere where classifiers are
appliedI On-line SSL can be used in order to make
tracking-by-detection systems more robust
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 90: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/90.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Conclusion
Semi-supervised Learning is a powerful learning paradigm withmany potential applications in Computer Vision
I It is often also the way how learning is done in natureI It can be applied virtually everywhere where classifiers are
appliedI On-line SSL can be used in order to make
tracking-by-detection systems more robust
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 91: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/91.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Outlook
We need to increase the robustness of SSL algorithms in order toleverage more applications
I Demand for more on-line Semi-Supervised MethodsI SSL from weakly-related unlabeled data
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 92: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/92.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
Outlook
We need to increase the robustness of SSL algorithms in order toleverage more applications
I Demand for more on-line Semi-Supervised MethodsI SSL from weakly-related unlabeled data
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II
![Page 93: CVPR2010: Semi-supervised Learning in Vision: Part 3: Algorithms and Applications](https://reader034.fdocuments.net/reader034/viewer/2022042623/547c5f37b47959ac508b4654/html5/thumbnails/93.jpg)
Graz University of Technology
SemiBoost & Visual Similarity Learning On-line Semi-supervised Boosting Semi-Supervised Random Forests On-line Manifold Regularization Conclusion & Outlook
References
BooksI O. Chapelle and B. Schoelkopf and A. Zien, “Semi-Supervised Learning”, The MIT Press, 2006
I Xiaojin Zhu and Andrew B. Goldberg, “Introduction to Semi-Supervised Learning”, Morgan & Claypool, 2009
Papers and ArticlesI C. Leistner, A. Saffari and H. Bischof, “MILForests: Multiple-Instance Learning with Randomized
Trees”,ECCV’10I C. Leistner, A. Saffari, J. Santner and H. Bischof: “,Semi-Supervised Random Forests”,ICCV’09I C. Leistner, A. Saffari, P.M. Roth and H. Bischof: “On Robustness of On-line Boosting – A Competitive
Study”,(ICCV) OLCV’09
I H. Grabner, C. Leistner and H. Bischof: “On-line Semi-Supervised Boosting for Robust Tracking”,ECCV’08
I B. Zeisl, C. Leistner, A. Saffari and H. Bischof: “On-line Semi-supervised Multiple-Instance Boosting”,CVPR’10
I C. Leistner, “Semi-Supervised Ensemble Methods for Computer Vision”, PhD-Thesis, Graz University ofTechnology, 2010
I A. Saffari, C. Leistner, M. Godec, J. Santner and H. Bischof, “On-line Random Forests”, (ICCV) OLCV’09I A. Saffari, C. Leistner, M. Godec and H. Bischof, “Robust Multi-View Multi-Class Boosting with Priors”,ECCV’10
I B. Kveton, M. Valko, M. Philipose and L. Huang, “Online Semi-Supervised Perception: Real-Time Learningwithout Explicit Feedback”, (CVPR) OLCV’10
I A. Saffari, C. Leistner and H. Bischof, “Regularized Multi-Class Semi-Supervised Boosting”,CVPR’09
I C. Leistner, H. Grabner and H. Bischof, “Semi-Supervised Boosting using Visual Similarity Learning”,CVPR’08
I A. Saffari, C. Leistner and H. Bischof, “Regularized Multi-Class Semi-Supervised Boosting”,CVPR’09
I A. Saffari, H. Grabner and H. Bischof, “SERBoost: Semi-supervised Boosting with ExpectationRegularization”,ECCV’08
Amir Saffari,Christian Leistner,Horst Bischof Semi-Supervised Learning in Computer Vision Part II