Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan...
-
Upload
teresa-patrick -
Category
Documents
-
view
216 -
download
0
Transcript of Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan...
![Page 1: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/1.jpg)
1
Hierarchical Subquery Evaluation for Active Learning on a Graph
Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow
CVPR 2014
University College London
![Page 2: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/2.jpg)
2
Cat
Dog
Horse
Large Image Collections
https://www.flickr.com/photos/cmichel67
![Page 3: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/3.jpg)
3
Large Image Collections
https://www.flickr.com/photos/cmichel67
Cat
Dog
Horse
Labeling large image collections is tedious
![Page 4: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/4.jpg)
4
Acquiring Annotations
https://www.flickr.com/photos/usnavy https://www.flickr.com/photos/rdecom
Crowdsourcing Specialized Knowledge
Expert time is valuable!
![Page 5: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/5.jpg)
5
Active Learning
Oracle
AL Algorithm
User Query
Label
UnlabeledDataset
![Page 6: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/6.jpg)
6
Number of user queries
TestAccuracy
1
0
Learning Curves
![Page 7: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/7.jpg)
7
Number of user queries
1
0
Learning Curves
TestAccuracy
![Page 8: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/8.jpg)
8
Number of user queries
1
0
Learning Curves
TestAccuracy
![Page 9: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/9.jpg)
9
Number of user queries
1
0
Learning Curves
TestAccuracy
![Page 10: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/10.jpg)
10
Learning Curves
Number of user queries
1
0
TestAccuracy
![Page 11: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/11.jpg)
11
Learning Curves
Number of user queries
1
0
We want the largest area under the learning curve
TestAccuracy
![Page 12: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/12.jpg)
12
Learning Curves
1
0
TestAccuracy
The number of unlabeled images can be very large!
![Page 13: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/13.jpg)
13
Active Learning Wish List
![Page 14: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/14.jpg)
14
• Fast updating of classifier for interactive labeling
Active Learning Wish List
![Page 15: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/15.jpg)
15
• Fast updating of classifier for interactive labeling• Exploit structure in unlabeled data
Active Learning Wish List
![Page 16: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/16.jpg)
16
• Fast updating of classifier for interactive labeling• Exploit structure in unlabeled data• Consistent performance across different datasets
Active Learning Wish List
![Page 17: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/17.jpg)
17
• Fast updating of classifier for interactive labeling• Exploit structure in unlabeled data• Consistent performance across different datasets• Make the most of the expert’s time
Active Learning Wish ListGraph Based
Semi-Supervised Learning
Perplexity Graph Construction
Our Hierarchical Subquery Evaluation
![Page 18: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/18.jpg)
18
Related Work
Video SegmentationFathi et al. BMVC 2011
Action DetectionBandla and Grauman ICCV 2013
Gaussian Random FieldsZhu et al. ICML 2003
Semantic SegmentationVezhnevets et al. CVPR 2012
RALF: Reinforced Active LearningEbert et al. CVPR 2012
…
Image ClassificationKapoor et al. ICCV 2007
…
![Page 19: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/19.jpg)
19
xiφ( ) =
Supervised Classification
![Page 20: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/20.jpg)
20
xjφ( ) =
Supervised Classification
![Page 21: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/21.jpg)
21
Supervised Classification
![Page 22: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/22.jpg)
22
Supervised Classification
Decision Boundary
![Page 23: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/23.jpg)
23
Semi-supervised learning using Gaussian fields and harmonic functions X. Zhu, Z. Ghahramani, J. LaffertyICML 2003
Fi = P(f(xi) == class1)
wij
Semi-Supervised Learning
![Page 24: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/24.jpg)
24
Semi-Supervised Learning
Fi = P(f(xi) == class1)
wij
![Page 25: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/25.jpg)
25
Graph Construction
Stochastic neighbor embeddingG. Hinton and S. RoweisNIPS 2002
![Page 26: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/26.jpg)
26
Graph Active Learning
![Page 27: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/27.jpg)
27
Example 2 Class Graph
![Page 28: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/28.jpg)
28
Example 2 Class Graph
Ground Truth
![Page 29: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/29.jpg)
29
Example 2 Class GraphActive Learning Strategies
![Page 30: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/30.jpg)
30
Active Learning Strategies
• Random
![Page 31: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/31.jpg)
31
Active Learning Strategies
• Random• Exploration – clusters
![Page 32: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/32.jpg)
32
Active Learning Strategies
• Random• Exploration – clusters• Exploitation – uncertainty
![Page 33: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/33.jpg)
33
Active Learning Strategies
• Random• Exploration – clusters• Exploitation – uncertainty
![Page 34: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/34.jpg)
34
Active Learning Strategies
• Random• Exploration – clusters• Exploitation – uncertainty• RALF – explore or exploit
Ralf: A reinforced active learning formulation for object class recognitionS. Ebert, M. Fritz, and B. SchieleCVPR 2012
![Page 35: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/35.jpg)
35
Active Learning Strategies
• Random• Exploration – clusters• Exploitation – uncertainty• RALF – explore or exploit• Expected Error Reduction – reduce future
error
Toward optimal active learning through sampling estimation of error reductionN. Roy and A. McCallum ICML 2001
![Page 36: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/36.jpg)
37
Expected Error Reduction
2 Labeled Points
Ground Truth
![Page 37: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/37.jpg)
38
Expected Error Reduction
Current ClassDistribution
Ground Truth
![Page 38: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/38.jpg)
39
Expected Error Reduction
Compute the Expected Error (EE) for each unlabled datapoint
Ground Truth
![Page 39: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/39.jpg)
40
Expected Error Reduction
? Hypothesize label 1
Ground Truth
Class 1 Class 2
![Page 40: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/40.jpg)
41
Expected Error Reduction
? Update model
Ground Truth
![Page 41: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/41.jpg)
42
Expected Error Reduction
? Hypothesize label 2
Ground Truth
Class 1 Class 2
![Page 42: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/42.jpg)
43
Expected Error Reduction
? Update model
Ground Truth
![Page 43: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/43.jpg)
44
Expected Error Reduction
? Compute EE
Ground Truth
![Page 44: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/44.jpg)
45
Expected Error Reduction
?
Hypothesize label 1
Ground Truth
Class 1 Class 2
![Page 45: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/45.jpg)
46
Expected Error Reduction
?
Update model
Ground Truth
![Page 46: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/46.jpg)
47
Expected Error Reduction
?
Hypothesize label 2
Ground Truth
Class 1 Class 2
![Page 47: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/47.jpg)
48
Expected Error Reduction
?
Update Model
Ground Truth
![Page 48: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/48.jpg)
49
Expected Error Reduction
?
Compute EE
Ground Truth
![Page 49: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/49.jpg)
51
Expected Error Reduction
Repeat for all unlabeled
nodes!O(N2)For Zhu et al.
Ground Truth
![Page 50: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/50.jpg)
52
Problems with EER
• Need to retrain the classifier with each unlabeled example (subquery) and for each different class label – O(N2)
At each step is it necessary to try every possible subquery?
![Page 51: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/51.jpg)
53
Active Learning Strategies
Lower Complexity
Performance RALFCVPR 2012
EERZhu 2003
Random
![Page 52: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/52.jpg)
54
Unsupervised Hierarchical Clustering
![Page 53: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/53.jpg)
Unsupervised Hierarchical Clustering
55
Authority-shift clustering: Hierarchical clustering by authority seeking on graphsM. Cho and K. Mu LeeCVPR 2010
…
![Page 54: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/54.jpg)
Unsupervised Hierarchical Clustering
56
…
![Page 55: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/55.jpg)
Unsupervised Hierarchical Clustering
57
…
![Page 56: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/56.jpg)
Unsupervised Hierarchical Clustering
58
Large clusters (exploration)
Boundary refinement (exploitation) …
![Page 57: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/57.jpg)
Our Hierarchical Subquery Evaluation
After 2 Queries
59
Ground Truth
![Page 58: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/58.jpg)
Our Hierarchical Subquery Evaluation
5.6 4.2
3.5After 2 Queries
Best EE
Next nodes to add to the active set
CurrentActive Set
60
Ground TruthRemaining Subqueries: 74
![Page 59: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/59.jpg)
Our Hierarchical Subquery Evaluation
Best EE
After 2 Queries
61
Ground Truth
6 2.15.6
3.5
4.2
Remaining Subqueries: 2
![Page 60: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/60.jpg)
Our Hierarchical Subquery Evaluation
6 2.1
3.21.1
After 2 Queries
62
Ground Truth
5.6
3.5
4.2
Remaining Subqueries: 0
![Page 61: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/61.jpg)
Our Hierarchical Subquery Evaluation
6 2.1After 3 Queries
3.21.1
Label for the example with the best EE is requested
After 2 Queries
63
Ground Truth
5.6
3.5
4.2
Remaining Subqueries: 0
![Page 62: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/62.jpg)
Our Hierarchical Subquery Evaluation
After 3 Queries
After 2 Queries
64
Ground TruthRemaining Subqueries: 72
![Page 63: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/63.jpg)
65
Results
![Page 64: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/64.jpg)
66
Results
1579 examples8 classes50 dim BoW PCA
![Page 65: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/65.jpg)
67
Results
![Page 66: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/66.jpg)
68
Results
Ralf: A reinforced active learning formulation for object class recognitionS. Ebert, M. Fritz, and B. SchieleCVPR 2012
![Page 67: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/67.jpg)
69
Results
![Page 68: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/68.jpg)
70
13 Different Computer Vision and Machine Learning Datasets
Results - Area Under Learning Curve
![Page 69: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/69.jpg)
71
13 Different Computer Vision and Machine Learning Datasets
Results - Area Under Learning Curve
![Page 70: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/70.jpg)
72
Summary
• Hierarchical graph based semi-supervised active learning O(N2) -> O(NlogN)
![Page 71: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/71.jpg)
73
Summary
• Hierarchical graph based semi-supervised active learning O(N2) -> O(NlogN)
• Robust to dataset type
![Page 72: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/72.jpg)
74
Summary
• Hierarchical graph based semi-supervised active learning O(N2) -> O(NlogN)
• Robust to dataset type • Best user query in the time available
![Page 73: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/73.jpg)
75
Future Work
• Representation learning – update graph structure during labeling
![Page 74: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/74.jpg)
76
• Representation learning – update graph structure during labeling
• Model different annotation costs
Future Work
![Page 75: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/75.jpg)
77
• Representation learning – update graph structure during labeling
• Model different annotation costs• Embed new datapoints into the graph
Future Work
![Page 76: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/76.jpg)
79
Come visit our poster 01-C-3
http://visual.cs.ucl.ac.uk/pubs/graphActiveLearning
![Page 77: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/77.jpg)
80
![Page 78: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/78.jpg)
81
Graph Construction Comparison
![Page 79: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/79.jpg)
82
Timings
![Page 80: Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.](https://reader038.fdocuments.net/reader038/viewer/2022110207/56649d745503460f94a546be/html5/thumbnails/80.jpg)
83