Towards Open World Recognition
Abhijit Bendale, Terrance BoultUniversity of Colorado of Colorado Springs
Poster no 85
2
Multi-Class Classification System
W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013
3
Out in the Real-WorldDetect New Category
Pool Table
Bowling Pin
Boxing glove
Calculator
Chess board
?
?
?
?
4
Open World Recognition
• World with Knowns (K) &Unknowns Unknowns (UU)
Detect as Unknown
• NU: Novel Unknowns
Label Data• LU: Labeled
Unknowns
Incremental Class Learning
• K: Known
Scale
5
Open Set Learning
Incremental Learning
Scalable Learning
Ristin+ (CVPR’14)
Yeh+ (CVPR’08)
Li+ CVPR’07)
Mensink+(PAMI’13)
Related Work
Jain+ (ECCV’14)
Scheirer+ PAMI’13)
Scheirer+(PAMI’14)
Deng+ (NIPS’11)
Marszalek+(ECCV’08)
Liu+ (CVPR’13)
Open World Recognition
6
Open Space in Classification
??
?
?
?
?
? ? ??
??
??
Open Set Recognition Empirical risk
functionRegularization constant
Open space risk
W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013
Closed SpaceOpen Space
7
NCM – Metric Learning
NCM Classifier with Metric Learning
T Mensink, J Verbeek, F Perronin, G Csurka “Distance based Image Classification: Generalizing to New Classes at Near Zero Cost” IEEE TPAMI 2013M Ristin, M Guillaumin, J Gall, L Van Gool “Incremental Learning of NCM Forests for Large-Scale Image Classification” CVPR 2014
8
Compact Abating Probability (CAP) Models
W Scheirer, L Jain, T Boult “Probability Models for Open Set Recognition” IEEE TPAMI 2014
Class Mean
9
Theorem 1: Open Space Risk for Model Combination
10
Theorem 2: Open Space Risk for Transformed Spaces
11
Opening an Existing Algorithm:Nearest Non-Outlier (NNO) Algorithm
W = Linear Transformation (weight matrix from metric learning)
Standard gamma functionIn volume of m-D ball Class mean for class i
τ is threshold for open world
Probability
12
Open World Evaluation
Training phase
Testing phase
Parameter Learning Phase Incremental Learning Phase
Closed Set TestingUnknown Categories
Known Categories
Open Set Testing
? ?
13
Training for Open World
NCM - ML NNO
• Parameter Learning with initial set of categories
• Estimation of τ for open set learning to balance open space risk• Optimize for Known vs Unknown Errors• Incrementally add new categories
14
ExperimentsDatasets
• ILSVRC’10: 1.2M training images, 1000 classes• ILSVRC’12: 1.2M training images, 1000 classes
Features• Dense SIFT features, Quantized into 1000 Bag of Visual Words• Publically available features• LBP, HOG, Dense SIFT (for ILSVRC’12)
Algorithms• Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013]• Nearest Non-Outlier Algorithm (NNO) [This Paper]• 1vSet [Scheirer etal PAMI 2013]• Linear SVM [Liblinear, Fan etal JMLR 2008]
50 Initial Categories
Increasing # of unknown categories during testing i.e. increasing openness of problem
Incrementally addingcategories during training
Closed Settesting
16
200 Initial Categories
Increasing # of unknown categories during testing i.e. increasing openness of problem
Incrementally addingcategories during training
Closed Settesting
500 known + 500 unknown categories
17
• Formalized Open World Recognition and showed how to “Open” an existing algorithm.
• NNO allows construction of scalable systems that can be updated incrementally
Conclusion & Future Work
See us at Poster no 85 …!!!!
• Exploring sophisticated novelty detectors, open world detection, “opening” other baseline algorithms etc.
• Open World Deep Learning methods• Happy to Collaborate…!!!
Top Related