Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual...
-
Upload
phoebe-rogers -
Category
Documents
-
view
220 -
download
1
Transcript of Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual...
![Page 1: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/1.jpg)
Evaluation of Research Theme CogB
![Page 2: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/2.jpg)
Objectives
• LEAR: LEArning and Recognition in vision
• Visual recognition and scene understanding– Particular objects and scenes– Object classes and categories– Human motion and actions
• Strategy : Robust image description + learning techniques
![Page 3: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/3.jpg)
Axes
• Robust image description – Appropriate descriptors for objects and categories
• Statistical modeling and machine learning for vision– Selection and adaptation of existing techniques
• Visual object recognition and scene understanding– Description + learning
![Page 4: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/4.jpg)
Overview
• Presentation of the team
• Positioning within INRIA and internationally
• Progress towards initial goals
• Main scientific contributions
• Future – next four years
![Page 5: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/5.jpg)
Team
• Creation of the LEAR team in July 2003
Permanent researchersContract engineers
Total
0
2
4
6
8
10
12
14
16
18
July 2003Nov 2005
July 2003
Nov 2005
![Page 6: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/6.jpg)
Positioning in INRIA
• Main INRIA strategic challenge:
Developing multimedia data and multimedia information processing
• The only INRIA team with object recognition as its central goal
• Expertise in image description and applied learning
![Page 7: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/7.jpg)
INRIA teams with related themes
• Imedia: indexing, navigation and browsing in large multi-media data streams
• TexMex: management of multi-media databases, handling large data collections and developing multi-media and text descriptors
• Vista: analysis of image sequences, motion descriptors
• Ariana: image processing for remote sensing
![Page 8: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/8.jpg)
International positioning
• In France and Europe: a few groups work on the problem (Amsterdam, Oxford, Leuven, TU Darmstadt)
• In the US: several groups use machine learning for visual recognition (CMU, Caltec, MIT, UBC, UCB, UCLA, UIUC)
• Competitive results compared to the above groups in– Image description (scale and affine invariant regions)– Classification and localization of object categories; winner of 14
out of 18 tasks of the PASCAL object recognition challenge– Learning-based human motion modeling
![Page 9: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/9.jpg)
Progress towards initial goals
• LEAR was created two and a half years ago
• Significant progress towards each goal, especially– Category classification and detection– Machine learning
• Scientific production – Publications (65 journals, conferences & books in 3 years, mainly
in the most competitive journals and conferences)– Software, databases available on our web page
• Collaborations (INRIA team MISTIS, UIUC in the US, ANU in Australia, Oxford, Leuven, LASMEA Clermont-Ferrand …)
![Page 10: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/10.jpg)
Progress towards initial goals
• Industrial contracts (MBDA, Bertin technologies,Thales Optronics, Techno-Vision project Robin)
• Research contracts (French grant ACI “Large quantities of data” MoviStar, EU network PASCAL, EU project AceMedia, EU project CLASS, EADS and Marie Curie postdoctoral grants)
• Scientific organization (Editorial boards of PAMI and IJCV; program committees/area chairs of all major computer vision conferences; organization of ICCV’03 and CVPR’05; vice-head of AFRIF; co-ordination of EU project CLASS, Techno-Vision project Robin and ACI MoviStar)
![Page 11: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/11.jpg)
Main contributions - overview
• Image descriptors– Scale- and affine-invariant detectors + descriptors– Local dense representations– Shape descriptors – Color descriptors
• Learning– Clustering– Dimensionality reduction– Markov random fields – SVM kernels
![Page 12: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/12.jpg)
Main contributions - overview
• Object recognition– Texture recognition– Bag-of-features representation– Spatial features (semi-local parts, hierarchical spatial model)– Multi-class hierarchical classification– Recognition with 3D models– Human detection
• Human tracking and action recognition– Learning dynamical models for 2D articular human tracking– 3D human pose and motion from monocular images
![Page 13: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/13.jpg)
Invariant detectors and descriptors
• Scale and affine-invariant keypoint detectors [IJCV’04]
– Matching in the presence of large viewpoints changes
![Page 14: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/14.jpg)
Invariant detectors and descriptors
• Evaluation of detectors and descriptors [PAMI’05, IJCV’06]
– Database with different scene types (textured and structured) and transformations
– Definition of evaluation criteria– Collaboration with Oxford, Leuven, Prague
• Database and binaries available on the web – 4000 access and 1000 downloads
![Page 15: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/15.jpg)
Image retrieval demonstration
![Page 16: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/16.jpg)
Image retrieval demonstration
![Page 17: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/17.jpg)
Dense representation
• Dense multi-scale local descriptors [ICCV’05]• Still local, but captures more of the available information• Clustering to obtain representative features
– our clustering algorithm deals with very different densities
• Feature selection determines the most characteristic clusters
![Page 18: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/18.jpg)
Bag-of-features for image classification
Classification
SVM
Extract regions Compute descriptors
Find clusters and frequencies
Compute distance matrix
![Page 19: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/19.jpg)
Bag-of-features for image classification
• Excellent results in the presence of background clutter
• Our team won all image classification tasks of the PASCAL network challenge on visual object recognition
bikes books building cars people phones trees
![Page 20: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/20.jpg)
Recognition with spatial relations
A
Approach [ICCV’05]:• Semi-local parts: point regions and similar geometric
neighborhood structure• Validation, i.e. part selection• Learn a probabilistic model of the object class
(discriminative maximum entropy framework)
![Page 21: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/21.jpg)
Recognition with spatial relations
Improved recognition for classes with structure
![Page 22: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/22.jpg)
Human detection [CVPR’05]
Histogram of oriented image gradients as image descriptor
SVM as classifier, importance weighted descriptors
Winner of the PASCAL challenge on human detection
![Page 23: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/23.jpg)
Human detection
![Page 24: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/24.jpg)
Evaluation of category recognition
• Techno-Vision project Robin (2005-2007)– Funded by the French ministries of defence and of research
• Construction of datasets and ground truth– Industrial partnership with MBDA, SAGEM, THALES, Bertin Tech,
Cybernetix, EADS and CNES– Production of six datasets with thousands of annotated images,
from satellite images to ground level images
![Page 25: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/25.jpg)
Evaluation of category recognition
• Evaluation metrics for category classification and localization in collaboration with ONERA and CTA/DGA
• Organization of competitions in 2006, 38 registered participants (research teams) at the moment
• Datasets, metrics and evaluation tools will be publicly available for benchmarking
![Page 26: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/26.jpg)
Learning based human motion capture
learning
[CVPR’04, ICML’04, PAMI’06], best student paper at the Rank Foundation Symposium on Machine Understanding of People
![Page 27: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/27.jpg)
Learning based human motion capture
![Page 28: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/28.jpg)
Future – next four years
• The major objectives remain valid
• Image description [low risk]– Learn image descriptors [PhD of D. Larlus]– Shape descriptors [postdoc of V. Ferrari]– Color descriptors [postdoc of J. Van de Weijer]– Spatial relations [PhD of M. Marszalek]
• Learning [medium risk]– Semi- & unsupervised learning, automatic annotation– Hierarchical structuring of categories– Existing collaborations, EU project CLASS, postdoc of J. Verbeek
![Page 29: Evaluation of Research Theme CogB. Objectives LEAR: LEArning and Recognition in vision Visual recognition and scene understanding –Particular objects.](https://reader036.fdocuments.net/reader036/viewer/2022062721/56649f205503460f94c38ee9/html5/thumbnails/29.jpg)
Future – next four years
• Object recognition– Object detection & localization [low risk]– Large number of object categories [medium risk]– Scene interpretation [high risk]
• Human modeling and action recognition– Pose & motion for humans in general conditions [PhD A. Agarwal]– Recognition of actions and interactions