Towards Hierarchical Place Recognition for
Long-Term Autonomy
Kirk MacTavish and Timothy D. Barfoot
ICRA Workshop on Visual Place Recognition in Changing Environments
June 2014
•Why use Place Recognition?
•How can we deal with variable lighting?
•What about Long-term Autonomy?
Motivation | Overview
2
Motivation | GPS Denied Environments
3
Mines
Planetary ExplorationUrban Canyons
Motivation | What is place recognition?
Relative localization is sufficient for many
tasks.
Have we beenhere before?
Cummins, M. and Newman, P., “FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance,” The International Journal of Robotics Research, 27(6):647–665, 2008.
Perceptual Aliasing Scene & Perspective Change
4
•Why use Place Recognition?
•How can we deal with variable lighting?
•What about Long-term Autonomy?
Motivation | Overview
5
11 am 6 pm
Motivation | Images Change over Time
6
• We can get some degree of lighting invariance from color-space manipulation.
• Dealing with Shadows: Capturing Intrinsic Scene Appearance for
Image-based Outdoor Localisation (Corke et al. 2013)
• Shady Dealings: Robust, Long- Term Visual Localisation using
Illumination Invariance (McManus et al. 2014)
• Lidar intensity images are unaffected by lighting conditions over the full day-night period.
Motivation | Lighting Invariance
7
Motivation | 24 Hours of Images
Place recognition compares places after significant time has
passed. This makes lighting invariance extremely important.
8
Motivation | Discretization
Camera
Lidar
9
The discretization is less obvious
•Why use Place Recognition?
•How can we deal with variable lighting?
•What about Long-Term Autonomy?
Motivation | Overview
10
It would be interesting to design the algorithm for operation over a 10 year period.
• Algorithms should be able to run in a constant computational budget for 10 years.
• Algorithms should be able to learn and adapt to the changing environment over the 10 year period.
Motivation | Long-Term Autonomy
11
• “Highly Scalable Appearance-Only SLAM – FAB-MAP 2.0” (Cummins and Newman, 2009)
• Linear complexity but very fast
• Are We There Yet? Challenging SeqSLAM on a 3000 km Journey Across All Four Seasons (Sunderhauf, Neubert and Protzel, 2013)
• Performs well under seasonal change, but still linear complexity and is sensitive to
camera alignment
• “Appearance-Based Loop Closure Detection for Online Large-Scale and Long-Term Operation” (Labbé and Michaud, 2013)
• Constant-time, but does not consider the whole map
Motivation | Long-Term Autonomy
12
Methodology | Roadmap
Place Recognition Lighting Change
Computational
Complexity
Hierarchy
Place
Discretization
LIDAR
Two Problems
One Solution
13
•Place Hierarchy
•FAB-MAP with groups
Methodology | Overview
14
Methodology | Computational Complexity
15
•Place Hierarchy
•FAB-MAP with groups
Methodology | Overview
16
Methodology | FAB-MAP with groups
City Centre Dataset Groups of 128 images
• Used the OpenFABMAPimplementation
• Adapted Bag-of-Words (BoW) descriptor to use features from groups of images, rather than single images
• Have not yet done hierarchical expansion
17
Methodology | FAB-MAP with groups
City Centre Dataset
Groups of 128 images
18
Bag-of-Words is a histogram of discretized
visual features
Methodology | FAB-MAP with groups
City Centre Dataset
Groups of 128 images
Single Image
Larger Group
BoW descriptors became less sparse with larger groups. This invalidates training.
19
Methodology | FAB-MAP with groups
The number of times a word must be seen in a group to be counted
as present
20
Single Image
Larger Group
BoW descriptors became less sparse with larger groups. This invalidates training.
Methodology | FAB-MAP with groups
Single Image
Larger Group
Group BoW descriptors are now as sparse as single images.
Training is useful again.
The number of times a word must be seen in a group to be counted
as present
21
•Datasets
•Results on camera images
•Results on LIDAR intensity
Results | Overview
22
Results | Datasets
City Centre New College
Oxford Mobile Robotics Group
23
Results | Datasets
KITTI Vision Benchmark SuiteOdometry Dataset
24
Results | Datasets
ASRL Sudbury LIDAR Dataset
25
Available: http://asrl.utias.utoronto.ca/datasets/abl-sudbury/
Anderson A, McManus C, Dong H, Beerepoot E, and Barfoot T D.
“The Gravel Pit Lidar-Intensity Imagery Dataset”.
University of Toronto Technical Report ASRL-2012-ABL001
•Datasets
•Results on camera images
•Results on LIDAR intensity
Results | Overview
26
Results | Camera | Confusion Matrices
KITTI-06 New College City Centre
27
La
rger
Gro
up
s o
f Im
ag
es
Results | Camera | Performance
28
Single-image FABMAP
Single-image FABMAP
Results | Camera | City Centre with groups
29
•Datasets
•Results on camera images
•Results on LIDAR intensity
Results | Overview
30
Results | LIDAR | Sudbury Dataset
31
Single-image FABMAP
• Develop the hierarchical inferencing algorithm.
• Develop a deeper high-level descriptor that can describe large places better than Bag-of-Words.
• Adapt the algorithm to use an unstructured LIDAR descriptor for continuous scans.
Future Work
32
http://asrl.utias.utoronto.ca
Email:
Web:
33
Towards Hierarchical Place Recognition for Long-Term Autonomy
Top Related