Post on 04-Jul-2020
Data Association Aware Semantic Mapping andLocalization via a Viewpoint-Dependent Classifier
Model
V. Tchuiev, Y. Feldman and V. Indelman
Technion - Israel Institute of Technology
vovatch@campus.technion.ac.ilyurif@cs.technion.ac.il
vadim.indelman@technion.ac.il
November 7, 2019
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 1 / 14
Introduction
We propose an object based SLAMapproach.
Classification is a key component for object based SLAM.
Key challenge: operation in perceptually aliased environments.Classification aliasingData Association (DA) aliasing
0 10 20 30 40 50
X coordinate [m]
0
10
20
30
40
50
Y c
oord
inate
[m
]
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1
1Right image from: Pathak, Shashank, Antony Thomas, and Vadim Indelman. ”A unified framework for data associationaware robust belief space planning and perception.” The International Journal of Robotics Research 37, no. 2-3 (2018): 287-315.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 2 / 14
Related Work
Existing works:
Consider most likely class semantic measurements.(Mu et al. 2016, Bowman et al. 2017)
Utilize a viewpoint dependent classifier with DA solved:(Velez et al. 2012, Teacy et al. 2015, Feldman and Indelman 2018)
Consider hypotheses for DA only:(Pathak et al. 2018)
Our work:
Considers semantic measurements of class probability vectors.
Uses a viewpoint dependent classifier model for DA-aware semantic SLAM.
Considers joint DA and classification hypotheses.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 3 / 14
Contribution
We utilize the coupling between classifier outputs and relativeviewpoint between object and camera to:
1 Assist in data association (DA) disambiguation.
2 Improve accuracy and reduce uncertainty of pose inference for therobot and the objects in the scene.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 4 / 14
Definitions and Formulation
We aim to maintain the hybrid belief:
P(Xk ,C , β1:k |Hk)
Xk : all robot and object poses.
xk : robot pose at time k .
C : class hypothesis of all objects.
βk : data association realization.
zgeok : geometric measurement of an object.
z semk : class probability vector of an object.
Z geok
.= {zgeok }, Z sem
k.
= {z semk }: geometric and semantic measurements.
Hk.
= {Z geo1:k ,Z
sem1:k , a0:k−1}: measurement history.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 5 / 14
Assumptions
A single robot within a static environment.
A known number of objects.
Models: motion P(xk |xk−1, ak−1), geometric P(Z geok |Xk , βk), and
classifier P(Z semk |Xk ,C , βk) are Gaussian.
The object observation modelP(βk |x rel) determines if DArealization is feasible givenrelative pose.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 6 / 14
The Classifier Model
zsemk ∈ RM is viewpoint dependent.
0 10 20 30 40 50
X coordinate [m]
0
10
20
30
40
50
Y c
oord
inate
[m
]
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
1 2 30.0
0.5
1.0
Cla
ss P
robabilit
y
The model is assumed Gaussian P(zsemk |c , x rel) = N (hc ,Σc), wherehc and Σc depend on object class c and relative pose x rel .
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 7 / 14
General Approach
Split the hybrid belief to continuous and discrete parts:
P(Xk ,C , β1:k |Hk) = P(Xk |C , β1:k ,Hk)︸ ︷︷ ︸bCβ1:k
[Xk ]
P(C , β1:k |Hk)︸ ︷︷ ︸wCβ1:k
bCβ1:k[Xk ] is the continuous belief given a class and DA realization.
wCβ1:k
is the weight of bCβ1:k[Xk ], computed separately for each C and
β1:k .
We keep all continuous beliefs with large enough weights.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 8 / 14
Continuous Belief and Weights
Continuous belief update:
bCβ1:k[Xk ] ∝ bCβ1:k−1
[Xk−1]P(xk |xk−1, ak−1)P(Zk |Xk ,C , βk)
Weight update:
wCβ1:k
∝ wCβ1:k−1
∫Xk
P(βk |Xk)bCβ1:k[Xk ]dXk
Small weights are pruned to keep the number of realizations small.
Viewpoint dependent classifier model in P(Zk |Xk ,C , βk) assists ininference DA, and reduces number of realizations when pruned.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 9 / 14
Experiment
Comparison between without and with classifier model.
Highly aliased scenario with 6 identical objects with differentorientations.
Uninformative prior on initial robot pose, causing multiple probablehypotheses.
Figure: Time k = 1, without (left) and with (right) classifier model
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 10 / 14
Experiment
Figure: Time k = 15, without (left) and with (right) classifier model
=⇒ With classifier:
fewer belief components
More accurate localization
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 11 / 14
Experiment
Figure: Time k = 15, without (left) and with (right) classifier model
=⇒ With classifier:
fewer belief components
More accurate localization
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 11 / 14
Experiment
Figure: Weight comparison, times k = 1 (left) and k = 15 (right)
=⇒ With classifier:
fewer belief components
stronger disambiguation
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 12 / 14
Experiment
Figure: Weight comparison, times k = 1 (left) and k = 15 (right)
=⇒ With classifier:
fewer belief components
stronger disambiguation
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 12 / 14
Conclusion and Summary
We propose an approach addressing DA and classificationambiguity that maintains a hybrid belief.
We utilized the coupling between object class and relative viewpointvia a viewpoint dependent classifier model.
Performance improvement in a highly aliased scenario wasdemonstrated for disambiguation and localization.
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 13 / 14
Thank you for listening!
V. Tchuiev, Y. Feldman and V. Indelman IROS 2019 Presentation November 7, 2019 14 / 14