Tracking by Sampling Trackers Junseok Kwon* and Kyoung Mu lee Computer Vision Lab. Dept. of EECS...
-
Upload
virgil-oliver -
Category
Documents
-
view
224 -
download
0
Transcript of Tracking by Sampling Trackers Junseok Kwon* and Kyoung Mu lee Computer Vision Lab. Dept. of EECS...
Junseok Kwon* and Kyoung Mu lee
Computer Vision Lab.Dept. of EECS Seoul National University, Korea
Homepage: http://cv.snu.ac.kr
Goal of Visual Tracking
Robustly tracks the target in real-world scenarios
Frame #1 Frame #43
Bayesian Tracking Approach Maximum a Posteriori (MAP) estimate
)Y|X(pmaxarg t:1tXt
Intensity
edge
}X,X,X{X st
yt
xtt
State Sampling
MAP estimate by Monte Carlo sampling
),Y|X(pmaxarg t:1)l(
tX )l(
t
N,,1l
X position
Y po
sitio
n
Sca
le
State space
Visual tracker
Guided by
Problem of previous works Conventional trackers have difficulty in
obtaining good samples.
Visual tracker
Tra
ckin
g e
nvir
onm
ent
changes
Fixed
can not reflect the changing
tracking environment
well.
Tracker space
Tracker sampling
Our approach : Tracker Sampling Sampling tracker itself as well as state
X positionY
posit
ion
Sca
le
X positionY
posit
ion
Sca
le
Tracker
#2
Tracker
#M
X positionY
posit
ion
Sca
le Tracker
#1
State sampling
Two challenges
How the tracker space is defined?
When and which tracker should be sampled?
Tracker space
Tracker
#1
Tracker
#2
Tracker
#M
Tracker space
Challenge 1 : Tracker Space Tracker space
Nobody tries to define tracker space. Very difficult to design the space because
the visual tracker is hard to be described.
Tracker space
Bayesian Tracking Approach
)Y|X(pmaxarg t:1tXt
1t1t:11t1tttt dX)Y|X(p)X|X(p)X|Y(p
Go back to the Bayesian tracking formulation
Updating rule
Bayesian Tracking Approach
1t1t:11t1tttt dX)Y|X(p)X|X(p)X|Y(p
What is important ingredients of visual tracker?
1. Appearance model
2. Motion model
3. State representation type
4. Observation type
Appearance
model
Mot
ion
mod
el
Stat
e
repr
esen
tatio
n
Observation
Tracker Space
Motion model
)M( t
Observation type )O( t
State representation type )S( t
Appearance model )A( t
Challenge 2 : Tracker Sampling Tracker sampling
When and which tracker should be sampled ? To reflect the current tracking environment.
Tracker space
Tracker
#m
Reversible Jump-MCMC
We use the RJ-MCMC method for tracker sampling.
1tA
2tA
|A|t
tA
Add Delete
Set of sampled appearance
models
1tM
2tM
|M|t
tM
Add Delete
Set of sampled motion models
1tS
2tS |S|
ttS
Add Delete
Set of sampled state
representation types
1tO
2tO
|O|t
tO
Add Delete
Set of sampled observation types
Sampled basic trackers
1tT 2
tT |O|S||M||A|t
ttttT
Sampling of Appearance Model Make candidates using SPCA*
The candidates are PCs of the target appearance.
Appearance models
* A. d’Aspremont et. al. A direct formulation for sparse PCA using semidefinite programming. Data Min. SIAM Review, 2007.
Sparse Principle Component Analysis*
Accept an appearance model With acceptance ratio
*tA
1i
*t
1t
5tj
itjjt:1t
*t
t*tt:1tt
*ttt:1t
*t
Alog)),X(Y(DD)Y,X|A(plogwhere
)A;A(Q)Y,X|A(p
)A;A(Q)Y,X|A(p,1min
Sampling of Appearance Model
Our method has the limited number of
models
The accepted model increase the total likelihood scores for recent frames
When it is adopted as the target reference
*tA
1i
1t
5tj
itjj )),X(Y(DD
Sampling of Motion Model
Make candidates using KHM* The candidates are mean vectors of the
clusters for motion vectors.
Motion models
K-Harmonic Means Clustering (KHM)*
* B. Zhang, M. Hsu, and U. Dayal. K-harmonic means - a data clustering algorithm. HP Technical Report, 1999
Sampling of Motion Model
Accept a motion model With acceptance ratio
*tM
1i
*titt:1t
*t
t*tt:1tt
*ttt:1t
*t
Mlog),D(VAR)Y,X|M(plogwhere
)M;M(Q)Y,X|M(p
)M;M(Q)Y,X|M(p,1min
Our method has the limited number of
models
The accepted model decreases the total clustering error of motion vectors for recent frames
When it is set to the mean vector of the cluster
*tM
1iit ),D(VAR
Sampling of State Representation
Make candidates using VPE* The candidates describe the target as the
different combinations of multiple fragments.
Vertical Projection of Edge (VPE)*
EdgePosi
tion
Intensity
* F.Wang, S. Yua, and J. Yanga. Robust and efficient fragments-based tracking using mean shift. Int. J. Electron. Commun., 64(7):614–623, 2010.
State representatio
n
Fragment 1
Fragment 2
Accept a state representation type With acceptance ratio
*t
itS
1i
F
1j
*tjt:1t
*t
t*tt:1tt
*ttt:1t
*t
Slog)f(VAR)Y,X|S(plogwhere
)S;S(Q)Y,X|S(p
)S;S(Q)Y,X|S(p,1min
Sampling of State Representation
Our method has the limited number of
types
The accepted type reduce the total variance of target appearance in each fragment for recent frames
*t
itS
1i
F
1jj )f(VAR
Sampling of Observation
Make candidates using GFB* The candidates are the response of multiple
Gaussian filters of which variances are different.
Gaussian Filter Bank (GFB)*
* J. Sullivan, A. Blake, M. Isard, and J. MacCormick. Bayesian object localisation in images. IJCV, 44(2):111–135, 2001.
Sampling of Observation Accept an observation type
With acceptance ratio
*tO
1i
1t
5tk,j
ik
ij
O
1i
1t
5tk,j
ik
ij
t:1t*t
t*tt:1tt
*ttt:1t
*t
Olog
),(DD
),(DD
)Y,X|O(plogwhere
)O;O(Q)Y,X|O(p
)O;O(Q)Y,X|O(p,1min
*t
*t
Our method has the limited number of
types
The accepted type makes more similar between foregrounds, but more different with foregrounds and backgrounds for recent frames
*t
*t
O
1i
1t
5tk,j
ik
ij
O
1i
1t
5tk,j
ik
ij
),(DD
),(DD
Tracker space
Overall Procedure
X positionY
posit
ion
Sca
le
X positionY
posit
ion
Sca
le
X positionY
posit
ion
Sca
le
Tracker #1
Tracker #2
Tracker #M
Tracker sampling
State sampling
Inte
racti
on
Qualitative Results
Qualitative Results
Iron-man dataset
Qualitative Results
Matrix dataset
Qualitative Results
Skating1 dataset
Qualitative Results
Soccer dataset
Quantitative Results
MC IVT MIL VTD Ourssoccer 53 116 41 23 17
skating1 172 213 85 8 8
animal 26 21 30 22 10
shaking 98 150 38 20 5
Soccer* 72 225 147 34 24
Skating1*
126 291 87 16 8
Iron-man 78 104 122 30 15
Matrix 123 50 57 80 12Average center location errors in pixels
IVT : Ross et. al. Incremental learning for robust visual tracking. IJCV 2007.MIL : Babenko et. al. Visual tracking with online multiple instance learning. CVPR 2009.
MC : Khan et. al. MCMC-based particle filtering for tracking a variable number of interacting targets. PAMI 2005.
VTD: Kwon et. al. Visual tracking decomposition. CVPR 2010.
Summary
Visual tracker sampler New framework, which samples visual
tracker itself as well as state. Efficient sampling strategy to sample the
visual tracker.
http://cv.snu.ac.kr/paradiso