CVPR 2006 New York City

Granularity and Elasticity Adaptation in

Visual Tracking

Ming Yang, Ying WuNEC Laboratories

AmericaCupertino, CA 95014

myang@sv.nec-labs.com

EECS Dept. Northwestern Univ.Evanston, IL 60208

yingwu@ece.northwestern.edu

CVPR 2008 Anchorage, Alaska 2

Motivation General targets exhibit enormous

variability and unpredictable changes.– rotation and scale changes– different degrees of deformations– partial occlusions

Most observation models tend to focus on certain characteristics of targets.

Adaptation of more aspects of target observation models is preferable.

CVPR 2008 Anchorage, Alaska 3

Appearance-based visual tracking

Two key aspects in designing appearance based observation models:– the abstraction level of features, – how to take into account the

geometrical structures of targets.

CVPR 2008 Anchorage, Alaska 4

Granularity vs. Elasticity Feature Granularity: the abstraction

level of features. – e.g. features describe attributes of a

pixel, a blob region or a whole object. Model Elasticity: the ability that the

model can tolerate geometrical changes among components.

CVPR 2008 Anchorage, Alaska 5

Comparisons

Comparisons of different tracking approaches in terms of their relative granularity and elasticity.

Granularity

Elasticity

super pixels SSD matching

bag of feature points

bag of patches

Mean-shift

( Appearance )

( Structure )

CVPR 2008 Anchorage, Alaska 6

The paradigm

We propose a general tracking paradigm.

The target is represented by a MRF of interest regions.

Adaptation of the feature granularity and model elasticity to maximize the likelihood of the MRF.

Featureextraction

Feature granularityadaptatoin

Tracking results

Coarse motionparameter estimation

Refinement byhypothesis testing

Interest regiondetection

Model elasticityadaptation

Input frames

CVPR 2008 Anchorage, Alaska 7

Target observation model

An MRF model of interest regions– X={xi}: the initial interest regions

– Y={yi}: the detected interest regions in every frame

Substantialize to different models– Every pixel is an interest region => Template– The target is one interest region => Meanshift

x1

x2 xi

y1

yiy2

CVPR 2008 Anchorage, Alaska 8

Target model construction Harris-Laplace interest region

detection– Represented by the location,

characteristic scale, and shape matrix

MRF model: pair-wise potential among overlapped interest regions.

{ , , , }ii i i I iu v S x

CVPR 2008 Anchorage, Alaska 9

Model the granularity and elasticity

The pair-site potential is defined based on the relative angles.

– The parameter models the elasticity. The likelihood of individual interest

region is defined using the Bahattachaya coefficient of feature histograms

– The scale ratio r regulates the image region to extract features so as to models the granularity.

CVPR 2008 Anchorage, Alaska 10

Motion estimation Coarse motion estimation

– The motion parameters are estimated independently based on the detected pair-wise cliques.

Motion parameters refinement– Jointly sample the motion parameters

and evaluate the posteriors of the hypotheses

( | ) ( ) ( | )i ii

P P P X Y X y x

CVPR 2008 Anchorage, Alaska 11

Feature granularity adaptation

Update the scale ratio by searching rt until a local maximum of

Rigid and stable targets => large ratio r can yield good matching

Partial occlusion or deformation happens => small ratio r may be appropriate.

CVPR 2008 Anchorage, Alaska 12

Model elasticity adaptation

Update the parameter in the pair-site potential function by maximizing the likelihood of the current tracking result:

The optimal is the variance of the observed angle differences.

CVPR 2008 Anchorage, Alaska 13

Experiment settings Up to 12 integration scales used in

Harris-Laplace interest region detection.

Features for the interest regions are 2D histograms in Normalized-RG space with 24*24 bins.

Interest regions matching:

Runs at 2-10 fps on a Pentium 3GHz desktop.

0( ( ), ( )) 0.75ti iH r H r T x y

CVPR 2008 Anchorage, Alaska 14

Illustration

CVPR 2008 Anchorage, Alaska 15

More tracking results

CVPR 2008 Anchorage, Alaska 16

Conclusion

A novel perspective of adapting

target observation models.

– able to automatically tune the

observation model’s focus on target’s

appearances and structures.

– flexible to incorporate different interest

region detection and features

extraction.