COMP9517ComputerVision

Tracking

5/09/16 1COMP9517S2,2016

Mo@onTracking•  Trackingistheproblemofgenera@nganinferenceaboutthe

mo@onofanobjectgivenasequenceofimages–  Whatdoweinferaboutanobject’sfutureposi@onfromasequenceof

measurements?

•  Applica@ons–  Mo@oncapture

•  Controlacartoon•  Modifythemo@onrecordtoobtainslightlydifferentmo@ons

–  Recogni@onfrommo@on•  Determinetheiden@tyoftheobject•  Tellwhatitisdoing

–  Surveillance•  Monitorac@vi@esandgiveawarningifitdetectsaproblemcase

–  Targe@ng•  DecidewhattoshootandhiRngit

5/09/16 2COMP9517S2,2016

Mo@onTracking•  Whenmovingpointsarenottaggedwithuniquetextureorcolourinforma@on,thecharacteris@csofthemo@onitselfmustbeusedtocollectpointsintotrajectories

•  Assump@on:–  Theloca@onofanobjectchangessmoothlyover@me–  Thevelocity(speedanddirec@on)ofanobjectchangessmoothlyover@me

–  Anobjectcanbeatonlyoneloca@oninspaceatagive@me

–  Twoobjectscannotoccupythesameloca@onatthesame@me

5/09/16 3COMP9517S2,2016

TrackingwithDynamicModels•  Trackingcanbeconsideredastheproblemofgenera@ngan(probabilis@c)inferenceaboutthemo@onofanobjectgivenasequenceofimages

•  Trackingisproperlythoughtofasanprobabilis@cinferenceproblem–  Themovingobjecthasinternalstate,whichismeasuredateachframe

– Measurementsarecombinedtoes@matethestateoftheobject

5/09/16 4COMP9517S2,2016

TrackingwithDynamicModels•  State:therepresenta@onofanobjectat@me(frame)t–  Posi@on–  Transforma@onparameters–  Class–  Etc

•  Measurement:theobserva@on–  Colour–  Texture–  Etc

5/09/16 5COMP9517S2,2016

TrackingwithDynamicModels•  Given–  Xi:thestateoftheobjectattheithframe–  Yi:themeasurementobtainedintheithframe

•  Therearethreemainproblems:–  Predic@on:predictthestatefortheithframebyhavingseenasetofmeasurementsy0,y1,…,yi-1

–  Dataassocia@on:selectthemeasurementsthatarerelatedtotheobject’sstate

–  Correc@on:correctthepredictedstatebyobtainedrelevantmeasurementsyi

5/09/16 6COMP9517S2,2016

),...,,|( 111100 −− === iii yYyYyYXP

),...,,|( 1100 iii yYyYyYXP ===

TrackingwithDynamicModels•  IndependenceAssump@ons(MarkovAssump@on)–  Onlytheimmediatepastma`ers

– Measurementsdependonlyonthecurrentstate

–  Theseassump@onsmeanthatatrackingproblemhasthestructureofinferenceonahiddenMarkovmodel

5/09/16 7COMP9517S2,2016

)|,...,()|()|,...,,( ikjiiikji XYYPXYPXYYYP =

)|(),...,,|( 1121 −− = iiii XXPXXXXP

Xi-1

Yi-1

Xi

Yi

Xi+1

Yi+1

LinearDynamicModels•  LinearDynamicModels–  Arandomprobabilitydistribu@onwithmeanandcovariance

–  Thestateisadvancedbymul@plyingitbysomeknownmatrixandthenaddinganormalrandomvariable

–  Themeasurementisobtainedbymul@plyingthestatebysomematrixandthenaddinganormalrandomvariableofzeromeanandknowncovariance

5/09/16 8COMP9517S2,2016

).,(~ ∑µNx

).,(~ 1 idiii xDNx ∑−

).,(~imiii xMNy ∑

ProbabilityDensityPropaga@on

5/09/16 9COMP9517S2,2016

ProbabilityDensityPropaga@on

5/09/16 10COMP9517S2,2016

TrackingwithDynamicModels•  TrackingasInference–  Predic@on

–  Correc@on

5/09/16 11COMP9517S2,2016

111011

111011101

11101110

),...,,|()|(

),...,,|(),...,,,|(

),...,,|,(),...,,|(

−−−−

−−−−−

−−−−

∫∫

∫

=

=

=

iiiii

iiiiii

iiiiii

dXyyyXPXXP

dXyyyXPyyyXXP

dXyyyXXPyyyXP

iiiii

iiii

i

iiiii

i

iiiiii

i

iiii

dXyyyXPXyPyyyXPXyP

yyyPyyyPyyyXPXyP

yyyPyyyPyyyXPyyyXyP

yyyPyyyXPyyyXP

∫ −

−

−−

−−−

=

=

=

=

),...,,|()|(),...,,|()|(

),...,,(),...,,(),...,,|()|(

),...,,(),...,,(),...,,|(),...,,,|(

),...,,(),...,,,(),...,,|(

110

110

10

110110

10

110110110

10

1010

TrackingwithDynamicModels•  Someconvenienttransforma@ons

5/09/16 12COMP9517S2,2016

),0;(),;( vxgvxg µµ −=

),;(),;( vmngvnmg =

)/,/;(),;( 2avaxgvaxg µµ =

),;(),0;(),;( baba vvxgduvugvuxg +=−∫∞

∞−µµ

),,,(),;(),;(),;( dcbafdb

bddbcdadxgdcxgbaxg

++

+=

TrackingwithDynamicModels•  KalmanFiltering–  Dynamicmodel:

–  Define:

–  Predic@on:

5/09/16 13COMP9517S2,2016

);,(~)|( 211 idiiii xdNXXP σ−−

).,(~)|( 2imiiii xmNXYP σ

21

21

21

21

12

1112

1

111011110

)(,

))(,;(

))(,;(),;(

),...,,|()|(),...,,|(

+−

−+−

−

+−

+−

−+−

+−−−

−−−−−

+==⇒

+∝

∝

=

∫

∫

iidiiii

iidiii

iiiidiii

iiiiiii

dXdX

dXdXg

dXXXgXdXg

dXyyyXPXXPyyyXP

i

i

i

σσσ

σσ

σσ

).,...,,|(__var),,...,,|(__ 1010 iiiiii yyyXPofianceyyyXPofmeanX −−++

σ

).,...,,|(__var),,...,,|(__ 110110 −

−

−

−−− iiiiii yyyXPofianceyyyXPofmeanX σ

TrackingwithDynamicModels•  KalmanFiltering–  Correc@on

5/09/16 14COMP9517S2,2016

⎟⎟⎠

⎞⎜⎜⎝

⎛

+=⎟

⎟

⎠

⎞

⎜⎜

⎝

⎛

+

+=⇒

∝

∝

=

−

−+

−

−−

+

−−

−

−

−

∫

222

22

222

22

22

110

110

11010

)()(

,)()(

))(,;(),;(

),...,,|()|(

),...,,|()|(),...,,|()|(),...,,|(

iim

imi

iim

iiimi

i

iiimiii

iiii

iiiii

iiiiii

mmymX

X

XXgXmyg

yyyXPXyP

dXyyyXPXyPyyyXPXyPyyyXP

i

i

i

i

i

σσ

σσσ

σσ

σσ

σσ

TrackingwithDynamicModels•  KalmanFiltering–thealgorithm(1-D)–  StartAssump@on: areknown–  UpdateEqua@on:Predic@on

–  UpdateEqua@on:Correc@on

5/09/16 15COMP9517S2,2016

−−

00 ,σx

21

2

1

)( +−

−

+−

−

+=

=

iidi

iii

d

xdx

iσσσ

⎟⎟⎠

⎞⎜⎜⎝

⎛

+=

⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

+

+=

−

−+

−

−−

+

222

22

222

22

)()(

)()(

iim

imi

iim

iiimii

m

mymx

x

i

i

i

i

σσ

σσσ

σσ

σσ

TrackingwithDynamicModels•  KalmanFiltering–thealgorithm(2-D)–  StartAssump@on: areknown–  UpdateEqua@on:Predic@on

–  UpdateEqua@on:Correc@on

5/09/16 16COMP9517S2,2016

−−Σ00 ,x

iiidi

iii

DDxDx

i

+−

−

+−

−

+Σ=

=

1

1

σσ

[ ][ ]

[ ] −+

−−+

−−−

Σ−=Σ

−+=

Σ+ΣΣ=

iiii

iiiiii

mTiii

Tii

MKIdxMyKxx

MMMKi

1

TrackingwithDynamicModels•  Par@cleFiltering

–  Anon-lineardynamicmodel–  Alsoknownvariouslyas:

•  Sequen@alMonteCarlomethod,•  bootstrapfiltering,•  thecondensa@onalgorithm,•  interac@ngpar@cleapproxima@onsand•  survivalofthefi`est

5/09/16 17COMP9517S2,2016

TrackingwithDynamicModels•  Par@cleFiltering

–  Thecondi@onalstatedensityat@metisrepresentedbyasetofsamplingpar@cleswithweight(samplingprobability).

–  Theweightdefinetheimportanceofasample(observa@onfrequency)

5/09/16 18COMP9517S2,2016

TrackingwithDynamicModels•  Par@cleFiltering–  Thecommonsamplingschemeisimportancesampling

•  Selec@on:selectNrandomsamples•  Predic@on:generatenewsampleswithzeromeanGaussianErrorandnon-nega@vefunc@on

•  Correc@on:computeweightscorrespondingtothenewsamplesusingthemeasurementequa@onwhichcanbemodelledasaGaussiandensity

5/09/16 19COMP9517S2,2016

TrackingwithDynamicModels•  Par@cleFiltering(A.VlakeandM.Isard,Ac@veContour)

5/09/16 20COMP9517S2,2016

TrackingwithDynamicModels•  Par@cleFiltering–thealgorithm(SIS),A.VlakeandM.Isard

5/09/16 21COMP9517S2,2016

TrackingwithDynamicModels•  Par@cleFiltering(A.VlakeandM.Isard,Ac@veContour)

5/09/16 22COMP9517S2,2016

Trackingcanbecomplex•  Lossofinforma+oncausedbyprojec@onofthe3Dworldona2Dimage,

•  Noiseinimages,•  Complexobjectmo+on,•  Non-rigidorar@culatednatureofobjects,•  Par@alandfullobjectocclusions,•  Complexobjectshapes,•  Sceneillumina+onchanges,and•  Real-+meprocessingrequirements.

5/09/16 23COMP9517S2,2016

ReferencesandAcknowledgements•  ShapiroandStockman2001•  Chapter19ForsythandPonce2003•  Chapter5Szeliski2010•  Imagesdrawnfromtheabovereferencesunlessotherwisemen@oned

5/09/16 COMP9517S2,2016 24