Laboratório de Óptica e Mecânica Experimental

A Movement Tracking Management Model A Movement Tracking Management Model with Kalman Filtering Globalwith Kalman Filtering Global Optimization Optimization

Techniques and Mahalanobis DistanceTechniques and Mahalanobis Distance

Raquel Ramos Pinho, João Manuel R. S. Tavares, Miguel Velhote Correia

Loutraki, 21–26 October 2005

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 2

Contents• Introduction;• Kalman Filter;• Incorporation of new data:

– Simplex Method;– Mahalanobis Distance;– Marker occlusion or appearance during movement tracking.

• Management Model;• Experimental Results;• Conclusions and Future work.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 3

Introduction• Motion capture video systems and interactive modelling systems can help automatic analysis and diagnosis of objects movement;• Many tracking applications may require tracking of several objects simultaneously, and involve problems as their (dis)appearance of the scene;• To track objects we used:

– A Kalman filter;– Optimization Techniques;– Mahalanobis Distance;– A Management Model.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 4

Kalman Filter• Optimal recursive Bayesian stochastic method;• In this work:

– the system state in each time step is the set of positions, velocities and accelerations of the tracked features (points);

– new measurements are incorporated whenever a new image frame is evaluated.

• One of its drawbacks is the restrictive assumption of Gaussian posterior density functions at every time step;

– Many tracking problems involve non-linear movement, human gait is just an example.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 5

Mahalanobis Distance and Optimization• For each position estimate there may exist at most one new measurement to correct its predicted position.• With Kalman’s usual approach the predicted search area for each tracked feature is given by an ellipse (whose area will decrease as convergence is obtained and vice-versa).

– Some problems:• there may not exist any feature in the search area or, as the

opposite, there might be several;• even if there is only one correspondence for each feature,

there is no guarantee that the best set of correspondencesis achieved.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 6

Mahalanobis Distance and Optimization• We propose the use of optimization techniques to obtain the best set of correspondences between the predictions and the measures;• To establish the best global set of correspondences we used the Simplex method;• The cost of each correspondence is given by the Mahalanobis distance.

Simplex Method:• Iterative algebraic procedure used to determine at least one optimal solution for each assignment problem;• Assignment formulation: one estimate = one measure.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 7

Mahalanobis Distance and OptimizationMahalanobis Distance:• Distance between two features is normalized by its statistical variations;• The Mahalanobis distance values will be inversely proportional to the quality of the prediction/measure correspondence; thus to optimize the correspondences, we minimize this cost function.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 8

Mahalanobis Distance and OptimizationOcclusion/Appearance:• Assignment restriction (1 to 1) not satisfied – problem solved with addition of fictitious variables:

– Features matched with fictitious variables are considered unmatched;

– Unmatched predicted position - it is assumed that the feature has been occluded, but the tracking process is maintained by including its predicted position in the measurement vector although with higher uncertainty;

– Unmatched measurement - we consider it as a new feature and initialize its tracking process.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 9

Management Model• When a feature disappear of the scene: Is it just occluded? It was removed definitively? Should we keep its tracking?• This decision is of greater importance if many features are being tracked, if the image sequence is long, if the tracking is in real-time, etc;• We use a management model in which a confidence value is associated to each feature:

– In each frame, if the feature is visible then the confidence value is increased, else it is decreased;

– If a minimum value of the confidence value is reached then is considered that the feature has definitively disappeared and itstracking will cease (if it reappears, its tracking will be initialised);

– In this work, the confidence values are integers between 0 and 5, and initialized as 3.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 10

Experimental Results•Synthetic data:

– Points A+B translated horizontally and C+D rotated:

Prediction Uncertainty Area Measurement Correspondence Results

A

B

C

D

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 11

Experimental Results (Cont.)

A

BC

D

Prediction Uncertainty Area Measurement Correspondence Results

– Continuation. ...Now, Points C+D inverted the rotation angle:

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 12

• Management of the tracking features - blobs (dis)appear randomly:Experimental Results (Cont.)

AB

C

DE

Steps“Blobs” 0 1 2 3 4

A - 3 4 5 5B 3 4 5 5 5C 3 4 5 5 5D 3 4 3 4 5E - 3 2 3 4

Prediction Uncertainty Area MeasurementCorrespondence Result

Confidence Values:

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 13

– Tracking 6 markers in human gait:

Experimental Results (Cont.)

1

23

4

5

•Real data:

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 14

Squared difference between predictions and measures:

Experimental Results (Cont.)

StepMarker 1 2 3 4 5 6

1

2

3

4

5

6

1.2 - -0.91.2 2

2.8

2

1.1 0.7 1.3

2.2

2.8

2.2

3.3

0.8 1.0 0.2 0.9

1.3

0.9

0.1

1.3

- 1

0.1

2.2

0.8 0.8 0.8

0.0 0.0 0.0

0.3 1.0 1.5 Average=1.2

•Real data:

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 15

- Tracking 5 markers in human gait:

Experimental Results (Cont.)

Prediction Uncertainty Area Measurement Correspondence Result

•Real data:

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 16

- Tracking persons in a shopping centre: Experimental Results (Cont.)

(5 frames interval)

•Real data:

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 17

Conclusions and Future Work• We proposed a methodology to track feature points along

image sequences based on:– a Kalman filter;– optimization techniques;– Mahalanobis distance;– a Management Model;

• With our approach, the best set of correspondences is guaranteed (with respect to the used cost function!);

• The used management model allows the tracking in continuous image sequences in real-time, as the features simultaneously tracked are continuously update.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 18

Conclusions and Future WorkFuture Work:• Consideration of other stochastic methods;• Adoption of matches one to several (and vice-versa);• Use the proposed tracking methodology in, for example,

human gait analysis.

A Movement Tracking Management Model with Kalman Filtering, Global Optimization Techniques and Mahalanobis Distance

Introduction Kalman Filter Mahalanobis Distance and Optimization Management Model Results Conclusions

Raquel R. Pinho, João Manuel R. S. Tavares, Miguel V. Correia 19

Acknowledgments• The first author would like to thank the support of the PhD

grant SFRH / BD / 12834 / 2003 of the FCT - Fundaçãopara a Ciência e a Tecnologia in Portugal

• This work was partially done in the scope of the project “Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles”, reference POSC/EEA-SRI/55386/2004, financially supported by FCT in Portugal.

União Europeia FEDER

Governo da República Portuguesa

A Movement Tracking Management Model with Kalman Filtering Global Optimization Techniques and Mahalanobis DistanceContentsIntroductionKalman FilterMahalanobis Distance and OptimizationMahalanobis Distance and OptimizationMahalanobis Distance and OptimizationMahalanobis Distance and OptimizationManagement ModelExperimental ResultsExperimental Results (Cont.)Experimental Results (Cont.)Experimental Results (Cont.)Experimental Results (Cont.)Experimental Results (Cont.)Experimental Results (Cont.)Conclusions and Future WorkConclusions and Future WorkAcknowledgments

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

/Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure true /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /NA /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /LeaveUntagged /UseDocumentBleed false >> ]>> setdistillerparams> setpagedevice