Yuanlu XuAdvisor: Prof. Liang Linmerayxu@gmail.com

Person Re-identification by Matching Compositional Template with Cluster

Sampling

Problem

Identifying The Same Person Under Different Cameras

Person Re-identification

Basic Assumption: 1. Face is unreliable due to view, low resolution and noises.2. People's clothes should remain consistent.

Large Intra-class Variations

Difficulty

Pose/View Variation Illumination Change Occlusion

Problem

Query Person

S vs. S M vs. S

Scene

Search

Multiple Setting

Representation

1. Body into 6 parts, limbs further into 2

symmetric parts.

2. Leaf nodes contain multiple instances.

3. Contextual relations between parts:

kinematics

symmetry.

Multiple-Instance Compositional

Template (MICT)

Problem Formulation

Given the template, the problem is

formulated as

Selecting an instance for each part.

Finding the matched part in target.

…

Matching-based Formulation

Problem Formulation

(a) Query Person (b) Test Scene

11 12 13

1112

13

2221 23 2421 22 23

24

31 32 34

33

3132

41 4244

43

41

42

(24,21)

(24,22)

(24,23)

(24,24)

(12,11)

(11,12)

(13,13)

(31,31)

(31,32)

(32,32)

(31,33)

(32,33)

(32,34) (41,41)

(41,42)

Candidacy Graph:

Vertices – possible matching pairs

Solving the problem:

Labeling vertices in the graph (selecting matching pairs)

NP hard – incorporating graph edges

Problem Formulation

(a) Query Person (b) Test Scene

11 12 13

1112

13

2221 23 2421 22 23

24

31 32 34

33

3132

41 4244

43

41

42

(31,31)

(31,32)

(32,32)

(31,33)

(32,33)

(32,34)

(12,11)

(11,12)

(13,13)

(41,41)

(41,42)

(24,21)

(24,22)

(24,23)

(24,24)

Compatible Edges:

Encouraging matching pairs to activate together in matching

Defined by contextual constraints

Problem Formulation

(31,31)

(31,32)

(32,32)

(31,33)

(32,33)

(32,34)

(12,11)

(11,12)

(13,13)

(41,41)

(41,42)

(24,21)

(24,22)

(24,23)

(24,24)

(a) Query Person (b) Test Scene

11 12 13

1112

13

2221 23 2421 22 23

24

31 32 34

33

3132

41 4244

43

41

42

Problem Formulation

Competitive Edges:

Depressing conflicting matching pairs being selected at the same time

Defined by matching constraints (31,31)

(31,32)

(32,32)

(31,33)

(32,33)

(32,34)

(12,11)

(11,12)

(13,13)

(41,41)

(41,42)

(24,21)

(24,22)

(24,23)

(24,24)

(a) Query Person (b) Test Scene

11 12 13

1112

13

2221 23 2421 22 23

24

31 32 34

33

3132

41 4244

43

41

42

Inference

Using Cluster Sampling [1] for inference:1. Sampling edges in candidacy graph to generate clusters.2. Randomly selecting/deselecting the clusters.3. Decide whether to accept the new state.

[1] J. Porway et al., “C4: Exploring multiple solutions in graphical models by cluster sampling”, TPAMI 2011.

(31,31)

(31,32)

(32,32)

(31,33)

(32,33)

(32,34)

(12,11)

(11,12)

(13,13)

(41,41)

(41,42)

(24,21)

(24,22)

(24,23)

(24,24)

State A Clusters

Cluster1

Cluster2

Re-identification

(31,31)

(31,32)

(32,32)

(31,33)

(32,33)

(32,34)

(12,11)

(11,12)

(13,13)

(41,41)

(41,42)

(24,21)

(24,22)

(24,23)

(24,24)

State B Clusters

Cluster2

Cluster1

Re-identification

Dataset

VIPeR Dataset:

1. Classic ReID dataset

2. Well-segmented people, limited pose/view3. Heavy illumination changes, lack occlusion

D. Gray et al., "Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features”, ECCV 2008.

Dataset

EPFL Dataset:

1. Cross-camera tracking dataset

2. Few people, shot scene provided, various pose/view

3. Little illumination changes, limited occlusions

F. Fleuret et al., "Multiple Object Tracking using K-Shortest Paths Optimization”, TPAMI 2011.

Query Instance Video Shot Target Individual

Dataset

CAMPUS-Human Dataset:

1. Camera and annotate by us

2. Many people, shot scene provided, various pose/view

3. Limited illumination changes, heavy occlusions

Query Instance Video Shot Target People

Result

Setting 1:

Re-identify people in segmented images, i.e. targets already localized.

Result

Setting 2:

Re-identify people from scene shots without provided segmentations.

Result

Evaluating feature and constraints effectiveness

Component Analysis

Conclusion

1. A solution for a new surveillance problem.

2. A person-based model, a graph-matching-based formulation, a more complete database for evaluation.

3. Exploring robust and flexible person models [1], efficient search method [2] in future.

[1] J. B. Rothrock et al., “Integrating Grammar and Segmentation for Human Pose Estimation”, CVPR 2013.[2] J. Uijlings et al., “Selective Search for Object Recognition”, IJCV 2013.

Published Papers

1. Yuanlu Xu, Liang Lin, Wei-Shi Zheng, Xiaobai Liu. “Human Re-identification by Matching Compositional Template with Cluster Sampling”. ICCV 2013.

2. Liang Lin, Yuanlu Xu, Xiaodan Liang, Jian-Huang Lai. “Complex Background Subtraction by Pursuing Dynamic Spatio-temporal Manifolds”. IEEE TIP 2014, under revision.

3. Yuanlu Xu, Bingpeng Ma, Rui Huang, Liang Lin. “Person Search in a Scene by Jointly Modeling People Commonness and Person Uniqueness”. ACMMM 2014, submitted.

QUESTIONS?

1. Given a candidacy graph and the current matching state , we first separate graph edges into two sets: set of inconsistent edges

and set of consistent edges in the other two cases.2. Next we introduce a boolean variable to indicate an edge is being turned on or turned off. We turn off inconsistent edges deterministically and turn on every consistent edge with its edge probability .

Cluster Sampling

Generating a composite cluster

3. Afterwards, we regard candidates connected by ”on” positive edges as a cluster and collect clusters connected by ”on” negative edges to generate a composite cluster .

Cluster Sampling

Generating a composite cluster

Composite Cluster Sampling

state transition probability ratio posterior ratio

Using Metropolis-Hastings method to achieve a reversible transition between twostates and , the acceptance rate of the transition is defined as

Composite Cluster Sampling

The state transition probability ratio is computed by

edges being turned off around ,

Composite Cluster Sampling

Inference Algorithm