Post on 25-Feb-2016
description
Representing, Learning, and Recognizing Non-Rigid Textures and Texture Categories
Svetlana Lazebnik Cordelia Schmid Jean PonceBeckman Institute Gravir Laboratory Beckman InstituteUIUC, USA INRIA, France UIUC, USA
Supported in part by the UIUC Campus Research Board, the UIUC/CNRS Collaborative Research Agreement, and the National Science Foundation under grant IRI-990709.
• 3D objects are never planar in the large,but they are always planar in the small.
• Representation: Local invariants andtheir spatial layout.
• Affine-invariant patches.
LeCun’03
(Lindeberg & Garding’97)(Mikolcajczyk & Schmid’02)
• Spatial selection • Shape selection• Affine adaption
Schaffalitzky & Zisserman (2001); Tuytelaars & Van Gool (2003)
(0,0)
(0,1)
(1,0)
Image 1 Image 2
Affine adaptation/Rectification process
Lindeberg & Garding (1997)Mikolcajczyk & Schmid (2002)Rectified patch
[Range spin images: Johnson & Hebert (1998)]
Intensity-Domain Spin Images
System architecture (Lazebnik, Schmid, & Ponce, CVPR’03)
[Signatures and EMD for image retrieval: Rubner, Tomasi, & Guibas (1998)]
• Signature: SS = { ( m1 , w1 ) , … , ( mk , wk ) }• Earth Mover’s Distance: D( SS , SS’’ ) = [i,j fij d( mi , m’j)] / [i,j fij ]
Texture retrieval/classification experiments
Schmid (2001); Varma & Zisserman (2002)
10 texture classes, with 20 samples per class.
NN classification
Ave
rage
reco
gniti
on ra
te
Ave
rage
reco
gniti
on ra
te
More retrieval/classification experiments: Brodatz database
• Picard et al. (1993, 1996)• Xu et al. (2000)
111 images divided into 9 windows
111 classes with 9 samples per class
Ave
rage
reco
gniti
on ra
te
Ave
rage
reco
gniti
on ra
te
T1 (brick) T2 (carpet) T3 (chair) T4 (floor 1) T5 (floor 2) T6 (marble) T7 (wood)
Multi-texture Samples
Texture Classes [NOTE: we do NOT use color information.]
A Two-Layer Architecture(Lazebnik, Schmid, & Ponce, ICCV’03)
Modeling:1. Use EM to learn a mixture-of-Gaussians model of
each texture class.2. Compute co-occurrence statistics of sub-class labels
over affinely adapted neighborhoods.
Recognition:1. Use the generative model to obtain initial class
membership probabilities.2. Use relaxation (Rosenfeld et al., 1976) to refine these
probabilities.
Malik, Belongie, Leung, & Shi (2001); Schmid (2001); Kumar & Hebert (2003)
Neighborhood Statistics
Estimate:• probability p(c,c’),• correlation r(c,c’).
Relaxation (Rosenfeld et al., 1976)
Iterate, for all regions i:
where
and wij=0 is region j is not in the neighborhood of i, with j wij=1.
Classification rates for single-texture images
10 training images per class, 10 test images per class.
Weakly-Supervised Modeling
Idea: Replace L mixture models with M components by a single mixture model with L x M components.
• Annotate each image with the set C of labels associated with classes occurring in it.
• Run EM:• E step: update class membership probabilities:
p (clm | x, C ) / p ( x | clm ) p ( clm | C ).• M step: update model parameters.
Nigam, McCallum, Thrun & Mitchell (2000)
T1 (brick) T2 (carpet) T3 (chair) T4 (floor 1) T5 (floor 2) T6 (marble) T7 (wood)
T1 (brick) T2 (carpet) T3 (chair) T4 (floor 1) T5 (floor 2) T6 (marble) T7 (wood)
Single-texture training images only
Single- and multi-texture training images
ROC Curves
10 single-texture images per class, 13 two-texture training images, 45 multi-texturetest images.
Effect of relaxation on labelingOriginal image
Top: before relaxation, bottom: after relaxation
Successful Segmentation Examples
Unsuccessful Segmentation Examples
Animal Dataset
• 10 training images for each animal + background, 20 test images per class.
Bradshaw, Scholkopf, & Platt (2001); Schmid (2001); Kumar & Hebert (2003)
• No manual segmentation.
Oh well..
• 3D Objects without distinctive texture
• Category-level recognition of 3D objects
• Please join us in trying to solve the 3D object recognition problem..