Robust Face Recognition with Occlusions in both Reference and Query Images
-
Upload
singingwei -
Category
Technology
-
view
493 -
download
0
description
Transcript of Robust Face Recognition with Occlusions in both Reference and Query Images
![Page 1: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/1.jpg)
Xingjie Wei, Chang-Tsun Li, Yongjian Hu
Department of Computer Science,
University of Warwick
http://warwick.ac.uk/xwei
Robust Face Recognition with
Occlusions in both Reference
and Query Images
![Page 2: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/2.jpg)
Outline
• Face recognition with occlusions
• Current methods
• Three occlusion cases
• Our methods
• Experimental results
2
![Page 3: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/3.jpg)
Face recognition with occlusions
• Intra-class variations > inter-class variations
• Causes imprecise registration of faces
poor recognition performance !
3
![Page 4: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/4.jpg)
Face recognition with occlusions
• Why is it so difficult?
• No prior knowledge of occlusion
• location,size,shape,texture -- unpredictable!
4
![Page 5: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/5.jpg)
Current methods
Reconstruction based methods [Wright et al. PAMI09, Yang et al. ECCV’10, Zhang et al. ICCV’11]
– An occluded probe image is represented
as a linearly combination of unoccluded
gallery images
– The probe image is assigned to the class
with the minimal reconstruction error
Assume the gallery/training images are clean 5
![Page 6: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/6.jpg)
Three occlusion cases
6
![Page 7: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/7.jpg)
Current methods
Very limited work considers the existence of
occlusions in both gallery and probe sets
• [Jia et al. FG’08,CVPR’09]
– Proposed a reconstruction based method in, as
well as an improved SVM
Depend on an occlusion mask trained through the use of
skin colour
• [Chen et al. CVPR’12]
– Uses the low-rank matrix recovery to remove the
occlusions
Requires faces to be well registered in advance 7
![Page 8: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/8.jpg)
Our method
• Dynamic Image-to-Class Warping
– An image a patch sequence
– Matching the Image-to-Class distance
No occlusion detection
No training phase
8
![Page 9: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/9.jpg)
Our method
• Face representation
– Natural order: forehead, eyes, nose and
mouth to chin does not change despite occlusion or imprecise registration
9
![Page 10: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/10.jpg)
Our method
• Image-to-Class distance
– from a probe sequence to
all the gallery sequences of
an enrolled class
– each patch in the probe
sequence can be matched
to a patch from different
gallery sequences
10
![Page 11: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/11.jpg)
Our method
11
Warping path:
:
(1,1,1)
(4,3,2) M
N
![Page 12: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/12.jpg)
Our method
• Constraints:
– Boundary
– Continuity
– Monotonicity
– Window constraint:
Maintains the order of facial features
12
Warping path:
:
![Page 13: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/13.jpg)
Our method
• Local cost
• Optimal overall cost
13
P
G
the optimal
![Page 14: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/14.jpg)
• Why does it work?
– Tries every possible warping path and select
the one with minimal overall cost
– Exploits the information from different gallery
images and reduce the effect of occlusions
14
Our method
![Page 15: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/15.jpg)
Our method Dynamic
Programming (DP)
15
![Page 16: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/16.jpg)
Experiments
• The FRGC database
– 44,832 images with different illuminations
and expressions, 100 subjects selected
• The AR database
– >4000 images with real disguise, 100
subjects selected
• Realistic images
– >2000 frontal view faces of strangers on
the streets, 80 subjects selected 16
![Page 17: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/17.jpg)
Synthetic occlusions • Uvs.O
– Gallery
– Probe
• Ovs.U
– Gallery
– Probe
• Ovs.O
– Gallery
– Probe 17
Block size: 0%~50%
of the image
Block location:
random and unknown
to the algorithm
![Page 18: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/18.jpg)
Synthetic occlusions
18
Uvs.O Ovs.U
Ovs.O
![Page 19: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/19.jpg)
Real disguises • Uvs.O
– Gallery
– Probe
• Ovs.U
– Gallery
– Probe
• Ovs.O
1. Gallery Probe
2. Gallery Probe 19
![Page 20: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/20.jpg)
Real disguises
20
![Page 21: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/21.jpg)
Realistic images
21
![Page 22: Robust Face Recognition with Occlusions in both Reference and Query Images](https://reader034.fdocuments.net/reader034/viewer/2022052321/5482589ab47959e20c8b4750/html5/thumbnails/22.jpg)
22
Thank you
• Questions ?
• Xingjie Wei
• http://warwick.ac.uk/xwei
• Department of Computer Science, University
of Warwick