Automatic Face Recognition under Component-Based Manifolds

CVGIP 2006

Wen-Sheng Chu (朱文生 ) and Jenn-Jier James Lien (連震杰 )

Robotics Lab. CSIE NCKU

Motivation

• Face recognition is hard due to several image variations:

illumination pose expression

Objective

• Recognize faces using multiple face patterns rather than a single one.

Person BPerson A Person BPerson A

Single input pattern Multiple input patterns

Automatic Acquisition of Facial Components

Detected Features

Normalized Illumination IB

Normalized Pose IR

Original Image

FaceDetection

Cropped Face I

Feature Point Detection

Face +veRemoval

Facial Components

Extraction

Band-passFiltering

Registration byAffine Warping

Rejected Non-face

Extracted Facial Components

2-Class SVM

Classifiers

Training Data of Feat

ures

P. Viola and M. Jones, “Robust Real-Time Face Detection”, IJCV 2004.

Automatic Acquisition of Facial Components

Detected Features

Normalized Illumination IB

Normalized Pose IR

Original Image

FaceDetection

Cropped Face I

Feature Point Detection

Face +veRemoval

Facial Components

Extraction

Band-passFiltering

Registration byAffine Warping

Rejected Non-face

Extracted Facial Components

2-Class SVM

Classifiers

Training Data of Feat

ures

Facial Feature Detector

• 2-class SVM with feature vector v:

• Reject false positives

),()( yxIxNyvA

),()( yxIxNyvG

G

A

v

vv

x x x

o o o o

Automatic Acquisition of Facial Components

Detected Features

Normalized Illumination IB

Normalized Pose IR

Original Image

FaceDetection

Cropped Face I

Feature Point Detection

Face +veRemoval

Facial Components

Extraction

Band-passFiltering

Registration byAffine Warping

Rejected Non-face

Extracted Facial Components

2-Class SVM

Classifiers

Training Data of Feat

ures

Registration & Illumination Normalization

Affine warping

Band-pass filtering85.0 GIGII RRB

RI

I

BI

Registration

IlluminationNormalization

Automatic Acquisition of Facial Components

Detected Features

Normalized Illumination IB

Normalized Pose IR

Original Image

FaceDetection

Cropped Face I

Feature Point Detection

Face +veRemoval

Facial Components

Extraction

Band-passFiltering

Registration byAffine Warping

Rejected Non-face

Extracted Facial Components

2-Class SVM

Classifiers

Training Data of Feat

ures

Facial Components Extraction

• Effects of pose and illumination are smaller in each local region compared with those in the holistic face image.

T. K. Kim, H. Kim, W. Hwang and J. Kittler, “Independent Component Analysis in A Local Facial Residue Space for Face Recognition”, PR, 2004.

Constrained Mutual Subspace Method (CMSM)

• Similarity between i and j == θc

• Use the variation of dissimilarity between subjects

θ

θc

K. Fukui and O. Yamaguchi, “Face Recognition Using Multi-viewpoint Patterns for Robot Vision”, ISRR 2003.

project project

subspace i subspace j

ic jc

constrainedsubspace

Constrained Subspace Generation

• Take nose for explanation:

The eigenvectors, w, selected in ascending order, are the basis of the constrained subspace, Snose.

Tnosei

nose

i

nose

i

T

kkNk

nosei NC BBxx/1 1

T

iii

nosenosenosenoseL

nose2

nose1 BBP w,)wP...PP(

Constraint subspace basis PCA basis

Projection onto Constrained Subspace

1. Projection basis vectors constrained subspace Snose

2. Normalizationlength(projected vector) 1

3. Orthogonalizationapplying Gram-Schmidt process to orthogonalize the normalized vectors

Snose

nose

iB nose

jB

Comparison between Normalized Manifolds

• The similarity of nose between subject i and subject j:

where are defined as the eigenvalues of ma

trix .• Similarity(i, j) == summing up the five canonical

correlations

nose

i

ti

nose

Sj

nose

Sit 2

1 cos/1)B,B(similarity

nose

i

NNTnose

Si

nose

Si

Tnose

Sj

nose

Sj

Tnose

Si

nose

Si

BBBBBB

Experiment Setup

#individuals 16

#sequence/individual 5

#second/sequence 10

#frame/second 10

Resolution 320x240

Size of Registration Template 100x125

Size of Facial Components

Eye-braw 40x15

Eyes 28x18

Nose 44x21

Mouth 28x40

Typical Samples in 3D Principal Component Space – Holistic Image

subject 1 (․)

subject 2 (․)

subject 3 (․)

subject 4 (․)

Original v.s. Projected Subspaces – Eye-braw

Original v.s. Projected Subspaces – Left Eye

Original v.s. Projected Subspaces – Right Eye

Original v.s. Projected Subspaces – Nose

Original v.s. Projected Subspaces – Mouth

Comparison

MethodsNN-facial

component

MSM-facial

component

CMSM-holistic

face

CMSM-facial component

Recognition Rate (%)

82.3 90.8 95.2 98.6

End

F&Q and thanks!