Face Identification for Humanoid Robot

ForschungsgruppeProf. Dr.-Ing. Kristian Kroschel

<Xin Wang>

Face-ID recognition module for OPASCA

Karlsruhe, <2nd Sep. 2008>


Outline for the presentation

Task und system requirement System overview Implementation steps with experiment results Face detection + Face recognition (Basic System)

• Eigenface , Fisherface Improvement solutions

• Filters, Composition training set , Classifier

System performance overview Future work Questions


Task und system Requirement

3

◆Task: Implementation of an optical face-ID recognition module using MATLAB with steering camera in real-time application

◆Requirement: Good recognition rate with robustness property Multi-person , Multi-places , Multi-illumination , Cross-poses , Cross-expressions , Cross-view of angle As the real situations the social robot will encounter !


Project Overview

5

◆System diagram:

Face Detection

FilteringSubspace Projection

Recognition System

ID


Implementation steps

◆System implementation diagram

Face Data-baseCreation

Eigenface

Fisherface

Filters(Gabor,

Histogram)

Classifier(N-Nearest,

Mahalanobis)

Composition Training Set

Core Algorithm Improvement SolutionTrain &Test Image



◆IITB research group face database 5 places , each with 2 situations ( with or without light) 50 * 50 pixel image size 6-9 persons with 50 pic / person in each situation Multiple illumination , cross-pose , cross-expression

and various view of angle.

For robustness test

purpose



◆Core Algorithm – Eigenface Algorithm ?

Near real-time performance and accuracy with robustness Problem for feature-based face recognition?



◆Core Algorithm – EigenfaceFeature extraction Images must share a number of similarities (as T-F domain

transform , exp. Fourier transform find basis)

Capture the variation in a set of images ,independent of any

judgment of features (Information theory based features)

How to do it?

？ A face image in our database is a point in 50*50=2500 high

dimensional image space , how to describe similarities

between images ? ( Image encoding, computation model)



◆Core Algorithm - EigenfaceComputation problem

Example for “ How far apart “ between two points (image) in 2D

d12= sqrt ( x△ 2 + y△ 2 ) ,

where x = x△ 2 - x1 , y = y△ 2 - y1

？Point in 2500 dimensions ?

Distance calculation means:

d = sqrt ( sum ( x△ 2 + y△ 2 + …. ))locations2500



◆Core Algorithm – EigenfaceSolutions They could be described by a relatively low dimensional

subspace (PCA , dimension reduction) Project the face images into an appropriately

chosen subspace and perform classification

by similarity computation

( distance, angle)



◆Core Algorithm -Eigenface1. Eigenface Initialization

Acquire an initial set of face

images (Training Set)

Each image is stored in a

vector of Size N

TiN

ii xxx ....1

TiN

ii xxx .....1



◆Core Algorithm – Eigenface2. Center and create data matrix

Each image as vector of size N

Data Matrix as size of N×P

p

i

iiix

pmmxx

1

1,

pxxxX ......21

M=



◆Core Algorithm – Eigenface3. Create covariance matrix , compute

eigenvalues and eigenvectors

(eigenfaces)

Covariance Matrix with size N×N

V (N×1)is set of eigenvectors ,with eigenvalues ∧

TXX

VV M eigenvectors correspoding toM largest non-zero eigenvalues



◆Core Algorithm – Eigenface 120 Eigenvectors(eigenface)

They are the orthogonal

basis to build a dimensional

reduction subspace

(facespace) Facespace with 120

dimensions , eigenvector->

weight



◆Core Algorithm – Eigenface4. Project training images and test

image

Projected train image and test

image in “facespace”,their

dimension are M×1jTjiTi yVyxVx ~..~



◆Core Algorithm – Eigenface5. Recognition(Similarity measurement) use Euclidian Distance , tested

image will be classified to nearest

euclidian distance picture in train

2

1

)~~()~,~( jk

M

k

ik

ji yxyxD

Test

Matched Image:



◆Core Algorithm – Eigenface Experiment results:

no light with light Test

Train min max avg std min max avg std

no light 0.82 1 0.9022 0.0667 0 1 0.3633 0.4614

with light 0 1 0.3367 0.4919 0.9600 1 0.9911 0.0145

Remark:the testing is based on the following condition , with 6-9 persons in each test for both training

set and testing pictures, and 50 pictures for each person in the test and training set

is assumed



◆Core Algorithm – Fisherface Main problems faced by Eigenface : robustness

Experiment in 5 rooms against each other : cross-database

Mean recognition rate: 20%-30%

Standard deviation : over 40%

Why?

The variations between the images of the same face due

to illumination and view direction are almost always larger

than image variations due to change in face ID.

--Moses,Adini,Ullman , ECCV’94



◆Core Algorithm – Fisherface Solution: find a subspace which is insensitive

to illumination change (Fisher Space)

Fisher subspace



◆Core Algorithm – Fisherface Fisher’s Linear Discriminant• Preserves separability of classes• Maximizes ratio of projected

between-classes

to projected within-class scatter Between-class scatter

Within-class scatter

C

iiW

Ti

Xxii

Smxmx ssi 1

,))((

Tii

C

iiB

mmmmnS ))((1

Where: Sw and SB areBoth N×N size and c :number of class,

m :mean of all imagemi : mean of class ith

ni : No. of images in ithclass



◆Core Algorithm – Fisherface Difference with Eigenface in

: how to solve eigenvector pro

VSVS WB

)(maxarg* VJVV

Finding an optimal linear mapping W



◆Core Algorithm – Fisherface Keep first C-1 eigenvectors to form

Fisherface subspace



◆Core Algorithm – Fisherface Experiment results

Eigen:Test2 Room1,Fisher Test6 Room1


Improvement steps

◆Gabor-wavelet filter for preprocessing Main problem : poor robustness in cross-database test

Even Fisherface only achieve around 40-50% rate,

10%-20% improvement against Eigenface method

(Maybe because our face database, quite various situations)

Possible Solution : choose one representation of image

that is insensitive to illumination

change ( Filtering step pre-step)

Gabor-Wavelet Filter Exploit salient visual properties, spatial

localization,orientation selectivity as retina


Improvement steps

◆Gabor-wavelet filter for preprocessing Complex Gabor function in space domain

Complex sinusoidal carrier

The Gaussian envelop

),(),(),( yxwyxsyxg r

)))(2(exp(),( 00 Pyvxujyxs

)))()((exp(),( 20

220

2rrr yybxxaKyxw

5 scales,8Phase ,

Real part


Improvement steps

◆Gabor-wavelet filter for preprocessing Too high dimension output?

2500*5*8=100000 dimension-> take real,

then mean them as output Experiment results (Test 16, use room5 worst case)

After Gabor

After Histeq


Improvement steps

◆Gabor-wavelet filter for preprocessing Testing from

Room5


Improvement steps

◆ Composition training set Initial idea : more information for system to “learn” Implementation: training pic – 6 pic/ person / room

4 other rooms used, 5 person

thus 6×5×4=120 pic totally

testing pic –

from other 1 room ,

24 random/ person , 5 person

TEST13,

ROOM1


Improvement steps

◆ Composition training set Why composition training achieve

best? Lambertian surface model :

given 3 images of a lambertian surface ->

albedo and surface normal can be

recovered Further Idea?

Active learning method for robot to

build its training database

automatically

spnpapE T)()()(


Improvement steps

◆ N-nearest Classifier / Distance

Test13, Room1

Result: 3%-5% improvement for avg and std


Project Overview

◆ System performance overview Within-database Test Both Eigenface and Fisherface algorithm achieve very good

recognition rate around 90%-99% with low Std around 1%-10% Cross-database Test Eigenface Algorithm rate drops a lot (in different illumination

condition )to around 20% with high Std around 40% Fisherface Algorithm shows better robustness , get rate around

40%-50% with Std around 30% Best Result after improvement in Cross-Database Test By using GaborFilter+Fisherface+4-Nearest Classifier get rate around 92.5% with Std 10.5% (Test 13 Room1 Condition)


Future work

Video based recognition system Main problem we still faced : Robustness, computational

complexity (efficiency)

Ideas? Facedetection+Gabor-wavelet+Adaboost+Fisher+

Bayesian classifier (or use SVM) Active learning model for building robot’s training

database automatically (face/not face? know/ unknow?)


Q&A

Thank you for your attention!

Questions?

40


Additional

◆Core Algorithm – Eigenface “Information theory approach of coding and decoding face

images may give insight into the information content of face

images, emphasizing the significant local and global

“features“ . Such features may or not be directly related to

our intuitive notion of face features , such as eyes, nose,

lips, or hair.

M. Turk and A. Pentland, “Eigenfaces for recognition” ,

Journal of Cognitive Science , pp, 73, 1991


Additional

◆How many eigenvectors we need in Eigenface From non-zero eigenvalue -> P-1 is upboundary From practical use -> ½ of image number other improvement-> get rid of first 3 eigenvectors

Experiment result:

Test15, 120 pic in

training


Additional

◆Solve SSS problem in Fisherface


Additional

◆Adaboost feature extraction for Gabor


Additional

◆Gabor filter vision


Additional

◆EBGM :

◆FERET DATABASE: http://face.nist.gov/colorferet/


Additional

ORL Facedatabase test: 5 persons selected and 5 pic/ per person used in both training and testing set, The results show all 100% rate for Eigen , Fisher and Fisher+Gabor method, Then I reduce the training set picture for only 1 pic / per person ,and 9 pic/

person in testing set , for 5 persons Even for Eigen algorithm , it achieves: Eigen’s Mean 0.8889 , Std 0.1361 So I think there is no doubt that other research paper states they get over 98%

recognition rate , it must depend on the face database they used. ORL facedatabase include very little illumination and view of angle changing .

ORL face database: http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html


<Subtopic>

<Title>

48

(P-C) × 1 Vector

(C-1) × 1 Vector

PCA

LDA

N×1 VectorX matrix

Face Identification for Humanoid Robot

Technology

Transcript of Face Identification for Humanoid Robot