BERC

Face Recognition Using Fisheye Camera

Yonsei University

Computer Visioin Lab.

Jae Kyu Suhr

berc.yonsei.ac.kr

Flow Chart

Face detection using skin color

Radial distortion removing

Perspective distortion removing

Training face data enrollment

by pinhole camera

Test image acquisition by fisheye camera

Eigenface generation

Face normalization

Face recognition

berc.yonsei.ac.kr

Training data acquisition and eigenface generation

berc.yonsei.ac.kr

Training Data Acquisition

Face images of 51 subjects were acquired by Nikon digital camera and normalized them using positions of two eyes and mouth.

Normalization method

Resolution: 50×67 pixels (gray scale)Left eye : (15,24)Right eye: (31,24)Mouth : (23,40)

berc.yonsei.ac.kr

Training image samples

Normalize

berc.yonsei.ac.kr

Training faces and eigenfaces

Training images (51) Eigenfaces

Meanface

berc.yonsei.ac.kr

Face detection

using foreground extraction and skin color

berc.yonsei.ac.kr

Foreground Extraction

Background Foreground+Background

Background subtraction

Morphological process (closing)

Extracted foreground

berc.yonsei.ac.kr

Skin Color-based Face Detection

Training face color

R G B

value

num

ber of pix

el

H S V

value

num

ber of pix

el

Y Cb Cr

value

num

ber of pix

el

berc.yonsei.ac.kr

Skin Color-based Face Detection

Morphological process (closing)

Skin detection

Labeling Detection result

berc.yonsei.ac.kr

Face Detection Results

berc.yonsei.ac.kr

Two-Step

Distortion Removing

berc.yonsei.ac.kr

1st step: Radial Distortion RemovingFisheye camera image Pinhole camera image

Equidistance projection model

er fθ=Perspective projection model

tanpr f θ=

tan( )ep

rr ff

=

Optical axis

Image plane

Incoming ray

Lensf

θ

rp

Image plane

Optical axis

Incoming ray

Lensf

θ

rp

berc.yonsei.ac.kr

2nd step: Horizontal Perspective Distortion Removing

θ

Camera 1 Camera 2

-1 -11 2 2=KR K =Hx x x

x1 : image point on the image from camera 1

x2 : image point on the image from camera 2

K : camera intrinsic parameters matrix

R : rotation matrix

berc.yonsei.ac.kr

Two Step Distortion Removing

Radial distortion Recticiation

Horizontal perspective distortion removing

berc.yonsei.ac.kr

Experimental Environment

1.3m

0.5m1.0~2.2m Fisheye camera

Illuminator

berc.yonsei.ac.kr

Test images

Enrolled face image

Input test images (height: 1.7m)

original rectified normalized

Input test images (height: 1.2m)

original rectified normalized

berc.yonsei.ac.kr

Test images

Enrolled face image

Input test images (height: 2.2m)

original rectified normalized

Input test images (height: 2.0m)

original rectified normalized

berc.yonsei.ac.kr

Test images

Enrolled face image

Input test images (height: 1.6m)

original rectified normalized

Input test images (height: 1.0m)

original rectified normalized

berc.yonsei.ac.kr

Test images

Enrolled face image

Input test images (height: 1.9m)

original rectified normalized

berc.yonsei.ac.kr

Matching Result (rank 1)

berc.yonsei.ac.kr

Matching Result (rank 2)

berc.yonsei.ac.kr

Matching Result (rank 3)

berc.yonsei.ac.kr

Failed Cases

berc.yonsei.ac.kr

0.5m1m 2m

0.5m

Operating Range

berc.yonsei.ac.kr

Thank youQ & A