Fusion of Face and Iris Biometrics from a Stand-Off

Video Sensor

Ryan ConnaughtonKevin W. Bowyer

Patrick Flynn

April 16, 2011Computer Vision Research Lab

Department of Computer Science & EngineeringUniversity of Notre Dame

Biometrics and Multi-Biometrics

BiometricTrait

Sensor MatcherBiometricSample Output

Multi-Modal Multi-Sensor

Multi-Sample

Multi-Algorithm

2

Redundancy at any stage is referred to as multi-biometrics

Fusion in Multi-Biometrics

Fusion: Combining information from multiple sources

Types of fusion:

– Signal Level

– Feature Level

– Score Level

– Rank Level

– Decision Level

33

Advantages and Disadvantages

Potential advantages of multi-biometrics:

– Increased recognition accuracy

– Wider population coverage & lower failure-to-acquire rates

– More difficult to spoof

Potential disadvantages:

– Increased computation time

– Increased acquisition time

– Increased sensor cost

4

Project Goal

Investigate the feasibility of multi-biometrics based on a single sensor

Specifically, combine multi-sample and multi-modal elements to create a system based on face and iris biometrics

Compare performance of multi-biometric approach to single biometric approach

5

Sensors – Iris on the Move (IOM)

Developed by Sarnoff Corp. [1]

Designed for Iris recognition

Stand-off and on-the-move

Array of 3 frontal video cameras

– Each frame is 2048 x 2048 px

– Average iris diameter is ~120 px

Synchronized NIR illumination

Image from K. W. Bowyer, K. Hollingsworth, and P. J. Flynn. Image understanding for iris biometrics: A survey. In Computer Vision and Image Understanding, volume 110, pages 281-307. 2008.

6

IOM Frame Example

7

Sensors – LG IrisAccess 4000 (LG-4000)

Developed by LG Iris [2]

High-quality iris sensor

Short-range, stationary subjects

Average iris diameter is ~250 px

Image from LG Iris Products and Solutions, 2010. URL http://www.lgiris.com/ps/products/irisaccess4000.htm

8

LG-4000 Image Example

9

Diagram of Approach

10

Preprocessing

Stitch and perform histogram matching between corresponding frames

Use template matching to determine translation required to align frames

11

Face Detection

Performed on stitched frames

OpenCV version Viola-Jones face detector used [3],[4]

– Trained on whole faces

Faces are cropped according to face detector's estimation of size and location

12

Eye Detection

Used for iris biometrics and for alignment during face matching

Performed in two phases

– Phase 1: Detect eyes in upper quadrants of previously detected faces

– Phase 2: Detect eyes in frames where no faces were found

Both phases use template matching approach to search for specular highlights

13

Face and Iris Matcher

Face Matcher

– Colorado State University's implementation of eigenface [5],[6]

– Mahalanobis Cosine: -1 to 1, -1 is perfect match

Iris Matcher

– Modified version of Daugman's algorithm [7]

– Normalized Hamming Distance: 0 to 1.0, 0 is perfect match

14

Fusion Summary

Multi-modal and multi-sample scenario

Test and compare multiple fusion approaches

– Score-level

– Rank-level

Three approaches:

– Min rule

– Borda count

– Sum rule

15

Min Fusion

Multi-sample, uni-modal, score-level fusion

MinIris = Min{ Ii,j | i=1...n, j=1...G }

MinFace = Min { Fi,j | i=1...m, j=1...G }

Ii,j = HD between i-th probe iris and j-th gallery iris

Fi,j = Mahalanobis distance between i-th probe face and j-th gallery face

n,m = number of irises and faces detected

G = number of gallery subjects

16

Borda Fusion

Multi-sample, multi-modal or uni-modal, rank-level fusion

For each probe biometric sample

– Sort gallery subjects by match score (best to worst)

– Cast votes for the top v-ranked gallery subjects

• BordaLinear: VoteWeightn = v + 2 – n

• BordaExp: VoteWeightn = 2v-n

Gallery subject with the most votes is the best match for that probe video

Three variations: BordaIris, BordaFace, and BordaBoth

17

Sum Fusion

Multi-sample, multi-modal or uni-modal, score-level fusion

Ii,k = HD between i-th probe iris and k-th gallery iris

FNormi,k = Normalized Mahalanobis distance between i-th probe face and k-th gallery face

n,m = number of irises and faces detected

α,β = weights assigned to face and iris modalities

18

Dataset

Collected 1,886 IOM video sets, spanning 363 subjects

– Ranged from 1 to 15 probe videos per subject

Iris gallery consisted of one left eye and one right eye for each subject

– Acquired with an LG-4000

Face gallery consisted of one full face image for each subject

– Manually selected and annotated from stitched IOM frames

– Earliest IOM video with full face available was used to generate gallery image

– Videos used to generate gallery images were not included in probe set

19

Detection Results

20

Face Matching Results

Mean match score:

-0.281 (σ = 0.213)

Mean non-match score:

0.000 (σ = 0.676)

Independent rank-one:

51.6% (5073/9833)

21

Iris Matching Results

Mean match score:

0.398 (σ = 0.053)

Mean non-match score:

0.449 (σ = 0.013)

Independent rank-one:

46.6% (13556/29112)

22

Rank-One Recognition Rates

23

Comparison Summary

24

Conclusions

Investigated fusion of face and iris biometrics from a single sensor

Conducted multi-modal experiments on a genuine dataset of 1886 videos of 363 subjects

Combined multi-modal and multi-sample biometrics, as well as score-level and rank-level fusion

Implemented the proposed multi-biometric workflow on a stand-off and on-the-move sensor

Thus far, the best tested multi-modal approach yielded an increase of 5.4% in rank-one recognition over uni-modal approach

25

Acknowledgments & Questions

Datasets used in this work were acquired under funding from the National Science Foundation under grant CNS01-30839, by the Central Intelligence Agency, and by the Technical Support Working Group under US Army Contract W91CRB-08-C-0093.

Current funding is provided by a grant from the Intelligence Advanced Research Projects Activity.

26

[1] J. Matey, O. Naroditsky, K. Hanna, R. Kolczynski, D. LoIacono, S. Mangru, M. Tinker, T. Zappia, and W. Zhao. Iris on the Move: Acquisition of Images for Iris Recognition in Less Constrained Environments. In Proceedings of the IEEE, volume 94, pages 1936-1947. November 2006.

[2] LG Iris. LG Iris Products and Solutions, 2010. URL http://www.lgiris.com/ps/products/irisaccess4000.htm.

[3] G. Bradski and A. Kaehler. Learning OpenCV. O'Reilly Media, Inc., 2008.

[4] P. Viola and M. Jones. Rapid Object Detection Using a Boosted Cascade of Simple Features. In 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), volume 1, pages 511-518, 2001.

[5] Colorado State University. Evaluation of Face and Recognition Algorithms, 2010. URL http://www.cs.colostate.edu/evalfacerec/algorithms6.html.

[6] M. Turk and A. Pentland. Face Recognition Using Eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1991), volume 1, pages 586-591, June 1991.

[7] J. Daugman. How Iris Recognition Works. In 2002 International Conference on Image Processing, volume 1, pages 33-36, 2002.