Grand Challenges in Biometrics“R d Th h d O i i b Bi i R h”or “Random Thoughts and Opinions about Biometrics Research”

or “Some Advice and Suggestions for Graduate Students in Biometrics”

Matthew TurkComputer Science Department and

Media Arts and Technology ProgramUniversity of California, Santa Barbara

http://www.cs.ucsb.edu/~mturk

All images are copyright by the owners

Caveats

• This is a personal view– I.e., a “soapbox talk”!

• Both “where the action is” and “where the action should be”the action should be

• There are several important things I ’ diwon’t discuss

– Data, standards, …

• Goal: Not to give the definitive list of challenges, but to generate discussion

BTAS’10

Biometrics Motivation (some facts and factoids)

• In the US, a car is stolen every 26 seconds• One out of every 10 notebook computers is

stolen• The average adult working in a large business

has 12 passwords to remember and spendshas 12 passwords to remember, and spends nearly a week in every year logging in to different systems

• Over 30 million border crossings per year at San Ysidro Port of Entry (San Diego – Tijuana)– 47 4 million in 200347.4 million in 2003

• There were 10,999 incidents of terrorism attacks worldwide in 2009 (US State Department)

BTAS’10

– 14,435 in 2007

Also… people are sometimes clueless

Hiding the key poorly

Not following the procedure

BTAS’10

Biometrics: “Something You Are”

• Something you have– E.g., key, access card

• Something you know– E.g., password/PIN, hidden entry

• Biometrics– Something you are

• E.g., fingerprint, face– Some way you behave

• E g gait signatureE.g., gait, signature

BTAS’10

Biometrics: Bio (life) + metrics (to measure)

1870 Bertillon develops anthropometrics, a method of identifying individuals based on body measurements

l d l l ifi i f fi i1892 Galton develops a classification system for fingerprints1903 NY State Prisons begin using fingerprints for identifying

criminals1936 Iris pattern for identification proposed1960s First research in automatic recognition of faces,

fingerprints, signatures, speech; FBI pushes development of t ti fi i t iti tautomatic fingerprint recognition systems

1970s/80s Additional biometric modalities introduced and explored1992 US government Biometric Consortium established1990s First systematic evaluations of biometric technologies2000s Biometrics become mainstream; used in border security,

public surveillance, passport control, laptops, building

BTAS’10

entry, etc.

Automatic Biometrics

• A number of biometric modalities have been proposed and investigated– Each has its strengths and limitations– Various levels of discrimination ability, robustness, usability,

intrusiveness, etc.– Each has applications/environments for which it is more/less suited

BTAS’10

Proposed biometric modalities

SignatureFingerprint

Nailbed (ridges in fingernail)g p

Facial appearanceIris texture

Finger geometryFacial expressions

Retinal vasculatureVoiceH d

Saccade patternsOdorSweat pores analysisHand geometry

Heat radiation of faceHeat radiation of hand

Sweat pores analysisDNA…Heat radiation of hand

Ear shapeGait

…

Potentially many more!

BTAS’10

GKeystroke dynamics

Primary sensing modalities for biometrics

• Imaging– Face, fingerprint, hand, retina, iris, ear, signature

• Audio– Voice

• Chemical• Chemical– Odor

• Biochemical– DNA

• Direct measurementHand keystrokes ear signature face– Hand, keystrokes, ear, signature, face

BTAS’10

Potential benefits of biometrics technologies

• Potential benefits include– Identification integrity– Information integrity– Increased security– Secure information access– Accurate access statistics– Communication security– Reduced costs – password maintenance no “buddy punching”– Reduced costs – password maintenance, no buddy punching– Fraud detection/prevention– Convenience – faster login, no passwords to remember– Non-repudiation – more difficult to refute transactions– Etc.

BTAS’10

For example…

Minority Report iris identification scene

BTAS’10

Desirable properties of biometrics

• Universality– Everyone has this characteristic

• Uniqueness– No two persons should be the same in terms of this characteristic

• Permanence• Permanence– The characteristic should be invariant with time

• Collectability– The characteristic can be measured quantitatively

BTAS’10

Desirable properties of biometrics (cont.)

• Performance– Accuracy, resource requirements, environmental factors

• Acceptability, convenience– To what extent people are willing to accept the system

• Circumvention• Circumvention– How easy it is to fool the system

• Cost– Is it affordable for the application domain?

• MobilityCan it be deployed in any location?– Can it be deployed in any location?

BTAS’10

Holy Grail or Grand Challenge

• The real grand challenge of biometrics is to create, develop, and commercialize systems that address all the

d f bi t i id ll f th t ti l b fitneeds for biometrics, provide all of the potential benefits, and exhibit all of the desirable properties– Works for everyone– 100% accurate in all conditions– Doesn’t degrade with time– Easy to enroll (or no explicit enrollment)– Easy to enroll (or no explicit enrollment)– Transparent (not cumbersome or inconvenient)– Inexpensive– Works anywhere, in all environments

BTAS’10

Intel – Lenovo commercial

BTAS’10

NEC commercial

BTAS’10

How to get there? Many research problems…

• Data acquisition– Digital image or audio, keystrokes, measurements…

• Data representationData representation– Efficiently represent the salient information

• Feature extraction– Sparse features from full datap

• Feature matching– Measure of similarity

• Search, organization, and scalability, g , y– Hundreds to millions

• Evaluation– Performance in context: FAR, FRR, ERR, ROC, …

• Integration– Over time, multiple biometrics

• Circumvention

BTAS’10

– Disguises, fake fingers, sunglasses…

Also… public perception of biometrics

CNN – FBI biometric database

BTAS’10

Recent Headlines

3M to Buy Biometrics Company Cogent for $943 Million

Safran to Buy L-1 Units for $1.6 Billion as BAE TakesSafran to Buy L 1 Units for $1.6 Billion as BAE Takes Remainder

Army Reveals Afghan Biometric ID Plan; Millions Scanned, y fg ; ,Carded by May

Canada to use biometrics to secure its borders

Georgia using biometrics in electoral process

Biometric use increasing on campusBiometric use increasing on campus

AuthenTec, UPEK to form new biometrics company

BTAS’10

UK expanding its use of biometrics

Other Recent Headlines

Biometric security inherently fallible, report says

Biometric systems: Not quite ready for prime time?Biometric systems: Not quite ready for prime time?

System failure (biometrics hand print device) results in long lines at CSULB rec. center

Iris Scan Coming to Airports, Focusing Terror Hunt on Innocent

Iris Scanner Border Test May Threaten Privacy

Iraqis fear the U S sharing their Biometric Data in the NextIraqis fear the U.S sharing their Biometric Data in the Next Phase of the Occupation

Minority Report Becomes a Reality For Immigrants

BTAS’10

Minority Report Becomes a Reality For Immigrants

Jan 2002

BTAS’10

Some Grand Challenges for the Field

• Actually a combination of personal challenges for researchers (esp. graduate students) and some specific t h i l h lltechnical challenges

• Biometrics is largely an applied research domain,Biometrics is largely an applied research domain, borrowing from pattern recognition, signal processing, information theory, sensor design, and other areas

H t k i ifi t t ib ti t th fi ld?– How to make significant contributions to the field?

• General encouragement: Ask deep questions and keep the g p q pbig picture in mind– Not just “What techniques can be used to compute features and

classify?

BTAS’10

classify?– Look for fundamental contributions that others can build on

1. Commitment to Doing Good Science

Biometrics is an “uncomfortable science”

Uncomfortable science is the term coined by statistician John Tukey for cases in which there is a need to draw an inference from a limited sample of data, where further samples influenced by the same cause system will not be available. More specifically, it involves the analysis of a finite natural phenomenon for which it is difficult to

h bl f i l f dovercome the problem of using a common sample of data for both exploratory data analysis and confirmatory data analysis. This leads to the danger of systematic bias through testing hypotheses suggested by the datathrough testing hypotheses suggested by the data.

Wikipedia

BTAS’10

Corollaries

• Use good methodology in running experiments and in reporting results– Not just “here’s our approach, our data, and our results”– Clear hypotheses– Solid experimental design– Meaningful analysis of results

• E.g., how do you choose which validation method to use?Th th t i th b t lt ??– The one that gives you the best results??

• Also: Resist publication biasp– Where the publication of research results depends on their nature

and direction– Report positive and negative (and inconclusive) results equitably

BTAS’10

Report positive and negative (and inconclusive) results equitably• The “file drawer problem”

Take care in experimental design and reporting

• Confidence in results

• Statistical testing frameworks– Fisher significance testing (inductive evidence against the null

hypothesis)hypothesis)– Neyman-Pearson hypothesis testing (competing hypotheses, Type I

and Type II errors)

N t h lt “ t ti ti ll i ifi t ” t• Note: such results are “statistically significant,” not “significant”

• Report and articulate results carefully– Inferential reasoning, not deductive reasoning

BTAS’10

Analogy – Adelson checker shadow

These squares have the same intensity values. Really!

d d lTed Adelson, MIT

An analogy – we sometimes see data the way we expect to see it, not necessarily the way it is

BTAS’10

necessarily the way it is.

An aside – Fisher vs. Neyman-Pearson

• Many texts present the subject matter as a hodgepodge mix of the two, but there are important philosophical and

th d l i l diff b t thmethodological differences between them

The misinterpretation of p values results in an overstatement of theThe misinterpretation of p values results in an overstatement of the against the null hypothesis. A consequence of this is the nuevidence mber of “statistically significant effects” later found to be negligible, to the embarrassment of the statistical community.g g y

Hubbard and Bayarri, “P Values are not Error Probabilities,” Nov. 2003

BTAS’10

1A. Incorporate Biometrics within Biometrics

• The terms “Biometrics” and “Biometry” have been used since early in the 20th century to refer to the field of d l t f t ti ti l d th ti l th ddevelopment of statistical and mathematical methods applicable to data analysis problems in the biological sciences.

• Statistical methods for the analysis of data and their application to various scientific contexts, e.g.:

A i lt l fi ld i t t th i ld f diff t– Agricultural field experiments to compare the yields of different varieties of wheat

– The analysis of data from human clinical trials evaluating the l ti ff ti f ti th i f direlative effectiveness of competing therapies for disease

– The analysis of data from environmental studies on the effects of air or water pollution on the appearance of human disease in a

i t

BTAS’10

region or country

Incorporate Biometrics within Biometrics (cont.)

• The International Biometric Society is an international society devoted to the development and application of t ti ti l d th ti l th d th d i thstatistical and mathematical theory and methods in the

biosciences (“Biostatistics”)

• It has been suggested that, since they were around first, our community be called “biometric authentication”

Oh ll !– Oh well…!

BTAS’10

Theory

• Develop a comprehensive theory of biometric information (BI)– The uncertainty about the identity of a person

based on sets of biometric feature measurements

• E.g., Youmaran and Adler developed an algorithm to calculate biometric information based on a set of features using the relativebased on a set of features, using the relative entropy measure– “Measuring Information Content in Biometric

F t ”Features”

Youmaran and Adler, “Measuring Information Content in Biometric Features ” 2009

BTAS’10

in Biometric Features, 2009

Test whole system performance

• Not just individual technique or device performance

• Given reasonable assumptions (about physical traits, human behavior, the hardware, and the software), what are the limits of biometric modalities and biometrics systems?the limits of biometric modalities and biometrics systems?

• A device or modality can have great experimental f b b ll b l i i !performance but be all but useless in practice!

BTAS’10

1B. Quantify robustness

• General definition: Robustness is the quality of being able to withstand stresses, pressures, or changes in procedure

ior circumstance• Robustness requires formal consideration of performance

in the face of variation or uncertainty in input variables,in the face of variation or uncertainty in input variables, assumptions, environmental conditions, etc.

R b ≠ L• Robust ≠ Low error rate• Robust ≠ Works well on our data set• Robust ≠ Works well on two data sets• Robust ≠ Works well on two data sets• Robust ≠ Uses a robust statistic

BTAS’10

Don’t claim robustness unless you can back it up!

2. Continuous Multimodal Biometrics

• “One shot” authentication systems have limited value in many scenarios– Short time for sensing– Fast decision needed– No ability to correct or backtrack– Limited sensing modalities available– Allow access for potentially unlimited periods of time

• Combine multimodal biometrics…– Using whatever modes are currently available

• …with continuous biometrics– Ongoing authentication, rather than “one shot”

to pro ide a contin o s assessment of ser

BTAS’10

• …to provide a continuous assessment of user authentication (e.g., sysop)

Continuous Multimodal Biometrics (cont.)

• Not just multibiometrics (typically N sources at once)• Implies need for soft biometrics and some method(s) for

non-biometric token tracking to keep track of user’s location over time– E.g., vision-based user trackingE.g., vision based user tracking

• Multibiometric terminology:– multisensory– multialgorithm– multi-instance (e.g., left and right fingers)( g , g g )– multi-sample– multi-modal

All are possible here

BTAS’10

• All are possible here

Continuous Multimodal Biometrics (cont.)

• But how to integrate information across modalities and across time?– What’s the value at time t of a authentication result at time t-1?– How to model uncertainty with different combinations of

modalities (hard and/or soft) at different times?

• Big challenges:H t th d t t i i d t ?– How to gather adequate training data?

– Using multiple traits may increase the enrollment and verification times, cause more inconvenience to the users and increase the

ll t f th toverall cost of the system.

BTAS’10

Fully leverage and integrate soft biometrics

• Many possible soft biometrics

• E.g., research on integrating “hard” and “soft” biometrics (Jain et al. 2004)

BTAS’10

3. Develop smarter sensors

• We don’t have to settle for the data we’re given!

• Learn from computational photography – design biometric sensors that integrate processing with sensing (in order to get better features/information)get better features/information)

• Various possibilities:– Low level (sensors)– Mid level (construction of data/features)– High level (context awareness – e.g., taking facial image at the g ( g , g g

right time or determining which facial image to use)

BTAS’10

Moore’s Law is your friend!

Year 1975– 0.001 CPU cycles/pixel of video streamy p

Year 2000– 57 cycles/pixel

Year 2025– 3.7M cycles/pixel

BTAS’10

(64k x speedup)

4. Some promising/needed research directions

• Increased efforts (and funding!) in:– Temporal modeling– Non-ideal and challenging conditions– Completely non-invasive biometrics– Revocable biometrics– Learning in biometrics– Human factors in biometrics

BTAS’10

Understanding and modeling temporal effects

• How people and their characteristics (features) change over time– E.g., effect of aging, facial hair growth, etc. on face recognition

BTAS’10

Biometrics in non-ideal conditions

• Regarding conditions, sensors, and people• E.g., recent work on blurry imagery for FR, matching high-

and low-resolution data, etc.

BTAS’10

Completely noninvasive biometrics

• For both surveillance and purposeful authentication

• E.g., Biometrics at a distance (remote biometrics) – 2006 Handbook by Tistarelli, Li, and Chellappa– DARPA’s Human Identification at a Distance (HumanID) projectDARPA s Human Identification at a Distance (HumanID) project

• Up to 150 meters distance

• Multibiometrics at a distance

• Need both new technologies and improvements in current technologiestechnologies

BTAS’10

Revocable biometrics

• Revocable biometrics are encoded with a distortion scheme that varies for each

li tiapplication.– Thus, they can be canceled

• Typically posed as a pre-processingTypically posed as a pre processing distortion:– During enrollment, the input biometric image is

subjected to a known distortion controlled by asubjected to a known distortion controlled by a set of distortion parameters.

– The distorted biometric sample can then be processed with standard biometrics algorithmsprocessed with standard biometrics algorithms, which are unaware that the features presented to them are distorted.

• Other approaches?

BTAS’10

• Other approaches?

Learning in biometrics

• Improve with experience– Not just with larger training sets

• Minimal enrollment possible– Enroll on the fly

• Learn changes over time (e g aging facial hair etc )• Learn changes over time (e.g., aging, facial hair, etc.)

• Data mining to discover effective authentication patterns and modalities

BTAS’10

Incorporate more human factors

• Explicitly invite the Human Factors community – E.g., joint research grants, keynote talks, program committees,

dit i l b deditorial boards, ….

“These are technologies that interface with human beings, and we've ignored h h ”the human part.”

Jim Wayman, SJSU

BTAS’10

Paper topics in some recent top conferences

P h i i t i diff tl ?

BTAS’10

Perhaps organize session topics differently?

5. Engage with policy issues

• Need to engage with the public policy debates– Take them seriously– Be reliable sources of information, not ideologues– Not just as a Pollyannaish advocate….

• The field, collectively, has a responsibility to do this– Great opportunities for interdisciplinary NSF funding!

BTAS’10

There are thoughtful viewpoints

Society inevitably will have to choose between two versions of ubiquitous surveillance: in one only the rich and powerfulof ubiquitous surveillance: in one, only the rich and powerful use and control the system to their own advantage. In the second, more democratic future, the watchers can also be

t h dwatched.

The question largely boils down to this: is privacy a right or a privilege?privilege?

David BrinDavid BrinThe Transparent Society1998

BTAS’10

…and others not-so-thoughtful

You already have zero privacy Get over itYou already have zero privacy. Get over it.

Scott McNealyCEO, Sun Microsystems1999

BTAS’10

…and Hollywood

Minority Report again…y p g

BTAS’10

Confronting Biometric Detractors

Dr. Robert AtkinsonPresident, Information Technology and Innovation Foundation2006 Bi t i C ti C f2006 Biometric Consortium Conference

• Distinguished detractors and opponentsDistinguished detractors and opponents– Detractors raise legitimate questions/concerns about the effectiveness

and value proposition of biometricsOpponents oppose biometrics for largely ideological reasons may be– Opponents oppose biometrics for largely ideological reasons, may be committed to stopping the deployment of biometrics

D fi d i i f / bj i• Defined six common categories of arguments/objections• Presented helpful advice on “persuading the detractors and

confronting the opponents”

BTAS’10

confronting the opponents

1. If possible, prove it in the private sector, then deploy to public sector.

2. Build in privacy from the beginning.3. Don’t over promise.4. Realize that in most cases opponents won’t play fair.5. Realize that government officials and the media are not

experts and are likely to believe the opponent’s claims evenexperts and are likely to believe the opponent s claims, even when they are false.

6. Make it clear that “could” is not the same as “will.”7. Point out that most systems are not perfect, yet they work

perfectly well.h b fi i l di f i d8. Focus on the benefits, including safety, security and

convenience.9 Keep the debate on the rules not on the technology

BTAS’10

9. Keep the debate on the rules, not on the technology.10. Educate, educate, educate.

6. Become a fully interdisciplinary practice

• E.g., a typical life sciences lab:– “Our laboratory is based in the James H. Clark Center at Stanford

d l id f t h i i l di l t lland employs a wide range of techniques including neural stem cell and tissue engineering methods, electrophysiology, molecular biology, neural activity imaging, animal behavior, and computational neural network modeling I am also a physician incomputational neural network modeling. I am also a physician in the psychiatry department, where I employ novel interventional high-speed (action potential-based) brain stimulation techniques in human patients for therapeutic purposes.”human patients for therapeutic purposes.

Deisseroth Lab, Stanford University

• Work with psychologists (and others) who study human behavior – possibility of exploring new behavioral (hard and soft) biometrics

• And statisticians life scientists sensor researchers

BTAS’10

• And statisticians, life scientists, sensor researchers (MEMS, nano, bio), sociologists, etc.

And speaking of “others”…

• Obviously, biometrics has a lot in common with face and gesture recognition– Face recognition– Behavioral analysis for surveillance– Behavior-based biometrics– Gesture as a biometric modality?

BTAS’10

FG2011 March 21-23, 2011, Santa Barbara, CA www.fg2011.org

BTAS’10

Summary of Challenges

• Commitment to doing good science• Continuous multimodal biometrics• Smarter sensors• Invest more effort into:

T l d li– Temporal modeling– Non-ideal and challenging conditions– Completely non-invasive biometrics– Revocable biometrics– Learning in biometrics– Human factors in biometricsHuman factors in biometrics

• Engage with policy issues• Become fully interdisciplinary

BTAS’10

Summary

• Biometrics is not just a technical problem for engineers and computer scientists

• A multidisciplinary approach is vital to truly “solve” biometrics – to understand it deeply– “Thinkers” and “builders” need to work togetherThinkers and builders need to work together

• Still, there is low-hanging fruit to be had, where specific biometrics technologies can be useful before all the Big P bl l dProblems are solved– (Good…!)

• Big Picture Thinking is required!

BTAS’10

Your assignment

• Invite someone new to dinner tonight• Discuss the state of the field

– Needs, driving forces, concerns and policy issues– The biometrics industry– The state of the art (products and research)The state of the art (products and research)– Ideas for good PhD theses– Ideas for your new company

T lk l i h fi ld (i i hi )• Talk to people not in the field (i.e., not in this room)– Other friends and family, undergraduates, etc.– How do they see the area, the needs, the concerns, usability, etc.?y , , , y,– Fresh (even naïve) perspectives can be very useful

• The emperor’s new clothes

BTAS’10

Discussion, questions?

http://ilab.cs.ucsb.edu

BTAS’10