Post on 23-Aug-2020
Image Processing forBiometric ApplicationsRuggero Donida Labati
Touchless Fingerprint and Palmprint Recognition Systems
Academic year 2016/2017
• Touchless fingerprint• Touchless palmprint• Publications
Summary
Touchless fingerprint
Traditional biometric systems
• Low usability and user acceptance:o Complex and highly cooperative acquisition
procedureso Can be perceived as privacy invasive
Improving the user acceptance
• Less-constrained biometrics:- touchless- at higher distances- uncontrolled scenarios- natural light conditions- on the move
Fingerprint biometrics• The most used biometric trait:
o high distinctivityo high permanence
• Touch-based sensors:o low usability and user
acceptanceo images with non-linear
distortions and low contrast regions
o latent fingerprint on the sensor platen
o sensibility to dust and dirt
Touchless fingerprint images
TouchlessTouch
• R. Donida Labati, V. Piuri, F. Scotti, TouchlessFingerprint Biometrics, CRC Press, August, 2015.
Possible applications oftouchless fingerprint biometrics
Touchless fingerprint:state of the art• Single view systems:
o enhancement + traditional recognition methods• 2D multiple view systems:
o mosaicing of three different viewso illuminator shaped as a ring-mirror
• 3D reconstruction:o Multiple viewso Structured lighto Photometric stereoo Depth from focuso Acoustic imaging
• Unwrapping methods:o parametric models (e.g. cylinder, sphere, set of rings)o non-parametric models based on minimization functions
Touchless fingerprint:some existing systems (1/2)
Mosaiking
Structured light
Touchless fingerprint:some existing systems (2/2)
Multiple views
Absorbed light
Fingerprint recognition on the move
Researched recognition techniques and their interoperability
The researchedtouchless recognition systems
• Pros:- less-constrainedo absence of distortions in the fingerprint images due
to different pressures of the finger on the sensoro more robust to dust and dirto more user acceptanceo possibility to use the recognition methods in mobile
devices with standard CCD cameras
• Cons:o longer computational timeo interoperability to be further studied
Two-dimensional samples
Single camera acquisition andframe selection
100 mm200 mm
LensStopStart
Featureextraction
NeuralNetwork
• 45 fetures related to:o shape of the ROIo gradient phase and moduleo quality of the focus estimated from the gradient
imageso ridge frequency (FFT and Gabor filters)o entropy
2D touchless fingerprint:touch-equivalent images
Orientationimage
Frequencyimage
Gaborfilters
Enhancement of the ridge pattern
ROIestimation
Enhancement of the ridge visibility
Enhancement of the ridge pattern
Imagebinarization
Three-dimensional minutia points
3D minutiae reconstruction
Features - Minutiae:
xcoordinate
ycoordinate
Angle Quality
- Local Fingercode: 4×2 el.
- HOG features: 3×3×9 el.
Three-dimensional samples and touch-equivalent images
Touchless 3D fingerprint recognition
R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Towardunconstrained fingerprint recognition: a fully-touchless 3-D system basedon two views on the move", in IEEE Transactions on Systems, Man, andCybernetics: Systems, 2015.
Contactless acquisition (1/4)
Contactless acquisition (2/4)
Camera A Camera B
Contactless acquisition (3/4)
Contactless acquisition (4/4)
Preprocessing
Segmentation
Extraction and matching of the reference points (1/2)
Extraction and matching of the reference points (2/2)
Refinement of the pairs of corresponding points
• Based on Thin Plate Spline
3D surface computationand image wrapping
1. Triangulation
2. Linear interpolation
Computation oftouch-equivalent images
• Enhancement– Background subtraction– Non-linear equalization (logarithm)– Butterworth low-pass filter
• Two-dimensional mapping
Two-dimensional mapping (1/2)
• Enrollment:– Compensate for rotations– Computation of 𝑁𝑁𝑅𝑅 rotations
Two-dimensional mapping (2/2)
Template computation
• Neurotechnology VeriFinger– Commercially available– Designed for touch-based
images
Matching
• Database entry– 𝑁𝑁𝑅𝑅 templates 𝑇𝑇𝑒𝑒– One for each rotation
• Live sample– 1 template 𝑇𝑇𝑓𝑓
Experimental results
• Datasets description• Accuracy of 3D reconstruction• Recognition performance• Robustness to finger misplacements• User acceptability• Interoperability• Overview of different technologies
Datasets description• Touchless - one session
– 2368 samples– 10 fingers, 30 volunteers, 8 acquisitions per finger
• Touchless - two sessions– 2368 samples– 10 fingers, 15 volunteers, 16 acquisitions per finger
• 8 acquisitions one year, 8 acquisition subsequent year• Touchless - misplaced fingers
– 1200 samples– 2 fingers (index), 30 volunteers, 20 acquisitions per finger
• Touch-based– One session– Two sessions
Accuracy of 3D reconstruction
• Average error: 0.03m
Recognition performance (1/2)
• Comparable to touch-based systems– One session
– Two-session
Recognition performance (2/2)
Robustness to finger misplacements
• Genuine and impostor match scores remainwell separated
User acceptability
• Survey performed using questionnaires• Results show preference towards contactless
recognition
Interoperability
• Accuracy level obtained by matching imagescaptured by different devices– Matching touchless with touch-based images
• 2 803 712 identity comparisons• EER = 2.00% with 𝑁𝑁𝑅𝑅 = 25• Less than EERs obtained in the literature with similar
experiments
Aspect Touch-based Touchless
Accuracy EER = 0.03% EER = 0.06%
Scalability High To be further investigated
Interoperability High To be improved
Security Latent fingerprints No latent fingerprints
Privacy Data protection techniques Data protection techniques
Cost 10$ to 5000$ 0$ to 5000$
Usability Medium High
User acceptance Medium High
Speed Template extraction + matching
3D reconstruction + template extraction + matching
Overview of different technologies
Three-dimensional samples andthree-dimensional templates
• Template computationo Minutiae extracted from the texture:
each minutiae is described by (i; x; y; z; θ; q)
o Two-dimensional Delaunay triangulation: each triangle is described by (I; L; θMax; C)
3D feature extraction and matching
• Matchingo Iterative algorithm:
searching of the triangle pairs minutiae alignment minutiae matching matching score computation
o Matching score = Nm/(N × M)
Example of 3D template
Computation of synthetictouchless fingerprint samples
R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Accurate 3D fingerprint virtual environment forbiometric technology evaluations and experiment design", Proc. of the 2013 IEEE InternationalConference on Computational Intelligence and Virtual Environments for Measurement Systems andApplications (CIVEMSA 2013), Milan, Italy, July 15-17, 2013, pp. 43-48.
Example of synthetictouchless fingerprint sample
Contactless three-dimensional reconstruction of acient fingerprints
Acquisition
Calibrationof the acquisition system
Image preprocessing and extraction of the
reference points
Point matching and triangulation
Surface estimation and texture mapping
Level 3 features
Conclusions
• Touchless fingerprint recognition:o systems based on two-dimensional samples can be used in
low-cost applications, but the samples present distortionso systems based on three-dimensional samples can obtain
comparable accuracy with respect to traditional systemso touchless systems are characterized by higher usability,
user acceptance, security, and scalabilityo touchless sysemts are partially compatible with AFIS
Touchless palmprint
Contactless and less-constrained palmprint recognition
• Less-constrained fingerprint recognitionhas been studied in previous workso Comparison methods are standard
and publicly available• Results enabled the study of
palmprint recognition methodso Palmprint features are similar to
fingerprint featureso Similar techniques for
acquisition and processing
Palmprint recognition:Comparison with fingerprints• Pros:
o Low resolutions (< 200 dpi) 500 dpi needed for fingerprints
o Can be acquired in more situations Manual workers, elder people
o User acceptabilityo Multibiometric system
Combination with fingerprint, finger shape, hand shape, etc.
• Cons:o High accuracy features not always
usable (e.g., minutiae)
Palmprint recognition:Taxonomy
Palmprint recognition:Contactless vs contact
• Pros:o Less distortiono No dirto Increased user acceptability
• Cons:o Low contrasto Complex backgroundo Sensible to lightingo Sensible to position
Palmprint recognition:3D vs 2D
• Pros:o Robust to lighting, occlusions,
noiseo Robust to spoofing attackso Invariant to position and distanceo Can use also 2D information
• Cons:o Complex equipmento Can be expensive
Palmprint recognition:State of the art (1/3)
• Contact-based 2D systemso CCD-based
scannero Optical deviceo Flatbed scanner
• Contact-based 3D systemso Structured light
illumination
Palmprint recognition:State of the art (2/3)
• Contactless 2D systems:o Cameraso Smartphoneso Webcams
• Contactless 3D systems:o Laser scanners
Palmprint recognition:State of the art (3/3)
• Recognition algorithms:o Ridge basedo Line basedo Subspace basedo Statisticalo Coding based
Researched methods:Palmprint acquisition systems
Contactless palmprint recognition at a fixed
distance
Fully contactless, less-constrained palmprint recognition with uncontrolled
distance
Fully contactless palmprint recognitionwith uncontrolled distance
Acquisition (1/2)
Image A Image B
• Special acquisition systemo Optimization of optics,
illumination, distances• Less-constrained acquisition
o Fully contactlesso Uncontrolled positiono Relaxed hando Palmprint must be visible
Horizontal orientation Small rotations are tolerated
Acquisition (2/2)
• Uniform illumination• Different setups and wavelengths studied
o Three downlights with white ledso Four blue led bars
3D palm reconstruction andmodel normalization• 3D reconstruction
o Point matching and triangulation Homography Cross-correlation
o Point cloud filteringo Surface estimation
• 3D normalization tocompensate rotationso Plane fitting
Palm is almost flato Trigonometry formulas
3D image registration andtexture enhancement• The model is reprojected on the
image planeo Using calibration informationo Normalized position
Invariant to the acquisition positionand distance
• Texture enhancemento Removal of the skin toneo Enhancement of the details of the palmo Removal of ridges
2D feature extraction and matching• SIFT-based alignment for horizontal
rotationso 3D features are not robust to horizontal
rotationso Extraction and matching of pointso Estimation of rotation and translation
RANSAC algorithm
• SIFT-based 2D feature extraction and matchingo Robust to uncontrolled acquisitions o Extraction and matching of SIFT descriptorso Refinement based on collinearity
3D feature extraction and matching
• Delaunay triangulation to refine the matcheso Similar groups of three points are more robusto Computation of 3D coordinateso Delaunay triangulationo Extraction of similar triangles
Match score
Experimental results:Accuracy of different illumination methods
• Palmprints capturedwith uncontrolleddistanceo 64 palms, 640 sampleso White light
Equal Error Rate= 4.13%
o Blue light Equal Error Rate
= 2.53%
Receiver Operating Characteristic
FMR = False Match RateFNMR = False Non-match Rate
Experimental results:Multiple comparisons
• Considered the bestof 3 comparisonso Maximum match score
• Combination of blueand whiteo Mean match score
EER = 0.08%
Fusionscheme EER (%)
FNMR @FMR
= 0.05%
FNMR @FMR
= 0.10%
FNMR @FMR
= 0.25%
FMR@FNMR= 0.10%
FMR@FNMR= 0.25%
Mean 0.08 0.10 0.09 0.07 0.06 0.00
Receiver Operating Characteristic
Experimental results:Robustness to hand orientation
• Hand positioned with differentroll orientationso Good tolerance
Experimental results:Robustness to illumination
• Different environmental illuminationso Laboratory acquisition, morning light,
afternoon light, artificial light• Match scores are not affected
Illumination situation
Match scores
Genuine comparisons
Impostor comparisons
Mean Std Mean Std
Laboratory acquisition 3179.8 942.1 3.3 1.8
Morning light 2677.4 941.6 2 0.8
Afternoon light 2748.9 903.2 2 0.8
Artificial light 2770.9 876.8 2 0.8
Experimental results:Evaluation of usability and social acceptance
• Usabilityo Evaluation of the quality of the sampleso Evaluation of the time needed for the acquisitiono Evaluation of users’ opinion
E.g., Is the acquisition comfortable?
• Social acceptanceo Evaluation of users’ opinion
E.g., Are you worried about hygiene issues? E.g., Do you feel that the system attacks your privacy?
Experimental results:Comparison with the literature (1/2)
• Based on the acquisitiono Fully contactless, less-constrained acquisitiono No pegso No dirt, sweat, or latent impressionso Faster acquisition, simpler setupo Less expensive than the methods based on 3D
models• Based on the accuracy
o Better accuracy than the methods based on 3D models and uncontrolled acquisitions
Comparison of methods in the literature
Reference Type of acquisition Device Size of dataset
(palms)EER(%)
Li et al., 2012Contact
2D
CCD-based with pegs 386 0.02
Cappelli et al., 2012 Optical device 160 < 0.01
Wang et al., 2012 Flatbed scanner 384 0.20
Li et al., 2012 Contact3D
CCD-based andprojector, with pegs 100 0.03
Jia et al. 2012 Contactless 2D
Mobile device 200 0.14
Tiwari et al., 2013 Ad-hoc device 602 0.06
Kanhangad et al. 2011 Contactless
3D
Laser scanner 354 0.22
ProposedMethod
Two-viewCCD-based 64 0.08
Publications
Publications (1/3)
• Research books1. R. Donida Labati, V. Piuri, F. Scotti, Touchless Fingerprint Biometrics, CRC Press, 2015. 2. A. Genovese, V. Piuri, F. Scotti, Touchless Palmprint Recognition Systems, S. Jajodia (ed.),
Springer International Publishing, September, 2014.
• Refereed Papers in International Journals3. R. Donida Labati, A. Genovese, E. Muñoz, V. Piuri, F. Scotti, "A novel pore extraction method for
heterogeneous fingerprint images using Convolutional Neural Networks", in Pattern RecognitionLetters, 2017 (to appear).
4. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Toward Unconstrained Fingerprint Recognition: a Fully Touchless 3D System Based on Two Views on the Move", in IEEE Transactions on Systems, Man, and Cybernetics: Systems, pp. 202-219, February, 2016.
5. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Touchless fingerprint biometrics: a survey on 2D and 3D technologies", in Journal of Internet Technology, pp. 325 - 332, May, 2014. ISSN: 1607-9264.
• Chapters in Books6. R. Donida Labati, F. Scotti, "Fingerprint", in Encyclopedia of Cryptography and Security (2nd
ed.), Springer, pp. 460 - 465, 2011.
Publications (2/3)
• Refereed Papers in Proceedings of International Conferences and Workshops
7. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Contactless Fingerprint Recognition: a Neural Approach for Perspective and Rotation Effects Reduction", in IEEE Workshop on Computational Intelligence in Biometrics and Identity Management, April 16 - 19, 2013.
8. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Two-view Contactless Fingerprint Acquisition Systems: a Case Study for Clay Artworks", in 2012 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications, September, 2012.
9. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Virtual Environment for 3-D SyntheticFingerprints", in 2012 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 48 - 53, July, 2012.
10. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Quality Measurement of UnwrappedThree-dimensional Fingerprints: a Neural Networks Approach", in 2012 International Joint Conference on Neural Networks, pp. 1123 - 1130, June, 2012.
10. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Fast 3-D Fingertip Reconstruction Using a Single Two-View Structured Light Acquisition", in IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BioMS 2011), pp. 1 - 8, September 28 , 2011.
Publications (3/3)
• Refereed Papers in Proceedings of International Conferences and Workshops (Cont’d)
11. R. Donida Labati, V. Piuri, and F. Scotti, "A neural-based minutiae pair identification method for touchless fingeprint images", in IEEE Symposium Series in Computational Intelligence 2011 (SSCI 2011), April, 2011.
12. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Measurement of the principal singular point in fingerprint images: a neural approach", in 2010 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA), pp. 18 - 23, September 6-8, 2010.
13. R. Donida Labati, V. Piuri, F. Scotti, "Neural-based Quality Measurement of Fingerprint Images in Contactless Biometric Systems", in The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1 - 8, July 18-23, 2010.