Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace...
Transcript of Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace...
![Page 1: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/1.jpg)
Face recognition
16-385 Computer VisionSpring 2018, Lecture 21http://www.cs.cmu.edu/~16385/
![Page 2: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/2.jpg)
• Homework 6 has been posted and is due on April 27th. - Any questions about the homework?- How many of you have looked at/started/finished homework 6?
• VASC Seminar today: Burak Uzkent, “Object Detection and Tracking on Low Resolution Aerial Images”.
- Monday 3-4pm, NSH 3305.
Course announcements
![Page 3: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/3.jpg)
• Face recognition.
• Eigenfaces.
• Fisherfaces.
• Deep faces.
• Face detection.
• Boosting.
• Viola-Jones algorithm.
Overview of today’s lecture
![Page 4: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/4.jpg)
Slide credits
Most of these slides were adapted from:
• Kris Kitani (16-385, Spring 2017).
• Fei-Fei Li (Stanford University).
• Jia-Bin Huang (Virginia Tech).
![Page 5: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/5.jpg)
![Page 6: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/6.jpg)
![Page 7: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/7.jpg)
![Page 8: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/8.jpg)
![Page 9: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/9.jpg)
![Page 10: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/10.jpg)
![Page 11: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/11.jpg)
![Page 12: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/12.jpg)
Face recognition: overview
•Typical scenario: few examples per face, identify or verify test example
•Why it’s hard?• changes in expression, lighting, age, occlusion,
viewpoint
•Basic approaches (all nearest neighbor)1. Project into a new subspace (or kernel space) (e.g.,
“Eigenfaces”=PCA)2. Measure face features3. Make 3d face model, compare shape+appearance, e.g.,
Active Appearance Model (AAM)
![Page 13: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/13.jpg)
Typical face recognition scenarios
• Verification: a person is claiming a particular identity; verify whether that is true
• E.g., security
• Closed-world identification: assign a face to one person from among a known set
• General identification: assign a face to a known person or to “unknown”
![Page 14: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/14.jpg)
Detection versus
Recognition
07-Nov-1714
Detection finds the faces in images Recognition recognizes WHO the person is
![Page 15: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/15.jpg)
Face Recognition
• Digital photography
07-Nov-1715
![Page 16: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/16.jpg)
Face Recognition
• Digital photography
• Surveillance
07-Nov-1716
![Page 17: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/17.jpg)
Face Recognition
• Digital photography
• Surveillance
• Album organization
07-Nov-1717
![Page 18: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/18.jpg)
Face Recognition
• Digital photography
• Surveillance
• Album organization
• Person tracking/id.
07-Nov-1718
![Page 19: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/19.jpg)
Face Recognition
• Digital photography
• Surveillance
• Album organization
• Person tracking/id.
• Emotions and
expressions
07-Nov-1719
![Page 20: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/20.jpg)
Applications of Face Recognition
• Surveillance
![Page 21: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/21.jpg)
Facebook friend-tagging with auto-suggest
![Page 22: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/22.jpg)
Face recognition: once you’ve detected and cropped a face, try to recognize it
Detection Recognition “Sally”
![Page 23: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/23.jpg)
What makes face recognition hard?
Expression
![Page 24: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/24.jpg)
What makes face recognition hard?
Lighting
![Page 25: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/25.jpg)
What makes face recognition hard?
Occlusion
![Page 26: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/26.jpg)
What makes face recognition hard?
Viewpoint
![Page 27: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/27.jpg)
![Page 28: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/28.jpg)
Simple idea for face recognition1. Treat face image as a vector of intensities
2. Recognize face by nearest neighbor in database
x
nyy ...
1
xy k
k
k argmin
![Page 29: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/29.jpg)
The space of all face images
•When viewed as vectors of pixel values, face images are extremely high-dimensional
• 100x100 image = 10,000 dimensions
• Slow and lots of storage
•But very few 10,000-dimensional vectors are valid face images
•We want to effectively model the subspace of face images
![Page 30: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/30.jpg)
100x100 images can
contain many things other
than faces!
07-Nov-1730
![Page 31: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/31.jpg)
The Space of Faces
• An image is a point in a high
dimensional space
• If represented in grayscale intensity,
an N x M image is a point in RNM
• E.g. 100x100 image = 10,000 dim
07-Nov-1731
Slide credit: Chuck Dyer, Steve Seitz, Nishino
![Page 32: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/32.jpg)
The space of all face images
•Eigenface idea: construct a low-dimensional linear subspace that best explains the variation in the set of face images
![Page 33: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/33.jpg)
The Space of Faces
• An image is a point in a high
dimensional space
• If represented in grayscale intensity,
an N x M image is a point in RNM
• E.g. 100x100 image = 10,000 dim
• However, relatively few high
dimensional vectors correspond
to valid face images
• We want to effectively model the
subspace of face images
07-Nov-1733
Slide credit: Chuck Dyer, Steve Seitz, Nishino
ɸ1
![Page 34: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/34.jpg)
Image space
Face space
• Maximize the scatter of the training images in face space
• Compute n-dim subspace such that the projection of the data points onto the subspace has the largest variance among all n-dim subspaces.
07-Nov-1734
![Page 35: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/35.jpg)
Eigenfaces: key idea
• Assume that most face images lie on a low-
dimensional subspace determined by the first k
(k<<d) directions of maximum variance
• Use PCA to determine the vectors or “eigenfaces”
that span that subspace
• Represent all face images in the dataset as linear
combinations of eigenfaces
07-Nov-1735
M. Turk and A. Pentland, Face Recognition using Eigenfaces, CVPR 1991
![Page 36: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/36.jpg)
Training images: x1,…,xN
07-Nov-1736
![Page 37: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/37.jpg)
Top eigenvectors: ɸ1,…,ɸk
07-Nov-1737
Mean: μ
![Page 38: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/38.jpg)
Visualization of eigenfaces
07-Nov-1738
Principal component (eigenvector) ɸk
μ + 3σkɸk
μ – 3σkɸk
![Page 39: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/39.jpg)
Eigenface algorithm• Training
1. Align training images x1, x2, …, xN
1. Compute average face
2. Compute the difference image (the centered data matrix)
07-Nov-1739
Note that each image is formulated into a long vector!
m =1
Nxiå
![Page 40: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/40.jpg)
Eigenface algorithm4. Compute the covariance matrix
4. Compute the eigenvectors of the covariance matrix Σ
5. Compute each training image xi ‘s projections as
6. Visualize the estimated training face xi
07-Nov-1740
xi ® xic ×f1, xi
c ×f2,..., xic ×fK( ) º a1,a2, ... ,aK( )
xi » m +a1f1 +a2f2 +...+aKfK
![Page 41: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/41.jpg)
Eigenface algorithm
07-Nov-1741
a1f1 a2f2 aKfK...Reconstructed training face
𝑥𝑖
xi ® xic ×f1, xi
c ×f2,..., xic ×fK( ) º a1,a2, ... ,aK( )
xi » m +a1f1 +a2f2 +...+aKfK
![Page 42: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/42.jpg)
Eigenvalues (variance along eigenvectors)
07-Nov-1742
![Page 43: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/43.jpg)
• Only selecting the top K eigenfaces reduces the dimensionality.
• Fewer eigenfaces result in more information loss, and hence less discrimination between faces.
Reconstruction and Errors
K = 4
K = 200
K = 400
07-Nov-1743
![Page 44: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/44.jpg)
Eigenface algorithm• Testing
1. Take query image t
2. Project into eigenface space and compute projection
3. Compare projection w with all N training projections
• Simple comparison metric: Euclidean
• Simple decision: K-Nearest Neighbor
(note: this “K” refers to the k-NN algorithm, different from the previous K’s referring to the # of principal components)
07-Nov-1744
t® (t -m)×f1, (t -m)×f2,..., (t -m) ×fK( ) º w1,w2,...,wK( )
![Page 45: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/45.jpg)
07-Nov-1746
Shortcomings• Requires carefully controlled data:
• All faces centered in frame
• Same size
• Some sensitivity to angle
• Alternative:
• “Learn” one set of PCA vectors for each angle
• Use the one with lowest error
• Method is completely knowledge free
• (sometimes this is good!)
• Doesn’t know that faces are wrapped around 3D objects (heads)
• Makes no effort to preserve class distinctions
![Page 46: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/46.jpg)
Summary for Eigenface
Pros• Non-iterative, globally optimal solution
Limitations
• PCA projection is optimal for reconstruction from a low dimensional basis, but may NOT be optimal for discrimination…
07-Nov-1747
![Page 47: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/47.jpg)
07-Nov-1748
Besides face recognitions,
we can also do
Facial expression recognition
![Page 48: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/48.jpg)
Happiness subspace
07-Nov-1749
![Page 49: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/49.jpg)
Disgust subspace
07-Nov-1750
![Page 50: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/50.jpg)
Which direction will is the first
principle component?
07-Nov-1751
![Page 51: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/51.jpg)
Fischer’s Linear Discriminant
Analysis
• Goal: find the best separation between two classes
07-Nov-1752
![Page 52: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/52.jpg)
Difference between PCA
and LDA
• PCA preserves maximum variance
• LDA preserves discrimination
• Find projection that maximizes scatter between
classes and minimizes scatter within classes
07-Nov-1753
![Page 53: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/53.jpg)
Illustration of the Projection
Poor Projection
x1
x2
x1
x2
• Using two classes as example:
Good
07-Nov-1754
![Page 54: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/54.jpg)
07-Nov-1755
Basic intuition: PCA vs. LDA
![Page 55: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/55.jpg)
07-Nov-17
56
LDA with 2 variables
• We want to learn a projection W such that the projection converts all the points from x to a new space (For this example, assume m == 1):
• Let the per class means be:
• And the per class covariance matrices be:
• We want a projection that maximizes:
𝑧 = 𝑤𝑇𝑥
![Page 56: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/56.jpg)
Fischer’s Linear Discriminant
Analysis
07-Nov-1757
![Page 57: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/57.jpg)
07-Nov-17
58
LDA with 2 variables
The following objective function:
Can be written as
![Page 58: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/58.jpg)
LDA with 2 variables
• We can write the between class scatter as:
• Also, the within class scatter becomes:
07-Nov-1759
![Page 59: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/59.jpg)
LDA with 2 variables
07-Nov-1760
• We can plug in these scatter values to our objective
function:
• And our objective becomes:
![Page 60: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/60.jpg)
LDA with 2 variables
• The scatter variables
07-Nov-1761
![Page 61: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/61.jpg)
2S
1S
BS
21SSS
W
x1
x2Within class scatter
Between class scatter
07-Nov-1762
Visualization
![Page 62: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/62.jpg)
Linear Discriminant Analysis (LDA)
• Maximizing the ratio
• Is equivalent to maximizing the numerator while keeping the denominator constant, i.e.
• And can be accomplished using Lagrange multipliers, where we define the Lagrangian as
• And maximize with respect to both w and λ
07-Nov-1763
![Page 63: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/63.jpg)
Linear Discriminant Analysis (LDA)
• Setting the gradient of
With respect to w to zeros we get
• This is a generalized eigenvalue problem
• The solution is easy when exists
07-Nov-1764
![Page 64: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/64.jpg)
Linear Discriminant Analysis (LDA)• In this case
• And using the definition of SB
• Noting that (μ1-μ0)Tw=α is a scalar this can be written as
• and since we don’t care about the magnitude of w
07-Nov-1765
![Page 65: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/65.jpg)
07-Nov-1766
LDA with N variables and C classes
![Page 66: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/66.jpg)
Variables• N Sample images:
• C classes:
• Average of each class:
• Average of all data:
N
xx ,,1
N
kk
xN 1
1
ikx
k
i
ix
N
1
07-Nov-1767
![Page 67: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/67.jpg)
Scatter Matrices
• Scatter of class i:
c
i
iWSS
1
c
i
T
iiiBNS
1
• Within class scatter:
• Between class scatter:
T
ik
x
ikixxS
ik
07-Nov-1768
![Page 68: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/68.jpg)
Mathematical Formulation
• Recall that we want to learn a projection W
such that the projection converts all the points
from x to a new space z:
• After projection:
– Between class scatter
– Within class scatter
• So, the objective becomes:
WSWSB
T
B
~
WSWSW
T
W
~
WSW
WSW
S
SW
W
T
B
T
W
B
optWW
max arg~
~
max arg
07-Nov-1769
𝑧 = 𝑤𝑇𝑥
![Page 69: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/69.jpg)
Mathematical Formulation
• Solve generalized eigenvector problem:
07-Nov-1770
WSW
WSWW
W
T
B
T
optW
max arg
miwSwSiWiiB
,,1
![Page 70: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/70.jpg)
Mathematical Formulation
• Solution: Generalized Eigenvectors
• Rank of Wopt is limited
– Rank(SB) <= |C|-1
– Rank(SW) <= N-C
miwSwSiWiiB
,,1
07-Nov-1771
![Page 71: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/71.jpg)
PCA vs. LDA
• Eigenfaces exploit the max scatter of the training images in face space
• Fisherfaces attempt to maximise the between class scatter, while minimising the within class scatter.
07-Nov-1772
![Page 72: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/72.jpg)
07-Nov-1773
Basic intuition: PCA vs. LDA
![Page 73: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/73.jpg)
Results: Eigenface vs. Fisherface
• Variation in Facial Expression, Eyewear, and Lighting
• Input: 160 images of 16 people
• Train: 159 images
• Test: 1 image
With glasses
Without glasses
3 Lighting conditions
5 expressions
07-Nov-1774
![Page 74: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/74.jpg)
Eigenfaces vs. Fisherfaces
07-Nov-1775
![Page 75: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/75.jpg)
Large scale comparison of methods
• FRVT 2006 Report
• Not much (or any) information available about methods, but gives idea of what is doable
![Page 76: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/76.jpg)
FVRT Challenge: interesting findings
• Left: Major progress since Eigenfaces
• Right: Computers outperformed humans in controlled settings (cropped frontal face, known lighting, aligned)
• Humans outperform greatly in less controlled settings (viewpoint variation, no crop, no alignment, change in age, etc.)
False Rejection
Rate at False
Acceptance Rate =
0.001
![Page 77: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/77.jpg)
• Most recent research focuses on “faces in the wild”, recognizing faces in normal photos
• Classification: assign identity to face• Verification: say whether two people are the same
• Important steps1. Detect2. Align3. Represent4. Classify
State-of-the-art Face Recognizers
![Page 78: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/78.jpg)
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Yang, Ranzato, & Wolf (Facebook, Tel Aviv), CVPR 2014
![Page 79: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/79.jpg)
Face Alignment
1. Detect a face and 6 fiducial markers using a support vector regressor (SVR)
2. Iteratively scale, rotate, and translate image until it aligns with a target face
3. Localize 67 fiducial points in the 2D aligned crop
4. Create a generic 3D shape model by taking the average of 3D scans from the USF Human-ID database and manually annotate the 67 anchor points
5.Fit an affine 3D-to-2D camera and use it to direct the warping of the face
![Page 80: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/80.jpg)
Train DNN classifier on aligned faces
Architecture (deep neural network classifier)
• Two convolutional layers (with one pooling layer)
• 3 locally connected and 2 fully connected layers
• > 120 million parameters
Train on dataset with 4400 individuals, ~1000 images each
• Train to identify face among set of possible people
Verification is done by comparing features at last layer for two faces
![Page 81: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/81.jpg)
Results: Labeled Faces in the Wild Dataset
Performs similarly to humans!(note: humans would do better with uncropped faces)
Experiments show that alignment is crucial (0.97 vs 0.88) and that deep features help (0.97 vs. 0.91)
![Page 82: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/82.jpg)
OpenFace (FaceNet)
FaceNet: A Unified Embedding for Face Recognition and Clustering, CVPR 2015 http://cmusatyalab.github.io/openface/
![Page 83: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/83.jpg)
MegaFace Benchmark
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale, CVPR 2016
![Page 84: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/84.jpg)
Detection versus
Recognition
07-Nov-1791
Detection finds the faces in images Recognition recognizes WHO the person is
![Page 85: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/85.jpg)
![Page 86: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/86.jpg)
![Page 87: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/87.jpg)
![Page 88: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/88.jpg)
![Page 89: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/89.jpg)
![Page 90: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/90.jpg)
![Page 91: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/91.jpg)
![Page 92: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/92.jpg)
![Page 93: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/93.jpg)
![Page 94: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/94.jpg)
![Page 95: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/95.jpg)
![Page 96: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/96.jpg)
![Page 97: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/97.jpg)
![Page 98: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/98.jpg)
![Page 99: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/99.jpg)
![Page 100: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/100.jpg)
![Page 101: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/101.jpg)
![Page 102: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/102.jpg)
![Page 103: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/103.jpg)
![Page 104: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/104.jpg)
![Page 105: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/105.jpg)
Multi-Scale Oriented Patches (MOPS)Multi-Image Matching using Multi-Scale Oriented Patches. M. Brown, R. Szeliski and S. Winder.
International Conference on Computer Vision and Pattern Recognition (CVPR2005). pages 510-517
Given a feature
Get 40 x 40 image patch, subsample
every 5th pixel(low frequency filtering, absorbs localization errors)
Subtract the mean, divide by standard
deviation(removes bias and gain)
Haar Wavelet Transform(low frequency projection)
![Page 106: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/106.jpg)
Haar Wavelets(actually, Haar-like features)
Use responses of a bank of filters as a descriptor
![Page 107: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/107.jpg)
Haar wavelets filters
Haar wavelet responses can be computed with filtering
image patch
-1-1
+1+1
Haar wavelet responses can be
computed efficiently (in constant time)
with integral images
![Page 108: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/108.jpg)
Integral Image
1 5 2
2 4 1
2 1 1
1 6 8
3 12 15
5 15 19
original
image
integral
image
![Page 109: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/109.jpg)
Integral Image
1 5 2
2 4 1
2 1 1
1 6 8
3 12 15
5 15 19
original
image
integral
image
Can find the sum of any block using 3 operations
![Page 110: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/110.jpg)
1 5 2
2 4 1
2 1 1
1 6 8
3 12 15
5 15 19
image integral image
What is the sum of the bottom right 2x2 square?
![Page 111: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/111.jpg)
![Page 112: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/112.jpg)
![Page 113: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/113.jpg)
![Page 114: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/114.jpg)
![Page 115: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/115.jpg)
![Page 116: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/116.jpg)
![Page 117: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/117.jpg)
![Page 118: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/118.jpg)
![Page 119: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/119.jpg)
![Page 120: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/120.jpg)
![Page 121: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/121.jpg)
![Page 122: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/122.jpg)
![Page 123: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/123.jpg)
![Page 124: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/124.jpg)
![Page 125: Face recognition - Carnegie Mellon School of Computer Science16385/s18/lectures/lecture21.pdfFace recognition: overview •Typical scenario: few examples per face, identify or verify](https://reader036.fdocuments.net/reader036/viewer/2022062609/60f2ef0616fc494acc7e1359/html5/thumbnails/125.jpg)
References
Basic reading:• Szeliski, Sections 14.1.1, 14.2.