Last tuesday, you talked about active shape models Data set of 1,500 hand-labeled faces 20 facial...
-
Upload
ella-williams -
Category
Documents
-
view
217 -
download
0
Transcript of Last tuesday, you talked about active shape models Data set of 1,500 hand-labeled faces 20 facial...
1
Last tuesday, you talked about active shape models
• Data set of 1,500 hand-labeled faces• 20 facial features (eyes, eye brows, nose, mouth, chin)• Train 40 individual regressors (x and y positions for each
facial feature)
Problems/Solutions
• Our regressors aren’t perfect, but they’re often wrong in conflicting ways
• Facial feature positions aren’t independent of each other– eg. knowing something about the
position of the eyes ought to provide a clue about the position of the nose
• Face alignment needs to be more than the sum of its parts– Active shape model
3
Shape model
• Given many examples of face shapes (and ignoring the image data)
• Find a lower dimensional model that can still represent the high dimensional examples
4
The Active Shape Model framework
• Build a model of face shapes to go along with our facial feature models, then combine the two into one output
Shape Features Alignment
Shape model
• Problem: We tried to describe our face using a 40-dimensional vector. That gives us a little bit too much flexibility:
• Solution: If 40 dimensions are too many, let’s build a low-dimensional model
Face Dog (?)
Learning a shape model
• Use Principal Component Analysis (PCA) on the feature positions– Represent each example as a 40-dimensional
vector, (x1, y1, x2, y2, …, x20, y20)
– Subtract the mean from each vector– Find the most important (principal) axes
• PCA gives us a total of 40 axes, but if we only select a few of them (say, the top six), we’ll get a low dimensional parameterization
ACTIVE APPEARANCE MODELS
• “Interpretation through synthesis”
• Form a model of the object/image (Learnt from the training dataset)
I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.
(Hard to automate)
• Shape Vector provides alignment
=
42
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Random Aside (can’t escape structure!)
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Antonio Torralba & Aude Oliva (2002)
Averages: Hundreds of images containing a person are
averaged to reveal regularities in the intensity patterns across
all the images.
Random Aside (can’t escape structure!)
Tomasz Malisiewicz, http://www.cs.cmu.edu/~tmalisie/pascal/trainval_mean_large.png
“100 Special Moments” by Jason Salavon
Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml
Given Shape Vectors (positions of set of keypoints)
• Shape Vector provides alignment
=
42Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Collect keypoints from training data in Ms whose columns are training shapes.
Subtract mean shape (s), then run SVD to get (Ms = Us S Vs’).
si = s + Usai
Once you have the shape model for each image, warp the model to the mean configuration:
Now, all faces are the same size and shape and place. So you can create a column vector out of the relevant pixels. Put all those columns together (one for each training image) into Mt
Subtract mean texture (t), then run SVD to get Mt = Ut S Vt’.
ti = t + Utbi
Face model
• All faces parameterized by mean texture t and mean shape s.
• Each face has shape parameter ai, and texture paramter bi.
• Given a shape and a texture, we construct a face as:– Texture = t + Utb
– Shape = s + Usa– Make the texture, then warp it onto the new shape.
– Awkward Mathematical notion.– Warp(Image, shape parameters)– W(t + Utb, s + Usa)
PCA Craziness
• The complete description of the face is [ai,bi].• Let’s put those into one column, make a matrix
M for all those columns and run PCA.
• Why the heck would we do this?
Average faces.
• Alignment is the key!
1. Warp to mean shape
2. Average pixels
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Random Aside
• Enhancing Gender
more same original androgynous more opposite
D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76
Playing with the Parameters
First two modes of shape variation First two modes of gray-level variation
First four modes of appearance variation
Final Projects
• 1 person: Implement any recent paper from CVPR/ ECCV/ ICCV, and apply to some new data.
• 2-3 people, small research project; perhaps starting with recent paper and substantial extension, or one of the following ideas
• Single Image Haze Removal Using Dark Channel Prior – http://research.microsoft.com/en-
us/um/people/jiansun/papers/dehaze_cvpr2009.pdf
Example 1 person projects…
Example 2-3 person projects
Matching historical and modern photos• http://www.thirdview.org/3v/rephotos/index.
html