Post on 08-Apr-2018
8/7/2019 Final Analysis of Facial Expression
1/36
Analysis of Facial ExpressionAnalysis of Facial Expression
using Artificial Neural Networkusing Artificial Neural Network
By
Shaik. Asgher Ali (05241A0509)P. Raghavendra (05241A0544)
B. Sai Kishore (05241A0559)
Under the esteemed guidance of
G.Mallikarjuna Rao(Dept. of CSE)
8/7/2019 Final Analysis of Facial Expression
2/36
AgendaAgenday Aimy Hardware and software Specificationsy Scopey Databasey Data Flow Diagramy Training the neural networky Testingy Screenshotsy Conclusiony Referencesy Demo
8/7/2019 Final Analysis of Facial Expression
3/36
AimAim
y The aim of our project is to develop an
facial expression analysis system which is
capable of recognizing basic human
emotions such as normal, happy,sadsurprised fear angry by takin geometrical
measures and appearance measures and
develop a neural network classificationbased on these messures.The network is
then trained using matlab to classify the
expressions.
8/7/2019 Final Analysis of Facial Expression
4/36
Hardware and softwareHardware and software
SpecificationsSpecificationsy Hardware Specification:
Processor: Intel Pentium 4 and above
Disk space: minimum 10GBRAM 512MB(minimum 1024 MB
recommended)
32 bit graphic card
y Software Specification:
Windows Xp/vista, MatLab, MS Excel
8/7/2019 Final Analysis of Facial Expression
5/36
ScopeScope
y Enable the construction of avatars thatreally simulate a persons facial expression.This is exiting prospectus for industries
like gamingy During e-commerce buying process, it
would be able to identify a personsgestures to know whether or not the
person is intended to make a purchasey To detect that a driver is getting sleepy
and to signal a alert signal
8/7/2019 Final Analysis of Facial Expression
6/36
DatabaseDatabase
y Facial expression images were obtained
from the yalefaces, JAFFE and facial
expression database
y The yalefaces database contained facial
images taken from 15 subjects
y The image sequences were digitized into
320 by 243 pixel arrays with 8-bitprecision for grayscale values
y The image format was gif
8/7/2019 Final Analysis of Facial Expression
7/36
Database continuedDatabase continued
y The Japanese Female Facial Expression
(JAFFE) database contained 150 facial
images posed by 10 Japanese female
models
y The photos were taken at the Psychology
Department in Kyushu University
y The image sequences were digitized into256 by 256 pixel arrays with 8-bit
precision for grayscale values
y
The image format was tiff
8/7/2019 Final Analysis of Facial Expression
8/36
Database continuedDatabase continued
y The Facial Expression database contained
56 facial images
y The image sequences were digitized into
64 by 64 pixel arrays with 8-bit precision
for grayscale values
y The image format was bmp
8/7/2019 Final Analysis of Facial Expression
9/36
Data Flow DiagramData Flow Diagram
8/7/2019 Final Analysis of Facial Expression
10/36
UseUse--case Diagramcase Diagram
8/7/2019 Final Analysis of Facial Expression
11/36
Sequence DiagramSequence Diagram
8/7/2019 Final Analysis of Facial Expression
12/36
Activity DiagramActivity Diagram
8/7/2019 Final Analysis of Facial Expression
13/36
Training the Neural NetworkTraining the Neural Network
y The training of the neural network was a veryimportant step in the overall classification of the facialexpressions
y Multi layered feed forward neural network wasdeveloped and trained to classify differentexpressions based on the fifteen parameters (8normalized real valued, and 7 binary) as input tonetwork
y The targets were the correspondingexpressions(normal, happy, sad, surprised, sleepy,
wink) each subject expressed which was in binaryformat
y The output was then converted to human readableformat so that it could be easily interpreted by us
8/7/2019 Final Analysis of Facial Expression
14/36
Training the NN continuedTraining the NN continued
y Network was trained by using data from 240images
y Training was performed using MATLAB
y Networks were trained using different number ofhidden layers ,different initial weights, ,different
number of neurons in the hidden layers and
different transfer functions (tansig and logsig)
y Each network had fifteen input nodes, eachcorresponding to the fifteen input parameters
8/7/2019 Final Analysis of Facial Expression
15/36
Training the NN continuedTraining the NN continued
y Since the normalized input data was in the rangeof -1 to 1, tansig function was used for the hiddenlayer neurons
y The output of the neural network has to be in
the 0 to 1 range. Hence, the logsig function wasused as the transfer function for the output layerneurons. The output of each node was convertedto a binary number (either 0 or 1)
y An output of 0.6 or more was forced to 1 and an
output of less than 0.6 was forced to 0. An outputof 1 indicated that particular expression waspresent and output of 0 indicated that particularexpression was absent
8/7/2019 Final Analysis of Facial Expression
16/36
Training the NN continuedTraining the NN continued
y The networks were trained using the
back propagation (trainbpx) technique
using MATLAB. The error goal was set
from 1*10-10 - 1*10-30 and the
maximum number of epochs used for
training were varied from 100-2500
8/7/2019 Final Analysis of Facial Expression
17/36
TestingTesting
y
After training the neural network ,it wassubjected to testing using data from
subjects not used in the training
y The output of each node was converted
to a binary number (either 0 or 1). An
output of 0.6 or more was forced to 1
and an output of less than 0.6 was forced
to 0y Table shows the different configurations
of the eight-node output string and its
corresponding output interpretations
8/7/2019 Final Analysis of Facial Expression
18/36
Code corresponding to expressionCode corresponding to expression
Node1Norma
l
Node2Happy Node3Sad Node4Surpris
ed
Node5Wink Node6Sleepy Node7Angry Node8Fear Neuralnetwor
k
output
1 0 0 0 0 0 0 0 Normal
0 1 0 0 0 0 0 0 Happy
0 0 1 0 0 0 0 0 Sad
0 0 0 1 0 0 0 0 Surprised
0 0 0 0 1 0 0 0 Wink
0 0 0 0 0 1 0 0 Sleepy
0 0 0 0 0 0 1 0 Angry
0 0 0 0 0 0 0 1 Fear
8/7/2019 Final Analysis of Facial Expression
19/36
ScreenshotsScreenshots
8/7/2019 Final Analysis of Facial Expression
20/36
ScreenshotsScreenshots
If training selected
8/7/2019 Final Analysis of Facial Expression
21/36
ScreenshotsScreenshots
y If testing selected
8/7/2019 Final Analysis of Facial Expression
22/36
ScreenshotsScreenshots
y Selection of an image
8/7/2019 Final Analysis of Facial Expression
23/36
ScreenshotsScreenshots
y
Real parameter extraction
8/7/2019 Final Analysis of Facial Expression
24/36
ScreenshotsScreenshotsy
Binary parameter extraction
8/7/2019 Final Analysis of Facial Expression
25/36
ScreenshotsScreenshotsy Output of analysis for happy
8/7/2019 Final Analysis of Facial Expression
26/36
ScreenshotsScreenshots
y Asking if satisfied
8/7/2019 Final Analysis of Facial Expression
27/36
ScreenshotsScreenshots
y If YES
8/7/2019 Final Analysis of Facial Expression
28/36
ScreenshotsScreenshots
y If No
8/7/2019 Final Analysis of Facial Expression
29/36
ScreenshotsScreenshotsy Output of analysis for sad
8/7/2019 Final Analysis of Facial Expression
30/36
ScreenshotsScreenshots
y
Output of analysis for angry
8/7/2019 Final Analysis of Facial Expression
31/36
ScreenshotsScreenshots
y Output of analysis for fear
8/7/2019 Final Analysis of Facial Expression
32/36
ConclusionConclusion
y Collected images with frontal facial
expressions from standard databases like
yalefaces,Jaffe,and facial expressions
y Eight real values were extracted from allthe images.Seven binary parameters were
extracted from all the images
y Real values were normalized by dividingall the horizontal distances with maximum
horizontal distance and all the vertical
distances with maximum vertical distance
8/7/2019 Final Analysis of Facial Expression
33/36
Conclusion continuedConclusion continued
y Neural network was developed by using
MATLAB 7.0
y The structure of neural network is that it
has 15 neurons in input layer,10 neuronsin hidden layer,and 8 neurons in output
layer compromising the output
y All the 15 parameters of images werepassed for training
8/7/2019 Final Analysis of Facial Expression
34/36
Conclusion continuedConclusion continued
y Training is a process in which network is
given input and desired output, and the
network learns to classify the expressions
by leaningy Learning is governed by backpropagation
algorithm
y The 15 parameters of images not used fortraining was passed for testing
y the network response is simulated with
input to give the desired output
8/7/2019 Final Analysis of Facial Expression
35/36
ReferencesReferencesy Websites:
http://ddsdx.uthscsa.edu/dig/itdesc.html
http://matworks.com
http://www.face-rec.orgy Textbooks:
Digital Image Processing by Rafael C.Gonzalezand Richard E.Woods
y Papers:
S.S. Kulkarni, N.P. Reddy, S.I Hariharan, FacialImage Based Mood Recognition Using
Committee Neural Networks
8/7/2019 Final Analysis of Facial Expression
36/36
DEMODEMO