Final Analysis of Facial Expression

8/7/2019 Final Analysis of Facial Expression

1/36

Analysis of Facial ExpressionAnalysis of Facial Expression

using Artificial Neural Networkusing Artificial Neural Network

By

Shaik. Asgher Ali (05241A0509)P. Raghavendra (05241A0544)

B. Sai Kishore (05241A0559)

Under the esteemed guidance of

G.Mallikarjuna Rao(Dept. of CSE)


2/36

AgendaAgenday Aimy Hardware and software Specificationsy Scopey Databasey Data Flow Diagramy Training the neural networky Testingy Screenshotsy Conclusiony Referencesy Demo


3/36

AimAim

y The aim of our project is to develop an

facial expression analysis system which is

capable of recognizing basic human

emotions such as normal, happy,sadsurprised fear angry by takin geometrical

measures and appearance measures and

develop a neural network classificationbased on these messures.The network is

then trained using matlab to classify the

expressions.


4/36

Hardware and softwareHardware and software

SpecificationsSpecificationsy Hardware Specification:

Processor: Intel Pentium 4 and above

Disk space: minimum 10GBRAM 512MB(minimum 1024 MB

recommended)

32 bit graphic card

y Software Specification:

Windows Xp/vista, MatLab, MS Excel


5/36

ScopeScope

y Enable the construction of avatars thatreally simulate a persons facial expression.This is exiting prospectus for industries

like gamingy During e-commerce buying process, it

would be able to identify a personsgestures to know whether or not the

person is intended to make a purchasey To detect that a driver is getting sleepy

and to signal a alert signal


6/36

DatabaseDatabase

y Facial expression images were obtained

from the yalefaces, JAFFE and facial

expression database

y The yalefaces database contained facial

images taken from 15 subjects

y The image sequences were digitized into

320 by 243 pixel arrays with 8-bitprecision for grayscale values

y The image format was gif


7/36

Database continuedDatabase continued

y The Japanese Female Facial Expression

(JAFFE) database contained 150 facial

images posed by 10 Japanese female

models

y The photos were taken at the Psychology

Department in Kyushu University

y The image sequences were digitized into256 by 256 pixel arrays with 8-bit

precision for grayscale values

y

The image format was tiff


8/36

Database continuedDatabase continued

y The Facial Expression database contained

56 facial images

y The image sequences were digitized into

64 by 64 pixel arrays with 8-bit precision

for grayscale values

y The image format was bmp


9/36

Data Flow DiagramData Flow Diagram


10/36

UseUse--case Diagramcase Diagram


11/36

Sequence DiagramSequence Diagram


12/36

Activity DiagramActivity Diagram


13/36

Training the Neural NetworkTraining the Neural Network

y The training of the neural network was a veryimportant step in the overall classification of the facialexpressions

y Multi layered feed forward neural network wasdeveloped and trained to classify differentexpressions based on the fifteen parameters (8normalized real valued, and 7 binary) as input tonetwork

y The targets were the correspondingexpressions(normal, happy, sad, surprised, sleepy,

wink) each subject expressed which was in binaryformat

y The output was then converted to human readableformat so that it could be easily interpreted by us


14/36

Training the NN continuedTraining the NN continued

y Network was trained by using data from 240images

y Training was performed using MATLAB

y Networks were trained using different number ofhidden layers ,different initial weights, ,different

number of neurons in the hidden layers and

different transfer functions (tansig and logsig)

y Each network had fifteen input nodes, eachcorresponding to the fifteen input parameters


15/36


y Since the normalized input data was in the rangeof -1 to 1, tansig function was used for the hiddenlayer neurons

y The output of the neural network has to be in

the 0 to 1 range. Hence, the logsig function wasused as the transfer function for the output layerneurons. The output of each node was convertedto a binary number (either 0 or 1)

y An output of 0.6 or more was forced to 1 and an

output of less than 0.6 was forced to 0. An outputof 1 indicated that particular expression waspresent and output of 0 indicated that particularexpression was absent


16/36


y The networks were trained using the

back propagation (trainbpx) technique

using MATLAB. The error goal was set

from 1*10-10 - 1*10-30 and the

maximum number of epochs used for

training were varied from 100-2500


17/36

TestingTesting

y

After training the neural network ,it wassubjected to testing using data from

subjects not used in the training

y The output of each node was converted

to a binary number (either 0 or 1). An

output of 0.6 or more was forced to 1

and an output of less than 0.6 was forced

to 0y Table shows the different configurations

of the eight-node output string and its

corresponding output interpretations


18/36

Code corresponding to expressionCode corresponding to expression

Node1Norma

l

Node2Happy Node3Sad Node4Surpris

ed

Node5Wink Node6Sleepy Node7Angry Node8Fear Neuralnetwor

k

output

1 0 0 0 0 0 0 0 Normal

0 1 0 0 0 0 0 0 Happy

0 0 1 0 0 0 0 0 Sad

0 0 0 1 0 0 0 0 Surprised

0 0 0 0 1 0 0 0 Wink

0 0 0 0 0 1 0 0 Sleepy

0 0 0 0 0 0 1 0 Angry

0 0 0 0 0 0 0 1 Fear


19/36

ScreenshotsScreenshots


20/36


If training selected


21/36


y If testing selected


22/36


y Selection of an image


23/36


y

Real parameter extraction


24/36

ScreenshotsScreenshotsy

Binary parameter extraction


25/36

ScreenshotsScreenshotsy Output of analysis for happy


26/36


y Asking if satisfied


27/36


y If YES


28/36


y If No


29/36

ScreenshotsScreenshotsy Output of analysis for sad


30/36


y

Output of analysis for angry


31/36


y Output of analysis for fear


32/36

ConclusionConclusion

y Collected images with frontal facial

expressions from standard databases like

yalefaces,Jaffe,and facial expressions

y Eight real values were extracted from allthe images.Seven binary parameters were

extracted from all the images

y Real values were normalized by dividingall the horizontal distances with maximum

horizontal distance and all the vertical

distances with maximum vertical distance


33/36

Conclusion continuedConclusion continued

y Neural network was developed by using

MATLAB 7.0

y The structure of neural network is that it

has 15 neurons in input layer,10 neuronsin hidden layer,and 8 neurons in output

layer compromising the output

y All the 15 parameters of images werepassed for training


34/36

Conclusion continuedConclusion continued

y Training is a process in which network is

given input and desired output, and the

network learns to classify the expressions

by leaningy Learning is governed by backpropagation

algorithm

y The 15 parameters of images not used fortraining was passed for testing

y the network response is simulated with

input to give the desired output


35/36

ReferencesReferencesy Websites:

http://ddsdx.uthscsa.edu/dig/itdesc.html

http://matworks.com

http://www.face-rec.orgy Textbooks:

Digital Image Processing by Rafael C.Gonzalezand Richard E.Woods

y Papers:

S.S. Kulkarni, N.P. Reddy, S.I Hariharan, FacialImage Based Mood Recognition Using

Committee Neural Networks


36/36

DEMODEMO

Final Analysis of Facial Expression

Documents

Transcript of Final Analysis of Facial Expression