Project Report Mid 8th Semester

8/2/2019 Project Report Mid 8th Semester

1/16

Pakistan Institute of Engineering and Applied Sciences

Department of Computer and Information Sciences

Facial Expressions Recognition using Ensembleclassifier approaches

ByAdnan Asif

Project SupervisorDr. Abdul Majid

Project Coordinator

Dr. Mazhar Hussain

Dated: March 22, 2012

Project Progress Report (mid 8th Semester)


2/16

Certificate of Approval

It is certified that the work contained in this document entitled as Project Progress

Report (mid 8th

Semester) for facial expressions recognition using ensemble classifier

approaches was carried out by Adnan Asifunder my supervision and in my opinion it is fully

adequate for the submission to project panel.

Approved by:

Signature: ______________

Supervisor Name: Dr. Abdul Majid

Dated: March 22, 2012


3/16

Preface

The purpose of this document is to let the project supervisor and panel know about progress of

the project. The document contains work carried out during the first half of 8 th semester.

A brief theory on Curvelet Transform was presented alongside with its application to Facial

expressions recognition in the previous report. Different machine learning techniques are also

applied to curvelet transform. The further study of BPNN application to facial expressions

recognition was carried out. A combination of curvelet transform with SVM, KNN and BPNN is

studied and their subjective and objective performance analysis was carried out.


4/16

Table of Contents

Contents

1

Project Plan ............................................................................................................................. 7

1.1 Semester 6 Activities ........................................................................................................ 71.1.1 Literature Survey on expressions recognition........................................................... 71.1.2 Literature Survey on facial expressions databases.................................................... 7

1.2 Summer break Activities .................................................................................................. 71.2.1 Literature Survey on feature extraction techniques .................................................. 71.2.2 Literature Survey on Classification techniques ........................................................ 7

1.3 Semester 7 Activities ........................................................................................................ 71.4 Semester 8 Activities ........................................................................................................ 7

2 Project Overview .................................................................................................................... 93 Databases .............................................................................................................................. 10

3.1 JAFEE ............................................................................................................................ 104 Image preprocessing techniques ........................................................................................... 10

4.1 Normalization ................................................................................................................. 104.2 Histogram Equalization .................................................................................................. 10

5 Feature Extraction techniques ............................................................................................... 105.1 Curvelet Transform ........................................................................................................ 10

6 Dimension Reduction techniques ......................................................................................... 106.1 Principal Components Analysis ..................................................................................... 106.2 K-fold Jack knife ............................................................................................................ 11

7 Classification Techniques ..................................................................................................... 117.1 Techniques used until now ............................................................................................. 117.2 Artificial Neural Network .............................................................................................. 11

7.2.1 Theory ..................................................................................................................... 117.3 Backpropagation Neural Network .................................................................................. 13

7.3.1 Theory ..................................................................................................................... 137.3.2 Implementation ....................................................................................................... 13

8 Experiments and Results ....................................................................................................... 159 Future work ........................................................................................................................... 1610 References ............................................................................................................................. 16


5/16

List of Figures

Figure 1 Some expressions from database .................................................................................... 10Figure 2 Natural neuron ................................................................................................................ 11Figure 3 Artificial Neuron ............................................................................................................ 12Figure 4 A simple BPNN .............................................................................................................. 15


6/16

List of Tables

Table 1 Results from SVM ........................................................................................................... 15Table 2 Results from KNN ........................................................................................................... 15Table 3 Results from BPNN ......................................................................................................... 16


7/16

1 Project Plan

1.1 Semester 6 Activities

1.1.1 Literature Survey on expressions recognition

o Introduction

o Application

o Finding Research groups

o Studying Review/Survey Papers

o Study & Implementing Basic/Well known techniques

1.1.2 Literature Survey on facial expressions databases

o JAFEE databaseo AR database

1.2 Summer break Activities

1.2.1 Literature Survey on feature extraction techniques

o Curvelet transform

o Comparison with wavelet transformation

1.2.2 Literature Survey on Classification techniques

o Learning classification tools such as ANN, SVM, and KNN.

o Study of Basic techniques used in this area and their implementation


o Implementation of existing techniques.

o Testing results of existing techniques.

o Implementation of Curvelet transform with different classification techniques.



8/16

o Comparison of implemented techniques.

o Analysis of these techniques.

o Thesis writing.


9/16

2 Project Overview

Read

Feature Extraction

Dimension Reduction

Training and testing data

Classifier

Decision based on majority voting

Image Database

Image preprocessing

Curvelet transform

Jack Knife

PCA

Ensemble

classifiers

KNN , SVM , ANN

Final

decision


10/16

3 Databases

3.1 JAFEE

The database contains 213 images of 7 facial expressions (6 basic facial expressions + 1 neutral)

posed by 10 Japanese female models. Each image has been rated on 6 emotion adjectives by 60Japanese subjects

Figure 1 Some expressions from database

4 Image preprocessing techniques

4.1 Normalization

4.2 Histogram Equalization

5 Feature Extraction techniques

5.1 Curvelet Transform

Images are decomposed by using discrete curvelet transform in this stage, curvelet coefficients in

different four scales and various angels will be obtained after performing fast curvelet transform.

All the curvelet coefficients are converted to a vector by row [1]

Theory and implementation of curvelet transform was discussed in previous report.

6 Dimension Reduction techniques

6.1 Principal Components Analysis

The amount of curvelet coefficient we obtained in previous stage is huge. To reduce the

dimensionality, PCA[1]

is applied to curvelet coefficient vector too in this facial expression

recognition method

Theory and implementation of PCA was discussed in previous report.


11/16

6.2 K-fold Jack knife

K fold jack knife is applied for the division in learning and testing data.

7 Classification Techniques

7.1 Techniques used until now

1. Support Vector machine (SVM)

2. K-Nearest Neighbor (KNN)

The details of the above techniques and their implementation as well as results have been

discussed in previous reports

7.2 Artificial Neural Network

7.2.1 Theory

An artificial neuron is a computational model inspired in the natural neurons. Natural neurons

receive signals through synapses located on the dendrites or membrane of the neuron. When the

signals received are strong enough (surpass a certain threshold), the neuron is activated and emits

a signal though the axon. This signal might be sent to another synapse, and might activate other

neurons.

Figure 2 Natural neuron


12/16

The complexity of real neurons is highly abstracted when modeling artificial neurons. These

basically consist of inputs (like synapses), which are multiplied by weights (strength of the

respective signals), and then computed by a mathematical function which determines the

activation of the neuron. Another function (which may be the identity) computes the output of

the artificial neuron (sometimes in dependence of a certain threshold). ANNs combine artificial

neurons in order to process information.

Figure 3 Artificial Neuron

The higher a weight of an artificial neuron is, the stronger the input which is multiplied by it will

be. Weights can also be negative, so we can say that the signal is inhibited by the negative

weight. Depending on the weights, the computation of the neuron will be different. By adjusting

the weights of an artificial neuron we can obtain the output we want for specific inputs. But

when we have an ANN of hundreds or thousands of neurons, it would be quite complicated to

find by hand all the necessary weights. But we can find algorithms which can adjust the weights

of the ANN in order to obtain the desired output from the network. This process of adjusting the

weights is called learning or training. The number of types of ANNs and their uses is very high.

Since the first neural model by McCulloch and Pitts (1943) there have been developed hundreds

of different models considered as ANNs. The differences in them might be the functions, the

accepted values, the topology, the learning algorithms, etc. I will discuss only an ANN which

learns using the backpropagation algorithm (Rumelhart and McClelland, 1986) for learning the

appropriate weights, since it is one of the most common models used in ANNs, and many others

are based on it. Since the function of ANNs is to process information, they are used mainly in

fields related with it. There are a wide variety of ANNs that are used to model real neural

networks, and study behavior and control in animals and machines, but also there are ANNs


13/16

which are used for engineering purposes, such as pattern recognition, forecasting, and data

compression.

7.3 Backpropagation Neural Network

7.3.1 Theory

Backpropagation was created by generalizing the Widrow-Hoff learning rule to multiple-layer

networks and nonlinear differentiable transfer functions. Input vectors and the corresponding

target vectors are used to train a network until it can approximate a function, associate input

vectors with specific output vectors, or classify input vectors in an appropriate way as defined.

Networks with biases, a sigmoid layer, and a linear output layer are capable of approximating

any function with a finite number of discontinuities. Standard backpropagation is a gradient

descent algorithm, as is the Widrow-Hoff learning rule, in which the network weights are moved

along the negative of the gradient of the performance function. The term Backpropagation refers

to the manner in which the gradient is computed for nonlinear multilayer networks. There are a

number of variations on the basic algorithm that are based on other standard optimization

techniques, such as conjugate gradient and Newton methods.

Properly trained backpropagation networks tend to give reasonable answers when presented with

inputs that they have never seen. Typically, a new input leads to an output similar to the correct

output for input vectors used in training that are similar to the new input being presented. This

generalization property makes it possible to train a network on a representative set of input/

target pairs and get good results without training the network on all possible input/output pairs.

7.3.2 Implementation

The backpropagation algorithm (Rumelhart and McClelland, 1986) is used in layered feed-forward ANNs. This means that the artificial neurons are organized in layers, and send their

signals forward, and then the errors are propagated backwards. The network receives inputs by

neurons in the input layer, and the output of the network is given by the neurons on an output

layer. There may be one or more intermediate hidden layers. The backpropagation algorithm

uses supervised learning, which means that we provide the algorithm with examples of the inputs


14/16

and outputs we want the network to compute, and then the error (difference between actual and

expected results) is calculated. The idea of the backpropagation algorithm is to reduce this error,

until the ANN learns the training data. The training begins with random weights, and the goal is

to adjust them so that the error will be minimal.

The activation function of the artificial neurons in ANNs implementing the backpropagation

algorithm is a weighted sum (the sum of the inputs xi multiplied by their respective weights wij )

(1)

As the activation depends only on the inputs and the weights. If the output function would be the

identity (output=activation), then the neuron would be called linear. But these have severe

limitations. The most common output function is the sigmoidal function:

(2)

As that the output depends only in the activation, which in turn depends on the values of the

inputs and their respective weights. Now, the goal of the training process is to obtain a desired

output when certain inputs are given. Since the error is the difference between the actual and the

desired output, the error depends on the weights, and there is a need to adjust the weights in

order to minimize the error. The error function can be written for the output of each neuron:

(3)

Square of the difference between the output and the desired target is taken because it will be

always positive, and because it will be greater if the difference is big, and lesser if the difference

is small. The error of the network will simply be the sum of the errors of all the neurons in the

output layer:

(4)

The backpropagation algorithm now calculates how the error depends on the output, inputs, and

weights. After these steps, the weights can be adjusted using the method of gradient descendent:

(5)


15/16

Figure 4 A simple BPNN

8 Experiments and Results

Following are the results of my experiments I performed up to the mid of 8th

semester.

Table 1 Results from SVM

Experiment Number of

classes

Number of

Coefficients

Kernel used Recognition

rate

Curvelet+PCA+SVM 7 38 RBF 93.43%

Curvelet+PCA+SVM 7 38 Linear 83.57%

Curvelet+PCA+SVM 7 38 Polynomial 84.98%

Table 2 Results from KNN


classes

Number of

Coefficients

Number of

nearest

neighbors

Recognition

rate

Curvelet+PCA+KNN 7 70 1 89.67%




16/16

Table 3 Results from BPNN


classes

Number of

Coefficients

Number of

hidden

layers

Number

of

Neurons

Recognition

rate

Curvelet+PCA+BPNN 7 70 1 40 82.63%

9 Future work

o PCA will be combined with LDA to increase performance (optional).

o Sensitivity, ROC etc. will be analyzed.

o In end, Ensemble will be taken of all the classifiers.

o Accuracy comparison of each class (7 classes).

10 References

[1] Cands. Emmanuel, Demanet. Laurent and Donoho.David ,et.al , Fast Discrete Curvelet

Transforms. Multiscale Modeling and Simulation, 5(3). pp. 861-899,2006

[2] Angshul Majumdar and Arusharka Bhattacharya , A Comparative Study in Wavelets,

Curvelets and Contourlets as Feature Sets for Pattern Recognition, The International Arab

Journal of Information Technology, 6(1), pp. 47-51, January 2009.

Project Report Mid 8th Semester

Documents

Transcript of Project Report Mid 8th Semester