Classification of EEG Signals for Brain-Computer Interface

Classification of EEG Signals for Brain-Computer Interface

www.azoft.com

Software Development


Introduction One of the current issues in medical science today is the classification of signals recorded from the brain via electroencephalography (EEG, which is an electrophysiological monitoring method to record electrical activity of the brain). Classification of cerebral cortex signals makes it possible to diagnose various neurological diseases (i.e. epilepsy), as well as create a brain-computer interface for managing robotic devices, including prosthetics for physically disabled patients.

Azoft R&D team, together with Sergey Alyamkin and Expasoft, decided to research the topic of building a brain-computer interface and participated in the «Grasp-and-Lift EEG Detection» competition organized by Kaggle. According to competition rules, participants were given 2 months to identify and classify various movements of the right hand with the lowest error rate using EEG.

The Project Task

To develop a model which helps classify various right-hand movements using EEG.

The Project Plan

1. Studying the Biological Aspects of the Project2. Data Pre-processing3. Identification of Brain Signal Appearance4. Choosing an Appropriate Machine Learning Algorithm5. Comparing Received Models and Sending the Best Result

Implementation

Studying Biological Aspects of the Project

The human brain serves as the center of the nervous system and can be divided into several levels (Image 1), each of which has its own functions:

1st level — the cortex — responsible for managing sensory and motor functions, as well as handling complex cognitive processes.

2nd level — basal ganglia — responsible for managing consensual movements and regulating muscle tone.

3rd level — hippocampus, hypophysis, hypothalamus, callosal gyrus, corpus amygdaloideum - mainly responsible for managing emotional reactions and states, and endocrine regulation.

4th level (the lowest) — reticular formation and other structures of brain stem — responsible for managing vegetative processes. [1]


Image 1. The general structure of the brain.


The image above illustrates the following brain structures: 1) cortex; 2) thalamus; 3) corpus callosum; 4) optic nerve; 5) eye; 6) amygdaloid body; 7) hypothalamus; 8) hypophysis; 9) hippocampus; 10) pons; 11) cerebellum; 12) spinal medulla; 13) medulla oblongata; 14) reticular formation.

As for the anatomic structure, the brain consists of two hemispheres — right and left.

Each hemisphere includes five lobes of cerebrum (Image 2):

1. frontal lobe;2. parietal lobe;3. occipital lobe;4. temporal lobe;5. insular lobe, insula.

Image 2. The lobes of cerebrum

All the data (anatomical, physiological, and clinical) proves that the cerebral cortex plays a leading role in the cerebral organization of physical processes. The cerebral cortex is most differentiated by structure and function areas of the brain.

The cerebral cortex is divided into the following elements: • Paleocortex

• Archicortex

• Mesocortex

• Neocortex


The neocortex of the human brain is most structurally complex - it takes up 96% of all the hemispherical surface. For this reason, we will primarily look at this new part of the brain.

All neocortex areas are constructed uniformly. The most typical for a human is the new six-layered cortex (Image 3), although the layer number is different in various brain parts. Layers differ from each other in their thickness, neuron structure, and arrangement.

The cerebral cortex is heterogeneous and even in one hemisphere has different cell compositions (Image 3). It helps to identify cytoarchitectonic areas of the cortex - the similarly organized centers.

Сytoarchitectonics is a study that looks at properties of various cerebral cortex structures, focusing primarily on cellular composition. In particular, cytoarchitectonics studies distinctive features of various cortex formations and their general cell structure characteristics: size and shape of cell elements, their layer distribution, their location density throughout the whole cortex cross-section and in certain layers, their division to the sublayers, the presence of certain cell forms in various layers, as well as cell distribution in vertical direction.

Image 3. The scheme of neuro and cytoarchitectonic structure of several cortex zones

The borders of cytoarchitectonic areas coincide with functionally specific neocortex areas, which are responsible for various sense organs. Different


studies have been conducted to broaden our knowledge about the human brain’s functional areas, for example via direct cortex stimulation by microelectrodes, positron emission tomography, or other methods using comparison of local destructions of cortex areas with observable behavior fluctuations. [2]

Let’s consider sensorimotor centers of the cerebral cortex in detail.

Image 4: Sensorimotor centers of the cerebral cortex of the human brain (according to scientific studies)

As shown in the image above, the sensorimotor centers of the human brain are: 1) root of the tongue; 2) larynx; 3) palate; 4) lower jaw; 5) tongue; 6) lower part of the face; 7) upper part of the face; 8) neck; 9) fingers; 10) hand; 11) arm from shoulder to wrist; 12) shoulder; 13) shoulder blade; 14) chest; 15) abdomen; 16) lower leg; 17) knee; 18) thigh; 19) toes; 20) the big toe; 21) four toes; 22) foot; 23) face; 24) pharynx. As shown in the image above, the motor area of the cortex is located between the blue and red lines, whereas the sensorimotor area is between the red and green lines. [3]

Finally, we learned that the main brain activity associated with hand movement is in the range from 7 to 30 Hz.


Data Preprocessing

Filtering

The main objects of the study in the EEG signal were eye movement, the movement of the electrodes, the contraction of muscles of the head and the heart, and 50-60Hz network interference. Compared to useful signals, interference caused by movement of eyes, muscles, and electrodes, are arranged at lower frequencies (from 0.1 Hz to 6 Hz). Therefore, we decided to use a bandpass filter with the bandwidth of 7 to 30 Hz.

According to the competition, the developed system should be causal. In other words, you cannot take data from the future in the specific moment of classification. In addition, following the competition rules, the signal classification will be precise if it enters the interval ±150 milliseconds, which equals to ±75 samplings. From there the maximal order of the bandpass filter might be 75*2 = 150.

It is important to note that the digital filtering contributed the least distortion in the signal, so we chose a filter with the finite impulse response. In addition, we used a Kaiser window with beta = 1 for eliminating the Gibbs effect.

Image 5: Amplitude frequency response and phase response characteristics of the derived filter


Image 6: Group delay of the derived filter

As is seen from the derived amplitude-response (blue color) and dynamic-response (red color) characteristics (Image 5), frequencies that are lower than 7 and higher than 30 strain out, and the phase response characteristic is linear. Also, the group delay of the filter equals 75 (Image 6) across the whole frequency line, which satisfies our initial requirements of the filter.

Downsampling

As for decimation, which is the process of reducing the sampling rate of a signal, we dropped it from 500 to 62.5 Hz. Since the maximum useful frequency is 30 Hz, taking into consideration the Nyquist theorem, the sampling rate must be greater than 60 Hz. During downsampling a bandpass filter with the bandwidth of 7 to 30 Hz helps to avoid aliasing.

Identification of Brain Signal AppearanceWe used several different approaches to identify typical signs of brain signals for classifications applicable to machine learning. This was necessary in order to reduce the amount of input data, exclude redundant information, and increase the accuracy of brain signal classification. The testing methods included principal component analysis, electrode selection from the biological viewpoint, and wavelet transform.

First of all, we decided to try one of the most popular methods of identifying typical signs - principal component analysis.


Principal Component Analysis

Principal component analysis has the following main idea related to the project task: the same signal can come from the cortex to several different electrodes (Image 7).

Image 7: Mixing signals on the way from the cortex to electrodes located on the head

If we assume that signals are linear mixing, then the signal vector from detectors:

where – column vector 32x1 – signals from detectors;

– column vector 10x1 – signals from the cortex;

– matrix 32x10 – unknown.

We can transform this correspondence to:

where – Moore-Penrose pseudoinverse.

If we know a matrix , we can set to the classificator entry not the given raw data but the improved data .


We tried to calculate a matrix using SVD (singular value decomposition).

All 32 signals band together to the signal matrix X. Every signal is a matrix column. Matrix X has the size Nx32. This matrix went through a singular value decomposition

,

where – unitary matrixes, - diagonal matrix with all the elements being positive and sorted out in descending order. has a size Nx32, – 32x32, – 32x32.

The first few matrix columns have to be the same deep signals, which appear in the brain when a patient begins to move a hand. The first few columns are the main components. Usually, artifacts are located in the last matrix columns.

When we implemented data analysis with SVD, we didn’t find a correlation between signals from the first matrix columns and the moments when a patient began to move a hand. For this reason, we had to stop using principal component analysis. It was assumed that deep brain signals are mutually orthogonal in calculating compulsory matrix by means of SVD – singular value decomposition. In practice, it turned out that they were not mutually orthogonal. Besides, in the experiment we had 32 detectors, which is not enough for good SVD. It would be better to use ICA (independent component analysis) for nonorthogonal signals. Unfortunately, we ran out of time and didn’t try ICA.


Electrode Selection From the Biological Viewpoint

When choosing electrodes, our decision was based on their spatial arrangement. In the following picture, the electrodes that provided data are labeled with the numbers from 1 to 32.

Image 8: Schematic illustration of electrodes on the surface of the head

There is a phenomenon called hemispheric asymmetry. Along with other meanings, it implies that the left hemisphere is responsible for the right side of the body, with the right hemisphere responsible for the left side of the body. Test subjects moved their right hand, which means a signal appeared in the left hemisphere. Based on this and according to Image 4, we chose the following electrodes: FC5, FC1, T7, C3, Cz, CP5, and CP1.


Wavelet Transform

Having chosen the required electrodes, we decided to focus on wavelet transform as our key method of brain signal identification.

Wavelet transform of one-dimensional signals is widespread in different areas of science - from seismic data processing to speech processing. This approach has a range of advantages compared to spectral analysis in the case of unsteady pulse signals.

The choice of a specific wavelet kind and type depends primarily on analyzed signals and analysis tasks. Today there are no clear methods which could help choose the optimal wavelet for a particular task. That’s why an expert often has to decide what wavelet function to use.

There is an implied principle of wavelet choice: the basic wavelet functions look has to be similar to the typical manipulated signal.

Gauss wavelet was chosen as the basic function gaus2:

Image 9. Gauss wavelet gaus2


Image 10. The result of wavelet transform for the EEG signal

Using this approach, we can analyze the fine structure of signals by means of their simultaneous localization based on both frequency and time.

Choosing an Appropriate Machine Learning Algorithm

When choosing the best machine learning algorithm for the task, we decided to apply the familiar convolutional neural networks (CNN), which are a part of deep learning algorithms.

Сonvolutional Neural Networks

Сonvolutional neural networks are among the most commonly used neural network types, especially for tasks that involve two-dimensional signals (i.e. images). Although, these networks can also work with regular signals.


Image 11: The example of CNN work with a one-dimensional signal

We set 32 one-dimensional signals to the CNN entry over a particular period of time.

Up to some experience, we identified that 4096 samples (up to downsampling) yield the lowest error rate in the final model.

Finally, we got the following model (Image 12):


Image 12: The derived model: CL is the convolutional layer, MP is map pooling, FC is fully connected layer, FM is feature masks.

The derived model enabled us to calculate the area under ROC curve, which equals 0.91983.

The increase in the number of masks, convolutional layers and changes of kernel didn’t bring an improvement. The area under the ROC curve varies between ~0.88 and ~0.91.

Random Forest

The second experimental algorithm was the so-called “random forest”, since it is one of the most popular methods for classification today. This algorithm consists of a set of decisions trees, and each of them gets its own dataset from the total dataset.

For better understanding of how the algorithm works let’s consider the example of the decision tree for an abstract task with two features (Image 13).

Initially elements from two different classes are on the plane. We divide them in a way to separate one class from the other by one horizontal


or vertical line. Next we divide the derived planes in the same way before we get the probability of dominant class higher than a certain limit or the unit depth comes to a maximum, or the element quantity becomes lower than a certain limit.

Image 13: The example of the decision tree for abstract task with two features

The decision tree is not a very stable and reliable algorithm. The algorithm’s high accuracy of classification and flexibility are attainable due to a set of decision trees.

We ran out of time to try a combination of “wavelet transform + random forest” because wavelet transform is very time-consuming.

Recurrent Neural Network

Recurrent neural networks (RNN) are the neural networks with feedback connections. Biological neural networks are recurrent. They have memory. Humans, as well as animals, have memory. Feedback connections gives the network the opportunity to obtain memory.

That allows it to not convey signal history to the network. Moreover,


the memory depth to the past can be determined during learning.

For the recurrent neural network, we used the raw data without removing atrifacts, because RNN does it. Therefore, the recurrent neural network has 32 entries.

We ran multiple experiments with the recurrent neural network, tested various architectures and delays in feedback connections. The network didn’t learn. There was a problem with memory – we could not create a complex network with more than two layers. Then we came to the conclusion that we need to try a special type of neural network – LSTM (long short term memory).

LSTM has inside specific memory units, which are made on the base of controlled backlink with a coefficient of 1.

Image 14. Memory unit of the LSTM recurrent neural network

The decision of whether to turn the backlink on or off is the responsibility of the other neural network part. Turning off the backlink leads to erasing information in the memory core. Thus, the network decides when to remember, use or forget the information.

Nevertheless, the complexity of the LSTM algorithm requires deeper investigation than it was possible under the competition circumstances. For these reasons, we decided to stop at the current stage, without obtaining LSTM testing results.


Conclusion Finally, our R&D team was able to obtain high quality classification of EEG signals during the process of hand movements. Thanks to the quality parameter “area under ROC curve”, which was derived as a result of using convolutional neural networks, we verified that deep learning algorithms are effective in various types of signal classification. Therefore, we can conclude that convolutional neural networks are one of the greatest deep learning methods for the development of brain-computer interfaces. In simple words, show us your EEG signals and we’ll figure out what you were doing at the moment of the EEG recording.

Literature [1] Homskaya E.D. Neiropsychologiya [Neuropsychology]. St. Petersburg, Piter, 2005. 496 p. (in Russian)[2] Kuo C-C, Luu P, Morgan KK, Dow M, Davey C, et al. Localizing Movement-Related Primary Sensorimotor Cortices with Multi-Band EEG Frequency Changes and Functional MRI (2014). Available at: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0112103 (Accessed 3 October 2015)[3] Saveliev S.V. Vozniknovenie mozga cheloveka [The emergence of the human brain]. Moscow, Vedi, 2010. 324 p. (in Russian)[4] H. P. Martinez, Y. Bengio, G. N. Yannakakis Learning deep physiological models of affect. Computational Intelligence Magazine, IEEE, Vol. 8, No. 2. (May 2013), pp. 20-33. Available at: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6496209 (Accessed 4 October 2015)[5] Model Random Forest dlya classificacii, realizaciya na c# (The Random Forest model for classification, c# implementation). Available at: https://habrahabr.ru/post/215453/ (Accessed 7 October 2015)

Classification of EEG Signals for Brain-Computer Interface

Science

Transcript of Classification of EEG Signals for Brain-Computer Interface