EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE...

15
EMD APPROACH TO MULTICHANNEL EEG DATA THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced Brain Signal Processing, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi, Saitama 351-0198, Japan [email protected] DANILO P. MANDIC Communication and Signal Processing Research Group, Department of Electrical and Electronic Engineering, Imperial College, Exhibition Road, London, SW7 2BT, United Kingdom [email protected] ANDRZEJ CICHOCKI Laboratory for Advanced Brain Signal Processing, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi, Saitama 351-0198, Japan [email protected] ANDRZEJ W. PRZYBYSZEWSKI Department of Neurology, University of Massachusetts Medical School, MA, USA [email protected] Human brains exhibit a possibility to control directly the intelligent computing applications in form of brain computer/machine interfacing (BCI/BMI) technologies. Neurophysiological sig- nals and especially electroencephalogram (EEG) are the forms of brain electrical activity which can be easily captured and utilized for BCI/BMI applications. Those signals are unfortunately usually very highly contaminated by external noise caused by the presence of di®erent devices in the environment creating electromagnetic interference. In this paper, we ¯rst decompose each of the recorded channels, in multichannel EEG recording environment, into intrinsic mode func- tions (IMF) which are a result of empirical mode decomposition (EMD) extended to multi- channel analysis. We present novel and interesting results on human mental and cognitive states estimation based on analysis of the above-mentioned stimuli-related IMF components. The IMF components are further clustered for their spectral similarity in order to identify only those carrying responses to present stimuli to the subjects. The resulting targets only Journal of Circuits, Systems, and Computers Vol. 19, No. 1 (2010) 215229 # . c World Scienti¯c Publishing Company DOI: 10.1142/S0218126610006037 215

Transcript of EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE...

Page 1: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

EMD APPROACH TO MULTICHANNEL EEGDATA — THE AMPLITUDE AND PHASECOMPONENTS CLUSTERING ANALYSIS

TOMASZ M. RUTKOWSKI

Laboratory for Advanced Brain Signal Processing,RIKEN Brain Science Institute, 2-1 Hirosawa, Wako-shi,

Saitama 351-0198, [email protected]

DANILO P. MANDIC

Communication and Signal Processing Research Group,Department of Electrical and Electronic Engineering,

Imperial College, Exhibition Road, London,SW7 2BT, United [email protected]

ANDRZEJ CICHOCKI

Laboratory for Advanced Brain Signal Processing,RIKEN Brain Science Institute,

2-1 Hirosawa, Wako-shi, Saitama 351-0198, [email protected]

ANDRZEJ W. PRZYBYSZEWSKI

Department of Neurology,University of Massachusetts Medical School, MA, USA

[email protected]

Human brains exhibit a possibility to control directly the intelligent computing applications inform of brain computer/machine interfacing (BCI/BMI) technologies. Neurophysiological sig-nals and especially electroencephalogram (EEG) are the forms of brain electrical activity whichcan be easily captured and utilized for BCI/BMI applications. Those signals are unfortunatelyusually very highly contaminated by external noise caused by the presence of di®erent devices inthe environment creating electromagnetic interference. In this paper, we ¯rst decompose each ofthe recorded channels, in multichannel EEG recording environment, into intrinsic mode func-tions (IMF) which are a result of empirical mode decomposition (EMD) extended to multi-channel analysis. We present novel and interesting results on human mental and cognitivestates estimation based on analysis of the above-mentioned stimuli-related IMF components.The IMF components are further clustered for their spectral similarity in order to identify onlythose carrying responses to present stimuli to the subjects. The resulting targets only

Journal of Circuits, Systems, and ComputersVol. 19, No. 1 (2010) 215!229#.c World Scienti¯c Publishing CompanyDOI: 10.1142/S0218126610006037

215

Page 2: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

reconstruction allows us to identify when and to which stimuli intelligent application user istuning at a time.

Keywords: Brain synchrony; brain signal processing; EMD application to EEG.

1. Introduction

Online brain states analysis based on noninvasive monitoring techniques such aselectroencephalogram (EEG) have received much attention recently due to thegrowing interest and popularity of research related to brain computer/machineinterfacing (BCI/BMI) techniques, owing to the very exciting possibility of com-puter-aided communication with the outside world. A new and growing interest inneuroscience, so-called steady-state potentials1!3 stimuli technique, which produceslonger in-time and more easy to detect within monitored EEG steady responsescontributes also to EEG signal processing's recent popularity.

EEG based brain stages monitoring is achieved in a noninvasive recording setup.The noninvasive brain monitoring method posesses several important and di±cultproblems. In terms of signal processing these include the detection, estimation,interpretation and modeling of brain activities, and cross-user transparency.4

It comes as no surprise, therefore, that this technology is envisaged to be at thecore of future \intelligent computing". Other industries which would bene¯t greatlyfrom the development of online analysis and visualization of brain states include theprosthetics, entertainment, virtual reality, and computer games industries, where thecontrol and navigation in a computer-aided application is achieved without resortingto using muscles, hands, or any gestures (peripheral nervous system in general).Instead, the onset of \planning an action" recorded from the head (scalp) surface,and the relevant information is \decoded" from this information carrier.

Apart from purely signal conditioning problems, in most BCI/BMI experimentsother issues such as user training and adaptation, inevitably cause di±culties andlimit a wide spread of this technology due to the lack of \generality" caused by cross-user di®erences.4 To help mitigate some of the above-mentioned issues, we propose tomake use of a new and growing interest in signal processing community technique ofempirical mode decomposition (EMD)5 which we extend to multichannel approachof parallel decomposition of single channel signals and further clustering ofso-obtained components among channels to track coherent (synchronized or corre-lated in spectral domain) activities in complex signals as EEG.

In the proposed approach, we analyze responses from experiments based on avisual stimuli which were conducted with in the Laboratory for Advanced BrainSignal Processing, BSI RIKEN, within the so-called steady-state visual evokedpotential (SSVEP) mode.1,6 Within this framework, the subjects are asked to focustheir attention on a simple °ashing stimuli, whose frequency is known to cause aphysiologically stable response traceable within EEG.6,7 This way, the proposedmultichannel and multimodal signal decomposition scheme uses the EEG captured

216 T. M. Rutkowski et al.

Page 3: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

by several electrodes, subsequently preprocessed, and transformed into informativetime!frequency traces, which very accurately visualize frequency and amplitudemodulations of the original signal.

EEG is usually characterized as a summation of extracellular currents caused bypost-synaptic potentials (intracellular) from a very large number of neurons whichcreate oscillatory patterns distributed and possible to record around the scalp. Thosepatterns in the known frequency ranges can be monitored and classi¯ed in synchronywith a stimuli given to the subjects. EMD utilizes empirical knowledge of oscillationsintrinsic to a time series in order to represent them as a superposition of componentswith well de¯ned instantaneous frequencies. These components are called intrinsicmode functions (IMF).

This paper is organized as follows. First the method of single channel EMDanalysis of EEG signals is presented. Next multichannel EEG analysis anddecomposition is discussed leading to a novel time!frequency synchrony evaluationmethod based on spectral IMF clustering. A new concept of multiple spatiallylocalized amplitude and frequency oscillations related to presented stimuli in time!frequency domain is described which let us obtain ¯nal traces of frequency andamplitude ridges coherent among the EEG channels. Finally examples of the analysisof the EEG signals are given and conclusions are drawn.

2. Methods

Weaimat looking at the level of detail (richness of information source) obtainable fromsingle experimental trial EEG signals, and compare the usefulness of a novel multi-channel signal decomposition approach in this context. The utilized approach is basedon multichannel extension of the EMD technique which was previously successfullyapplied to EEG soni¯cation and mental states estimation in Refs. 1!3. The EMDapproach rests on the identi¯cation of signal's nonstationary and nonlinear featureswhich represent di®erent modalities of brain activity captured by the EEG dataacquisition system (g.USBampr of Guger Technologies). This novel method allowedus previously to create slowly modulated tones representing changing brain activitiesamong di®erent human scalp locations where EEG electrodes were localized.

In the current application, we propose to look at the level of details revealed in singleEEG channels decomposed separately into IMFs as discussed in detail in the nextsection and later compared across the multiple channels. After application of Hilberttransform to all IMFswe can track and visualize revealed oscillations of amplitude andfrequency ridges. Similarity of these oscillations among channels revealed in form ofpairwise correlations identify components which are synchronized or not with theonsets and o®sets of the presented stimuli. The proposed approach is a completely\data driven" concept. All the IMFs create semi-orthogonal bases created from theoriginal EEG signals and not introduced arti¯cially by the method itself.

EMD Approach to Multichannel EEG Data 217

Page 4: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

2.1. EMD for EEG analysis — A single channel case

The IMF components obtained during EMD analysis should approximately obey therequirements of

(i) completeness;(ii) orthogonality;(iii) locality;(iv) adaptiveness.5

To obtain an IMF it is necessary to remove local riding waves and asymmetries,which are estimated from local envelope of minima and maxima of the waveform.There are several approaches to estimate signals envelopes and we have discussedthem previously.3

The Hilbert spectrum for a particular IMF allows us later to represent the EEG inamplitude — instantaneous frequency — time plane. An IMF shall satisfy the twoconditions:

(i) in the whole dataset, the number of extrema and the number of zero crossingsshould be equal or di®er at most by one;

(ii) at any point of IMF the mean value of the envelope de¯ned by the local maximaand the envelope de¯ned by the local minima should be zero.

The technique of ¯nding IMFs corresponds thus to ¯nding limited-band signals. Italso corresponds to eliminating riding waves from the signal, which ensures that theinstantaneous frequency will not have °uctuations caused by an asymmetric wave-form. IMF in each cycle is de¯ned by the zero crossings. Every IMF involves only onemode of oscillation, no complex riding waves are thus allowed. Notice that the IMF isnot limited to be a narrow band signal, as it would be in traditional Fourier orwavelets decomposition, in fact, it can be both amplitude and frequency modulatedat once, and also nonstationary or nonlinear.

The process of IMF extraction from a signal x(t) (\sifting process"5) is based onthe following steps:

(1) determine the local maxima and minima of the analyzed signal x(t);(2) generate the upper and lower signal envelopes by connecting those local maxima

and minima, respectively, by the chosen interpolation method (e.g., linear,spline, cubic spline, piece-wise spline3,5);

(3) determine the local mean m(t), by averaging the upper and lower signalenvelopes;

(4) subtract the local mean from the data:

h1"t# $ x"t# !m1"t# : "1#

218 T. M. Rutkowski et al.

Page 5: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

Ideally, h1"t# is an IMF candidate. However, in practice, h1"t# may still contain localasymmetric °uctuations, e.g., undershoots and overshoots; therefore, one needs torepeat the above four steps several times, resulting eventually in the ¯rst (optimized)IMF. In order to obtain the second IMF, one applies the sifting process to the residue

"1"t# $ x"t# ! IMF1"t# ; "2#

obtained by subtracting the ¯rst IMF from x(t); the third IMF is in turn extractedfrom the residue "2"t# and so on. One stops extracting IMFs when two consecutivesifting results are close to identical; the EMD of the signal x(t) may be written as:

x"t# $Xn

k$1

IMFk"t# % "n"t# ; "3#

where n is the number of extracted IMFs, and the ¯nal residue "n"t# can either be themean trend or a constant. The EMD is obviously complete, since Eq. (3) is anequality: the original signal can be reconstructed by adding all IMFs and the ¯nalresidue. Note that the IMFs are not guaranteed to be mutually orthogonal, but inpractice, they often are close to orthogonal3; it is also noteworthy that the IMFs areadaptive, i.e., they depend on the signal x(t) as anticipated for the data drivenmethod.

2.2. Huang!Hilbert spectra with amplitude and frequency ridges

From the obtained in previous section IMFs corresponding time!frequency rep-resentations can be produced by applying the Hilbert transform to each component.5

As a result of Hilbert transform application to each IMF the data can be expressed astime!frequency domain in form of analytic complex signals formed as

IMFk;an"t# $ IMFk"t# % iIMFk;HT"t# $ IMFk"t# %1

!

Z %1

!1

IMFk"t 0#t! t 0

dt 0 ; "4#

where IMFHT is the Hilbert transformed version of IMF. The analytic signal isfurther polar-decomposed as

IMFk;an"t# $ AIMFk"t# exp"i"IMFk

"t## ; "5#

where

AIMFk"t# $

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!IMFk"t#2 % IMFk;HT"t#2

q"6#

is the instantaneous amplitude, and

"IMFk"t# $ arctan

IMFk;HT "t#IMFk"t#

" #"7#

is instantaneous phase of each IMFk, respectively. The Hilbert transform allows us todepict the variable amplitude (Fig. 1(d)) and the instantaneous frequency (Fig. 1(c))

EMD Approach to Multichannel EEG Data 219

Page 6: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

200 400 600 800 1000

!10

0

10

EEG signal

[am

plitu

de]

200 400 600 800 1000

!10

0

10

[am

plitu

de]

200 400 600 800 1000

!10

0

10

[am

plitu

de]

200 400 600 800 1000

!10

0

10

[samples]

[am

plitu

de]

(a) Time domain preprocessed EEG.

HilbertHuang timefrequency distribution

norm

(fs)

0 260 521 782 1043

0.50.375

0.25

0.125

0.0

norm

(fs)

0 260 521 782 1043

0.50.375

0.25

0.125

0.0

norm

(fs)

0 260 521 782 1043

0.50.375

0.25

0.125

0.0

[samples]

norm

(fs)

0 260 521 782 1043

0.50.375

0.25

0.125

0.0

24681012

24681012

24681012

24681012

(b) Huang!Hilbert spectra.

0 260 521 782 10430.0

0.125

0.25

0.375

0.5

norm

(fs)

Frequency ridges

0 260 521 782 10430.0

0.125

0.25

0.375

0.5

norm

(fs)

0 260 521 782 10430.0

0.125

0.25

0.375

0.5

norm

(fs)

0 260 521 782 10430.0

0.125

0.25

0.375

0.5

[samples]

norm

(fs)

(c) Frequency ridge traces.

0 260 521 782 10430

5

10

15

[am

plitu

de]

Amplitude ridges

0 260 521 782 10430

5

10

15

[am

plitu

de]

0 260 521 782 10430

5

10

15

[am

plitu

de]

0 260 521 782 10430

5

10

15

[samples]

[am

plitu

de]

(d) Amplitude ridge traces

Fig. 1. Four plots of four EEG channels in each panel recorded synchronously during SSVEP experiment.The steady-state response can be visually spotted in the range of 150!650 samples. Panel (a) presents timedomain EEG preprocessed plots; (b) their Huang!Hilbert spectra; (c) and (d) the frequency and ampli-tude ridges in Hilbert spectra domain (solid line: ¯rst; dashed line: second; dotted line: third; dash-dottedline: fourth IMF, respectively).

220 T. M. Rutkowski et al.

Page 7: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

in the form of very sharp and localized functions of frequency and time (in contrast toFourier expansion, for example, where frequencies and amplitudes are ¯xed for itsbases). Such an approach is very suitable for the nonstationary EEG analysis andcommon/synchronized activities within certain channels. An example of Huang!Hilbert spectrograms of four EEG channels recorded simultaneously is presented inFig. 1(b).

2.3. EMD application to multichannel EEG signals

Using the above procedure in a single channel mode the EEG signals from chosenelectrodes could be decomposed separately forming subsets of IMF functions, fromwhich low frequency drifts and high frequency spikes could be further removed. Themost interesting part of EEG is usually in the middle range frequencies. To analyzemultichannel EEG signal sets recorded synchronously in a single experiment wepropose to decompose all channels separately preventing possible oscillatory infor-mation leaking among the channels. The so-obtained IMFs sets can be furthercompared as in case of four EEG signals presented in Fig. 1(a) which are furtherEMD decomposed and visualized in form of Huang!Hilbert spectrograms5 as inFig. 1(b). The traces of amplitude and frequency modulation ridges8 obtained fromHilbert transformation of separately processed IMFs and further plotted together arepresented in Figs. 1(d) and 1(c), respectively. Ridges are the continuous traces withinspectrograms of frequency and amplitude oscillations as ¯rst introduced in Ref. 8.The areas of steady-state stimulation can be easily spotted in amplitude traces of asingle IMF on all channels in Fig. 1(d) and subsequently in form of stable frequencyridge during stimulation with very strong oscillations before and after the stimuli inall channels as in Fig. 1(c).

The combined result of analysis of seven EEG channels from locations around thehuman head during similar SSVEP stimuli BCI/BMI paradigm is shown also inFig. 5. There are seven time domain EEG traces plotted with only two amplitude(AR) and frequency (FR) ridges of the only components showing synchrony with thestimuli. Those components were obtained based on spectral clustering technique andthresholding described in the next section.

2.4. Spectral clustering of EMD components

In order to compare all IMFs extracted from the analyzed channels we propose totransform them to Hilbert domain in order to capture spectral content carried by allof them. The spectra are further treated as feature vectors and compared for theirsimilarity across the channels.

We decided to compare several distance measures in order to test their usabilityfor IMF components clustering resulting in EEG interference separation from thesignals.

EMD Approach to Multichannel EEG Data 221

Page 8: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

The distances were evaluated in form of a hierarchical cluster analysis using a setof dissimilarities for the n objects to be clustered. Such a procedure was performed9

with utilization of \R" package.10 Initially, each vector representing power spectrumvalues is assigned to its own cluster and then the algorithm proceeds iteratively, ateach stage joining the two most similar clusters. The procedure continues until thereis just a single cluster. At each stage distances between clusters are recomputed bythe Lance!Williams dissimilarity update formula with a single linkage clusteringmethod. The single linkage method is closely related to the minimal spanning treeconcept and it adopts a \friends of friends" strategy for clustering.9 Results of suchprocedure are presented in Figs. 2(a), 2(b), 3(a), 3(b), 4(a), and 4(b), where two setsof clusters are visualized. The following distance measures were tested (written herefor two vectors x and y):

2.4.1. Euclidean distance

The Euclidean distance is a usual square distance between the two vectors (two-norm). A result of clustering with this measure (not very good though with most ofthe IMFs clustered together) is presented in Fig. 2(a).

2.4.2. Maximum distance

The maximum distance is calculated as a maximum distance between two com-ponents of x and y (supremum norm). A result of clustering with this measure (herealso most of the IMFs created a single cluster) is presented in Fig. 2(b).

2.4.3. Manhattan distance

The Manhattan distance is based on an absolute distance between the two vectors(one-norm). A result of clustering with this measure (here again most of the IMFscreated a single cluster) is presented in Fig. 3(a).

2.4.4. Canberra distance

Canberra distance is calculated as

dC $X

i

jxi ! yijjxi % yij

; "8#

with terms with zero numerator and denominator omitted from the sum and treatedas if the values were missing. A result of clustering with this measure (here the resultis more interesting but unfortunately again low frequency components were mixedwith those of higher frequencies) is presented in Fig. 3(b).

2.4.5. Minkowski distance

Minkowski distance is calculated as the p-norm, which is the pth root of the sum ofthe pth powers of the di®erences of the components. A result of clustering with this

222 T. M. Rutkowski et al.

Page 9: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

Tp8_imf9F8_imf10

O2_imf3O2_imf1

O2_imf2O2_imf4

O2_imf6O2_imf5

F8_imf8O1_imf9

O2_imf7O2_imf8Tp8_imf7O1_imf7

Tp7_imf1Fpz_imf1Tp8_imf1O1_imf1

F7_imf1F8_imf1F8_imf9O1_imf8

Tp7_imf2Fpz_imf2

F7_imf6O2_imf9F8_imf7

Tp7_imf7Fpz_imf7

F8_imf6O1_imf2F7_imf2F8_imf2

Tp8_imf2F7_imf7F8_imf3

Fpz_imf10F7_imf10

Tp7_imf11Tp8_imf8Tp7_imf9Tp7_imf8F7_imf9

Fpz_imf8F7_imf8F7_imf3O1_imf3F7_imf5

Tp7_imf3Fpz_imf3Tp8_imf3F8_imf5

Tp8_imf5F8_imf4

O1_imf5Tp7_imf5Fpz_imf5

O1_imf6F7_imf4O1_imf4

Tp8_imf4Tp7_imf4Fpz_imf6

Tp7_imf6Tp8_imf6Fpz_imf4

Tp7_imf10Fpz_imf9

0 1000 2000 3000 4000

Result of the cluster analysis in form

of a cluster dendogram

IMFs power spectra with euclidean distance measure

(a) Euclidian

Tp8_imf9F8_imf10

O2_imf3F8_imf8O1_imf9

O2_imf6O2_imf7O2_imf8

O2_imf4O2_imf5

O1_imf7Tp8_imf7O2_imf2

O2_imf1F8_imf9O1_imf8F8_imf7F7_imf6

Tp7_imf7Fpz_imf7

F8_imf6F7_imf7O2_imf9F8_imf3

Tp8_imf1Tp7_imf1Fpz_imf1Tp7_imf2Fpz_imf2O1_imf6

Fpz_imf6O1_imf1F7_imf1F8_imf1O1_imf2F7_imf2F7_imf3O1_imf3O1_imf4F8_imf4

Tp7_imf3Fpz_imf3F8_imf2

Tp8_imf2F7_imf4

Tp7_imf4Fpz_imf4

Tp7_imf10Fpz_imf9Tp8_imf3Tp8_imf4Tp7_imf6Tp8_imf6O1_imf5

Tp7_imf5Fpz_imf5

F7_imf5F8_imf5

Tp8_imf5F7_imf8

Fpz_imf10F7_imf10

Tp7_imf11Tp7_imf9Tp8_imf8Tp7_imf8F7_imf9

Fpz_imf8

0 1000 2000 3000 4000

Result of the cluster analysis in form

of a cluster dendogram

IMFs power spectra with maximum distance measure

(b) Maximum

Fig. 2. IMF spectral clustering results for Euclidian and maximum distances methods.

EMD Approach to Multichannel EEG Data 223

Page 10: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

O2_imf1O2_imf3

O2_imf2Tp8_imf9

Tp7_imf1Fpz_imf1

O1_imf1Tp8_imf1

F7_imf1F8_imf1

O2_imf4O2_imf8

O2_imf5O2_imf6F8_imf10

O1_imf7O2_imf7

Tp8_imf7Tp7_imf2Fpz_imf2F7_imf3O1_imf3O2_imf9

Tp7_imf7Fpz_imf7

F7_imf6F8_imf7

Tp7_imf3Fpz_imf3F8_imf6

F7_imf10Tp7_imf11Fpz_imf10

F7_imf7F7_imf8O1_imf6O1_imf8

Tp8_imf8Fpz_imf8F7_imf9

Tp7_imf8Tp7_imf9

Tp7_imf4F7_imf4F8_imf4O1_imf4F7_imf5

Tp8_imf4Fpz_imf4

F8_imf5Tp8_imf5Fpz_imf6Tp8_imf6Tp7_imf6

Tp7_imf10Fpz_imf9

O1_imf5Tp7_imf5Fpz_imf5

F8_imf3Tp8_imf3

F8_imf9F8_imf8O1_imf9

O1_imf2F7_imf2F8_imf2

Tp8_imf2

0 5000 10000 15000

Result of the cluster analysis in form

of a cluster dendogram

IMFs power spectra with Manhattan distance measure

(a) Manhattan

Fpz_imf10Tp7_imf11

O2_imf1O2_imf2

Tp7_imf2Fpz_imf2

O1_imf2F7_imf2F8_imf2

Tp8_imf2Tp7_imf3O2_imf4F8_imf4

O2_imf3Fpz_imf3

Tp8_imf4Fpz_imf4F7_imf10F8_imf5

O1_imf4Tp7_imf6O1_imf9F7_imf4

Tp7_imf4F8_imf7

F7_imf5O1_imf8

F7_imf6Tp8_imf8O1_imf6

F7_imf8Fpz_imf8

O1_imf5F7_imf7F7_imf9

Tp7_imf5F8_imf9

Tp8_imf6F8_imf6

Fpz_imf9Tp7_imf10

Tp7_imf8Tp7_imf9

Tp8_imf5Fpz_imf5F8_imf10F8_imf8

Tp8_imf9Fpz_imf6

O2_imf8O1_imf7O2_imf7

Tp8_imf7Tp7_imf7O2_imf5O2_imf9O2_imf6

Fpz_imf7F8_imf3

Tp8_imf3F7_imf3O1_imf3Tp7_imf1Fpz_imf1

O1_imf1Tp8_imf1

F7_imf1F8_imf1

0 20 40 60

Result of the cluster analysis in form

of a cluster dendogram

IMFs power spectra with Canberra distance measure

(b) Canberra

Fig. 3. IMF spectral clustering results for Manhattan and Canberra distances methods.

224 T. M. Rutkowski et al.

Page 11: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

Tp8_imf9F8_imf10

O2_imf3O2_imf1

O2_imf2O2_imf4

O2_imf6O2_imf5

F8_imf8O1_imf9

O2_imf7O2_imf8Tp8_imf7O1_imf7

Tp7_imf1Fpz_imf1Tp8_imf1O1_imf1

F7_imf1F8_imf1F8_imf9O1_imf8

Tp7_imf2Fpz_imf2

F7_imf6O2_imf9F8_imf7

Tp7_imf7Fpz_imf7

F8_imf6O1_imf2F7_imf2F8_imf2

Tp8_imf2F7_imf7F8_imf3

Fpz_imf10F7_imf10

Tp7_imf11Tp8_imf8Tp7_imf9Tp7_imf8F7_imf9

Fpz_imf8F7_imf8F7_imf3O1_imf3F7_imf5

Tp7_imf3Fpz_imf3Tp8_imf3F8_imf5

Tp8_imf5F8_imf4

O1_imf5Tp7_imf5Fpz_imf5

O1_imf6F7_imf4O1_imf4

Tp8_imf4Tp7_imf4Fpz_imf6

Tp7_imf6Tp8_imf6Fpz_imf4

Tp7_imf10Fpz_imf9

0 1000 2000 3000 4000

Result of the cluster analysis in form

of a cluster dendogram

IMFs power spectra with Minkowski distance measure

(a) Minkowski

O2_imf5Tp7_imf5O1_imf5

Fpz_imf5F8_imf8F8_imf9

F7_imf10F8_imf10O1_imf9

Fpz_imf10Tp7_imf11Tp8_imf9O1_imf8

Tp8_imf8Tp7_imf9Tp7_imf8F7_imf9

Fpz_imf8Fpz_imf9F7_imf6F8_imf7F7_imf7O2_imf7Tp7_imf7O2_imf9

Tp7_imf10O2_imf8O1_imf7

Fpz_imf7F7_imf8

Tp8_imf7Tp8_imf6Tp7_imf6O2_imf6

O1_imf6F8_imf6

Fpz_imf6O2_imf4

F7_imf5F8_imf5

Tp8_imf5Fpz_imf3

Tp7_imf3F7_imf3O1_imf3

Tp8_imf3F8_imf4

O1_imf4F8_imf3

F7_imf4Tp8_imf4

O2_imf3Tp7_imf4Fpz_imf4

Tp7_imf2Fpz_imf2

Tp8_imf2O1_imf2O2_imf2F7_imf2F8_imf2

Tp8_imf1F7_imf1F8_imf1

O2_imf1O1_imf1

Tp7_imf1Fpz_imf1

0.0 0.2 0.4 0.6 0.8

Result of the cluster analysis in form

of a cluster dendogram

(1-correlation) betw een IMFs power spectra as distance measure

(b) (1-correlation coe±cient)

Fig. 4. IMF spectral clustering results for Minkowski and (one-minus-correlation coe±cient) distancesmethods.

EMD Approach to Multichannel EEG Data 225

Page 12: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

measure (here as the result again the low frequency components were mixed withthose of higher frequencies which) is presented in Fig. 4(a).

2.4.6. (One-minus-correlation) distance

One-minus-correlation distance is based on correlations between vectors x and yevaluated \as a distance measure". This result presented in Fig. 4(b) is the mostinteresting, since it allowed us to separate two sets of clusters.

For the best result of correlation distance measure the ¯rst set is for distancesbelow 0.15 and those components from di®erent channels are classi¯ed as broad-spectrally very similar and originating from nonstimuli related brain sources. Thevisualization of the remaining IMFs is presented in Fig. 5 and it vividly con¯rms thestrength of the proposed method. The remaining set of clusters with distances below0.15 was rejected.

200 400 600-40-20

020

original EEG

EE

G(F

pz)

200 400 6000

5

AR

(Fpz

)

two amplitude ridges

200 400 6000

0.5two frequency ridges

FR

(Fpz

)

200 400 600

-40-20

02040

EE

G(F

7)

200 400 6000

5

AR

(F7)

200 400 6000

0.5

FR

(F7)

200 400 600

-200

20

EE

G(F

8)

200 400 6000

5

AR

(F8)

200 400 6000

0.5

FR

(F8)

200 400 600

-100

10

EE

G(T

P7)

200 400 6000

5

AR

(TP

7)

200 400 6000

0.5

FR

(TP

7)

200 400 600

-20

0

20

EE

G(T

P8)

200 400 6000

5

AR

(TP

8)

200 400 6000

0.5

FR

(TP

8)

200 400 600-5

0

5

EE

G(O

1)

200 400 6000

5

AR

(O1)

200 400 6000

0.5

FR

(O1)

200 400 600

-10

0

10

EE

G(O

2)

[samples]200 400 600

0

5

AR

(O2)

[samples]200 400 600

0

0.5

FR

(O2)

[samples]

Fig. 5. The result showing the power of the proposed method to analyze multichannel EEG recordings.The ¯rst column shows noisy EEG signals, while the second and third columns depict only amplitude (AR)and frequency (FR) traces of components synchronized with the stimuli and subsequently correlatedamong channels (solid line for the ¯rst and dashed line for the second IMF synchronized with the stimuli).

226 T. M. Rutkowski et al.

Page 13: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

!200

0

200Original EEG contaminated by EOG

EE

G1

!200

0

200

EE

G2

!100

0

100

EE

G3

200 400 600 800 1000 1200 1400!100

0

100

EE

G4

(a) Original EEG contaminated by EOG

200 400 600 800 1000 1200 1400

!505

C1

Estimated components with classical InfoMax ICA

200 400 600 800 1000 1200 1400

!505

C2

200 400 600 800 1000 1200 1400

!505

C3

200 400 600 800 1000 1200 1400

!505

C4

(b) Classical ICA

200 400 600 800 1000 1200 1400!5

0

5

C1

Estimated sources with FastICA

200 400 600 800 1000 1200 1400!5

0

5

C2

200 400 600 800 1000 1200 1400!5

0

5

C3

200 400 600 800 1000 1200 1400!5

0

5

C4

(c) FastICA

!5

0

5Estimated sources with SOBI algorithm

C1

!5

0

5

C2

!5

0

5

C3

200 400 600 800 1000 1200 1400!5

0

5

C4

(d) SOBI

200 400 600 800 1000 1200 1400!200

0

200Filtered ocular muscle noise with proposed approach

C1

200 400 600 800 1000 1200 1400!200

0

200

C2

200 400 600 800 1000 1200 1400!100

0

100

C3

200 400 600 800 1000 1200 1400!50

0

50

C4

(e) EMD & clustering: EOG ¯ltered out.

200 400 600 800 1000 1200 1400!50

0

50

EE

G4

200 400 600 800 1000 1200 1400!50

0

50

EE

G3

200 400 600 800 1000 1200 1400!50

0

50

EE

G2

200 400 600 800 1000 1200 1400!50

0

50Filtered EEG with proposed approach

EE

G1

(f ) EMD & clustering: recovered EEG only.

Fig. 6. Comparison of contemporary blind separation algorithms in panels (b)!(d) with the proposedapproach in panels (e) and (f ) in application for ocular muscle interference (EOG) removal from EEG. Theoriginal contaminated EEG signals are presented in panel (a).

EMD Approach to Multichannel EEG Data 227

Page 14: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

3. Conclusions

An automatic framework to separate interfering stimuli related responses withinEEG has been presented. This has been achieved by proposing a novel EEGdecomposition technique, which allows a °exible sub-band signal decompositionwhile preserving the nonlinear and nonstationary features of the signals which is verycrucial for brain activity analysis. The so-obtained components from each EEGchannel processed separately have been further transformed to the Hilbert domainand compared within amplitude and phase domains using the clustering technique inorder to identify those similar (spectrally correlated) across channels.

The resulting reconstruction has allowed us to separate common nonstimulirelated EEG subcomponents from the target (stimuli related) brain activity in thedata-driven signal processing approach without information leakage between chan-nels. The proposed approach was tested in several EEG recording sessions in amultiple subjects con¯rming the results presented here.

For a comparison we included results with contemporary blind source separationalgorithms such as classical ICA11 in Fig. 6(b), FastICA12 in Fig. 6(c), and SOBI13 inFig. 6(d). These contemporary methods were not able to separate strong ocularmuscle interference (EOG) from neurophysiological signals (EEG) (compare originalrecordings in Fig. 6(a)). The proposed approach was able to separate ocular artifacts,without additional scaling problems what was an issue occurring in the other com-pared approaches. The EMD-based technique presented in this paper was able toseparate EOG interference (see Fig. 6(e)) from target pure EEG (see Fig. 6(f))preserving scaling and shape distortion (compare with Fig. 6(a)). The strength of theproposed technique is based on adaptive ¯ltering design in the completely datadriven approach.

This is a step forward in EEG signal processing applications which could be usefulprimarily for creating user friendly BMI that would be °exible, adaptive, andresponse automatic detection focused resulting in fast estimation of user attention tothe stimuli.

Acknowledgments

This work was supported in part by JSPS and Royal Society under the Japan-UKResearch Cooperative Program, 2007 & 2008 and AOARD grant \Brain SignalProcessing for the Study of Sleep Inertia".

References

1. T. M. Rutkowski, F. Vialatte, A. Cichocki, D. Mandic and A. K. Barros, Auditoryfeedback for brain computer interface management — An EEG data soni¯cationapproach, Knowledge-Based Intelligent Information and Engineering Systems, LectureNotes in Arti¯cial Intelligence, Vol. 4253 (Springer, Berlin & Heidelberg, 2006),pp. 1232!1239.

228 T. M. Rutkowski et al.

Page 15: EMD APPROACH TO MULTICHANNEL EEG DATA … · EMD APPROACH TO MULTICHANNEL EEG DATA — THE AMPLITUDE AND PHASE COMPONENTS CLUSTERING ANALYSIS TOMASZ M. RUTKOWSKI Laboratory for Advanced

2. T. M. Rutkowski, A. Cichocki, A. L. Ralescu and D. P. Mandic, Emotional states esti-mation from multichannel EEG maps, Advances in Cognitive Neurodynamics ICCN 2007Proceedings of the International Conference on Cognitive Neurodynamics, eds. R. Wang,F. Gu and E. Shen, Neuroscience (Springer, Berlin & Heidelberg, 2008), pp. 695!698.

3. T. M. Rutkowski, A. Cichocki and D. P. Mandic, EMDsonic — An empirical modedecomposition based brain signal activity soni¯cation approach, Signal ProcessingTechniques for Knowledge Extraction and Information Fusion, Information Technology:Transmission, Processing and Storage (Springer, 2008), pp. 261!274.

4. J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller and T. M. Vaughan,Brain!computer interfaces for communication and control, Clin. Neurophysiol. 113(2002) 767!791.

5. N. Huang, Z. Shen, S. Long, M. Wu, H. Shih, Q. Zheng, N. C. Yen, C. Tung and H. Liu,The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis, Proc. Math. Phys. Eng. Sci. 454 (1998) 903!995.

6. S. P. Kelly, E. C. Lalor, C. Finucane, G. McDarby and R. B. Reilly, Visual spatialattention control in an independent brain!computer interface, IEEE Trans. Biomed.Eng. 52 (2005) 1588!1596.

7. E. Niedermeyer and F. L. Da Silva (eds.), Electroencephalography: Basic Principles,Clinical Applications, and Related Fields, 5th edn. (Lippincott Williams & Wilkins,2004).

8. A. W. Przybyszewski and T. M. Rutkowski, Processing of the incomplete representationof the visual world, Issues in Intelligent Systems Paradigms, eds. O. Hryniewicz,J. Kacprzyk, J. Koronacki and S. Wierzchon, Problemy Wspolczesnej Nauki — Teoria iZastosowania — Informatyka (Akademicka O¯cyna Wydawnicza EXIT, Warsaw,Poland, 2005), pp. 225!235.

9. F. Murtagh, Multidimensional clustering algorithms, COMPSTAT Lect. 4 (1985).10. R Development Core Team, R: A Language and Environment for Statistical Computing.

(R Foundation for Statistical Computing, Vienna, Austria, 2008), http://www.R-project.org.

11. S. Makeig, A. Bell, T.-P. Jung and T. Sejnowski, Independent component analysis ofelectroencephalographic data, Information Processing Systems, Vol. 8, eds. D. Tour-etzky, M. Mozer and M. Hasselmo (MIT Press, Cambridge, MA, 1986), pp. 145!151.

12. A. Hyvärinen, J. Karhunen and E. Oja, Independent Component Analysis (John Wiley &Sons, 2001).

13. A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso and E. Moulines, A blind source sep-aration technique using second-order statistics, IEEE Trans. Signal Process. 45 (1997)434!444.

EMD Approach to Multichannel EEG Data 229