International Journal of Psychophysiology 73 (2009) 53–61

Contents lists available at ScienceDirect

International Journal of Psychophysiology

j ourna l homepage: www.e lsev ie r.com/ locate / i jpsycho

Mining EEG–fMRI using independent component analysis

Tom Eichele a,⁎, Vince D. Calhoun b,c, Stefan Debener d

a Department of Biological and Medical Psychology, University of Bergen, 5009 Bergen, Norwayb The Mind Research Network, Albuquerque, New Mexico, United Statesc Department of ECE, University of New Mexico, Albuquerque, New Mexico, United Statesd Biomagnetic Center, Dept. of Neurology, University Hospital Jena, Erlanger Allee 101, D-07747 Jena, Germany

⁎ Corresponding author. Tel.: +47 45224919; fax: +4E-mail address: tom.eichele@psybp.uib.no (T. Eichel

0167-8760/$ – see front matter © 2009 Elsevier B.V. Aldoi:10.1016/j.ijpsycho.2008.12.018

a b s t r a c t

a r t i c l e i n f o

Article history:

Independent component an Received 8 September 2008Received in revised form 26 November 2008Accepted 23 December 2008Available online 15 February 2009

Keywords:ICAPCAEEGERPfMRISingle trial analysisGroup analysis

alysis (ICA) is a multivariate approach that has become increasingly popular foranalyzing brain imaging data. In contrast to the widely used general linear model (GLM) that requires theuser to parameterize the brain's response to stimuli, ICA allows the researcher to explore the factors thatconstitute the data and alleviates the need for explicit spatial and temporal priors about the responses. In thispaper, we introduce ICA for hemodynamic (fMRI) and electrophysiological (EEG) data processing, and one ofthe possible extensions to the population level that is available for both data types. We then selectivelyreview some work employing ICA for the decomposition of EEG and fMRI data to facilitate the integration ofthe two modalities to provide an overview of what is available and for which purposes ICA has been used. Anoptimized method for symmetric EEG-fMRI decomposition is proposed and the outstanding challenges inmultimodal integration are discussed.

© 2009 Elsevier B.V. All rights reserved.

1. Introduction

Since its inception in 1992 (Frahm et al., 1992; Kwong et al., 1992;Ogawa et al., 1992) functional magnetic resonance imaging hasbecome a major tool to study human brain function (Amaro andBarker, 2006; Bandettini et al., 2000; Huettel et al., 2004). This waspropelled by the non-invasiveness, excellent spatial resolution,flexible experiment design and repeatability. The signal in fMRI isbased on changes in magnetic susceptibility of the blood during brainactivation, it does not directly reflect neuro-electric activity in thebrain. The signal is based upon the complex coupling of neuronalactivity, metabolic activity and blood flow parameters in the brain(Heeger and Ress, 2002; Logothetis and Wandell, 2004; Raichle andMintun, 2006). The hemodynamic response is delayed and smoothedrelative to the neuronal activity, and it is usually not possible toreconstruct the neuro-electric process from the hemodynamicprocess. Nevertheless, the hemodynamic signal remains a veryinformative surrogate for neuronal activity. Moreover, what fMRIalone lacks in temporal resolution and direct relationships withneuro-electric activity could be compensated by the combinationwithconcurrent EEG measurements (Debener et al., 2006; Eichele et al.,2005).

A wide variety of studies have been performed to delineate theneuronal correlates of cognitive functions (for a review see e.g. Cabeza

7 555 89872.e).

l rights reserved.

and Nyberg, 2000), use activity patterns to predict perception andbehavior, identify aberrant regional activation in clinical populations,and many more. Inferences made from fMRI data rely most often onpre-specified temporal models, i.e. the correlation between apredicted time course that models the activation to stimuli/responsesand the measured data. However, in many studies the temporaldynamics of event related or intrinsic regional activations are difficultto predict due to the lack of awell-understood brain-activationmodel.In contrast, independent component analysis (ICA) is an exploratory,data-driven tool that can reveal inter-subject and inter-eventdifferences in the temporal dynamics of the fMRI signal without aprior model. ICA is increasingly utilized as a tool for evaluating thehidden spatio-temporal structure contained within electrophysiolo-gical and hemodynamic brain imaging data. The strength of ICA is itsability to reveal function-relevant dynamics for which a temporalmodel cannot be specified a priori (Calhoun and Adali, 2006; Eicheleet al., 2008b), or is not available, such as in resting state data(Damoiseaux et al., 2006; Fox and Raichle, 2007). Below, we willintroduce an ICA generative model and the extension of ICA to groupdata. We will review how the method has been used in fMRI, EEG andconcurrent EEG-fMRI research, and identify some of the outstandingchallenges for future work.

2. Independent component analysis

ICA is a multivariate statistical method used to uncover hiddensources from multiple data channels (e.g., electrodes, microphones,

54 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

images) such that these sources aremaximally independent. Typically, itassumes a generative model where observations are assumed to belinear mixtures of statistically independent sources. Unlike principalcomponent analysis (PCA), which decorrelates the data, ICA includeshigher-order statistics to achieve independence. An intuitive example ofICA can be given by a scatter-plot of two independent signals s1 and s2.Fig. 1a (left, middle) show the projections for PCA and ICA, respectively,for a linear mixture of s1 and s2 and Fig. 1a shows a plot of the twoindependent signals (s1 and s2) in a scatter-plot. PCAfinds the orthogonalvectors a1,a2, but cannot identify the independent vectors. In contrast,ICA is able to find the independent vectors of the linearly mixed signals(s1,s2), and is thus able to restore the original sources.

A typical ICA model assumes that the recorded signal consists ofstatistically independent and non-Gaussian sources that are linearlymixed. Consider an observed M–dimensional random vector denotedby x=[x1,x2,⋯,xM]T:

x = As ð1Þ

where s=[s1,s2,⋯,sN ]T is an N-dimensional vector whose elements arethe random variables that refer to the independent sources and AM×N

is an unknown mixing matrix. Typically M≥N, so that A is usually offull rank. The goal of ICA is to estimate an unmixing matrix WN×M

such that y given by

y = Wx ð2Þ

is a good approximation to the ‘true’ sources s.Two commonly used ICA algorithms for EEG and fMRI data derived

within these formulations are Infomax (Bell and Sejnowski, 1995) andFastICA (Hyvärinen et al., 2001). Both Infomax and FastICA typicallywork with a fixed nonlinearity or one that is selected from a smallset, e.g., two in the case of extended Infomax (Lee et al., 1999). Thesealgorithms work well for symmetric distributions and are lessaccurate for skewed distributions and for sources close to Gaussian.An intuitive introduction into ICA is provided by Stone (Stone, 2004),and a detailed mathematical description by Hyvärinen (Hyvärinenet al., 2001).

3. ICA of EEG data

The initial application of single-subject temporal ICA to EEG datawas introduced by Makeig using multichannel event related poten-tials (Makeig et al., 1997). Since then, ICA of EEG signals has becomepopular for a wide user community, facilitated through the open-source toolbox EEGLAB (www.sccn.eeglab.edu). The use of ICA formulti-channel EEG recordings has been reviewed (Onton et al., 2006),and a conceptual framework for using ICA for the study of event-related brain dynamics has been proposed by Makeig and colleagues(Makeig et al., 2004b).

Fig. 1. Scatterplot of two independent, mixed signals illustrates the need for higher order stat

The popularity of ICA is in large part due to two key features. First,it is a powerful way to remove artefacts from EEG data (Jung et al.,2000a,b), and second, it helps to disentangle otherwise mixed brainsignals (Makeig et al., 2002). One of the most common artefactstypically identified by ICA is shown in Fig. 2. Eye blinks can often beeasily identified by their characteristic topography and time course.Also very common in EEG recordings are ICs reflecting lateral eyemovements, and muscle activity (EMG). ICA from high-density EEGrecordings typically reveals a number of these components. Beyondartifacts, various examples have been published demonstrating thepotential of ICA for the separation of event-related brain activitypatterns (Debener et al., 2005a,b; Delorme et al., 2007; Makeig et al.,2002; Onton et al., 2005). These studies have combined ICA withsingle-trial EEG analysis, thereby allowing exploration of braindynamics beyond the evoked fraction of the signal that is preservedin the ERP. Typically, the inspection of non-artefact ICs suggests thatsome can be specifically linked to stimulus processing and explaincondition-specific variance, which affords functional inferences aboutevent-related component activity (for a review, see Onton andMakeig, 2006).

4. ICA of fMRI data

Following its first application to fMRI (McKeown et al., 1998), ICAhas been successfully utilized in a number of fMRI studies, especially inthose that have proven challenging to analyze with the standardregression-type approaches (CalhounandAdali, 2006;McKeown et al.,2003). Spatial ICA of fMRI finds systematically non-overlapping,temporally coherent brain regions without constraining the shape ofthe temporal response. Note that ICA can be used to discover eitherspatially or temporally independent components (Stone, 2004). Due tothe data structure, in which the number of observed volume elements(voxels) by far exceeds the number of observed time points, mostapplications to fMRI use the former approach and seek componentsthat are maximally independent in space. The aim of ICA is then tofactor a two dimensional data matrix (voxels-by-timepoints) into aproduct of a set of time courses and a set of independent spatialpatterns. The choice of spatial or temporal independence has beensomewhat controversial, although these are just two possible model-ing assumptions. McKeown et al. (1998) argued that the sparsedistributed nature of the spatial pattern for typical cognitive activationparadigms would work well with spatial ICA (sICA). Furthermore,since the proto-typical image artifacts are also sparse and localized,e.g., vascular pulsation, CSF flow signals in the ventricles, or breathinginduced motion (signal localized to strong tissue contrast neardiscontinuities), the Infomax algorithm with a sparse prior is verywell suited for spatial analysis, and has also been used for temporal ICAas have decorrelation-based algorithms (Calhoun et al., 2001; Petersenet al., 2000). Stone et al. (1999) proposed a method which attempts tomaximize both spatial and temporal independence. An interesting

istics as an alternative to orthogonal projection as a means to faithfully un-mix the data.

Fig. 2. Illustration of artefact removal from EEG data by means of ICA. A Section of selected channels from a multi-channel EEG recording is shown, with ongoing EEG oscillations inthe alpha range evident at occipital electrodes and two eye blinks at fronto-polar channels. B Un-mixing of the EEG data into a set of independent components. Each component canbe described on the basis of a spatial pattern (map) and a time course (activation). C Back-projection of all but components 1 and 2 reveals artefact-corrected EEG data.

55T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

combination of spatial and temporal ICAwas presented by Seifritz et al.(2002). They used an initial sICA to reduce the spatial dimensionalityof the data by locating a region of interest in which they then sub-sequently performed temporal ICA to study in more detail the struc-ture of the response in the human auditory cortex.

5. Group ICA models for EEG and fMRI

Unlike univariate methods such as the general linear model (GLM),ICA does not naturally generalize to a method suitable for drawinginferences about observations from multiple subjects. For example,when using the GLM, the investigator specifies a fixed set ofregressors, and so drawing inferences about group data comesnaturally, since all individuals in the group share the same regressors.In ICA, by contrast, different individuals in the group will havedifferent time courses and component maps. Even if components arevery similar across subjects they require a subsequent clustering effortto be grouped together. Accordingly, it is not immediately clear how todraw inferences about group data by using single-subject ICA.

To overcome this problem, several multi-subject ICA approacheshave been proposed (Beckmann and Smith, 2005; Calhoun et al.,2001; Esposito et al., 2005; Guo and Pagnoni, 2008; Lukic et al., 2002;Schmithorst and Holland, 2004; Svensen et al., 2002). The variousapproaches differ in terms of how the data are preprocessed andorganized prior to the ICA analysis and what types of outputs aregenerated. One possibility is to perform single-subject ICA and then

attempt to combine the output into a group post hoc by using aclustering metric or simple correlation of the components (Calhounet al., 2001; Esposito et al., 2005). This has the advantage of allowingfor unique spatial and temporal features, but has the disadvantagethat the components are not necessarily unmixed in the same way foreach subject.

Other approaches involve ICA computed on condensed group data.The advantage of these approaches is that they perform one single ICAon the group data, in which subject-specific and condition-specificcontributions can be identified. Note that a temporal concatenationapproach allows for unique time courses for each subject, but assumescommon group maps. A spatial concatenation approach on the otherhand allows for unique maps but assumes common time courses. Itappears that temporal concatenation works better for fMRI data (Guoand Pagnoni, 2008; Schmithorst and Holland, 2004). Temporalconcatenation is implemented in MELODIC (http://www.fmrib.ox.ac.uk/fsl/) and in GIFT (http://icatb.sourceforge.net/). GIFT additionallyimplements a back-projection step which returns subject specific mapsand timecourses for further analysis (Calhoun et al., 2001; Eichele et al.,2008a). A detailed comparison of several group ICA approachesincluding temporal concatenation and tensor ICA is provided in recentpapers (Guo and Pagnoni, 2008; Schmithorst and Holland, 2004).

While group spatial ICA ismore prominent in fMRI, it has beenmoretypical in EEG research to employ single-subject temporal ICA andcluster individual component sets along features of interest forinferences about groups of subjects (Debener et al., 2005a,b; Onton

56 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

et al., 2005, 2006). In the caseof grouptemporal ICAonEEG timedomaindata, the spatial concatenation, aggregation anddata reductionwithPCAprecedes component estimation, enabling the identification of compo-nents that contribute to event-related potentials (Eichele et al., 2008a),and is implemented in EEGIFT (http://icatb.sourceforge.net/). A similargroup ICAmodel in combinationwithpartial least squareswasproposedrecently by Kovacevic (Kovacevic and McIntosh, 2007). The drawbackhowever is that processes which are not well time-locked acrosssubjects, such as ongoing EEG rhythms, cannot be satisfyinglyreconstructed in this model. The accuracy of component detection andback-reconstructionwith suchagroup ICAmodel dependson thedegreeof intra- and inter-individual latency jitterof event relatedEEGprocesses(see also Moosmann et al., 2008).

We focus on the group ICA approach implemented in the GIFTsoftware,which is freely available andhas been designed for analysis ofEEG and fMRI data. GIFT uses multiple data reduction steps followingdata concatenation, and enables a statistical comparison of individualmaps and time courses. GIFT essentially estimates a mixing matrixwhich has partitions that are unique to each subject. Once the mixingmatrix is estimated, the component maps for each subject can becomputed by projecting the single subject data onto the inverse of thepartition of the unmixing matrix that corresponds to that subject.Mathematically, if we let Xi=Fi−1Yi be the L×V reduced data matrixfrom subject i, where Yi is the K×V data matrix containing thepreprocessed fMRI or EEG data, Fi−1 is the L×K reducing matrixdetermined by the PCA decomposition, V is the number of fMRI voxelsor EEG timepoints,K is the number of fMRI time points or EEG channelsand L is the size of the dimension following reduction. The reduceddata from all subjects is concatenated into a matrix and reduced usingPCA to N dimensions the number of components to be estimated. TheN×V reduced, concatenated matrix for the M subjects is

X = G−1F−11 Y1v

F−1M YM

24

35: ð3Þ

where G−1 is an N×M reducing matrix (also determined by a PCAdecomposition) and is multiplied on the right by the LM×Vconcatenated data matrix for the M subjects. Following ICA estima-tion, we can write X= AS, where A is the N×N mixing matrix and S isthe N×V component fMRI map or EEG timecourse. Substituting thisexpression for X into Eq. (3) and multiplying both sides by G, we have

GAS =F−11 Y1v

F−1M YM

24

35: ð4Þ

Partitioning the matrix G by subject provides the followingexpression

G1v

GM

24

35AS =

F−11 Y1v

F−1M YM

24

35: ð5Þ

We then write the equation for subject i by working only with theelements in partition i of the above matrices such that

GiASi = F−1i Yi: ð6Þ

Thematrix Si in Eq. (6) contains the single subject maps for subjecti and is calculated from the following equation

Si = GiA� �−1

F−1i Yi: ð7Þ

We now multiply both sides of Eq. (6) by Fi and write

Yi≈FiGi A Si; ð8Þ

which provides the ICA decomposition of the data from subject i,contained in the matrix Yi. The N×V matrix Si contains the N sourcemaps in fMRI and timecourses in the EEG modality and the K×Nmatrix FiGi A is the single subject mixing matrix and contains the fMRIcomponent time course or the EEG component topography for each ofthe N components Fig. 3.

6. Application of ICA to concurrent EEG-fMRI recordings

It has become popular to collect multiple types of imaging andother (e.g. genetic) data from the same participants, often in settingswhere relatively large groups are sampled. Each imaging methodinforms on a limited domain and typically provides both common andunique information about the problem in question. Approaches forcombining or fusing data in brain imaging can be conceptualized ashaving a place on an analytic spectrum with meta-analysis (highlydistilled data) to examine convergent evidence at one end and highlydetailed large-scale computational modeling at the other end (Husainet al., 2002). In between are methods that attempt to perform a directdata fusion (Horwitz and Poeppel, 2002). One promising data fusionapproach is to first process each image type and extract features fromdifferent modalities. These features are then examined for relation-ships among the data types at the group level. This approach allows usto take advantage of the ‘cross’-information among data types andwhen performing multimodal fusion provides a natural link amongdifferent data types (Ardnt, 1996; Savopol and Armenakis, 2002). Herewe focus selectively on ‘integration by prediction’, where typicallysome feature from the EEG (e.g. alpha power, P300 amplitude) isconvolvedwith a canonical hemodynamic response function and usedas a predictor of hemodynamic activity in a GLM. Integration-by-prediction is based on the assumption that the hemodynamicresponse is linearly related to local changes in neuronal activity, inparticular local field potentials (Heeger and Ress, 2002; Lauritzen andGold, 2003; Logothetis et al., 2001). Since large-scale synchronousfield potentials are the basis for the scalp EEG (Nunez, 1995), suchintegration can be achieved by investigating correlations betweenBOLD and scalp EEG. This can be done either continuously over time,as in the study of background rhythms and epileptic discharges in theEEG (for a review see Laufs et al., 2008), or in the context of inducingvariation in some cognitive operation (Debener et al., 2006, 2005;Eichele et al., 2005). ICA has been used in this context on many levels:for artifact reduction during preprocessing, for feature extraction fromthe EEG, decomposition of fMRI, parallel decomposition of EEG andfMRI, and joint ICA of multimodal data.

6.1. EEG artifact reduction

Besides using ICA for feature extraction and making inferences, ithas been successfully employed during pre-processing for denoisingof EEG data acquired in the scanner. Temporal ICA is very useful forartifact reduction in EEG data (Jung et al., 2000a,b), and it is similarlyuseful in pre-processing of simultaneously acquired EEG–fMRI data.Principally, the same artifacts affect the in-scanner EEG recording asoutside the scanner. In addition, in-scanner EEG recordings sufferfrom further artifacts, most prominently the pulse artifact. A numberof authors have used ICA for removing the pulse artifact, given thattemplate-based rejection algorithms (Allen et al., 2000, 1998) do notalways sufficiently control for this type of artifact. ICA has been shownto be helpful in reducing the pulse artifact only at lower field strengths(Debener et al., 2008, 2007; Mantini et al., 2007a,b). Since this artifactseems to violate the spatial stationarity assumption inherent intemporal ICA, other existing tools such as optimal basis sets (OBS)

Fig. 3. Prototypical independent components from fMRI data. The figure shows the activation maps of nine independent components from an event related fMRI study (Eichele et al.,2008b) rendered onto the MNI template at representative transverse slices. The maps are shown in neurological convention (left hemisphere is on the left). Activations are plotted inred, deactivations in blue. To the left of each map, the hemodynamic response functions within the respective ICs as estimated via deconvolution from 1–20 s after stimulus onset, inarbitrary (range-scaled) amplitude units are displayed. The group average from the 13 participants is plotted as a solid line, error bars indicate±1 S.E.M., dotted lines represent allindividual estimates. The empirical HRFs were used to estimate single trial amplitudes in the fMRI data. (For interpretation of the references to colour in this figure legend, the readeris referred to the web version of this article.)

57T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

might be more advantageous (Niazy et al., 2005). It is our experiencethat ICA returns better unmixing results when used subsequent topulse artifact correction with OBS or template-based rejectionalgorithms (Debener et al., 2007).

6.2. EEG feature extraction

Another powerful option is to use ICA to identify and select a brainsignal of interest, such as the power modulation of the posterior alpharhythm (Feige et al., 2005), frontal theta (Scheeringa et al., 2008), orsingle trial variability in the error-related negativity (ERN, Debeneret al., 2005). ICA here provides an unmixed and denoised sourcewaveform, and the assumption is that this improves the correlationbetween EEG and fMRI signals. This approach assumes that unmixeddata allows a better matching of specific EEG timecourses to theirunderlying spatial sources in the fMRI. An intuitive example is activityin the 8–12 Hz range, and a common ICA observation often ignored inthe literature is that this frequency range is occupied by at leastposterior alpha and central mu sources. These rhythms are regionallyand functionally separable, and it is easily conceivable that samplingalpha activity from single or grouped posterior electrodes in which allthese activities mix provides a less clear measure of any of these sub-processes than decomposition and selection of one feature (Feige

et al., 2005). Similar considerations apply to other rhythms, and toevent related activity. For example, the isolation of an ERN-liketopography and time course separates the variability associated withresponse-locked error-related processing from stimulus-locked andbackground activity (Debener et al., 2005).

In other circumstances where it is difficult to define a feature ofinterest it seemsmore appropriate to employ ICA for artifact reductiononly and choose the back-projected,mixed EEG rather than the sourcesfor correlation analysis with the fMRI (Eichele et al., 2005). The resultsfrom both method choices may provide comparable results (Bagshawand Warbrick, 2007). However, one caveat is that temporal indepen-dence in the EEG does not necessarily equate with different regionalgenerators of the activity in the fMRI, such that a variety of scenariosare conceivable inwhich unmixing of the EEG could reduce sensitivityof the EEG–fMRI correlation. Also, although the mixing problem isacknowledged and addressed in the EEG, this is not extended in thesepublications to the treatment of fMRI data, where it exists as well.

6.3. fMRI decomposition

While the work cited above used ICA to decompose EEG data for abetter integration with the fMRI BOLD signal, more recentlyalternative approaches have been used. Here, ICA is applied to address

58 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

the spatio-temporal mixing problem that is inherent in fMRI data aswell, while the EEG signals were not subjected to ICA. For instance,Mantini et al. (2009, 2007b) decomposed resting state and eventrelated fMRI data with fastICA and subsequent component clusteringacross subjects and extracted distributed regional networks. In theresting state data (Mantini et al., 2007b), the BOLD signals of thesenetworks were correlated with power fluctuations in the delta, theta,alpha, beta, and gamma bands from concurrently recorded EEG. Eachnetwork was characterized by a signature that involved variablecontributions from the different bands. The interesting idea of thiswork is the perspective that wide-band EEG activity principallydescribes large-scale brain networks more comprehensively thannarrow-band activity. In the event related data (Mantini et al., 2009),the amplitude fluctuation of the P300 revealed correlations withattentional networks. The approach illustrates a means of how tomeaningfully reduce the multiple testing problems in fMRI withoutdefining regions of interest a priori. However, again, only onemodalitywas unmixed. In this case the EEG remained mixed.

6.4. Parallel EEG-fMRI decomposition

The two approaches described above each solve a part of themixing problem and make way for refined spatio-temporal mapping.However, the choice to unmix only one modality but not the other issomewhat inconsistent with the reasoning that leads researchers touse ICA in the first place. We now lay out how parallel unmixing couldbe used for EEG-fMRI integration.

The motivation for implementing parallel EEG-fMRI decomposi-tion is based on the data structure and the presumed physiology ofsignal generation in EEG and fMRI. First, consider that a concurrentexperiment generates a multidimensional dataset consisting of about104 to 105 volume elements by 102 to 103 timepoints sampled in thefMRI, by 102 to 103 timepoints by 102 to 103 trials by 32–128 scalpchannels sampled in the EEG, by a number of participants thatconstitute a group study, providing a rich and complex source ofinformation. The utility of ICA lies in the visualization and exploratoryassessment of such data. Second, we can assume that processing ofsimple stimuli and tasks produces widespread event-relatedresponses that are spatially and temporally mixed across the brain(Baudena et al., 1995; Halgren et al., 1995a,b; Halgren and Marinkovic,1995; Makeig et al., 2004a; Onton et al., 2006). The scalp EEG samplesa spatially degraded version of brain activity, where the potential is amixture of independent timecourses from large-scale synchronousfield potentials. Similarly, the neurovascular transformation of theelectrophysiological activity into hemodynamics provides temporallydegraded and spatially mixed signals across the fMRI volume(Calhoun and Adali, 2006; McKeown et al., 2003).

The spatial and temporal mixing in both modalities createssituations in which prediction of fMRI activity by EEG features in amass–univariate voxel-by-voxel framework may be difficult sinceneither the predictor, nor the response variables are any likely torepresent single and coupled sources of variability. Thus, oneimprovement to achieve a more symmetric treatment of the data isto unmix both modalities in parallel. We have developed such ananalysis framework for multisubject data that employs Infomax ICA(Bell and Sejnowski, 1995) to recover a set of statistically independentmaps from the fMRI, and independent time-courses from the EEGseparately (Eichele et al., 2008a). We utilized the approach that isdescribed above, i.e. create aggregate data containing observationsfrom all subjects, estimate a single set of ICs and then back-reconstructthese in the individual data (Calhoun et al., 2001; Schmithorst andHolland, 2004). The analysis is done on the single trial level ratherthan averaged data (cf. Calhoun et al., 2006), and it does not assume ajoint mixing matrix for both modalities (cf. Calhoun et al., 2006; cf.Moosmann et al., 2008). The components are matched acrossmodalities by correlating their trial-to-trial modulation, where we

can use the convolved trial-by-trial modulation in EEG components topredict the fMRI, or (vice versa) employ deconvolution and single trialestimation on the fMRI component timecourses, and use these aspredictors for the trial-by-trial amplitude modulation in the EEG.Parallel group ICA provides a means to disentangle and visualize largescale networks both in their spatial and temporal form, given coherentneuronal sources jointly express scalp electrophysiologic and hemo-dynamic features (Calhoun and Adali, 2006; Debener et al., 2006;Makeig et al., 2004a; McKeown et al., 2003; Onton et al., 2006).

6.5. Joint ICA for multimodal data

Joint ICA (jICA) is another approach, which enables the jointdecomposition of multi-modal data that have been collected from thesame sample of subjects. Here, we primarily consider a set of extractedfeatures from each subject's data, and these data form the multipleobservations. Given two (or more) sets of group data, XF and XG weconcatenate the two datasets side-by-side to form XJ and write thelikelihood as

L Wð Þ =YNn=1

YVv=1

pJ;n uJ;v

� �; ð9Þ

where uJ=WxJ. Here, we use the notation in terms of randomvariablessuch that each entry in the vectors uJ and xJ correspond to a randomvariable, which is replaced by the observation for each samplen=1,…,Nas rows of matrices UJ and XJ. When posed as a maximum likelihoodproblem,weestimate a jointunmixingmatrixW such that the likelihoodL(W) is maximized.

Let the two datasets XF and XG have dimensionality N×V1 andN×V2, then we have

L Wð Þ =YNn=1

YV1

v=1

pF;n uF;v

� � YV2

v=1

pG;n uG;v

� � !; ð10Þ

Depending on the data types in question, the above formula can bemade more or less flexible. This formulation characterizes the basicjICA approach and assumes that the sources associated with the twodata types modulate the same way across N subjects. Note that theassumption of the same linear covariation for both modalities is fairlystrong. However, it has the advantage of providing a parsimoniousway to link multiple data types and has been demonstrated in avariety of cases with meaningful results, for example for fusing ERPswith fMRI contrast images (Calhoun, Adali et al., 2006). The extensionof joint ICA for concurrent single trial data is described in a simulationstudy by Moosmann and colleagues (Moosmann et al., 2008).Although robust results can be acquired with this methodology,broader validity of the application to real single trial data wouldrequire an iterative procedure with estimation and removal of thehemodynamic transfer function prior to fusion (see below).

7. Challenges

The methodological and conceptual development in the field ofmultimodal integration is ongoing, and ICA plays a prominent role inthis effort. With respect to EEG-fMRI integrationwewill highlight twoissues that are currently being addressed. First, we discuss the utilityof hemodynamic deconvolution and single trial estimation in thefMRI, and second, the need formore flexiblemodeling of the EEG-fMRIcoupling.

7.1. Deconvolution

An outstanding problem that was not addressed in the workdescribed above is the assumptions made about the hemodynamic

59T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

response function (HRF). Most previous concurrent EEG-fMRI studiesassumed a generic transfer HRF to the transient neuronal responseevoked by discrete sensory stimulation (e.g. derived from Boynton et al.,1996) tomodel thehemodynamic activation,whereby the parameters ofthe HRF such as shape and latency of the peak and undershoot areimplicitly assumed to befixed across the entire brain and across subjects.However, quite some variability of the HRF exists across regions andsubjects (Aguirre et al., 1998; Glover, 1999; Handwerker et al., 2004).Inclusion of temporal anddispersion derivative terms of theHRF into themodel can alleviate suchdifferences on thefirst level. However, it ignoresthe potential for amplitude bias induced by model mismatches due tovariable hemodynamic delays and is typically not helpful for 2nd levelrandom effects analyses (Calhoun et al., 2004). Therefore, in order toachieve a more sensitive and less biased analysis it is more desirable toestimate subject- and region-specificHRFs. Another consideration is thatthe eliciting conditions that generate a response which conforms to thecanonical HRF are not necessarily valid for intrinsic/resting state activityand also more complex cognitive operations. An illustrative example ofestimating theHRF fromEEG-data has beenpublished by deMunck et al.(2008, 2007). Here, HRFs were directly estimated from the alpha powermodulation in concurrent EEG–fMRI data, and indeed show systematiclatency and shape differences from the canonical HRF. The regionalactivation with significant HRFs was much more widespread thanwhat

Fig. 4. EEG–fMRI integrationwith deconvolution. The spatial ICA of the fMRI data results in indthe IC timecourses is used for prediction of EEG activity. In order to recover the amplitude mtiming and an assumed HRF length is multiplied with the IC timecourse, yielding individually aonset, yielding a design matrix (X) with predictors for each trial 1..n. The regression of the desiβn). In the EEG, a group temporal ICA provides independent source timecourses, whose trial-b

was previously reported, which can be taken as an indication that thesparser activation patterns in earlier studies (Goldman et al., 2002;Moosmann et al., 2003) could have resulted from poor sensitivity.

Apart from estimating the HRF from the EEG, event related HRFscan be estimated from independent component timecourses throughdeconvolution (Eichele et al., 2008b). While typically noise-tolerantmodels are needed for voxel-by-voxel deconvolution, ICA forms largeregions of interest with quasi-denoised time courses such thatdeconvolution can be done simply by forming a convolution matrixof the stimulus onsets with an assumed duration of the HRF. FMRIsingle trial response amplitudes can then be recovered by fitting adesignmatrix (X) containing separate predictors for the onset times ofeach trial convolved with the estimated HRF onto the IC timecourse,estimating the scaling coefficients (β) in themultiple linear regressionmodel y=β·X+ε using least squares. In this approach, the HRF isestimated and then effectively removed from the hemodynamic data,leaving just the amplitude modulation across trials, which has thesame resolution as the EEG trial-by-trial dynamics (see Fig. 4).

7.2. Integration

There are a variety of ways how one can conceive the couplingbetween electrophysiology and hemodynamic signals. Linear regression

ividual maps and timecourses. The single trial HRF amplitude modulation estimated fromodulation (AM), the pseudoinverse of a convolution matrix generated from the stimulusnd regionally specific HRFs. These HRFs are then convolved separately with each stimulusgn onto the IC timecourse (y) yields the single trial amplitude modulation for this IC (β1..y-trial modulation are extracted and correlated with the fMRI activity.

60 T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

between fMRI and EEG is usually used in combination with ICA toinvestigate links between modalities (e.g., Debener et al., 2006). Insearching for such one-to-one mappings it is assumed that a particularEEG feature is related to a particular fMRI network. Although this isjustifiable since both the fMRI maps and EEG topographies are differentfrom each other and stationary, it appears oversimplified since inprinciple several fMRI networks can affect several EEG signals. Oneexplanation might be that one component is a generator, i.e. it has anopen field configuration, is reasonably large and close to the cortexsurface to generate a scalp potential, while other componentstransiently and remotely modulate this source, while being electro-physiologically silent on the scalp, since these otherwise should inducetopographic variability in the EEG. This explanation already implies thatthe fMRI networks are coupled with each other, which is reflected intheir functional connectivity (Eichele et al., 2008b; Jafri et al., 2008), andeffective connectivity (Stevens et al., 2007). Another related explanationis that the fMRI networks might be considered spatially independentnodes in regionally distributed, functionally coherent source networks,such that many nodes can contribute directly or indirectly to the scalppotentials in a given paradigm/task. Here, the EEG components,although temporally independent on the scalp, would not represent asingle source but still a weighted average of multiple spatiallyindependent regional sources. Both these explanations are physiologi-cally plausible (Baudena et al.,1995; Halgren et al.,1995a,b; Halgren andMarinkovic,1995), and although it is typically not the aimof integration-by-prediction to directly solve the inverse problem, it is helpful toconsider in which way the topographies and timecourses relate to eachother.

Ifwe assume thatmany fMRI networks affect a particular EEG featureone way of addressing this is by employing multiple regression with adesign that contains all fMRI trial-by-trial modulations for prediction ofEEG activity. The problem of collinearity between regressors can beaddressed by orthogonalization (Andrade et al., 1999), stepwiseselection, or a decomposition of the fMRI component timecourses intoset of unrelated factors by means of e.g. PCA or ICA (similar to theprocedure in Seifritz et al., 2002). However, such a treatment alsomeansthat the interpretation of the results becomes less straightforward.Other options include canonical correlation analysis (CCA) in order totreat the problem. Here, however, 2nd level inference is less welldefined. Also, a joint (temporal) ICA of the sIC and tIC trial-by-trialmodulations appears feasible (Calhoun et al., 2006; Moosmann et al.,2008), but has as yet not been explored in detail. A conceptually moreadvanced approach that, however, needs detailed specifications wouldbe to adapt dynamic causal models that explain the EEG/ERP responsesas changes in the effective connectivity between independent sources(Friston, 2005; Friston et al., 2003; Garrido et al., 2007).

8. Conclusion

ICA is a powerful data driven approach that has been successfullyused to analyze EEG, fMRI, and simultaneous EEG-fMRI data. It hasbeen shown repeatedly that the integrations of bothmodalities can beachieved on the (statistical) source level, as provided by ICA. Theoverview provided here demonstrates the utility and diversity of thevarious existing ICA-based approaches for the analysis of brainimaging data. Two key challenges in the field, the estimation of theHRF and a more flexible way of data integration, were identified.Multimodal integration by means of ICA is still under development.However, we expect ICA to contribute to more refined ways ofachieving comprehensive joint mapping of electrophysiology andhemodynamic signals.

Acknowledgment

This work was supported by a grant from the L. Meltzer UniversityFund (801616) to TE, and by the National Institutes of Health, under

Grants 1 R01 EB 000840, 1 R01 EB 005846, and 1 R01 EB 006841 toVDC.

References

Aguirre, G.K., Zarahn, E., D'Esposito, M., 1998. The variability of human, BOLDhemodynamic responses. Neuroimage 8, 360–369.

Allen, P.J., Polizzi, G., Krakow, K., Fish, D.R., Lemieux, L., 1998. Identification of EEG eventsin the MR scanner: the problem of pulse artifact and a method for its subtraction.Neuroimage 8, 229–239.

Allen, P.J., Josephs, O., Turner, R., 2000. A method for removing imaging artifact fromcontinuous EEG recorded during functional MRI. Neuroimage 12, 230–239.

Amaro Jr., E., Barker, G.J., 2006. Study design in fMRI: basic principles. Brain Cogn. 60,220–232.

Andrade, A., Paradis, A.L., Rouquette, S., Poline, J.B., 1999. Ambiguous results infunctional neuroimaging data analysis due to covariate correlation. Neuroimage 10,483–486.

Ardnt, C., 1996. Information Gained by Data Fusion. SPIE Proc., vol. 2784.Bagshaw, A.P., Warbrick, T., 2007. Single trial variability of EEG and fMRI responses to

visual stimuli. Neuroimage 38, 280–292.Bandettini, P.A., Birn, R.M., Donahue, K.M., 2000. Functional MRI. Background,

methodology, limits and implementation. In: Cacioppo, J.T., Tassinary, L.G.,Berntson, G.G. (Eds.), Handbook of Psychophysiology. Cambridge UniversityPress, Cambridge, UK.

Baudena, P., Halgren, E., Heit, G., Clarke, J.M., 1995. Intracerebral potentials to rare targetand distractor auditory and visual stimuli. III. Frontal cortex. Electroencephalogr.Clin. Neurophysiol. 94, 251–264.

Beckmann, C.F., Smith, S.M., 2005. Tensorial extensions of independent componentanalysis for multisubject FMRI analysis. Neuroimage 25, 294–311.

Bell, A.J., Sejnowski, T.J., 1995. An information-maximization approach to blindseparation and blind deconvolution. Neural Comput. 7, 1129–1159.

Boynton, G.M., Engel, S.A., Glover, G.H., Heeger, D.J., 1996. Linear systems analysis offunctional magnetic resonance imaging in human V1. J. Neurosci. 16, 4207–4221.

Cabeza, R., Nyberg, L., 2000. Imaging cognition II: An empirical review of 275 PET andfMRI studies. J. Cogn. Neurosci. 12, 1–47.

Calhoun, V., Adali, T., 2006. Unmixing fMRI with independent component analysis. IEEEEng. Med. Biol. Magazine 25, 79–90.

Calhoun, V.D., Adali, T., McGinty, V., Pekar, J.J., Watson, T., Pearlson, G.D., 2001. fMRIactivation in a visual-perception task: network of areas detected using the generallinear model and independent component analysis. NeuroImage 14, 1080–1088.

Calhoun, V.D., Adali, T., Pearlson, G.D., Kiehl, K.A., 2006. Neuronal chronometry of targetdetection: fusion of hemodynamic and event-related potential data. Neuroimage30, 544–553.

Calhoun, V.D., Adali, T., Pearlson, G.D., Pekar, J.J., 2001. A method for making groupinferences from functional MRI data using independent component analysis. Hum.Brain Mapp. 14, 140–151.

Calhoun, V.D., Stevens, M.C., Pearlson, G.D., Kiehl, K.A., 2004. fMRI analysis with thegeneral linear model: removal of latency-induced amplitude bias by incorporationof hemodynamic derivative terms. Neuroimage 22, 252–257.

Damoiseaux, J.S., Rombouts, S.A., Barkhof, F., Scheltens, P., Stam, C.J., Smith, S.M.,Beckmann, C.F., 2006. Consistent resting-state networks across healthy subjects.Proc. Natl. Acad. Sci. U. S. A. 103, 13848–13853.

de Munck, J.C., Goncalves, S.I., Huijboom, L., Kuijer, J.P., Pouwels, P.J., Heethaar, R.M.,Lopes da Silva, F.H., 2007. The hemodynamic response of the alpha rhythm: an EEG/fMRI study. Neuroimage 35, 1142–1151.

de Munck, J.C., Goncalves, S.I., Faes, T.J., Kuijer, J.P., Pouwels, P.J., Heethaar, R.M., Lopes daSilva, F.H., 2008. A study of the brain's resting state based on alpha band power,heart rate and fMRI. Neuroimage 42 (1), 112–121.

Debener, S.,Makeig, S., Delorme, A., Engel, A.K., 2005a.What is novel in the novelty oddballparadigm? Functional significance of the novelty P3 event-related potential asrevealed by independent component analysis. Brain Res. Cogn. Brain Res. 22, 309–321.

Debener, S., Ullsperger,M., Siegel,M., Fiehler, K., von Cramon, D.Y., Engel, A.K., 2005b. Trial-by-trial coupling of concurrent electroencephalogram and functional magneticresonance imaging identifies the dynamics of performance monitoring. J. Neurosci.25, 11730–11737.

Debener, S., Ullsperger, M., Siegel, M., Engel, A.K., 2006. Single-trial EEG–fMRI revealsthe dynamics of cognitive function. Trends Cogn. Sci. 10, 558–563.

Debener, S., Strobel, A., Sorger, B., Peters, J., Kranczioch, C., Engel, A.K., Goebel, R., 2007.Improved quality of auditory event-related potentials recorded simultaneouslywith 3-T fMRI: removal of the ballistocardiogram artefact. Neuroimage 34, 587–597.

Debener, S., Mullinger, K.J., Niazy, R.K., Bowtell, R.W., 2008. Properties of theballistocardiogram artefact as revealed by EEG recordings at 1.5, 3 and 7 T staticmagnetic field strength. Int. J. Psychophysiol. 67, 189–199.

Delorme, A.,Westerfield,M.,Makeig, S., 2007.Medial prefrontal theta bursts precede rapidmotor responses during visual selective attention. J. Neurosci. 27, 11949–11959.

Eichele, T., Calhoun, V.D., Moosmann, M., Specht, K., Jongsma, M.L., Quiroga, R.Q.,Nordby, H., Hugdahl, K., 2008a. Unmixing concurrent EEG–fMRI with parallelindependent component analysis. Int. J. Psychophysiol. 67, 222–234.

Eichele, T., Debener, S., Calhoun, V.D., Specht, K., Engel, A.K., Hugdahl, K., von Cramon, D.Y.,Ullsperger, M., 2008b. Prediction of human errors by maladaptive changes in event-related brain networks. Proc. Natl. Acad. Sci. U. S. A. 105, 6173–6178.

Eichele, T., Specht, K., Moosmann,M., Jongsma,M.L., Quiroga, R.Q., Nordby, H., Hugdahl, K.,2005. Assessing the spatiotemporal evolution of neuronal activation with single-trialevent-related potentials and functional MRI. Proc. Natl. Acad. Sci. U. S. A. 102,17798–17803.

61T. Eichele et al. / International Journal of Psychophysiology 73 (2009) 53–61

Esposito, F., Scarabino, T., Hyvarinen, A., Himberg, J., Formisano, E., Comani, S., Tedeschi,G., Goebel, R., Seifritz, E., Di Salle, F., 2005. Independent component analysis of fMRIgroup studies by self-organizing clustering. Neuroimage 25, 193–205.

Feige, B., Scheffler, K., Esposito, F., Di Salle, F., Hennig, J., Seifritz, E., 2005. Corticaland subcortical correlates of electroencephalographic alpha rhythm modulation.J. Neurophysiol. 93, 2864–2872.

Fox, M.D., Raichle, M.E., 2007. Spontaneous fluctuations in brain activity observed withfunctional magnetic resonance imaging. Nat. Rev. Neurosci. 8, 700–711.

Frahm, J., Bruhn, H., Merboldt, K.D., Hanicke, W., 1992. Dynamic MR imaging of humanbrain oxygenation during rest and photic stimulation. J. Magn. Reson. Imaging 2,501–505.

Friston, K.J., 2005. A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci.360, 815–836.

Friston, K.J., Harrison, L., Penny, W., 2003. Dynamic causal modelling. Neuroimage 19,1273–1302.

Garrido, M.I., Kilner, J.M., Kiebel, S.J., Friston, K.J., 2007. Evoked brain responses aregenerated by feedback loops. Proc. Natl. Acad. Sci. U. S. A. 104, 20961–20966.

Glover, G.H., 1999. Deconvolution of impulse response in event-related BOLD fMRI.Neuroimage 9, 416–429.

Goldman, R.I., Stern, J.M., Engel Jr., J., Cohen, M.S., 2002. Simultaneous EEG and fMRI ofthe alpha rhythm. Neuroreport 13, 2487–2492.

Guo, Y., Pagnoni, G., 2008. A unified framework for group independent componentanalysis for multi-subject fMRI data. Neuroimage 42, 1078–1093.

Halgren, E., Marinkovic, K., 1995. General principles for the physiology of cognition assuggested by intracranial ERPs. In: Ogura, C., Koga, Y., Shimokochi, M. (Eds.),Recent Advances in Event-Related Brain Potential Research. Elsevier, Amsterdam,pp. 1072–1084.

Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Liegeois, C., Chauvel, P., Musolino, A.,1995a.Intracerebral potentials to rare target and distractor auditory and visual stimuli. I.Superior temporal plane and parietal lobe. Electroencephalogr. Clin. Neurophysiol.94, 191–220.

Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., Devaux, B., Vignal, J.P.,Biraben, A., 1995b. Intracerebral potentials to rare target and distractor auditory andvisual stimuli. II. Medial, lateral and posterior temporal lobe. Electroencephalogr.Clin. Neurophysiol. 94, 229–250.

Handwerker, D.A., Ollinger, J.M., D'Esposito, M., 2004. Variation of BOLD hemodynamicresponses across subjects and brain regions and their effects on statistical analyses.Neuroimage 21, 1639–1651.

Heeger, D.J., Ress, D., 2002. What does fMRI tell us about neuronal activity? Nat. Rev.Neurosci. 3, 142–151.

Horwitz, B., Poeppel, D., 2002. How can EEG/MEG and fMRI/PET data be combined?Hum. Brain Mapp. 17, 1–3.

Huettel, S.A., Song, A.W., McCarthy, G., 2004. Functional Magnetic Resonance Imaging.Sinauer, Sunderland, MA.

Husain, F.T., Nandipati, G., Braun, A.R., Cohen, L.G., Tagamets, M.A., Horwitz, B., 2002.Simulating transcranial magnetic stimulation during PET with a large-scale neuralnetwork model of the prefrontal cortex and the visual system. NeuroImage 15,58–73.

Hyvärinen, A., Karhunen, J., Oja, E., 2001. Independent Component Analysis. John Wiley& Sons, New York.

Jafri, M.J., Pearlson, G.D., Stevens, M., Calhoun, V.D., 2008. A method for functionalnetwork connectivity among spatially independent resting-state components inschizophrenia. Neuroimage 39, 1666–1681.

Jung, T.P., Makeig, S., Humphries, C., Lee, T.W., McKeown, M.J., Iragui, V., Sejnowski, T.J.,2000a. Removing electroencephalographic artifacts by blind source separation.Psychophysiology 37, 163–178.

Jung, T.P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., Sejnowski, T.J.,2000b. Removal of eye activity artifacts from visual event-related potentials innormal and clinical subjects. Clin. Neurophysiol. 111, 1745–1758.

Kovacevic, N., McIntosh, A.R., 2007. Groupwise independent component decompositionof EEG data and partial least square analysis. Neuroimage 35, 1103–1112.

Kwong, K.K., Belliveau, J.W., Chesler, D.A., Goldberg, I.E., Weisskoff, R.M., Poncelet, B.P.,Kennedy, D.N., Hoppel, B.E., Cohen, M.S., Turner, R., et al., 1992. Dynamic magneticresonance imaging of human brain activity during primary sensory stimulation.Proc. Natl. Acad. Sci. U. S. A. 89, 5675–5679.

Laufs, H., Daunizeau, J., Carmichael, D.W., Kleinschmidt, A., 2008. Recent advances inrecording electrophysiological data simultaneously with magnetic resonanceimaging. Neuroimage 40, 515–528.

Lauritzen, M., Gold, L., 2003. Brain function and neurophysiological correlates of signalsused in functional neuroimaging. J. Neurosci. 23, 3972–3980.

Lee, T., Girolami, M., Sejnowski, T., 1999. Independent component analysis using anextended infomax algorithm for mixed subgaussian and supergaussian sources.Neural Comput. 11, 417–441.

Logothetis, N.K., Wandell, B.A., 2004. Interpreting the BOLD signal. Annu. Rev. Physiol.66, 735–769.

Logothetis, N.K., Pauls, J., Augath, M., Trinath, T., Oeltermann, A., 2001. Neurophysio-logical investigation of the basis of the fMRI signal. Nature 412, 150–157.

Lukic, A.S., Wernick, M.N., Hansen, L.K., Strother, S.C., 2002. An ICA algorithm foranalyzing multiple data sets. Int.Conf.on Image Processing (ICIP).

Makeig, S., Jung, T.P., Bell, A.J., Ghahremani, D., Sejnowski, T.J., 1997. Blind separation ofauditory event-related brain responses into independent components. Proc. Natl.Acad. Sci. U. S. A. 94, 10979–10984.

Makeig, S., Westerfield, M., Jung, T.P., Enghoff, S., Townsend, J., Courchesne, E.,Sejnowski, T.J., 2002. Dynamic brain sources of visual evoked responses. Science295, 690–694.

Makeig, S., Debener, S., Onton, J., Delorme, A., 2004a. Mining event-related braindynamics. Trends Cogn. Sci. 8, 204–210.

Makeig, S., Debener, S., Onton, J., Delorme, A., 2004b. Mining event-related braindynamics. Trends Cogn. Sci. 8, 204–210.

Mantini, D., Perrucci, M.G., Cugini, S., Ferretti, A., Romani, G.L., Del Gratta, C., 2007a.Complete artifact removal for EEG recorded during continuous fMRI usingindependent component analysis. Neuroimage 34, 598–607.

Mantini, D., Perrucci, M.G., Del Gratta, C., Romani, G.L., Corbetta, M., 2007b.Electrophysiological signatures of resting state networks in the human brain.Proc. Natl. Acad. Sci. U. S. A. 104, 13170–13175.

Mantini, D., Corbetta, M., Perrucci, M.G., Romani, G.L., Del Gratta, C., 2009. Large-scalebrain networks account for sustained and transient activity during target detection.Neuroimage 44, 265–274.

McKeown, M.J., Makeig, S., Brown, G.G., Jung, T.P., Kindermann, S.S., Bell, A.J., Sejnowski,T.J., 1998. Analysis of fMRI data by blind separation into independent spatialcomponents. Hum. Brain Mapp. 6, 160–188.

McKeown, M.J., Hansen, L.K., Sejnowski, T.J., 2003. Independent component analysis offunctionalMRI:what is signal andwhat is noise? Curr. Opin. Neurobiol.13, 620–629.

Moosmann, M., Ritter, P., Krastel, I., Brink, A., Thees, S., Blankenburg, F., Taskin, B., Obrig,H., Villringer, A., 2003. Correlates of alpha rhythm in functional magnetic resonanceimaging and near infrared spectroscopy. Neuroimage 20, 145–158.

Moosmann, M., Eichele, T., Nordby, H., Hugdahl, K., Calhoun, V.D., 2008. Jointindependent component analysis for simultaneous EEG–fMRI: principle andsimulation. Int. J. Psychophysiol. 67, 212–221.

Niazy, R.K., Beckmann, C.F., Iannetti, G.D., Brady, J.M., Smith, S.M., 2005. Removal ofFMRI environment artifacts from EEG data using optimal basis sets. Neuroimage 28,720–737.

Nunez, P.L., 1995. Neocortical Dynamics and Human EEG Rhythms. Oxford UniversityPress, New York.

Ogawa, S., Tank, D.W., Menon, R.S., Ellermann, J.M., Kim, S.G., Merkle, H., Ugurbil, K.,1992. Intrinsic signal changes accompanying sensory stimulation: functional brainmapping with magnetic resonance imaging. Proc. Natl. Acad. Sci. U. S. A. 89,5951–5955.

Onton, J., Makeig, S., 2006. In: Neuper, C., Pfurtscheller, G. (Eds.), Information-basedModeling Of Event-Related Brain Dynamics. Progress in Brain Research, vol. 159.Elsevier, Amsterdam, pp. 99–120.

Onton, J., Delorme, A., Makeig, S., 2005. Frontal midline EEG dynamics during workingmemory. Neuroimage 27, 341–356.

Onton, J., Westerfield, M., Townsend, J., Makeig, S., 2006. Imaging human EEG dynamicsusing independent component analysis. Neurosci. Biobehav. Rev. 30, 808–822.

Petersen, K., Hansen, L., Kolenda, T., Rostrup, E., Strother, S., 2000. On the independentcomponents of functional neuroimages. Int.Conf.on ICA and BSS, pp. 615–620.

Raichle, M.E., Mintun, M.A., 2006. Brain work and brain imaging. Annu. Rev. Neurosci.29, 449–476.

Savopol, F., Armenakis, C., 2002. Mergine of heterogeneous data for emergencymapping: data integration or data fusion? Proc.ISPRS.

Scheeringa, R., Bastiaansen, M.C., Petersson, K.M., Oostenveld, R., Norris, D.G., Hagoort,P., 2008. Frontal theta EEG activity correlates negatively with the default modenetwork in resting state. Int. J. Psychophysiol. 67, 242–251.

Schmithorst, V.J., Holland, S.K., 2004. Comparison of three methods for generatinggroup statistical inferences from independent component analysis of functionalmagnetic resonance imaging data. J. Magn. Reson. Imaging 19, 365–368.

Seifritz, E., Esposito, F., Hennel, F., Mustovic, H., Neuhoff, J.G., Bilecen, D., Tedeschi, G.,Scheffler, K., Di Salle, F., 2002. Spatiotemporal pattern of neural processing in thehuman auditory cortex. Science 297, 1706–1708.

Stevens, M.C., Kiehl, K.A., Pearlson, G.D., Calhoun, V.D., 2007. Functional neuralnetworks underlying response inhibition in adolescents and adults. Behav. BrainRes. 181, 12–22.

Stone, J.V., 2004. Independent Component Analysis: A Tutorial Introduction. MIT press,Cambridge, MA.

Stone, J.V., Porrill, J., Buchel, C., Friston, K., 1999. Spatial, temporal, and spatiotemporalindependent component analysis of fMRI data. Proc.Leeds Statistical ResearchWorkshop.

Svensen, M., Kruggel, F., Benali, H., 2002. ICA of fMRI Group Study Data. NeuroImage 16,551–563.