1

ERPFeaturesandEEGDynamics:AnICAPerspective

ScottMakeigandJulieOnton

smakeig@ucsd.edujulie@sccn.ucsd.edu

SwartzCenterforComputationalNeuroscienceInstituteforNeuralComputationUniversityofCaliforniaSanDiego

LaJolla,California,USA

SubmittedversionofDec.8,2008

Aninvitedchapterfor:S.Luck&E.Kappenman(2008).OxfordHandbookofEvent‐RelatedPotentialComponents.

NewYork,OxfordUniversityPress

2

Prefatoryremarks

Followingtheadventofaveragingcomputersintheearly1960s,event‐relatedpotential(ERP)averaging became the first functional brain imagingmethod to open a window into humanbrain processing of first sensory and then cognitive events, and the first to demonstratestatistically reliabledifferences in thisprocessingdependingon thecontextual significanceoftheseevents–or theirunexpectedabsence.Yet thesameresponseaveragingmethods,noweasily performed on any personal computer, may have helped brain electrophysiologicalresearch remain split into two camps. For nearly half a century now, researchers mainly inpsychologydepartmentshave recordedhumanscalpelectroencephalographic (EEG)dataandstudied the features of human average ERPs time‐locked to events and behavior, whileresearchersmainlyinphysiologydepartmentsmeasuredaveragedevent‐relatedchangesinthenumberofspikesemittedbysingleneurons inanimals,capturedfromhigh‐passfiltered localfield potential (LFP) recordings from microelectrodes. Since the spatial scales of thesephenomenaare sodifferent, these twogroupshavehad little to say tooneanother. In fact,relationshipsbetween thesequitedifferentaveragebrain responsemeasurescanbe learnedonlybystudyingthespatiotemporalcomplexitiesofthewholeEEGandLFPsignalsfromwhichtheyarerespectivelyextracted,andbyunderstandingnotonlytheiraveragebehavior,butthecomplexitiesoftheirmoment‐to‐momentdynamicsaswell.

Many open questions remain about the nature and variability of theseelectrophysiologicalsignals,includingthefunctionalrelationshipsoftheirdynamicstobehaviorand experience. This investigation is now beginning, with much more remaining to bediscoveredabout thedistributedmacroscopicelectromagneticbraindynamics that allowourbrainstosupportustooptimizeourbehaviorandbrainactivitytomeetthechallengeofeachmoment.Fromthispointofview,keyopenquestionsforthoseinterestedinunderstandingthenatureandoriginsofaveragescalpERPsarehowtoidentifythebrainsourcesofEEGandERPdynamics, their locations, and their dynamic inter‐relationships. Toadequately address thesequestions,newanalysismethodsarerequiredandarebecomingavailable.

Ourresearchoverthepastdozenyearshasconvincedus,andanincreasingnumberof

other researchers, that using independent component analysis (ICA) to find spatial filters forinformationsources inscalp‐recordedEEG(andother)data,combined inparticularwithtrial‐by‐trial visualization and time/frequency analysis methods, are a powerful approach toidentifying the complex spatiotemporal dynamics that underlie both ERP averages and thecontinuallyunfoldingandvaryingbrainfieldpotentialphenomenatheyindex.Inessence,ICAisamethodfortrainingorlearningspatialfiltersthat,whenappliedtodatacollectedfrommanyscalp locations, each focus on one source of information in the data. Characterizing theinformation content of data rather than its variance (the goal of previous signal processingmethods)isapowerfulnewapproachtoanalysisofcomplexsignalsthatisbecomingevermoreimportant for datamining of all sorts. Applied to EEG data, ICA tackles ‘head on’ themajorconfounding factor that has limited the development of EEG‐based brain imaging methods,namelythebroadspreadofEEGsourcepotentialsthroughthebrain,skull,andscalpandthemixingofthesesignalsateachscalpelectrode.

3

I.EEGsourcesandsourceprojectionsIn this chapter, we consider the relationship between ongoing EEG activity as recorded inevent‐relatedparadigms,andtrialaveragestimelockedtosomeclassofexperimentalevents,knownasevent‐relatedpotentials(ERPs).WefirstdiscusstheconceptoftheERPasaveragingpotentialsgeneratedbyspatiallycoherentactivitywithinanumberofcorticalEEGsourceareasaswellasnon‐brainsourcestypicallytreatedasdataartifacts.Weuse“ERP‐image”plottingtovisualize variability in EEG dynamics across trials associatedwith events of interest using anexampledataset.Wethenintroducetheconceptanduseofindependentcomponentanalysis(ICA)toundotheeffectsofsourcesignalmixingatscalpelectrodesandtoidentifyEEGsourcescontributingtotheaveragedERP.Wenotethat“independentcomponents”arebrainornon‐brainprocessesmoreorlessactivethroughoutthedataset,andthusrepresentaquitedifferentuseoftheterm“component”inthetitleandelsewhereinthisvolume.Afterintroducingsomebasictime/frequencymeasuresusefulforstudyingtrial‐to‐trialvariability,wetakeanotherlookat trial‐to‐trial variability, now focusing on the contributions of selected independentcomponentprocessestotherecordedscalpsignals.Wehopethechapterwillhelpthereaderinterested inevent‐relatedEEGanalysis tothinkcarefullyabouttrial‐to‐trialEEGstabilityandvariability. The latter we suggest largely reflects not “ERP noise” but instead the brain’scarefully constructed response to the highly individual, complex, and context‐defined“challenges”posedbyunfoldingevents.WhatisanEEGsource?Afundamentalfactaboutelectrophysiologicalsignalsrecordedatanyspatialscaleisthattheyreflectand indexemergentpartialcoherence(inbothtimeandspace)ofelectrophysiologicaleventsoccurringatsmallerscales.Brainelectrophysiologicalsignalsrecordedbyrelativelylargeand/or distant electrodes can be viewed as phenomena emerging from the possibly onequadrillionsynapticeventsthatoccur inthehumanbraineachsecond.Theseevents, inturn,arisewithinthestillmorevastcomplexityofbrainmolecularandsub‐moleculardynamics.Thesynchroniesandnear‐synchronies,intimeandspace,ofsynapticandnon‐synapticneuralfielddynamicsprecipitatenotonlyneuralspikes,butalsootherintracellularandextra‐cellularfieldphenomena – both those measurable only at near field (e.g., within the range of a neuralarbor),andthoserecordedonlyatfarfield(inparticular,aselectricallyfarfromthebrainasthehumanscalp).Theemergenceof spatiotemporal field synchronyornear‐synchronyacrossanareaofcortex isconceptuallyakintotheemergenceofagalaxy intheplasmaofspace.Botharespontaneouslyemergentdynamicphenomenalargeenoughtobedetectedandmeasuredatadistance–viaEEGelectrodesandpowerfultelescopes,respectively.

The emergence of synchronous or near‐synchronous local field activity across some

portionofthecorticalmantlerequiresthatcellsinthesynchronizedcorticalareabephysicallycoupledinsomemanner.Abasicfactofcorticalconnectivityisthatcortico‐corticalconnectionsbetween cells are highly weighted toward local (e.g., shorter than 0.5‐mm) connections,

4

particularlythosecouplingnearbyinhibitorycellswhosefastgap‐junctionconnectionssupportthespreadofnear‐synchronousfielddynamicsthroughlocalcorticalareas(MurreandSturdy,1995; Stettler et al., 2002). Also important for sustaining rhythmic EEG activity arethalamocortical connections that are predominantly (though not exclusively) organized in aradial, one‐to‐one manner (Frost and Caviness, 1980). EEG is therefore likely to arise asemergentmesoscopic patterns (Freeman,2000)of local field synchronyornear‐synchrony incompact thalamocortical networks. Potentials arising from vertical field gradients associatedwithpyramidalcellsarrayedorthogonaltothecorticalsurfaceproducethelocalfieldpotentialsrecordedonthecorticalsurface(Luck,2005;Nunez,2005).Synchronous(ornear‐synchronous)field activity across a cortical patch produces the far‐field potentials that are conveyed byvolumeconductiontoscalpelectrodes.Bothscalpanddirectcorticalrecordingsagreethat innearly all cognitive states, such locally coherent field activities arise within many parts ofhumancortex,oftenwithdistinctivedynamicsignaturesindifferentareas.Directobservationsin animals report that cortical EEG signals are indeed associated with sub‐centimeter sizedcorticalpatcheswhose spatialpatterns resemble ‘phase cones’ (like ‘pond ripples,’ (FreemanandBarrie,2000))orrepeatedlyspreading‘avalanche’events(BeggsandPlenz,2003),thoughmore adequate multi‐resolution recording and modeling are needed to better define theirspatiotemporalgeometryanddynamics.

Inthischapter,wewillusethetermEEGsource tomeanacompactcorticalpatch(or

occasionally, patches)withinwhich temporally coherent local field activity emerges, therebyproducing a far‐field potential contributing appreciably to the EEG signals recorded on thescalp.Wewill use thephrase sourceactivity to refer to thevarying far‐fieldpotential arisingwithinanEEGsourceareaandvolume‐conductedtothescalpelectrodes.RecordedEEGsignalsarethen,inthisview,thesumofEEGsourceactivities,contributionsofnon‐brainsourcessuchas scalp muscle, eye movement, and cardiac artifacts, plus (ideally small) electrode andenvironmentalnoise.

NotethattheactivitycontributedbyacorticalsourcetotherecordedEEGtypicallydoes

not comprise all the local field activity within the cortical source domain, since potentialsrecordedwithsmallcorticalelectrodesatdifferentpointsinacorticalsourcedomainmayonlybeweaklycoherentwiththefar‐fieldactivitythatispartiallycoherentacrossthedomainandisthereforeprojectedtothescalpelectrodes.Thatis,onlytheportionofthelocalfieldactivityinasourcedomainthat issynchronousacrossthedomainwillcontributeappreciablytothenetsourcepotentialsrecordedbyscalpelectrodes.Thus,corticalelectrophysiologyisbyitsnaturemultiscale, its properties differ depending on the size of the recording electrodes and theirdistance from the source areas in ways that are currently far from adequately observed ormodeled. Scalp EEG recordings predominantly capture the sum of locally coherent sourceactivitieswithinanumberofcorticalsourcedomains,plusnon‐brainartifactsignals.RolesofEEGsourceactivitiesTheprimaryfunctionofourbrainistoorganizeandcontrolourbehavior'inthemoment'soasto optimize its outcome. For many neurobiologists, field potential recordings have been

5

consideredofpossibleinterestatbestonlyaspassive,indirect,andquitepoorstatisticalindicesof changes in neural spike rates, their primary measure of interest. In fact, however, thevariations inelectricalpotentialrecordedbyEEGorLFPelectrodesbetterreflectvariations inconcurrent dendritic synaptic input to neurons, input that may or may not provoke actionpotentials. Action potential generation is provoked by receipt of sufficient dendritic inputwithin a brief (several‐millisecond) time window. The emergence of synchronized local fieldpotentials across a cortical areamay therefore reflect changes in occurrence of joint spikingeventsacrossgroupsofassociatedneuronsinthatarea.Somerecentexperimentalresultsalsosuggestthatlocalfieldpotentialsmayalsoactivelyaffectspiketiminganddegreeofsynchronybetween of neurons within a partially‐synchronized source domain, biasing their joint spiketiming towards (or away from) concentration into brief, potent volleys (Voronin et al., 1999;Francisetal.,2003;Radmanetal.,2006).Bythismeans,smallstatisticaladjustments in jointspike timingof neuronswith commonaxonal targets effectedby spatially synchronized localfieldpotentialsmightbeassociatedwithlargechangesineffectiveneuralcommunication,andthencewithbehaviorandbehavioraloutcome(Friesetal.,2007).

According to recent reports, the timing and phase of extracellular fields may alsoenhance orweaken the effects of concurrent input on future cell and areal responsivity, byaffectingtheamountoflong‐termsynapticpotentiation(LTP)producedbythatinput(DanandPoo, 2004). Thus, locally synchronous (or near‐synchronous) field activity arising withincompact cortical source areasmay not only weakly index neural dynamics on spatial scaleslargerthanasingleneuron,butmayalsoplayamoredirectandactiverole inorganizingthedistributed brain dynamics that support experience, behavior, and changes inpsychophysiological state. The spatiotemporaldynamics of cortical field synchrony, and theirrelationshiptoneuralspiketiminghavestillbeenrelatively littlestudied.Thereis likelymuchmore to be discovered about relationships between extra‐cellular fields and intracellularpotentialsinlivingbrains.

SpatialsourcevariabilityTheconceptthatanEEGsourcerepresentstheemergenceofsynchronousfieldactivityacrossacorticalpatchisundoubtedlyasimplificationoftheactualmorecomplex,multi‐scaledynamicsthat producenear‐synchronous activitywithin an EEG source domain. Although the conceptthat EEG is produced by synchronous field activity in small cortical domains is supportedqualitatively by fMRI results that are generally dominated by roughly cm‐scale or smallerpockets of enhanced activity, and by a few reports of direct field potential grid recordings(Bullock, 1983; Freeman and Barrie, 2000), when cortical activity in animals is viewed at asmaller (sub‐mm) scaleusingoptical imaging, smaller‐scalemovingwavesofelectromagneticactivityareobserved(Arielietal.,1995).However,simplecalculationof thephasedifferencebetweentheedgesandthecenterofa‘pondripple’patternatEEGfrequencies,basedontheestimated(cm2‐scale)domainsizesandobservedtravelingwavevelocities(1‐2m/s),suggeststhatthespatialwavelengthoftheradiating‘ripples’isconsiderablylargerthanthesizeofthedomain,meaningthatthetopographicscalpprojectionofacortical‘phasecone’isclosetothatoftotallysynchronousactivityacrossthepatch,asinoursimplifiedEEGsourcemodel1.

6

However,larger‐scaletravellingormeanderingwavesatslow(1‐3Hz)deltaorinfraslow

(<1Hz)EEGfrequencieshavealsobeenobservedinepilepsy,sleep,andmigraine(Massiminietal., 2004), and (near 12‐Hz) sleep spindles may also ‘wander’ over cortex in concert withspatially varying activity in coupled regions of the thalamic reticular nucleus (Rulkov et al.,2004). Sufficiently detailed recordings from high‐density, multi‐resolution arrays are not yetavailable to allow models of the relationship between these moving activity patterns andapparentlystableEEGsourcedynamicsinwakinglife.TemporalsourcevariabilityA hallmark of EEG is that its temporal dynamics are highly non‐stationary and exhibitcontinuouschangesonalltimescales(Linkenkaer‐Hansenetal.,2001).ChangingEEGdynamicsindex changes in and between local synchronies that are driven or affected by a variety ofmechanisms including sensory information as well as broadly projecting brainstem‐basedarousal or ‘value’ systems identified by their central neurotransmitters – dopamine,acetylcholine, serotonin, neurepinephrine, etc. These ‘neuromodulatory’ systems based inbrainstemareasprojecttowidespreadcorticalareasandareverylikelyanimportantsourceoftemporalvariabilityinthespatiotemporalcoherencethatproducesfar‐fieldsignalsrecordedatthescalp,variabilitythatgivesflexibilityandindividualitytoourdistributedbrainresponsessoas to respondmost appropriately to theparticular challenge of themoment. Ranganath andRainer (Ranganath and Rainer, 2003) have reviewedwhat is known and still unknown aboutthesesystemsandtheirinteractionswithcorticalfieldpotentials.VolumeconductionandsourcemixingExperimentalneurobiology suggests that thespontaneousemergenceofpartial coherenceofcomplexrhythmictemporalpatternsoflocalfieldactivityacrosscompactcortical‘phasecone’or ‘avalanche’domainsa fewmmor larger indiameterproduce thescalpEEGandthereforeERPsignals.Thedifferencesbetweendistinctpartsofneuronswithinthepartially‐synchronizedand nearly‐aligned pyramidal cell domain sum coherently both in local recordings and asmeasuredatanydistanceafterpassingbyvolumeconduction through interveningconductivemedia including cortical grey and white matter, cerebral‐spinal fluid (CSF), skull, and skin(Akalin‐AcarandGencer,2004;GencerandAkalin‐Acar,2005).Theverybroadcorticalsourcefieldpatterns(eachgenerallyresemblingthedouble‐lobedpatternironfilingstakewhenplacedaround a bar magnet) are attenuated by the partial resistance of these media, and theirpropagationpatternsarespatiallydistortedattissueboundarieswhereconductancechanges.Thebroadlyprojecting, spatially distorted, and severely attenuated signals are then summedwithinconductiveEEGelectrodesattachedtothescalp(Nunez,1977).

ForwardandinversemodelingA first question for EEG/ERP researchers, therefore, is or should be how to separate therecorded EEG activities recorded at all the scalp channels into a set of activities originating

7

withindifferentspatialsourcedomains.Findingtheappropriatespatialfiltersis,unfortunately,technically difficult, and were any arbitrary 3‐D arrangement of source configurationsphysiologically possible, any number of them could be found that would produce the samescalp potentials (Grave de Peralta‐Menendez and Gonzalez‐Andino, 1998). A biophysicalsolution to this so‐called inverse problem must begin with construction of a forward headmodel specifying (1) where in the brain the electromagnetic sources may be expected toappear, and (2) in which orientations, and (3) how their electromagnetic fields propagatethrough the subject’s head to the recording electrodes (Akalin‐Acar and Gencer, 2004).Fortunately, the well‐grounded assumption that the brain EEG sources are cortical sourcepatches whose field patterns are oriented near‐perpendicular to the local cortical surfaceallowsmorephysiologicallyplausibleestimatessincetheseassumptionsallowafairMRimage‐derivedmodeloftheshape,location,andlocalorientationofsubjectcortex(Fischletal.,2004).ConstructingindividualizedcorticalmodelsforEEGanalysisrequiresextensivecomputationandexpensiveMRheadimages,andthustheprocessisstillrarelycarriedoutforroutineEEG/ERPexperiments.While adequate headmodels are needed to develop EEG into a 3‐D functionalbrain imaging modality, different analysis goals may require various degrees of anatomicaccuracy,anduseofstandardheadmodelsmaysufficeformanyanalysispurposes.

Givenanaccurateforwardheadmodel,theinverseproblemisstillunderdeterminedifmultiplesourcescontributetotheobservedscalppotentialdistributionwhosesourcesaretobedetermined.Incontrast,thesolutionismuchsimpler,anditsanswerbetterdetermined,ifthescalpmapswhosesourceprojectionstheysumaresimplemapsrepresentingtheactivityofonlyonesource.BothEEGandMEGresearchershavelongattemptedtoconsiderscalpmapsofamplitudepeaksinaverageERPstobesimplemaps.Thatis,theyattempttouseERPaveragingasameansofspatialfilteringtoeliminateprojectionsofEEGsourceareasnotdirectlyinvolvedinthebrainresponsetothetime‐lockingevents.

Inmanycases,takingthedifferencebetweentwoaverageERPstime‐lockedtorelated

sets of experimental events may further restrict the number of strongly contributing brainsourceareas.Unfortunately,trialaveragingordifferencingisrarelycompletelyeffectiveforthispurpose, since meaningful sensory (as well as purely cognitive) events rapidly perturb thestatistics ofmany EEG sources andother (subcortical) brain processes (Halgren et al., 1998).Therefore, except for very early sensory brainstem and cortical potentials, ERP maps sumactivities thatarise, typically, inmanysourcedomains.Thismeans that scalpmapsofERPorERPdifference‐wavepeaksareonlyrarelysimplemapsrepresentingpotentialsprojectedtothescalpfromasinglesource.TooptimallyestimatethesourcedistributionresponsibleforEEGorERPdata,itisdesirabletofindabetterwaytoisolatesimplemapsrepresentingtheprojectionofsinglesourcescontributingtothedata,asubjectwewillreturntoinSectionIII.

II.ERPtrialaveragingandtrialvariabilityThe top panel of Figure 3.1 shows a single‐subject ERP for all 238 scalp channels

averagedover500dataepochstime‐lockedtoonsets(alatency0)ofaninfrequentlypresented

8

visualtargetdiscinavisuospatialselectiveattentiontask(Makeigetal.,1999b;Makeigetal.,1999a;Makeigetal.,2002;Makeigetal.,2004b).ERPtracesforall238channelsareoverlaidonthesameplotaxis.InterpolatedscalpmapsshowtheERPscalpdistributionatfourindicatedlatencies. Thebottompanel shows theERPaverageof thesame500epochs,butnowtime‐lockedtothesubject’sbuttonpressineachtrial.Inbothpanels,thedatawereaveragedafterremovingartifactsproducedbyeyemovements,eyeblinks,electrocardiographic(ECG)activity,and electromyographic (EMG) from scalp and neck muscles using independent componentanalysis (ICA), as explained in Section III2,3. We will use these data through this chapter toexplore relations between the average ERPs as shown in this figure and the single EEG dataepochsthatwereaveragedtoproducethem.

Figure 3.1. ERP tracesfrom each of 238 scalpchannels averaged over 500EEGepochs inasinglesubject,time locked to (A) continuallyanticipated but infrequentlypresented visual target stimuliin a selective visuospatialselective attention task, and(B) immediately followingspeededsubjectbuttonpressescued by target presentation2.Solid lines cutting verticallythrough theERPchannel tracebundles lead to cartoon headsshowingthe interpolatedscalppotential distribution at themomentindicated.Thetrial‐averagingmodel

Over the past near half‐century, the predominantmethod for reducing thecomplexity of event‐relatedEEG data collected in sensoryand cognitive paradigms is toform event‐related potential(ERP) averages of trial recordstime‐locked to sets ofexperimental events assumedby the experimenter togenerate the same or

essentiallysimilarbrainresponses.TogainarealisticunderstandingofthefeaturesofERPtrial

9

averagesandtheirrelationshiptotheunderlyingbraindynamics,itisimportanttounderstandboth the strengths and limitations of ERP averaging. Thephysiologicalmodel underlying ERPaveraging is that cortical processing of sensory (or other) event information follows a fixedspatiotemporal sequence of source activities, and that this processing produces a fixedsequence of deviations in scalp potentials whose distribution reflects the locations of theircorticalgenerators.However,thesetracesofthecorticalprocessingsequenceareobscuredinsingleresponseepochsbytypicallymuchlargerongoingEEGactivitiesgeneratedinmanybrainareas,aswellasartifactsgeneratedinnon‐brainstructures.Crucially,theseongoingactivitiesareassumedtobeunaffectedbythetime‐lockingeventsofinterest.

Thus, in the ERP averagingmodel, EEG epochs are assumed to sum (1) a temporallyconsistent event‐related activity sequence (“the evoked response”), plus (2) event‐unrelatedongoing or spontaneous EEG activities (not contributing to the ERP). Under thesecircumstances, averaging a sufficient number of event‐locked epochs subtracts, cancels, orspatially filters out the unrelated brain activities, leaving a single average response epochdominated by the consistent event‐related activity sequence, recorded on the scalp as theflowingERPfield ‘movie.’TheamplitudesofEEG(orothernon‐brain)activitiesunaffectedbythe time‐locking events that remain in the average will be approximately 1/N1/2 of theamplitudeof those activities in the single trials,whereN is thenumberof epochs averaged.Thus, achieving a faithful representation of the actual (and typically relatively small) evokedresponsesequenceusuallyrequiresaveragingarelativelylargenumberofevent‐relatedepochsknownorassumedtocontainthesametime‐lockedERPsequence.

TounderstandhowERPaveragingleadstoareductioninevent‐unrelatedEEGactivity,wefirstneedtodefinethephaseofanEEGsourcesignal.Thesimplestsenseofthetermmightbe the sign or polarity of the recorded potential at a given time point, either positive ornegativeinrelationtosomebaselinepotential(typicallyestablishedbyaveragingthepotentialsrecordedinsomeperiodoftherecordingassumedtobeunaffectedbytheeventsofinterest).Amorespecificmeaningforthe‘phase’ofasignalatsometimepointandfrequencyisasthephase of a best‐fitting brief, tapered sinusoid at the given frequency centered on that timepoint.ThusthephaseofanEEGsourceorscalpsignal,atanygiventimepointandfrequency,isdefinedbytherelationofitsvalueatthattimepointtoitsvaluesinanenclosingwindowoftimepoints.Notethatatagiventimepointasignalhasadifferentphaseateachfrequency.Also,sinceEEGsignalsarerelativelysmooth,EEGphasedifferencesbetweenneighboringtimepointscannotvaryfreelybutmustchangesmoothly.4 ERPaveragingcanremovethecontributionsofthosesourceactivitiesunrelatedtothetime‐lockingeventsbymeansofphasecancellation,whichworksasfollows. Ifagivensourcesignalisunaffectedbythetime‐lockingevents,andifthetimingoftheexperimentaleventsisnot based on the ongoing EEG signals, their phase at each latency and frequencywill differrandomlyacrosstrials.Mathematically,thesumofrandom‐phasesignalsatagivenfrequencytends to become smaller and smaller (at that frequency) as the number of summed trialsincreases.Wecanseethismosteasilybyconsideringthesignsofthesignals(+or‐)insteadoftheirphases.Ifthesignsofasetofsignalepochvaluesatagivenlatencyarerandom,thenin

10

theaverageofthoseepochsthepositive‐phaseandnegative‐phasevaluesindifferentepochswillpartiallycanceleachother,andthemagnitudeoftheaverageepochatthatlatencywillbesmallerthantheaverageofthesamevaluesweretheyallofthesamesign.

Similarly, if at a given analysis frequency (say for example 9 Hz), the single‐trial EEGsignalshavearandomphasedistributionwhenmeasuredinatimewindowcenteredatsomelatency (say, 200ms following the time‐locking event), then the vectors that canbeused torepresenttheiramplitudesandphasesatthatfrequencywillbemoreorlessevenlydistributedaround thephase circle, and theexpected lengthof the vector averageof these vectorswillbecome smaller as the number of trial vectors averaged increases, provided that the exacttimingofexperimentaleventscannotbepredictedbythebrain.Onaverage,thelengthoftheaveragephasevectorwilldecreaseasthesquarerootofthenumberoftrialsaveraged.Inthisway, trialaveragingfiltersoutall featuresof thedatathatarenotwhollyorat leastpartiallyphase‐lockedtothetime‐lockingeventsatanyfrequencyandlatency.

It is important tounderstandhowevent‐relatedphase‐lockingandtime‐lockingdiffer.

Forexample,imagineasetofEEGtrialepochs,time‐lockedtoaparticulartypeofevent,thateachcontainaburstof10‐Hzalphabandactivitycentered500msafterthetime‐lockingevent.Further, imagine that these alpha bursts, while undeniably time‐locked to the experimentaleventsofinterest,mayexhibitanyphaseat500ms(ascending,descending,etc.).Thesebursts,therefore,arenotphase‐lockedtotheevents.AnERPaverageofenoughsuchepochswouldthereforecontainlittletraceof10‐Hzactivityat500ms,eventhoughtthisisastrikingfeatureofthesingle‐trialdata.Thisisbecausetrialaveragingfiltersoutallactivitythatisnotbothtime‐lockedandphase‐lockedtothetime‐lockingevent..Thus,scalpERPsdonotcapturealloftheconsistently event‐related dynamics in the averaged EEG epochs, but only those dynamicprocesses that affect thephasedistributionof their signals at someanalysis frequencies andtrial latencies. We will consider this question again in Section IV (see also Chapter 2, thisvolume).

However, ifagivenbrainsourcecontributesa fixedactivitysequencetoagivenscalpchannel signal in every trial, at each analysis frequency and epoch latency the phase of itscontributions to thescalpsignalswillbeconsistentacross trials,and theERPaverageat thatscalpchannelwillcontainallofthatsource’sactivitysequence,withoutdiminution.Ifthephaseofitssingle‐trialactivityatagivenfrequencyandlatencyisvariableandonlyweaklyconsistent,relative to a true randomphase process, then the sourcewill contribute onlyweakly to thescalpERP.Ifthesourceactivityhastrulyrandomphaseatthegivenfrequencyandlatency,thenitsERPcontributionwillbeminimalandfurtherdecreasingasmoretrialsareaveraged.

If all the EEG sources that project to a scalp channel have fixed evoked activity

sequences,thentheircollectivecontributiontothechannelineachtrialwillbethesumofalltheirsourceactivities,andtheaverageERPatthatchannelwillbetheaverageofthesummedsource activities at each trial latency. Thus sourcemixing that occurs at the scalp electrodecoulddecrease(or increase)theapparentERPmagnitudethroughthesameprocessofphasecancellation.Butagain,onlythosesignalsthatarephase‐lockedtothetime‐lockingeventsfrom

11

trialtotrialwillberetainedintheaverage.Andconversely,foractivityatagivenfrequencyandlatency tobe removed fromthesignalbyaveraging,only the signalphase in the single trialsneedberandom.

Limitationsofevent‐relatedaveragingThemeanofanydistribution is simplyonestatisticalmeasureof thedistribution–astatisticthat, if provided apart from other statistics, may be informative and/or misleading. Forexample,tellingaNewGuineanunacquaintedwithAmericansthattheaverageadultAmericanheight is 5’6” (1.68 m) might give him or her an adequate concept of the distribution ofAmerican adult heights – assuming the shapes and the widths of the (near‐normal) heightdistributions in the two cultures are not dissimilar. However, sendingMartian scientists thearithmeticallyequallycorrectinformationthattheaveragehumanishalfmaleandhalffemalemightwellengenderquiteincorrectideasabouthumanbiologyandsociety.Theproblemhereisthathumansexualphysiologyhasnotonebuttwoquitedistinctmodes(femaleandmale),information that is not captured in or conveyed by the average. Thus, the average of adistribution may or may not in itself provide or suggest a useful and realistic model of theunderlying distribution or its features. This may be even more problematic for time seriesaverages that sum disparate activities of many distinct brain and non‐brain sources whosedetailed features are of primary interest, including their spatial and temporal trial‐to‐trialvariability.Spatialvariabilityofevent‐relatedactivityNotehowthescalptopographiesoftheERPsinbothpanelsofFigure3.1differslightlybeforeandafterthebuttonpress.IfERPspatialvariationsweregeneratedwithinordirectlyunderthescalp itself, such changes would reflect potential changes occurring directly below themoststrongly affected electrodes. Such an interpretation, while having naïve appeal, is howevercontrarytotheanatomicandbiophysicalfactsaboutvolume‐conductedcorticalfieldpotentialsthat actually produce scalp EEG signals, as summarizedabove. The verybroad ‘point‐spread’pattern of potentials propagating out by volume conduction from each cortical source areameansthateachsourcecontributestosomeextenttothesignalsrecordedatnearlyallofthescalpelectrodes,andcontributesappreciablytomanyofthem.

Further,ifeachsourceareaisspatiallyfixed(ornearlyso),byitselfitcannotproduceamovingtopographicpatternoffieldactivityonthescalp–itcanonlyproduceproportionalandsimultaneouschangesacrossalltheelectrodesinitsprojectionpattern.Therefore,changesinthescalpmapofaverageERPdata(asinFig.3.1)mustreflectsumsoftime‐varyingpotentialsprojected in thebroad andhighly‐overlapping scalppatterns from several spatially‐fixed EEGsource activities, each contributing to the ERP in spatially and temporally overlapping timewindows. This view is compatible with fMRI results showing that cortical activations anddeactivationsmainly occur within compact cortical domains – though direct high‐resolution,multi‐scaleobservationsofelectrocorticalactivitythatcouldconstitute‘groundtruth’evidenceforthisassumptionarenotyetavailable.

12

Twomainpointshereare,first,thatbasicbiophysicalknowledgeisnotconsistentwith

an interpretation of ERP potentials as exclusively (or even principally) reflecting activitygenerateddirectlybeloweachelectrode,afactitiseasytolosesightofwhenfocusingonthedetailsofsingle‐channelERPorEEGwaveforms.Second,althoughwhenanimated,changesinhigh‐densityERPtimeseriesappeartoflowacrossthescalp,featuresofmostERPsrecordedincognitive experiments are much more likely produced by sums of time‐varying activities ofrelatively small, spatially‐fixed cortical generator domains, each with a broad ‘point‐spread’pattern of projection to the scalp surface. Looking ahead a bit: Although the exact sizedistributionofthesedomainsisnotyetknown,fairlypreciseindicationsoftheircenterscanbeobtained bymethods that spatially filter the scalp EEG data to focus on single sources (seeSectionIII).ERPsasspatialfilters?Early ERP analysis attempted to deal with the difficulty in interpretation posed by volumeconduction and scalp mixing by assuming that event‐related averaging provides sufficientspatial filteringof themanysourcesignals reaching thescalpelectrodesso thatactivity fromonlyoneaffectedsourceareacontributestoeachERPamplitudepeak.Thatis,someearlyERPresearchershopedthatthesequenceofpeakscomprisingERPwaveformswouldeachspatiallyfilteroutallactivitiesnotgeneratedinasinglecorticalarea.

However,subsequentresearchclearlysuggestedthatsoonaftersensorysignalsarrivein

cortexfollowingmeaningfulevents,multipleEEGsourcesbegintocontributetoERPaverages.It has now been shown that in animals coordination of activities between early visual areas(Grinvald et al., 1994) andbetweenprimary visual and auditory cortex (Foxe and Schroeder,2005) begins as early as 30ms after stimulus onsets, and invasive recordings from epilepticpatients for clinical purposes show that by atmost 150ms after presentation ofmeaningfulvisualstimuli,thephasestatisticsoflocalfieldprocessesarealteredinmanypartsofthebrain,both cortical and subcortical (Klopp et al., 2000). Thus, the somewhat different scalpdistributions inthesingle‐channelERPscalpfieldmapsofFig.3.1infactrepresentdifferentlyweighted mixtures of mean event‐related activities, time‐locked to subject button presses,fromseveraltomanycorticalsourceshavingbroadandstronglyoverlappingscalpprojections.Also, direct cortical recordings from both animals and humans show that, through activitycycles through thalamocortical and other network connections involving significant delays,singlecorticalareasoftenproducemulti‐peakedcomplexesinsteadofsingleresponsepeaksinresponse to single stimulus presentation events (Swadlow and Gusev, 2000), adding to thespatialoverlappingofsourceactivitiesinERPwaveforms.

Thus,responseaveragingisnotanefficientmethodforspatialfilteringofevent‐related

EEG data since it typically does not produce a sequence of simple maps each reflecting theprojectiontothescalpofasinglecorticalsource.ComputingtheERP‘differencewave’ateachscalp channel between ERPs in two contrasting conditions may more effectively isolatecondition differences in the activity of a single generator or set of generators, although in

13

general the effectiveness of this approach cannot be guaranteed. Finding more effectivemethods for spatial filtering of EEG data into EEG source activities is therefore of urgentimportancetohumanelectrophysiologyresearch.Temporalvariabilityofevent‐relatedactivityAnotherproblemwithmodelingevent‐relatedEEGdynamicsbasedonERPaveragesalone isthat ERP averaging collapses and thus conceals both the orderly (event‐, task‐, or context‐related)aswellasthedisorderly(event‐,task‐,andcontext‐unrelated)trial‐to‐trialvariabilityintherecordedEEGscalpandsourcesignals–givingnowayfortheusertodeterminetherelativeproportionsortypesofthesetwoclassesofsignalvariationsthatarepresentinthesingletrials.AmodelofbrainactivitybuiltsolelyonanERPaverageofscalpactivityinevent‐relatedepochsmustfailtoincludemanyaspectsofthebrainprocessesthatproduceit.Ifthesingle‐trialEEGepochs each sum activity and time‐varying activities from multiple spatial sources whosedynamicsaretightly linkedtomultipletaskorcontext‐relatedfactors,focusingsolelyontheiraveragemaydiscouragestudyoftheirorderlytrial‐to‐trialvariations.

Averagingitselfsimplifiesnotonlythespatialpattern,butalsothetemporalpatternsofthesignalsaveraged,retainingonlythatportionofthesignals (typically,asmallportion)thatarebothtime‐lockedandphase‐lockedtothetime‐lockingsignals (asexplainedaboveand inChapter2,thisvolume).Fartoooftentrial‐to‐trialvariabilityofEEGsignalsissimplydismissedbyresearchers(eitherexplictlyorimplicitly),asrepresentingirrelevantbrain‘noise’–withoutsufficientconsiderationorevidenceforthisassumption.Toconsiderthispointmorecarefully,letusexaminesomeaspectsofthetrial‐to‐trialtemporalvariabilityinscalpEEGactivitybeforeandafter targetpresentations in the selectivevisual attention task session forwhichFig.3.1showedtheERPaverages.

Afirststeptowardsunderstandingtrial‐by‐trialvariabilityinEEGepochstime‐lockedto

someclassof time‐lockingevents is to findusefulways tovisualize theirvariability. JungandMakeig have developed amethod of sorting the order of single trials by some criterion andthenplotting themashorizontalcolor‐coded lines ina rectangular imagetheycalled theERPimage(Makeigetal.,1999b;Jungetal.,2001).InERP‐imageplots,single‐trialtracesdrawnashorizontal colored lines, with color (instead of vertical placement) encoding potential. Thecolored lines canbe fused intoa rectangular imageand smoothed, ifdesired,witha verticalmovingaveragetobringouttrends.Crucially,theorderinwhichthetrialsaresorted(bottomto top) need not follow the actual time order of the trials, but can be based on any othercriterion.Unlike,theaverageERP,whichisfixed,theERPimagesresultingfromdifferenttrialsortingorderscandifferdramaticallyfromeachother,eachbringingoutoneormoreaspectsoftrial‐to‐trialvariabilityinthedata.

14

Figure3.2.SixERP‐imageplotsof nearly 500 visual targetresponse epochs (same dataas in Fig. 3.1) at a scalpelectrode near the vertex(referred to right mastoid). Ineachpanel,trialpotentialsaresorted(frombottomtotop)inthe order indicated, thensmoothed (vertically) with a20‐trial moving window, andfinally color‐coded (see colorbar on lower right). In leftpanels (A‐C), the trials arealigned to the moment ofstimulusonset ineachtrial. Inright panels (D‐F), they arealigned to themoment of thesubjectbutton‐press response.The trace below each imageshows the ERPaverageof thetrials. Stimulus‐locked ERPpeaks N2 and LPC (latepositive complex) are labeledin A. The trial sorting criteria

are indicated in eachERP‐imagepanel.Horizontal arrows show thevalue (C)andphase (E,F)sortingwindows. The ERP images illustrate thewide variety of trial‐to‐trial differences in thedata.

PanelAofFigure3.2usesthismethodtovisualizenearly500singletrialstimelockedto

visual target stimulus presentations at a scalp site near the vertex (Cz) – the same stimulus‐lockedtrialsaveragedtoformtheERPshowninFig.3.1A,withthe(vertical)orderoftrialsheresorted in their original time order. For easier visualization of event‐related patterns acrosstrials,ineachpanelofFig.3.2wehavesmoothedthesingle‐trialdata(vertically)witha20‐trialmoving window. The solid vertical line in panel A marks the onset of the visual target, theverticaldashedlinethesubject’smedianbuttonpresslatency.TheaverageERP(shownbelowthepanel)appearstohaveonlyasmallandtemporallydiffuselatepositivecomplex(LPCor,forhistoricalreasons,“P300”)feature,herepeakingnear400msafterstimulusonset.

Panel B shows the same data trials, but here sorted by the latency of the subject’s

buttonpress(dashedtrace),againsmoothingwitha20‐trialmovingaverage.Wenowseethatinmosttrialsa(muchmoredistinct)LPCfollowsthesubject’sbuttonpressbyabout120ms,withLPCamplitudesmaller in long‐RTtrials(nearthetopoftheERP‐imagepanel).Thispanel

15

showswhichfeaturesofthepost‐stimulusERPareprimarilytime‐lockedtothestimulusonsetitself (e.g., the negative (blue) pre‐response N2 peak), and which to the subject behavioralresponse (the following LPC and two sparser ensuing positive wave fronts). The trial‐to‐triallatencyvariabilityoftheLPCcannotbelabeledasirrelevanttrial‐to‐trial‘noise,’andcannotbededucedfromthestimulus‐lockedtrialaverage(bluetracebelowpanelsAandB)inwhichtrial‐to‐trialvariabilitytime‐lockedtothebuttonpressratherthanthestimulusonsetistemporally“smearedout.”

PanelC showsagain thesametrials,heresorted insteadbymeanpotential in theN2

response ERP latencywindow indicated by the dashed lines.We see that only in about thebottom half of the trials is the mean potential in this window actually negative. But theseincludeabouta (bottom) thirdof the trials inwhich thenegativity is relatively large, thereby“outweighing” the contributions of the trials in which the single‐trial value is positive, thusproducinganegativepeakintheaverageERP(againshownbelowtheERP‐imagepanel).

Thecurvingpost‐RTpositive(red‐orange)wavefrontsintheERP‐imageplotinpanelBrevealthattheevokedLPCcanbemoreaccuratelyrepresentedbyanERPaverageofthesamedataepochstime‐alignedtothesubjectbuttonpressratherthantostimulusonset–asinpanelD. In particular, the averagemotor response‐aligned ERP (belowpanelD) better reflects theabruptonset,slope,anddurationofthepost‐motorLPCinthesingletrialsthanthestimulus‐alignedERP(belowpanelA).AsinpanelB,panelDshowsthatthepost‐buttonpresspositivityisgenerallystrongerin(lowermost)trialswithshortbuttonpresslatencies,andisweakorevenabsent in (uppermost) trials with long response latencies. Also, the slightly curving (red)responsecolumninpanelDsuggeststhatthetimelockingoftheLPCpeaktothebuttonpressis only relatively constant, the LPC appearing slightly wider and centered slightly later inshortest‐latencytrials,relativetoothertrials. These scalp data, measuring the potential difference between an “active” electrodenearthevertex(Cz)anda“reference”electrodeontherightmastoid(seecartoonheadabovepanel A), actually sum broadly spreading projections of field activities projecting by volumeconduction fromseveral tomanycortical sources.Themeanpower spectrum for these trials(abovepanelA)containsastrongalpha‐bandpeakat10Hz.PanelEshowsthesameresponse‐aligned trials as in D, now sorted according to the best‐fitting 10‐Hz phase in the indicatedthree‐cycle(300‐ms)widetrialsortingwindowcenteredontheLPC.InthisERP‐imageviewofthe data, the central positive (red) LPC peak of the activity in the single trials forms a redcurvingwave frontnear thecenterof thesortingwindow.Thepeak latencydifference (frombottommosttotopmosttrialsintheimage),clearlyvisibleinpanelE,ishiddeninpanelDusingadifferenttrial‐sortingorder. Panel F shows again the same response‐aligned trial data, but now sorted by alphaphase in thepre‐response (butpost‐stimulus)periodbetween thedotted lines.Here,we seethatongoingalphaactivity that is randomphasebefore thebuttonpress (as reflected in theperfectlydiagonalpositive andnegativewave fronts in the sortingwindow),but theensuingLPCpositivitypeakinginthisviewabout120msafterthebuttonpressisnearlyverticalinthis

16

view,andthereforeappearstobeindependentofthephasetheprecedingalphabandactivity(whichmightwellhaveadifferentsetofsourcesthanthosegeneratingtheLPC).NotethattheLPC positivity is again wider than the tighter peak obtained in the alpha‐sorted view of thesamedatainpanelE,thoughpanels(D‐F)allvisualizeaspectsofthesamedataandhavethesamemotorresponse‐lockedERP. Whatconclusionscanwedrawfromthesesixquitedifferentwaysofplottingthesamedata?NotethateachpanelhighlightsadifferentwayofseparatingtheERPtrialaverageintoaset of single‐trial activities, followed bymoving‐average smoothing to bring out trial‐to‐trialtrendsinthetrial‐sorteddataimages.Eachpanelrepresents,inaway,adecompositionofthesingle‐trialdatahighlighting some foreground featuresand smoothingother typesof trial‐to‐trial variability to form a kind of background “noise” (as it were). Which of thesedecompositionsinthissense–ifanyofthem–isthemostphysiologicallyrealisticor‘correct’decomposition? Arithmetically, there is no difference between them; in each case the sametrialdata,alignedasinpanelsA‐CorD‐F,havethesameERPaverage,nomatterhowthetrialsaresorted–justas5penniesareequallythesumof3+1+1or1+3+1coinsubsets.

The standardmodel underlying ERP analysis is that the EEG data essentially sum, (1)contributions to the ERP occurring in each trial, and (2) other (undefined) variable activitiesunaffected by the time‐locking events and therefore not contributing to the ERP. Is thisstandard (“ERP + background”) decomposition of the single‐trial signalsmore physiologically“realistic,”inanysense,thanthedifferentimplied“decompositions”ofthedataintothequitedifferent features that arehighlighted (plus thoseobscured) in these six quitedifferent ERP‐imagepanels?Inparticular–doanyoftheseviewsparsethedataintophysiologicallydistinctsourcecontributions?

From theseERP‐image representationsof thedata,wecanat least see someways inwhichanaverageERPof the trials is simplyone statisticalmeasureof them,ameasure thatdoesnotrevealtheirorderly(thoughcomplex)temporalvariationsormultiplespatialsources.Ineachtrial,thesubjectperformedthesametask–attemptingtoproducequickbuttonpressresponsestotargetstimuliwhilewithholdingresponsestonon‐targets.PanelBclarifiesatleastoneaspectoftrial‐to‐trialEEGvariabilitydirectlyrelatedtosubjectbehavior(e.g.,theirmanualresponse latency). What other trial‐to‐trial differences, either in stimulus features (here, intarget location),and/or inthetrialcontext (here,thehistoryofprecedingstimuliandmanualresponses)mayhavealtered thenatureof the ‘challenge’posed to thesubject’sbrain–andthus affected aspects of trial‐to‐trial EEG variations? Neither the ERP averages nor these sixERP‐imagepanelsanswerthesequestions.TherearemanyotherpossibledecompositionsintoputativeunderlyingEEGsourceactivitiesthatexaminationofitstrialaverageERPcannotruleinor out as reflecting physiologically valid distinctions among spatial sources and their event‐relatedtemporalpatterns.

Attheleast,thesepanelsillustratethefactthattrial‐to‐trialvariationsinEEGdataarenot simply “EEG noise.” Rather, they include several types of orderly trial‐to‐trial variabilitylinkedtoseveralEEGsourceand/orbehavioralandtaskparameters.PanelsC‐F, inparticular,

17

pose an interesting question. Is the LPC activity at this channel, time‐aligned to the subjectbuttonpressesdominatedbypositive‐phasealphaactivity(aspanelEsuggests)orbyabroaderfixed‐latency,centralLPCpeak(as inpanelF)?Orperhaps,bybothtypesofactivityarising indifferent cortical domains and both projecting to the vertex to differing relative extents indifferenttrials?

A simpleERPmodelof the trial‐to‐trial variability inEEGdata represents thenon‐ERP

portionofthedataassummingcontributionsofbrainsourceactivitiesthatareunaffectedbythetime‐lockingevents.Inanextremeversionofthismodel,theamplitudeoftheERPmightbeassumed to be identical in every trial, with any trial differences reflecting additional task‐irrelevantEEG(ornon‐brainartifact)activity.ButFigure3.2suggeststhatsuchastrictversionof themodel isunlikely toadequatelycapturethetrial‐to‐trialvariability in theevent‐relateddynamicsinthesetrials.

Figure 3.3. ERP‐image plots separating the ERPimageinpanelA,inwhichthetrialsaresortedbymean potential between the two dashed linessurrounding thepost‐responseERPpeak, into thesumof (B) an ERP‐imageof thebest‐fitting (non‐negative)mean‐ERPcontributiontoeachtrial,and(C) the remaining unexplained portion of eachtrial. As in Fig. 3.2, the mean of the single‐trialdata ineachpanel isshownbelowthepanel.TheERP has been largely (though not completely)removed from the lower panel data. Thisdecomposition is compatiblewith theassumptionthat the single‐trial data sum a single ERPresponseof variable amplitude (B) plus unrelatedEEGactivity (C).However,manyEEGsourcesmaymake separate contributions to the data and totheERPaverage.RegressionofthewholeaverageERP trace on the single data trials does not takeinto account the spatiotemporal variability of theindependent sources of trial‐to‐trial variation, forexample the partial sorting by value remainingwithin the sorting window and near 300 ms.(Verticalsmoothingwindowwidth20trials).PanelAofFigure3.3showsthesametrialdataasin Fig. 3.2, but now sorted bymean potential inthe indicated post‐response LPC data window.

PanelsBandCvisualizeanERP‐baseddecompositionof thedata inpanelA,all threepanelsusing the same trial sorting order (sorting by LPC amplitude). Panel B shows the estimatedcontributionofthemeanERPtoeachtrial,asdeterminedbyfindingthebestleast‐squarefitofthemeanERPtoeachsingle‐trialepochbutnotallowingnegativeERPtrialweightsinthefew

18

lowesttrials.PanelCshowstheremaining(non‐ERP)dataforeachtrial.Thus,thesumofthevalues inpanelsBandCarethewhole‐trialdataasshowninpanelA.At least intwolatencywindowsinPanelC(90‐150msand200‐300ms)exhibitsystematicdifferencesfromarandomtrial distribution, indicating trial‐to‐trial variability of potentials in thosewindows is partiallyindependent of the overall amount of ERP‐like activity in the trial. But is this attempteddecompositionofthesingle‐channeldata(inpanelA) intoERPandnon‐ERPdataportions(inpanelsB andC) aphysiologically valid separationbetween completely response‐locked (ERP)andresponse‐independent(non‐ERP)corticalsourceprocessesinthedata?

Howcanwebegintofindanswerstothisquestion?Tomodelthenatureofthehighlyvariablesignalsoccurringduringtheserecordeddataepochs,ideallyweshouldfirstfindawayto separate the whole scalp EEG data into a set of functionally and physiologically distinctsourceactivities.

III.SeparatingEEGsourcesusingIndependentComponentAnalysis(ICA)

In 1995 the first author and colleagues at Salk Institute (La Jolla, CA) performed the firstdecompositionofmulti‐channelEEGdataintoitsmaximallyindependentcomponents(Makeig,1996) using a then‐new and elegant ‘infomax’ algorithm (Bell and Sejnowski, 1995) thatfollowedinsights,afewyearsearlier,thatweightedsumsofindependentsourcesignalsshouldbe separable ‘blindly’ into the individual source signalswithout advance knowledge of thenature of the source processes, as had been thought necessary (Jutten and Herault, 1991;Comon,1994).Theprototypicalexampleofthisproblemisthe‘cocktailpartyproblem’inwhichanarrayofmicrophonesrecordsmixturesofthevoicesofseveralpeopletalkingatonceatacocktailparty.Individually,therecordingssoundlikeindecipherable‘cocktailpartynoise.’Theblindsourceseparationproblemistodeterminehowtocombinetherecordedsignalssoastoseparate out each speaker’s voice, “blind” to any knowledge of the nature or properties ofindividualsources.

Atroot,theinsightallowingthesolutiontothisproblemisthattheindividualspeakers’voices are the only sources of independent information in the recorded data. By adaptingrandomlyweightedsumsoftherecordedsignalsinsuchawayastomakestheweighted‐sumsignalsmoreandmoretemporallyindependentofeachother,theunmixingprocessmustfinallyarriveatproducingtheindividualvoicesignals.Insignalprocessingterms,thejointmicrophonedataisseparatedintoitsmaximallyindependentsignalcomponents,whichmustbetheoriginalvoicesourcessincetheyaretheonlypossibleindependentsourcesintherecordedmixtures.

When the process proceedswithout relying on any knowledgeof the qualities of the

individual source signals (forexample,whether thevoicesaremaleor female), theunmixingprocessiscalledblindsourceseparation.Usingtemporalindependencetoseparatethesourcesignalsisaformofblindsourceseparationcalledindependentcomponentanalysis(ICA).

Readingabout the ICAsolution to thecocktailproblem in the influentialpaperofBell

andSejnowskibeforeitspublicationin1995,thefirstauthorsuspectedthatthesamemethod

19

should be applicable to EEGdata. The results of the first EEGdecomposition (Makeig, 1996)werehighlypromising,andsubsequentworkoverthenextdozenyearsormorehasconfirmedthe ability of ICA to identify both temporally and functionally independent source signals inmulti‐channelEEGorotherelectrophysiologicaldata.ICAineffectcreatesasetofspatialfiltersthat cancel out all but a single source signal. It can be thought of as a method of datainformation‐drivenbeamformingthatfocusesspatialfiltersonphysicalsourcesofEEGsignals–separatingoutdistinctbraingeneratorprocessesaswellasnon‐brain(artifact)signals.

Figure3.4.SchematicflowchartforIndependent Component Analysis(ICA) data decomposition andback‐projection. ICA applied to amatrix of EEG scalp data (uppermiddle) findsan ‘unmixing’matrixof weights (W, upper left) that,when multiplied by the (channelsby timepoints) Scalp datamatrix,gives a matrix of independentcomponent (IC) activities oractivations(lowerright).Thisisthe

process of ICA decomposition (downward arrow) of the data into maximally temporallyindependentprocesses, eachwith itsdistinct time seriesand scalpmap.Theprocessofback‐projection (upwardarrow) recaptures theoriginal scalpdatabymultiplying the ICactivationsmatrix (lower right) by thematrix of independent component (IC) scalpmaps (lower center)whosecolumnsgivetherelativeprojectionweightsfromeachcomponenttoeachscalpchannel.TheICscalpmapor‘mixing’matrix(W‐1,lowercenter)istheinverseofthe‘mixingmatrix’(W,upper left). In simple matrix algebra form, if the indicated scalp data matrix is X and thecomponentactivationsmatrixU,thenalgebraicallyWX=UandX=W‐1U.Here,Wisamatrixofspatialfilters learnedbyICAfromtheEEGscalpdatathat,whenappliedtothedatafindstheactivityprojectionsoftheunderlyingEEGsourceprocesses,andtheICactivations(lowerright).This general schematic holds for all “complete” linear decomposition methods returning asmanycomponentsastherearedatachannels.

Moreformally,alineardecompositionofa(channelsbytimepoints)signalmatrixisitsrepresentationasanyweightedsumofcomponentsignalmatricesofthesamesize(thesamenumbers of channels and time points). Figure 3.4 schematically visualizes the simplematrixalgebraic formulation of the linear signal decomposition used in ICA. The scalp data channelsignalsareformedintoamatrix(topcenter).ICAdecompositionfindsanunmixingmatrix(W)which,whenmultipliedbythedatamatrix,decomposesthedata(downwardspointingarrow)intoamatrixof independentcomponent (IC)signalscalledthe ICactivations (lowerright),ofthesamesizeastheinputscalpdata.MultiplyingtheICactivationsmatrixbytheinverseoftheunmixing matrix W (lower middle) reconstitutes or back‐projects the original scalp datachannels(upwardspointingarrow).

20

Theinverseoftheunmixingmatrix,W‐1,isthecomponentmixingmatrix(lowercenter)whosecolumnsgivetherelativestrengthsandpolaritiesoftheprojectionsofonecomponentsource signal to each of the scalp channels. In the figure, the values in the columns of themixingmatrixarecolorcodedandinterpolatedontocartoonheadstovisualizethetopographicprojectionpatternsorscalpmapsassociatedwitheachofthesources.

ThecomponentscalpmapsfoundbyICAdecompositionarenotconstrainedtohaveany

particularrelationshiptoeachother(unlikeinPCAdecomposition).Theymaybehighly(thoughnotperfectly)correlated.Theymayalsohaveany(simpleorcomplex)spatialpattern,althoughin practice scalp maps for components truly accounting for a distinct source process(contributing independent temporal information to the data) must reflect the relativeprojectionsofthesourceprocesstotheindividualscalpchannels.Thus,asourcecomprisedofspatiallycoherentlocalfieldactivityacrossacorticalpatchmusthaveascalpmapthatmatchesthatofasingletinybattery(dipole)placedin“theelectricalcenterofmass”(asitwere)ofthesourcepatchandcalleditsequivalentdipole(Scherg,1990).However,ICscalpmapsmay,againhaveanyform,dependingonthepatternofthesourceprojectiontothescalpelectrodes,andonthedegreeofdominanceofa(maximally)independentcomponentbyasinglesignalsource.Whatisanindependentcomponent?Beforegoingfurther,wemustfirstdiscussabasicterminologicalconfound. ICA(likePCAandother linear decompositions) uses the term ‘component’ tomean something quite differentthanitsuseelsewhereinthisvolume(includingitstitle),i.e.asacontractionoftheterm“ERPcomponentfeaure”–someidentifiablefeatureinanERPwaveform(typicallyassociatedwithasinglepeak).Bybroaderdefinition,anERPcomponentmaybeanyfunctionallydistinctfeatureor portion of an ERP waveform, i.e. a feature with a functionally distinct relationship toexperimental parameters, and/or an ERP feature generated in a particular brain region (seeChapter1,thisvolume).

In this chapter,however,wewill use the termcomponentprocess (orcomponent for

short)tomeansomeportionofanentiremulti‐channelrecordeddatasetseparatedfromtheremainingrecordeddatabylineardecomposition.Tominimizeconfusion,wewillsubstituteaterminological equivalent, ‘ERP peak’ or ‘ERP peak feature,’ for the more usual term ‘ERPcomponent.’ Thus again, in this chapter components will not refer to ERP peaks or otherfeaturesbuttoEEGsourceprocesses,eachaccountingforsomeportionofthecontinuousEEGactivity(atalltimepoints)formingamulti‐channeldataset.Eachdatasetcomponentnaturallythenalsoaccountsforsomeportion(large,small,ornegligible)ofanyERPaverageofepochsdrawnfromthedataset.

AsshowninFig.3.4,anindependentcomponent(process)ofanEEGdataset(orICfor

short) comprises both a fixed scalp map and a time series giving its relative polarity andamplitude (or “activation”) ateach timepoint. The scalpmap shows the relativeweightsorprojection strengths (and polarities) of the projection from the component process to eachelectrode location. The component activation time series gives the relative amplitude and

21

polarityof thecomponent’sactivityateach timepoint.BecausewedefineanEEGsourceasbeingspatiallystable,acomponentscalpmapremainsconstantovertime.Theback‐projectionofeachcomponentprocesstoeachscalpchannel istheproductofthecomponentactivationtime series with the scalp map weight for that channel. The IC back‐projection to all thechannels is the portion or component of the scalp data (at all channels) contributed by thecomponentprocess.Theoriginalchannelsignalsarethesumsoftheback‐projectedactivitiesofall the independentcomponents.That is, thescalpdataare thecollectionofall thesummedback‐projectionsofalltheindependentcomponentstoallthechannels.

In simple matrix algebra form, if the scalp data matrix is X and the componentactivationsmatrixU,thenalgebraicallyWX=U,whereWistheunmixingmatrixofspatialfilterslearnedby ICAdecompositionof theEEG scalpdata. This equation simply says that applyingspatialfiltersWtothedataX(bysimplematrixmultiplication)givestheactivationtimecoursesof the independent componentprocesses. The converseprocess that reconstitutes thedatafrom the components is, algebraically, X =W‐1UwhereW‐1 (thematrix inverse ofW) is thecomponent mixing matrix. These same equations can be used to represent any lineardecompositionmethod,thoughothermethodsmayusedifferentnamesforthematrices.IndependentcomponentsofEEGdataItisimportanttounderstandthateachscalpEEGrecordingchannelisitselfineffectaspatiallyfilteredmeasureofthevaryingscalppotential field,recordingonlythetime‐varyingpotentialdifference between two scalp electrodes, the so‐called “active” electrode and one or more“reference”electrodes–atwhichbrainandnon‐brainsignalsarepotentiallyjustas“active”asthe so‐called “active” electrode. ICA attempts to replace these scalp channel electrode‐differencefilterswithICfiltersusingotherlinearelectrodecombinationschosensoastopassindividualEEGsourcesignalswhilerejectingallothersources.ThedegreeofsourcefidelityICAcanachievedependsonthenumberofdatachannelsversusthenumberofactivesources–aswellasonthelengthandqualityofthedata.

Before considering in detail the assumptions underlying ICA and giving heuristicguidelines for how to apply it, let us first show model examples of independent EEGcomponentsor ICs. ICsofEEGdatacanberoughlyseparatedintothreetypes, ICsaccountingfor brain and non‐brain (artifact) processes, respectively, and small ICs whose maps andactivities appear noisy and are poorly if at all replicated from session to session. This lastcategorycanbeconsidereda‘noisy’partoftheEEGsignalsthatICAisnotabletoresolveintocomponents dominated by a single source (althoughnot every small IC fits this description).BetweenthesethreeICcategories,theremaybeICsin‘greyareas’whoseassignmenttooneofthese threecategories isdifficult.Here, letus first consider ICsclearlyaccounting foractivityfromparticularnon‐brain(artifact)sources.

22

Figure 3.5. Typicalcomponent properties offour non‐brain independentcomponent processesaccounting respectively foreye blinks, lateral eyemovements, left post‐auricular electromyographic(EMG) activity, andelectrocardiographic (ECG)activity, from the 238‐channel EEG recordingstudied in Figures 3.2 and3.3. Upper panels show theinterpolated componentscalp map, activity ERPimage, average ERP (belowERP image), and meanpower spectrum. Lowerpanel shows the maximallyindependentactivitiesofthefourprocessesduringafive‐second period. Note thecharacteristic activityelements, also seen in theERP‐imagerepresenta‐tions.

Independentnon‐braincomponentprocesses:Noiseorsignals?ICAcharacteristicallyseparatesseveralimportantclassesofnon‐brainEEGartifactactivityfromtherestoftheEEGsignalintoseparatesourcesincludingeyeblinks,eyemovementpotentials,electromyographic(EMG)andelectrocardiographic(ECG)signals,linenoise,andsingle‐channelnoise (Jungetal., 2000b; Jungetal., 2000a). This importantbenefitof ICAdecompositionofEEGdatawasapparent fromthe firstattempt toapply it (Makeig,1996). ICAhas thus foundinitial use in many EEG laboratories simply as a method for removing eye blinks and otherartifacts from data. For data sets heavily contaminated by eye blinks or other artifacts, forinstancedatacollected fromyoungchildren, theability toanalyzebrainactivity indata trialsincludingeyemovementartifactscanmeanthedifferencebetweenanalyzingandrejectingthesubjectdataaltogether.

Unlike regression‐based methods for artifact removal, ICA artifact separation allowsartifact subtraction (often called artifact ‘correction’) without requiring a separate (‘pure’)

23

referencechannelforeachsignal.Inpractice,regressionmethodsriskeliminatingbrainsignalsthatalsoprojecttothe(impure)referencechannel(e.g.,frontalbrainsourcesalsoprojecttoan‘electrooculographic(EOG)channel’neartheeyes).Figure3.5showsscalpmaps,spectra,andERP‐image plots (above their trial‐average ERPs) for four typical independent artifact sourcecomponents separated by ICA from the visual selective attention task session considered inearlier figures. Thehighly distinct activity features separated from thedataby ICAmake thequalitativeimplicationsoftemporalindependenceclear.Therecoveredcomponentwaveformsarethemosttemporallydistinctportionsoftherecordeddata.SeparationbyICAofnon‐brainsourceprocessesallowdetailedanalysisof the separatedsourceprocess timecourses.Note,forexample,thatthesubjectrefrainedfromblinkingforoverasecondfollowingtargetstimuluspresentations(Fig.3.5A).

When,ashere,theelectrodemontageincludesbothheadandnecksites,scalpmapsof

headmusclecomponentsexhibitacharacteristicpolarityreversalattheinsertionpointofthemuscle into the skull, with the direction of the dipole inversion follow the direction of themusclefibers.NotethescalpmapassociatedwithanEMGcomponentsignal(Fig.3.5B).NotealsotheabruptchangesinEMGactivitylevelinthecomponentERP‐imageplotneartrials180,300, and 370 (lower left), a common occurrence for scalp muscle activity recorded duringexperimentsinwhichthesubjectissittingcomfortablywhileattemptingtominimizetheirheadandeyemovements.Thesemarkedchangesinactivitylevelofthismusclewerelikelyneitherwillfully controlled nor noted by the subject. By so clearly separating non‐brain processescontributing toEEGdata, ICAallows theseactivities tobeanalyzedas concurrently recordedbiological(orother)signalsinsteadofsimplybeingrejectedasnon‐brain“artifacts.”Spatiallystereotypedversusnon‐stereotypedartifactsItisimportant,however,tounderstandthedistinctionbetweenspatiallystereotypednon‐brainsignalsources,suchaseyeblinksandscalpmuscleactivitiesthatalwaysprojectwiththesametopographicpattern to thescalpchannels,andnon‐stereotypednon‐brain signalphenomenathathavevaryingspatialscalpprojections.Consider,forexample,thecaseofanunrulysubjectwhovigorously scratcheshisorher scalp fora secondor twoduring theEEG recording.Thisquickly produces a seriesof somehundredsof EEGdatapoints (i.e., EEG scalpmaps)whosetopographicpatternsdonotmatcheachothernorappearelsewhereinthedata.Theone‐time‐onlyappearanceofeachof thesescalpmaps is ineffect temporally independentofallotherdata sources,possiblyhugely increasing thenumberof ‘temporally independent’ sources ICAneedstoseparateintoafinitenumberofcomponentactivities.Further,duringthisperiodthechanges inelectrodecontactwiththeskinmayalterthespatialpatternwithwhichtheotherbrainandnon‐brainsignalsourcesprojecttotheelectrodearray,violatingtheICAassumptionthatthesespatialprojectionpatternsarestablethroughoutthedata.

Thus, includinga stretchofdatadominatedby thisorotherspatiallynon‐stereotyped

(SNS) artifact in the data given to ICA for decomposition can only limit the success of thedecomposition at identifying physiologically distinct EEG source processes. Such SNS periodsmaybeidentifiedbyeyewhilescrollingthroughthedata,byuseofsimpleheuristics(Delorme

24

etal.,2007a),bysimilarobservationsofapreliminaryICAdecompositionofthewholedata,orevenautomaticallyduring ICAtrainingbycomputingtheprobabilityofeachdatapoint fittingthecurrentICAmodelandrejectinghighlyimprobabledatapointsfromfurthertraining.

Cases intermediate between spatially stereotyped and non‐stereotyped artifacts are

phenomena with spatially stereotyped but non‐stationary scalp patterns, for example slowblinks(OntonandMakeig,2006),ballistocardiographic(BCG)cardiacartifactsrecordedwithinahigh‐field magnetic resonance scanner (Debener et al., 2008), and slow waves in sleep(Massiminietal.,2004).Insuchcases,ICAtypicallyfindsasmallsetofmaximallyindependentcomponents that each capture one time period of the repeating activity pattern, therebyseparatingitfromothersourceactivities.

Figure 3.6. Equivalent dipolesfor six maximally independentbrain source components. Neareach IC index (ranked in orderof variance contributed to thescalp data), the residualvariance (r.v.)of theequivalentdipole model across the 238‐channel component scalp mapisindicated,basedonfittingthemeasured 3‐D electrodelocations to an individualizedthree‐shell boundary elementmethod (BEM) head model. Allthe residual variances are low(< 6%), indicating that thecomponent maps arecompatible with an origin in asingle (or, IC8, in dual bilateralcortical patches). Theequivalent dipoles (center) arelikely situated somewhat closer

to the cortical surface than the locations of the equivalent model dipoles. The five‐secondactivationperiods shown in the lowerpanel give representative (not concurrent) examplesofbursts of frontalmidline (IC21) theta (and higher frequency) activity, and posterior (IC3, IC4)alphasourceactivitiesforthreeofthecomponents.Independentbraincomponentprocesses

Using ICA solely to remove non‐brain source processes,while valuable, does not exploit thepowerofICAtoseparatetheactivitiesof individualbrainsourcesthatcontributetothescalpdata.Somemightaskthatsincenopartofthebrainactswhollyindependentlyfromtherestof

25

thebrain,howcan ICAdecompositionextractphysiologicallymeaningful component signals?TheanswertothisquestionisthatICAfindsthemaximallyindependentcomponentsforadataset,eveniftracesofdependenceremainbetweenthem.Thisdependencemightbetransient–for example the occasional strong similarity of occasional evoked activity in otherwisemaximally independent left and right lateral occipital processes produced by central visualstimuluspresentation(Makeigetal.,2002).Orthedependencemightbelimited,forexampleonlyreflectedinweaklycoherent,lowamplitude,highfrequencyactivity.

Even in cases inwhich two ICshave remantmutualdependence, i.e.when their jointactivities could be said to form "a dependent two‐dimensional subspace" of the data, ICAshould still separate the activity of this subspace from the activities of other, singleindependentcomponentprocesses.Forexample,amovingscalpartifactproducedbysloweyeblinksmightbeseparatedintotwoormoreindependentcomponents,eachaccountingforonephaseofthemovingblinkpotential.Inthiscase,thespatiotemporallyoverlappingcomponentactivitieswoulddiffer fromoneanother,notallowingtheirparsingasasingle IC.Thoughnotcompletely independentofeachother, thetimecoursesof thepartiallydependent ICsmightstillbeindependentofanyotherICtimecourseinthedataandsufficientlydifferentfromeachothertorequiremorethanoneIC.

Applied to high‐density EEG data of adequate length and quality, ICA decomposition

typicallyproducesfromonetothreedozencomponentswithlowmutualinformationandscalpmapshighlycompatiblewithanorigininasinglecorticalpatch(oroccasionallyinabilaterallysymmetricpairofcorticalpatches),asinourdefinitionofanEEGsourceinSectionI(above).Asan example, Figure 3.6 shows 3‐D scalp maps and equivalent dipole locations in anindividualizedsubjectboundaryelementmethod(BEM)headmodelforsixindependentbraincomponentsseparatedby ICA fromthesamerecordeddataas inearlier figures.Besideeachcomponentscalpmap,theresidualscalpmapvariance(r.v.)notexplainedbythebest‐fitsingle(or dual bilateral) equivalent dipole model is shown. The component scalp maps are nearlydipolar,i.e.nearlymatchingthecomputedprojectionofasingleequivalentdipole(orbilateraldipole pair), whose locations are here within an individualized boundary element method(BEM)headmodelbuiltfromasubjectMRimage(OostenveldandOostendorp,2002).

Again, thecomputed ICsingleequivalentdipole locationscannot represent thespatial

distributionofthecorticalgeneratordomains.Instead,theyrepresentthecomputedpositions(intheBEMheadmodel)ofvanishinglysmallorienteddipoleswhosescalpprojectionpatternsmatch most closely the actual IC component maps (across all electrodes). In general, anequivalentdipole for a cortical patch source is typicallydeeper in thebrain than the corticalpatchitself(Scherg,1990).Recentadvancesindistributedinversesourcelocalizationmethodssuggest that itmay soonprovepossible toestimate,using subjectMR images, thepatch (orpatches)ofsubjectcortexthatmostlikelyconstituteeachICsourcedomain.Suchagoalislikelynot reachable by the alternate strategy of first computing an ERP average and then findinginverse sourcedistributions of oneormore ERP scalpmaps, since the ERP scalpmap at anypointintimeistypicallyaweightedmixtureofcontributionsfromseveralcorticalsourceareas.

26

ICAassumptionsAs illustrated in Figures 3.5 and 3.6 (above), ICA decomposition has proven to be highlysuccessfulforstudyingEEGdata–Why?Animportantpartoftheanswermustbethatthereisan approximate fit, at least, between ICA assumptions and the physiological nature of EEGsources themselves (Makeig et al., 2004a; Onton and Makeig, 2006; Onton et al., 2006).Basically, ICA “blindly” separates the scalp data given it into component processes whosespatial and temporal properties are not known in advance, based on the following fiveassumptions:1. Thatthecomponentsourcelocations(andtherebytheirtopographicprojectionpatternsto

thescalpsensors)arefixedthroughoutthedata.2. Thattheprojectedcomponentsourceactivitiesaresummedlinearlyatthesensors.3. That there are no differential delays involved in projecting the source signals to the

differentsensors.4. Thattheprobabilitydistributionsoftheindividualcomponentsourceactivityvaluesarenot

preciselyGaussian.5. Thatthecomponentsourceactivitywaveformsare(maximally)temporallyindependentof

oneanother.The last (‘independence’) assumption (5) can be translated informally as saying that thecomponent source activity time patterns are maximally distinct from one another. Moretechnically,asetofsignalsaretemporally independent, inthesenseusedfor ICA, ifknowingtheactivity(µV)valuesofanysubsetofthesignalsatagiventimepointgivesnoclueabouttheactivity values of any subset of remaining sources at the same time point. Thus eachcomponentsourcesignalis, inaparticularsense,anindependentsourceofinformation inthedata, contributing a temporal pattern not in any way determinable from the values (at thesametimepoint)oftheothercomponentsourcesignals.

The spread of information‐based signal processing into nearly every signal processingapplicationareainthelastdecade(JuttenandKarhunen,2004)derivesprimarilyfromthebasicinterest of investigators in all research areas in identifying the sources of information thatcontribute to their multi‐dimensional data. However, the value of ICA for decomposing anysignal isdeterminedbythedegreetowhichtheICAassumptionsfitthemanner inwhichthedata are actually generated and recorded. For EEG signals, the assumptions of simplesummationat theelectrodes (2)and lackofdifferentialdelay (3)aremetprecisely.Thenon‐Gaussian distribution assumption (4) is plausible for EEG sources generated by nonlinearcorticaldynamicsaswellasfornon‐brainartifactsources includingcardiacsignals, linenoise,muscle signals, eyeblinks andeyemovements, etc. that arenot themselves sumsof smalleruncorrelatedsignals.

Asmentioned earlier, the ICA spatial source stationarity assumption (1) is consistent

with indirect evidence from fMRI and other brain imaging methods, and the independenceassumption (5) is consistentwith theverysparse long‐rangecortico‐cortical couplingand the

27

predominantly radial thalamocortical connectivity profile. However, both these assumptionshavelimitations. Inparticular,opticaldyerecordings inanimalsof local fieldpotentialsatthemillimeterandsmallerscalerevealmovingwavepatterns(Arielietal.,1995),andcomparisonof ICA solutions across a group of subjects participating in the same task suggests that thespatially stable EEG sources separated from the data by ICA depend in part on the task thesubjectisperforming(Onton,2005).Thus,furtherresearchisneededonmethodsofidentifyingspatial lability in EEG source data (Anemuller et al., 2003) and for identifying changes in thespatial distribution of the sources as subject task, strategy, or preoccupation changes (Lee,2000).However,givenahypotheticalswitchbetweentwositesofEEGsignalgenerationasthesubject alternately performs two tasks, ICA should in theory return two components eachshowingthetask‐relatedactivityonlyduringoneperformancecondition.Dual‐dipoleICprocessesFromtheviewpointofICAdecomposition,anEEGsourceisnothingmorethananindependenttime course of information in the data, whatever its scalp projection pattern. The scalpprojections (and hence, scalp maps) of ICA components are thus constrained only by theprojectionpatternsoftheactualphysicalsourcesofthedata.Cortical(orother)sourcesignalsarising in separate cortical patches may be partially or wholly synchronized if the separatepatches are physically linked by densewhitematter tracts (such as corpus callosum), or areidentically stimulated. In this case, ICA decomposition will (rightly) return a componentsumming the scalpprojectionsof the two (physical) sourcepatches. For example, a single ICtypicallyaccountsforeyeblinkartifactsfromthetwoeyes,whosesynchronizedsmallupwardmovementsduringtheblinkinduceelectricalactivityaccountedbyanequivalentdipolelocatedineacheye.Similarly,ICAmayreturnoneormorebraincomponentswhosescalpmapssumthe projections of two equivalent dipoles, usuallywith bilaterally near symmetrical locationsand scalp projections, compatiblewith patches connected by corpus callosum. Theoretically,cortical activities on either endof anydensewhitematter tractmight synchronize and theiractivities be combined into a single IC, though to us this has not yet been conclusivelydemonstrated.ItmakesnosensetosaythatICAfailstoseparate“sources”inthiscase,unlessoneforexample(re)definestheterm“EEGsource”tomeanactivity inasinglecorticalpatch.However, in practice the number of dual‐dipolar” ICs is relatively small (except formost ICsaccountingforeyemovements,thankfully).ICAambiguityDiscussionsofthepolarityandamplitudeambiguityinherentinICactivationsinsomeearlyICApapershavebeenconfusingtosomereaders.Infact,thisambiguityispresentonlywhentheICactivationsandICscalpmapsareconsideredseparately.Wemightsaythatthesignandscalingof the (back‐projected) component in thedata is split (arbitrarily)between its activationandscalpmap.Since‐1×‐1=1, invertingthesignsofbothanICactivationanditsscalpmapwillnotchangetheirproduct,theback‐projectionoftheICintotheoriginaldata,whichwillretainitsoriginalpolarity.TheseambiguitiesshouldbekeptinmindwhenexaminingorcomparingICactivationsorscalpmaps.

28

However,theµVscalingoftheback‐projectedICscalpactivityispreciselytheproductofthescalpmapvalueswiththeactivationtimeseries.Thus,ICAdecompositiondoesnotlosethisinformation, as is sometimes mistakenly suggested. Also, while IC potentials at the corticalsurfacearealsoproportionaltotheICactivation,accuratesourcelocationandelectricalheadmodelsareneededtodeterminetheactualICstrengthonorinthecortex,sincethisdependson the resistance between scalp and cortex, which in turn varies across heads and sourcelocations.

Note that, ICA does not itself sort the components into any fixed order. Thus,

decompositions of similar data, even data from the first and second halves of the samerecordingsession,arenotguaranteedtoreturnICsinthesameorder.ICsfromdifferentdatasetsneedtobecomparedwitheachotherusingoneormoremeasuresoftheirtimecoursesand/orscalpmaps,forexampletheirpowerspectraandequivalentdipolelocations.

NumberofdatachannelsHow many data channels should be used for ICA filtering to be successful? The mostcomputationally efficient and robust ICAmethods, such as infomax ICA, neither increasenordecreasethedimensionalityofthedata–theyfindthesamenumberofcomponentsasthereare data channels and are therefore called “complete” decompositionmethods5. HowmanyindependentsourcescontributetoEEGdata?Itishighlylikelythattherearealwaysmore(brainand non‐brain) sources with distinct (e.g., near‐independent) time courses and unique scalpmapsthananypossiblenumberofrecordingchannels,sincesynchronizedcorticalfieldactivitylikely occurs, at least transiently, at more than one spatial scale, and to some extentuncorrelatednoise isgeneratedateachof theelectrodes.Mostsuchsourceactivitieswillbesmalltonegligible,buttheirpresenceguaranteesthatthenumberofdegreesoffreedomoftherecordeddatawillneverbelessthanthenumberofdatachannels.

Data contributions from numbers of sources beyond the number of availablecomponentdegreesoffreedom(i.e.,beyondthenumberofdatachannels)willbemixedintosomeoralloftheresultingcomponents,therebyaddingakindof‘noise’totheresultsofthedecomposition. The noise inherent to ICA decomposition of EEG data is evidenced by theindeterminate scalpmapsof thevery smallest ICs inahigh‐dimensionaldatadecomposition,ICsthatmaynotprovestableunderrepeateddecompositionandwhosescalpmapsareoftenfar from ‘dipolar’ (i.e., resembling the projection of a single dipolar source). Because of theneed for ICA to ‘mix’ all of the EEG sources into the available number of components,decomposing data with a larger number of (clean signal) channels may be preferable whenthereareenoughdatatodecomposethem(seefollowing).Butdecomposingasmallernumberofchannelswilllikelyprovebeneficialaswell.DatarequirementsSuccessful ICA decomposition requires an adequate amount of data. We may say,metaphorically, that the independenceofmanysourcesignalscannotbe“expressed” inbrief

29

mixturesofthem.To“express”theirindependence(orlessmetaphorically,foranICAalgorithmto recognize it), a considerable amount of data is typically required. Thus, successful ICAdecomposition typically profits from being applied to a large amount of data, typically theentire collection of continuous data or extractedand then concatenated single trials fromanevent‐relatedEEG/ERPtasksession.ThemostfrequentmistakeresearchersmakeinattemptingtoapplyICAtotheirdataistoattempttoapplyICAdecompositiontotoofewdatapoints.Forhighchannelnumbers(64ormore),wesuggest it isoptimaltodecomposeanumberoftimepoints at least 20 or more times the number of channels squared. This is only a heuristicstandard, not a strict minimum. For very dense scalp arrays, this standard could require anunreasonable amount of data. For example, to decompose256‐channel data 20× 2562 timepointsat256points/secwouldrequireover80minutesofrecordingtimeandoccupynearly1.5GB,thoughbythissamestandardfor64‐channeldataa22‐minuterecordingoccupyingabout0.35GBwouldsuffice.

WearenotassureabouttheinfluenceofsamplingrateonICAdecomposition.Doublingthesamplingrateduringarecordingperiodshortenedbyhalfmightnotproduceaseffectiveadecomposition, since the higher frequencies captured in the data acquired with a highersampling rate would be small, relative to lower frequency activity, and might have lowersource‐signal‐to‐noise ratio. See Onton & Makeig (Onton and Makeig, 2006) for furtherdiscussion.

Optimallythedatashouldbefromaperiodinwhichthesubjectispredominantlyinthesamestate(forexample,awakeandattentive),andperformingthesametypeoftaskortasks.Although standard ICAmethods are theoretically able to separatedata into sources that areprincipally active at different periods in the data set, a promising newer mode of ICAdecomposition allows learningmultiple sets of independent components wherein each timepointisassociatedwithonlyonedecomposition(Palmer,2008).

Finally,itshouldbenotedthatICAisreferencefree,sinceanyre‐referencingofthedata

thatpreservesitsdimensionalitydoesnotchangeits informationcontentor itssources.Afterre‐referencing,theICscalpmapswillchangebutICactivationdynamicsandequivalentmodeldipole locationsshouldnotchangeexceptasaresultofnormalstatisticalvariability,which istypicallysmallforICswithhighly‘dipolar’scalpmaps.ICAversusPCA

Another well‐known method of linear decomposition of multi‐channel data, principalcomponentanalysis (PCA),transformsmulti‐channeldataintoasumofuncorrelatedprincipalcomponents so named because they each, in sequence, account for the most possible (or‘principal’)varianceintheremaininguncorrelated(ororthogonal)portionofthesignaldatanotaccounted forby theprecedingprincipal components.By contrast, independent components(ICs)producedbyICAhavenonaturalorder–thoughitiscommontosortthembydescendingvarianceofthe(back‐projected)scalpdatatheyeachaccountfor.Again,foreitherPCAorICAthewholescalpdataarethesumoftheindividualcomponentcontributions.Thesimplesystem

30

ofFig.3.4thusappliestoPCAaswell,thoughforPCA,W‐1iscalledtheeigenvectormatrixandWisitsinverse.Also,inPCAboththeeigenvectormatrixandtheactivationsorfactorweightsmatricesarenormalized,andaninterveningdiagonalmatrixE,theeigenvaluesmatrix,isusedtoholdtherelativescalingofthecomponents(i.e.,X=W‐1EU),whilethecolumnsofthemixingmatrixaswellastherowsoftheactivationsmatricesareeachnormalized(tohaveunityroot‐mean‐squarevalues).

The important difference between ICA and PCA is in their quite different goals orobjectives.Wemay say that PCA attempts to lump togethermaximum signal variance fromhowevermanysourcesintoasfewprincipalcomponentsaspossible,whereasICAattemptstosplit the signal into its separate information sources, regardlessof their variance.ThismakesPCAusefulforcompressingthenumberofdimensionsinthedatawhilepreservingasmuchaspossible of the data variance. However, elimination of low‐variance PCs for the purpose ofdimensionreductionmostlikelydeletesportionsofnearlyallthesourceactivities,notjustthesmaller ones. When data length is not long enough to successfully decompose all availablechannels, another possibility is to perform ICA decomposition of data from some channelsubset. The relative value of these two approaches (principal subspace versus channelsubspace)isdifficulttoevaluateinadvance.

The ‘maximum successive variance’ objective of PCA also forces both the principalcomponent activities and the scalp projections (scalp maps) to be mutually uncorrelated(orthogonal).Sincethescalpprojectionsofbrain(andnon‐brain)sourcesarerarelythemselvesorthogonal, thisproperty forces all but the first very fewprincipal component scalpmaps toresemblecheckerboardsthatcannotreasonablyrepresenttheactivityofsingleEEGsources.Ingeneral, principal component maps do not resemble the projection of a single EEG sourceunlessonesource(often,eyeblinks)ortwosourceswithnear‐orthogonalmaps(forexample,lateralandverticaleyemovements)dominatethesignalvariance.

For this reason, some ERP researchers advocate the use of post‐PCA component

rotationmethodsdevelopedforearlierfactoranalysisapproaches,suchasVarimaxorPromax(Dien et al., 2005). These may help focus the scalp maps of the very first components toemphasizeafewlargesourceactivities(suchaseyeblinkartifactsandlateraleyemovements),butbothsimulationsandactualdecompositionsshowtheirpowertoaccomplishthisformanybrain and non‐brain sources pales in comparison to ICAmethods when properly applied tosufficientdata(Makeigetal.,1999b;Makeigetal.,2002).

Independenceamongsourcewaveforms,however,isamuchstrongerassumptionthan

the simple absence of correlations between source pair signals. Substituting the strongerassumptionof independencebetweencomponentactivities insteadofrequiringthemonlytobeuncorrelatedallowsICAtoreturnindependentcomponents(ICs)havingany(non‐identical)scalpmaps.EveryICscalpmapisthenfreetorepresenttheprojectionofasinglebrainornon‐brain signal source, whereas PCA component maps are constrained to be uncorrelated andthereforemost have a “checkerboard” appearance not compatible with a single cortical (orother)sourceprojection.

31

Theoretically, exact independence is such a strict requirement that it can never be

established forEEGsignalswith finite length. ICAalgorithms, therefore,mayatbestproducecomponentswithmaximal independence by ensuring that components continuallyapproachindependence as the ICA algorithm iteratively applied to the data. The degree of ICindependenceachievedmaydifferfordifferentdatasetsandalsofordifferentICAalgorithmsapplied to the same dataset. Our discussion of the nature of brain EEG sources (Section I)implies that the more independent the recovered IC activities, the more dipolar (oroccasionally,bilateraldual‐dipolar)theICscalpmapsofbraincomponents,atrendsupportedbyrecenttests(Delorme,unpublisheddata).

Figure3.7.ERPimagesforsixmidlineand one bilateral brain independentcomponents(ICs) inthesamesessionas earlier figures (compare Fig. 3.6),withtrialssortedinthesameorderasFig.3.3,andscaledastheycontributeto the near‐vertex channel signalimaged in Fig. 3.3. There, the trialswere sorted by the amplitude of theLPC peak centered 120 ms after thebuttonpress (at latency0ms)at thenear‐vertex channel shown on thehead cartoon (middle right panel).The sum of the signals projected bythese six component processes isshown in the same panel (note

differenceincolorscale).Thesummedcontributionsoftheother232non‐artifactcomponentstothesamechannelareshowninthelowerrightpanel.ThechannelERP‐imageplot(Fig.3.3A)isthusthesumofthetwoERP‐imagepanelswithgreybackgroundsattherightofthisfigure.Separately,thecontributionstotheERPofeachoftheothercomponentssummedinthelowerrightpanelaresmallerthanthoseofthesixcomponentswhosecontributionsareshownintheotherpanels.IndependentcomponentcontributionstosingletrialsandERPsBy definition and design, independent component processes contribute nearly independenttemporal variability to sets of single‐trial epochs. Each IC represents an independent EEGprocesswhosecontinuousactivityvariationsinthesingletrialsareavailableforinspectionandanalysis.Inparticular,brain‐basedICswithnear‐dipolarscalpmapsmayeachbepresumedtoindex the near‐synchronous field activity arising in a single patch of cortical neuropile (oroccasionally, simultaneously in two bilaterally symmetric and likely tightly‐coupled corticalpatches). Examining the trial‐to‐trial variability in the IC activities relative to a set of time‐lockingeventsmayallowamoredetailedunderstandingofevent‐relatedbraindynamicsthan

32

examination of raw scalp channel data themselves, since the effects of source (and artifact)mixingbyvolumeconductionhavebeenremovedorstronglyreducedbyICA.As an example of this, Figure 3.7 shows ERP‐image plots of the activities of six independentcomponentsinthesameresponse‐lockedsingle‐trialdataasearlierfigures.IneachERPimage,ICactivity isscaled (inµV)as in itprojects tothenear‐vertex (Cz)electrode.Locationsof theequivalentICdipolesareshowninthecentralpanel.TrialsareorderedexactlyasinFig.3.3,bytheamplitudeof the LPCpeak120msafter thebuttonpress (0ms) at the vertex.Note thequite high degree of overlap of the ‘dipolar’ scalp maps of these midline components thatcontributetemporallyindependentcontributionstotherecordedEEGsignals.

The sum of the signals projected by these six component processes is shown in themiddle right (grey) panel. Note that the trial order used here, which sorts the trials by ERPamplitude at the selected channel (as in Fig. 3.3B), only partially sorts the post‐responseamplitude of each IC activation. This is shown by the uneven gradations of the post‐motorresponsepositivityinthecomponentERP‐imagepanelsandinthesumoftheircontributionsatthe same near‐vertex channel (middle right panel). This implies that contributions to theactivity fitting themeanERP template inFig.3.3B sumvarying spatial combinations of theseandotherbrainsourceprocessesinthedifferenttrials.

NotealsoinFig.3.7thedifferentpeaklatenciesoftheLPCpeakforICs2,6,and15(top

row).ThetrialorderingselectedinFig.3.3basedontheamplitudeoftheaverageERPignoresthesesingletrialandcomponentprocessdifferences.Itsortstrialsaccordingtotheamplitudeoftheaveragesummedcontributionsoftheseandotherindependentsources,ratherthanonthevaryingamplitudesandlatenciesoftheindividualsourceprocesses.

The summedand similarly smoothed (smaller) contributionsof all theother232non‐artifactcomponentstothesamechannelinthesingletrialsareshowninthelowerright(grey)panel.ThechannelERPimageinFig.3.3Aisthusthesumofthetworight(grey)panels,aswellasthesumofthetwoERP‐imagepanelsFig.3.3BandC.Neitherofthe‘remainder’ERPimages(Fig.3.3CorFig.3.7lowerright)suggestasatisfactorymodelingoftheLPCassummingjusttwofactors–i.e.neitherthe‘six‐ICs’modelimagedinFig.3.7(lowerright)northe‘invariant‐ERP’model in Fig. 3.3 (B and C). Using the ICAmodel, however, wemay examine, for example,whether the particular trial‐to‐trial differences in the LPCwindow of each identified ICmayindicate that its response varieswith somedimensionof the varying trial context (e.g., eachtrial’sparticularcognitiveandbehavioraldemandsanddemandhistory).

Figure3.8showsthataportionoftheICtrial‐to‐trialvariabilityhighlightedinFig.3.7is

indeed linked inorderlywaystobehavioraltrialdifferences. ItshowsERP‐imageplotsforthesame six independent components (ICs) as in Fig. 3.7, again scaled as they contribute to thecentralscalpchannel(centerright)butheresortedbysubjectreactiontimeandthensmoothedwith awider (50‐trial)movingwindow tomore clearly visualize trends. Note that thewideraveragingwindowreducestheoverallamplitudeoftheimageddatathroughphasecancellation

33

of trial‐to‐trial ICvariability inneighboring trials (compare thecolorµVscale limitsherewiththoseinFig.3.7).

The middle right panel shows the summed contribution of the six ICs to the whole

channel signal, and the lower right panel, again the remainder of thewhole channel signalstheydonotaccountfor.ThisviewrevealsthataportionofthetrialvariabilityevidencedinFig.3.7istightlylinkedtodifferencesinsubjectreactiontime.ForanteriorsourcesICs2and8,thenegativityprecedingthebuttonpress istime‐lockedtostimulusonsets,whilethesubsequentLPCismainlytime‐lockedtothesubjectbuttonpress.ForcentralmidlinesourcesIC3andIC4,thenegativityonsetandoffsetaretime‐lockedtothestimulusandbuttonpressrespectively.PosteriorsourcesIC3andIC4appeartoexhibitpartialphaseresettingoftheiralphaactivitiesfollowingstimuluspresentations(seeSectionIV).ThesinglescalpchanneldataandERPsumallthese(anddoubtlessother)event‐relatedsourceprocesscontributions.

Figure 3.8. ERP images for thesame six independentcomponents (ICs) as in Fig. 3.7,againscaledastheycontributetothe central scalp channel (centerright cartoon head) but heresorted by subject reaction timeand then smoothed with abroader(50‐trial)movingwindow(note color scales). The middleright panel shows the summedcontribution of the six ICs to thewhole channel signal,withagain(lower right) the differencebetween the whole signals andthe sum of these six sourcecontributions. This view reveals

thataportionofthetrialvariabilityevidencedinFig.3.7istightlylinkedtodifferencesinsubjectreactiontime.ForanteriorsourcesICs2and8,thenegativityprecedingthebuttonpressistime‐lockedtostimulusonsets,whilethesubsequentLPCismainlytime‐lockedtothesubjectbuttonpress.ForcentralmidlinesourcesIC3andIC4,thenegativityonsetandoffsetaretime‐lockedtothe stimulus and button press respectively. Posterior sources IC3 and IC4 appear to exhibitpartial phase resetting of their alpha activities by stimulus presentations. The single scalpchannelsignalsumsallthese(andother)event‐relatedsourcedynamics.Itmaybeworththereader’seffort toexamineagaincarefully thetrialvariability indifferentdimensionsvisualizedforscalpchanneldatainFigs.3.2and3.3,andforICdatainFigs.3.7and3.8.

34

IndependentcomponentclusteringTocompare,group,orfurtheraverageERPsacrosssubjectsand/orsessions,channeldataaretypically identified by the labeled (Cz, Pz, etc.) ormeasured (x,y,z) channel positions on thescalp.Thoughequatingofequivalentscalp locationsacrosssessionsandsubjects isadequatefor many purposes, it ignores the variety of individual cortical configuration differences,particularly in the positions and orientations of cortical sulci that may orient anatomicallyequivalent EEG source projections toward different scalp areas in different subjects. In thiscase, functionally equivalent sources may have quite different scalp maps, and thereforeelectrodesatanalogous locationswill recorddifferentweightedmixturesof sourceactivities.Thus,forexample,signalsfrom‘myCz’and‘yourCz’maynotbeequivalent,evenifourbrainshaveequivalent cortical areas that function identically. Thisproducesunavoidable and rarelyconsideredvariabilityinscalprecordingsthatarecomparedoraveragedacrosssubjects.

Figure 3.9. Equivalent modeldipole locations, mean scalpmaps, and cluster projectionenvelopes of four (of 22)independentcomponentclustercontributions to a visualstimulusERP(grandmeanover12 subjects) in a visualattention‐shift experiment. ERPenvelopes show only the mostpositive and most negativechannel values at eachresponse latency. Here, theenvelope (see text) of thegrand‐average back‐projectionof the indicated IC cluster iscolor‐filled. The outer blacktraces are the envelope of thewhole grand‐mean ERP afterremoving contributions ofcomponent clusters and outliercomponentsaccountingforeye,muscle, and other non‐brain

artifacts.ThebottomtwopanelsshowclustersaccountingformostoftheP1peakinthegrand‐meanERP.Theuppertwopanelsindicatetheportionsofthegrand‐meanERPaccountedforbya central posterior cluster (blue, with maximal contribution to the peak labeled P2) and amidlinecluster(red,withmaximalcontributiontothelaterpeaklabeledP3).

Since under favorable circumstances ICA can separate scalp‐recorded signals into thevolume‐conductedactivitiesofmaximallyindependentbrainsources,itmaybemoreaccurate

35

to group, compare, and characterize functionally equivalent clusters of ICs across subjectsand/or sessions. Finding these IC equivalence classes is the challenge of IC clustering acrosssubjects and/or sessions. IC clustersmay be selected on the basis of their equivalent dipolelocations,ERPs,and/orothermeasures.6

Figure 3.9 (above) shows a sample application of IC clustering to a grandmean ERPaveraging data from 12 subjects who participated in a visual attention‐shift experiment.Throughout the experiment, subjectsmade speededmanual choice responses to indicate inwhichdimension(shapeorcolor)thelateraltargetstimulus(presentedat0ms)differedfromasimultaneously presented neutral background stimulus. In the 12 subjects’ ICA‐decomposeddata, we identified 22 clusters of similarly located and similarly reacting ICs by comparingequivalent dipole locations, mean power spectra and event‐related spectral perturbations(ERSPs, seeSection IV) in three stimulus conditions. Figure3.9 focusesonagrand‐meanERPtimelockedtostimuluspresentation(atlatency0s)inonecondition.ThecentralpanelshowsICequivalentdipolelocationsforfourof22identifiedICclusters.

Theblacktracesinthefourtopandbottomplotpanelsshowtheenvelopeofthegrand

meanERP(i.e., itsmaximumandminimumchannelvaluesateach latency).Thefourtopandbottompanelsshowthecluster‐meanscalpmapsandtheboundariesof thecoloredregions,the envelopes of those portions of the grand‐mean ERP accounted for by each of the fourclusters.7EnvelopeplottingallowstheERPcontributionsofoneormoreICsorICclusterstobevisuallycomparedwiththeenvelopeofthewholescalpERP.

The lower panels show two lateral occipital IC clusters (see the green and purple IC

dipoles)thataccountedfornearlyallthebilateralpositivepeaknear110msintheERP,plusalatersustained“ridge‐like”feature.TheuppertwopanelsshowtheportionsofthegrandmeanERPaccountedforbyacentralposteriorcluster(blue)whosemaximumERPcontributionwastothepositivepeaknear220ms,andamidlinecluster (red)thatcontributedmaximallytoalaterpositivepeaknear350ms.

Notethatalthoughthemodeldipolesarerepresented,forvisualconvenience,assmall

balls,theactualuncertaintyintheirindividuallocationsisratherlarger,asarethedistributionsofcorticalterritoryacrosswhichsynchronizedlocalfieldactivity(inourmodel)producethefar‐field potentials recorded by the scalp electrodes. Sources of dipole location error in Fig. 3.9include possible differences in recorded electrode positions relative to each other and thescalp,errorsinco‐registeringtheelectrodestotheheadmodel,anddifferencesinheadshape,andpossibledifferencesinheadtissueconductivityparameters.Althoughtheequivalentmodeldipolelocationsshowninthemiddlepanelarerelativelytightlygrouped,theirspreadmayalsoreflectdifferencesinthelocationsoffunctionallyequivalentcorticalareasacrosssubjects,sincesimilarities between activity measures were also considered in assigning components toclusters. ICclusteringisrequiredtocompareICAdecompositionsfrommorethanonesubjectorsession. Itcanbeusedtounderstandthe locationsanddynamicsof independentcomponent

36

processescontributingtoaverageERPsaswellastotheunaveragedsingletrials.ICclusteringprovidesaninvolvedbutunderfavorablecircumstances,webelieveamoreadequateanswerto the inverse problem of estimating the distributed sources of ERP scalp maps and therelationship of the source dynamics to experimental events and conditions. In particular, ICclustering gives a more adequate solution than simply attempting to model the distributedcorticalsourcesofERPscalpmapsthemselves. ICclusteringalsoallowstestingfordifferenceswithin and/or between subject groups reflected in the presence or absence of ICs in one ormoreclustersand/orondetailsoftheclusteredIClocationsoractivities.

IV.Time/frequencyanalysisofevent‐relatedEEGdataTime‐lockedbutnotphase‐locked:Event‐relatedspectralperturbations(ERSPs)TounderstandtherelationshipofERPfeaturestotheevent‐relateddynamicsoftheentireEEGsignalsfromwhichtheyarederived,itisconvenienttousetime/frequencyanalysisthatmodelsthesingle‐trialdataassumminganever‐changingcollectionofsinusoidalburstsacrossawidefrequencyrange.NotethatproducingthisrepresentationofthedatadoesnotmeanthattheEEGisnecessarilycomposedofsuchbursts,orthattheburstshapeorwindowemployedintheanalysis is necessarily a physiologically accurate template. Rather, as Joseph Fourier firstshowed for heat flows along a copper tube, frequency analysis, and later non‐stationarytime/frequencyanalysis,canbeusedtorepresentanytemporalactivitypattern,notlimitedtothose portions of the recorded signals that do indeed resemble single time/frequency basiselements, e.g. symmetric and smoothly tapered bursts at a single frequency. However, thefrequentappearanceofperiodicitiesatmultiplefrequenciesisaclearandremarkablefeatureofEEGrecordsandthispropertyofthesignalsshowquiteclearandspatiallydistinctchangesaccompanychangesinarousalandattention,makingtime/frequencyanalysisclearlyusefulforEEGanalysis.

Ratherthanaveragingtherecorded(‘time‐domain’)event‐relateddataepochsdirectly,onemayaveragetheirtime/frequencytransforms(seealsoChapter2,thisvolume).Averagingtime/frequencypowerorlogpowervaluesinaregulargridoftime/frequencywindowsgivesanevent‐related spectrogram that is nearly always dominated by relatively large low‐frequencyactivities.Normalizingtheresult,therefore,bysubtractingthemeanlogpowerspectrumwithinsomedefined‘baseline’period(pre‐stimulusorotherwise,asrelevanttotheanalysis)allowsacolor‐coded time/frequency imageofmean log spectraldifferenceswe call theevent‐relatedspectralperturbation (ERSP) image (Makeig,1993).Basing theERSPon changes in logpowerimplicitly assumes a multiplicative model by which EEG spectral changes represent themultiplication or division of the baseline power at each frequency in each latency windowrelativetothetime‐lockingevents.

37

Figure 3.10. Event‐relatedtime/frequencyanalysisofthesetofindependentcomponenttrialsshowninFig.3.7and3.8(upper left). Mean event‐related spectral perturbation(ERSP, top) and inter‐trialcoherence (ITC, bottom)time/frequency images for aleftward‐pointing mid‐frontalindependent component (IC2)time‐locked to button pressesfollowing target stimuli.Regions of non‐significantdifference from baseline (p<

.001,uncorrected)aremaskedwithlightgreen.Thetop(ERSP)imagerevealsthatmeanalphabandpowerat10Hzincreasesweaklyfollowingthebuttonpress.Onaverage,low‐betaactivity(15‐20Hz)activitiesfirstdecreaseslightly, then increaseafter400ms.Activityatthebaselinespectralpeak(6Hz,seetopleftbaselinespectrumplot)doesnotchange,thoughactivitybelow5Hzismaximalatthebuttonpress.Thebottom(ITC)imageshowsthat4‐Hzactivitybecomespartially but significantly phase‐locked around the button press, meaning the portion of thecomponentERP(lowertrace)near4Hzisstatisticallysignificant(comparetheERPtracebelow),asareitsweak10‐Hz“scalloping”between‐50and200ms.Componentactivationunits(‘act.’)areproportionaltoscalpµV.Thestatisticallysignificantchangesinmeanspectralpowerinthebeta band, shown in the upper panel, are not associated with significant ERP features andtherefore represent changes in component activity time‐aligned but not phase‐aligned to thebuttonpresses.

Determiningeithertheamplitudeorthephaseofactivityataparticulartime/frequencypoint involves matching the data in a window surrounding the given time point to theoscillatory basis element (typically a tapered sinusoidal burst or ‘wavelet’). Tomeasure lowfrequencies, thiswindowmustberelatively long, limiting the frequencyrangeconsidered forshort data epochs. Also, event‐related changes in spectral power may last longer thansignificantfeaturesintheERP.Forthesereasons,ourowntypicaltime/frequencyanalysesuseepochs including at least 1 s before the time‐locking event and continuing to 2 s or morefollowing it, allowing a frequency decomposition based on a three‐cycle tapered sinusoidalwaveletdownto3Hz.

ThemeanERSPofasetofevent‐relateddataepochscanindexevent‐relateddynamics

thatleavenotraceatallintheERPaverageofthesameepochs,asfirstshownforalphabandactivity by Pfurtscheller and Aranibar (Pfurtscheller and Aranibar, 1977). Thus the ERSPtransformoftheaverageERPforasetofdataepochs,whileofpossible interesttocompute,maybear littleornoresemblancetotheaverageERSPforthesamecollectionofepochs.Forone,significantERSPfeaturesmay longoutlast thereliableERPfeatures.Forexample,Figure

38

3.10 (top panel) shows a mean event‐related spectral perturbation (ERSP) time/frequencyimagefora left‐frontal independentcomponent(IC2)time‐lockedtobuttonpressesfollowingtargetstimuli(fromthesamesessionasFig.3.1‐3.8).Regionsofnon‐significantdifferencefrombaseline(herep<.001,uncorrectedformultiplecomparison)aremaskedwithlightgreen.TheERSPimagerevealsthatmeanalphabandpowerjustbelow10Hz increasesweaklyfollowingthe button press, whilemean low‐beta activity (15‐20 Hz) in two frequency ranges increaseaftermostmarkedlyafter400ms.Activityatthe6‐Hzbaselinespectralpeak(seetopleftside‐facingbluebaselinespectrum)doesnotchange, thoughactivitybelow5Hz increasesweaklyaroundthebuttonpress,andthendecreasesbeginning200msafterthebuttonpress.

SpectralpowerintheaverageERPisoftenreferredtoasthespectrumofactivityevokedbyevents,whilechangesinspectralpowerappearingintheERSParedubbedchangesinducedby events. However, this terminological distinction should not suggest that the two arenecessarilyphysiologicallydistinct.Toseethis,weneedtoconsiderchangesinphasestatisticsassociatedwithexperimentalevents.

Phase‐lockingacrosstrials:inter‐trialcoherence(ITC)TheERSPdisregardscompletelytheconsistencyorinconsistencyofthephaseoftheactivityateachfrequencyandlatencyinasetofevent‐relatedepochs.Inter‐trialcoherence(ITC),ormoreprecisely, inter‐trialphasecoherence,introducedas‘phase‐lockingfactor’byTallon‐Baudryetal.(Tallon‐Baudryetal.,1996),measuresthedegreeofconsistency,acrosstrials,ofthephaseof the best‐fitting time/frequency basis element at each latency/frequency point. Phaseconsistency ismeasuredonascale from0 (noconsistency,phaseacross trials is randomanduniformaroundthephasecircle)to1(phaseperfectlyconsistentacrosstrials).TheITCforanyfinitesetofrandomly‐selecteddataepochswilltypicallynotbe0.Therefore,itisimportanttocomputeabaselinethresholdfortheappearanceofsignificantlynon‐randomphasecoherence.AnITCreliabilitythresholdforasetoftrialdatacanbefoundusingeitherparametricornon‐parametricstatisticalmethods(Mardia,1972;DelormeandMakeig,2004).

ItisimportanttonotethattheITCandERSPimagesforagivensetofevent‐lockeddataepochsmayhavefeworevennocommonfeatures.Forexample, inFig.3.10thepost‐motorresponseincreasesinalphaandtheninbeta‐bandpowerinthefrontalmidlineICspectrumarenotmirroredbysignificantchangesinITCatthesamelatenciesandfrequencies.

However,there isan intimaterelationshipbetweenthe ITCandtheERP. Inparticular,

theoccurrenceofasignificantERPpeakorotherfeaturerequiressignificantITC(seeSectionII).Inthissense,asignificantERPvalueatanytimepointreflectsandrequiressignificantITCvaluesatoneormorefrequenciesatthattimepoint(exceptinodd,improbablecases).NotealsothattheITCforanyfrequencymaybesignificantevenatlatenciesatwhichthemeanpotentialERPvalue is0.Forexample, if the0value in theERPoccursduring thezerocrossingofanalphaoscillation in each trial, then the ITC at that alpha frequency might be highly significant,althoughtheERPatthattimepointmighthaveavalueof0.TheITCmayalsobesignificant,ata

39

particularlatency,atmorethanonefrequency.Ifso,thiswillbereflectedintheshapeoftheERPwaveformsurroundingthelatencyinquestion.

Forexample, intheITCimageinFigure3.10(lowerpanel),activitynear4Hzbecomes

partially but significantly phase‐locked around the button press event (0 ms), meaning theportion of the component ERP (bottom trace) at 4 Hz is statistically significant. As well, ITCbecomes(barely)significantat10Hz,afactreflectedintheweak10‐Hz‘scalloping’intheERPwaveformbetween‐200and+300ms.Noother ITC frequencies,andthereforenootherERPfrequencies,aresignificantlydifferentfromchance.ThestatisticallysignificantERSPchangesinmean spectral power in the beta and low‐gammabands, shown in the upper panel, are notassociated with significant ITC features and therefore represent component activity that isphase inconsistent, i.e. not phase‐aligned (or phase‐coherent) across trials, and so does notcontributesignificantlytothetrial‐averageERP.

ERPsandpartialphaseresetting

The relatively low peak ITC values in Fig. 3.10 (~0.4) are not unusual for longer‐latency ERPfeatures.Analternatemodel,firstproposedforselectedevent‐relateddataasearlyas1974bySayers (Sayersetal.,1974), isknownasthephase‐resettingorpartialphase‐resettingmodel.Phase resetting refers toaphenomenon seenboth inmathematicalmodelsand inbiologicalsystems inwhich the phase of an ongoing periodicity (e.g., the cardiac or circadian cycle) isresettoafixedvaluerelativetothedeliveredperturbingstimulus.Forexample,briefexposureto strong lightdelivered toadark‐adapted rat (orhuman) at almost anyphaseof thewake‐sleepcycle,will tend to reset thecycle toa fixedphasevalue (Winfree,1980;Czeisleretal.,1986;Honmaetal.,1987;Tass,1999).Atthefrequencyoftheongoing,spontaneousrhythm,anITCmeasuretime‐lockedtocomparableeventsdeliveredatrandomtimepointsthroughoutthesessionwillbecomesignificantasthephaseoftherhythminsomeormostofthetrialsisresettoafixedvalue.Ifthephaseoftherhythmicactivitythentendstocontinuetoadvanceinaregularmannerfromits initialresetvalue,theITCtime‐lockedtotheeventsof interestwillremain significant for some number of cycles until, across trials, natural variability randomlyseparatestheadvancingphasevalues.

The term ‘phase resetting’ has been applied to EEG dynamics in a less formal sense,since inmostcasesthere isnoconstant,ongoingrhythmforexperimentaleventstoperturb.Rather,inmanycasesthesignalcontainsonlyintermittentburstsofalphaorotherfrequencyspindles of various lengths. The term ‘phase resetting’, therefore, can be formally applied insomestatisticalsensetomeanthatthephasestatistics(asmeasured,forexample,bytheITC)aretransientlyperturbedfollowingeventsofinterest.If,wheneverrhythmicactivityatagivenfrequency is present, its phase distribution following the time‐locking events becomes non‐uniform,ITCwillincreaseandmaytendtoremainsignificantforaslongastherhythmicactivityispresent.

40

Figure3.11.ERPsandpartialphase resetting. The rightpanel shows an ERP‐imageplotvisualizingtheresponsesin over 400 single trials of abilateral occipitalindependent component (IC)process followingpresentationofaletteratthecentral fixation point of asubject participating in aworking memory task. Thebilateral component (IC6,sixthbyvarianceexpressedinthe data) produces aresponselargelyresemblinga

one‐cycle sinusoid at 9 Hz. The mean ERP trace (below the ERP image) plots its mean timecourse,timelockedtostimulusonset.TheERSPtrace(belowthat)showsthatthemeanlevelof9‐Hzenergyinthedata,duringthisperiod,is15dBormorehigherthaninthepre‐stimulus(orensuing)period.The ITCtrace(belowthat)confirmsthatduringthisERPfeaturethephaseofthe entire 9‐Hz activity in the trials is highly consistent (ITC approaching 1). The bluebackgroundsshowp< .01probability limits,demonstratingthatall threemeasuresarehighlysignificantlydifferentfrombaselineinthisperiod.Thepanelontheleftshowsaquitedifferentsetofover100datatrialsforamedial(ormedialbilateral)occipital ICprocessinthefive‐boxtaskofFig.3.1inwhichstimuliwerepresentedaboveandleftofacentralfixationcross,whilethe subject retained fixation. The large amount of alpha band activity produced by this ICprocessunderthesecircumstanceslikelyreflects‘alphaflooding’ofrelevantvisualcortexwhenvisual attention is forced by the task to remain elsewhere in the visual field (Worden et al.,2000).

Figure3.11showsERP‐imageplotsfortwosetsofvisual‐stimuluslockedtrialdatafrom

two ICs captured in different subjects under rather different task conditions. In panel 3.11A(left), the stimulus is a briefly‐flashed disk presented at a central, visually unattended targetsquare located above a central fixation cross during a visual selective attention task. The ICshown here has a bilateral equivalent dipole model in or near primary visual cortex, andproduces abundant alpha‐band activity (see power spectrum), likely reflecting the ‘alphaflooding’ofvisualcorticalareassensitivetothefovealfixationregionwhenthesubjectplaceshisorhervisualattentionelsewhereinthevisualfield(Wordenetal.,2000).Thisalphaactivityappears to be ‘partially phase‐reset’ (ITC ~ 0.4) for nearly 500ms (5 alpha cycles) followingstimulus presentation. In the ERP image, trials are sorted by alpha phase in a three‐cyclewindow ending 50 ms after stimulus onset. The possibility of ‘partial phase resetting’ issuggestedbythebendingandthennear‐verticalalignmentofthepositiveandnegativewavefrontsbeginningnear100msintheERPimage,whentheITCbecomessignificant.

41

Note that the visual evidence presentedby this ERP‐image, including the finding of a

significant ITC(lower ITCtrace),arenot inthemselvessufficientevidencetoprovethatthesedatatrulyfita‘phaseresetting’model(formorediscussion,seeChapter2,thisvolume).Nordotheynecessarilyruleouta‘true‐ERP’modelforthedata,e.g.amodel inwhichthesameERP(upperERPtrace) resemblinganalphaburst is simplyadded toongoingalphaandotherEEGactivity inevery trial (as inFig.3.3),and that theongoingalphaactivity is in turn reduced inamplitudejustenoughtomaketotalmeanalphapowerateachlatencyconstant,asobservedhere (middle ERSP trace). However, keep in mind that these data are the result of spatialfilteringby ICAofasingle independentsource,very likelyfocusedonasinglesourcearea(orcloselyspacedmedialbilateralareas),giventhehighly‘dipolar’fromoftheICscalpmap.Itthusseems tous physiologically implausible that, following these visual events, this same corticalsource areaproducesongoing random‐phase alpha activityplus a fixedbutwhollyunrelatedalpha‐burst ERP. Several groups have recently proposed measures to further test phaseresettingmodels ondata such as these (Mazaheri and Jensen, 2006;Hanslmayr et al., 2007;Martinez‐Montesetal.,2008).Ultimately, the issuewill likelybesettledbyfittingconcurrentscalpandintracranialEEGrecordingstogenerativemodelsofcorticalfielddynamics,aprocessbegunbygroupsstudyinghumanbrainresponsesduringcorticalmodeling(Wangetal.,2005).

Inpanel3.11B(right),ontheotherhand,thetime‐lockingstimulusisaletterpresentedat fixation in a letterworkingmemory task. The spectrumof the bilateral lateral‐occipital IC(inset)hasonlyaweakalphabandpeak,andnosignofprolongedalpha‐bandphaseresettingfollowing the highly stereotyped (ITC > 0.8) component IC stimulus‐evoked response (whichcontributes strongly to the P1‐N1‐P2 features of the full scalp ERP, not shown). At thefrequency best fitting the ERP complex (9Hz),mean single‐trial amplitude during the ERP isnearly6times(over15dB)higherthanthemeanamplitudeofactivityatthesamefrequencyinthepre‐stimulusbaseline.Phase‐sortingthesingletrialsat9Hzinawindowending50msafterstimulusonset(asinA)showsthatthephaseoftheweakalphaactivitypresentinsingletrialsduring thebaselineperiodhasnoobviouseffecton the latenciesof the subsequentevoked‐responseactivity in thesametrials.For this ICstimulus response, therefore,a ‘partialphase‐resettingmodel’ seemsunnaturalanda ‘trueERP’modeladequate.However,evenhereonemayaskwhether,forexample,thefrequencypeakoftheERP(9Hz)maynotalsobeapeakofthespontaneous(baseline)spectrumofthiscorticalarea.

Manyauthorshaveattemptedtodrawaharddistinctionbetweenevokedandinducedevent‐related activities, definingevoked activity as being activity completely time‐locked andphase‐lockedtothestimulus(ITC=1)andtherebycomposingtheERP,whiletheremainderofthesingle‐trialactivity,havingnophaselockingtothetime‐lockingevents(ITC=0)isdefinedasinduced(Galambos,1992).Whilethisdistinctionmaybeusefulforsomepurposes,drawingthisterminological distinction does not mean this decomposition of the EEG signal into evoked(ERP)activityplus induced (otherEEG)activityhasanynaturalphysiologicalbasis. Thinkof astackof fivepennies–Again,does this stack ‘really’ sumtwogroupsof twoand three,orofgroupsoffourandone?Infact,thestackofpenniesretainsnotraceofhowitwasconstructedandthuscannotbesaidtobeanymore‘really’3+2than4+1,nomatterhowitwasoriginally

42

constructed. The sameapplies to themodel of event‐relatedEEGdata illustrated in Fig. 3.3:EEG data = ERP + Other, amodel that, as ICA decomposition and Figs. 3.7 and 3.8 suggest,disregardsthevaryingsingle‐trialcontributionsofspatiallyseparabledatainformationsources,someclearlylinkedtotrial‐by‐trialbehavioraldifferences.

Figs. 3.7 and 3.8 suggest that scalp ERPs sum channel activity arising from different

mixturesofspatialsourceprocessesindifferenttrials.ButhowshouldwethinkoftheaverageresponseofasingleIC?AssumingthatanICactivationdoesindexlocally‐synchronousornear‐synchronousfieldactivityofasinglepatchofcortex,cantheICactivityproducingtheIC“ERP”activity(strictlytime‐lockedtothesetofevokingevents)bephysiologicallydistinctfromother(non phase‐locked) EEG activity originating at the same moments in presumably the samecorticalpatch?

Linear summation in cortex, even of direct sensory input and ongoing cortical dynamics,appearsphysiologicallyimplausiblewithoutstrongnonlinearinteractions.Fiserandcolleagues(Fiser et al., 2004) have noted that even at prototypical sensory cortex – the input layer ofprimaryvisualcortex(inferrets),onlyafewpercentofthesynapsesdeliverinformationdirectlyfromtheeyesviathelateralgeniculatenucleus(LGN).Inaccordwiththisfact,theyreportthat“at all ages including the mature animal, correlations in spontaneous neural firing [duringnatural vision] were only slightly modified by visual stimulation, irrespective of the sensoryinput. These results suggest that in both the developing and mature visual cortex, sensoryevokedneuralactivityrepresentsthemodulationandtriggeringofongoingcircuitdynamicsbyinputsignals,ratherthandirectlyreflectingthestructureoftheinputsignalitself”(Fiseretal.,2004). If this is the case even for V1, it should not be less so for cortical areas that are notprimarysensoryareas.Clearly,deeperunderstandingof theEEGdynamicchangesassociatedwith sensory andother eventswill requiremoredetailedobservation andmodeling of braindynamicsatmultiplespatialscales. IntermsofEEGresearch,moredetailedobservationsandmodelingareneededoftrial‐by‐trialdifferencesinoscillatoryactivityanditsrelationshiptoitstransformationbyexperimentalevents.

Event‐relatedcoherenceCognitive events –moments atwhichwe apperceive the significance of some sensory eventandmentally ‘grasp’ its immediateconsequencesforourattentionandbehavioralplanning–mustinvolveand/orproducecomplexanddistributedchangesinEEGdynamics.Furthermore,somemechanismofinformationtransferbetweenbrainregionsmustexistthatisdynamicallydependent both on the nature of the stimulus and its relation to subject expectations andintentions. A possible mechanism for this transfer may be indexed by transient temporalcoupling between pairs of sources relative to experimental events. One measure of thiscouplingisevent‐relatedcoherence(ERC)(DelormeandMakeig,2003).

The preponderance of coherence observed between scalp channel signal pairs is

accounted for by ICA as deriving from common IC projections to both scalp channels. Anamplitude change in activity of a single ICmayproduce a change in themeasured (zero‐lag)

43

scalpchannelcoherencewithoutanyactualbraincoherencechangesoccurring.Bymaximallyreducing the effects of volume conduction on the data, ICA decomposition allows a moreprincipledstudyoftransientorintermittentcoherencebetweenICsourceactivities.

Recently,weused ICAdecompositionof target responsedata fromthe same five‐boxtaskusedhereforillustrativepurposestoshowthatbriefandweaklyspatiallycoherentthetawave complexes arise in frontalmidline, somatomotor, and parietal cortex inmany subjectsfollowing significant events (Makeig et al., 2004b), often beginning in frontal polar cortex(Delorme et al., 2007b). But how can the activities of ‘independent’ components be(occasionally) phase coherent? As described previously, ICA decomposition actually derivesmaximallyindependentcomponents–thisallowsthediscoveredICactivitypatternstoexhibitoccasional transient dependence – for example, in the five‐box data at one frequency in atmostafifthofthetrials.Inthisandrelatedcases,wefoundthepartialcoherencestohavenon‐zerophase lags, and to remainwheneach component ERPwas (artificially) regressedout ofeach single trial activity and coherencewas computed only on the remainder. Event‐relatedcoherenceisanothermeasurethatcannotbededucedfromERPwaveformsalone,andcannotbeconfidentlyinterpretedwhencomputedforpairsofscalp‐channelsignals.ICApreservesonlythosecoherences that represent transientcouplingof the frequency‐domainactivitiesof twoEEGsourceswithafixedlatencydifference.

A ‘close up’ example of similar ‘phase reorganization’ in human brain was recently

provided by the study byWang and colleagues of event‐related local field activity in multi‐channel ‘thumbtack’ electrodes pushed through a small piece of intact cortex in anteriorcingulatebefore its clinically required removal inabrainoperation (Wangetal., 2005).Theyreportedthatthetabandactivitywasgeneratedinsuperficiallayersofanteriorcingulatecortex(ACC) both before and after presentations of a variety of task‐relevant stimuli, while afterpresentationsphase‐lockingbetweenACCandotherbrainareasincreasedtransiently.

V.MeetingtheChallengeoftheMomentFor us to survive and thrive, at eachmoment our brainmust integrate its awareness of itspresentsituationandenvironment,includingexistingplansforactionand/orinaction,withitsemergingsensoryexperienceandmnemonicassociations.Itmustoptimallyengageorreviseitsattentionaldistribution,actionplans,andphysiologicalbodystate inawayadequatetomeetthechallengeofthemoment.

This volume summarizes the results of nearly fifty years of scientific experience instudyingtheshapesandsizesofaverageevent‐relatedpotential(ERP)responsesofscalpEEGsignals tosensoryorotherevents, responses thatdepend in largepartonthesignificanceoftheeventstothesubjectandonthecontext inwhichtheyoccur.EEGistheoldestandmostnon‐invasive functionalbrain imagingmodality; it is also the leastexpensiveandmosthighlyportable.ThecontinuingpromiseofEEGbrainimagingisthatthehighlylabiledynamicsofEEGscalp fields, signaling changes in local field synchronywithin andbetween cortical areas, can

44

providedetailedindicesofchangesinhumanattentional,intentional,andaffectivestate,bothposthoc andeven, toan increasingextent,online,withpotentially importantapplications tobasic scientific research, to clinical andworkplacemonitoring, and to other fields of humaninterestandendeavor.

In this chapter,we have discussed the origins in local cortical synchrony of both EEG

signalsandERPwaveformsderivedfromthem.WehavedefinedtheconceptofanEEGsource,basedonbothEEGanalysisandphysiologicalevidence,andhavedemonstrated theutilityofindependentcomponentanalysis(ICA)forseparatingmulti‐channelEEGrecordingsintoasetoftemporally and functionally independent brain and non‐brain source processes. Finally, wehaveshownasimpleexampleofusingICAdecompositiontostudythesourcesthatcontributeto(aswellasthosethatcontaminate)ERPs,andtheiractivitiesinthesingle‐trialEEGdata.WehavegivenexamplesofusingERP‐imageplottingtovisualizethedependenceofEEGresponsesin single trials on behavioral, EEG, or other parameters, have introduced time/frequencyanalysisintheformofinter‐trialcoherence(ITC)toshowthattheactivitycapturedinaverageERPs reflects trial‐to‐trial phase consistency, andhave the introduced the concept that someERPfeaturesmayreflectreorganization(orperturbation)oftheexacttimingorphasestatisticsofongoingactivity inthesamecorticalareas,as longsuggestedby investigatorsfamiliarwithcontroltheoryandotherdynamicmodelingmethods.

WebelievetheincreasinglyurgentchallengeforthefieldofERPandmoregeneralEEG

research is todiscover thebrain sourcedynamics thatproduce the characteristic featuresofevoked responses and to model the trial‐by‐trial (and condition‐by‐condition) differences inEEG(andERP)dynamicsassociatedwiththe largevarietyofevents thatunfoldcontinually inourdailylives,withinanever‐evolvingsituationalcontext–eventsthatposeawidevarietyofchallengestowhichourbrainsrespondeffectively.

Webelievethistobeanexcitingtimetostudyhumanelectrophysiology,anerainwhich

non‐invasiveEEGrecordingismovingtowardfulfillingitspromiseofbecomingatruefunctionalbrain imaging modality. Current knowledge and understanding of EEG dynamics is likely toadvancesteadilyasnewanalysistoolsdevelopedforthispurposebecomemorewidelyapplied.OneresultshouldbeadeeperandfullerunderstandingofthenatureandsignificanceofERPfeatures.

LaJollaCADecember,2008

45

Endnotes1Forexample,at20Hzandtravelingat1m/s,aradiating‘pondripple’wouldreachtheedgeofa1‐cmsourcedomain in5ms,one tenthofa20‐Hzcycle.Therefore, therewouldonlybea2π/10 = 36 degree phase lag between the center and the edge of the patch, and thespatiotemporal pattern of potentials at scalp electrodeswould be highly correlatedwith thepatternproducedbycompletelysynchronous20‐Hzactivityacrossthesame1‐cmdomain.2TheseEEGdatawere collected synchronously from250 scalpplus four infra‐ocular and twoelectrocardiographic (ECG) electrodes with an active reference (Biosemi, Amsterdam) at asampling rateof256Hzand24‐bitA/Dresolution.Onsetsandoffsetsof targetdiscs,aswellsubject button presses, were recorded in a simultaneously acquired event channel. Therecordingmontagecoveredmostoftheskull,forehead,andlateralfacesurface,omittingchinandfleshycheekareas.Locationsoftheelectrodesrelativetoskulllandmarksforeachsubjectwere recorded (Polhemus, Inc.). Electrodes with grossly abnormal activity patterns wereremovedfromthedata,leaving238channels.Afterre‐referencingtodigitallylinkedmastoids,thedataweredigitally filteredtoemphasizefrequenciesabove1Hz.Dataperiodscontainingbroadlydistributed,high‐amplitudemusclenoiseandother irregularartifactswere identifiedbytestsforhighkurtosisorlow‐probabilityactivityandremovedfromanalysis.Occurrenceofeye blink, other eye movement, or isolated muscle noise artifact was not a criterion forrejection. Remaining data time points were then concatenated and submitted todecomposition by extended infomax ICA using the binica function available in the EEGLABtoolbox (http://sccn.ucsd.edu/eeglab). Decompositions used extended‐mode infomax ICA(Makeig et al., 1997)with default training parameters. Extended infomaxwas used to allowrecoveryofanycomponentswithsub‐gaussianactivitydistributions,including60‐Hzlinenoisecontamination. ICA components clearly and predominantly accounting for eye movement,muscle,cardiac,single‐channel,orotherartifactualactivitywereremovedfromtheERPdata.Both the target stimulus‐locked and motor response‐locked epochs analyzed in the figureswerereferredtoameanbaselineina500‐msperiodbeforetargetstimulusonsets.3Data figures in this chapter were produced using software tools from the freely availableEEGLABMatlabsoftwareenvironment(sccn.ucsd.edu/eeglab/).Thesingle‐subject256‐channeldatasetfromwhichwederivedmostofthefigureswasrecordedandfirststudiedbyDelormeetal.(Delormeetal.,2007b)andisavailablefordownloadinrawandinEEGLABformatsfromtheEEGLABwebsite(above).4In particular, the phase of a digitally recorded signal cannot be defined above its Nyquistfrequency(halfofitssamplingrate)andisambiguousatitsNyquistfrequency.5Methodsthatfindmorecomponentsareavailable,butrequirenarrowersourceassumptionsandmorecomputationtime.

46

6EEGLAB includes Matlab‐based tools for applying, evaluating, and exploring componentclustering.7Data used for the IC cluster figure were collected by Klaus Gramann at the University ofMunich from 12 subjects performing a visual feature discrimination task. Theelectroencephalogram(EEG)wasrecordedcontinuouslyatasamplingrateof500Hzusing64Ag/AgClelectrodesmountedonanelasticcap.EEGsignalswereamplifiedusinga0.1–100‐Hzbandpass filter and filtered off‐line using a 1–40‐Hz bandpass. All electrodes were recordedreferencedtoCzandthenre‐referencedoff‐linetolinkedmastoids.AverageERPsinan800‐msepoch were computed relative to a 200‐ms pre‐stimulus baseline. ICA decomposition usedextendedinfomax.ICswereclusteredacrosssubjectsusingEEGLABclusteringfunctionsbasedontheirrespectivedynamicsunderthreetargetstimulus‐differenceconditions(whetherornotthe target had a different color, different shape, or both than the accompanying standardstimuli).OnlyICswhoseequivalentdipoleprojectiontothescalphadaresidualvariancefromthe IC scalpmapbelow15%andanequivalentdipole locationwithin thebrainvolumewereconsideredforclustering.TheseICswereseparatedinto22clustersbasedontheirequivalentdipole locations,event‐relatedspectralperturbations (ERSPs)and inter‐trialcoherences(ITCs)inthe500msfollowingtargetonsets.

47

ReferencesAkalin‐Acar Z, Gencer NG (2004) An advanced boundary element method (BEM)implementation for the forwardproblemofelectromagneticsource imaging.PhysMedBiol49:5011‐5028.

Anemuller J, Sejnowski TJ, Makeig S (2003) Complex independent component analysis offrequency‐domainelectroencephalographicdata.NeuralNetw16:1311‐1323.

Arieli A, Shoham D, Hildesheim R, Grinvald A (1995) Coherent spatiotemporal patterns ofongoing activity revealedby real‐timeoptical imaging coupledwith single‐unit recording inthecatvisualcortex.JNeurophysiol73:2072‐2093.

Beggs JM, Plenz D (2003) Neuronal avalanches in neocortical circuits. J Neurosci 23:11167‐11177.

Bell AJ, Sejnowski TJ (1995) An information‐maximization approach to blind separation andblinddeconvolution.NeuralComput7:1129‐1159.

BullockTH(1983)Electricalsignsofactivityinassembliesofneurons:compoundfieldpotentialsasobjectsofstudyintheirownright.ActaMorpholHung31:39‐62.

ComonP (1994) IndependentComponentAnalysis,aNewConcept.SignalProcessing36:287‐314.

CzeislerCA,Allan JS, StrogatzSH,Ronda JM,SanchezR,RiosCD,FreitagWO,RichardsonGS,Kronauer RE (1986) Bright light resets the human circadian pacemaker independent of thetimingofthesleep‐wakecycle.Science233:667‐671.

DanY,PooMM(2004)Spiketiming‐dependentplasticityofneuralcircuits.Neuron44:23‐30.Debener S, Mullinger KJ, Niazy RK, Bowtell RW (2008) Properties of the ballistocardiogramartefactas revealedbyEEGrecordingsat1.5,3and7Tstaticmagnetic fieldstrength. Int JPsychophysiol67:189‐199.

Delorme A, Makeig S (2003) EEG changes accompanying learned regulation of 12‐Hz EEGactivity.IEEETransNeuralSystRehabilEng11:133‐137.

DelormeA,Makeig S (2004) EEGLAB: an open source toolbox for analysis of single‐trial EEGdynamicsincludingindependentcomponentanalysis.JNeurosciMethods134:9‐21.

DelormeA, Sejnowski T,Makeig S (2007a) Enhanceddetectionof artifacts in EEGdatausinghigher‐orderstatisticsandindependentcomponentanalysis.Neuroimage34:1443‐1449.

Delorme A, Westerfield M, Makeig S (2007b) Medial prefrontal theta bursts precede rapidmotorresponsesduringvisualselectiveattention.JNeurosci27:11949‐11959.

Dien J, Beal DJ, Berg P (2005) Optimizing principal components analysis of event‐relatedpotentials:matrixtype,factorloadingweighting,extraction,androtations.ClinNeurophysiol116:1808‐1825.

FischlB,SalatDH,vanderKouweAJ,MakrisN,SegonneF,QuinnBT,DaleAM(2004)Sequence‐independentsegmentationofmagneticresonanceimages.Neuroimage23Suppl1:S69‐84.

FiserJ,ChiuC,WelikyM(2004)Smallmodulationofongoingcorticaldynamicsbysensoryinputduringnaturalvision.Nature431:573‐578.

FoxeJJ,SchroederCE(2005)Thecaseforfeedforwardmultisensoryconvergenceduringearlycorticalprocessing.Neuroreport16:419‐423.

FrancisJT,GluckmanBJ,SchiffSJ(2003)Sensitivityofneuronstoweakelectricfields.JNeurosci23:7255‐7261.

48

FreemanWJ(2000)Mesoscopicneurodynamics: fromneurontobrain. JPhysiolParis94:303‐322.

FreemanWJ,BarrieJM(2000)AnalysisofspatialpatternsofphaseinneocorticalgammaEEGsinrabbit.JNeurophysiol84:1266‐1278.

FriesP,NikolicD,SingerW(2007)Thegammacycle.TrendsNeurosci30:309‐316.FrostDO,CavinessVS,Jr.(1980)Radialorganizationofthalamicprojectionstotheneocortexinthemouse.JCompNeurol194:369‐393.

GalambosR(1992)Acomparisonofcertaingammaband(40‐Hz)brainrhythmsincatandman.In:InducedRhythmsintheBrain(BasarE.BTH,ed),pp201‐216.Boston:Birkhauser.

GencerNG,Akalin‐AcarZ(2005)Useoftheisolatedproblemapproachformulti‐compartmentBEMmodelsofelectro‐magneticsourceimaging.PhysMedBiol50:3007‐3022.

Grave de Peralta‐Menendez R,Gonzalez‐Andino SL (1998) A critical analysis of linear inversesolutionstotheneuroelectromagneticinverseproblem.IEEETransBiomedEng45:440‐448.

GrinvaldA,LiekeEE,FrostigRD,HildesheimR (1994)Corticalpoint‐spreadfunctionand long‐rangelateralinteractionsrevealedbyreal‐timeoptical imagingofmacaquemonkeyprimaryvisualcortex.JNeurosci14:2545‐2568.

Halgren E, Marinkovic K, Chauvel P (1998) Generators of the late cognitive potentials inauditoryandvisualoddballtasks.ElectroencephalogrClinNeurophysiol106:156‐164.

HanslmayrS,KlimeschW,SausengP,GruberW,DoppelmayrM,FreunbergerR,PecherstorferT,BirbaumerN(2007)AlphaphaseresetcontributestothegenerationofERPs.CerebCortex17:1‐8.

Honma K, Honma S,Wada T (1987) Phase‐dependent shift of free‐running human circadianrhythmsinresponsetoasinglebrightlightpulse.Experientia43:1205‐1207.

JungTP,MakeigS,WesterfieldM,TownsendJ,CourchesneE,SejnowskiTJ(2000a)Removalofeyeactivityartifactsfromvisualevent‐relatedpotentialsinnormalandclinicalsubjects.ClinNeurophysiol111:1745‐1758.

JungTP,MakeigS,WesterfieldM,TownsendJ,CourchesneE,SejnowskiTJ(2001)Analysisandvisualizationofsingle‐trialevent‐relatedpotentials.HumBrainMapp14:166‐185.

Jung TP, Makeig S, Humphries C, Lee TW, McKeown MJ, Iragui V, Sejnowski TJ (2000b)Removing electroencephalographic artifacts by blind source separation. Psychophysiology37:163‐178.

Jutten C, Herault J (1991) Blind Separation of Sources .1. An Adaptive Algorithm Based onNeuromimeticArchitecture.SignalProcessing24:1‐10.

Jutten C, Karhunen J (2004) Advances in blind source separation (BSS) and independentcomponentanalysis(ICA)fornonlinearmixtures.IntJNeuralSyst14:267‐292.

Klopp J, Marinkovic K, Chauvel P, Nenov V, Halgren E (2000) Early widespread corticaldistributionofcoherentfusiformfaceselectiveactivity.HumBrainMapp11:286‐293.

LeeTW,Lewicki,M.S.(2000)ThegeneralizedGaussianmixturemodelusingICA.InternationalWorkshoponIndependentComponentAnalysis.

Linkenkaer‐Hansen K, Nikouline VV, Palva JM, Ilmoniemi RJ (2001) Long‐range temporalcorrelationsandscalingbehaviorinhumanbrainoscillations.JNeurosci21:1370‐1377.

Luck SJ (2005) An introduction to event‐related potentials and their neural origins. In: AnIntroductiontotheEvent‐RelatedPotentialTechnique,pp1‐50:TheMITPress.

49

MakeigS(1993)Auditoryevent‐relateddynamicsoftheEEGspectrumandeffectsofexposuretotones.ElectroencephalogrClinNeurophysiol86:283‐293.

MakeigS,DebenerS,OntonJ,DelormeA(2004a)Miningevent‐relatedbraindynamics.TrendsCognSci8:204‐210.

Makeig S, Jung TP, Bell AJ, Ghahremani D, Sejnowski TJ (1997) Blind separation of auditoryevent‐related brain responses into independent components. Proc Natl Acad Sci U S A94:10979‐10984.

MakeigS,WesterfieldM,TownsendJ,JungTP,CourchesneE,SejnowskiTJ(1999a)Functionallyindependentcomponentsofearlyevent‐relatedpotentials inavisual spatialattention task.PhilosTransRSocLondBBiolSci354:1135‐1144.

MakeigS,WesterfieldM,JungTP,CovingtonJ,TownsendJ,SejnowskiTJ,CourchesneE(1999b)Functionally independent components of the late positive event‐related potential duringvisualspatialattention.JNeurosci19:2665‐2680.

MakeigS,WesterfieldM, JungTP,Enghoff S,Townsend J,CourchesneE, SejnowskiTJ (2002)Dynamicbrainsourcesofvisualevokedresponses.Science295:690‐694.

MakeigS,DelormeA,WesterfieldM,JungTP,TownsendJ,CourchesneE,SejnowskiTJ(2004b)Electroencephalographic brain dynamics followingmanually responded visual targets. PLoSBiol2:e176.

Makeig S, Bell, A.J., Jung, T.P., Sejnowski, T.J. (1996) Independent component analysis ofelectroencephalographicdata.AdvancesinNeuralInformationProcessingSystems8:145‐151.

MardiaKV(1972)Statisticsofdirectionaldata.NewYork,NY:AcademicPress.Martinez‐MontesE,Cuspineda‐BravoER,El‐DeredyW,Sanchez‐BornotJM,Lage‐CastellanosA,Valdes‐SosaPA(2008)Exploringevent‐relatedbraindynamicswithtestsoncomplexvaluedtime‐frequencyrepresentations.StatMed27:2922‐2947.

Massimini M, Huber R, Ferrarelli F, Hill S, Tononi G (2004) The sleep slow oscillation as atravelingwave.JNeurosci24:6862‐6870.

MazaheriA, JensenO(2006)Posterioralphaactivity isnotphase‐resetbyvisualstimuli.ProcNatlAcadSciUSA103:2948‐2952.

MurreJM,SturdyDP(1995)Theconnectivityofthebrain:multi‐levelquantitativeanalysis.BiolCybern73:529‐545.

NunezP(1977)Thedipolelayerasamodelforscalppotentials.TITJLifeSci7:65‐72.Nunez P, Srinivasan, R. (2005) Electric fields of the brain; The neurophysics of EEG OxfordUniversityPress.

Onton J,Makeig S (2006) Information‐basedmodeling of event‐related brain dynamics. ProgBrainRes159:99‐120.

Onton J, Westerfield M, Townsend J, Makeig S (2006) Imaging human EEG dynamics usingindependentcomponentanalysis.NeurosciBiobehavRev30:808‐822.

Onton J, Makeig,S. (2005) Independent Component Analysis (ICA) source locations varyaccordingtotaskdemands.OrganizationforHumanBrainMappingAbstracts.

OostenveldR,OostendorpTF(2002)ValidatingtheboundaryelementmethodforforwardandinverseEEGcomputationsinthepresenceofaholeintheskull.HumBrainMapp17:179‐192.

PalmerJA,Makeig,S.,Kreutz‐Delgado,K.Rao,B.D.(2008)NewtonMethodfortheICAMixtureModel.ICASSP:1805‐1808.

50

PfurtschellerG,AranibarA(1977)Event‐relatedcorticaldesynchronizationdetectedbypowermeasurementsofscalpEEG.ElectroencephalogrClinNeurophysiol42:817‐826.

Radman T, Parra L, Bikson M (2006) Amplification of small electric fields by neurons;implicationsforspiketiming.ConfProcIEEEEngMedBiolSoc1:4949‐4952.

RanganathC,RainerG(2003)Neuralmechanismsfordetectingandrememberingnovelevents.NatRevNeurosci4:193‐202.

RulkovNF, Timofeev I, BazhenovM (2004)Oscillations in large‐scale cortical networks:map‐basedmodel.JComputNeurosci17:203‐223.

SayersBM,BeagleyHA,HenshallWR(1974)ThemechansimofauditoryevokedEEGresponses.Nature247:481‐483.

SchergM(1990)Fundamentalsofdipolesourcepotentialanalysis.AdvancesinAudiology6:40‐69.

StettlerDD,DasA,BennettJ,GilbertCD(2002)Lateralconnectivityandcontextualinteractionsinmacaqueprimaryvisualcortex.Neuron36:739‐750.

SwadlowHA,GusevAG (2000) The influenceof singleVB thalamocortical impulsesonbarrelcolumnsofrabbitsomatosensorycortex.JNeurophysiol83:2802‐2813.

Tallon‐BaudryC,BertrandO,DelpuechC,PernierJ (1996)Stimulusspecificityofphase‐lockedandnon‐phase‐locked40Hzvisualresponsesinhuman.JNeurosci16:4240‐4249.

TassP(1999)PhaseResettinginMedicineandBiology:StochasticModellingandDataAnalysisSpringer.

VoroninLL,VolgushevM,SokolovM,KasyanovA,ChistiakovaM,ReymannKG(1999)Evidencefor an ephaptic feedback in cortical synapses: postsynaptic hyperpolarization alters thenumberofresponsefailuresandquantalcontent.Neuroscience92:399‐405.

Wang C,Ulbert I, SchomerDL,Marinkovic K,Halgren E (2005) Responses of human anteriorcingulate cortex microdomains to error detection, conflict monitoring, stimulus‐responsemapping,familiarity,andorienting.JNeurosci25:604‐613.

WinfreeAT(1980)Thegeometryofbiologicaltime.Biomathematics8.WordenMS,FoxeJJ,WangN,SimpsonGV(2000)Anticipatorybiasingofvisuospatialattentionindexed by retinotopically specific alpha‐band electroencephalography increases overoccipitalcortex.JNeurosci20:RC63.