EEG Evidence for Audiovisual Speech Perception Deficits in...
Transcript of EEG Evidence for Audiovisual Speech Perception Deficits in...
EEGEvidenceforAudiovisualSpeechPerceptionDeficitsinParkinson’s
Disease:AResearchProposal
GeenaBruno
Introduction:
EEGstandsforelectroencephalogram,has32electrodesontheHydroCelGeodesic
SensorNet(HCGSN),andisavailablewithdensitiesof32,64,128,or256channelnet.Itis
connectedtoanamplifierandamplifiedelectricalsignalscomingfromthebrainare
recorded.Researchersareinterestedinthebrain'sresponsetoaspecificstimuli(Khamis-
Dakwar,2012).EEGdoesnotshowwhereactivationoriginates,butrather,thetimingof
howaperson’sbrainrespondstocertainstimuliinataskpresented.Anevent-related
potential(ERP)istheaverageelectricalactivitycorrelatedwiththespecifictypeof
stimuluspresentation;researchershavetosegmentpartsofEEGtogetERPs.ERP
componentsaredefinedintermsofacombinationofpolarity,latency,andscalp
distribution,andaresensorycognitiveandmotorprocessesrelatingtobehaviorand
thought(Kappenman&Luck,2011).Mismatchnegativity(MMN)isanERPcomponentthat
peaksto150–250millisecondswhenthereisachangeofstimulus(Khamis-Dakwar,2012);
forexample,ifthestimulusisanoddballparadigm,havingastandardanddeviantsound
(ta,ta,ta,da,ta).MMNalsohasthelargestamplitudeatfrontalandcentralelectrodesites
oftheskull,andthepatientdoesnotneedtorespondinordertogetanMMNresponseon
theEEG(FonaryovaKeyet.al,2005).Thisisgreatforbabieswhocannotrespondontheir
own,orforpatientswhohavecognitivedeficits.
Motorspeechdisordersaredifficultiesrelatingtoproblemsofmovement,resulting
fromaneurologicaldisorderorinjurythataffectsmotorplanning,programming,
coordination,timing,andexecutionusedforspeech;thesedisorderscanalsoaffect
respiration,resonation,phonation,andarticulation(OwensJr.et.al,2014)Thebasal
ganglia(BG)isasubcorticalareaofthebrainthatregulatesmotorfunctioningand
maintainspostureandmuscletone.Ithasdirectandindirectpathways,andifdamagedit
resultsineitherslowedmovementsasinParkinson’sdiseaseorinvoluntarymovementsas
inHuntington’sdisease(OwensJr.et.al,2014).TheBGisalsoinvolvedinonline
monitoringofauditoryfeedbackduringspeech(DeKeyseret.al,2016).Damagetothe
neuralcircuitryconnectingtheprefrontalandBGareasofthebraincausesanincreasein
syntacticandvoiceonsettimingerrorsonsingle-wordproductiontasks(Walsh&Smith,
2011).
Dysarthriaisagroupofspeechdisordersresultingfromdisturbancesinthecentral
andperipheralnervoussystemsthatcontrolmusclesofspeechproduction.Specifically,in
Parkinson’sdiseaseitisdescribedashypokineticdysarthria.Hypokineticdysarthriaisa
BGcontrolcircuitpathologyevidentinvoice,articulation,prosody.Ithaseffectsofrigidity,
difficultyinitiatingmovement,reducedforceandrangeofmotioninarticulators,slowbut
sometimesfastrepetitivemovementsonspeech,reducedvocalloudnesswithharsh–
hoarsequality,slowspeakingratewithburstofrapid-firearticulation,excessiveinlong
pauses,prolongedsyllables,mono-loudness,andreducedphonationtime(Freed,2000;
Owenset.al,2014).IndividualswithParkinson’sdiseasealsohavehigherlipaperture
variability,portrayinglessconsistentoralmotorcoordinationthanthecontrolparticipants
(Walsh&Smith,2011).Sensorimotorintegrationdeficitsmayresultfromdysfunctionsof
feedforwardandfeedbackcontrolinpeoplewiththisdisease(DeKeyseretal,2016).
Importantly,peoplediagnosedwithParkinson’sdiseasehavepoortemporaldiscrimination
fortactile,auditory,andvisualstimuli,alsoknownasproprioceptivedeficits(Duffy,2005),
whichwillhaveaneffectonauditory-visualintegration.
Researchshowsthatmultisensoryintegrationincreasesspeedofdetection,
enhancessensorysensitivity,andcorrectlyidentifiesevents:“Multisensoryintegrationis
mosteffectiveandthereforeelicitmaximalbehavioralenhancementswhenlessintenseor
weakandambiguousindividualstimuliareapplied”(Freiherret.al,2013).Thisisnotonly
evidentinpeoplewithadisorderordisease,butalsoinnormalhearingpeople;whenyou
areatabarorloudrestaurant,yourelyonvisualcueslikelookingatsomeone'smouthto
helpyouunderstandwhatthepersonissayingtoyou.ThisisalsocalledthePrincipleof
InverseEffectiveness;multisensoryintegrationhelpscounteracttheconsequencesof
unisensorydeterioration(Freiherret.al,2013).Also,substantialworkinthespeech
productionandspeechperceptionliteraturesupportsthenotionthatspeechperception
andproductionhaveabidirectionalinfluenceononeanother(DeKeyseret.al,2016).To
addtothis,visualarticulatoryinformationintegrateswiththemotorsystemduringspeech
production(Veneziaet.al,2016).
OnewaytoelicitanMMNresponseistousetheMcGurkeffect,whichiswhatis
beingusedinthisresearchproposal.TheMcGurkeffectisaperceptualillusionthatshows
theinfluenceofvisualspeechinformationontheperceptionofspeech(Cienkowski&
Carney,2002).Whatthepatientseesinfluenceswhatthepatienthears(Franciscoet.al,
2017).TheMcGurkEffectshowsthatmannerofarticulationandvoicingaretransmitted
mostefficientlyfromtheauditoryportionofthestimulusandplaceofarticulationis
transmittedbestbythevisualportion(Hessler,et.al,2013).
Thisstudywillinvestigatethepossibledifferencesinaudiovisualspeechperception
processinginindividualswithParkinson’sdiseaseandhealthyindividualsasindexedby
theMMNcomponentrecordedusinganEEGnet.
Methods
A. Participants
Therewillbe10individualsinthePDgroupand10healthyindividualsinthe
controlgroup.HealthyindividualsarematchedwithpatientswithPDbygenderandage.
PatientswithPDwerediagnosedbyaneurologistusingtheUnifiedParkinson’sDisease
RatingScale.Hypokineticdysarthriawasdiagnosedbyspeechlanguagepathologistsusing
theSentenceIntelligibilityTestandArizonaBatteryforCommunicationDisordersof
Dementia.Eachparticipantreceivedaspeechevaluationthatincludedacasehistoryofhis
orherspeechandlanguage,anoralmechanismexamination,perceptualjudgmentsof
respiration,phonation,resonance,articulation,andprosodyduringsustainedphonation,a
diadochokinetictask,andreadingandspontaneousspeech.Participantinterviewsand
reviewofPDparticipants’medicalrecordswereusedtodeterminethatallparticipantshad
negativehistoriesforheadtraumaandforpre-existingcommunication,memory,
neurologic,orpsychiatricproblems.
B.EEG
High-densityEEGwillberecordedfromparticipantswhiletheyareexposedto
experimentalstimuli.IntheAVMcGurkparadigm,participantsviewastandard
presentationofcongruentauditoryandvisualinformation(e.g.articulationof/ba/)
interspersedwithadeviantpresentationofincongruentvisual(e.g.articulationof/ga/)
dubbedovertheoriginalaudiostimulus.InEEGexperimentsutilizingtheMcGurkeffect,
thecongruentaudiovisualpresentationof/ba/ispresentedrepeatedlyasthestandard
stimulus.InfrequentpresentationoftheMcGurkstimulus,theincongruentaudio
presentationof/ba/pairedwithvisualpresentationof/ga/,generatestheMMN(Saint-
Amouret.al,2007).Inthisparadigmthepresentedaudioisconsistently/ba/andonlythe
visualstimuluschanges.
AnadditionalAVconditionutilizinganinverseMcGurkdeviantwillbeimplemented
toexploretheeffectsofincongruenceasmediatedbymodality.TheinverseMcGurk
condition(AVinverse,labeledAI)utilizescongruentpresentationofauditoryandvisual
/ba/asastandardstimuluswithachangetoauditory/ga/,whilemaintainingvisual/ba/
forthedeviantstimulus.Healthyindividualsperceivetheauditoryaspectofthestimulus,
ordonotfuseresponsesintoasinglepercept,butratherperceivebothsounds
simultaneously(/b-ga/).TheAIconditioninthepresentstudywouldprovideadirect
contrastwiththeAVcondition,elucidatingtheinfluenceofmodalityonincongruent
deviancedetection.
Avisual-only(VO)controlconditionisalsonecessarytoensurethatthederived
MMNisduetoAVintegrationprocesses(visualinformationchangingtheauditorypercept)
ratherthanresponsestochangeinvisualstimulus(SaintAmouretal.,2007).
Thedatawillbeanalyzedwithingroupsandbetweenconditionsaswellasacross
conditionsandbetweengroups,withinthetimewindowsofinterest,toevaluatewhether
thereweresignificantdifferencesinMMNpeakamplitude.
Condition Standard80%
Deviant20%
DeviantPercept
AV auditory/ba/andvisual/ba/
auditory/ba/andvisual/ga/
/da/
AI auditory/ba/andvisual/ba/
Auditory/ga/andvisual/ba/
/ba/or/b-ga/
VO Visual/ba/
Visual/ga/ /ga/
Figure 1
Thisexperimentwillutilizea128-channelEEGsystem(ElectricGeodesics,Inc.).The
128electrodesarearrangedinapredictablegeodesicpositionrelativetooneanotherina
sensornet.Theelectrodesareheldtogetherbyafineelastomerandcontainasilver
chloride-platedcarbonfiberembeddedinaplasticsubstrate.Eachelectrodehassponge
insertsthataresoakedinanelectrolytesolutionofpotassiumchlorideandwaterbefore
usetoensureoptimumconductivity.Eachparticipantwillwearanetthatfitsthe
circumferenceoftheirhead,whichwillhavebeenmeasuredbytheresearcher.Inaddition,
measurementsaretakentolocatethevertexoftheskullinordertoaccuratelyplacethe
net.Afterthenetisplacedontheparticipant,itisidealtohavetheindividualsitinachair
inasound-attenuatedroom.Theparticipantsitsinfrontofacomputermonitorthat
presentsthestimuliandthesensornetconnectstoacalibratedamplifier.
Figure2
C.Stimuli
Theauditory-visual(AV)stimulithroughoutthisexperimentwerecreatedtoevoke
theMcGurkMMN.Stimuliwerecreatedbydigitalrecordingofafemalenativespeakerof
AmericanEnglishsaying/ba/and/ga/.Digitalvideo(CanonVixiaHFR50)andcorrelating
audio(BlueMicYetiPro,www.bluemic.com)wererecordedatasamplingrateof44.1KHz
andaframerateof24images/second;theylatertrimmedforatotaldurationof300ms
pertoken.Theplacesofarticulationfor/ba/and/ga/differgreatly.Sincetheauditory
distinctioninthisparadigmdependsonplaceofarticulation,videosegmentsbeganinthe
preparatoryarticulatoryposition—closedlipsfor/ba/andopenlipsfor/ga/.Thespeaker
wasinstructedtoopenhermouthminimally.Visualinspectionofvideosegmentsensured
thatjawcoordinationwasconsistentbetweenthe/ba/and/ga/videosegments.Video
framewascroppedusingAppleiMovietorevealonlythespeaker’smouthinorderto
constrainthevisualpresentationandtoavoideye-movementartifactsduringEEG
recording.TheaudiotrackswereseparatedfromthevideoandeditedinPraatwith50ms
rise/falltoavoidclickartifactsintherecording,andamplitudesnormalizedto70dB.The
vowelsegmentfromone/ba/recordingwasremovedinPraatandthespliced/a/segment
wasusedwiththeonsetsforboth/b/and/g/,sotheonlydifferenceintheaudioisthe
consonantsegment.Theaudiotrackfor/ba/wasdubbedoverthevideotracksofboth
/ba/and/ga/,creatingcongruent(auditory/ba/,visual/ba/)andincongruentMcGurk
(auditory/ba/,visual/ga/)AVstimuliinAppleiMovie.OnsetoftheAVstimulibeginswith
thecontrastivearticulatoryposition,closedlipsfor/ba/andopenmouthfor/ga/.TheAI
conditionwassimilarlycreated,withaninverseMcGurkdeviant(auditory/ga/dubbed
ontovisual/ba/).
Visual-only(VO)stimuliconsistedofthesame300ms/ba/and/ga/videotracks
withaudioremoved.EEGepochsweresegmentedtocoincidewiththeonsetofthe
auditorycomponentofstimulustomorespecificallyexaminetheneurophysiological
responsetothevisualinfluenceonauditoryperception(Hessleretal.,2013).
Figure 3
a. OddballParadigm
StimulifortheAV,AI,andVOconditionswillbepresentedinanoddball
paradigm.Thisiswhenthestandardswerepresentedfor80%oftrialsandthedeviantsfor
20%oftrials.Eachconditionhas450totaltrials,with360standardsand90deviants.
Stimuliwillbepresentedpseudorandomlyinordertoensurethatatleasttwostandards
camebeforeeverydeviantandthatdeviantswerenotplayedconsecutively.The
interstimulusinterval(ISI)forallconditionsis600ms.Presentationofeachconditionis
counterbalancedbetweenparticipantswithineachgroup.
Figure 4
B. Experiment/BehavioralMeasuresandProcedures
a.BehavioralMeasures
1.TheSentenceIntelligibilityTest(SIT)has1,100sentences(100
sentencesrangingfromfiveto15wordsinlength).Participantswillbeevaluatedusing10
to15sentencestotestintelligibilityandwillbeaskedtorepeatsentencesreadbythe
researcher.Thiswillhelptomakesureparticipantshaveahighintelligibilityrateto
understandwhattheyneedtodothroughouttheexperimentandtheinstructionsthey
needtofollow(Yorkston,Beukelman,&Tice,1996).
2.TheArizonaBatteryforCommunicationDisorders(ABCD)isa
comprehensiveassessmentandscreeningofdementia.Itincludes14subteststhatevaluate
verbalepisodicmemory,linguisticcomprehension,visuospatialconstruction,etc.The
ABCDwasstandardizedonAlzheimer'sandParkinson’sdiseasepatients(Bayles&
Tomoeda,1991).Itisimportantthatpatientshaveatleast80%ormoreontheABCDin
ordertoparticipateinthisstudy.
c.Participationintheexperimentshouldinvolvetwovisits:
Visit1:
1. Participantswillbeshownthelab,equipment,andproceduresbyan
experiencedspeech-languagepathologistwithexperiencediagnosingand
treatingindividualswithPD.CaregiverswillaccompanypatientswithPDand
questionswillbeencouragedthroughoutthisfamiliarizationperiod.
2. Participantswillbepresentedwithaconsentformthattheparticipant,
caregiver(ifnecessary),andresearcherhavetosign.Thespeech-language
pathologistgivestimeforparticipantstoreadtheformandalsoprovidesa
verbalexplanation.Last,theparticipantandcaregiverareinformedofany
riskfactorsandoftheiroptiontowithdrawatanytimeduringthe
experiment.
3. Followingcompletionofconsentprocedures,participantsparticipatedin
administrationofstandardizedtests
4. Visit1willvaryintimedependingoneachparticipant’sneeds
Visit2:
1.Afterassessingiftheparticipantswereappropriateforthestudy,the
participantswillbeinvitedbacktocompletetheexperimentaltasks.Onthesecondlab
visit,aconsentform(thesameasforvisit1)willbepresentedagainandtherewillbe
opportunitiesforquestions.
2.Hearingmustbescreenedat500,1000,2000,and4000Hzwithinarange
of40–20dBbeforethestartoftheexperiment.
3.Thecircumferenceoftheheadwillbemeasuredtochoosethecorrectnet
sizeandthevertexwillbemarkedontheskulltoensurethatalltheelectrodesare
correctlyplaced.
4.Theparticipantwillbeseated80cmfromthecomputermonitor,ina
sound-attenuatedroom.Theparticipantswillbegiveninsertheadphones;thered-
codedinsertgoesintotherightearandtheblueintheleft.Avideocamerawillbein
theroomtogivetheresearchervisualinformationabouttheparticipantduringthe
experiment.Theparticipantwillberemindedtosignalatanytimeduringthe
experimentifheorshedoesnotwishtocontinue.Theamplifiermustbechecked
andcalibratedbeforethenetisconnected,andimpedances(lossofsignalbetween
scalpandsensor)mustbemeasured.Inordertoimproveimpedances,the
electrodesneedtobeadjustedasnecessarysothattheyareingoodcontactwiththe
participant’sscalp.
5.ExperimentalEEGtaskswillbepresentedinrandomorderand
counterbalancedacrossparticipants.Taskswillbepresentedinshortrunsofless
than10minutestominimizefatigueandreducehabituationthatinterfereswith
MMNelicitation.Participantswillbeencouragedtotakeshortbreaksbetweenruns.
6.Aftercompletingtheexperimentaltasks,thespeech-languagepathologist
willremovetheEEGnetandtheparticipantwillbeaskedquestions.Visit2maylast
approximately60minutesorlonger,dependingonparticipant’sneedsandnumber
anddurationofbreaks.
ProposedResults:
Speech-productionimpairmentinPDmayimpacthigher-levellinguisticand
cognitiveprocessingasindexedbytheMcGurkMMN.Thecomparisongroupisexpectedto
showtheclassicMMNinresponsetotheMcGurkeffectintheAVconditionbutnottheAI
condition.ItisanticipatedthatresponsesinthePDgroupwillbemorevariablethaninthe
comparisongroup.ThismeansthatindividualswithPDnotgettinganearlyautomatic
fusionMMNresponseintheAVmayhaveadeficitinaudiovisualintegration.Weexpectthe
PDgroup’sresponsetobesimilarintheAVandAIconditions,indicatingthattheycan
detecttheoddballdeviantbutdonotexperienceAVintegration.ResultsforVOstimuliwill
beconsistentwithinthePDgroupandcomparisongroup,demonstratingthattheresults
arenotduetosimplechangedetectionofthevisualstimulus.
Resultswillalsorevealasmalleramplitudeandlongerlatencyresponsein
individualswithPDascomparedtocontrols.ThismeansthatpatientswithPDwillhavea
slowerresponsetimeandhaveatendencytowardlatergoingresponse.Thiswillchange
theirMMNresponsetolooklikeaP300response(anotherERPcomponentderivedfrom
EEGmeasurements).TheseresultsexplainthatindividualswithPDhaveadeviancein
detectionofstimuliandaninsensitivitytomodality.
Figure 5 (examples of MMN and P300 responses)
Figure 6 (Jahshan et. al, 2012) MMNisrecordedatlargestoverthefrontocentralelectrodes,regardlessofwhere
theprocessishappening.Theimagebelowisarepresentationofthisimplication.
Figure7
DiscussionandClinicalImplication:
TheresultsoftheproposedstudywillhelpusunderstandifdeficitsinAV
integrationarepartofParkinson’sdiseaseoraresultoftheongoingexperienceofliving
withaspeechimpairment.Asstatedinliterature,patientswithPDhaveproprioceptive
deficitsthatcausepoortemporaldiscriminationfortactile,auditory,andvisualstimuli
(Duffy,2005).ThiscanhaveaneffectonAVintegration,andtherapymaynotchangethese
deficits.Also,inmanytherapytechniquesforpatientswithspeech,language,andhearing
disorders,speechtherapistsemphasizetheimportanceoftheprincipleofinverse
effectiveness,orusingmultiplesensestocounteractthenegativeeffectsofweakeningone
sensorysystem(Freiherr,2013).Whilewetendtousevisualarticulatorycuesduring
speechtherapy,wedon’tknowifvisualarticulatoryinformationisfacilitativeor
overloadingthesysteminpatientswithPD.
References
Bayles,K.A.,&Tomoeda,C.K.(1991).ArizonaBatteryforCommunicationDisordersofDementia.
Tucson,AZ:CanyonlandsPublishing.
Cienkowski,K.M.,&Carney,A.E.(2002,November).Auditory-visualspeechperceptionand
aging.EAR&HEARING,23(5),439-449.
DeKeyser,K.,Santens,P.,Bockstael,A.,Botteldooren,D.,Talsma,D.,DeVos,S.,…DeLetter,
M.(2016,October).Therelationshipbetweenspeechproductionandspeechperception
deficitsinParkinson’sdisease.JournalofSpeech,Language,andHearingResearch,59,915-
931.
Duffy,J.R.(2005)MotorSpeechDisorders:Substrates,DifferentialDiagnosis,andManagement
(2nded.).ElsevierMosby.
FonaryovaKey,A.P.,Dove,G.O.,&Maguire,M.J.(2005).LinkingBrainwavestotheBrain:An
ERPPrimer.DEVELOPMENTALNEUROPSYCHOLOGY,27(2),183-215.DOI:
10.1207/s15326942dn2702_1
Francisco,A,A.,Jesse,A.,Groen,M.A.,&Mcqueen,J.M.(2017,January).Ageneral
audiovisualtemporalprocessingdeficitinadultreaderswithdyslexia.Journalof
Speech,Language,andHearingResearch,60,144-158
Freed,D.(2000).Motorspeechdisorders:diagnosisandtreatment.SanDiego,CA:Singular
ThomsonLearning.
Freiherr,J.,Lundstrom,J.N.,Habel,U.,&Reetz,K.(2013,December).Multisensoryintegration
mechanismsduringaging.FrontiersinHumanNeuroscience,7,1-6.
Hessler,D.,Jonkers,R.,Stowe,L.,&Bastiaanse,R.(2013).Thewholeismorethanthesumofits
parts–AudiovisualprocessingofphonemesinvestigatedwithERPs.Brainand
language,124(3),213-224.
Kappenman,E.S.,&Luck,S.(2011).ERPComponents:TheUpsandDownsofBrainwave
Recordings.TheOxfordHandbookofEvent-RelatedPotentialComponents,3-30.
doi:10.1093/oxfordhb/9780195374148.013.0014
Khamis-Dakwar,R.(2012).Translationaldisplayofneurophysiologicalinvestigations.In
CommunicationSciencesandDisorders.In:Goldfarb,R.(Ed.).TranslationalSLP/A:
EssaysinhonorofDr.SadanandSingh.PluralPublishing,Inc.IntroductionToERP(2012)
Owens,R.E.,Jr.,Metz,D.E.,&Farinella,K.A.(2014).IntroductiontoCommunication
Disorders:ALifespanEvidence-BasedPerspective,FifthEdition.(5thed.).Pearson.
Saint-Amour,D.,DeSanctis,P.,Molhoim,S.,Ritter,W.,&Foxe,J.J.(2007,February).Seeing
voices:High-intensityelectricalmappingandsource-analysisofthemultisensory
mismatchnegativityevokedduringtheMcGurkIllusion.Neuropsychologia,45(3),
587-597
Venezia,J.H.,Fillmore,P.,Matchin,W.,Isenberg,A.L.,Hickock,G.,&Fridriksson,J.(2016).
Perceptiondrivesproductionacrosssensorymodalities:Anetworkforsensorimotor
integrationofvisualspeech.NeuoroImage,(126),196-207.
Walsh,B.,&Smith,A.(2011,June).Linguisticcomplexity,speechproduction,and
comprehensioninParkinson’sDisease:BehavioralandphysiologicalIndices.Journalof
Speech,Language,andHearingResearch,54,787-802.
Yorkston,K.M.,Beukelman,D.,&Tice,R.(1996).SentenceIntelligibilityTestforWindows
[Computersoftware].Lincoln,NE:TiceTechnology.