Differences)betweennative)and …aschafer/Schafer_JSLS_2017_slides.pdfDifferences)betweennative)and...

73
Differences between native and nonnative English listeners in the use of prosodic focus and event structure to anticipate discourse structure and resolve reference Amy J. Schafer University of Hawai‘i JSLS 2017, Kyoto, July 2017 [email protected] @amyjschafer

Transcript of Differences)betweennative)and …aschafer/Schafer_JSLS_2017_slides.pdfDifferences)betweennative)and...

Differences  between  native  and  non-­‐native  English  listeners  in  the  use  of  prosodic  focus  

and  event  structure  to  anticipate  discourse  structure  

and  resolve  referenceAmy  J.  Schafer

University  of  Hawai‘i

JSLS  2017,  Kyoto,    July  2017 [email protected] @amyjschafer

Theres Grüter(University  of  Hawai‘i)

Hannah  Rohde(University  of  Edinburgh)

Plus  Amber  Camp,  Wenyi Ling,  Aya Takeda,  and  our  wonderful  team  of  research  assistants  (University  of  Hawai‘i)

NSF  Standard  Grant  BCS-­‐1251450

2

Humans  have  expectations  about  how  situations  in  the  real  world  will  unfold

3

situation  model

expectationsabout  who  &  what  will  be  mentioned  next

We  use  our  general  knowledge  of  events  to  create  situation  models  for  a  discourse,  which  guide  our  expectations  for  how  it  will  unfold,  such  as  who  will  be  mentioned  next

4

situation  model:completed  event

increased  expectation  to  hear  about  

what  happened  next

Wenyi gave Nancy some chocolates.

5

SheNancy …

increased  expectation  to  hear  about  the  causeor  about  event  details

Wenyi gave Nancy some chocolates.Wenyi was giving Nancy some chocolates.

SheWenyi …

6

SheNancy …

situation  model:ongoing  event

situation  model

expectationsabout  who  &  what  will  be  mentioned  next

grammatical  aspect

7

Focus  of  today’s  talk

•To  what  extent  is  the  comprehension  of  language  influenced  by  the  anticipation of  upcoming  discourse  material?• In  native  speakers  of  English  • And  in  non-­‐native  Japanese-­‐ and  Korean-­‐speaking  learners  of  English

•Specifically,  is  anticipation  for  next-­‐mention  influenced  by:• Event  structure  (completed  versus  ongoing  events)?• Prosody?

8

Aspect  and  co-­‐reference

•Significant  effects  of  grammatical  aspect  on  co-­‐reference  for  native speakers  of:• English(e.g.,  Rohde,  Kehler &  Elman,  2006;  Kehler,  Kertz,  Rohde  &  Elman,  2008;  Ferretti,  Rohde,  Kehler &  Crutchley,  2009)• Japanese(Ueno  &  Kehler,  2010,  2016)

• Korean(Kim,  Grüter &  Schafer,  2013,  in  prep.)

ØWill  it  also  affect  non-­‐native  speakers?

9

Prosody  and  co-­‐reference

• Little  research  -­‐ even  for  native  speakers  of  English•Mixed  results• Prosodic  form  rarely  described  in  detail

•Here:  Use  of  contrastive  prosody  (prosodic  focus)

10

Contrastive  prosody

• Increases  attention  to  the  focused  phrase  (e.g.,    Cutler  &  Foss,  1977;  Fraundorf et  al,  2010)

•Attracts  relative  clauses  (e.g.,  Schafer  et  al,  1996)

•Facilitates  comprehension  of  alternatives  (e.g.,  Dahan et  al,  2002;  Ito  &  Speer,  2008)• Hang  the  green  drum.  • Now  hang  the  BLUEL+H* drum…

•Primes  alternatives  (“contrast  associates”)  (e.g.,  Braun  &  Tagliapietra,  2010;  Husband  &  Ferreira,  2015)

11

Suprasegmentals and  second  language  learners  (L2ers)• Humans  begin  adapting  to  their  native-­‐language  suprasegmental system  prenatally• L2  perception  of  suprasegmentals is  shaped  by  the  L1  system  (e.g.,  L1  intonation  affects  L2  lexical  tone  perception)• Suprasegmentals are  widely  perceived  to  be  challenging  for  L2ers,  in  production  and  perception• Yet  the  acquisition  of  L2  prosody/intonation  is  understudied• Even  though  it  is  valuable  to  both  basic  and  applied  questions

12

Contrastive  prosody  involves  multiple  levels  of  representation…who…? discourse/information  structure,  QUD{Marianna,  Beth} alternative  sets[Marianna]F syntactic  marking

L+H* presence/type  of  pitch  accentMariANna f0,  duration

• A  learner  must  identify  sets  of  acoustic  cues  and  make  appropriate  mappings  to  higher-­‐level  linguistic  distinctions• As  researchers,  we  need  to  tease  apart  processing/learning  at  different  levels• E.g.,  is  prosodic  prominence  (e.g.,  expanded  f0)  detected  without  it  being  appropriately  mapped  to  alternative  sets  or  alternative  Questions  Under  Discussion?

13

Assuming  effects  of  contrastive  prosody  in  native  speakers:Will  native-­‐Japanese  and  native-­‐Korean  learners  of  English  demonstrate  effects?• Both  Japanese  and  Korean  use  prosodic  prominence  (e.g.,  expanded  f0)  to  mark  contrastive  focusPredict  at  least  basic  “prominence”  effects

• Both  differ  from  English  in  their  intonational phonology  English  has  multiple  types  of  pitch  accents;  J/K  do  not

• Japanese  learners  show  difficulty  with  English  pitch  accents(e.g.,  Takeda  et  al  2015a,b;  Nakamura,  Symposium  talk)

• Numerous  other  differences  between  J/K  and  English  Morphological  focus  markers,  use  of  pronouns,  scrambling…Predict  more  L1/L2  differences  for  “higher-­‐level”  decisions

14

Discourse  comprehension  in  native  speakers:  A  dynamic  process  with  incremental  updates

15

“Wenyi …first  mention,  subject,  topic  -­‐>  important  to  discourse“WenyiSource gave  …

transfer  event,  completed  -­‐>  what’s  the  result?“WenyiSource gave  NancyGoal …

goal,  2nd human  -­‐>  new  possible  topic“Wenyi gave  Nancy  some  chocolates.  She  …

pronoun  -­‐>  subject/topic  bias“Wenyi gave  Nancy some  chocolates.  She  ate  them…  à SheNancy

She  had  found  them…à SheWenyi

coherence  relation

Processing  by  non-­‐native  speakers

• Incremental• Potentially  different  weighting  of  cues

(e.g.,  less  reliance  on  ‘syntax’,  more  on  ‘discourse’?)• Potentially  less  anticipatory• Lexical  semantics:  yes(e.g.,  Chambers  &  Cooke,  2009)•Morphosyntax:  mixed  findings(see  Kaan,  2014,  for  review)•Discourse:  ?    (current  studies)

16

The  RAGE hypothesis

Non-­‐native  speakers  have:  reduced  ability  to  generate  expectations  

(Grüter,  Rohde  &  Schafer,  2014,  2016)

• Stated  strongly,  to  push  forward  research  questions

• Attention  to  expectations,  to  refine  “processing  difficulty”

• Requires  us  to  sharpen  our  views  of  when/how  processing  decisions  take  place,  in  native  speakers  and  in  non-­‐native  speakers

17

Experiments

Part  1:  Story  continuation  experiments

•Participant  reads/hears  the  beginning  of  a  story•Then  provides  a  continuation  in  their  own  words

•Open-­‐ended  task;  rich  linguistic  information:• Next-­‐mention  preferences  (referent  of  the  continuation  subject)• Coherence  patterns(e.g.  whether  the  next  sentence  describes  an  outcome  of  the  previous  event,  provides  an  explanation,  etc.)

19

Experiment  1:  Written  story  continuationAspect  and  prompt  type

Grüter,  Rohde  &  Schafer  (2017)

20

Exp1:  Participants

•39  native  speakers  of  English

•48  non-­‐native  speakers• 23  native  Japanese• 25  native  Korean•Mostly  international  or  exchange  students  at  UH• Proficiency:  majority  are  B2  (Independent  Users)  or  A2  (advanced  Basic  Users)  in  the  Council  of  Europe  framework

21

Exp1:  Stimuli  and  design2  (aspect)  x  2  (prompt  type)  design

COMPLETED TRANSFER-­‐OF-­‐POSSESSION EVENT (PERFECTIVE)Emily  brought a  drink  to  Melissa.  She _______________________  Emily  brought a  drink  to  Melissa.   __________________________

ONGOING TRANSFER-­‐OF-­‐POSSESSION EVENT (IMPERFECTIVE)Emily  was  bringing a  drink  to  Melissa.  She ____________________Emily  was  bringing a  drink  to  Melissa.   _______________________

Previous  research  on  prompt  type:  • Pro-­‐forms  in  subject  position  à increased  reference  to  the  previous  subject  for  native  speakers  of  English,  Japanese,  and  Korean

Latin  square  design,  5  items/condition  +  20  fillers(10  verbs:  bring,  feed,  give,  mail,  pass,  push,  roll,  serve,  take,  throw)

22

Annotation  for  co-­‐referenceEmilySOURCE brought/was  bringing  a  drink  to  MelissaGOAL (She)  ______

She thought Melissa was thirsty.(SOURCE-­‐continuation)

Melissa said “Thank you.”(GOAL-­‐continuation)

She did not want it. (ambiguous:  4/4%  of  L1/L2  data)

It was Coke. (other:  12/13%  of  L1/L2  data)

23

Annotation  for  coherenceEmily  brought/was  bringing  a  drink  to  Melissa.  (She)  _______________

She thought Melissa was thirsty. (EXPLANATION)

She gave her Coke. (ELABORATION)

Emily dropped it on the ground. (VIOLATED EXPECTATION)

Melissa drank it. (OCCASION)

Melissa said “Thank you.” (RESULT)

(Hobbs,  1979;  Kehler,  2002)

24

Exp1:  Co-­‐reference  results25

L1 L2

0.00

0.25

0.50

0.75

1.00

Pronoun Free prompt Pronoun Free prompt

Prop

ortio

n of

Sou

rce−

refer

ence

resp

onse

s

AspectImperfectivePerfective

Native  speakers:• Sig.  effect  of  Aspect  • Sig.  effect  of  PromptØ rep.  of  previous  work

Non-­‐native  speakers:• Significant  effect  of  PromptØ even  though  pronoun  acquisition  can  be  difficult  for  non-­‐native  speakers• Reduced  effect  of  Aspect  (n.s.;  sig  Aspect*Group  interaction)Ø consistent  with  RAGE  hypothesis

Exp1:  DiscussionNative  speakers• Generate  expectations  based  on  Aspect  (ongoing/completed  event)  and  Verb  type  (transfer  event)• Update  those  when  a  pronoun  appears

Non-­‐native  speakers•More  likely  to  delay  decisions  until  they  become  required• Less  likely  to  anticipate  co-­‐reference;  less  effect  of  Aspect  on  co-­‐reference  – even  though  they  use  it  in  their  L1• Respond  at  the  prompt  without incorporating  aspectual  information

26

Experiment  2:  Spoken story  continuationContrastive  prosody  on  Source/Goal

Schafer,  Rohde  &  Grüter,  2015Schafer  et  al.,  2015,  in  prep.

27

Exp2:  Stimuli  and  design2  (aspect)  x  2  (contrastive  prosody  location)  design

COMPLETED EVENT (PERFECTIVE)DAVID served  Paul  a  pint  of  beer.  He  _______________________David  served  PAUL a  pint  of  beer.  He  _______________________

ONGOING EVENT (IMPERFECTIVE)DAVID was  serving  Paul  a  pint  of  beer.  He  ______________________David  was  serving  PAUL a  pint  of  beer.  He  ______________________

• All  prompts  begin  with  written  pronouns

Latin  square  design,  5  items/condition  +  20  fillers(10  verbs:  bring,  e-­‐mail,  feed,  give,  hand,  pass,  present,  roll,  serve,  throw)

28

Exp2:  Stimuli

Source  contrast:BRENDA  fed  Anne  a  bowl  of  soup.  (She  __)

Goal  contrast:Brenda  fed  ANNE a  bowl  of  soup.  (She  __)

29

Brenda fed Anne a bowl of soup

L+H* L-H% L*+H !H* !H* !H* L-L%

125

450

200

300

400

Pitc

h (H

z)

Time (s)0 3.44

Brenda fed Anne a bowl of soup

L*+H !H* L+H* L-H% H* !H* L-L%

125

450

200

300

400

Pitc

h (H

z)

Time (s)0 3.3

The  L+H*  L-­‐H%  contour•Type  of  “Rise-­‐Fall-­‐Rise”  (RFR)  contour•Suggests  contrast  (L+H*)  and  continuation  (L-­‐H%)•Marks  contrastive  topics(e.g.,  Steedman,  2000;  Constant,  2012;  Roberts,  2012)What  about  Fred?  What  did  he  eat?  FREDL+H L-­‐H%        ate  the  BEANSH*  L-­‐L%• Implicates  alternatives  and  promotes  discourse  continuations  that  express  contrast(Dennison,  2010;  Kurumada et  al,  2014;  Dennison  &  Schafer,  2017;  Sugawara,  Symposium  talk)The  pencil  WASL+H sharpL-­‐H%        …but  now  it’s  dull.

30

Exp2:  Specific  prosodic  hypotheses

• Prosodic  boost?• (Anticipate)  co-­‐reference  to  prominent/focused  entitiesØMore  Source  reference  with  Source  contrastØMore  Goal  reference  with  Goal  contrast

• Parallelism  (topic  continuity)?•Maintain  a  parallel  information  structure;  answer  a  parallel  sub-­‐question  of  the  Question  Under  Discussion

Ø Continue  Source  reference  when  there  is  Goal  contrast;  mention  an  Alternative  Goal,  e.g.:(David  served  PAUL  a  pint  of  BEER.  HeDavid gave  MARY  some  NUTS.)  

31

She  …

wanted to make her happy.

32

Exp2:  Co-­‐reference  results33

Native  speakers:Aspect:  significantProsody:  signif.Ø Prosodic  boost

Non-­‐native:Aspect:  n.s.  (p=.3)Ø replicates  Exp1Ø supports  RAGE

Prosody:  signif.Ø Prosodic  boost

Ø Increased  reference  to  prominent  antecedent  in  each  groupØProsody  is  not always  difficult  for  non-­‐native  speakers

L1 L2

Imperf. Perf. Imperf. Perf.0.00

0.25

0.50

0.75

1.00

Prop

. of S

ourc

e re

fere

nce Contrast location

SourceGoal

Contrastive  prosody

• Acoustic  prominenceDAVID…

• Psychological  salienceMAHO…

• Semantic  contrast  (alternative  set)

WENYI…

• Expectations  for  a  continuation  that  licenses  contrast  (e.g.,  explanation;  violated  expectation;  parallel)

Wenyi gave  FRED…

34

Exp2:  Discussion

Native  speakers•Expectations  based  on  Aspect  and  Prosody

Ø By  assumption  – need  online  evidence  for  verification

Non-­‐native  speakers  •Prosodic  effect,  but  is  it  anticipatory,  and  is  it  really  driven  by  information  structure?Ø Let’s  test  a  case  less  easily  explained  by  simple  prosodic  prominence

35

Experiment  3:  Spoken story  continuationContrastive  prosody  on  pronoun

Schafer  et  al.,  2015-­‐BUCLD  poster,  in  prep.

36

Exp3:  Stimuli  and  design2  (aspect)  x  2  (prosody)  design

COMPLETED EVENT (PERFECTIVE)David  served  Paul  a  pint  of  beer.  He obviously  _______________David  served  Paul  a  pint  of  beer.  HE obviously  _______________

ONGOING EVENT (IMPERFECTIVE)David  was  serving  Paul  a  pint  of  beer.  He obviously  ____________David  was  serving  Paul  a  pint  of  beer.  HE obviously  ___________

• Spoken  prompts,  which  include  adverbs  to  allow  natural  truncation• Adverbs  fit  with  explanations,  common  with  Source  &  Goal  ref.

Latin  square  design,  5  items/condition  +  20  fillers(10  verbs:  bring,  e-­‐mail,  feed,  give,  hand,  pass,  present,  roll,  serve,  throw)

37

Exp3:  StimuliDavid  served  Paul  a  pint  of  beer.

Unstressed  pronoun: Stressed  pronoun:He obviously  ___ HE obviously  ___

38

Exp2  vs.  Exp3:  Comparison…

• L+H*  L-­‐H%  prosodic  manipulation;  similar  realization• Co-­‐reference  of  target  pronoun  to  Source/Goal

Exp 2:  Source  contrast: Exp 3:  Stressed  pronoun:DavidSource served… HE obviously  ___

39

…  and  contrast

Exp2• L+H*  L-­‐H%  on  an  earlier  referent  (e.g.,  David)  à•Multiple  ways  to  achieve  Prosodic  BoostØ Prosodic  focus  draws reference

Exp3• L+H*  L-­‐H%  on  target  pronoun  (e.g.,  HE)  à•Cannot  rely  on  Prosodic  boostØ Prosodic  focus  expected  to  shift reference  

40

Exp3:  Specific  prosodic  hypotheses• Form  effect?• Fuller  expression  à Reference  to  less  salient  entity(Gundel et  al.,  1993)  ØMore  non-­‐subject  (i.e.,  Goal)  reference  with  stressed  pronoun

• ‘Flip’  effect?• Choose  a  contextually  plausible  alternative  to  the  referent  preferred  for  an  unstressed  pronoun  (Hirschberg  &  Ward,  1991;  Kameyama,  1999)  Ø Stressed  pronoun  flips  preference  of  unstressed  pronoun

• Coherence  effect?• Anticipate  plausible  coherence  for  the  focal  structure;  choose  reference  accordingly  (Cummins  &  Rohde,  2015;  Kehler,  2005;  Kehler et  al.,  2008)  ØMore  causality  in  continuationsØ Difficulty  for  non-­‐native  speakers

41

Exp3:  Co-­‐reference  results42

Native  speakers:Aspect:  significantProsody:  significant

Non-­‐native:Aspect:  n.s.Prosody:  n.s.Asp*Pros:  significantPairwise:Ø Aspect  for  unstr:  sigØ Stress  for  imperf:  marg.

ØContrastive  prosody  IS difficult  for  non-­‐native  speakers  when  it  requires  more  complex  processing

L1 L2

Imperf. Perf. Imperf. Perf.0.00

0.25

0.50

0.75

1.00

Prop

. of S

ourc

e re

fere

nce Pronoun stress

UnstressedStressed

Exp3:  Discussion

•Significant  effect  of  prosody  only  for  native  speakersNon-­‐native  speakers:Ø Difficulty  with  higher-­‐level  decisions  about  information  structure  and/or  expectations  for  coherence

•Reduced  effect  of  Aspect  in  non-­‐native  speakers  àincremental  processing  effect• Aspect  is  n.s.  for  Stressed  pronoun:  HE  <pause>  obviously

Ø Reference  is  determined  during  the  pause• Effect  of  Aspect  for  Unstressed  pronoun:  He  obviously

Ø Draw  on  Aspect  when  constructing  the  VP

43

Prosody  and  co-­‐reference:  Summary  so  far

Native  speakers• Effects  of  contrastive  prosody  in  both  experiments

Non-­‐native  speakers• Effects  only  when  it  was  on  a  potential  antecedentØ Success  in  using  contrastive  prosody  may  depend  on  the  complexity  and  timing  of  steps  required  to  make  the  relevant  mappings

Ø Need  online  evidence  to  better  understand  group  differences  and  potential  anticipatory  effects

44

Part  2:  Visual  world  experimentsWhat  happens  during  the  silence  between  sentences?  

Will  we  see  expectations  for  next-­‐mention  before the  continuation  begins?

45

Visual  world  paradigm

•Spoken  story  +  comprehension  question•Visual  scene  depicts  entities  in  the  story•Gaze  is  tracked  as  the  story  unfolds•Captures  dynamic  changes  in  attention  to  entities  

46

1st sentence:'Donald brought

Melissaa fancy drink.'

[Silence]

2nd sentence:'REF obviouslyliked hosting

parties.'

0.0

0.2

0.4

0.6

0.8

−4000 −2000 0 2000Time from pronoun onset (ms)

Prop

. loo

ks

AOISourceGoalTheme

Experiment  4:  Visual  world  paradigmAspect  (with  broad  focus)

Grüter,  Takeda,  Rohde  &  Schafer,  2016,  2017,  in  prep.

47

Exp4:  Participants

Age(in  years)

VersantEnglish  Test1(overall  score,  range  20-­‐80)

LexTALEEnglish2(%  correct)

Self-­‐ratedEnglish  

proficiency(out  of  10)

L1-­‐English(n=54) 23  (18-­‐49) -­‐-­‐ -­‐-­‐ 9  (6-­‐10)

L2-­‐English  (n=35) 28  (18-­‐46) 60  (39-­‐80) 71  (49-­‐100) 6  (1-­‐9)

• Various  L1s:  19  Chinese,  4  Japanese,  2  Spanish,  2  Indonesian,  8  other• First  exposure  to  English  M  =  11  yrs  (3-­‐21)• Performed  similarly  to  participants  in  Exp1  on  the  ‘Knowledge-­‐of-­‐Aspect’  task

1Pearson  (2011;  http://www.versanttest.com),  2Lemhöfer  &  Broersma  (2012;  www.lextale.com)    

48

Exp4:  Stimuli  and  design

5  items/condition  (Latin  square)  =  20  experimental  items;  40  non-­‐transfer  fillers

49

• 2  (aspect)  x  2  (reference)  design

Context  (completed/ongoing)Donald  {  brought  /  was  bringing  }  Melissa  a  fancy  drink.  

Continuation  (pronoun  gender  matches  Source/Goal){  HeDonald /  SheMelissa  }  obviously  liked  hosting  parties.

Exp4:  Trial  structure

preview [2000ms]

context Donald  brought  Melissa  a  fancy  drink.

silence [2500ms]

continuation He obviously  liked  hosting  parties.pause [250ms]

[1500ms]

question Who  liked  hosting  parties?

{mouseclick on  box  corresponding  to  answer}

50

Exp4:  Predictions

Following  a  transfer-­‐of-­‐possession  event  and  prior  to  the  encounter  of  a  referential  expression,

1)   Native  speakers  will  look  at  the  Goal  more  following  perfective  than  imperfective  verbs;

2)   Non-­‐native  speakers’  looks  will  not  be  modulated  by  aspect.

51

collapsing  over  Aspect  and  Reference

Exp4:  Eyegaze overview(N  =  54  native  speakers  of  English)

1st sentence:'Donald brought

Melissaa fancy drink.'

[Silence]

2nd sentence:'REF obviouslyliked hosting

parties.'

0.0

0.2

0.4

0.6

0.8

−4000 −2000 0 2000Time from pronoun onset (ms)

Prop

. loo

ks

AOISourceGoalTheme

52

collapsing  over  Reference

[Silence]'REF obviously liked

hosting parties.'

0.0

0.2

0.4

0.6

0.8

−2000 −1000 0 1000 2000Time from pronoun onset (ms)

Prop

. loo

ks

AOI−AspectSource−Perf.Source−Imperf.

53

Native  speakers,  looks  to  Source  by  Aspect

collapsing  over  Reference

[Silence]'REF obviously liked

hosting parties.'

0.0

0.2

0.4

0.6

0.8

−2000 −1000 0 1000 2000Time from pronoun onset (ms)

Prop

. loo

ks

AOI−AspectSource−Perf.Goal−Perf.Source−Imperf.Goal−Imperf.

Native  speakers,  looks  to  Source  &  Goal

54

Analysis  region:  -­‐2000  to  200

Native  speakers:Aspect:  significant

Exp4:  Native  versus  non-­‐native  speakers

55

à Non-­‐native  speakers  do  not show  an  anticipatory  effect  of  Aspect...…but  they  do  show  an  anticipatory  bias  toward  the  Goal

[Silence]2nd sentence:

'REF obviously liked hosting parties.'

[Silence]2nd sentence:

'REF obviously liked hosting parties.'

L1 L2

−2000 −1000 0 1000 2000 −2000 −1000 0 1000 20000.0

0.2

0.4

0.6

0.8

Time from pronoun onset (ms)

Prop

. loo

ks AOI−AspectSource−Perf.Goal−Perf.Source−Imperf.Goal−Imperf.

Exp4:  Discussion

•Anticipatory  effect  of  Aspect  in  the  native  speakers,  but  not  in  the  non-­‐native  group•Anticipatory  Goal  bias,  consistent  with  the  semantics  of  transfer-­‐of-­‐possession  verbs,  in  both groups.

Ø Non-­‐native  speakers  are  able  to  create  discourse-­‐level  expectations  based  on  verb  semantics,  but  they  may  not  be  able  to  use  the  grammatical-­‐aspect  cue  to  dynamically  update  their  expectations.

56

Experiment  5:  Visual  world  paradigmRicher  context;  Aspect  &  Prosody

57

Exp5:  Story  structurePreambleThe  party  was  held  at  a  bar  near  Emily,  Melissa  and  Kira’s  house.

Context  (2  x  3  design)Emily  brought  Melissa  a  fancy  drink.

Continuation  (ambig.  pronoun/Goal  name,  between  items){She/Melissa}  obviously  liked  hosting  parties.

58

Exp5:  Context  sentence• 2  (aspect)  x  3  (prosody)  design

COMPLETED EVENT (PERFECTIVE)EMILY brought  Melissa  a  fancy  drink.  Emily  brought  Melissa  a  fancy  drink.  Emily  brought  MELISSA a  fancy  drink.  

ONGOING EVENT (IMPERFECTIVE)EMILY was  bringing  Melissa  a  fancy  drink.  Emily    was  bringing  Melissa  a  fancy  drink.  Emily  was  bringing  MELISSA a  fancy  drink.  

59

Latin  square  design,  3  items/condition  =  18  experimental  items;  36  fillers

60

Exp5:  Discourse  information

Preambles•Mention  an  alternative  entity  to  the  Source  and  Goal  argumentsà should  ease  the  construction  of  an  alternative  set  (e.g.,  EMILY,  not  Kira)

• Allow  identification  of  Source  and  Goal  entities  to  take  place  before  the  context  sentence• Provide  context  for  the  critical  ongoing/completed  event

Visual  scene• Depicts  the  3rd human  and  an  alternative  theme,  providing  concrete  options  for  contrasting  sentencese.g.:  Emily  brought  Melissa  a  fancy  drink.  She  gave  Kira  a  noisemaker.

61

Exp5:  Trial  structure

preamble The  party  was  held  at  a  bar  near  Emily,  Melissa  and  Kira’s  house.

silence [2500ms]context EMILY was  bringing  Melissa  a  fancy  

drink.silence [2500ms]continuation She obviously  liked  hosting  parties.silence [500ms]

[2000ms]question Who  liked  hosting  parties?

{mouseclick on  box  corresponding  to  answer}

62

Exp5:  PredictionsAspect• Replicate  the  patterns  found  in  Exp2  and  Exp4

Ø Native  speakers  will  look  at  the  Goal  more  following  perfective  than  imperfective  verbs.

Ø Non-­‐native  speakers’  looks  will  not  be  modulated  by  aspect

Prosody• Effects  in  both  groups  during  the  continuation  (cf.  Exp2)  • During  the  silent  region,  native  speakers  will  increase  their  looks  to  prosodically focused  antecedents• Non-­‐native  speakers’  looks  during  the  silent  region  will  not  be  modulated  by  prosody

63

But  note:  the  preambles  could  facilitate  the  generation  of  expectations  in  each  group

collapsing  over  Group,  Aspect,  and  Prosody

Exp5:  Eyegaze overview(N  =  47  native  speakers;  19  non-­‐native  – unbalanced  across  lists)

64

Preamble +silence

[last 1000 ms]

Context sentence:'Emily brought Melissa

a fancy drink.'[silence]

Continuation:'[Ref] obviously liked

hosting parties.'

0.0

0.2

0.4

0.6

0.8

−5000 −2500 0Time from pronoun onset (ms)

Prop

. loo

ks AOISourceGoalThemeAlt. theme3rd human

(preliminary  data)

Exp5:  Focus,  context  regionSource focus Broad focus Goal focus

NativeNon−native

−6000 −5000 −4000 −3000 −2000 −6000 −5000 −4000 −3000 −2000 −6000 −5000 −4000 −3000 −2000

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

Time from pronoun onset (ms)

Prop

. loo

ks

ProsodySource focusBroad focusGoal focus

AOISourceGoalTheme

65

(preliminary  data)

à In  both  groups,  prosodic  focus  increases  looks  to  the  focused  entity

Exp5:  Aspect,  silence  +  continuation  

[silence]Continuation:

'[Ref] obviously liked hosting parties.'

0.0

0.2

0.4

0.6

0.8

−3000 −2000 −1000 0 1000 2000Time from pronoun onset (ms)

Prop

. loo

ks AOI−AspectGoal*−Perf.Goal*−Imperf.Source*−Perf.Source*−Imperf.

66

collapsing  over  Group  and  Prosody

(preliminary  data)

Goal*=  Goal  +  ThemeSource*=  all  other  AOIs

à Collapsing  over  group  and  prosody,  an  anticipatory  pattern  for  Aspect  during  the  silent  region  

Analysis  region

Non−native

Native

−3000 −2000 −1000 0

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

Time from pronoun onset (ms)

Prop

. loo

ks AOI−AspectGoal*−Perf.Goal*−Imperf.Source*−Perf.Source*−Imperf.

67

Non−native

Native

−3000 −2000 −1000 0

0.0

0.2

0.4

0.6

0.8

0.0

0.2

0.4

0.6

0.8

Time from pronoun onset (ms)

Prop

. loo

ks AOI−FocusGoal*−Goal focusGoal*−Source focusSource*−Goal focusSource*−Source focus

Aspect  *  Group:theme  +  silence  region

Focus  *  Group:  theme  +  silence  region

(preliminary  data)

Not  shown:  Broad  focus

à Aspect  pattern  in  both  groups

à Prosody  differs  by  group

Exp5:  Discussion  (tentative)

❓Anticipatory  effect  of  aspect  in  both  groups?❓Aspect  effect  emerges  in  non-­‐native  speakers  with  the  presence  of  preambles,  which  reduce  the  processing  load  during  the  context  sentence?

❓Prosody  during  silence:  too  early  to  say… stay  tuned!(these  preliminary  data  are  unbalanced  across  lists  for  the  prosody  factor,  especially  for  the  non-­‐native  group)

68

General  discussion  and  conclusionNative  speakers• Use  aspect  and  prosody  to  determine  reference• Anticipate reference  on  the  basis  of  aspect…  and  maybe  prosody

Non-­‐native  speakers• Show  a  reduced  ability  (weaker  tendency)  to  generate  expectations…which  is  modulated  by  cue  strength  and  processing  demands• Show  difficulty  with  higher-­‐level  decisions  about  information  structure  (including  focus-­‐sensitive  expectations  for  coherence  relations)Ø Converging  evidence  with  Takeda  et  al,  2015;  Nakamura,  Symposium  talk

Overall• Discourse  processing  draws  on  multiple  sources  of  information• Shaped  by  the  combination  of  linguistic  knowledge,  real-­‐world  knowledge,  and  processing  resources

Ø Research  value  in  carefully  considering  details  of  linguistic  representations  and  online  process  of  constructing/anticipating  them

69

Thanks  to  all  of  you,  and…• NSF  Standard  Grant  BCS-­‐1251450• Co-­‐authors:  Theres Grüter,  Hannah  Rohde,  Amber  Camp,  Wenyi Ling,  and  Aya Takeda• Fabulous  RAs  at  the  University  of  Hawai‘i:  

Bonnie  Fox,  Catherine  Gardiner,  Uy-­‐Di  Nancy  Le,  Victoria  Lee,  Maho Takahashi,  and  (not  pictured)  A.  Blake,  Ivana  Matson,  Eric  Stepans,  and  Alexis  Toliva

70

ReferencesArnold,  J.  E.  (2010).  How  speakers  refer:  The  role  of  accessibility. Language  and  Linguistics  Compass,  4,  

187-­‐203.Braun,  B.  &  Tagliapietra,  L.  (2011).  On-­‐line  interpretation  of  intonational meaning  in  L2.  Language  and  

Cognitive  Processes,  26,  224-­‐235.Carlson,  K.  (2001).  The  effects  of  parallelism  and  prosody  in  the  processing  of  gapping  sentences.  Language  

and  Speech,  44,  1-­‐26.Carlson,  K.  (2002).  Parallelism  and  prosody  in  the  processing  of  ellipsis  sentences.  New  York:  Routledge.Chambers,  C.  G.,  &  Cooke,  H.  (2009).  Lexical  competition  during  second-­‐language  listening:  Sentence  

context,  but  not  proficiency,  constrains  interference  from  the  native  lexicon.  Journal  of  Experimental  Psychology:  Learning,  Memory  and  Cognition,  35,  1029–1040.  

Constant,  N.  (2012).  English  rise-­‐fall-­‐rise:  A  study  in  the  semantics  and  pragmatics  of  intonation.  Ling.  and  Philosophy,  35,  407-­‐442.  

Cummins,  C.,  &  Rohde,  H.  (2015).  Evoking  context  with  contrastive  stress:  effects  on  pragmatic  enrichment.  Frontiers  in  Psychology,  6.

Cutler,  A.  &  Foss,  D.  J.  (1977).  On  the  role  of  sentence  stress  in  sentence  processing.  Language  and  Speech,  20,  1-­‐10.

Dahan,  D.,  Tanenhaus,  M.,  &  Chambers,  C.  G.  (2002).  Accent  and  reference  resolution  in  spoken-­‐language  comprehension.  Journal  of  Memory  and  Language,  47,  292-­‐314.

Dennison,  H.  (2010).  Processing  implied  meaning  through  contrastive  prosody.  (Unpublished  doctoral  dissertation).  Univeristy of  Hawaii,  Manoa.

Dennison,  H.  &  Schafer,  A.  J.  (2017).  Processing  intonationally implicated  contrast  versus  negation  in  American  English. Language  and  Speech,  60, 174-­‐199.

Ferretti,  T.  R.,  Rohde,  H.,  Kehler,  A.,  &  Crutchley,  M.  (2009).  Verb  aspect,  event  structure,  and  coreferentialprocessing.  Journal  of  Memory  and  Language,  61,  191-­‐205.  

Fraundorf,  S.  H.,  Watson,  D.  G.,  &  Benjamin,  A.  S.  (2010).  Recognition  memory  reveals  just  how  contrastive  contrastive  accenting  really  is.  Journal  of  Memory  and  Language,  63,  367-­‐386.

Grüter,  T.,  Rohde,  H.,  &  Schafer,  A.  (2014).  The  role  of  discourse-­‐level  expectations  in  non-­‐native  speakers'  referential  choices.  In  W.  Orman &  M.  J.  Valleau (Eds.)  Proceedings  of  the  38th  Annual  Boston  University  Conference  on  Language  Development  (BUCLD).  (pp.  179-­‐191).  Cascadilla:  Somerville,  MA.

71

References  cont.Grüter,  T.,  Rohde,  H.,  &  Schafer,  A.  (2017).  Coreference and  discourse  coherence  in  L2:  The  roles  of  

grammatical  aspect  and  referential  form.  Lnguistic Approaches  to  Bilingualism.Gundel,  J.  K.,  Hedberg,  N.,  &  Zacharski,  R.  (1993).  Cognitive  status  and  the  form  of  referring  expressions  in  

discourse.  Language,  69,  274–307.  Hirschberg,  J.  &  Ward,  G.  (1991).  Accent  and  bound  anaphora.  Cognitive  Linguistics,  2, 101-­‐121.  Hobbs,  J.  R.  (1979).  Coherence  and  coreference.  Cognitive  Science,  3,  67-­‐90.Husband,  E.  M.,  &  Ferreira,  F.  (2016).  The  role  of  selection  in  the  comprehension  of  focus  alternatives.  

Language,  Cognition  and  Neuroscience,  31(2),  217-­‐235.Ito,  K.  &  Speer,  S.  R.  (2008).  Anticipatory  effects  of  intonation:  Eye  movements  during  instructed  visual  

search.  Journal  of  Memory  and  Language,  58,  541-­‐73.Kaan,  E.  (2014).  Predictive  sentence  processing  in  L2  and  L1:  What  is  different?  Linguistic  Approaches  to  

Bilingualism,  4,  257–282.  Kameyama,  M.  (1999).  Stressed  and  unstressed  pronouns:  complementary  preferences.  In  P.  Bosch  and  R.  

van  der  Sandt (Eds.),  Focus:  Linguistic,  Cognitive,  and  Computational  Perspectives.  (pp.  306–21).  Cambridge  University  Press.  Cambridge.  

Kehler,  A.  (2002).  Coherence,  reference,  and  the  theory  of  grammar.  Stanford,  CA:  CSLI  Publications.Kehler,  A.,  Kertz,  L.,  Rohde,  H.,  &  Elman,  J.  (2008).  Coherence  and  coreference revisited.  J.  of  Semantics  25,  

1-­‐44.Kim,  K.,  Grüter,  T.  &  Schafer,  A.  J.  (2013,  March).  Effects  of  event-­‐structure  and  topic/focus-­‐marking  on  

pronoun  reference  in  Korean.  Poster  presented  at  the  26th  Annual  CUNY  Conference  on  Human  Sentence  Processing,  Columbia,  SC.  

Kim,  K.,  Grüter,  T.  &  Schafer,  A.  J.  (2014,  March).  Effects  of  morphological  and  prosodic  focus  cues  on  topic  maintenance  in  Korean.  Poster  presented  at  the  27th  Annual  CUNY  Conference  on  Human  Sentence  Processing,  Columbus,  OH.

Kurumada,  C.,  Brown,  M.,  Bibyk,  S.,  Pontillo,  D.  F.,  &  Tanenhaus,  M.  K.  (2014).  Is  it  or  isn’t  it:  Listeners  make  rapid  use  of  prosody  to  infer  speaker  meanings.  Cognition,  133,  335-­‐342.

72

References  cont.Rohde,  H.,  Kehler,  A.,  &  Elman,  J.  L.  (2006).  Event  structure  and  discourse  coherence  biases  in  pronoun  

interpretation.  In  R.  Sun  (Ed.),  Proceedings  of  the  28th  Annual  Conference  of  the  Cognitive  Science  Society  (pp.  617-­‐622).  Mahwah,  NJ:  Lawrence  Erlbaum  Associates.

Roberts,  C.  (2012).  Information  structure  in  discourse:  Towards  an  integrated  formal  theory  of  pragmatics.  Sem.  &  Prag.,  5,  1-­‐69.  

Schafer,  A.  J.,  Carter,  J.,  Clifton  Jr.,  C.,  &  Frazier,  L.  (1996).  Focus  in  relative  clause  construal.  Language  and  Cognitive  Processes,  11,  135-­‐63.

Schafer,  A.  J.,  Takeda,  A.,  Camp,  A.,  Rohde,  H.,  &  Grüter,  T.  (2015).  Effects  of  contrastive  intonation  and  grammatical  aspect  on  processing  coreference in  Mainstream  American  English.  Proceedings  of  the  18th  International  Congress  of  the  Phonetic  Sciences,  Glasgow,  Scotland.

Sorace,  A.  (2011).  Pinning  down  the  concept  of  ‘interface’  in  bilingualism.  Linguistic  Approaches  to  Bilingualism,  1,  1-­‐33.

Steedman,  M.  (2000).  Information  structure  and  the  syntax-­‐phonology  interface.  Linguistic  Inquiry,  31,  649-­‐689.

Takeda,  A.,  Anderson,  V.,  Schafer,  A.  J.,  &  Schwartz,  B.  D.  (2015).  Non-­‐native  speakers’  sensitivity  to  prosodic  marking  of  information  structure.  Poster  presented  at  the  28th  Annual  CUNY  Conference  on  Human  Sentence  Processing,  Los  Angeles,  CA.

Takeda,  A.,  Schafer,  A.  J.  &  Schwartz,  B.  D.  (2015).  L2  online  sensitivity  to  English  prosodic  marking  of  new  and  contrastive  discourse  status.  Talk  presented  at  the  40th BUCLD,  Boston,  MA.

Ueno,  M.,  &  Kehler,  A.  (2010).  The  interpretation  of  null  and  overt  pronouns  in  Japanese:  Grammatical  and  pragmatic  factors.  In  S.  Ohlsson and  R.  Catrambone (Eds.),  Proceedings  of  the  32nd  Annual  Meeting  of  the  Cognitive  Science  Society  (pp.  2057-­‐2062).  Austin:  Cognitive  Science  Society.

Ueno,  M.,  &  Kehler,  A.  (2016).  Grammatical  and  pragmatic  factors  in  Japanese  pronoun  interpretation.  Linguistics,  54, 1165-­‐1221.  

73