Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and)...

44
Experience Sampling and Ecological Momentary Assessment Arthur A. Stone Department of Psychiatry and Behavioral Science Stony Brook University Stony Brook, NY USA For the Conference on “The Future of Survey Research,” November 2012 Disclosures: AAS is a Senior ScienNst with the Gallup OrganizaNon and Senior Consultant with PRO ConsulNng, inc.

Transcript of Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and)...

Page 1: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Experience  Sampling  and  Ecological  Momentary  Assessment  

Arthur  A.  Stone  Department  of  Psychiatry  and  Behavioral  Science  

Stony  Brook  University  Stony  Brook,  NY  USA  

For  the  Conference  on  “The  Future  of  Survey  Research,”    November  2012  

Disclosures:  AAS  is  a  Senior  ScienNst  with  the  Gallup  OrganizaNon  and  Senior  Consultant  with  PRO  ConsulNng,  inc.  

Page 2: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Overview  

•  Why  go  into  a  day  in  detail?  

•  Conceptual  posiNon  •  Methods  for  ESM  and  EMA  •  Pros  and  Cons  from  a  survey  perspecNve  

•  AlternaNves  to  ESM/EMA  –  End  of  Day  Diaries  –  Yesterday  Diaries  –  ReconstrucNng  Yesterday  

Page 3: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Why  go  into  a  day  in  detail?  

•  To  capture  data  on  experiences  and  behavior  as  accurately  as  possible  –  To  most  capture  daily  experiences  and  behavior  without  fear  

of  recall  bias  

–  To  generate  high  levels  of  Ecological  Validity  •  To  study  Nme  usage  

–  Experiences  or  acNviNes  changing?  •  To  study  diurnal  paWerns  of  experience  and  behavior  •  To  examine  within-­‐day  contemporaneous  and  lagged  

associaNons  among  experiences  and  behaviors  •  To  link  experienNal  and  behavioral  data  with  real-­‐

Nme,  physiological  data  

Page 4: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Conceptual  posiNon  

Page 5: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Conceptual  PosiNon  Regarding  Measurement  

•  “Experienced  UNlity”  versus  “Remember  UNlity”  •  Immediate  informaNon  is  stored  in  Experien(al  Memory  

–  Stored-­‐lived,  fleeNng  •  Memories  of  experiences  are  stored  in  Episodic  Memory  •  Beliefs  about  experiences  are  stored  in  Seman(c  Memory  

•  As  Recall  Period  increases,  a  shi[  from  ExperienNal/Episodic  to  SemanNc  –  Robinson  &  Clore’s  work  on  reacNon  Nme  

Page 6: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Conceptual  PosiNon  Regarding  Measurement  

•  Cogni(ve  Heuris(cs  impact  recollecNon  –  Rules  of  Thumb,  but  out  of  awareness  

–  Peak-­‐end  –  Current  levels  

•  As  a  proxy  for  recall  period  •  As  altering  accessibility  to  past  memories  

–  Variability  –  Dura(on  neglect  

•  These  issues  are  recognized  by  survey  research  world  and  by  regulatory  agencies  –  Bradburn’s  influenNal  1987  Science  arNcle  on  

autobiographical  memory  in  surveys  

Page 7: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Conceptual  PosiNon  Regarding  Measurement  

•  Concept:  Ecological  Validity  •  SituaNons  need  to  be  

representaNvely  sampled  from  the  relevant  universe  of  situaNons  

•  Bias  can  be  introduced  into  measurement  

Egon  Brunswik  

Page 8: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Conceptual  PosiNon  Regarding  Measurement  

•  Procedures  to  avoid  bias  due  to  recall  –  Limit  recall  period  –  ReconstrucNon/reinstanNaNon  of  recall  period  –  More  precise  quesNons  about  sought  a[er  informaNon  –  Limit  queries  to  informaNon  that  can  be  recalled  

•  e.g.,  salient  events  may  be  possible  to  recall  •  e.g..,  difficulty  with  highly  fluctuaNng  states  

•  ImplicaNons  for  Survey  Research  –  If  only  short  periods  are  reasonable,  how  can  surveys  

incorporate  such  assessments  –  What  do  single,  short-­‐term  assessments  mean?  

•  Are  they  of  any  value?  •  Are  they  reasonable  for  characterizing  groups?  •  What  Ns  are  needed  in  single  assessment  surveys?  

Page 9: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Methods  for  ESM  and  EMA  

Page 10: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Overview  of  CollecNon  Methods  

MORNING   NIGHT  

MORNING   NIGHT  

MORNING   NIGHT  

MORNING   NIGHT   MORNING   NIGHT  

1  

2  

3  

4,  5  

?  

Page 11: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Real-­‐Nme  Data  Capture  

•  Capturing  Real-­‐(me  Experience  –  Experience  Sampling  Method  (ESM)  

•  Csiksezentmahalyi  

•  Paper-­‐and-­‐pencil  –  Ecological  Momentary  Assessment  (EMA)  

•  Context  of  medical  research  

•  GeneralizaNon  of  ESM  to  sampling  other  modaliNes  –  EMA  assesses  phenomena  at  the  moment  they  occur  –  EMA  is  dependent  upon  careful  Nming  of  assessments  –  EMA  methods  usually,  but  not  always,  involve  a  substanNal  number  of  repeated  observaNons  

–  EMA  measurements  are  made  in  the  environments  that  subjects  typically  inhabit  

Page 12: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Electronic  Diary  Screens  

Page 13: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Real-­‐Nme  Data  Capture  

•  PROs  –  No  recall  bias  –  tapping  experiences  –  Mood  can  be  associated  with  environmental  

qualiNes  –  Can  capture  diurnal  rhythms  

•  CONs  –  Point  esNmates  –  Sampling  framework  –  Can  miss  important  events  –  Burdensome/expensive  

•  ModificaNons  –  “Coverage”  model:  Small  retrospecNve  periods  

(minutes/hours)  to  increase  coverage  of  day  

Page 14: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

AlternaNves  to  ESM/EMA  End-­‐of-­‐Day  Diaries  

Page 15: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

End-­‐of-­‐Day  Diaries  

•  EOD  diaries  are  the  oldest  method  for  capturing  near  real-­‐Nme  data  

•  Require  recall  for  up  to  24  hours  –  LiWle  evidence  that  major  bias  due  to  most  heurisNcs  

•  Mode  of  AdministraNon  (applies  to  following  methods)  –  Paper-­‐and-­‐pencil  

•  “Parking  Lot”  compliance  •  Forward-­‐filling  

–  Electronic  diary  •  Time-­‐date  stamp  

–  InteracNve  Voice  Recording  (IVR)  •  Few  visual  cues  

–  Internet  •  Household  availability  of  internet  access  

Page 16: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

End-­‐of  Day  Diaries  

•  PROs  –  Rapid  assessment  of  day’s  experience  

–  Allows  for  extended  length  studies  –  Low  parNcipant  burden  –  RelaNvely  inexpensive  

•  CONs  –  Possible  recall  bias  over  day,  though  liWle  –  Poor  resoluNon  of  day  

•  Diurnal  rhythms  

•  False  PA—NA  associaNon  

–  Compliance  issues  with  some  Modes-­‐of-­‐AdministraNon  

Page 17: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

End-­‐of  Day  Diaries  

•  NaNonal  Study  of  Daily  Experience  –  Part  of  MIDUS  survey  

(Almeida)  –  1,031  respondents;  8  

consecuNve  brief  telephone  interviews  

–  Daily  NA:  Worthless;  Hopeless;  Nervous;  Restless;  Fidgety  

–  Daily  PA:  Cheerful;  In  Good  Spirits;  Extremely  Happy;  Calm  and  Peaceful;  SaNsfied;  Full  of  Life  

Page 18: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

AlternaNves  to  ESM/EMA  Yesterday  Diaries  

Page 19: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Recall  of  Yesterday  

•  Ask  person  to  report  about  yesterday  –  Advantage:  Can  be  administered  anyNme  during  day  

–  Advantage:  Covers  the  enNre  day  –  Disadvantage:  Longer  recall  period  

•  Gallup  OrganizaNon  Daily  Survey  –  Since  2  January  2008,  1,000  RDD  interviews  per  day  –  Several  quesNons  about  Yesterday  –  Did  you  experience  the  following  feelings  during  A  LOT  OF  

THE  DAY  yesterday?  How  about  __________?  NO/YES    

–  Enjoyment,  Sadness,  Stressed,  Worried  

–  Data  presented  is  of  337,000  interviews  conducted  in  2008  

Page 20: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Recall  of  Yesterday  

Page 21: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Recall  of  Yesterday  

Page 22: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Recall  of  Yesterday  

•  PROs  –  Very  brief  assessment  

–  Low  parNcipant  burden  –  Low  cost  

•  CONs  –  Longer  recall  period  –  Current  affect  may  impact  recall  

–  Low  resoluNon  regarding  the  day  

Page 23: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Day  ReconstrucNon  Method  (DRM)  

•  Beyond  EOD  diaries  by  incorporaNng  reconstrucNon  of  recall  period  

•  IntenNon  was  to  develop  a  method  that  could  reproduce  ESM/EMA  from  the  perspecNve  of  yesterday  

•  RaNonale:  ReconstrucNon  of  day  via  Nme-­‐use  –  Kahneman,  D.,  Krueger,  A.  B.,  Schkade,  D.  A.,  Schwarz,  N.,  &  Stone,  

A.  A.  (2004).  A  survey  method  for  characterizing  daily  life  experience:  the  day  reconstrucNon  method.  Science,  306(5702),  1776-­‐1780  

Page 24: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Day  ReconstrucNon  Method  (DRM)  

•  Texas;  Columbus,  OH  and  Rennes,  France  

       N  ≈  800-­‐900;  samples  of  women  age  18+  ini(ally.  •  Respondents  were  given  3  envelopes,  which  were  

opened  sequenNally.    Each  envelope  contained  a  quesNonnaire.    The  quesNonnaires  were  labeled  “Packets”  1-­‐3  – Packet  1  contained  standard  life  saNsfacNon  quesNons  and  

some  demographics.    

– Packet  2  asked  respondents  to  segment  the  preceding  day  (a  work  day)  into  episodes,  like  scenes  in  a  movie  (mean  =  15  episodes);  didn’t  collect  diaries  from  them  at  the  end.  

– Packet  3  asked  detailed  quesNons  about  the  seung  of  each  episode  and  about  their  feelings  (shown  next)  

Page 25: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Day  ReconstrucNon  Method  (DRM)  

Page 26: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Day  ReconstrucNon  Method  (DRM)  

Page 27: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Day  ReconstrucNon  Method  (DRM)  

Page 28: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Day  ReconstrucNon  Method  (DRM)  

Page 29: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  [Princeton  Affect  &  Time  Use  Survey  (PATS)]  

•  24-­‐hour  Nme  diary,  beginning  at  4:00  AM  the  day  before  the    interview  (yesterday)  and  ending  at  4:00  AM  on  the  day  of  the  interview  (today)  

•  Collect  all  acNviNes  engaged  in  during  the  day  with  ATUS  instrument    

•  NIA-­‐supported  “Experimental  Module”:  Randomly  selects  3  awake  episodes  in  proporNon  to  duraNon  (without  replacement)  and  ask  quesNons  about  feelings  a[er  complete  Nme  diary  

Page 30: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  Princeton  Affect  &  Time  Use  Survey  (PATS)  

Page 31: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  Princeton  Affect  &  Time  Use  Survey  (PATS)  

Page 32: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  Princeton  Affect  &  Time  Use  Survey  (PATS)  

Page 33: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  Princeton  Affect  &  Time  Use  Survey  (PATS)  

Page 34: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  Princeton  Affect  &  Time  Use  Survey  (PATS)  

•  Does  DRM  method  faithfully  reproduce  EMA?  –  Small  scale  study  (n=168)  comparing  PATS  with  EMA  

–  Three  days  of  6  EMA  random  reports  –  Following  day  telephone  interview  about  the  day  

Page 35: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

American  Time  Use  Survey  (ATUS)  Princeton  Affect  &  Time  Use  Survey  (PATS)  

•  PROs  –  RelaNvely  rapid  assessment  of  emoNons  

–  Sampling  of  episodes  is  possible  –  Associated  with  Nme-­‐usage  

–  MulNple  modes  of  administraNon  

•  CONs  –  Technologically  sophisNcated  –  Skilled  interviewers  necessary  –  Costly  –  30-­‐45  minutes  

Page 36: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  Methods  for  Survey  Research  

Page 37: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  

•  Several  on-­‐going  efforts  to  create  “survey-­‐friendly”  versions  of  single-­‐day  

•  Context:  Wellbeing  assessment  •  Jacqui  Smith  at  ISR/U  Michigan  

•  Stony  Brook’s  versions  

•  Are  being  tested  in:  –  Health  ReNrement  Survey  

–  English  Longitudinal  Study  of  Aging  –  ROBUST  

Page 38: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  

From  the  ISR  group  

Page 39: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  

From  the  ISR  group  

Page 40: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  

From  the  Stony  Brook  group.  

Page 41: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  Small-­‐scale  study  comparing  affect  for  the  day  between  duraNon-­‐weighted  DRM  and  “overall  yesterday”  assessments.  (N=49).  

SWB-­‐SF:  SubjecNve  Wellbeing,  Short-­‐Form  

Page 42: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  

Page 43: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Transforming  

Page 44: Experience)Sampling)and) Ecological)Momentary)AssessmentExperience)Sampling)and) Ecological)Momentary)Assessment Arthur)A.)Stone) Departmentof)Psychiatry)and)Behavioral)Science) Stony)Brook)University)

Summary  

•  There  is  considerable  interest  in  characterizing  daily  experiences  and  behaviors  in  real-­‐Nme  or  near  real-­‐Nme  

•  There  should  be  a  clear  raNonale  for  aWempNng  to  collect  detailed  daily  informaNon  in  large-­‐scale  surveys,  because  there  are  logisNc  challenges  

•  Several  studies  have  been  successful  with  alternaNves  to  ESM/EMA  

•  However,  it  is  not  yet  enNrely  clear  how  well  EOD  and  Yesterday  approaches  replicate  EMA,  though  there  is  promising  recent  evidence