Handbook of EmotionElicitation and Assessment

Series in Affective Science

Series Editors

Richard J. Davidson

Paul Ekman

Klaus Scherer

The Nature of EmotionFundamental QuestionsEdited by Paul Ekman

and Richard J. Davidson

Boo!Culture, Experience, and the Startle Reflexby Ronald Simons

Emotions in PsychopathologyTheory and ResearchEdited by William F. Flack Jr. and

James D. Laird

What the Face RevealsBasic and Applied Studies of Spontaneous

Expression Using the Facial ActionCoding System (FACS)

Edited by Paul Ekman and ErikaRosenberg

ShameInterpersonal Behavior, Psychopathology,

and CultureEdited by Paul Gilbert and

Bernice Andrews

Affective NeuroscienceThe Foundations of Human and Animal

Emotionsby Jaak Panksepp

Extreme Fear, Shyness, and SocialPhobia

Origins, Biological Mechanisms, andClinical Outcomes

Edited by Louis A. Schmidt andJay Schulkin

Cognitive Neuroscience of EmotionEdited by Richard D. Lane andLynn Nadel

The Neuropsychology of EmotionEdited by Joan C. Borod

Anxiety, Depression, and EmotionEdited by Richard J. Davidson

Persons, Situations, and EmotionsAn Ecological ApproachEdited by Hermann Brandstätter and

Andrzej Eliasz

Emotion, Social Relationships, andHealth

Edited by Carol D. Ryff andBurton Singer

Appraisal Processes in EmotionTheory, Methods, ResearchEdited by Klaus R. Scherer, Angela

Schorr, and Tom Johnstone

Music and EmotionTheory and ResearchEdited by Patrik N. Juslin and

John A. Sloboda

Nonverbal Behavior in Clinical SettingsEdited by Pierre Philippot,

Robert S. Feldman, andErik J. Coats

Memory and EmotionEdited by Daniel Reisberg and

Paula Hertel

Psychology of GratitudeEdited by Robert A. Emmons and

Michael E. McCullough

Thinking about FeelingContemporary Philosophers

on EmotionsEdited by Robert C. Solomon

Bodily SensibilityIntelligent ActionBy Jay Schulkin

Who Needs Emotions?The Brain Meets the RobotEdited by Jean-Marc Fellous and

Michael A. Arbib

What the Face RevealsBasic and Applied Studies of Spontaneous

Expression Using the Facial ActionCoding System (FACS), SecondEdition

Edited by Paul Ekman andErika L. Rosenberg

The Development of Social EngagementNeurobiological PerspectivesEdited by Peter J. Marshall and

Nathan A. Fox

Handbook of Emotion Elicitation andAssessment

Edited by James A. Coan andJohn J. B. Allen

E D I T E D B Y

James A. Coan

John J. B. Allen

Handbook of EmotionElicitation and Assessment

12007

3Oxford University Press, Inc., publishes works that furtherOxford University’s objective of excellencein research, scholarship, and education.

Oxford New YorkAuckland Cape Town Dar es Salaam Hong Kong KarachiKuala Lumpur Madrid Melbourne Mexico City NairobiNew Delhi Shanghai Taipei Toronto

With offices inArgentina Austria Brazil Chile Czech Republic France GreeceGuatemala Hungary Italy Japan Poland Portugal SingaporeSouth Korea Switzerland Thailand Turkey Ukraine Vietnam

Copyright © 2007 by James A. Coan and John J. B. Allen

Published by Oxford University Press, Inc.198 Madison Avenue, New York, New York 10016

www.oup.com

Oxford is a registered trademark of Oxford University Press

All rights reserved. No part of this publication may be reproduced,stored in a retrieval system, or transmitted, in any form or by any means,electronic, mechanical, photocopying, recording, or otherwise,without the prior permission of Oxford University Press.

Library of Congress Cataloging-in-Publication DataHandbook of emotion elicitation and assessment / [edited by] James A. Coan, John J. B. Allen.

p. cm.—(Series in affective science)ISBN 978-0-19-516915-71. Affect (Psychology) 2. Emotions. I. Coan, James A. II. Allen, John J. B. III. Series.BF511.H355 2007152.4072—dc22 2006014136

9 8 7 6 5 4 3 2 1

Printed in the United States of Americaon acid-free paper

www.oup.com

Contributors vii

Introduction: Organizing the Tools and Methods of Affective Science 3James A. Coan and John J. B. Allen

I Emotion Elicitation

1 Emotion Elicitation Using Films 9Jonathan Rottenberg, Rebecca D. Ray, and James J. Gross

2 The International Affective Picture System (IAPS) in the Study of Emotion and Attention 29Margaret M. Bradley and Peter J. Lang

3 The Directed Facial Action Task: Emotional Responses Without Appraisal 47Paul Ekman

4 Emotional Behaviors as Emotional Stimuli 54James D. Laird and Sarah Strout

5 Probing Unconscious Emotional Processes: On Becoming a Successful Masketeer 65Stefan Wiens and Arne Öhman

6 Social Psychological Methods of Emotion Elicitation 91Eddie Harmon-Jones, David M. Amodio, and Leah R. Zinner

7 Emotion Elicitation Using Dyadic Interaction Tasks 106Nicole A. Roberts, Jeanne L. Tsai, and James A. Coan

8 Combining Music With Thought to Change Mood 124Eric Eich, Joycelin T. W. Ng, Dawn Macaulay, Alexandra D. Percy, and Irina Grebneva

9 Emotion Elicited by Primary Reinforcers and Following Stimulus-Reinforcement Association Learning 137Edmund T. Rolls

Contents

10 Emotion Elicitation With Neurological Patients 158Robert W. Levenson

II Emotion Assessment

11 Assessing Positive and Negative Affect via Self-Report 171Elizabeth K. Gray and David Watson

12 The Information-Processing Approach to Emotion Research 184Edward Wilson, Colin MacLeod, and Lynlee Campbell

13 Observer-Based Measurement of Facial Expression With the Facial Action Coding System 203Jeffrey F. Cohn, Zara Ambadar, and Paul Ekman

14 Use of Automated Facial Image Analysis for Measurement of Emotion Expression 222Jeffrey F. Cohn and Takeo Kanade

15 Measuring Emotion-Related Vocal Acoustics 239Michael J. Owren and Jo-Anne Bachorowski

16 The Specific Affect Coding System (SPAFF) 267James A. Coan and John M. Gottman

17 Continuous Measurement of Emotion: The Affect Rating Dial 286Anna Marie Ruef and Robert W. Levenson

18 Assessing Understanding and Appraisals During Emotional Experience: The Developmentand Use of the Narcoder 298Nancy L. Stein and Marc W. Hernandez

19 The Time Sampling Diary (TSD) of Emotional Experience in Everyday Life Situations 318Hermann Brandstätter

20 Methodological Considerations in the Study of Emotion Across Cultures 332David Matsumoto and Seung Hee Yoo

21 Considerations in Studying Emotion in Infants and Children 349Heather A. Henderson and Nathan A. Fox

22 Conceptual, Theoretical, and Methodological Issues in Inferring Subjective Emotion Experience:Recommendations for Researchers 361Lis Nielsen and Alfred W. Kaszniak

III Methods for Understanding the Biological Bases of Emotion

23 Studying Emotion in Animals: Methods, Materials, and Training 379Lisa A. Parr and Katalin M. Gothard

24 The Psychophysiological Laboratory 398John J. Curtin, David L. Lozano, and John J. B. Allen

25 Investigating Human Emotion With Lesions and Intracranial Recording 426Ralph Adolphs

26 Functional Magnetic Resonance Imaging and the Study of Emotion 440Catherine J. Norris, James A. Coan, and Tom Johnstone

Name Index 461

Subject Index 475

vi Contents

Ralph AdolphsDepartment of Neurology, University

of Iowa, andHumanities and Social Sciences,

California Institute of Technology

John J. B. AllenDepartment of PsychologyUniversity of Arizona

Zara AmbadarDepartment of PsychologyUniversity of Pittsburgh

David M. AmodioDepartment of PsychologyNew York University

Jo-Anne BachorowskiDepartment of PsychologyVanderbilt University

Margaret M. BradleyNIMH Center for the Study of

Emotion and AttentionUniversity of Florida

Hermann BrandstätterJohannes-Kepler-University of Linz,

Austria

Lynlee CampbellSchool of PsychologyUniversity of Western Australia

James A. CoanDepartment of PsychologyUniversity of Virginia

Jeffrey F. CohnDepartment of PsychologyUniversity of Pittsburgh

John J. CurtinDepartment of PsychologyUniversity of Wisconsin, Madison

Eric EichDepartment of PsychologyUniversity of British Columbia

Paul EkmanDepartment of PsychiatryUniversity of California, San Francisco

Contributors

Nathan A. FoxDepartment of Human DevelopmentUniversity of Maryland

Katalin M. GothardDepartment of PsychologyUniversity of Arizona

John M. GottmanRelationship Research Institute,

Seattle

Elizabeth K. GrayPsychology DepartmentNorth Park University, Chicago

Irina GrebnevaDepartment of PsychologyUniversity of British Columbia

James J. GrossDepartment of PsychologyStanford University

Eddie Harmon-JonesDepartment of PsychologyTexas A&M University, College Station

vii

Heather A. HendersonDepartment of PsychologyUniversity of Miami

Marc W. HernandezDepartment of PsychologyUniversity of Chicago

Tom JohnstoneWaisman Center for Brain Imaging

and BehaviorUniversity of Wisconsin, Madison

Takeo KanadeRobotics InstituteCarnegie Mellon University

Alfred W. KaszniakPsychology DepartmentUniversity of Arizona

James D. LairdDepartment of PsychologyClark University

Peter J. LangNIMH Center for the Study of

Emotion and AttentionUniversity of Florida

Robert W. LevensonDepartment of PsychologyUniversity of California, Berkeley

David L. LozanoMindware Technologies, Gahanna,

Ohio

Dawn MacaulayDepartment of PsychologyUniversity of British Columbia

Colin MacLeodSchool of PsychologyUniversity of Western Australia

David MatsumotoDepartment of PsychologySan Francisco State University

Lis NielsenNational Institute on AgingNational Institutes of Health

Joycelin T. W. NgDepartment of PsychologyUniversity of British Columbia

Catherine J. NorrisWaisman Center for Brain Imaging

and BehaviorUniversity of Wisconsin, Madison

Arne ÖhmanDepartment of Clinical NeuroscienceKarolinska Institute, Stockholm

Michael J. OwrenDepartment of PsychologyCornell University

Lisa A. ParrYerkes National Primate Research

Center

Alexandra D. PercyDepartment of PsychologyUniversity of British Columbia

Rebecca D. RayDepartment of PsychologyStanford University

Nicole A. RobertsDepartment of PsychologyUniversity of Wisconsin, Milwaukee

Edmund T. RollsDepartment of Experimental

PsychologyUniversity of Oxford

Jonathan RottenbergDepartment of PsychologyUniversity of South Florida

Anna Marie RuefBoston Veterans Administration

Medical Hospital

Nancy L. SteinDepartment of PsychologyUniversity of Chicago

Sarah StroutDepartment of PsychologyClark University

Jeanne L. TsaiDepartment of PsychologyStanford University

David WatsonDepartment of PsychologyUniversity of Iowa

Stefan WiensStockholm University and Karolinska

Institute

Edward WilsonSchool of PsychologyUniversity of Western Australia

Seung Hee YooDepartment of PsychologySan Francisco State University

Leah R. ZinnerDepartment of PsychologyUniversity of Wisconsin, Madison

viii Contributors

Handbook of EmotionElicitation and Assessment

This page intentionally left blank

Introduction

Organizing the Tools and Methods of Affective Science

James A. Coan

John J. B. Allen

3

Affective science—the scientific study of emotion and emo-tion-related processes—is now a mature domain of inquiry,with its own standardized measures, induction procedures,data analytic challenges, subdisciplines, core theoretical de-bates, and so on. Though long associated with psychology,researchers in the affective sciences can now be found in avariety of disciplines. Psychologists, biologists, sociologists,geneticists, neuroscientists, ethologists, economists, behav-ioral ecologists, and even physicians each contribute theirspecific expertise like pieces of a puzzle, because emotionsdistribute their echoes and effects at every one of these lev-els and more.

A cursory scan of titles in the Oxford University PressSeries in Affective Science bears this out. By 2003, the mam-moth Handbook of Affective Sciences (Davidson, Scherer, &Goldsmith, 2003) offered as near an exhaustive overview ofthe field as we are likely to see soon, making it clear that emo-tions are implicated in domains of inquiry as broad and di-verse as brain-behavior relationships, behavior genetics,personality, social bonding and interaction, evolution, cul-ture, decision making, psychopathology, and health. To date,both the series and the Handbook of Affective Sciences havefocused on overviews of results and explications of theoreti-cal developments in this diverse field. Few texts devote spaceto explicit discussions of the empirical tools and method-ological challenges that collectively allow emotion researchto proceed, despite the fact that the study of emotion oftenrequires highly specialized designs, instruments, and strate-

gies. Indeed, new techniques are proliferating at an impres-sive rate, often without heed to the specific incremental ad-vantages they may or may not offer. Well-validated andsubstantially understood measures are frequently neglectedin favor of expeditious progress, convenience, or both. Thefield is now at a point at which a large number of excellentelicitation procedures and assessment approaches exist, andthe broader application of these procedures and approachesstands to increase interlaboratory standardization and, bydoing so, to increase the speed and accuracy with whichemotion research can be communicated among peers. Inshort, it is time for a handbook that organizes and details themajor methodological accomplishments of this multifacetedfield, and that is what we hope to provide with the Hand-book of Emotion Elicitation and Assessment.

In publishing this handbook, however, we emphaticallydo not wish to stifle the creative development of innovativemethods. On the contrary, our hope is that this book willaccelerate the development of new elicitation and assessmentprocedures by discouraging researchers from reinventingapproaches for which sufficient resources currently exist. Inadvocating the expanded use of standard and well-researchedmethods and techniques, we, in fact, hope to encouragecurrent and future emotion researchers to cast their collec-tive gaze with greater creativity and determination towarddomains that lie beyond the field’s current reach.

It is also our intent that this handbook serve as some-thing more than a mere collection of tools. Some chapters

4 Handbook of Emotion Elicitation and Assessment

address broad methodological problems (e.g., working withinfants and children, measuring subjective emotional expe-rience). Others seek to assist the broader community of af-fective scientists in being critical and conversant on topicsstill beyond the scope of many laboratories (e.g., compara-tive research, functional neuroimaging). Still others reviewand recommend general methodological orientations andstrategies (e.g., thinking like a social psychologist in design-ing emotion elicitations). Ultimately, this collection shouldserve as a pragmatic resource for emotion researchers in needof both specific guidance and general advice. Scales are pre-sented and described, stimuli are reviewed in a methodologi-cal context, coding systems and detailed assessment tools aresuggested, innovative methodologies are proposed, currentmethodological problems are highlighted, and general rec-ommendations are expressed. It is a book of resources—akind of bookshelf consultant—for the affective scientist, orfor the scientist whose foray into the affective sciences maybe in its beginning stages.

The volume is organized into three general sections. Thefirst addresses the laboratory elicitation of emotion, the sec-ond discusses the assessment of emotion, and the thirdfocuses on methods that support research on emotion’s po-tential biological underpinnings.

Part I: Emotion Elicitation

Section I (chapters 1–10) covers a diversity of strategies foreliciting emotion in the laboratory. The selection of topicsunder elicitation procedures reflects our view that no singlesensory domain is paramount—that indeed emotions canarise through a variety of modalities and that these modali-ties (even some that can be controversial) need to be wellunderstood and properly implemented if comparable resultsacross laboratories can be achieved.

Opening this section, Rottenberg, Ray, and Gross (chap-ter 1) offer detailed instructions on the acquisition and useof standardized emotional film clips, and Bradley and Lang(chapter 2) discuss the proper use of their well-known In-ternational Affective Picture System (IAPS). Other chaptersprovide detailed instructions and recommendations for us-ing emotional behavior. Ekman (chapter 3) provides detailedrecommendations for using his Directed Facial Action (DFA)task, and Laird and Strout (chapter 4) review the implemen-tation of what they refer to as a “family” of elicitation tech-niques that utilize emotional behaviors. Wiens and Öhman(chapter 5) clarify the often murky methodological waterssurrounding the elicitation of “unconscious” emotion viamasking techniques and provide invaluably specific recom-mendations for their use.

Of course, although essential (especially when imple-mented properly), films, images (perhaps especially maskedones), and emotional behaviors can be limited in their abil-ity to elicit intense forms of emotion in the laboratory or,

indeed, some forms of emotion altogether. Famous amongthe elicitation-resistant laboratory emotions is anger, but theelicitation of intense or “authentic” emotional responses inthe laboratory is, in general, challenging. Harmon-Jones,Amodio, and Zinner (chapter 6) offer recommendations forthe use of strategic laboratory social interactions in the elici-tation of emotion with high levels of both experimental con-trol and ecological validity. In chapter 7, Roberts, Tsai, andCoan review a specific strategy for eliciting emotional reac-tions that are not only authentic but often quite intense. Theirdyadic interaction task offers the possibility of studying emo-tional processes in social contexts and at levels of intensitythat are often beyond what is achievable—either method-ologically or ethically—using ordinary laboratory challenges.

Eich, Ng, Macaulay, Percy, and Grebneva (chapter 8)describe their MCI technique, a theoretically based elicitationprocedure that places music (M) and contemplation (C) inan idiographic (I) context, such that individuals are not sub-jected to other experimental procedures until they are known(via occasional measurements) to be sufficiently induced intoa target mood. Their instructions include lists of widely avail-able pieces they have used for this purpose, and with theadvent of online music stores (e.g., iTunes), acquisition ofthis music has become inexpensive and convenient.

Rolls (chapter 9) reviews the use of primary reinforcers inthe elicitation of emotion and emotional processes (e.g., physi-ological processes associated with emotion). In providing hisrecommendations, he also outlines a broad model of emotionsas reflecting brain systems that respond to rewards and pun-ishments. In this way, Rolls invokes fundamental evolution-ary processes that foreshadow later chapters on the biologicalunderpinnings of emotion and emotional responding. Simi-larly, Levenson (chapter 10) reviews methodological issues inthe elicitation of emotion in neurological patients. The readeris particularly referred to this chapter for Levenson’s excellentreview of a broad array of elicitation techniques, many of whichdo not receive dedicated chapters in this volume.

Part II: Emotion Assessment

Emotion elicitation is of little use if we are incapable of re-liably measuring emotional responses. Leading off this sec-tion, Gray and Watson (chapter 11) describe the history andproper use of a variety of paper-and-pencil scales designedto assess both state and trait affect, with particular emphasison the widely used Positive and Negative Affect Schedule(PANAS). It would be hard to overestimate the impact of suchscales, especially because the reliable and valid measurementof subjective experience is such a formidable task (as notedby Nielsen and Kaszniak later in chapter 22). Wilson,MacLeod, and Campbell follow this in chapter 12 with adiscussion of information-processing approaches to under-standing emotion and emotional effects. This work offers anew and fruitful heuristic for understanding the role of emo-

Introduction 5

tion in a variety of cognitive contexts, with a specific focuson our conceptual understanding of cognitive vulnerabili-ties to affective disorders such as depression and anxiety.

Cohn, Ambadar, and Ekman (chapter 13) introduce be-havior coding with their review of the Facial Action CodingSystem (FACS). Their chapter discusses the major facial ele-ments associated with emotional expression and describesprocedures for extended training and certification in FACScoding. In chapter 14, Cohn and Kanade describe their im-portant ongoing work in the domain of automated facialanalysis (AFA), an emerging technology that holds the prom-ise of automating FACS coding. Owren and Bachorowski(chapter 15) present methods for measuring a complimen-tary mode of emotional expression in their comprehensivediscussion of vocal acoustics associated with emotional re-sponding, and, synthesizing across channels of expression,Coan and Gottman (chapter 16) describe the Specific Affect(SPAFF) Coding System for coding emotional behavior at theconstruct level. This system trains coders to be constantlymindful not only of emotional facial expressions but also ofvocal acoustic properties and verbal content.

Experience sampling and the analysis of time are of increas-ing interest to affective scientists, and with good reason. Tech-nical and data analytic advances in recent years have made thestudy of these important variables—long neglected as theyhave been—more available and affordable. Ruef and Levenson(chapter 17) describe a device for continuous, fluid reportingof emotional responding through time called the affect ratingdial. The basic design and implementation of this tool offers ahost of advantages to researchers interested in the chronotropyof emotional responding, the study of psychophysiologicalcoherence in emotional responding, and the synchrony ofemotional responding among interacting dyads. Stein andHernandez (chapter 18) then describe the Narcoder, a pro-gram that allows researchers greater access to the richness ofemotional thoughts and language, as well as the ways in whichverbal content can indicate emotional understanding and sub-jective well-being. In chapter 19 Brandstätter brings the analy-sis of time together with a detailed look at subjective emotionalexperience in his discussion of the time sampling diary (TSD).Brandstätter has been a pioneer in applying methods of fre-quent experience sampling across time and situations. Hischapter offers clear instructions for implementing his particularapproach, including innovative data analytic methods for usewith TSD data, once collected. Researchers will find his ap-proach broadly applicable, as well as a useful foundation forfuture iterations of time sampling methodologies.

Practical methodological concerns associated with spe-cial populations and problems are highlighted in the nextthree chapters. In chapter 20, Matsumoto and Yoo walk thereader through the myriad methodological issues that fre-quently arise in the study of emotion across cultures. Matsu-moto and Yoo take the reader beyond simple two-culturecomparison studies, offering descriptions of different typesof cultural comparisons, as well as recommendations for

maximizing the interpretability, repeatability, and impact ofsuch studies. Similarly, Henderson and Fox (chapter 21) offera host of pragmatic and theoretical recommendations for thesound study of emotion in infants and children. Develop-mental milestones (e.g., in social-cognitive development andattentional control) hold implications for the kinds of ques-tions that can likely be asked in such populations, and thevalidity of parental reports of child and infant emotionalityraise additional methodological and pragmatic concerns.Ethical issues also come to the fore in dealing with infantsand children in ways that are not as common with adultpopulations. Finally, Nielsen and Kaszniak (chapter 22) raisea number of vexing questions about the nature and measure-ment of subjective emotional experience. These questionsrange from how well such reports actually capture the rich-ness of experience to whether such reports are capable ofproviding insights about underlying emotional events, espe-cially at the level of physiology. Rather than cautioningaffective scientists to shy away from measuring emotional ex-perience (as, they note, so many have done), Nielsen andKaszniak celebrate the “rich phenomenality of emotion” andcall on researchers to expand their notions of experiencemeasurement to include dimensions of motivational, percep-tual, and cognitive awareness and to explore alternative re-porting methodologies.

Part III: Methods for Understandingthe Biological Bases of Emotion

When first considering a section on methods for understand-ing the biological bases of emotion, we quickly realized thatthe section could easily expand into a book of its own. In fact,several related books currently exist (e.g., Cacioppo, Tassinary,& Berntson, 2000), few or none with an explicit emphasis onemotion. We heartily recommend these books to affective sci-entists intent on exploring the biology of emotion. We alsonote, however, that emphasis on the biological bases of emo-tion—and of virtually every other domain of psychologicalscience —is on the rise, both in the media and as a function offunding priorities at most granting agencies. At the very least,affective scientists now need to be conversant on a varietyof topics related to the physiology and evolution of emo-tion. Chapters in this section thus satisfy a number of practi-cal goals.

Parr and Gothard (chapter 23) introduce readers to theassumptions, general methods, and ethics of conducting re-search on emotion in nonhuman primates. Much of what theyhave to say is applicable to a wide range of animals. Theirchapter sets the tone for many in this section in that its func-tion is to acquaint readers with what they will need inorder to gain access to and become proficient with nonhu-man primate populations, as well as how to become betterconsumers of the literature in this area. In chapter 24, Curtin,Lozano, and Allen provide readers with a quite comprehen-

6 Handbook of Emotion Elicitation and Assessment

sive introduction to the psychophysiological laboratory. Thispragmatic chapter approximates a manual for readers, assum-ing sufficient funding, to actually set up a psychophysiologicallaboratory of their own. The authors note that psychophysi-ological equipment continues to become both cheaper andbetter with each passing year, thus making it easier for therelative novice to break in.

The two final chapters introduce readers to two broaddomains of affective neuroscience: work with lesion patientsand intracranial recording and functional magnetic resonanceimaging (fMRI). Both provide sufficient methodologicalknowledge that interested readers should become signifi-cantly better consumers of this literature, but both may alsoserve as excellent introductions to these domains for indi-viduals intending to actually commence research in theseexciting areas. Moreover, as more affective scientists forgecollaborations with individuals who have access either to rareneurological patient populations or hugely expensive neuro-imaging facilities, these chapters can serve to provide a depthof understanding of these approaches that exceeds the levelof relatively informal meetings and conversations but fore-goes very steep investments in formal training and time.

Adolphs begins, in chapter 25, with descriptions of typesof lesion studies while noting that lesion methods in humansare absolutely vital for the establishment of causal links be-tween neural activity and behavior. Indeed, he asserts (andwe agree) that other methods, such as fMRI, are as sole mea-sures incapable of identifying causal links between neuralstructures and emotional behaviors. (This alone is an impor-tant methodological lesson, worthy of a great deal of empha-sis.) As Adolphs makes abundantly clear, lesion methodshave grown extremely sophisticated and stand to make pow-erful contributions to our understanding of the neurobiol-ogy of emotion. Norris, Coan and Johnstone (chapter 26)follow with their discussion of functional neuroimaging inthe study of emotion. With their chapter, Norris and col-leagues strive not only to provide methodological informa-tion critical to the design, analysis, and understanding offMRI research but also to place fMRI in a broader concep-tual context, commenting on the impact fMRI has hadon the field of affective science. They note that althoughfMRI has provided unprecedented access to neural pro-cesses, inferential overreaching and misunderstanding hasfrequently occurred. They discuss the reasons for such mis-understanding and make recommendations for avoiding itin the future.

Our Core Assumption: Methods Follow Questions

In assembling a book that emphasizes tools and methods,we were gratified to have one of our core assumptions con-firmed by our contributing authors: that the great diversity

of methodological approaches represented here reflects thefact that affective science is highly theory-driven. Althoughsome have argued—and it is doubtless true—that the toolsof science themselves often serve as powerful heuristics fornew theoretical developments in the sciences (Gigerenzer,1991), we find this to be somewhat less true (but not en-tirely untrue) of the study of emotion. Perhaps because“grand theories” of emotion have existed for some time, toolshave often been adapted, co-opted, and developed for thepurpose of settling contentious debates or disconfirmingcommonly held notions; that is, with prespecified proposi-tions in mind. It is perhaps for this reason that the study ofemotion has long served as a magnet for interdisciplinarycollaboration. It is increasingly apparent that a completeunderstanding of any emotional process is going to requireattention paid to multiple levels of analysis, from the culturalto the behavioral, psychological, experiential, physiological,and molecular. This practice will increase in frequency in thecoming years and decades, with significant progress comingfrom the use of a wide variety of approaches, within andacross studies. We find this prospect exciting and hope thatreaders of this volume will begin to enjoy a greater familiar-ity with a diversity of methods, all the while maintaining afocus on questions they are interested in answering as op-posed to the specific methods they are interested in using.

Conclusion

We said earlier that we hoped, with the Handbook of EmotionElicitation and Assessment, to provide a volume that organizesand details the major methodological achievements in theaffective sciences. We feel that this mission has been verynearly accomplished—“very nearly” because there are somany more methods and approaches that we would like tohave included (e.g., quantitative and molecular genetics, sta-tistical methods for affective scientists, etc.) but that took usa step or two too far from our original purpose. Neverthe-less, our sincere hope is that readers will discover a substan-tial trove of practical assistance between these covers and thatthis assistance will prove useful for years to come.

References

Cacioppo, J. T., Tassinary, L. G., & Berntson, G. G. (Eds.).(2000). Handbook of psychophysiology. New York: CambridgeUniversity Press.

Davidson, R. J., Scherer, K. R., & Goldmsith, H. (Eds.). (2003).Handbook of affective sciences. New York: Oxford UniversityPress.

Gigerenzer, G. (1991). From tools to theories: A heuristic ofdiscovering in cognitive psychology. Psychological Review,98, 254–267.

IEmotion Elicitation

This page intentionally left blank

1Emotion Elicitation Using Films

Jonathan Rottenberg

Rebecca D. Ray

James J. Gross

9

Research on emotion has undergone explosive growth dur-ing the past few decades, marked by new theories (e.g., evo-lutionary analyses; Tooby & Cosmides, 1990), methods (e.g.,anatomically based systems for coding facial expressivebehavior; see chapter 13, this volume), and findings (seeCacioppo & Gardner, 1999). Some of the research in thisarea has been correlational, focusing on factors that naturallycovary with emotional processes, such as chronological age,physical health, or social status. However, experimental re-search also has flourished, focusing on emotional processesin the context of relatively well-controlled laboratory envi-ronments. Our chapter on the use of emotion-eliciting films,like many of the contributions to the Handbook of EmotionElicitation and Assessment, lies squarely within this second,experimental tradition.

Scientists who take an experimental approach have hadat least two distinct motives for eliciting emotion. First, emo-tion has been used as an independent variable in manipula-tions that demonstrate the important contribution made byemotion to a diverse array of phenomena, ranging from ag-gression (Zillman & Weaver, 1999) to helping behavior (Isen,Daubman, & Nowicki, 1987). Second, emotion has beenused as a dependent—or outcome—variable in work thathas illuminated several emotion-related phenomena, suchas self-reported experience (Duclos & Laird, 2001), facialexpressive behavior (Ekman, Friesen, & Ancoli, 1980), au-tonomic or central nervous system activation (LeDoux,1996; Levenson, 1988), and individual differences in emo-

tion responding (e.g., Gross, Sutton, & Ketelaar, 1998;Rottenberg, Kasch, Gross, & Gotlib, 2002).

Whatever their motivation for studying emotion, experi-mentalists have required a reliable means of eliciting emo-tion in an ethically acceptable fashion. Happily, investigatorshave made vast improvements over the buckets of frogs(Landis, 1924) and other ad hoc measures of the past, mov-ing toward more tightly controlled and replicable emotionelicitation procedures. Indeed, as the other chapters in thisvolume attest, many different emotion elicitation techniqueshave now come to fruition, including images and sounds(Bradley & Lang, chapter 2; Wiens & Öhman, chapter 5, thisvolume), expressive behavior (Ekman, chapter 3; Laird &Strout, chapter 4, this volume), scripted and unscripted so-cial interactions (Harmon-Jones, Amodio, & Zinner, chap-ter 6; Roberts, Tsai, & Coan, chapter 7, this volume), andmusic (Eich, Ng, Macaulay, Percy, & Grebneva, chapter 8,this volume).

The development of films as emotion elicitors has paral-leled this wider maturation of emotion science. For much ofthe past half-century, researchers have selected individualfilm clips using relatively informal criteria, often to elicit adiffuse state of anxiety or stress (e.g., Lazarus, Speisman,Mordkoff, & Davison, 1962). In fact, until recently, the em-pirical record concerning films was scanty, prompting res-ervations about the reliability and validity of film-basedemotion induction procedures (Polivy, 1981). There havebeen two notable efforts to build a scientific database

10 Emotion Elicitation

concerning films by formalizing film selection criteria andassembling a standardized library of emotion stimuli capableof eliciting specific emotional states. Philippot (1993) pre-sented normative viewing data (N = 60) from a set of 12 filmclips that elicited six emotional states and reported successfor stimuli that elicited amusement, sadness, and a neutralstate. Gross and Levenson (1995), also working from a dis-crete emotions perspective, presented normative viewing data(N = 494) from 16 films targeting eight emotions and re-ported success for stimuli that elicited amusement, anger,contentment, disgust, sadness, surprise, a neutral state, and,to a lesser extent, fear.

As emotion science has matured, the palette of viableemotion elicitation techniques has grown. Increasingly, inves-tigators face a baffling array of techniques to elicit emotion.Unfortunately, the published literature offers little explicitguidance on these issues, forcing investigators to base theirdecisions on informal rules of thumb, idiosyncratic trainingexperiences, or personal communications. With these needsin mind, we intend this chapter to be a guide for investigatorscontemplating the use of short film clips to elicit emotion. Inthe sections that follow, we first outline the general task ofeliciting emotion in a laboratory. Second, we compare theproperties of films to other laboratory emotion induction pro-cedures. Third, we discuss how to use films in different ex-perimental contexts and avoid potential pitfalls. Fourth, weoffer examples of films that have worked well in our own re-search applications. Finally, we close with reflections on thefuture evolution of films as an emotion elicitation procedure.

Eliciting Emotion in the Laboratory

Historically, the task of eliciting emotion in the laboratory hasbeen made all the more difficult because emotion itself has beensuch an elusive construct. With different investigators employ-ing their own idiosyncratic and often widely discrepant defi-nitions of emotion, it is no wonder that there has beenconsiderable confusion as to which procedures reliably elicitemotion. Fortunately, affective scientists have increasinglymoved toward a consensual understanding of key affectiveprocesses, a move that has greatly facilitated the systematicstudy of these processes (Ekman, 1992; Russell, 1991).

Here we follow common usage and view emotions as atransient but coordinated set of responses that occur whenan individual faces a situation (real or imagined) that is rel-evant to salient personal goals. Like others, we view emotionsas being multicomponential, typically involving changes incognitive, experiential, central physiological, peripheralphysiological, and behavioral response systems (Lang, 1978).For example, anger may be reflected by thoughts concern-ing revenge, feelings of great distress, an elevated heart rate,and an attack on the source of one’s anger. Finally, like othercontemporary researchers, we distinguish emotions fromother affectively laden concepts. For example, compared with

an emotion, a mood is a longer, slower moving state that isless tied to specific objects or elicitors (Watson, 2000).

Guided by this conception of emotion, laboratory emo-tion elicitation procedures include a broad array of effortsto evoke a brief affective response in one or more emotionresponse systems via some type of stimulus. Figure 1.1 high-lights several of the key features of the emotion-generativeprocess as it unfolds during a laboratory elicitation proce-dure (whatever that procedure may be).

The left of the figure draws attention to the fact thatemotions elicited in the laboratory usually are not created denovo but rather arise from preexisting affective states. Indeed,affect has been understood as a stream with a continuous,or tonic, output (e.g., Russell, 2003). Emotions occur asbursts of activity, or waves, superimposed against the back-drop of this affective stream. The interaction between pha-sic emotion and tonic affect is not well understood and isclearly an important avenue for future research (Neumann,Seibt, & Strack, 2001). Indeed, an inspection of Figure 1.1suggests that it is quite difficult to distinguish the waves ofemotion from background affective tone, as there are nononarbitrary criteria for deciding when one phenomenonends and the other begins. Finally, the different “peaks”within Figure 1.1 illustrate a related complexity—that emo-tional impulses have no uniform signature and instead ex-hibit a variable duration and morphology. The notion ofaffective chronometry (Davidson, 1998) signals the emerg-ing scientific interest in characterizing the variability of theemotion waveform by decomposing it into a number of tem-poral subcomponents, such as latency, rise time, magnitude,duration, and offset.

It should be noted that Figure 1.1, although useful forheuristic purposes, simplifies the emotion-generative processin many ways. For example, emotional responses are plot-ted with a single line, a representation that assumes that dif-ferent emotion response elements (e.g., emotion experience,behavior, and physiology) exhibit synchrony during emotionactivation. Theoretically, a high degree of synchrony betweenemotion response systems has been seen as essential in help-ing an organism mobilize a response to a challenge (e.g., makea fast getaway when a bear charges; Levenson, 1999). Em-

Magnitude

Time

Start Stop

Figure 1.1. Affective responding over the course of a labora-tory emotion elicitation procedure.

Emotion Elicitation Using Films 11

pirically, however, it is clear that emotional response systemsdo not covary perfectly in their activity (Gross, John, &Richards, 2000; Lang, 1978), with modest intercorrelationsbetween emotion response systems often obtained (Mauss,Wilhelm, & Gross, 2004; Ruch, 1995), and even dissociatedactivity observed in some settings (e.g., Lacey, 1967).

Another limitation of Figure 1.1 is that it does not acknowl-edge the role of individual differences. A growing body offindings demonstrates that individual differences influenceemotion generation at every stage of the process. These influ-ences occur as a function of affectively toned traits such asdispositional mood (Watson, Clark, & Tellegen, 1988), emo-tional reactivity (Kagan & Snidman, 1999), emotion regula-tory styles (Gross, 1998), and metaemotions (Salovey, Mayer,Goldman, Turvey, & Palfai, 1995), as well as personality traits(Larsen & Ketelaar, 1991; Gross et al., 1998), physical healthstatus (Ritz & Steptoe, 2000), and other subject characteris-tics such as gender, race, class, and culture (e.g., Vrana &Rollock, 2002). Individual (and group) differences representan important theme to which we return later.

Because emotion is a multifaceted process about whichso much is still unknown, no single technique can serve allpurposes for eliciting emotion in the laboratory. Thus thevery same properties of a given induction procedure thatmake it a valuable procedure to probe emotion in many con-texts may represent liabilities in others. With an eye to help-ing investigators decide whether and how films might servetheir research aims, we now compare films with other com-mon elicitation procedures.

Using Films to Elicit Emotions

A nonexhaustive list of procedures that have been used toelicit emotion in the laboratory includes: images, sounds,self-statements, facial and body movements, scripted andunscripted social interactions, hypnosis, drugs, relived orimagined scenes, music, and odors (see Martin, 1990). Theseprocedures differ from one another in a large number of ways.To facilitate comparisons, we highlight seven key dimensionsthat are salient to the selection and use of these procedures.The approximate location of films along these seven dimen-sions is presented in Figure 1.2.

Intensity

Emotional intensity can be viewed in several ways. Forpresent purposes, we consider the intensity of emotionalresponses in terms of the two conceptually separable (butoften correlated) dimensions of (1) response strength and (2)response breadth.

Experimentalists face important ethical constraints re-garding the strength of the emotional responses they mayelicit, even when participants are carefully selected and de-briefed. The experience of intense negative emotions can be

painful and traumatic, and even the experience of intensepositive emotions can be associated with a loss of control thatis aversive. Films are capable of eliciting mild or strong emo-tional responses. For a number of positive and negativeemotions, films rival or exceed the response strength that canbe elicited ethically with other procedures.

The relative potency of films may be due in part to theintrinsic power of carefully crafted, external, and dynamicallyvarying stimuli. We suspect it is also due to the relativelypermissive cultural mores that surround film emotion. In theUnited States and other Western countries, cinema and tele-vision traffic widely in graphic and emotionally explicit ma-terial, and it is likely that the presence of these media givesexperimenters who use films a somewhat freer hand to elicitstrong responses (even negative ones) without creating asense of harm or ethical violation. By contrast, other proce-dures, such as hypnosis or confederate procedures that elicitstrong negative states, may be readily perceived as deceptiveor manipulative, and the effects of these induction proce-dures may be quite difficult to remove via debriefing (Ross,Lepper, & Hubbard, 1975).

There is also good evidence that films are capable of elic-iting activations across many of the response systems asso-ciated with emotion (e.g., experience, behavior, autonomicand central physiology; Averill, 1969; Gross & Levenson,1993; Karama et al., 2002; Palomba, Sarlo, Angrilli, Mini, &Stegagno, 2000). To the extent that investigators want to elicitbroad, multisystem responses, films may hold advantages

intensity

complexity

demand characteristics

standardization

temporal resolution

ecological validity

attentional captureFigure 1.2. Films as an emotion elicitation procedure.

12 Emotion Elicitation

over other procedures. Anecdotal evidence suggests that self-statements have somewhat weaker effects on behavior andphysiology; facial movement has relatively weaker effects onexperience; and music has relatively weaker effects on physi-ology. Published data indicate that, whereas some film clipsare capable of generating multi-system activations (e.g., Rot-tenberg, Gross, Wilhelm, Najmi, & Gotlib, 2002), many filmclips will not. One difficulty in sorting out this issue is thatfilms, like many elicitation procedures, are generally normedonly on the basis of self-reported emotion experience (a limi-tation of film validation procedures that is discussed morefully later). In light of the relatively loose coupling amongemotion response systems, even the most robust self-reportednorms provide no guarantee that a film will generate behav-ioral and physiological activations.

Complexity

When considering the complexity of films, what is perhapsmost striking is the variability of film clips on this dimension.A film can be a still, silent image, such as a fixation cross or testpattern, or a dynamic visual and auditory sequence that de-picts complex themes. In their traditional incarnation, the filmclips that are used for emotion induction tend to be dynamic,multimodal, and reliant on meaningful narrative. Given thesefeatures, therefore, most emotion-eliciting films will be relativelyhigh in cognitive complexity. Indeed, even a very simple film,such as a sequence depicting an arm being amputated (e.g.,Gross & Levenson, 1993) likely requires considerably moreappraisal (Frijda, 1988) than competing procedures to elicitdisgust, such as ingesting a bitter taste (reliant on a primitivereflex). On the one hand, films share with other narrative-basedprocedures (e.g., relieved emotion tasks) the ability to elicitcognitively sophisticated emotional states such as nostalgia. Onthe other hand, films impose relatively high levels of cognitivedemand on participants—potentially a drawback when test-ing special populations, such as young children, infants, orcognitively impaired adults (e.g., those with schizophrenia).

Attentional Capture

Emotion elicitation procedures also vary in how much of par-ticipants’ attention they require to operate. Masked stimulimake limited demands on attention and operate on emotionalmost totally outside of participants’ conscious awareness (seeWiens & Öhman, chapter 5, this volume). By contrast, theVelten (1968) procedure directs participants to read a seriesof emotional self-statements, a task with high attentional re-quirements that preclude performance of most other concur-rent tasks (e.g., filling out questionnaires, answering interviewquestions). As a dynamic display that engages both visual andauditory modalities, film clips are also typically fairly high inattentional capture. In fact, film effectiveness can be degradedby competing demands placed in either modality (e.g., gradi-ent coil noise). Therefore, films may be a suboptimal proce-

dure in experimental settings in which participants must carryout a secondary task (e.g., mental arithmetic).

Demand Characteristics

Films are embedded in experimental contexts that vary con-siderably in their level of demand. Some experimental coverstories are more likely to provoke demand (e.g., the film is partof a neuroticism test) than others (e.g., the film is part of amemory test). Likewise, the specific instructions that accom-pany film viewing also influence demand. For example, beforeshowing a sad film, Marston, Hart, Hileman, and Faunce (1984,page 128) presented the instructions, “Let yourself experiencewhatever emotions you have as fully as you can, don’t try tohold back or hold in your feelings.” In part out of demandconcerns, we (for this film, The Champ, and for others) use thesimpler instructions to “please watch the film carefully.” Finally,film demand characteristics are also film-content-dependent.A film depicting a man eating dog feces suggests a fairly trans-parent intent to elicit disgust, whereas the intent of a film de-picting landscape scenery is relatively opaque.

Bearing all of these caveats in mind, film clips can elicitemotion with relatively low levels of demand. Furthermore,films often generate effects in response systems that are typi-cally seen as being outside of participants’ volitional control(e.g., heart rate). By contrast, procedures such as self-state-ments, relived emotion, and hypnosis almost invariably con-tain strong cues concerning the targeted affective state. Thesecues may be less obvious for directed facial movement orconfederate procedures; these procedures are thus probablylower in demand than films.

Standardization

Although threats to standardization are present in any labo-ratory procedure, the stimulus content, presentation appa-ratus, and viewing conditions can all be tightly controlledwith film clips. The standardization of films is therefore high,allowing for the potential replication of effects across labo-ratories (Gross & Levenson, 1995). Films share this highdegree of standardization with other normative media, suchas slides and music. Confederate interaction procedures tendto be less standardized than films because experimenterscannot totally regiment the dynamic interplay between hu-man participants. Relived emotion procedures are less stan-dardized than films simply because by-person idiographicvariation is the source of their power. The high standardiza-tion of films does not, of course, guarantee that films will beeffective equally for all participants (see later section on in-dividual and group differences).

Temporal Considerations

Emotion researchers differ widely in their requirements fortemporal resolution, or granularity. For example, phenom-

Emotion Elicitation Using Films 13

ena that are modified by emotion over seconds or millisec-onds (such as the startle reflex, event-related brain poten-tials, or brain activations) require data collection techniquesthat accommodate a high degree of temporal resolution. Still-picture paradigms (see Bradley & Lang, chapter 2, this vol-ume), in which stimuli are presented in relatively short trials(approximately 6 s) that are averaged together to increasemeasurement reliability, have been used for this purpose.Films, in their prototypical use, are much lower in temporalresolution and range from about 1 to 10 minutes in length.Because emotions are a relatively rapid phenomenon, withonsets and offsets over seconds, films (and other low-resolution procedures such as confederate interactions) willproduce epochs of data that are heterogeneous in emotionalactivation. Experimenters who use films must consider pro-cedures to extract the emotional phenomena of interest fromthese longer epochs (e.g., a priori criteria, whole period av-erages), an issue also discussed in more detail later.

Ecological Validity

Like many of the stimuli that make us emotional in real life,film clips represent a dynamic display of prototypic situa-tions relevant to well-being and survival (e.g., loss, danger;Tooby & Cosmides, 1990). From this standpoint, films ap-pear to be high in ecological validity. At the same time, boththeoretical and empirical uncertainty surround this issue(e.g., Ritz & Steptoe, 2000). On the one hand, film emotionappears to be real and robust. For example, about one-thirdof female participants overtly weep in response to a sad filmwe have used (e.g., Rottenberg, Gross, et al., 2002), and ourbest films in other emotion categories produce similar results(e.g., visible gagging to disgust films, convulsive laughter toamusing films). On the other hand, film emotion is a kindof aesthetic emotion (Frijda, 1989) that requires a “willingsuspension of disbelief” for its operation. That is, participantsbecome emotional in response to films in spite of (or becauseof) the fact that film images are an illusion of reality.

In sum, although some questions remain concerning theecological validity of films, films are probably more naturalis-tic (and hence more generalizable) than a number of othertechniques such as directed facial movement or hypnosis,which resemble very few everyday life situations. In situationsin which high ecological validity is required, one might do wellto employ scripted or unscripted social interaction procedures,as they elicit emotions that are as real and robust as film emo-tions while requiring no willing suspension of disbelief.

General Considerations When Using Films

Thus far, we have focused on whether films are the rightemotion elicitation procedure to use in a given research con-text. In the following section, we consider how to use filmprocedures once one has decided that films are appropriate.

With films, as with many techniques used in experimentalpsychology, the devil is in the details. With this in mind, wediscuss several factors that influence how well films work indifferent settings and offer suggestions for avoiding commonpitfalls.

How Should Emotion Be Measured?

Not surprisingly, judgments regarding the success or failureof film-based emotion elicitation efforts often hinge on howemotion itself is measured. Here we discuss two issues rele-vant to the measurement of emotion when using films: (1)proximity of activation and measurement periods and (2) theextraction of emotion.

Emotions are evanescent. Therefore, delays, even shortones, between the activation of emotion by a film and theassessment of emotion by an experimenter can introducemeasurement error. Further compounding this error is theprospect that the time course of an emotional response var-ies by emotion response system (e.g., facial expressive behav-ior may have a faster offset than emotion experience). Thecosts of delay are well illustrated by the common practice ofassessing self-report responses to films retrospectively usingquestionnaires. As time elapses between the film’s end andthe questionnaire’s completion, the elicited affect is likely tofade and/or be distorted by errors or systematic biases in recall(Levenson, 1988). To avoid problems associated with delayedretrospective reports and to obtain continuous measures ofexperience that parallel continuous measures of other re-sponse domains (e.g., behavior and autonomic psychophysi-ology), there has been a growing interest in rating dialmethodologies, which afford continuous measures of emotionexperience, in either online or cued-review rating formats(Fredrickson & Kahneman, 1993; Gottman & Levenson,1985; Ruef & Levenson, chapter 17, this volume)

One concern about frequently assessing emotion expe-rience is that the act of repeated measurement can potentiallyalter the emotional response itself, a concern that does notapply as strongly to the physiological and behavioral responsesystems, which may be monitored continuously withoutinterfering substantially with emotional responses. Decisionsregarding when and how to assess emotion experience re-quire that the experimenter balance the desire for valid andperhaps even continuous emotion experience reports againstthe competing desire not to interfere with emotional respond-ing during film and postfilm (recovery) periods. In our ownwork, we typically assess central and peripheral physiologi-cal responses, videotape expressive behavior, and—depend-ing on the study—use a mix of retrospective and real-timeemotion experience ratings.

A second measurement issue concerns the extraction ofemotion from film viewing periods. Many researchers—our-selves included—have relied on overall period averages tomeasure experiential, behavioral, and physiological reactiv-ity during a film clip. Overall average response is a useful

14 Emotion Elicitation

summary statistic and provides an important starting pointfor data analysis. At the same time, we have alluded to thefact that films have relatively low temporal resolution andtypically create heterogeneous epochs of data. This meansthat the period average strategy will almost invariably includenonemotional epochs and/or epochs in which nontargetedemotions were elicited, effectively “watering down” the emo-tion data of principal interest. Furthermore, an exclusivefocus on overall averages may detract from other potentiallyinformative parameters of emotion (e.g., threshold, rise time,variability). One promising alternative to period averages isto extract time windows of data based on a priori responsecriteria—such as rating dial measures of emotion experience,facial movements that match intended prototypes (Rosenberg& Ekman, 1994), or behavioral or physiological profiles in-dicative of the target emotional state (Davidson, Ekman,Saron, Senulis, & Friesen, 1990).

What Kind of Baseline Should Be Used?

However one measures emotion, and whichever way oneextracts particular periods of interest, it is necessary to esti-mate the impact of a film compared with some point of ref-erence. In fact, because acute emotional responses are usuallysuperimposed on some prior affective state, it is difficult, ifnot impossible, to draw inferences about the nature of a filmeffect without a relevant baseline period. Thus another ba-sic issue for implementing film procedures is establishing aproper point of comparison, or baseline, against which toassess the effects of a given film clip.

What type of baseline should one use? Historically, a rest-ing state has been a major workhorse. In our experience, how-ever, there are drawbacks associated with resting baselines: (1)rest may not be a representative state of the organism; (2) itmay create a floor that precludes detection of deactivation ef-fects; (3) rest instructions may introduce unwanted variabil-ity, as participants differ radically in their ability to comply (seealso Christoff, Ream, & Gabrieli, 2004; Levenson, 1988). Toavoid these drawbacks, we have in our own work moved to-ward use of neutral-emotion-film baselines (e.g., Rottenberg,Gross, et al., 2002). In addition, a film baseline also has thedesirable feature of controlling for the effects of viewing adynamic external stimulus (Piferi, Kline, Younger, & Lawler,2000). Specific relatively neutral film clips are recommendedfor baseline later in this chapter.

Even when a baseline has been well constructed, it willnot be useful as a comparison condition unless it is timedappropriately. Levels of responding in experiential, behav-ioral, and physiological channels are never static but drift,or fluctuate, throughout a laboratory session. Delays betweenbaseline and activation periods introduce the confoundingeffects of time. Therefore, it is often useful to have multiplebaseline periods, and baselines must be positioned tempo-rally proximal to the emotion film (ideally in a contiguousposition). In designs that employ multiple emotion films, the

issue of response drift is acute and strongly militates for theuse of multiple baselines.

How Can Film Clips Be Matched?

A related consideration is how to best match films to allowstrong inferences about emotion effects. That is, when two(or more) emotion film conditions are compared, emotionresearchers will usually want to infer that observed condi-tion effects are due to emotion rather than film differences.The complexity of films complicates this inference: Filmsdiffer from one another on a large number of potentiallyconfounding characteristics (e.g., length, intensity, complex-ity, core themes, presence and number of human figures,color, brightness, picture motion; see Detenber, Simons, &Bennett, 1998). Moreover, investigators usually have fewdegrees of freedom in matching stimuli on these character-istics because: (1) the pool of effective films available for anygiven target emotion is often small (i.e., 2–5 films); (2) thenumber of possibly relevant dimensions of difference amongfilms is so large; and (3) even slight variations in the editingof film clips can dramatically alter their effectiveness as emo-tion elicitors.

Because it is rare that films can be matched across allcharacteristics, investigators must match along a few char-acteristics that have the highest priority. In our own workwe have matched films based on length, theoretically impor-tant dimensions (e.g., activation level), known effects in theliterature, and study-specific aims (e.g., an investigation ofstimulus meaning matched films on thematic content;Rottenberg, Gross, & Gotlib, 2005). Decisions about match-ing should be made explicitly, with reference to the goals ofthe particular study.

How Many Film Clips Should Be Used?

Our discussion of baseline and of film stimulus confoundshave suggested several noise sources that can obscure the“signal” of emotion. How can one boost this signal? Oneapproach is to sample emotion for extensive periods by us-ing several film exemplars for a given emotion and/or verylong stimulus presentations. Indeed, psychometric theoryargues that aggregation over multiple film exemplars (andlong films) should increase measurement reliability (Epstein,1983).

However sensible this approach to sampling emotion maybe, it is often problematic to carry out in practice because:(1) different films designed to target a particular emotion donot always generate equivalent responses; (2) the risk ofhabituation, sensitization, or fatigue effects increases withmore film presentations; (3) longer films are more heteroge-neous and often less effective than shorter films (e.g., longneutral films can become aversive); and (4) practical con-straints related to participant availability and attention spanoften make it impossible to use multiple exemplars for each

Emotion Elicitation Using Films 15

target emotional state. For these reasons, we have used acompromise sampling strategy in our own research: For eachtarget emotion we use one (or two) relatively short filmstimuli that are typically between 1 and 3 minutes in lengthand that are as homogenous as is possible. Our strategy toboost the signal of emotion therefore places a heavy burdenon stimulus selection (and reducing sources of noise).

What Is the Psychological ContextDuring Film Viewing?

The sensitivity of emotion to psychological context is an in-teresting (and bedeviling) aspect of emotion that can disruptthe standardization of film elicitations of emotion. Films areoften shown in the context of complex, multisession stud-ies, in which participants complete questionnaires, have sen-sors attached, are videotaped, and interact with one or moreexperimenters. Although an investigator may be particularlyinterested in participants’ emotional responses to one aspectof this complex experience (i.e., the films), it is important toremember that several aspects of the experimental protocolmay trigger emotional responses that compete with (and evensupersede) participants’ emotional responses to the film clips.Of course, it is impossible for an investigator to control (oreven to be aware of) every aspect of the psychological context.Nevertheless, three aspects of the psychological context standout as threats to standardization and can be controlled and/ormonitored: (1) timing, (2) order, and (3) prior viewing.

A first consideration is timing, or when a film is presentedin the course of a laboratory session. It would be unwise toassume that a sad film presented at the end of a grueling2-hour laboratory session will be evaluated in the same wayas it would at the beginning of the session, given the greaterlikelihood of increased participant fatigue (Morgan, 1920)and reactivity to repeated laboratory tasks (Thompson &Spencer, 1966). A second consideration is the order in whicha film is presented within a laboratory session. For example,certain orders of film presentation may be more susceptibleto carryover effects than others. Several studies indicate thatresidual affective states that match the valence of a new emo-tion stimulus will enhance the response to the new stimulus(e.g., Branscombe, 1985; Neumann et al., 2001), suggestingthat carryover effects are most probable when films of likevalence are presented in blocked order. A third element ofthe psychological context that poses a threat to standardiza-tion is the film-viewing histories of participants. Participantsoften have previous experience with films used in laboratoryprocedures, simply because many of the best emotion filmclips are edited segments of commercially available entertain-ments. Prior viewing has been associated with a heightenedreport of the target emotion (Gross & Levenson, 1995), andit may influence the experience of viewing film clips in otherways (e.g., expectation effects).

In our own work, when emotion-specific effects are im-portant to us, we make a point of ensuring that each film

occurs in each position within the experimental protocol. Wealso try to limit carryover from one film to the next usingtemporal spacing (e.g., with self-report assessment periods),as well as nonemotional distractor tasks (e.g., copying geo-metric figures; Gross, et al., 1998). With respect to the issueof prior film viewing, we routinely ask participants (bothduring pilot testing of the films and during the experimentitself) whether they have seen the film before. We then usethis information to control for the effects of prior viewing.

Does the Physical Context Matter?

One important determinant of participants’ responses to filmstimuli is the physical setting in which films are presented.Emotional reactivity to films has been associated with mun-dane aspects of the experimental situation such as room light-ing (Knez, 1995), larger display size (Detenber & Reeves,1996; Lombard, 1995), warmer room temperature (Ander-son, Deuser, & DeNeve, 1995), and color (as opposed toblack and white; Detenber, Simons, & Reiss, 2000). Physi-cal setting also includes participants’ proximity and accessto other people. Laboratory film procedures (like all emo-tion procedures) are socially embedded phenomena. Indi-viduals may report differing reactions to films as a functionof the group size (e.g., whether films are viewed in group orindividual session formats), and these effects may differ byemotion (Jakobs, Manstead, & Fischer, 2001). Even withinthe context of single-subject paradigms, subtle changes in thephysical arrangements may influence reactivity via theimplied social presence of others (Fridlund, 1992). Forexample, the presence of video recording equipment ina participant room may increase self-consciousness thatdampens or enhances behavioral responses. In our ownwork, we have used a 20-inch monitor positioned about5 feet from the participant in solitary film viewing sessionsconducted in a living-room-like laboratory room withdimmed lights. Throughout experimental sessions, partici-pants and experimenters are in contact via an intercom.Cameras used to record participants’ expressive behavior arediscreetly hidden behind darkened glass panels in order tominimize participants’ self-consciousness.

What Is the Role of Individualand Group Differences?

A dramatic example of the power of individual differencesto influence the outcome of emotion elicitation proceduresis hypnotic emotion inductions, which are not usable in themajority (70–75%) of people who not highly hypnotizable(Bower, 1981). Individual differences also influence reactiv-ity to emotion film clips. For example, variations in self-reported neuroticism and extraversion have been shown toinfluence negative and positive reactions to films, respectively(e.g., Gross et al., 1998). Likewise, biological traits, such asresting electroencephalographic (EEG) asymmetry in anterior

16 Emotion Elicitation

regions of the brain, have also been shown to predict filmreactivity (Wheeler, Davidson, & Tomarken, 1993). How oneproceeds in the face of these individual differences dependslargely on one’s research aims. Some researchers (e.g., per-sonality researchers) welcome variation on these factors be-cause these differences are the focus of study (e.g., Berenbaum& Williams, 1995). In other cases, such as work on basicemotion processes (e.g., forms of self-regulation), these in-dividual differences may constitute nuisance variance thatinterferes with the detection of other subtle yet importanteffects.

The influence of group membership (e.g., linguistic, gen-der, racial, or socioeconomic) on emotion film reactivity isat an early stage of investigation and is an important area forfuture research. Preliminary evidence, however, indicates thatthe Gross and Levenson (1995) films generalize to other lin-guistic groups (e.g., German-language speakers; Hagemannet al., 1999). Emotion film effects have also generalized acrossdifferent ethnic groups in some samples (Gross & Levenson,1995; Tsai, Levenson, & Carstensen, 2000), but additionalstudy of this issue is needed. Other findings suggest that gen-der may be a particularly important influence on film reac-tivity: Women, relative to men, have been shown to exhibitstronger reports of emotional experience (Gross & Levenson,1995; Hagemann et al., 1999), to be more expressive (Kring& Gordon, 1998), and to exhibit differential neural activa-tions to emotion elicitors (Karama et al., 2002). Treatmentof these group differences, we believe, should hinge on one’swider research aims: where group differences are the objectof study, individuals with certain group memberships canbe oversampled (Canli, Desmond, Zhao, & Gabrieli, 2002);where group differences represent confounds, they can beaddressed by screening (e.g., studying each sex separately;Gross & Levenson, 1993) or by using post hoc statisticalcontrols.

In our own work, we are cautious about assuming thatfilm reactivity will generalize across groups in the absence ofstrong normative data. We have found that the successful useof film clips entails extensive piloting of films and attentionto sample composition, and we regularly report the genderand ethnic composition of study participants so as to allowothers to better compare our results with their own. If webelieve (as we often do) that there may be important gendereffects, we may conduct an initial study with a single gender(often women) and then replicate and extend results of thisinitial study by using a mixed-sex sample. Likewise, for treat-ment of individual differences, we recommend the use ofinstruments to screen out participants who have individualdifference profiles that might contaminate results and/orcollecting data on these individual difference variables toenable post hoc statistical control (e.g., Wheeler et al., 1993).In our own work, we typically obtain a wide array of indi-vidual difference measures, most often via the Internet, be-fore the experimental session.

Recommended Film Clips for ElicitingDiscrete Emotional States

In this section, we first discuss the process of finding, editing,and validating film clips. We then recommend a number of filmclips that meet our criteria for eliciting specific target emotions,drawing in part on the film library described in Gross andLevenson (1995). Toward the end of this section, we considerother film clips that may be useful but that do not target dis-crete emotions. The number of proven emotion elicitors re-mains relatively modest. We remark on some of the stumblingblocks that have stood in the way of developing and validatinga larger library of films and present recommendations for over-coming these obstacles in future stimulus development.

Developing and Validating Film Clips

Clearly, it would be desirable to have an extensive databaseof valid film stimuli that rivals those of other normativeemotional stimuli (e.g., pictures, words, and sounds). Per-haps most enviable in this respect is the International Affec-tive Picture System (IAPS; Lang, Bradley, & Cuthbert, 1995),a database of hundreds of colored pictures that have beenstandardized on large normative participant samples forjudged pleasure and arousal, distributed internationally toscores of researchers, and used extensively in cognitive, so-cial psychological, and biobehavioral studies. The compara-tively modest size of the library of well-validated film clipsprompts a consideration of obstacles that often arise in thecourse of developing new stimulus films.

First, films are complicated to develop because, unlikeslides, they are embedded within another source. For theforeseeable future, at least, in-house production of emotionfilms falls beyond the technical, dramaturgical, and financialmeans of most laboratories. Extracting emotion films from acandidate source is an iterative process that involves a num-ber of steps, including nominating candidate sources, infor-mally screening scenes from candidate sources, frame editingclips taken from candidate sources, collecting pilot data onedited clips, reediting clips on the basis of the pilot ratings(assuming that the initial ratings look promising), and, finally,collecting normative data from participant samples based onthe final edited version of the film clip.

In our experience, we have been repeatedly surprised atthe fragility of the film extraction process. Often, film seg-ments that are powerfully emotionally evocative in the con-text of the larger film fail to elicit emotion when the film clipis viewed on its own, particularly if the participant has notseen the film. We also have found that even if a film segmentsurvives its surgery, the emotional impact can vary as a re-sult of relatively minor variations in the editing of either themusic, the image, or both.

A second factor that has hindered the development of alarge library of films has been a lack of systematic communi-

Emotion Elicitation Using Films 17

cation among scholars. Whereas still pictures and othermaterials are shared widely, uncertainty concerning the fairuse of commercial films for scholarly purposes has slowedthe development of a centralized repository where film clipsmight be stored, copied, and distributed. The developmentof the Internet, however, has made it easier for scholars todevelop centralized locations that post frame-editing instruc-tions for generating stimulus films (which does fall under fairuse). As we move forward into the future, we expect theInternet will remain an important resource for building thecommunity of researchers who develop and use emotionfilms.

A third factor that has made it difficult to build a largelibrary of film clips is the disparate (and often incommensu-rate) film validation procedures. In our own research, we havefollowed the practice of carefully pretesting each of the filmswe are thinking of using with a sample that is matched tothe intended research population. We collect validation dataon 8–10 films at a time in group-format sessions that lastapproximately 1 hour (fatigue and the duration of films lim-

its the number of films that can be validated). We obtain abroad range of emotion experience reports immediately af-ter each film clip is shown. As is evident from the other con-tributions to this volume, there are many different ways toassess emotion experience.

The specific emotion terms we usually use span a broadrange of theoretically important discrete negative and positiveemotional states. We typically also include the term confu-sion to assess how easy it is for the participant to understandthe film clip outside the original film context. The use of awide range of terms allows us to compare films we employto elicit different target states. Within a given study, we of-ten find it useful to use several terms that converge on a tar-get emotional state so that we may create target composites.For example, in a study of amusement and sadness, we usedthe rating form shown in Figure 1.3. On the form, partici-pants rate the greatest emotion that was experienced duringthe preceding film, using both discrete emotion (specific) anddimensional (pleasant vs. unpleasant) terms. Participants rateeach term on 9-point Likert (0–8) scales that for discrete

POST FILM QUESTIONNAIRE

The following questions refer to how you felt while watching the film.

0 1 2 3 4 5 6 7 8not at all/ somewhat/ extremely/ none some a great deal

Using the scale above, please indicate the greatest amount of EACH emotion you experienced while watching the film.

_____ amusement _____ anger _____ anxiety _____ confusion _____ contempt _____ disgust

_____ embarrassment _____ fear _____ guilt _____ happiness _____ interest _____ joy

_____ love _____ pride _____ sadness _____ shame _____ surprise _____ unhappiness

Did you feel any other emotion during the film? O No O Yes If so, what was the emotion? ______________________

How much of this emotion did you feel? _____

Please use the following pleasantness scale to rate the feelings you had during the film. Circle your answer:

0 1 2 3 4 5 6 7 8unpleasant pleasant

Had you seen this film before? O No O Yes

Did you close your eyes or look away during any scenes? O No O Yes

Figure 1.3. Postfilmquestionnaire used in filmvalidation.

18 Emotion Elicitation

emotions are anchored by not at all and extremely and thatfor the dimensional valence item is anchored by unpleasantand pleasant. We also allow participants to rate any otheremotion they may have felt during the film and ask whetherthey looked away during the film (in which case they maynot have seen important parts of the film).

When deciding whether or not a film is efficacious (in-cluding the films we recommend here), we have relied onintensity and discreteness as the two primary metrics. Thatis, intensity refers to whether a film receives a high meanreport on the target emotion relative to other candidatefilms. Discreteness refers to the degree to which participantsreport feeling the target emotion more intensely than allnontargeted emotions. One way to formalize the discrete-ness metric is to use an idiographic “hit rate” statistic thatis the percentage of participants who indicated that theyhad felt the target emotion at least 1 point more intenselythan other, nontargeted emotions. Different films that pu-tatively elicit the same target state can be compared statis-tically by combining intensity and discreteness scores intoa success index, in which each intensity score represents az score derived by normalizing intensity scores for all com-parison films and in which each discreteness score repre-sents a normalized discreteness value relative to allcomparison films. We have recently added to these metricsfor judging the success of a film the recommendation thatfilms have high alphas for the target composites and be lowin reports of confusion.

We emphasize that our reliance on retrospectively as-sessed self-reports of emotion (obtained in group-viewingsessions) for film validation is practical, not philosophical.We do not view any of the major emotion response systems(i.e., experience, behavior, physiology) as the “gold standard.”Given the loose coupling between emotional response sys-tems, self-report ratings will not perfectly predict which filmswill produce behavioral or physiological activations. We en-courage researchers to collect validation data from as manyresponse systems as possible to afford more precise state-ments concerning the effects of a given film. We acknowl-edge, however, that resource limitations often make itimpossible to validate film materials using the full range ofbehavioral and/or physiological measures and the same view-ing contexts (e.g., individual sessions) as will be used in theexperiment itself.

Our Film Recommendations

Both our emotion film recommendations and the criteria onwhich they are based build on and extend our past efforts(Gross & Levenson, 1995). In this work, we have generallywanted to elicit specific discrete emotional states (e.g., an-ger, sadness) rather than more diffuse states of positive ornegative activation (Watson, 2000). Therefore, in stimulusdevelopment we have searched for films that are relativelyhigh on discrete emotions (the target) and as low as possible

on other related emotions. We should note that frequentlywe have found that our initial intuitions as to how a film clipwould work have been wrong, and a film clip that we be-lieved would elicit a discrete state produced what might bestbe characterized as a diffuse state of positive or negative ac-tivation and was hence discarded. We illustrate this point bycontrasting self-report profiles of two films that were bothdeveloped to elicit disgust. Unexpectedly, a film depictingan employee wounded in an industrial accident (Figure 1.4,panel A) was far less successful from the standpoint of dis-creteness than a film depicting the surgical amputation of anarm (e.g., Figure 1.4, panel B).

In the following sections, we offer our current recom-mendations regarding film clips that will elicit neutral orrelatively discrete emotional states. We organize these rec-ommendations according to target emotion. One point ofdifference from Gross and Levenson’s set of target emotionsshould be noted: Mild levels of contentment that were pre-viously considered separately are now considered underNeutral. Table 1.1 presents validation data for these filmsacross a core set of items to facilitate comparisons. In ap-pendix 1, frame instructions are presented for creatingmany of these stimuli from commercially available sources.Additional film instructions, as well as copies of noncom-mercial films, are available at http://www.cas.usf.edu/psychology/fac_rottenbergJ.htm.

SURPSADN

A. Amputation of Arm Film Clip

7.00

6.00

5.00

4.00

3.00

2.00

1.00

0.00AMUS CTEN ANGE DISG FEAR

SURPSADN

B. Industrial Accident Film Clip

7.00

6.00

5.00

4.00

3.00

2.00

1.00

0.00AMUS CTEN ANGE DISG FEAR

Figure 1.4. Different response profiles obtained for two filmstargeting disgust. Participants could report Amusement (AMUS),contentment (CTEN), Anger (ANGE), disgust (DISG), Fear(FEAR), Sadness (SADN), or Surprise (SURP).

http://www.cas.usf.edu/psychology/fac_rottenbergJ.htmhttp://www.cas.usf.edu/psychology/fac_rottenbergJ.htm

Emotion Elicitation U

sing Film

s19

Table 1.1Recommended Films for Eliciting Discrete Emotional States

Target EmotionMean (SD) Self-Reported Emotion

Film Clip Sex AMUS ANGE CFUS DISG EMBA FEAR HAPP INTE SADN SURP

Amusement

Harry M (N = 29) 5.45 (1.23) 0.39 (0.72) 0.55 (0.85) 0.74 (1.32) 2.55 (2.01) 0.23 (0.82) 3.39 (1.71) 4.45 (1.43) 0.13 (0.43) 1.90 (2.33)F (N = 41) 5.61 (1.28) 0.24 (0.62) 0.22 (0.53) 0.22 (0.73) 2.10 (2.07) 0.35 (0.98) 3.32 (1.82) 3.63 (1.93) 0.17 (0.67) 1.27 (1.72)

Robin M (N = 28) 5.89 (1.17) 0.32 (0.67) 0.71 (1.18) 0.50 (0.92) 0.82 (1.44) 0.07 (0.26) 4.68 (1.96) 4.79 (1.34) 0.14 (0.45) 2.07 (2.12)F (N = 34) 5.82 (1.99) 0.21 (0.49) 0.70 (1.67) 0.91 (1.71) 0.53 (1.02) 0.06 (0.24) 4.59 (2.09) 4.50 (2.29) 0.18 (0.46) 1.94 (2.23)

Cosby M (N = 14) 5.21 (2.36) 0.07 (0.27) 0.21 (0.58) 0.57 (1.40) 0.79 (1.53) 0.07(0.27) 3.71 (2.43) 3.64 (1.87) 0.07 (0.27) 1.14 (2.41)F (N = 24) 5.20 (1.76) 0.08 (0.27) 0.31 (0.68) 0.38 (0.90) 0.35 (0.89) 0.04(0.20) 4.23 (1.66) 4.62 (1.88) 0.04 (0.20) 1.77 (2.05)

Whose Line M (N = 13) 7.23 (1.01) 0.62 (1.12) 0.54 (1.13) 1.85 (2.79) 0.92 (1.44) 0.31 (0.85) 5.92 (1.93) 6.08 (1.89) 0.08 (0.28) 3.38 (2.27)F (N = 15) 6.87 (1.19) 0.07 (0.26) 0.87 (1.46) 2.07 (2.58) 1.80 (2.91) 0.20 (0.56) 5.27 (2.60) 5.47 (2.64) 0.47 (1.81) 3.47 (2.47)

Anger

Bodyguard M (N = 27) 1.34 (1.61) 5.03 (1.82) 1.21 (1.11) 4.69 (1.61) 1.10 (1.76) 1.62 (1.57) 0.76 (1.33) 3.66 (2.02) 3.07 (2.12) 1.66 (1.97)F (N = 33) 0.61 (1.12) 5.36 (1.39) 1.82 (2.21) 4.94 (1.80) 0.61 (1.25) 2.15 (2.00) 0.42 (0.90) 3.15 (1.62) 4.21 (2.13) 1.21 (1.76)

Cry Freedom M (N = 21) 0.78 (1.62) 5.87 (1.96) 3.09 (2.73) 5.74 (1.76) 1.78 (2.58) 3.00 (2.92) 0.83 (1.64) 4.09 (2.11) 5.22 (2.17) 2.86 (2.75)F (N = 36) 0.14 (0.42) 6.17 (1.68) 2.28 (2.25) 5.33 (2.48) 0.72 (1.65) 3.69 (2.41) 0.22 (0.72) 3.22 (2.26) 5.56 (1.93) 2.42 (2.56)

Disgust

Pink Flamingos M (N = 20) 2.40 (2.39) 0.95 (1.50) 1.85 (2.13) 6.60 (1.39) 0.85 (1.76) 0.45 (1.05) 0.55 (1.61) 1.20 (2.12) 0.90 (1.77) 3.05 (2.56)F (N = 31) 2.47 (2.56) 0.47 (1.22) 1.87 (2.27) 6.34 (1.54) 1.12 (2.08) 0.38 (1.13) 0.34 (0.83) 1.88 (1.86) 0.29 (1.10) 3.72 (2.43)

Amputation M (N = 74) 1.23 (1.72) 0.68 (1.17) 2.22 (1.94) 5.00 (2.22) 0.51 (1.15) 1.74 (1.84) 0.27 (0.63) 2.65 (2.12) 0.93 (1.46) 2.12 (2.27)F (N = 71) 0.42 (1.20) 0.66 (1.50) 2.30 (2.43) 6.19 (1.92) 0.32 (0.88) 2.15 (2.36) 0.15 (0.73) 2.68 (2.37) 0.76 (1.56) 2.00 (2.34)

Foot Surgery M (N = 11) 0.45 (0.82) 0.18 (0.41) 1.82 (2.27) 4.91 (2.30) 0.36 (0.81) 0.45 (1.04) 0.09 (0.32) 3.00 (2.57) 0.27 (0.91) 0.82 (1.94)F (N = 18) 0.56 (1.15) 0.39 (0.78) 2.00 (1.94) 4.44 (2.62) 0.39 (1.20) 1.78 (2.44) 0.17 (0.51) 2.44 (2.28) 0.28 (0.75) 1.50 (2.04)

Fear

Shining M (N = 23) 1.39 (1.37) 0.65 (1.27) 2.91 (2.26) 0.39 (0.78) 0.22 (0.42) 3.26 (2.03) 0.96 (1.22) 4.61 (1.27) 0.70 (1.26) 1.74 (2.05)F (N = 36) 0.83 (1.23) 0.17 (0.38) 1.92 (2.25) 0.00 (0.00) 0.00 (0.00) 4.61 (2.07) 0.19 (0.75) 3.89 (1.72) 0.17 (0.45) 1.08 (1.65)

Lambs M (N = 31) 2.65 (2.36) 1.74 (1.53) 1.61 (1.54) 2.39 (1.96) 0.48 (0.81) 3.87 (2.46) 1.70 (1.97) 4.81 (1.52) 0.74 (1.13) 2.19 (2.04)F (N = 40) 1.07 (1.39) 0.80 (1.14) 0.88 (1.52) 1.80 (2.08) 0.28 (0.68) 4.45 (2.23) 0.60 (1.01) 4.32 (1.95) 0.53 (1.38) 1.88 (2.14)

Neutral

Sticks M (N = 19) 1.05 (1.65) 1.37 (1.71) 3.58 (2.52) 0.84 (1.26) 0.21 (0.42) 0.16 (0.38) 0.79 (1.62) 1.11 (1.56) 0.53 (1.26) 1.16 (1.68)F (N = 36) 0.83 (1.21) 0.92 (1.46) 1.92 (2.31) 0.39 (0.80) 0.14 (0.49) 0.33 (1.37) 0.75 (1.16) 0.92 (1.32) 0.11 (0.52) 0.62 (1.02)

Denali M (N = 12) 2.33 (2.06) 0.00 (0.00) 0.58 (1.08) 0.00 (0.00) 0.08 (0.29) 0.25 (0.45) 3.75 (1.91) 4.54 (1.50) 0.67 (1.50) 0.42 (1.16)F (N = 12)