Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC...
-
Upload
eve-stacks -
Category
Documents
-
view
216 -
download
0
Transcript of Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC...
Jonathan GratchUSC Institute for Creative Technologies
Joint work with
Stacy MarsellaUSC Information Sciences Institute
Jonathan GratchUSC Institute for Creative Technologies
Joint work with
Stacy MarsellaUSC Information Sciences Institute
The Architectural Role of Emotions in Cognitive Systems
The Architectural Role of Emotions in Cognitive Systems
Outline
Emotions are adaptiveCan inform cognitive system design
Ground in implemented cognitive system– Mission Rehearsal Exercise system– Cognitive Appraisal Theory– Illustrate impact on architecture design
General implications for cognitive systems
Outline
Outline
• Revolutionary progress in emotion research– Neurophysiology of emotion (Damasio, LeDoux)– Appraisal theories (Frijda, Lazarus, Scherer)
• Emotions appear adaptive (in moderation?)– Decision-making ─ Focus of attention– Learning ─ Social relationships – Belief formation ─ Communication
• Growing interest in “emotional” systems with focus on modeling human behavior– HCI (non-verbal recognition and generation) – User and human behavior modeling– Believability/Entertainment
Adaptive Role of Emotions
Outline
• Can inform intelligent behavior in general• Motivate behavior• Balancing competing goals• Balancing reaction and deliberation• Disambiguating stimuli in light of existing beliefs and commitments
• Abstract and formalize as information processing– Not new idea:
• Simon(1967), Oatley&Johson-Laird(1987), Sloman
– Revisit in light of new findings• In intelligent systems• In theories of emotion
Architectural Perspective
Grounding: Virtual Humans
Face-to-face interaction
Verbal & non-verbal behaviorSwartout, Gratch, Hill, Hovy, Johnson,
Marsella, Narayanan, Rickel, Traum, …
Marsella, Johnson & Labore
Mission Rehearsal Exercise
Social Training Simulation• Explore high-stakes social interactions in safety of VR
Mission Rehearsal Exercise
• Team decision-making in crisis situations:– Non-scripted real-time interactions
– Planning, replanning, and plan execution
– Teamwork, distributed authority and responsibility
– Collaborative, mixed initiative dialogue
– Multi-party conversations
– Verbal and non-verbal communication
– Emotionally-biased behavior
•
Mission Rehearsal Exercise
• Assumptions/Limitations:– Tightly focused task-related dialogue
– Near-expert decision makers
– Stylized vocabulary (military speak)
– Stylized virtual environment
•
Speech Recognition (HTK)
Semantic Parser
Motion/ Gesture Scheduler (Beat)
Text to Speech (Festival)
World Simulator
Animation System
BDI
Haptek
Com
mun
icat
ion
Bus
Audio (Protools)
Voice Input
Vega
Projection System
Speakers (10.2)
Soar Planning
DialogueAction Selection
Per
cept
ion
NLG
Emotion
NLU pragmatics
Child Healthy:False
AccidentIntend: FalseBlame: unresolved
Assist Eagle 1-6:False
Eagle 1-6 AssistDesire: LT
Belief: False
Child-HealthyDesire: SGTBelief: False
Probability: 75%
Get MedevacResponsibility:LTIntend: True
Medevac Available:True
Past FuturePresent
Cognitive Representation
Soar’s Working Memory
Planning Perception Dialogue Action
Soar operators
Child Healthy:False
AccidentIntend: FalseBlame: unresolved
Assist Eagle 1-6:False
Eagle 1-6 AssistDesire: LT(+50)
Belief: False
Child-HealthyDesire: SGT(+80)
Belief: FalseProbability: 75%
Get MedevacResponsibility:LTIntend: True
Medevac Available:True
Past Events Future PlansPresent
Cognitive Representation
• Causal Interpretation– Combines decision-theoretic plans with models of
belief and intention– Uniform representation of past, present, future– Agent centric subjective view
Architectural Role of Emotion
• Began with view “emotion as veneer” Ended up as central organizing construct – Initial problem:
• how to convey emotion in interactive setting?
– Built mechanism to infer plausible emotions• In response to simulation events • In response to user interventions
– But discovered resolved architectural issues• Coherence is more than skin deep• Build it and they will come
How to convey emotion
• Cognitive Appraisal Theory– Influential and well-established theory
Arnold, Frijda; Lazarus; Ortony, Clore & Collins; Scherer; Smith
– Emphasizes tight coupling between• Emotion
• Cognition
• Motivation
Goals, BeliefsExternal Events
Coping
Emotion
Appraisal
Problem-focused
Emotion-focused
Cognitive Appraisal Theory
Coping
Smith and Lazarus’ cognitive-motivational-emotive system
Goals, BeliefsExternal Events Appraisal
Appraisal
• Appraisal = Situation assessment
– Compare beliefs, desires and intentions with
external circumstances
Appraisal
• Characterize via appraisal variables– Desirability– Likelihood – Urgency– Unexpectedness– Causal attribution (causality, agency, blame/credit)– Coping potential (controllability, adaptability)
• Superset of criteria considered by cog systems– Decision theory: desirability, likelihood– Scheduling: desirability, urgency
CopingProblem-focused
Emotion-focused
• Coping = Response strategy– Characterized by ontology of coping strategies
Emotion
Coping Strategies
Goals, BeliefsExternal Events
• Problem-focused (act on the world)– Action execution
– Planning
– Seek instrumental social support
• Analogous to:– Deliberative or reactive problem solving
– Team negotiation
Coping Strategies
• Emotion-focused (act on belief)– Denial– Find silver lining– Shift blame– Distancing
• Not typically considered by cog systems systems– More than a decision (e.g. abandon current plan)
• Provides self-justification for why• Related to motivational / explanatory coherence • Leads to persistent change in behavior
Coping Strategies
Modeling Appraisal and Coping
Past Future
Planning Perception Dialogue Action
Soar’s Working Memory
Soar Operators
Modeling Appraisal and Coping
• Appraisal as plan-evaluation– Causal interpretation mediates agent-environment relationship– Define appraisal variables in terms of features of interpretation– Fast, reactive, parallel
• Coping as generalized plan criticsMap to operators that change interpretation– Problem-focused execute step, add plan step– Emotion-focused
• Denial Change belief• Find silver lining Change utilities• Shift blame Change causal attribution
Dialogue moves• Distancing Drop goal / intention
Child Healthy:False
AccidentBlame: unresolved
Assist Eagle 1-6:FalseEagle 1-6 Assist
Desire: LTSatisfied: False
Child-HealthyDesire: SGT
Satisfied: False
Distress: 80
Sgt’s Appraisal of Accident from his perspective
Perspective: Self (Sgt)Desirability: -80Likelihood: 100% Blame/Credit: unresolved
Ap
praisal
Child Healthy:False
AccidentBlame: unresolved
Assist Eagle 1-6:False
Distress: 80
Child-HealthyDesire: SGT
Satisfied: False
Distress: 80
Perspective: LieutenantDesirability: -80Certainty: 100%Blame/Credit: unresolved
Sgt’s Appraisal of Accident from Lieutenant’s Perspective
Eagle 1-6 AssistDesire: LT
Satisfied: False
Ap
praisal
Child Healthy:False
AccidentBlame: unresolved
Assist Eagle 1-6:FalseEagle 1-6 Assist
Desire: LTSatisfied: False
Child-HealthyDesire: SGT
Satisfied: False
Distress: 80
Cop
ing
Child Healthy:False
AccidentBlame: unresolved
Assist Eagle 1-6:FalseEagle 1-6 Assist
Desire: LTSatisfied: False
Distress: 80
Sgt’s OwnPerspective
Cop
ing
Child-HealthyDesire: SGT
Satisfied: False
Child Healthy:False
AccidentBlame: unresolved
Assist Eagle 1-6:FalseEagle 1-6 Assist
Desire: LTSatisfied: False
Distress: 80
Child-HealthyDesire: SGT
Satisfied: FalseProbability: 75%
Distress: 80
Cop
ing
MakeAmends
Get MedevacResponsibility:LT
Problem-Focused Coping: Form intention to help Boy
Child Healthy:False
AccidentBlame: MOM
Assist Eagle 1-6:FalseEagle 1-6 Assist
Desire: LTSatisfied: False
Distress: 80
Child-HealthyDesire: SGT
Satisfied: FalseProbability: 75%
Distress: 80
Cop
ing
MakeAmends
Get MedevacResponsibility:LT
ShiftBlame
Emotion-Focused Coping: Blame Mother
Child Healthy:False
AccidentBlame: unresolved
Assist Eagle 1-6:FalseEagle 1-6 Assist
Desire: LTSatisfied: False
Distress: 80
Child-HealthyDesire: SGT
Satisfied: FalseProbability: 75%
Distress: 80
Cop
ing
MakeAmends
Get MedevacResponsibility:LT
ShiftBlame
Personality
Architectural Implications
• Emotion as central control construct– Planning (inform course-of-action selection)
– NLU (inform reference resolution)
– Dialogue (prompt dialogue initiative)
– NLG (biases sentence generation strategies)
– Non-verbal expression
General Implications
• Emotion and Reflection– Appraisal is form of self-reflection / focus of attention
• Emotion as decision-making– Generalization of decision-theory
– More to the world than probabilities and utilities
• Emotion and plausible reasoning– Emotion-focused coping motivate preference/beliefs
– Attempt to construct coherent motivational explanation
– Non-rational but adaptive?
General Implications
• Emotion and Learning– Focus learning on “emotionally salient” events
– Appraisal variables as features / case indexes
Conclusion
• Emotion is form of information processing– Arguable adaptive
• Juggling competing goals and commitments
• Focusing cognitive resources
• Enforcing coherence
– Arguable unexplored by cognitive systems
Focus=1Lt: U9 “Secure a landing zone”Committed(lt,7), 7 authorized, Obligation(sgt,U9)Sgt: U10 “First we should secure the assembly area”Disparaged(sgt, 7), endorsed(sgt,2)Lt: U11“Secure the area”Committed(lt,2), 2 authorized, Obligation(sgt,U11)Sgt: U12 “Yes sir”Committed(sgt,2), Push(2)Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr})Goal8: Start-conversation(sgt, {1sldr,2sldr,…},2)Goal8 Sgt: U13 “Squad leaders listen up!”Goal7 Sgt: U14 “I want 360 degree security”Push(3)Goal9:authorize 3Goal9 Sgt: u15“1st squad take 12-4”Committed(sgt,3), 3 authorizedPop(3), Push(4)Goal10: authorize 4Goal10 Sgt: u16“2nd squad take 4-8”Committed(sgt,4), 4 authorizedPop(4)…A10: Squads moveA10: grounds U13-U18,… ends conversation about 2, realizes 2Pop(2), Push(7)
Dialogue Example: Sgt’s Behavior
Render Aid
Secure Area
Secure 12-4
Secure 8-12 Secure Accident
Secure 4-8
Squads in area
A=Lt, R=Sgt
A=Sgt, R=1sldr
A=Sgt, R=2sldr
A=Sgt, R=4sldrA=Sgt, R=3sldr
Area Secure
1
2
34
5 6
Decomposition
DecompositionSecure LZ
A=Lt, R=Sgt
7
• Appraisal as a mediating variable– Direct mappings (e.g. Hayes-Roth personality model)
– Indirect mappings
• Direct: More links, No insight on how to map• Indirect: more constrained. More modular
More than a theory of emotion
World stateBeliefsDesiresPersonality
Behavior
World stateBeliefsDesiresPersonality
BehaviorAppraisal Variables
Mediating Variable
• Appraisal Mediates Personality
Personality Variable Appraisal Variables Behavior
e.g Extroversion Control Hope
Penley & Tomaka (2002)
• Appraisal Mediates Culture
Culture Variable Appraisal Variables Behavior
e.g. Uncertainty avoidance Threat Fear Kupperbusch et al