Comparative Approaches to Emotion-Oriented Architectures (WP 7: Emotion in Cognition and Action)

Comparative Approaches

to Emotion-Oriented Architectures(WP 7: Emotion in Cognition and Action)

Last Plenary

:-/ :-)Lola Cañamero (UH)

Plenary 3, 4-6 June 2007, Paris, France

humaine WP7: Emotion in Cognition and Action2

WP7: The area

Scope: investigating computational models of emotional influences in cognition and action

Enhance behavior & interactions of emotion-oriented systems

Feedback to emotion theorists (synthetic approach, operationalize)

Exemplar: Comparative approaches to emotion-oriented architectures: assumptions, integration challenges, and guidelines for future research => Output: edited collection

Divided in 4 elements:Emotion in embodied cognition and action

Emotion in reflective cognition and action

Emotions in bridging the gap between embodied and reflective C&A

Emotions in social cognition and interaction

Groups: UH, OFAI, Bari, Paris8, DIST, GERG, HW, EMPL38, CNR, USC, ICCS, KCL, UM, INESC-IST, EFPL, Miralab, FT-RD, UOXF, USFD


WP7 Exemplar: the four elements

4

123

4


Where are we? exemplar timeline

Stage 1: Critical analysis of state of the art and needs Months 1 - 18: iterations to define problems and exemplar; uncover

assumptions and needs

Stage 2: Integration challenges and key development goals Month 19: workshop

Months 19-39: theoretical and practical work on integration challenges

Stage 3: Conclusions and guidelines for future researchMonth 40: start developing “guidelines” for future research; chapter

proposals and abstracts => moved to earlier date (D7e, Month 35)

Month 42 (JUNE 15!!!!!): drafts of chapters due

Months 43-48: chapters reviewed and revised

Month 48: book to publisher


Months 1-5 (D7b): state of the art, analysis (1)

Review of key achievements:Emotion-based architectures (action selection, learning, memory)

Appraisal and cognitive systems

User modeling

ECAs and virtual environments

Key conceptual problemsMechanisms underlying the involvement of emotions in cognition and action

Emotion elicitors (which factors activate those mechanisms?)

Emotions as cognitive modes

Relations among emotion, value systems, motivation and action


Months 1-5 (D7b): state of the art, analysis (2)

Key integration challengesProblems arising from theories and models (diversity, poor understanding)Diversity of computational frameworks & modeling approaches

Embodied AI, dynamical systemsSymbolic AIHybrid systemsSocial simulation

Key development goals“Grounding problem” of artificial emotionsDissolving the “mind-body” problemUntangling the “knot of cognition”: links emotion – intelligenceMeasuring progress & the contributions of emotions to our systems


Months 6-11 (D7c): approach to exemplar

Best approach to fulfill horizontal goals in our area:Comparative approaches to emotion-oriented architectures: assumptions, integration challenges, and guidelines for future research

Key ideas:“Comparative approaches”

welcome the diversity of conceptual and computational models and frameworksde-emphasize idea of a “unified” model for an emotion-based architecture (misleading goal at this point) -> complements “blueprint”

“Assumptions, integration challenges and guidelines for future research” stresses the nature of our principled integration effort in setting sound grounds


Months 6-11 (D7c): elements of exemplar

Focused working groups, integration at various levels:WG1: Emotion in “lower-lever” cognition and action

UH, GERG, CNRS EPML 38, KCL

WG2: Emotion in “higher-level” cognition and actionUni. Bari, France Telecom RD, GERG, CNR-ISTC, QUB, UA

WG3: Bridging gap between “lower-” and “higher-level” C & A OFAI, HW, INESC-ID, IST, EPFL, USC

WG4: Emotion in Social Cognition and InteractionOFAI, UH, Paris8, MIRALab, DIST, DFKI

Output: Edited bookReflection based on “proof-of-concept” designs and implementations


Support to Network activities

Presentations (posters) at Plenary 1Presentations at WP3 workshop: “Contributions from robotic models of emotions” & several “hands-on demonstrations”Presentations at WP4 workshop: links wp7-wp4, Markov-based analysisPresentations & posters at WP6 workshop (ToM, affect-based imitation)Cross-WPs links (meetings in Saarbruecken, Geneva, Santorini, Paris):

WP3, blueprint; conceptual clarificationWP4, constraints from cognition-action to signals/signs processingWP6, integration internal models-expressive behaviorsWP8, mental states underlying external manifestations of persuasion

Co-organization (with WP3) of symposium on architectures of computational models at Plenary 2, May 2005Working visits (UH & OFAI to GERG & MIRALab, etc)Support actions (sessions) to WPs 3, 6, 8 planed at workshop (July 2005)


Dissemination & “external” activities

Symposium “Architectures for Modeling Emotion”, AAAI Spring Symposium, Stanford, March 2004Symposium “Dimensions of Sociality”, Vienna, Nov. 2004Symposium “Motivational and Emotional Roots of Cognition and Action”, AISB’05 @ UH, April 2005Symposium “Mind-Minding Agents”, AISB’05Co-organization (with WP3) symposium “Architecture of Computational Models” at ISRE in Bari, July 2005Co-edition (with WP6) special issue Humanoid RobotsVarious press reports, numerous scientific articles published or submitted (e.g. contribution to special issue NNets)


Input from March 2005 project review

No “recommendations” for WP7 but some “comments”:C1: Establish more clearly synergies with other WPs

WP3: interplay theories / implementations meetingsWP6 (+WP4): complementarity towards human-like capabilities (special issue); emotion-attention interplay for social interactionWP8: cognitive emotion models for dialog, communication, persuasionWP10: working towards standards (joint handbook chapter); ethics

C2: Distinction between “lower-level” and “higher-level” misleadingElements renamed to make focus more precise and avoid confusion

C3: Provide more details on plans to “bridge the gap”Element 3 re-structured and made more concrete

C4: Robotic implementations shouldn’t be toy demonstrations of problems

Closer integration with emotion theory and formal analysisMore prominent use of ECAs


Core achievements in 2005

Definition of exemplar (D7d)WP 7 workshop, London, July 2005 (D7a)Other workshops to develop/support WP7 exemplar but not funded by HUMAINENew partner (CNRS-EPML38, development), involvement of other new partners (USC, CNR)Contribution to 2005 “high quality” dissemination deliverable and co-edition of 2006 oneJoint conceptual, design, implementation workPublications Other dissemination activities and esteem factors


Feedback from March 2006 review

Develop potential synergies with other projects

IST’06 Networking Session

Further improve links with the other workpackages and provide clear assessment means of this progress

Co-edition of special journal issue with WP6Cross-currents symposiumACE’06 symposiumInvolvement of WP4 in follow-up proposal


Core achievements in 2006

Good progress in the 4 elements of the exemplar

Increased links with other WPs & projectsSessions: Cross-currents, Summer School, IST’06, ACE’06

Co-edition of 2006 “high quality” dissemination deliverable with WP6 (IJHR special issue)

Publications 9 joint (4 journal, 5 conf / wksp), 21 single institution (8 journal, 13 conf / w)

Outline book submissions

Other dissemination activities and esteem factorsEdition (4), conf. organization (ACII, ACE, Ro-Man, EpiRob), inv. talks (10)

Follow-up project merging E1 + E4 (+ WP4-WP6): FEELIX GROWING


E1: Emotion in embodied cognition and action

Interactions between emotion & cognition-action as occurring through the “body”

UH, EPML38, Paris8, KCL , GERG

Subtasks:E1.1 – Emotional modulation of perception-action in embodied agents: proximal causes, development, evolution

E1.2 – Analysis of embodied emotion-oriented architectures and behavior of robots: ethological + mathematical

E1.3 – Novelty detection and emotion-attention interactions (ECAs)


Modulation of Per-Ac loops

Proximal causation Development

Actuator Sensors

E

D

Fixed Nodes

Depot NodeInternalSensors

Possible Initial Network

E

D

Evolution

QuickTime™ and a decompressor

are needed to see this picture.

Motivations

Mfatigue

Mcold

Bfeed

Bwarmup

Behaviors

Physiology

sheat

sfoo

d

dtemp

denergy

External Stimulus

Bumper Bavoid

Bsearch




E2: Emotion in reflective cognition and action

Influence of emotions in cognition-action from the perspective of subjective perception and reasoning (introspection, linguistic accounts)

Bari, CNR, FT-RD, UM, USC

Subtasks:Role of BDI&E models and relation to rationality and psych. theory:

Emotional conflict, cognitive dissonance

Emotion and anticipation

Validation of cognitive models of emotion activation by means of ‘sensitivity analysis’ & their extension to the ‘interpretation’ of emotional expressions displayed by the user

Application & comparison of models for emotion activation and recognition to dialogs


Emotional Mind

DynamicEmotion table

Selecting a context and a personality

Selecting an eventFiring the

event

Simulationhistory

Mind at T0

Setting a personality set and a context set


Emotional Mind in action




E3: Emotions in bridging gap

Role of emotions in relating behavioral meaning and symbolic representations

OFAI, HW, USC, INESC, IST; GERG, KCL, UOXF, UM

Subtasks: E3.1 – A scenario-based survey of bridging functions of emotions

E3.2 – Improving upon symbolic models of reflective cognition & action

E3.3 – Improving upon embodied models of cognition & action

E3.4 – Bridging the gap between micro- (individual-based) and macro- (social) views on social functions of emotion


Scenario-based evaluation and design

Goal: Understand state of modeling across different disciplinesChallenge: Substantial differences in concrete scenarios addressed (over 12). Different affect-related phenomena modeled at different granularities in settings of different complexityApproach: to compare systems, the functional role of “emotion” (use of term) must be explicatedFocus on architectural building blocks:

Data structures, processes, interactions between processesFixed vs. dynamic/implicit paths of communicationExplicit differentiation of contexts of information processing (“modules”, “levels”, “stages”,…)

Bridging between such contextsDerive best practices for the development of computational models of emotion


Improving upon symbolic reflective models

Suitable building blocks to model emotional processes?

Question foundations of symbolic architectures: symbolic “shortcuts” need to be motivated explicitly

Parallel embodied processes as basic behavioral components

Challenge: realize reflective and symbolic processes “on top”

Concurrent processes and resource management

Resources

Processes

Meta-Processes


Improving upon embodied models

Further extension of previous work on emergent affective and personality model that integrates perception, motivation, action selection, planning and memory

Autobiographical memory

Group level dynamics

AutobiographicMemory Model(Wan Ching Ho)

EmotionalParameters

Synthetic GroupDynamics


Bridging micro-macro gap

From inwards-oriented appraisal towards social-communicative behavior

Social emotions as result of supra-individual process of co-regulated reactions

Extension of appraisal theory modelAssessment of sequential evaluation check model

Improving sensing and rapidly reacting to human emotional signals

Integration of socially situated theory emphasizing centrality of social goals and contingent behavior

Planned workshop (October 2007, USC/ISI)


E4: emotions in social cognition and interaction

Roles of emotions in social cognition and interaction; emotions, cognition and action not modeled from the perspective of the individual but of the

interaction itself.

OFAI, MIRALab, EPFL, ICCS-NTUA, DIST, UH, EPML-38, UBari, CNR, U. Sheffield

Subtasks:E4.1 – Towards socially meaningful emotional agents: Closing the emotion recognition-generation-expression loop

E4.2 – Socially situated nature of emotions: Socially situated affective dialogue


Closing the emotion rec-gen-exp loop

QuickTime™ and aDV - PAL decompressor


Emotion recognition fromfull body motion

ECA copying observedexpressive gestures




The Premio Paganini experiment

Stimulus material : Canon from the Musical Offering (J.S Bach)

Measure of motoric activation: 4 special videocameras (50fps)

Audio: recording of both direct violin and in ambience

Physiological data (BioMuse): ECG (Electrocardiogram) EMG (Electromyogram)

Multimodal integration: Data Synchronization and Analysis (EyesWeb XMI)

Description of the set up


Cross-currents symposium, June 2006

1. Dynamical systems as a framework to bridge gaps in emotion research? L. Cañamero (UH, coord), R. te Boekhorst (UH), A. Flykt (Mid Sweden U), P. Gaussier (EPML38), N. Korsten (KCL)

2. Closing the emotion recognition-generation-expression loop J. Gratch (USC, coord), A. Blanchard (UH), G. Castellano (DIST), A. Egges (Miralab), K. Karpouizis (ICCS), C. Peters (Paris8)

3. Beyond the blackbox vs process models alternative: reflective emotion models in comparison, with their mental ingredients, grain size, application perspectives and limits

F. de Rosis (Bari, coord), Peter Goldie (UM), Stacy Marsella (USC), Sabine Payr (OFAI), Isabella Poggi (CNR)

4. Avenues to bridge gaps between "embodied" and "reflective" systems P. Petta (OFAI, coord), Nienke Korsten (KCL), Robert Marsh (UH), Sandy Louchart (HW), Fiorella de Rosis (Bari)


Networking Session @ IST06

“Embodied Emotion, Cognition and Action for Autonomous and Interactive Artifacts”

Aims:provide framework to explore opportunities for interaction among projects with a common interest in embodied emotion and cognition

draft a longer-term “research agenda” for this area

Presentations FP6 projects: HUMAINE, euCognition, ICEA, ENACTIVE, MindRACES, S2S2, TAI-CHI, CALLAS

Challenges, needs and other projects identified

Over 100 participants

Follow-up session @ Plenary07


Book: proposed submissions

4 chapters from E1: (1) modulation Per-Ac loops; (2) neuromodulation; (3) dynamical systems analysis; (4) novelty and attention

3 chapters E2:(5) emotional conflict; (6) empathic dialogue agent; (7) emotion and anticipation

4 chapters E3:(8) AS architecture for virtual humans; (9) hybrid affective mind; (10) improving upon symbolic models; (11) improving upon embodied models

5 chapters E4:(12) emotion sharing & understanding; (13) PerAc models of imitation; (14) analysis of movement dynamics for emotion recognition; (15) full-body motion and gesture analysis for recognition; (16) socially situated affective dialogue

External input to each section


Socio-emotional development

ICCS

FEEL, Interact, eXpress: a Global appRoach to develOpment With INterdisciplinary GroundingFP6-IST-045169, December 2006 – May 2010


Objectives

Identification of key evaluation scenarios (types of problems) in global socially situated development of autonomous agents

identify cross-disciplinary benchmarks (scenarios and methods) for a comparative evaluation

Investigation of the roles of emotion, interaction, expression and their interplays in bootstrapping & driving socially situated development

implementation and testing of robotic systems that improve existing work

Integration of: (a) the above “capabilities” in at least 2 different robotic prototypes, and (b) feedback across the disciplines involved

platform for grounded long-term multidisciplinary research (roadmap)

Identification of needs towards achieving standards in: (a) design of scenarios and problem typologies,

(b) evaluation metrics,

(c) design of everyday robotic platforms and related technology.


WP4: “Feel” and development

TasksCross-disciplin. training & critical analysis models of emotion in development Sample key work in psychology to inspire / support robotic studies

Emotion elicitation in spontaneous vs induced imitationRoles of + & - emotion in attachment and emotion regulation

Implementing & testing in robots selected key aspects: Hedonic processes and their roles in motivation and emotion regulation in social interactionSelected mechanisms for the detection / recognition of emotions in social interactions (modal & amodal)Attachment processes & their roles in exploration, learning and adaptation to social environment


“Interact” and development

TasksCross-disciplin. critical analysis models of interaction in development

Sample key work in psychology to inspire / support robotic studies Emotions in social referencing

Emotion in joint attention (chimps w differential rearing conditions)

Implementing & testing in robots selected key aspects: Joint attention, particularly the role of gaze direction

Task learning by observation/imitation; effect of (emotional) user feedback

Interaction and “emotional resonance”


“Express” and development

TasksCross-disciplinary critical analysis models of expression in development Sample key work in psychology to inspire / support robotic studies

Normal & impaired development of emotional resonance & recognitionPerception of emotion in human vs robotUse of FACS for robots

Implementing & testing in robots selected key aspects: Development of emotional expression related to social interactionUse of expression as signalling for communication (no link to “internal” emotional state)Use of expression as manifestation of an “internal” emotional state

Comparative Approaches to Emotion-Oriented Architectures (WP 7: Emotion in Cognition and Action)

Documents

Transcript of Comparative Approaches to Emotion-Oriented Architectures (WP 7: Emotion in Cognition and Action)