Affective Computing andHuman Robot Interaction

a short introductionJoost Broekens

Telematica Institute, Enschede,

LIACS, Leiden University.

Overview

• What is emotion?

• What theoretical angles can we take to study emotion?

• What is affective computing + examples ?

• Why is affective computing useful?

15 min coffee break.

• Interactive Human-Robot Interaction

• Human emotional expressions as feedback to a learning robot

Emotion (what is it)?

• Common emotions: fear, anger, happiness, sadness, surprise, disgust

• Short episode of synchronized system activity triggered by event:

– subjective feelings (the emotion we normally refer to),– tendency to do something (action preparation),– facial expressions,– evaluation of the situation (cognitive evaluation, thinking),– physiological arousal (heartbeat, alertness).

• Affect (affectie)= related to emotion and mood:– emotion : short term, high intensity, object directed,– mood : unfocused, long term, low intensity,– affect : sometimes seen as abstraction of emotion/mood in terms of,

• positiveness/negativeness and activation/deactivation (Russell).

Emotion (why do we have it)?

• Situational evaluation and communication.

• Heuristic relating events to personal goals, needs and beliefs:– evaluates personal relevance of event (Scherer) and helps decision-

making (Damasio),– fast reactions and action preparation (Frijda),– influence information processing and decision making (Damasio):

• e.g, adaptation, attention, search.

• Communication medium:– communicate internal state (show feelings),– alert others (direct attention),– show empathy (show understanding of situation).– Give feedback (evaluate behavior).

Emotion (what does it do)?

• Emotion and affect influence thought and behavior:– The kind of thoughts we have

• Mood congruency (e.g., positive moods triggers positive thought)

– The way we process information• Broad vs. narrow look (Goschke & Dreisbach) (forest vs. trees)• A lot vs. a little processing effort (Scherer, Forgas)

– What we think about things• Emotion/mood as information (Clore & Gasper) (how does this feel?)• Emotion as belief anchor (Frijda & Mesquita) (idea “fixation”)

– How we learn and adapt• Emotion/affect as social reinforcement (emotion expression as signal)• Emotion/affect as intrinsic reinforcement (use my own emotion as feedback)• Emotion as “metaparameter” to control learning process (explore vs. exploit)

Emotion (how can we study it)?

• Biological/Neurological (Damasio, Panksepp, LeDoux, Rolls):– aim: find the necessary / sufficient brain areas and circuits involved in

emotions /feelings / self-regulation / adaptation.

• Biological/Evolutionary (Darwin, Ekman):– aim: why emotions exist in the first place, what’s the utility of emotion.

• Cognitive/Psychological (Scherer, Frijda):– aim: understand relation cognition/emotion, why does event e results in

emotion x while:• the same event e may result in a different emotion, and• other events may result in the same emotion x.

• Social (Ekman, and others):– aim: understand the role of emotion in communication.

What is affective computing?

• Computing that relates to, arises from, or deliberately influences emotions (Picard).

• Different types of computer models:– recognize emotions (perception, in),– interpret/elicit emotions (cognition/behavior, processing),– emotional influence on behavior (adaptation, attention),– show emotions (expression, out), and– complex mixtures of these…

• An affective computing system is– a system of computational processes that perceives, expresses, interprets,

or uses emotions,– e.g. a robot, a virtual character, a tutor agent, a fridge, etc…

Some examples…

SIMS 2 (Electronic Arts)

• Entertainment: emotions are used to provide entertainment value.

Kismet (Breazeal)

• Social: Kismet, A framework, using a humanoid head expressing emotions, to study:– effect of emotions on human-machine interaction.

– learning of social robot behaviors during human-robot play.

– joint attention.

Mission Rehearsal Exercise (Gratch & Marsella)

• Cognitive: study the influence of artificial emotions on – planning mechanism of virtual characters,

– training effect on trainees (emotion might enhance effect)

Assistive robotics

• Aibo (Sony, Japan)Entertainment robot

• I-Cat (Philips, NL)Robot assistant for elderly people

• Paro (Wada et al, Japan)Robot companion for elderly

• Huggable (MIT, USA)Robot companion for elderly

Why is affective comp. useful (1)?

• From a theoretical point of view…

• Advance emotion theory– simulated agents that elicit emotions to study possible psychological

structure of emotions (Scherer),– emotion as artificial motivator (e.g. a bored robot that explores)

(Canamero).

• Understand intelligence and adaptation– laws and rules are not sufficient for understanding or predicting human

behavior and intelligence (e.g. how to sift thru many possible choices) (Damasio, Picard)

– simulated learning agents influenced by emotions (e.g., emotion as reward) (Breazeal, Broekens),

– Artificial Intelligence, Artificial Life?

Why is affective comp. useful (2)?

• From a practical point of view…

• Facilitate Human-Computer (Robot) interaction– use emotions in simulated-agent plans (to simulate human reasoning)

(Gratch & Marsella),– communication and joint attention (Breazeal, MIT)– robot acceptation (Heerink, Telin)– Persuasive design (VR training, tutor agents (Gratch & Marsella,

Nijholt))– Interactive robot learning (second part of this lecture).

• Entertainment– Computer games such as the Sims (EA), Aibo Robot (Sony).

Affective Computing in short…

• Affect has different meanings related to emotion.– emotion : short term, high intensity, object directed,– mood : unfocused, long term, low intensity,– affect : sometimes seen as abstraction of emotion/mood in terms of,

• positiveness/negativeness and activation/deactivation.

• Affective computing is about computing with emotions– “a system of computational processes that perceives, expresses, interprets, or uses

emotions”

• Why?– advance emotion theory,– understand intelligence,– facilitate Human-Computer (Robot) interaction,– entertainment.

Interactive Robot Learning

(Real-time interaction to control robot behavior)

Interactive Robot Learning:a special case of HRI

• Goals Human Robot Interaction:– Develop more natural and effective ways of interacting between robots

and humans.– Allow non-expert interaction.– Allow flexibility of robot behavior.

• How?– Gesture recognition– Cognitive robot architectures– Machine learning (artificial adaptation)– Affective computing

• Emotion recognition• Emotion expression

Example, Guidance and Feedback

• Interactive robot learning– Learning by Example

• Imitation learning (see Breazeal & Scassellati)• E.g., gesture repetition

– Learning by Guidance (Thomaz & Breazeal)• In general: future directed learning cues, such as• Directing attention• Communicating motivational intentions (e.g., anticipatory reward)• Proposing actions

– Learning by Feedback• Learn by trial (+feedback) and error (-feedback) and exploration.• Additional human-based reinforcement signal (Breazeal & Velasquez; Isbell

et al; Mitsunaga et al; Papudesi & Hubert)

Why study Interactive Robot Learning?

• Robot perspective: Interactive robot learning– Facilitate human-robot interaction– Study learning and adaptation– Future:

• enable robots to interactively cooperate in an efficient manner with humans in ways that are natural to humans.

• Psychological perspective: Understanding mechanisms– How can affect influence learning?– How can affect influence attention processes?– Joint Human-Robot attention– Study learner-teacher relations– Etc..

EARL

• A framework to study the influence of Emotion on Adaptive behavior in the context of Reinforcement Learning:– emotion as reinforcement (reward/punishment),

– emotion as influence on learning process,

– emotion as information,

– artificial emotion resulting from learning process

EARL

• A typical EARL setup:– Simulated robot in a

– Maze learning (navigation) tasks

– Reinforcement learning (RL) approach to robot learning• Learn by trial (+feedback) and error (-feedback) and exploration.

– Robot has own model of emotion

– Robot head to express emotion

– And…• Webcam and emotion recognition to interpret emotions

Experiment: Human Affect as Reinforcement to Robot

• An experiment to study interactive robot learning

• Simulated Robot learns to find food in a maze.– Explore unknown environment (try actions).

– Reinforcement Learning• Feedback from environment based on taken actions• + feedback=repeat• - feedback=don’t repeat

– Learn which sequence of actions gives best +feedback.

• Robot also uses human facial expressions as +/- feedback

Facial Expression as Reinforcement

• Affective signal as additional reinforcement– Web cam

– Emotional expression analysis

– Positive emotion (happy) = reward

– Negative emotion (sad) = punishment

– So: in addition to feedback from environment emotional expression is used in learning as rhuman, a social reward coming from the human observer

Sad / happy

- / +

Robot behaviorto screen

Reinforcement-based robot learning

path wall food humanfeedback

2

b

Food (+)

Agent

Wall (-)

HumanFeedback

Path

Start

Experiment: results

• Test learning difference between standard agent and social agents

– Standard agent uses rewards environment to update behavior.

– Social agent uses rewards from environment and reward from human emotional signal.

• Results:– Social agent learns the path to the food quicker

(just believe me, so that I don’t have to explain the graphs)

• Earl demo…

Interactive Robot Learning in short…

• A special case of Human Robot Interaction– Goal HRI: more efficient, flexible, personal, pleasant human robot interaction– Interactive Robot Learning: Imitation, Feedback, Guidance.

• Affective Interaction Feasible and useful– can be used as guidance (MIT robot studies), and– feedback (experiment shown).

• Why?– Robot perspective

• Facilitate human-robot interaction• Study learning and adaptation

– Psychological perspective• How can affect influence learning?• How can affect influence attention processes?• Joint Human-Robot attention• Study learner-teacher relations