Emotion-Based Interactive Installation

22
Emotion-Based Interactive Installation Francisco Ventura, António Pedro Oliveira and Amílcar Cardoso University of Coimbra, Portugal

Transcript of Emotion-Based Interactive Installation

Page 1: Emotion-Based Interactive Installation

Emotion-Based Interactive Installation

Francisco Ventura, António Pedro Oliveira and Amílcar Cardoso

University of Coimbra, Portugal

Page 2: Emotion-Based Interactive Installation

2

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 3: Emotion-Based Interactive Installation

3

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 4: Emotion-Based Interactive Installation

Introduction

4

We prepared an installation to explore the interactive capabilities of an Emotion-Driven Music Engine (EDME)

This installation allows participants to experience and influence the emotional state of a computational system

The system perceives emotions from the environment by using a camera and expresses emotions to the environment with music and images by using, respectively, loudspeakers and a projector

Page 5: Emotion-Based Interactive Installation

5

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 6: Emotion-Based Interactive Installation

EDME

6

EDME is a computer program that intends to produce music expressing a desired emotion

It is composed by an offline stage and a real-time stage

At the offline stage, pre-composed music is segmented, emotionally classified and stored in a music base

At the real-time stage, EDME selects from a music base the segments that are closer to a desired emotion

Page 7: Emotion-Based Interactive Installation

EDME

7

Transformation and sequencing algorithms are applied on selected segments to form song-like structures

These song-like structures are then synthesized and given to the listener

Real-Time Stage

Offline Stage

Page 8: Emotion-Based Interactive Installation

8

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 9: Emotion-Based Interactive Installation

Interactive Installation The installation is composed by a computer

system that uses a camera to take pictures of the environment

Page 10: Emotion-Based Interactive Installation

Interactive Installation The installation is composed by a computer system that uses a camera to take pictures of the environment

The system analyses these pictures to recognize the number of participants and quantity of movement

System recognizes:• Number of Participants• Quantity of Movement

Page 11: Emotion-Based Interactive Installation

Interactive Installation The installation is composed by a computer

system that uses a camera to take pictures of the environment

The system analyses these pictures to recognize the number of participants and quantity of movement

These two variables are used to define, respectively, the values of valence and arousal of the emotional state of the system

System recognizes:• Number of Participants• Quantity of Movement

Emotional State of the

System

Page 12: Emotion-Based Interactive Installation

Interactive Installation

12

The installation is composed by a computer system that uses a camera to take pictures of the environment

The system analyses these pictures to recognize the number of participants and quantity of movement

These two variables are used to define, respectively, the values of valence and arousal of the emotional state of the system

The emotional state influences the expression of music and images to the environment

System recognizes:• Number of Participants• Quantity of Movement

Emotional State of the

System

Emotional expression (Music & Images)

Page 13: Emotion-Based Interactive Installation

Interactive Installation

13

The number of participants is determined by the percentage of change between scenario (image taken when the system starts) and current image

Quantity of movement is determined by the change between current and last image

Page 14: Emotion-Based Interactive Installation

Interactive Installation

14

EDME generates music according to the values of valence and arousal of the system

As images are emotionally classified with valence, the selection of images is only based on this emotional dimension

Page 15: Emotion-Based Interactive Installation

Interactive Installation

15

The emotional behaviour of the system is controlled by a multi-agent system

Page 16: Emotion-Based Interactive Installation

16

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 17: Emotion-Based Interactive Installation

Experiments

17

We made a preliminary experiment with students with the objective of observing the interaction between the system and participants

Page 18: Emotion-Based Interactive Installation

Experiments

18

The second experiment was focused in obtaining participants feedback via a questionnaire of 7 questions

Example of questions: Music expressed expected emotion? Importance of images in the emotional expression? System expressed expected emotion?

Page 19: Emotion-Based Interactive Installation

Experiments

19

Answers from the questionnaire show us that: the system correctly related arousal with the amount of

movement and valence with the number of participants the system was successful in the expression of emotions with

both music and images

Page 20: Emotion-Based Interactive Installation

20

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 21: Emotion-Based Interactive Installation

21

Outline Introduction EDME Interactive Installation Experiments Demonstration Conclusion

Page 22: Emotion-Based Interactive Installation

Conclusion

22

EDME is prepared to be used in interactive contexts

People experienced and influenced the emotional state of the system

Both music and images are relevant in the emotional expression of the system

Installation assumes no variation of environmental conditions; however, it can be adapted to deal with this situation by changing the scenario