User Experience: Beyond Cognition and Emotion UX-next... · undersandni g, or creri a i for...
Transcript of User Experience: Beyond Cognition and Emotion UX-next... · undersandni g, or creri a i for...
-
User Experience: Beyond Cognition and Emotion … Matthias RAUTERBERG Eindhoven University of Technology – TU/e The Netherlands 2014
-
Interaction Paradigms in Computing
time 1960 2020
Personal computing Man-machine-interaction
1980 2000
Cooperative computing Computer-mediated-interaction
Social computing Community-mediated-interaction
Cultural computing Cross cultural-interaction
© Matthias Rauterberg, 2014 Eindhoven University of Technology 2/28
-
What is Culture? Culture is the integration pattern of human behavior that includes - attitudes, - norms, - values, - beliefs, - actions, - communications and language - institutions of a race, ethnic, religious and/or social group. The word culture comes from the Latin root colere (to inhabit, to cultivate, or to honor). In general, it refers to human activity; different definitions of culture reflect different theories for understanding, or criteria for valuing, human activity. Anthropologists use the term to refer to the universal human capacity to classify experiences, and to encode and communicate them symbolically. They regard this capacity as a defining feature of the genus Homo.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 3/28
-
Attitudes Norms Values Beliefs
Etc.
Attitudes Norms Values Beliefs
Etc.
conscious
unconscious
© Matthias Rauterberg, 2014 Eindhoven University of Technology 4/28
“It is not the consciousness of men that determines their being, but, on the contrary, their social being that determines their consciousness.”
Karl MARX [1818-1883]
-
© Matthias Rauterberg, 2014 Eindhoven University of Technology 5/28
Salem, B.; Nakatsu, R.; Rauterberg, M. (2009). Kansei experience: Aesthetic, emotions and inner balance. International Journal on Cognitive Intelligence and Natural Intelligence, vol. 3, no. 2, pp. 18-36.
-
Daniel Kahneman Map of Bounded Rationality: A Perspective on Intuitive Judgement and Choice . Nobel Prize Lecture, 8 December 2002
© Matthias Rauterberg, 2014 Eindhoven University of Technology 6/28
-
consciousness * propositional representation
* synchronization via global workspace
phenomenal consciousness * internal processing
unconsciousness * parallel processing in modular subsystems
verbal behaviour
nonverbal behaviour
multi modal sensory input
mul
ti m
odal
act
uato
r out
put
Hofmann, W., & Wilson, T. (2010). Consciousness, introspection, and the adaptive unconscious. In B. Gawronski & B. K. Payne (Eds.), Handbook of Implicit Social Cognition: Measurement, Theory, and Applications (pp. 197-215). New York: Guilford Press.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 7/28
-
© Matthias Rauterberg, 2014 Eindhoven University of Technology 8/28
Sandy PENTLAND
-
After 1910 the discoveries of Carl Gustav JUNG about the collective unconscious and the related archetypes were challenging. Jung dreamt a great deal about the dead, the land of the dead, and the rising of the dead. These represented the unconscious itself -- not the "little" personal unconscious that Freud made such a big deal out of , but a new collective unconscious of humanity itself , an unconscious that could contain all the dead, not just our personal ghosts. Jung began to see the mentally ill as people who are haunted by these ghosts, in an age where no-one is supposed to even believe in them. If we could only recapture our mythologies, we would understand these ghosts, become comfortable with the dead, and heal our mental illnesses.
(1875-1961)
© Matthias Rauterberg, 2014 Eindhoven University of Technology 9/28
-
© Matthias Rauterberg, 2014 Eindhoven University of Technology 10/28
-
© Matthias Rauterberg, 2014 Eindhoven University of Technology 11/28
The Hero's Journey
Joseph CAMPBELL [1904-1987]
-
A collaborative project between two PhD students
Leonid IVONIN
• Generation of ideas • Statistical analysis • Writing of articles
Shared activities
• Processing of physiological signals and application of data mining methods
• Development of technical infrastructure for the experiments
• Sensing application (ArcheSense)
Huang-Ming CHANG
• Identification, selection, and preparation of audiovisual stimuli for elicitation of psychological states
• Selection of appropriate questionnaires
• Presenting application (ArcheBoard)
© Matthias Rauterberg, 2014 Eindhoven University of Technology 12/28
How to design for the unconscious?
-
2.Elicitation 3.Recognition Emotional
Responses
Mapping
Affective Stimuli
1.Explicit Emotions (e.g. joy, anger, sadness, etc.)
The State of the Art
© Matthias Rauterberg, 2014 Eindhoven University of Technology 13/28
-
Elicitation Recognition Smile
Mapping
The State of the Art
The emotion of Joy
© Matthias Rauterberg, 2014 Eindhoven University of Technology 14/28
-
• Perceptual Quality Black, White
• Physical Content Dove
• Symbolic meaning Peace, Spirit, etc.
What induces emotions?
Problem of Current States
Emotion Appraisal
• Barrett, L. F. (2012). Emotions are real. Emotion, 12(3), 413–29.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 15/28
-
Number of participants: • 34
Two types of stimuli: • Visual (pictures) • Auditory (sounds)
Five categories of stimuli: • Archetypal • Positive-relaxing • Positive-arousing • Neutral • Negative
Six stimuli per category Physiological data:
• Heart Rate • Galvanic Skin Response • Skin Temperature Examples of visual stimuli
The findings indicated a significant relationship between the categories of stimuli (including archetypal) and physiological signals.
Hypothesis: Psychological states related to archetypal stimuli would lead to different patterns of physiological activations.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 16/28
Study 1 - method
-
Number of participants: • 36
Two types of stimuli: • Film clips with explicit emotions • Film clips with archetypal experiences
Five categories of explicit stimuli: • Neutral • Amusement • Fear • Joy • Sadness
Physiological data: • ECG (HR + HRV) • Skin conductance (response + level) • Respiration • Skin temperature
Duration of each stimulus: Approximately 5 minutes
Self-report ratings after every stimulus
Eight categories of archetypal stimuli: • Anima • Animus • Hero departure • Hero initiation • Hero return • Mentor • Mother • Shadow
Hypothesis: Various archetypal experiences lead to recognizable patterns of physiological activations that could be differentiated using computational intelligence algorithms.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 17/28
Study 2 - method
-
Dynamic patterns of the heart rate responses of the participants to the film clips presentations.
The mean values and 95% confidence intervals of the HR are represented with the bold lines and the vertical bars for each of the psychological condition.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 18/28
Study 2 - results
-
kNN SVM Naïve Bayes LDA
Archetypal experiences
Classification rate 78 82 85.5 83
Cross-validated classification rate 74 75.5 79.5 79.5
Explicit emotions
Classification rate 77.6 75.2 78.4 77.6
Cross-validated classification rate 72 71.2 74.4 74.4
Archetypal experiences and explicit emotions
Classification rate 62.8 55.7 72.6 69.8
Cross-validated classification rate 49.8 68.3 57.2 61.2
Classification Performance Achieved with Different Methods
© Matthias Rauterberg, 2014 Eindhoven University of Technology 19/28
Study 2 - results
Classification methods: k-nearest neighborhood (kNN), support vector machine (SVM), naïve Bayes, and linear discriminant analysis (LDA), Only the best accuracy is reported.
-
More information about ArcheSense can be found at http://hxresearch.org.
ArcheSense is a tool for evaluation of human experience with products or media based on physiological data of people.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 20/28
A tool for evaluation of unconscious human experience
ArcheSense tool
http://hxresearch.org/�
-
Number of subjects: • 23
Two types of stimuli: • Film clips with explicit emotions • Film clips with archetypal experiences
Five categories of explicit stimuli: • Neutral • Active-pleasant • Active-unpleasant • Passive-pleasant • Passive-unpleasant
Physiological data: • ECG (HR + HRV) • Skin conductance (response + level)
Duration of each stimulus: • Approximately 1 minute
Number of stimuli per category: 3 Self-report ratings after every stimulus
Seven categories of archetypal stimuli: • Anima • Hero departure • Hero initiation • Hero rebirth • Hero return • Mentor • Mother • Shadow
Hypothesis: Various archetypal experiences lead to recognizable patterns of physiological activations that could be differentiated using computational intelligence algorithms.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 21/28
Study 3 - method
-
© Matthias Rauterberg, 2013 22/6
-
Rain drops down
Reborn from Fire
Reborn from Fire and Thunder Strom
“God is in the rain.”
© Matthias Rauterberg, 2014 Eindhoven University of Technology 23/28
-
Categories of the film clips Number of states
Self-reports Physiological data
Anima, hero departure, hero initiation, hero rebirth, hero return, mentor, shadow
7 28.0 36.7
Anima, hero departure, mentor, shadow 4 42.0 53.3
Anima, hero initiation, mentor, shadow 4 43.1 57.1
Anima, hero rebirth, mentor, shadow 4 38.4 52.9 Anima, hero return, mentor shadow 4 40.6 56.1
Active-pleasant, active-unpleasant, neutral, passive-pleasant, passive-unpleasant
5 50.4 50.7
Active-unpleasant, neutral, passive-pleasant, passive-unpleasant
4 64.9 57.2
Classification methods: k-nearest neighborhood (kNN), support vector machine (SVM), naïve Bayes, linear discriminant analysis (LDA), and Adaptive Boosting with decision trees (AdaBoost). Only the best accuracy is reported.
Comparison of the classification accuracy achieved using the self-report questionnaires and the physiological data (between-subject classification).
Archetypes
Explicit emotions
© Matthias Rauterberg, 2014 Eindhoven University of Technology 24/28
Study 3 - results
-
Study #1: • ‘Exploratory’ study • Pictures and sounds (presentation for 6 sec) • 1 archetype, 4 explicit emotions • Between-subject design • Statistically significant results • 23.3% classification accuracy (5 classes)
Study #2: • Film clips (length 5 min) • 8 archetypes, 5 explicit emotions • 1 stimuli per class • Between-subject design • Statistically significant results • 79.5% classification accuracy (8 classes)
Study #3: • Film clips (length 1 min) • 7 archetypes, 5 explicit emotions • 3 stimuli per class • Between-subject and within-subject designs • Statistically significant results • 57.1% classification accuracy (4 classes, between-subject),
70.3% classification accuracy (7 classes, within-subject)
© Matthias Rauterberg, 2014 Eindhoven University of Technology 25/28
Overview
-
Performance of Predictive Models
Recognition Technique
Affective Stimuli Archetypal Symbols Explicit Emotions
Self-report Data (Conscious) Poor Superb
Physiological Data (Unconscious) Good Good
Classification Rate in General
Implicit Emotion Emotion toward Archetypal Symbols
State of the Art
< =
© Matthias Rauterberg, 2014 Eindhoven University of Technology 26/28
-
Three main conclusions can be drawn: (1) Conscious and unconscious reactions are different.
(2) The unconscious is sensitive to archetypes.
(3) The unconsciousness is at least important as the consciousness.
© Matthias Rauterberg, 2014 Eindhoven University of Technology 27/28
-
Thank you for your attention. A door goes open to a new world…
© Matthias Rauterberg, 2014 Eindhoven University of Technology 28/28
-
© Matthias Rauterberg, 2013 29/27
Chang H.M., Ivonin L., Díaz M., Català A., Chen W., Rauterberg M. (2014). Enacting archetypes in movies: Grounding the unconscious mind in emotion-driven media.Digital Creativity (published online) [IF=0.370] [SJR]. Chang HM., Ivonin L., Díaz M., Català A., Chen W., Rauterberg M. (2014). Unspoken emotions in movies: The basis of emotion-driven storytelling systems. Informatik-Spektrum, vol. 37, no. 6, pp. 539-546 (online) [IF=0.148] [SJR]. Ivonin L., Chang HM., Díaz M., Català A., Chen W., Rauterberg M. (2014). Beyond cognition and affect: Sensing the unconscious. Behaviour & Information Technology(published online) [IF=0.856] [SJR]. Chang H.M., Ivonin L., Díaz M., Català A., Chen W., Rauterberg M. (2013). From mythology to psychology: Identifying archetypal symbols in movies. Technoetic Arts: A Journal of Speculative Research, vol. 11, no. 2, pp. 99–113 [SJR]. Ivonin L., Chang H.M., Chen W., Rauterberg M. (2013). Unconscious emotions: Quantifying and logging something we are not aware of. Personal and Ubiquitous Computing, vol. 17, no. 4, pp. 663-673 [IF=2.853] [SJR]. Chang H.M., Diaz M., Catala A., Chen W., Rauterberg M. (2014). Mood boards as a universal tool for investigating emotional experience. In: A. Marcus (ed.), Design, User Experience and Usability - DUXU 2014, (Part IV, Lecture Notes in Computer Science, vol. 8520, pp. 220–231), Berlin - Heidelberg New York: Springer. Chang H.M., Ivonin L., Diaz M., Catala A., Chen W., Rauterberg M. (2013). Experience the world with archetypal symbols: A new form of aesthetics. In: N. Streitz, C. Stephanidis (eds.), Distributed, Ambient, and Pervasive Interactions DAPI - HCI international (Lecture Notes in Computer Science, vol. 8028, pp. 205–214), Berlin - Heidelberg: Springer [IF=0.499]. Chang H.M., Ivonin L., Diaz M., Catala A., Chen W., Rauterberg M. (2013). What do we feel about archetypes: Self-reports and physiological signals. In: Proceedings of 21st European Signal Processing Conference (September 9-13, 2013, Marrakech, Morocco; paper no. 1569742621). Chang H.M., Ivonin L., Chen W., Rauterberg M. (2013). Feeling something without knowing why: Measuring emotions toward archetypal content. In: M. Mancas, N. d’ Alessandro, X. Siebert, B. Gosselin, C. Valderrama, T. Dutoit (eds.), Proceedings of Intelligent Technologies for Interactive Entertainment - INTETAIN (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 124, pp. 22–31), © Institute for Computer Sciences, Social Informatics and Telecommunications Engineering: Springer.
http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DIGICREAT2014journal-online.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DIGICREAT2014journal-online.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DIGICREAT2014journal-online.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DIGICREAT2014journal-online.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DIGICREAT2014journal-online.pdf�http://www.tandfonline.com/doi/full/10.1080/14626268.2014.939985�http://www.scimagojr.com/journalsearch.php?q=144891&tip=sid&clean=0�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INFSPEK2014journal.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INFSPEK2014journal.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INFSPEK2014journal.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INFSPEK2014journal.pdf�http://link.springer.com/journal/287/37/6/page/1�http://link.springer.com/journal/287/37/6/page/1�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INFSPEK2014journal-online.pdf�http://www.scimagojr.com/journalsearch.php?q=145041&tip=sid&clean=0�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/BIT2014journal.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/BIT2014journal.pdf�http://www.tandfonline.com/doi/full/10.1080/0144929X.2014.912353�http://www.scimagojr.com/journalsearch.php?q=12125&tip=sid&clean=0�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/TechnoeticArts2013journal.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/TechnoeticArts2013journal.pdf�http://www.scimagojr.com/journalsearch.php?q=6000153223&tip=sid&clean=0�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/PUC12journal.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/PUC12journal.pdf�http://www.scimagojr.com/journalsearch.php?q=22315&tip=sid&clean=0�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DUXU2014-paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/DUXU2014-paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/HCIinternational2013paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/HCIinternational2013paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/EUSIPCO2013paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/EUSIPCO2013paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/EUSIPCO2013paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/EUSIPCO2013paper.pdf�http://www.eurasip.org/Proceedings/Eusipco/Eusipco2013/papers/1569742621.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INTETAIN2013paper.pdf�http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/INTETAIN2013paper.pdf�
Slide Number 1Slide Number 2Slide Number 3Slide Number 4Slide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Slide Number 20Slide Number 21Slide Number 22Slide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29