4 The Perception of Emotion 3 2 in the Singing Voice · The Perception of Emotion in the Singing...
15
The Perception of Emotion in the Singing Voice Emilia Parada-Cabaleiro 1,2 , Alice Baird 1,2 , Anton Batliner 1,2 , Nicholas Cummins 1,2 , Simone Hantke 1,2,3 , Björn Schuller 1,2,4 1 Chair of Embedded Intelligence for Health Care and Wellbeing, Augsburg University, Germany 2 Chair of Complex and Intelligent Systems, University of Passau, Germany 3 MISP Group, MKK, Technische Universität München, Germany 4 GLAM - Group on Language, Audio & Music, Imperial College London, UK This work was supported by the European Union's Seventh Framework and Horizon 2020 Programmes under grant agreements No. 338164 (ERC StG iHEARu) and No. 688835 (RIA DE-ENIGMA).
Transcript of 4 The Perception of Emotion 3 2 in the Singing Voice · The Perception of Emotion in the Singing...
The Perception of Emotion in the Singing Voice
Emilia Parada-Cabaleiro1,2, Alice Baird1,2, Anton Batliner1,2, Nicholas Cummins1,2, Simone Hantke1,2,3, Björn Schuller1,2,4
1Chair of Embedded Intelligence for Health Care and Wellbeing, Augsburg University, Germany2Chair of Complex and Intelligent Systems, University of Passau, Germany
3MISP Group, MKK, Technische Universität München, Germany4GLAM - Group on Language, Audio & Music, Imperial College London, UK
This work was supported by the European Union's Seventh Framework and Horizon 2020 Programmes under grant agreements No. 338164 (ERC StG iHEARu) and No. 688835 (RIA DE-ENIGMA).
emotional content is an ever
increasing feature music retrieval
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 2
In a digital world
Web emotional playlist as…
Music can induce…
emotions
Which kind of music evokes emotions?
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 3
ORCHESTRATION?trumpet, xylophone…
Rhythm?binary,
syncopated…
HARMONY?tonal, modal,
jazz…
ARTICULATION?
>, stacc, sfz…DYNAMICS?mf, ppp, ff…
Voice as communication channel
The capacity to express emotion
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 4
Rethoric of ancient Greece and Rome:
Cicero, Aristotle
Work songs Religious chant
Lullaby
Speech Singing
LISTENING RETRIEVING ASSISTANCE
The study of emotions in singing
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 5
AUTOMATICSPEECH EMOTION
RECOGNITION
SYSTEMS
FOR
SINGING VOICE
DIGITAL ORGANIZATION
LIBRARIES ONLY ACOUSTIC FEATURES
MUSICAL
• The singing voice can naturally express emotions• Music retrieval by emotional content is ever increasing• The Musical features that relate to emotion are still unknown
for automatic recognition of emotions in singing voice
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 6
Research goal…To evaluate which musical features of the singing voice relate to listeners’ perception of emotions
Considering that…
Methodology: Listening test
– Categorical– Dimensional – Models for MIR (e. g., MIREX 5-cluster model)
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 7
Theory of emotions
– Bi-dimensional (more suitable for music evaluation)– 7 level rating scale– iHEARu-PLAY crowdsourcing platform https://www.ihearu-play.eu
Perception test
Methodology: Dataset
– Random Splicing, Band Pass filtering …
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 8
Electronic manipulation in speech
6 Italian folk songs (Canzone Romana in Roman dialect)
– Random splicing – 0.5sec per segment(to disrupt Rhythmic-melodic Contour)– Reversing(to disrupt Musical Syntax)– Global tempo manipulation – 25% slower, 50% faster (to disrupt Tempo)
Italian Folk Dataset
Overview of the study
• 104 sung chunks (52 for each gender) • Lasting 11.8 sec average (sd of 2.9 sec)• Sung by 6 singers (3 for each gender)• 26 not manipulated, 26 random-spliced, 26
reversed, 26 global tempo manipulated• 24 listeners (18-30 years, sd of 3.5 years) • 13 German, 11 non German• Bi-Dimensional test (arousal/valence, 7 level)• None of them Italian speaker.
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 9
p (sig) Clear / random splicing Clear / reversing
Clear / tempo manipulation
dimension
Mean differences between clean and manipulated signals (ANOVA ) Starred results indicate p < .05 in Tukey’s post hoc test
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 10
Results
valence (f/m)
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 11
Results
Clean/random splicing
Clean/reversing
Clean/tempo manipulation
Mean differences between clean and manipulated signals for each singer. ID1, ID2, ID3 (female); ID4, ID5, ID6 (male); ID1 and ID5 (Allegro).
Starred results indicate p < .05 in Tukey’s post hoc test
Clean/reversingvalence (f/m)
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 12
ResultsIN – IndianTI – Tunesian + IranianEU – British + Spanish
Clean/tempo manipulationvalence (f/m)
Clean/random splicing
• The perception of emotions in Italian folk seems to be less influenced by manipulation techniques in German listeners.
• Confirming previous studies, culture may influence the perception of emotion in music.
• Rhythmic-melodic contour is linked to both dimensions.
• Musical syntax and allegro Tempo (fast speed) has shown to be specially related to Valence.
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 13
Conclusions
• Further perception tests - larger listener groups from a variety of musical traditions.
• Emotional models for music – connect our work with MIR and MDL studies.
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 14
Future Work
Questions?
This work was supported by the European Union’s Seventh Framework and Horizon 2020 Programmes
under grant agreement No. 33164 (ERC StG iHEARu) and No. 688835 (RIA DE-ENIGMA)
Parada-Cabaleiro et al. The Perception of Emotion in the Singing Voice 15
ihearu.eu de-enigma.eu
For further information:[email protected]
