A Musical System For Emotional Expression
-
Upload
antonio-oliveira -
Category
Engineering
-
view
100 -
download
0
Transcript of A Musical System For Emotional Expression
![Page 1: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/1.jpg)
A Musical System For Emotional Expression
António Pedro OliveiraUniversity of Coimbra, Portugal
![Page 2: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/2.jpg)
2
Outline of the Presentation
Introduction Background Emotion-Driven Music Engine
(EDME) Conclusion
![Page 3: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/3.jpg)
3
Outline of the Presentation
Introduction Background Emotion-Driven Music Engine
(EDME) Conclusion
![Page 4: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/4.jpg)
Motivation Emotions are widely accepted as being
an important factor in the society Music is almost everywhere and it is a
powerful stimulus capable of influencing our emotions
Computational systems with the capability of producing music with an appropriate emotional content have an enormous application potential
4
Introduction Background EDME Conclusion
![Page 5: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/5.jpg)
Aim
Conceive a computational system for the control of the emotional content of produced music, so that it expresses a given emotional specification
Music: solely instrumental
5
Introduction Background EDME Conclusion
![Page 6: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/6.jpg)
6
Outline of the Presentation
Introduction Background Emotion-Driven Music Engine
(EDME) Conclusion
![Page 7: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/7.jpg)
Emotional Expression with Music
Four approaches: Music Transformation Music Composition Music Selection/Classification Hybrid Approaches
Our approach: Hybrid that consists in combining
selection/classification with transformation
7
Introduction Background EDME Conclusion
![Page 8: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/8.jpg)
8
Outline of the Presentation
Introduction Background Emotion-Driven Music Engine
(EDME) Conclusion
![Page 9: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/9.jpg)
Architecture – Offline Stage
9
Introduction Background EDME Conclusion
![Page 10: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/10.jpg)
Architecture – Online Stage
10
Introduction Background EDME Conclusion
![Page 11: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/11.jpg)
Experiments Initial phase
11
Introduction Background EDME Conclusion
![Page 12: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/12.jpg)
Initial phase Manually Built Knowledge Base (short
version) Happy music: high loudness, major scale Sad music: violin, slow tempo Activating music: high loudness, fast
tempo Relaxing music: low loudness, slow tempo
12
Introduction Background EDME Conclusion
![Page 13: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/13.jpg)
Experiments Stages of the experiments
13
Introduction Background EDME Conclusion
![Page 14: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/14.jpg)
Experiments First Experiment - Preliminary
Evaluation of the Classification Module Hypothesis: There is a small amount of
features that may predict arousal/valence Valence: 2 features, CC – 0.76 Arousal: 4 features, CC – 0.77
14
Introduction Background EDME Conclusion
![Page 15: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/15.jpg)
Experiments
Second experiment (two parts) First part - Extended Evaluation of the
Classification Module Hypothesis: There is a small amount of
features that may predict arousal/valence Valence: 4 features, CC – 0.70 Arousal: 3 features, CC – 0.77
15
Introduction Background EDME Conclusion
![Page 16: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/16.jpg)
Experiments Second part - Analysis of Audio
Features Hypothesis: There are audio features
emotionally-relevant Valence: Spectral Sharpness and Loudness
are important Arousal: Spectral Similarity, Spectral
Dissonance and Spectral Sharpness are important
16
Introduction Background EDME Conclusion
![Page 17: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/17.jpg)
Experiments
Third experiment (three parts) First part - Improvement of the
Classification Module Hypothesis: There is a small amount of
features that may predict arousal/valence Valence: 5 features, CC – 0.69 Arousal: 3 features, CC – 0.71
17
Introduction Background EDME Conclusion
![Page 18: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/18.jpg)
Experiments Second part – Evaluation of the Transformation
Algorithms Hypothesis: It is possible to change musical features to
transform emotional content High positive coefficients for tempo were confirmed The increase of register correlates positively with valence
and negatively with arousal Some of the features can be helpful in finding scales more
appropriate to some emotions Instruments are essentially relevant to the arousal Change from normal to staccato articulation has a
correlation with the increase of valence
18
Introduction Background EDME Conclusion
![Page 19: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/19.jpg)
Experiments Third part – Melodic Analysis
Hypothesis: The analysis of the melodic line alone turns the emotionally-relevant features more visible
19
Introduction Background EDME Conclusion
Emotional Dimension CC Features
Valence – data of first experiment 0.79 5 features
Valence – data of second experiment 0.62 5 features
Valence – data of third experiment 0.41 4 features
Arousal – data of first experiment 0.85 4 features
Arousal – data of second experiment 0.72 4 features
Arousal – data of third experiment 0.54 3 features
![Page 20: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/20.jpg)
Calibration and Validation
Hypothesis: 13 features are enough to discriminate emotional expression
Valence: 7 features, CC – 0.85 Arousal: 6 features, CC – 0.83
Use SAM to obtain emotional answers Controlled environment Statistical analysis:
System’s classification and subject’s classification are probably measuring the same concept
20
Introduction Background EDME Conclusion
![Page 21: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/21.jpg)
21
Outline of the Presentation
Introduction Background Emotion-Driven Music Engine
(EDME) Conclusion
![Page 22: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/22.jpg)
Contributions The system proposed has the
advantage of being able to produce outputs of acceptable quality quite independently from the music base
It is also quite flexible: the music base can be completely redefined to adapt to the specific needs of a given scenario
22
Introduction Background EDME Conclusion
![Page 23: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/23.jpg)
Contributions The system is also reliable, thanks to
the experimental calibration using different subjects
We adopted both the discrete and dimensional representation of emotions
We used techniques of human emotional recognition for validation and calibration of the system
23
Introduction Background EDME Conclusion
![Page 24: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/24.jpg)
Future Work
Emotion-Driven Music Composition Artificial Intelligence approaches
Test in applications contexts Healthcare Entertainment
24
Introduction Background EDME Conclusion
![Page 25: A Musical System For Emotional Expression](https://reader036.fdocuments.net/reader036/viewer/2022081605/58ed5b201a28ab60658b4635/html5/thumbnails/25.jpg)
The End
25