Affective UX: Challenges in UX involving affective computing
1 Roberto Bresin - An Introduction to Affective Music – 2004.08.30 An Introduction to Affective...
-
Upload
cullen-womble -
Category
Documents
-
view
217 -
download
1
Transcript of 1 Roberto Bresin - An Introduction to Affective Music – 2004.08.30 An Introduction to Affective...
1Roberto Bresin - An Introduction to Affective Music – 2004.08.30
An Introduction to Affective Music,
Theory and Some Applications
Roberto Bresin
http://www.speech.kth.se/music/performance
2Roberto Bresin - An Introduction to Affective Music – 2004.08.30
OutlookAim
To explain how it is possible to communicate different emotions with the same music score
Part I: The science of music performanceAnalysis & synthesis of music performance• The most important techniques for measuring and modelling a
performance• The acoustical cues of importance for communicating expressivity• How the use of acoustical cues can influence the performance style• A rule-based system for the synthesis of music performance
Part II: Emotion in music performance• Emotionally expressive music performance• Real-time Visualization of Musical Expression • Examples• Applications
– Emotional colouring of music performance– expressive ringtones in mobile phones – visual display of emotion in music performance
3Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Part I
The Science of Music Performance
4Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Musical Communication
composer musician instrument listener
score gestures sound
Computational modelsComputational models
5Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The MusicianThe musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
6Roberto Bresin - An Introduction to Affective Music – 2004.08.30
What is communicated?
• The music
• Emotions
• Imagined and real motion• …
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
7Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Score - Score - Notes, harmony, melody, Notes, harmony, melody, rhythm, pitch, texture, instrumentsrhythm, pitch, texture, instruments
Performance - Performance - Tempo, phrasing, Tempo, phrasing, articulation, intonationarticulation, intonation
AuditiveAuditive
Environment - Concert, club, Environment - Concert, club, home, live/recordinghome, live/recording
AudienceAudience
Social/culturalSocial/cultural
Body movements and gesturesBody movements and gestures
People, clothes, stage lightning, etc.People, clothes, stage lightning, etc.
VisualVisual
MemoryMemory
Musical knowledgeMusical knowledge
Past experiencePast experience
Different factors in the communication
8Roberto Bresin - An Introduction to Affective Music – 2004.08.30
What can be studied?
• What is a musical performance?
• Emotional communication: Accuracy,
musical factors
• Emotional affect
• Couplings to motion: Musicians gestures and
the resulting sound
• Visual perception of musicians body
movements
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
9Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The Score and the Performance
How important is the performance?
• Dead-pan by computer and sampler• Schumann’s Träumerei by Alfred Brendel
10
20
30
40
50
60
70
80 ²DR [%] Horowitz 65
-10
-20
-30
1 2 3 4 5 6 7 8 9
10
20
30
40
50
60
70
80
-10
-20
-30
10
10
10
20
30 ²DR [%] Schnabel
-10
-20
1 2 3 4 5 6 7 8 9
30
-10
-20
10
20
30
40
50 ²DR [%] Brendel
-10
-20
-30
1 2 3 4 5 6 7 8 9
20
30
40
50
-10
-20
-30
IOI (%) BrendelIOI (%) Brendel
Tim
e d
evia
tio
n f
rom
sco
re
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
10Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The Score and the Performance
10
20
30
40
50
60
70
80 ²DR [%] Horowitz 65
-10
-20
-30
1 2 3 4 5 6 7 8 9
10
20
30
40
50
60
70
80
-10
-20
-30
10
10
10
20
30 ²DR [%] Schnabel
-10
-20
1 2 3 4 5 6 7 8 9
30
-10
-20
10
20
30
40
50 ²DR [%] Brendel
-10
-20
-30
1 2 3 4 5 6 7 8 9
20
30
40
50
-10
-20
-30
IOI (%) BrendelIOI (%) BrendelTim
e d
evia
tio
n f
rom
sco
re
IOI (%) IOI (%) SchnabelSchnabel
IOI (%) IOI (%) Horowitz 65Horowitz 65
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
11Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Collecting Data of Expressive Performances
• Expert musicians (Lars Frydén for KTH)– Expertise is translated into rules
• Measurements of recorded performances– Commercial recordings (CDs)– Computer controlled acoustical instruments
(Disklavier, Böserndorfer)
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
12Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Design of Performance Rules
Performance rules obtained mainly with 2 methods:
• analysis-by-synthesis• analysis-by-measurement
Generative grammar for automatic music performance
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
13Roberto Bresin - An Introduction to Affective Music – 2004.08.30
MUSICSCORE(MIDI)
PERFORMEDMUSIC(MIDI)
DIRECTOR MUSICES
(performance rules)
PROGRAMMERPROFESSIONAL
MUSICIAN
NEW / MODIFIEDRULE
K values (Rule quantity)
Analysis-by-Synthesis of Music Performance
14Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Dead-pan, K=0
Exaggerated, K = 4.4
Moderate, K = 2.2
Inverted, K = -2.2
Duration contrast ruleIn
ter o
nset
I nte
rval d
evia
t ion
s (
%)
15Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Analysis-by-Measurement Designing Articulation Rules
3 main classes of articulation:
• Legato (overlapping)
• Staccato (detaching)
• Repetition
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
16Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Legato and Staccato Tones
IOI = Inter-onset-Interval
DR = Tone Duration
KOT = Key Overlap Time
KDT = Key Detached Time
17Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Data
• Natural• Glittering• Dark• Heavy• Light• Hard• Soft• Passionate• Flat
5 pianists played the
same score
9 times on a
DisklavierThe first sixteen bars of the
Andante movement of W A
Mozart’s Piano Sonata in G
major, K 545
1 pianist played 13 Mozart piano sonatas
on a computer-monitored Bösendorfer
18Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Mean Legato (KOR)
0
5
10
15
20
25
Fla
t
Da
rk
So
ft
Lig
ht
Na
tura
l
Glit
teri
ng
Pa
ssio
na
te
Ha
rd
He
avy
Adjective
Me
an
KO
R (
%)
0
5
10
15
20
25
P1 P2 P3 P4 P5Pianist
Me
an
KO
R (
%)
Legato articulation rule
19Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Mean Staccato(Key Detached Ratio, KDR)
0
25
50
75
100
Heavy
Passio
nate
Dark
Soft
Fla
t
Hard
Natu
ral
Glit
tering
Lig
ht
Adjective
Mean
KD
R (
%)
0
25
50
75
100
Na
tura
l
Glit
teri
ng
Da
rk
He
avy
Lig
ht
Ha
rd
So
ft
Pa
ssio
na
te
Fla
t
Adjective
Mea
n K
DR
(%
)
P1 P2 P3 P4 P5
Staccato articulation rule
19
20Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Context Influence in Staccato Production
Amount of staccato (KDR) in different contexts for the 2nd note in a three notes pattern.
(S = staccato note, N = Non–staccato note)
NSN
0
10
20
30
40
50
60
-100 -75 -50 -25 0 25 50 75 100
KDR (%)
Fre
qu
en
cy (
N)
Previous KDR
Current KDR
Next KDR
NSS
0
10
20
30
40
50
60
-100 -75 -50 -25 0 25 50 75 100
KDR (%)
Fre
qu
en
cy (
N) Previous KDR
Current KDR
Next KDR
SSS
0
10
20
30
40
50
60
-100 -75 -50 -25 0 25 50 75 100
KDR (%)
Fre
qu
en
cy (
N)
Previous KDR
Current KDR
Next KDR
SSN
0
10
20
30
40
50
60
-100 -75 -50 -25 0 25 50 75 100
KDR (%)
Fre
qu
en
cy (
N)
Previous KDR
Current KDR
Next KDR
64
66
68
70
72
74
76
SSS NSS SSN NSN
Staccato context
KD
R (
%)
21Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Legato and walking
Staccato and running
Step and key overlap time
-50
0
50
100
150
200
250
300
350
400
0 500 1000 1500 2000
Tc/2 (ms), IOI
Td
su (
ms)
, KO
T
Normal step frequency
High step frequency
Low step frequency
Bresin & Battel
MacKenzie and Van Eerd
Repp
Step and key detached time
050
100150200250300350400450
0 200 400 600
Tc/2 (ms), IOI
Tai
r (m
s), K
DT
Running
Natural
Heavy
Adagio
Allegro
Presto
Default mezzostaccato
Control model for
step sounds
LegatoLegato and and SStaccatotaccato AAllude to llude to WWalking and alking and RRunning?unning?
22Roberto Bresin - An Introduction to Affective Music – 2004.08.30
WalkingWalking RunningRunning
Footsteps
23Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Controlling Footsteps
Pd model for crumpling sounds controlled with performance rules
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
24Roberto Bresin - An Introduction to Affective Music – 2004.08.30
KTH Performance Rules
• Descriptions of different performance principles used by
musicians
• General applicability
• K values change the overall quantity of each rule
• Context dependency
• ~ 30 rules
• 30 years of research at KTH
ScoreScore RulesRules PerformancePerformance
K valuesK values
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
25Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Director MusicesA program for modelling music performancehttp://www.speech.kth.se/music/performance
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
26Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Performance Rules
Phrasing Phrase archFinal
ritardandoPunctuationHigh loud
Harmonic/melodic tension Harmonic/melodic charge
Repetitive patterns and grooves Swing
Articulation PunctuationStaccato/legato
Accents Accent rule
Ensemble timing Ensemble swingMelodic sync
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
27Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Phrase Arch Rule
Dead-pan
Exaggerated
ΔIO
I (%
)
28Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Moderate
Inverted
Phrase Arch Rule Δ
IOI
(%)
29Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The DM system (~30 rules)
Differentiation RulesExample: Duration Contrast Rule
Grouping RulesExample: Phrase Articulation Rule
Synchronization/Ensemble RulesExample: Ensemble timing
Other RulesExample: Repetition Articulation Rule
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
30Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Part II
Emotion in Music Performance
31Roberto Bresin - An Introduction to Affective Music – 2004.08.30
HappyHappy oror sadsad musicmusic??
32Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Mapping from Emotional Expression to Rule Parameters
For each emotion:
Select a palette of rule parameters according to previous findings
Mapping
Emotional expression
Rule parameters
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
33Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Cues for the simulation of
emotions in music
performance
(by A.Gabrielsson and P.Juslin,
Psychology of Music, 1996, vol. 24)
• No expression
• Tenderness
• Solemnity
• Happiness
• Sadness
• Anger
• Fear
Synthesis of Emotional ExpressionThe musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
34Roberto Bresin - An Introduction to Affective Music – 2004.08.30
• TENDERNESSslow mean tempo (Ga96)slow tone attacks (Ga96)low sound level (Ga96)small sound level variability (Ga96)legato articulation (Ga96)soft timbre (Ga96)large timing variations (Ga96)accents on stable notes (Li99)soft duration contrasts (Ga96)final ritardando (Ga96)
• HAPPINESSfast mean tempo (Ga95)small tempo variability (Ju99)staccato articulation (Ju99)large articulation variability (Ju99)high sound level (Ju00)little sound level variability (Ju99)bright timbre (Ga96)fast tone attacks (Ko76)small timing variations (Ju/La00)sharp duration contrasts (Ga96)rising micro-intonation (Ra96)
• ANGERhigh sound level (Ju00)sharp timbre (Ju00)spectral noise (Ga96)fast mean tempo (Ju97a)small tempo variability (Ju99)staccato articulation (Ju99)abrupt tone attacks (Ko76)sharp duration contrasts (Ga96)accents on unstable notes (Li99)large vibrato extent (Oh96b)no ritardando (Ga96)
• SADNESSslow mean tempo (Ga95)legato articulation (Ju97a)small articulation variability (Ju99)low sound level (Ju00)dull timbre (Ju00)large timing variations (Ga96)soft duration contrasts (Ga96)slow tone attacks (Ko76)flat micro-intonation (Ba97)slow vibrato (Ko00)final ritardando (Ga96)
• FEARstaccato articulation (Ju97a)very low sound level (Ju00)large sound level variability (Ju99)fast mean tempo (Ju99)large tempo variability (Ju99)large timing variations (Ga96)soft spectrum (Ju00)sharp micro-intonation (Oh96b)fast, shallow, irregular vibrato (Ko00)
Positive Valence
Negative Valence
High ActivityLow Activity
From Juslin (2001)
35Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Tempo
Loudness
Timbre
Encoding Decoding
Cue Utilization Cue Utilization
The Listener
Articula.
The Performance The Performer intention expressive cues judgment
Accuracy
others
Anger Anger
rPerformer rListener
Matching
.87
.26 .47 .63
-.26
.22 .55 .61
-.39
.92
Lens model: quantifies the expressive communication between performer and listener
36Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Example: SADNESS
Expressive Cue Analysis Synthesis (Director Musices)
Tempo Slow Tone IOI is lengthened by 30%
Sound level Moderate or low Sound level is decreased by 6 dB
Articulation Legato Tone duration = Tone IOI
Time deviations Moderate Duration Contrast Rule (k = -2)
Phrase Arch Rule applied on phrase level (k = 1.5)
Phrase Arch Rule applied on sub-phrase level (k = 1.5)
Final ritardando Yes Obtained from the Phrase Arch Rule
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
37Roberto Bresin - An Introduction to Affective Music – 2004.08.30
IOI deviations
dB deviations
articulation
Example: SADNESSModel Score
38Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Synthesis of Emotion: Listening Test Results
Percentage of “right” classification (%)
0102030405060708090
100
FE
AR
AN
GE
R
HA
PP
INE
SS
SA
DN
ES
S
SO
LE
MN
ITY
TE
ND
ER
NE
SS
NO
EX
PR
ES
SIO
N
Ekorrn
Mazurka
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
39Roberto Bresin - An Introduction to Affective Music – 2004.08.30
ConclusionsConclusions
Emotional expressioncan be derived directlyfrom the music score,
simplyby enhancing music structure
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
40Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Better Monophonic Ringtones!
Today (nom.)
Natural
:-) Happy
>:-( Angry
:-( Sad
=|:-| Solemn
Mozart G minor
0 2 4 6 8 10 12 -1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
0 2 4 6 8 10 12 -1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8 0 2 4 6 8 10 12 -1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
0 2 4 6 8 10 12 -1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
0 2 4 6 8 10 12 -1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
0 2 4 6 8 10 12 -1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Dead-pan
Happy
Angry
Natural
Sad
Solemn
www.notesenses.com
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
41Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Usher, Burn
MechanicalMusicalHappy
Jennifer Ellison, Bye Bye Boy
MechanicalMusicalRomantic
Better Polyphonic Ringtones!
www.notesenses.com
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
42Roberto Bresin - An Introduction to Affective Music – 2004.08.30
pDM –performance rules in real-timeAnders Friberg + MEGA project (IST EU)
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
43Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Real-time Visualization of
Musical Expression
44Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Background
Feel-Me project
Design a computer program for teaching students to play expressively
The system includes a tool for automatic extraction of acoustic cues (CUEX):
pitch, duration, sound level, articulation, vibrato, attack velocity, spectrum
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
45Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Aim
Design a tool for real-time visual feedback to expressive performance
Mapping of acoustic cues:
– Non-verbal– Intuitive– Informative (including emotional expression)
Previous studies: cross-modality speeds stimuli discrimination
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The Listener Recognition of emotionVisualisation of musical expression
46Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Cue analysis
Expression mapper
Audio Emotion
TempoSound level
Articulation...
Implementations
CUEX
Simplified real-timeversion
Mult. Regression
Fuzzy inspired
Recognition of EmotionThe musicianMusical communicationModelling music performanceEmotional colouringApplications
The ListenerRecognition of emotionVisualisation of musical expression
47Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Experiment
2 melodies, Brahms (minor) & Haydn (Major)3 instruments (piano, guitar, saxophone)12 performances per instrument (12 emotional intentions)
24 colour nuances8 levels of hue 2 levels of brightness 2 levels of saturation
2 groups of 11 subjects each(1 group per melody)
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The ListenerRecognition of emotionVisualisation of musical expression
48Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Experiment: main resultsHUE
Happiness YellowFear BlueSadness Violet & BlueAnger RedLove Blue & Violet
BRIGHTNESS
Observed tendency:Minor tonality Low brightness (Dark colours)Major tonality High brightness (Light colours)
Interaction involving sadness:Even for major tonality low brightness is preferred for all instruments
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The ListenerRecognition of emotionVisualisation of musical expression
49Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Experiment: main resultsSimilar colour palettes within instruments
DISGUST (SAX)
0
0,5
1
1,5
2
2,5
3
3,5
4
red
red
_d
ark
red
_lig
ht
ora
ng
e
ora
ng
e_
da
rk
ora
ng
e_
ligh
t
yello
w
yello
w_
da
rk
yello
w_
ligh
t
gre
en
gre
en
_d
ark
gre
en
_lig
ht
cya
n
cya
n_
da
rk
cya
n_
ligh
t
blu
e
blu
e_
da
rk
blu
e_
ligh
t
vio
let
vio
let_
da
rk
vio
let_
ligh
t
ma
ge
nta
ma
ge
nta
_d
ark
ma
ge
nta
_lig
ht
Haydn
Brahms
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The ListenerRecognition of emotionVisualisation of musical expression
50Roberto Bresin - An Introduction to Affective Music – 2004.08.30
Experiment: main resultsSHAME (SAX)
0
1
2
3
4
5re
d
red
_d
ark
red
_lig
ht
ora
ng
e
ora
ng
e_
da
rk
ora
ng
e_
ligh
t
yello
w
yello
w_
da
rk
yello
w_
ligh
t
gre
en
gre
en
_d
ark
gre
en
_lig
ht
cya
n
cya
n_
da
rk
cya
n_
ligh
t
blu
e
blu
e_
da
rk
blu
e_
ligh
t
vio
let
vio
let_
da
rk
vio
let_
ligh
t
ma
ge
nta
ma
ge
nta
_d
ark
ma
ge
nta
_lig
ht
Haydn
Brahms
SHAME (GUITAR)
0
1
2
3
4
5
red
red
_d
ark
red
_lig
ht
ora
ng
e
ora
ng
e_
da
rk
ora
ng
e_
ligh
t
yello
w
yello
w_
da
rk
yello
w_
ligh
t
gre
en
gre
en
_d
ark
gre
en
_lig
ht
cya
n
cya
n_
da
rk
cya
n_
ligh
t
blu
e
blu
e_
da
rk
blu
e_
ligh
t
vio
let
vio
let_
da
rk
vio
let_
ligh
t
ma
ge
nta
ma
ge
nta
_d
ark
ma
ge
nta
_lig
ht
Haydn
Brahms
SHAME (PIANO)
0
1
2
3
4
5
red
red
_d
ark
red
_lig
ht
ora
ng
e
ora
ng
e_
da
rk
ora
ng
e_
ligh
t
yello
w
yello
w_
da
rk
yello
w_
ligh
t
gre
en
gre
en
_d
ark
gre
en
_lig
ht
cya
n
cya
n_
da
rk
cya
n_
ligh
t
blu
e
blu
e_
da
rk
blu
e_
ligh
t
vio
let
vio
let_
da
rk
vio
let_
ligh
t
ma
ge
nta
ma
ge
nta
_d
ark
ma
ge
nta
_lig
ht
Haydn
Brahms
51Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The ExpressiBall Expressive performance space as a mapping of acoustical cues and emotions
X Tempo Color EmotionY Sound level Shape Articulation Z Attack velocity & Spectrum energy
Lo
ud
So
ft
Slow Fast
StaccatoAngryFast attackHigh energy
Lo
ud
So
ftSlow Fast
LegatoSadSlow attackLow energy
52Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The ExpressiBall
DEMO!
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The ListenerRecognition of emotionVisualisation of musical expression
53Roberto Bresin - An Introduction to Affective Music – 2004.08.30
ExpressiBall:Current & Future Work…
Sonification of the ExpressiBall
Set-up depending on instrument/spectrumUse complex colour palettes (pictures?)
Usability test with students
Other possible applications: ”Colour Monitor” in discotheques Computer screen saver …
The musicianMusical communicationModelling music performanceEmotional colouringApplications
The ListenerRecognition of emotionVisualisation of musical expression
54Roberto Bresin - An Introduction to Affective Music – 2004.08.30
OutlookAim
To explain how it is possible to communicate different emotions with the same music score
Part I: The science of music performanceAnalysis & synthesis of music performance• The most important techniques for measuring and modelling a
performance• The acoustical cues of importance for communicating expressivity• How the use of acoustical cues can influence the performance style• A rule-based system for the synthesis of music performance
Part II: Emotion in music performance• Emotionally expressive music performance• Real-time Visualization of Musical Expression • Examples• Applications
– Emotional colouring of music performance– expressive ringtones in mobile phones – visual display of emotion in music performance
55Roberto Bresin - An Introduction to Affective Music – 2004.08.30
The End