Tension in Music : Cognition, Emotion, Brain, Movement Carol L. Krumhansl Department of Psychology...
-
Upload
jayson-radley -
Category
Documents
-
view
216 -
download
0
Transcript of Tension in Music : Cognition, Emotion, Brain, Movement Carol L. Krumhansl Department of Psychology...
Tension in Music :Cognition, Emotion, Brain, Movement
Carol L. KrumhanslDepartment of Psychology
Cornell UniversityIthaca, NY 14853
USA
Tension: Linking Cognition, Emotion, Brain, and Motion
WIPPPPPWIPPPPP
Work in Past Published in Press Present PlannedWork in Past Published in Press Present Planned
"…expectation is always ahead of the music, creating a background of diffuse tension against which particular delaysarticulate the affective curve and create meaning.”
“Not only does music use no linguistic signs but, on one level at least, it operates as a closed system, that is, it employs no signs or symbols referring to the non-musical world of objects, concepts, and human desires. …This puzzling combination of abstractness with concrete emotional and aesthetic experience can, if understood correctly, perhaps yield useful insights into more general problems of meaning and communication.”
L. B. Meyer, Emotion and Meaning in Music, 1956
Duration of each beat in the music as performed
Mozart, Piano Sonata, K. 282Krumhansl, Music Perception, 1996
Tension and cognition
Judgments of sections ends
Duration of each beat in the music as performed
Duration of each beat in the music as performed
Judgments of new musical ideas
Krumhansl, Music Perception, 1996
(a) octave (root) level: 0 (0)(b) fifth level: 0 7 (0)(c) triadic level: 0 4 7 (0)(d) diatonic level: 0 2 4 5 7 9 11 (0)(e) chromatic level: 0 1 2 3 4 5 6 7 8 9 10 11 (0)
Lerdahl, Tonal Pitch Space, 2001
Diatonic basic space, set to I/C (C = 0, C# = 1, …B = 11).
Computing Distance from d minor (vi) chord in F major keyto C major (I) chord in C major key
Region DistanceF major to C major
Chord Distanced minor to C major
Basic Space DifferencesNew entries in Basic Space
Tensing Relaxing
Slide 40Slide 40
Krumhansl, Music Perception, 1996
Chopin Prelude
0
25
50
75
100
125
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47
Event
Tension
Judged Predicted
Messiaen Quartet
0
25
50
75
100
125
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
Event
Tension
Judged Predicted
Messiaen, Quartet for the end of time
Bach Chorale
0
25
50
75
100
125
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41
Event
Tension
Judged Predicted
Bach ChoraleCristus, der ist mein Liebe
WagnerParsifalGrailTheme
Diatonic
Chromatic
Lerdahl & Krumhansl, Music Perception.in press
Wagner Diatonic Hierarchical Right-Branching +
0
25
50
75
100
125
1 2 3 4 5 6 7 8 9
Event
Tension
Judged Predicted
Wagner ChromaticShifting Diatonic
Sequential to Hierarchical
0
25
50
75
100
125
1 2 3 4 5 6 7 8 9
Event
Tension
Judged Predicted
Fourier Balance OneFB1
FB 1
Stable for minor second, major seventhUnstable for fourth, fifth, and tritone
Stable for a diatonic scale
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Tipping the (Fourier) balances:A geometric approach to representing pitch structure in non-tonal music
FB 2
Stable for tritone, minor second, major seventh, fourth and fifthUnstable for minor third, major sixth
Stable for an octatonic scale
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Fourier Balance TwoFB2
FB 1 FB 2 FB 3
FB 4 FB 5 FB 6
-25
0
25
50
75
100
Y
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0
Event
Predicting Judged Tension with FB model
Fourier Balance ModelR-squared (2,5) = .999
Predicting Judged Tension with TPS ModelR-squared (2,5) = .942
Compare Fourier Balance Model to Tonal Pitch Space Model
Y Judged Tension Predicted from FB model Predicted from TPS model
Predicting Judged Tension with FB ModelR-squared (18,21) = .823
Predicting Judged Tension with TPS ModelR-squared (2,37) = .758
0
10
20
30
40
50
60
70
80
90
100
Y
0 10 20 30 40
Rows
Y Judged Tension Predicted from FB model Predicted from TPS model
MessiaenQuartet for the End of Time
Summary:Fourier Balance model fits judged tension better thanTonal Pitch Space model
but:The two models are closely connected
International Congress on Music Perception and CognitionBologna, August. 2006
A Proto-Music Theoryfrom Unsupervised Learning
Carol L. KrumhanslDavid G. Rand
Background
Experiments demonstrate listeners are sensitive to statistically frequent patterns
“Statistical learning”
adults -- cross-cultural (Krumhansl, et al., 1999, 2000)
adults -- melodies on artificial tone-sets (Oram & Cuddy, 1995)
infants -- tone and syllable sequences (Saffran, et al. 1999)
Limitations
Low-order statistics for sequences
P(c1), P(c2 | c1), P(c3 | c1 c2)
Results show effects of acculturation only in higher-order statistics (Krumhansl et al., 1999, 2000)
Automatic discrimination of musical styles only with higher-order statistics (Krumhansl, 2000)
Coding of music ignores durations of tones
c1 c2 c3 … ck
Questions
Can a statistical learning model distill musically interpretable patterns from a musical corpus?
What does such a model show about the relevance of rhythm to melodic structure?
ADIOS (Automatic DIstillation Of Structure)Zach Solan, David Horn, Eytan Ruppin (Tel-Aviv University),Shimon Edelman (Cornell University)
Two classes of syntax modelsLanguage-specific theory of syntax (generative theory)General-purpose statistical or distributional learning models
ADIOS has features of bothDistillation of rule-like regularities out of the acquired knowledgeKnowledge acquired only from “raw” distributional information
Representational Data Structure (RDS)
Directed Graph Input: raw unlabeled corpus data (not tagged for part of speech, only BEGIN and END of each sentence)
Node = one constituent (word)
Directed edge is inserted if transition between constituents exists in corpus
Pattern Acquisition (PA) Algorithm
A Pattern is a similarly structured sequence of constituents that recurs in the corpus.
An Equivalence Class is a set of constituents from different paths that occur in the same position in a pattern
This is the syntax that ADIOS is distilling
Bundles are formed when two or more paths run in parallel
and dissolved when more paths leave the bundle than stay in
Criterion for judging pattern significance
For path c1 c2 c3 … ck:
S = e-(L/k)2 P(c1,c2,…,ck) log( P(k)(path) / P(2)(path)), where:
L = typical context length k = length of the candidate path
P(k) (path) = P(c1)P(c2|c1)P(c3|c1 c2)…P(ck|c1 c2 c3 … ck)“k-gram”
P(2) (path) = P(c1) P(c2|c1) P(c3|c2)… P(ck|ck-1)“random walk”
Bootstrapping
The identification of new equivalence classes is done using acquired equivalence classes - “bootstrapping”
Musical Corpus
Musical Themes from Classical Corpus in Themefinder300 themes in C major, 300 themes in A minor Monotone (no harmony)
C major scale tones: C D E F G A B
A minor natural scale tones: A B C D E F G
Kern format:• 2aa 2ee 4dd 8cc 8b 4a 4a 2ee 2b 4a 8g 8f 4e 4e #
2=half note, 4=quarter note, 8=eighth note, etcG = G below middle C, g = G above middle C, gg= G an octave and a fifth above middle C, etc.* = begin# = end
Each duration-pitch pair is treated as an independent constituent
There is NO information given to ADIOS to indicate that:
16a and 8a are the same pitch (with different durations)16a and 16b are the same duration (with different pitches)16G and 16g are the same pitch in different octaves (octave equivalent)
Comparison with Musicians Judgments43 Major, Minor, Major Shuffled, Minor Shuffled, Major Scrambled, Minor Scrambled11.1 years instruction on musical instruments, 3 or more music theory courses
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Overall Summary
ADIOS distills interpretable structure from musical corpusMusically interpretable Patterns and Equivalence ClassesTonal hierarchy is evident in Patterns and Equivalence ClassesWhen trained on Major, can correctly discriminate Major vs MinorWhen trained on Major and Minor, can discriminate Original vs Shuffled (duration-pitch pairs moved)
ADIOS develops “Proto-Music Theory” Not fully developed, but sufficient to determine:Neighbors (pitch proximity)Same pitch independent of durationSame duration independent of pitchOctave equivalent pitchesHarmonically related pitchesMajor vs. MinorOrder of duration-pitch pairs
Tension and Emotion
Krumhansl, An exploratory study of musical emotions and psychophysiologyCanadian Journal of Psychology, 1997
Experimental Design
Dynamic RatingsSadFearHappyTension
Sad Excerpts:Tomaso Albinoni, Adagio in G minor for Strings and OrchestraSamuel Barber, Adagio for Strings, Op. 11
Fear Excerpts:Gustav Holst: Mars -- the Bringer of War from The PlanetsModest Mussorgsky, Night on Bare Mountain
Happy Excerpts:Antonio Vivaldi, The Four Seasons, La Primavera (Spring), Danza pastoraleHugo Alfven, Midsommarvaka
Physiological Measures Recorded at 1-second Intervals
1) cardiac interbeat interval (IBI), measured in milliseconds, with shorter IBIs taken to indicate a higher level of cardiovascular arousal2) pulse transmission time to the finger (FPTT), measured in milliseconds, with shorter pulse transmission times indicative of greater autonomic (sympathetic) activation3) finger pulse amplitude (FPA), a measure of the amount of blood in the periphery, with reduced amplitude indicating greater vasoconstriction and associated with greater autonomic (sympathetic) activation4) pulse transmission time to the ear (EPTT), another measure of blood flow5) respiration intercycle interval (ICI), measuring the time between successive inspirations in milliseconds6) respiration depth (RD), which is the point of maximum inspiration minus the point of maximum expiration7) respiration-sinus asynchrony (RSA)8) systolic blood pressure (SBP)9) diastolic blood pressure (DBP)10) mean arterial pressure (MAP)11) skin conductance level (SCL), with increased skin conductance indicative of greater autonomic (sympathetic) activation12) temperature on the finger (TEM) measured in degrees Fahrenheit.
Correlations between Dynamic Emotion Ratings and Dynamic Physiology Ratings
Sad Ratings Fear Ratings Happy RatingsInterbeat-Interval
.14*** -.01 -.15***Finger Pulse Transmission Time
.10** .16*** -.09**Finger Pulse Amplitude
-.14*** -.31*** .24***Ear Pulse Transmission Time
.07* .15*** .03Respiration Intercycle-Interval
.05 -.12*** -.16***Respiration Depth
.00 .00 -.09*Respiration Sinus Asynchrony
-.02 -.11*** -.13***Systolic Blood Pressure
.37*** -.23*** -.14***Diastolic Blood Pressure
.41*** -.24*** -.14***Mean Arterial Pressure
.37*** -.25*** -.10**Skin Conductance Level
-.36*** .06 -.08*Finger Temperature
-.35*** -.20*** .21***
Factor analysis of correlations between physiological measures
Distinct groupings
Blood SCL Respiration Blood Flow Heart Blood FlowPressure Temp Rate Finger Rate Ear
Functional Magnetic Resonance Imaging (fMRI)
The neural mechanisms of tonal and rhythmic expectations were studied in two ways:
Stimulus- introducing either tonal or rhythmic violations (or both)
Task-judge either the tonal structure or the rhythmic structure -(or passively listen to the melodies)
Tension and Brain
Musical Sequences: Melodies composed by Diego Vega, Cornell University (6 sec), piano timbre. Sample sequences:
Original
Tonal Violations
Rhythmic Violations
Tonal and RhythmicViolations
Passive Listening: Tonal and Rhythmic ViolationsThis analysis contrastedmusical sequences containing both Tonal and Rhythmic violations with musical sequences with no violations.
Even when the subjects were in not performing a task, activations in:
superior temporal cortex (especially on the right)
Active Judgments: Tonal and Rhythmic Violations
This analysis also contrastedMusical Sequences containing both Tonal and Rhythmic violations with musical sequences with no violations (as before)
Judged whether a violation occurred
Superior temporal activation (as for Passive Listening), in addition, when making either Tonality or Rhythm Judgments:
right dorsolateral frontal right
inferior frontal (bilateral)
"The cognitive representation of an event unit involving human motion can bedescribed as some small set of relatively stable, preparatory motions, followed bythis relatively unstable, completing motion. Such a schematic structure enablesthe perceiver to anticipate temporal relations within an unfolding event, and to fitsuccessive parts of the temporal sequence into this anticipated structure.”
M. Lasher, Cognitive Psychology, 1981.
"The interaction between movement and sound is the most fundamental element ofdance. The dance does not mimic the music -- there is not a particular part of themusic for every gesture and step -- but the basic "kinetic feel" or "energy shape"of the music is expressed in the dance. The choreographer uses the music not onlyfor its rhythmic pulse, but also as a source of emotional and structural ideas. Thus,elements of the music are often observed in the dance.”
K. Teck, Movement to Music, 1990.
Tension and motion
Tension and motion in dance
Experimental Design
Music W. A. Mozart, Divertimento No. 15 Bb, Minuetto
Dance George Balanchine, School of AmericanBallet
Subjects Music Lessons 9.2 yearsDance Lessons 7.3 years
Conditions Music OnlyDance OnlyBoth Music and Dance
Tasks Section End (discrete judgment)during Tension (continuous judgment )videotape New Idea (discrete judgment)
Emotion Expressed (continuous judgment)
Task after Emotion Quality (overall judgment)videotape
Intrinsic Relationships Between Music and Dance
Rhythmic accent, meter, sounds produced by the dancers
Dynamic volume of musical and choreographic
Textural number of instruments/performers, homophony versus polyphony, counterpoint
Structuralcorresponding motives or figures, phrases, structures
Qualitative choreomusical parallels of tessitura, timbre, articulation, dissonance/consonance
Mimetic choreography imitates a particular sound in the music
P. Hodges (1992) Relationships between score and choreography in 20th century dance. London: Mellen.
"To see Balanchine's choreography … is to hear the music with ones eyes…The choreography emphasizes relationships of which I had hardly been aware…and the performance was like a tour of a building for which I had drawn the plansbut never explored the result.”
I. Stravinsky, quoted in S. Jordan, Dance Chronicle, 1993
Exploratory Study of Anticipating Human Movement in Dance
Barycenter
Center of sub-regionincluding head, trunk,legs
Stop Position
Target PositionJudged Position
Camurri, Krumhansl, Mazzarino, Volpe, 2004
Marcello Wanderley, IRCAM thesis, 2002Non-obvious performance gestures
(Not needed to produce tone)
Normal Expression, Exaggerated Expression,Immobile
Tension and motion in music performance
Experimental Design
Continuous Judgments of TensionContinuous Judgments of Phrasing
while:watching performance (no sound)hearing performance (no image)both watching and hearing performance
Stravinsky, Second of Five Pieces for Solo Clarinet
Tension judgments in experiment
Phrasing judgments in experiment
Analysis of data with
Functional Data AnalysisJ. Ramsay
Audio, Visual, Audio and Visual conditionssimilar for judgments of phrasing
But complex interactions between Visual and Audioin judgments of tension
Vines, Wanderley, Levitin, Krumhansl, Cognition, 2006
Tension in facial and body gesture
Cognition Emotion Motion
Brain
Behavior
Music
Tension
L. B. MeyerEmotion and Meaning in Music
1956
"…expectation is always ahead of the music, creating a background of diffuse tension against which particular delaysarticulate the affective curve and create meaning.”
“Not only does music use no linguistic signs but, on one level at least, it operates as a closed system, that is, it employs no signs or symbols referring to the non-musical world of objects, concepts, and human desires. …This puzzling combination of abstractness with concrete emotional and aesthetic experience can, if understood correctly, perhaps yield useful insights into more general problems of meaning and communication.”
L. B. Meyer, Emotion and Meaning in Music, 1956
Cognition Emotion Motion
Brain