Corpus Analysis of Rock Music

Trevor de Clercq

Assistant ProfessorIthaca College

Department of Music Theory, History, and Composition

Math, Music, and the Brain (Biology 22020)Dec. 11, 2012

What is corpus analysis / corpus research / corpus study?

A corpus is a body of data

• the text of some collection of books (linguistics)

• the syllables in a collection of newspaper articles

• the chords in a collection of musical compositions

• the notes in the melodies of a collection of songs

Corpus study asks questions about this data, e.g.:

• Do composers tend to use dissonant chords in middle sections?

• Does music slow down at moments of rapid harmonic motion?

• Do melodies more often fall or rise at the end of phrases?

How can we do corpus-based music research?

Encode music to allow searching

• humdrum (David Huron, Ohio State)

• music21 & python (Michael Cuthbert, MIT)

• custom-based text encoding

Search for patterns in encoded music

• computational analysis

– offers some level of objectivity in analysis

• statistics and probability

Math and music, but what about the brain?

Corpus research can explain aspects of music cognition

• Our perception and conception of different musical styles is (at least in part) based on our knowledge of typical patterns in those styles.

• Corpus research helps identify those patterns and thus offers a window into our how we perceive and categorize diverse musical styles.

Corpus research into rock

• Rock is one of the most popular, most listened-to kinds of music in modern America (and many other countries as well).

• If we take the view that people’s music perception is—at least partly—shaped by statistical regularities in the music they hear, then studying rock may shed interesting light on the music perception of modern Western listeners.

Our goal:

To get statistical evidence about patterns in rock music

A collaborative project with David Temperley (Eastman)

What is rock music? Choosing the corpus: a broad definition

Rolling Stone magazine “500 Greatest Songs of All Time” (2004)

(the “RS 500”)

1: “Like a Rolling Stone” (Bob Dylan, 1965)

2: “Satisfaction” (The Rolling Stones, 1965)

3: “Imagine” (John Lennon, 1971)

4: “What’s Going On” (Marvin Gaye, 1971)

5: “Respect” (Aretha Franklin, 1967)

....

30: “I Walk The Line” (Johnny Cash, 1956)

44: “Georgia On My Mind” (Ray Charles, 1960)

256: “Paranoid Android” (Radiohead, 1997)

346: “California Love” (Dr. Dre and 2Pac, 1996)

399: “Enter Sandman” (Metallica, 1991)

What patterns to investigate?

Musical patterns can exist within many domains.

• harmony, melody, rhythm, timbre

We chose to study patterns in harmony first.

• How do harmonic patterns in rock compare or contrast to harmonic patterns in other styles?

• Can we expect certain patterns of harmony in rock music?

• Does rock music have operative harmonic principles at all?

Our work so far

• initial publication (dealing with harmony):

– de Clercq, Trevor and David Temperley. (2011). “A corpus analysis of rock

harmony.” Popular Music 30/1: 47-70.

• Popular Music article reports on a 100 song subset of RS 500:

– 20 top songs from each decade, ‘50s – ‘90s (“RS 5x20”)

• harmonic analyses (plus melodic transcriptions and timing data) available online at:

http://theory.esm.rochester.edu/rock_corpus/

Crash course in harmony

• Music theorists traditionally categorize harmonic entities (i.e., “chords”) via Roman numerals.

• Roman numerals describe triads built scale degrees.

C major scale

C major triads

Roman numerals describe classes of pitches.

D major triads

V chords in D major

Diatonic and (some) non-diatonic triads in the “key” of “C”

Harmony in certain styles displays particular patterns.

Common-practice music (e.g., Bach, Beethoven, Brahms)

• Pre-dominants (ii, IV) typically move to Dominants (V, viio)

• Dominants (V, viio) typically move to Tonics (I)

• Phrase model: T – (PD) – D – T

• e.g., “The Four Seasons,” Spring, mv. 1 (A. Vivaldi, 1725)

• e.g., String Quartet #51, menuetto (F. J. Haydn, 1790)

Similar principles are found in jazz music (ii – V – I)

Common-practice harmonic patterns can be found in rock

• “Twist and Shout” (The Beatles, 1963)

• I – IV – V

Other songs go against common-practice harmonic patterns

• “The Lemon Song” (Led Zeppelin, 1963)

• I – V – IV – I (blues cadence)

• “Louie Louie” (The Kingsmen, 1963)

• I – IV – v – IV

What (if any) are organizational principles of harmony in rock?

• Music theorists give conflicting views

• Walter Everett, 2004: "Making Sense of Rock's Tonal Systems". Music Theory Online. 10/4 (December).

=> rock as common-practice system

• Ken Stephenson, 2002What to Listen for in Rock: A Stylistic Analysis. New Haven: Yale University Press.

=> rock as opposite to common-practice system

• Allan Moore, 2001Rock: The Primary Text: Developing a musicology of rock. Aldershot, UK: Ashgate.

=> rock as a modal system

Timbre and texture strongly influence our perception of styles

.... but harmony (and other factors) play roles

Ex: Vitamin String Quartet

• LZT

• DDHW

• LGT

How to encode the corpus?

• Songs individually analyzed by both authors• Recursive notational system

“Da Doo Ron Ron” (The Crystals, 1963)

A: I | IV | V | I |

In: I |*4Vr: $A*2 I | IV | I | V | $A I |*2So: $A*2Ou: $A*4

S: [Eb] [12/8] $In $Vr*2 $So $Vr $Ou

% Bohemian RhapsodyA: bVI V #IV V |B: vi | ii . ii42 viih7 |C: IV64 I . . |

In1: [Bb] vi7 | V7/V | V7 | I | vi | [Eb] V7 | I | ii/V | V/V | $A*2 [Bb] IV I6 | viix42/V V64 | |Vr1a: [Bb] I |*3 vi | ii | ii V | I | vi | ii | viih7 . . ii64 |Vr1b: [Eb] I . . V65 | $B V | I V6 | Vr1b1: [Eb] $Vr1b vi iv | I | [2/4] |Vr1b2: [Eb] $Vr1b $B V | I V6 | $B bVII . bVII/bVII vi/bVII |Vr1: $Vr1a $Vr1b1Vr2: $Vr1a $Vr1b2

Br1: [A] I | | $C*2 IV64 I IV64 I | . . IV64 I |Br2: [A] III64 V/III | bIII64 V | I |*3 [2/4] V |Br3: [Eb] I | $A*2 $C*2 IV I6 | V/V V | IV I6 viix42/V ii7 | $A*2Br4: [Eb] I V I . | V . . I | . V I V | . . . I | . V I V | . . . I | V . . I | V | bIII |Br5: [Eb] bVI V/VII VII V/bIII | bIII V I . | V . . I | IV I V . | I IV64 | V/iii iii | V |*4Rf1: [Eb] [12/8] I |*3 V/V | V | . I | V | [6/8] . bVII | [12/8] V | . I | IV | ii | V | ii | V | ii V | ii V |Rf2: [Eb] [12/8] I |*3 V/V | bIII IV #IV | bVI | IV | V |*3

Ln: [Eb] [4/4] I V6 | vi . V6/vi vi | V6/vi vi V I | V/iii iii | IV I |Ref: [Eb] vi iii | vi iii | vi iv | V11 | I IV64 | I viix43/V | V6 iv6/II | [F] V | . I | | |

Pt1: $In1 $Vr1 $Vr2Pt2: $Br1 $Br2 $Br3 $Br4 $Br5 $Rf1 $Rf2Pt3: [4/4] $Ln $RefS: [Eb] $Pt1 $Pt2 $Pt3

Recursive computer program “expands” harmonic analyses

Expanded version of “Da Doo Ron Ron” (The Crystals, 1963)

I | | | | | IV | V | I | | IV | V | I | ....

... and also creates a CHORD LIST

start end key chord chromatic relative root0.00 5.00 E I 05.00 6.00 E IV 56.00 7.00 E V 77.00 9.00 E I 09.00 10.00 E IV 5(and so on....)

... So that we can then run statistics on the data

Statistics show that harmonic analysis is (somewhat) subjective

• agreement on chromatic relative root (e.g. I vs. IV): 92.4 %

• agreement on absolute root (e.g. A vs. D): 94.4 %

(Rolling Stones, “Satisfaction”)

• agreement on key (or pitch center): 97.3 %

(Lynyrd Skynyrd, “Sweet Home Alabama”)

• # of songs with 100% agreement: 39

• # of songs where agreement was between 90-99%: 39

(The following statistics are averages of those from DT and TdC)

Statistics show information on harmonic palette(zeroth-order probabilities)

Top five chords: I, IV, V, bVII, VI. Very different from common-practice music, especially IV>V and high freq. of bVII.

Statistics show information on harmonic palette over time

(zeroth-order probabilities)

Statistics show information on harmonic syntax

(first-order probabilities)

Transitions from one chord (antecedent) to another (consequent)

Relationships between distribution probabilities

Distribution of roots overall and in pre- and post-tonic positions

IV chord seems to function as a preparation for tonic

Harmonic information abstracted from key or function

Root motions in the RS 5x20 corpus by interval size

Lots of motion (up or down) by P4; M2 next most common

Root motions on a “line of fifths”

Chord vectors

For each chromatic relative root, we created a vector of 99 values (one value for each song), 1 if the song contains the chromatic relative root and 0 otherwise. Correlating these vectors for a pair of chords gives a measure of how much they occur together (not necessarily adjacently) in the same songs.

Chord vectors (cont’d)

Chord vectors (cont’d)

Correlations above 0.350 are circled.

Chord pairs with high correlations (above 0.350)

• IV and V

• VI, II, and III

• bIII, bVI, and bVII

Correlations suggest some sort of modal harmonic organization.

Conclusions from harmonic data

• Rock has its own harmonic logic, very different from that of common-practice music

• IV is the most common non-tonic chord in rock, and is especially common preceding the tonic

• Rock does not show strong asymmetries in root motion; ascending and descending 5th motions are roughly equally common

• Frequency of root motions corresponds strongly to circle-of-fifths distance

• Patterns of co-occurrence suggest “flat-side” harmonies tend to occur together, as do “sharp-side” harmonies

What about scales in rock music?

With our melodic transcriptions, we can answer questions about the types of pitch collections (i.e., “scales”) used in rock music.

- Does rock have a consistent “scale”? Can we distinguish diatonicfrom chromatic scale-degrees, as we do in classical music?

- Do rock songs group into natural categories with regard to theirscalar organization – analogous to classical major and minor?

There’s been much speculation about these questions. With regard to scales, a variety of frameworks have been applied to rock:

- Common-practice major and minor scales

- Pentatonic scales

- Diatonic modes (Mixolydian, etc.)

- Blues scales

- Scales arising from harmonic progressions

...but there’s little consensus on this issue, and little hard evidence has been put forth.

Notation

We devised a simple notation for transcribing the melodies. For “Hey Jude”, for example, here is a transcription of the first main section:

[F] [OCT=4] ...5 | 3....356 | v2.....23 | 4.^1..175 | 6.543.........5. | 6.6...6.21.7.16. | 5...v1236 | 5..54377 | 1....... |

Vertical bars indicate measures. Each measure is evenly divided into N segments, where N is a power of 2 (assuming duple meter). If a segment contains a note onset, that is indicated with the scale-degree of the note, otherwise a dot is used.

[F] [OCT=4] ...5 | 3....356 | v2.....23 | 4.^1..175 | 6.543.........5. | 6.6...6.21.7.16. | 5...v1236 | 5..543.7 | 1....... |

Major-scale degrees are assumed, unless otherwise indicated (e.g. “b7”). Each pitch is assumed to be the closest representative of that scale-degree to the previous note, unless indicated by “v” (which shifts an octave down) or “^” (which shifts an octave up).

[F] indicates the tonal center; [OCT=4] indicates the octave of the first note (middle C = C4).

Problems

We found the task to be quite difficult—a good deal more difficult than the harmonic analysis.

- It’s not always obvious which vocal line is the “melody” (we’ll see an example in a minute)

- Some notes are indeterminate as to pitch – “blue notes” that fall between two pitches, or notes with gliding pitch, or notes that are quasi-spoken

- Some notes are indeterminate as to rhythm (or seem to require very complex rhythmic notation)

(These problems of transcription are not unique to rock but occur with many kinds of music.)

Problems

Here’s an especially complex example – a phrase from Otis Redding’s “I’ve Been Loving You Too Long” (here’s TdC’s transcription, in conventional notation):

There’s a slight scoop up to the first long note, and then a significant glide downwards at the end of it; should these be notated or not? (TdC notates the second, not the first.) Should the rhythm of the next phrase be notated literally (as TdC does) or is it just an expressively stretched rendition of a simpler rhythm (e.g. ee | e e e q. )?

Problems

Eight songs on the list were judged to have no melody at all—either rap / hip hop songs, or those in which the pitches of the melody were largely indeterminate. This left a corpus of 192 songs.

Level of agreement

We both analyzed eight songs so as to examine the level of agreement between us. We were in agreement only 74% of the time. (On the harmonic analyses it was much higher: 92.4%.)

In some cases, we simply did not agree as to which line was the melody. However, there were also more substantive disagreements. Generally, TdC seemed to take a “literal” approach, transcribing exactly what was sung; DT seemed to go for the intended or implied notes.

An Example

Here’s just one example of the kind of disagreements that arose between us—a phrase from “London Calling,” by the Clash.

ANALYSES OF THE DATA

Scale-Degree Distribution

What is the overall distribution of scale-degrees in the data—that is, pitch-classes in relation to the tonic?

Let’s examine this in comparison to some other distributions.

Scale-degree distribution for classical music (gathered by Temperley from a corpus of excerpts from the Kostka-Payne theory textbook, grouping major and minor together)

Scale-degree distribution for classical music (gathered by Temperley from a corpus of excerpts from the Kostka-Payne theory textbook, grouping major and minor together)

_ _ _ _ _ _ _ (Major-scale degrees)

The seven major degrees are most common (partly because there are more major than minor pieces). Next are the minor degrees—b3, b6, and b7. (7 is much more common than b7.) b2 and #4 (“chromatic” degrees) are least common.

Comparing the scale-degree distribution from our rock harmonic analyses (counting each pitch-class once for every chord that it occurs in), we see a fairly similar pattern...

As in classical music, b2 and #4 are least common. 7 is still morecommon than b7, but the difference is smaller. The 6 > b6 differenceis greatly increased.

Now we add in the melodic rock data (blue). (We count each note separately, not weighted for duration.)

Similar to the harmonic rock data and classical data. But now, b7 > 7, by a considerable margin. Note also the very low value for b6 (but still higher than b2 and #4).

Our melodic data (as well as our harmonic data) support the idea that rock as a whole reflects a “global” scale collection including all 12 degrees except for b2 and #4. (Temperley’s “supermode” [2001]).

This is an important commonality with common-practice music, where b2 and #4 are the least common degrees (chromatic in both major and minor).

The fact that b7 > 7 in our melodic data is interesting, since neither classical major (above) nor classical minor (below) reflect this. (Data is again from the Kostka-Payne corpus.)

Major

Minor

Our data may explain a curious feature of Krumhansl & Kessler’s (1982) key-profiles. In their probe-tone experiments, given minor-key contexts, b7 was given a slightly higher rating than 7, though 7 occurred in the contexts and b7 did not. The K&K minor key-profile:

Perhaps this reflects the influence of rock melodies. (K&K’s subjects were trained classical musicians, but undoubtedly had heard a lot of popular music as well.)

Scales in Rock

To what extent do rock songs reflect conventional scales—modal, pentatonic, etc.?

As a first approach, we can represent each song with a binary 12-valued vector, showing which scale-degrees occur in the song and which ones do not. A song that uses only the major scale would therefore have this vector:

1 0 1 0 1 1 0 1 0 1 0 1

(1 b2 2 b3 3 4 #4 5 b6 6 b7 7)

The most common vectors, with their frequency of occurrence (out of 192 songs): Num occ.1. 101011010101 29 (major)

2. 101011010100 17 (“diatonic hexachord”)

3. 101111010110 10 (bluesy?)

4. 101111010100 7 (bluesy?)

5. 101010010100 7 (major pentatonic)

Not terribly revealing because it does not consider how often each scale-degree is used in a song. (Note also that these 5 scales account for only 70 songs, less than 40% of the total.)

A second approach: Take a song and a scale, and consider whether the degrees of the scale are the most frequently occurring scale-degrees in the song. If so, we declare that song to “match” that scale. (So this allows some “chromaticism” in relation to the scale.)

We can then consider how many songs match various scales.

Examining some scales that are of particular interest:

Num matches101011010101 42 (Major)

101011010110 12 (Mixolydian)

101101010110 9 (Dorian)

101101011010 6 (Aeolian or“natural minor”)

101010010100 50 (major pent)

100101010010 21 (minor pent)

This data does not give much support to the idea that rock is largely “modal”. There are relatively few songs in which the most frequent scale-degrees form a diatonic mode (other than major). There’s some evidence of pentatonicism, though this matching method favors scales with fewer notes.

Clustering

Both of these approaches suggest that there is a good deal of variability in the scale content of songs: from a purely statistical perspective, many different scales are used.

Can we simplify this picture in any way? If we were to “cluster” rock songs into a small number of categories as to their scalar content, what would the categories look like?

A “brute-force” approach: Let’s classify a song as “major” if 3 > b3, “minor” otherwise. We create a “major” distribution from all the major songs and a “minor” distribution from all the minor ones; what do these distributions look like?

Here are the results. (The “major” category has 121 songs, the “minor” has 71. Fifty-two of these songs use both 3 and b3.)

The “major” distribution reflects the major scale; the five degrees of the major pentatonic are the most common (1, 2, 3, 5, 6).

In the “minor” distribution, the five minor pentatonic degrees are most frequent (1, b3, 4, 5, b7); the next most frequent are 2 and 6, suggesting Dorian mode. (6 > b6, but the difference is smaller than in major.)

This approach simply imposes a kind of major/minor categorization. Could we take a more data-driven approach to sorting the songs into clusters?

A “hill-climbing” approach:

1. Randomly sort the songs into two categories.

2. For each category, construct an aggregate scale-degree distribution.

3. Within each category, measure the match of each song to the category’s distribution (here we use cross-entropy). Combining all these values produces a measure of the “quality” of that clustering.

4. Randomly shift one song from one category to the other, and repeat the process. If this change improves the quality of the clustering, keep it; if not, don’t.

5. Iterate this process many times.

The result (after 800 iterations):

Once again, we get something that looks roughly like “major” vs. “minor” (almost identical to the “brute-force” approach above!). Category 1 has higher values for b3 and b7 (and a relatively higher value for b6); category 2 has higher values for 3 and 7.

Note that this method is not guaranteed to find the optimal solution. However, the process was repeated several times, starting with different random sortings, and always converged on the same solution.

What about repeating the process, but sorting the songs into threecategories? The result:

Category 1 (blue, 41 songs) looks like major pentatonic (1, 2, 3, 5 and 6 are highest)

Category 2 (red, 73 songs) looks like major diatonic (or diatonic hexachord)

Category 3 (green, 78 songs)—almost identical to the “minor” category we saw before.

Category 3—the “minor” category—is of particular interest:

As noted earlier, the minor pentatonic degrees are most frequent, followed closely by 2. But both 3 and 6 are quite common. This also resembles the “blues” scale, though this is defined in different ways (#4 is often included in the blues scale). It is also the union of the major and minor pentatonic scales.

Many of the songs in this category are 1) early (1950s) rock’n’roll songs, 2) songs by “hard rock” or blues-based bands like the Rolling Stones, or 3) soul songs such as Aretha Franklin’s “Respect.”

Conclusions

1. Rock melody, like rock harmony, reflects a global scale in which all twelve chromatic scale-degrees are fairly common except for b2 and #4. (b6 is, however, a borderline case.)

2. The biggest difference between rock’s melodic distribution and that of common-practice music is the fact that b7 > 7.

3. Statistical clustering methods suggest some kind of (loosely speaking) major/minor dichotomy in rock. One category of songs features 3 > b3, 7 > b7, and 6 >> b6; the other category features b3 > 3, b7 > 7, and a smaller difference between 6 and b6.

Conclusions (cont’d)

4. The rock “minor” is, however, quite different from classical minor. b7 > 7 (unlike in classical minor); 6 > b6 (unlike in classical minor); and 3 is quite common. Further work is needed to determine whether this distribution really represents a consistent melodic practice, or perhaps several quite different ones.

Directions for Further Work

Our work raises a number of questions that deserve investigation:

1. How does the scale-degree distribution of rock change over time (from 1950 to 2000)?

2. We ignored the duration of notes in our tallies; would the distributions change significantly if notes were weighted by their duration?

3. What is the absolute distribution of pitch-classes in rock? Does it show a strong preference for certain pitch-classes over others?

Many other kinds of issues could be addressed with our data:

- What are the characteristic melodic patterns of rock? No doubt rock features certain phenomena that are found in many other styles, such as a preference for small intervals. But are there characteristic melodic gestures that are unique to rock?

- To what extent is melody in rock constrained by harmony? (Some have suggested that these constraints are much weaker in rock than in common-practice music.) Does our harmonic data yield a similar “clustering” of songs to the melodic data?

5. Rhythm....?! Many questions could be asked here as well.

Thank you for your attention!