10/3/2015ESL Conference: Kananaskis 1 It Don’t Mean a Thing If It Ain’t Got that Swing.
It Don’t Mean a Thing - Jazzomat · 2018-07-08 · It Don’t Mean a Thing (If It Ain’t Got...
Transcript of It Don’t Mean a Thing - Jazzomat · 2018-07-08 · It Don’t Mean a Thing (If It Ain’t Got...
It Don’t Mean a Thing
(If It Ain’t Got That Swing)
Estimating Swing Ratios and Soloist Micro-timing
from Jazz Recordings with Aligned Beat Grids
Christian Dittmar
© AudioLabs, 2014
Author
Title of Presentation
2
Overview
Introduction
Swing ratio
Ensemble timing
Automatic method
Swing ratio estimation
Log-lag ACF analysis
Preliminary results
Dataset / Comparison to prior work
Example cases
Log-lag ACF
Futher steps
© AudioLabs, 2014
Author
Title of Presentation
3
Introduction: Swing ratio
„Jazz music is supposed to swing“ (Friberg & Sundström
2002)
Lengthening odd (on-beat) 8th notes
Shortening even (off-beat) 8th notes
Ratio of long 8th to short 8th swing ratio
1 straight binary 2 tied-triplet
1 2 3 4 5 6 7 8
𝑠𝑟
© AudioLabs, 2014
Author
Title of Presentation
4
Introduction: Swing ratio
Jazz drummers often use typical cymbal pattern inducing
the swing (4th, long 8th, short 8th, 4th, long 8th, etc. …)
Relation between swing ratio and relative note durations:
𝑙𝑜𝑛𝑔 8𝑡ℎ = 𝑠𝑟
1 + 𝑠𝑟
𝑠ℎ𝑜𝑟𝑡 8𝑡ℎ = 1
1 + 𝑠𝑟
𝑠𝑟 = 𝑙𝑜𝑛𝑔 8𝑡ℎ
𝑠ℎ𝑜𝑟𝑡 8𝑡ℎ
𝑙𝑜𝑛𝑔 8𝑡ℎ + 𝑠ℎ𝑜𝑟𝑡 8𝑡ℎ = 1
© AudioLabs, 2014
Author
Title of Presentation
5
Introduction: Swing ratio
Illustrative example: Swing ratio 1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Time in Seconds
Wavefo
rm /
Am
plit
ude E
nvelo
pe
Cymbal pattern at 120 BPM and swing ratio 1
𝑠𝑟 =0.25
0.25= 1
© AudioLabs, 2014
Author
Title of Presentation
6
Introduction: Swing ratio
Illustrative example: Swing ratio 2
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Time in Seconds
Wavefo
rm /
Am
plit
ude E
nvelo
pe
Cymbal pattern at 120 BPM and swing ratio 2
𝑠𝑟 =0.333
0.167= 2
© AudioLabs, 2014
Author
Title of Presentation
7
Introduction: Swing ratio
Illustrative example: Swing ratio 3
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Time in Seconds
Wavefo
rm /
Am
plit
ude E
nvelo
pe
Cymbal pattern at 120 BPM and swing ratio 3
𝑠𝑟 =0.375
0.125= 3
© AudioLabs, 2014
Author
Title of Presentation
8
Introduction: Swing ratio
Illustrative example: Swing ratio 4
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Time in Seconds
Wavefo
rm /
Am
plit
ude E
nvelo
pe
Cymbal pattern at 120 BPM and swing ratio 4
𝑠𝑟 =0.4
0.1= 4
© AudioLabs, 2014
Author
Title of Presentation
9
Introduction: Ensemble timing
Friberg‘s observations:
Less pronounced swing of the soloist (wind instruments)
Large onset delay w.r.t. on-beat
Small onset delay w.r.t. off-beat (coincide with cymbal)
© AudioLabs, 2014
Author
Title of Presentation
10
Automatic method: Swing ratio estimation
Compute spectrogram from solo excerpts via STFT
Detection of percussive onsets candidates
Heuristics to retain only onsets in upper freq. band
Detection of onset ‚triplets‘ within one beat (plus tolerance)
Derivation of swing ratio
Matching of soloist onsets to the previously found ‚triplets‘
Derivation of swing ratio from matched ‚triplets‘
Measurement of onset delay w.r.t. on-beat and off-beat
© AudioLabs, 2014
Author
Title of Presentation
11
Automatic method: Log-lag ACF analysis
Ommit error-prone steps:
Cymbal onset detection
‚triplet‘ selection from onsets
Autocorrelation on onset detection function
Resample ACF to logarithmically-spaced lag-axis
Tempo differences turn into shifts (Gruhne & Dittmar 2009)
Common rhythmic patterns are retained
© AudioLabs, 2014
Author
Title of Presentation
12
Automatic method: Log-lag ACF analysis
Illustrative example: Swing ratio 1
294 243 200 165 136 112 93 76 63 520
0.1
0.2
0.3
0.4
0.5
Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 1
Log-lags in BPM
Peridic
ity s
alie
nce
120
BPM
Eighth
(240 BPM)
© AudioLabs, 2014
Author
Title of Presentation
13
Automatic method: Log-lag ACF analysis
Illustrative example: Swing ratio 2
294 243 200 165 136 112 93 76 63 520
0.1
0.2
0.3
0.4
0.5
Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 2
Log-lags in BPM
Peridic
ity s
alie
nce
120
BPM
Eighth
(240 BPM)
© AudioLabs, 2014
Author
Title of Presentation
14
Automatic method: Log-lag ACF analysis
Illustrative example: Swing ratio 3
294 243 200 165 136 112 93 76 63 520
0.1
0.2
0.3
0.4
0.5
Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 3
Log-lags in BPM
Peridic
ity s
alie
nce
120
BPM
Eighth
(240 BPM)
© AudioLabs, 2014
Author
Title of Presentation
15
Automatic method: Log-lag ACF analysis
Illustrative example: Swing ratio 4
294 243 200 165 136 112 93 76 63 520
0.1
0.2
0.3
0.4
0.5
Log-Lag ACF of cymbal pattern at 120 BPM and swing ratio 4
Log-lags in BPM
Peridic
ity s
alie
nce
120
BPM
Eighth
(240 BPM)
© AudioLabs, 2014
Author
Title of Presentation
16
Preliminary results: Dataset
Subset of the Jazzomat data
Rhythm feel: Swing
Manually aligned beat grids
available
Manually transcribed solo wind
instruments play 8th lines
Relatively steady drum beat
Only ‘reliable’ results shown here
Drummers: Soloists:
© AudioLabs, 2014
Author
Title of Presentation
17
50 100 150 200 250 300 3500.5
1
1.5
2
2.5
3
3.5
4
Tempo in BPM
Sw
ing r
atio
Drummers' swing ratio vs. tempo
Billy Higgins
Art Taylor
Tony Williams
Max Roach
Carl Allen
Preliminary results: Drummers‘ swing ratio
Friberg & Sundström 2002: Jazzomat @ AudioLabs:
© AudioLabs, 2014
Author
Title of Presentation
18
100 150 200 250 300 3500.5
1
1.5
2
2.5
Tempo in BPM
Sw
ing r
atio
Soloists' swing ratio vs. tempo
John Coltrane
Art Pepper
Wayne Shorter
Dizzy Gillespie
Sonny Rollins
Preliminary results: Soloists‘ swing ratio
Friberg & Sundström 2002: Jazzomat @ AudioLabs:
© AudioLabs, 2014
Author
Title of Presentation
19
100 150 200 250 300 350-40
-20
0
20
40
60
80
100
120
Tempo in BPM
On-b
eat
dela
y in m
s
Soloists' on-beat deleay vs. tempo
John Coltrane
Art Pepper
Wayne Shorter
Dizzy Gillespie
Sonny Rollins
Preliminary results: Soloists‘ on-beat delay
Friberg & Sundström 2002: Jazzomat @ AudioLabs:
© AudioLabs, 2014
Author
Title of Presentation
20
100 150 200 250 300 350-40
-20
0
20
40
60
80
100
120
Tempo in BPM
Off
-beat
dela
y in m
s
Soloists' off-beat deleay vs. tempo
John Coltrane
Art Pepper
Wayne Shorter
Dizzy Gillespie
Sonny Rollins
Preliminary results: Soloists‘ off-beat delay
Friberg & Sundström 2002: Jazzomat @ AudioLabs:
© AudioLabs, 2014
Author
Title of Presentation
21
50 100 150 200 250 300 3500.5
1
1.5
2
2.5
3
3.5
4
Tempo in BPM
Sw
ing r
atio
Drummers' swing ratio vs. tempo
Billy Higgins
Art Taylor
Tony Williams
Max Roach
Carl Allen
Preliminary results: Example cases
Art Pepper – Blues for Blanche:
28.5 29 29.5 30 30.5 31
Cymbal: Billy Higgins
Soloist: Art Pepper
Cymbal beat grid
Tapped beat grid
Time in Seconds
MelID: 2 - Art Pepper - Blues for Blanche
© AudioLabs, 2014
Author
Title of Presentation
22
50 100 150 200 250 300 3500.5
1
1.5
2
2.5
3
3.5
4
Tempo in BPM
Sw
ing r
atio
Drummers' swing ratio vs. tempo
Billy Higgins
Art Taylor
Tony Williams
Max Roach
Carl Allen
Preliminary results: Example cases
Steve Lacy – Let‘s Cool One:
85.5 86 86.5 87 87.5 88 88.5
Cymbal: Billy Higgins
Soloist: Steve Lacy
Cymbal beat grid
Tapped beat grid
Time in Seconds
MelID: 157 - Steve Lacy - Let's Cool One
© AudioLabs, 2014
Author
Title of Presentation
23
50 100 150 200 250 300 3500.5
1
1.5
2
2.5
3
3.5
4
Tempo in BPM
Sw
ing r
atio
Drummers' swing ratio vs. tempo
Billy Higgins
Art Taylor
Tony Williams
Max Roach
Carl Allen
Preliminary results: Example cases
Clifford Brown – George‘s Dilemma:
57 58 59 60 61 62 63 64
Cymbal: Max Roach
Soloist:
Cymbal beat grid
Tapped beat grid
Time in Seconds
MelID: 36 - Clifford Brown - George's Dilemma
© AudioLabs, 2014
Author
Title of Presentation
24
Preliminary results: log-lag ACF
Log-lag ACF shape correlates to swing ratio
Very noisy Normalized log-lag ACFs vs. swing ratio
Log-lags in BPM
Sw
ing
ratio
294 243 200 165 136 112 93 76 63 52
1.1811
1.3957
1.6192
1.8092
2.1355
2.553
© AudioLabs, 2014
Author
Title of Presentation
25
Further steps
More elaborate onset detection for cymbals
NMF-based drum transcription (Dittmar & Gärtner 2014)
Refine onsets using center of gravity in onset frame
Automatic detection of outliers
Systematically evaluate usage of log-lag ACF
Ingest automatically refined beat grid back to Jazzomat DB
Correct editorial metadata inconsistencies
© AudioLabs, 2014
Author
Title of Presentation
26
References
Anders Friberg & Andreas Sundström, „Swing Ratios and Ensemble Timing in
Jazz Performance: Evidence for a Common Rhythmic Pattern“, Music
Perception, 2002
Matthias Gruhne & Christian Dittmar, “Improving Rhythmic Pattern Features
Based on Logarithmic Preprocessing”, Proceedings AES International
Conference, 2009
Christian Dittmar & Daniel Gärtner, “Real-time transcription and separation of
drum recordings based on NMF decomposition”, Proceedings of the 17th
International Conference on Digital Audio Effects, 2014