Geber Ramalho Automatic Music Accompaniment: a 10-year research program.
-
Upload
ariel-wiggins -
Category
Documents
-
view
220 -
download
4
Transcript of Geber Ramalho Automatic Music Accompaniment: a 10-year research program.
Geber Ramalho – SBCM’03
3Why to build such systems?
Arrangement assistant (Rhythm session)
Rehearsal partner
Performance (Stage) partner
Tutoring system
In other markets, such as computer games
Geber Ramalho – SBCM’03
4Roadmap
Music Accompaniment
Fragment reuse approach
Research retrospective• ImPact System: jazz bass lines
• Chord Prediction
• D’Accord Guitar
Research perspectives
The Task
ReasoningReasoning
Knowledge:
- Music theory, phrases, ...- skills ...
Goals:- ??
Actions:- Choose and play chords, melody,rhythms
executionexecutionperceptionperception
Chords: ... A7(b9 | Cm7 | F7 | Fm7 | Bb7 | Ebmaj7 . . .
Solo: ...
Piano: ...
Drums: ...
Bass: ...
Agent
Geber Ramalho – SBCM’03
6Challenges
Few criteria for evaluating a lot of possible solutions
Knowledge acquisition: hard to justify note by note
Collective creation in real time• Fast and coordinated reactions and planning
Geber Ramalho – SBCM’03
8Main Approaches
Creating from scratch note by note• Grammars, Logical Rules, Probability,… for choosing notes
– Ex.: Jonhson-Laird´s, Cybernetic-Composer, etc.
Fragments reuse• Creating new accompaniments by appending music
fragments, retrieved from a library, to one another– Ex.: Band-in-a-Box, ImPact, Hodgson’s, etc.
Geber Ramalho – SBCM’03
9From scratch note by note
Probabilistic derivational tree (mostly used)• Or markovian chain approaches
Grammars or rules• Use notes from the local tonality
• Possibly combined with random choices “prefer short notes at the beginning of the phrase”
Constraint-based programming• Filter-out bad notes or phrases
Etc.
.
35
65
50
50
2560
15
100
100
100
100
100
Geber Ramalho – SBCM’03
10
Melodic and Rhythmic Fragments
Dm7 A7
Miles Davis:
Important role in creation (specially in Jazz)
• Learning by imitation
• Pattern-oriented musicology
Geber Ramalho – SBCM’03
11
Melodic and Rhythmic Fragments
Minimize knowledge acquisition effort
• They are available (scalability)
• Adequate level of granularity
• Embody a knowledge hardly formalized
Improves expressiveness
Cm7(b5) F7
Em7(b5) A7
stepwise, dissonant, low density, ascending, etc.
Chord-based, consonant, medium density, etc.
Geber Ramalho – SBCM’03
12Fragments Reuse
Idea (Case-Based Reasoning)• Solve new problems (e.g., chord sequence) adapting
solutions (e.g., musical fragments) used in previous problems (e.g., played before)
Questions• How long should be the fragments?
• How to represent them?
• How to index them?
• Why to choose a particular fragment?
• How to adapt a retrieved fragment to the new context?
Geber Ramalho – SBCM’03
13Questions
Fragments Length • Plausibility (variable length) vs. Simplicity (fixed length)
• Reactivity (short) vs. Recurrence (long)
Fragment indexation (attributes)• Context
– Underlying chords, position, tempo,...
• Musical properties – Dissonance, density, range, melodic contour,…
• Time window (how far to go to the past?)
|Em7(b5) | A7(b9) | Cm7 | F7 | Fm7 ...
Geber Ramalho – SBCM’03
14Questions
Fragment Choice• Query Q = (C, D) ,
– where C = {c1,...,ck} context descriptionD = {d1,...,dj} desired musical
properties
• Retrieval function F(Q,L), where L is the library– Random– Based on similarity– Based on weights given by the user
Geber Ramalho – SBCM’03
16Main Loop
1) Segment the given Chord Grid into “Chord Chunks”e.g., II-, II-V-I, IV-II-V-I,…
2) Determine the desired “musical properties” of the segment
a - Activate PACTs (intentions)
b - Select the PACTs concerning the segment;
c – Assemble these PACTs to compose the query
3) Retrieve a fragment and adapt it
Geber Ramalho – SBCM’03
17Rules for Activating PACTs
Examples
• IF the agent is playing at the theme exposition, THEN “play chord notes until the next section”
• IF the next chord is highly dissonant, THEN “use chromatic scale during it”
• IF the drummer is playing a RIFF, THEN “choose a similar rhythm”
Geber Ramalho – SBCM’03
21
Given the chord sequence (and a melody) played so far, which is the next chord?
No precise rules for chord prediction• It depends on the music style
• the musicians themselves
• the very song being played, ...
How to extract structural information from songs?
Chord Prediction
Geber Ramalho – SBCM’03
22Our approach on chord prediction
Previous approach (Thom & Dannemberg 95)• n-gram probabilistic models algorithm
• Error rate 42%, for songs in C tonality and without prediction of chords duration
Our Hybrid system• Prediction technique: neural networks (=> prior knowledge on
typical chord sequences)
• Online adaptation: knowledge based (=> sequence tracking rules)
Chords SequenceTracker
NeuralNet next
chord
Geber Ramalho – SBCM’03
23Results
On Jazz standards in their original tonality
• MLP-backpropagation: error rate = 11,2%
• Depending on the song, error rate decreases 3% per repetition in average
Sound Example: So Nice
1 Cm7 2 Dm7b5 G7 3 Cm7 4 Cm7Am7 Em
5 Bbm7Bbm7 Em7
6 Eb7 7 AbMaj7 8 AbMaj7
9 Dm7b5 10 G7 11 Dm7b5 G7 12 Cm7
13 Am7b5Bm
14 Ab7 15 Dm7b5 16 G7Am
17 Cm7 18 Dm7b5 G7 19 Cm7 20 Cm7
21 Bbm7 22 Eb7 23 AbMaj7 24 AbMaj7
25 AF
26 EbMaj7 27 Dm7b5 G7Dm7b5
28 Cm7
29 F7 30 Fm7 Bb7Fm7
31 EbMaj7 21 Dm7b5 G7
1st execution
7 erros (in red): 4, 5, 13, 16, 25, 27, 30
Sound Example: So Nice
1 Cm7 2 Dm7b5 G7 3 Cm7 4 Cm7
5 Bbm7Bbm7 Em7
6 Eb7 7 AbMaj7 8 AbMaj7
9 Dm7b5 10 G7 11 Dm7b5 G7 12 Cm7
13 Am7b5 14 Ab7 15 Dm7b5 16 G7Am
17 Cm7 18 Dm7b5 G7 19 Cm7 20 Cm7
21 Bbm7 22 Eb7 23 AbMaj7 24 AbMaj7
25 A 26 EbMaj7 27 Dm7b5 G7Dm7b5
28 Cm7
29 F7 30 Fm7 Bb7Fm7
31 EbMaj7 21 Dm7b5 G7
2nd execution
4 erros (in red): 5, 16, 27, 30
Sound Example: So Nice
3rd execution
0 erros!
1 Cm7 2 Dm7b5 G7 3 Cm7 4 Cm7
5 Bbm7 6 Eb7 7 AbMaj7 8 AbMaj7
9 Dm7b5 10 G7 11 Dm7b5 G7 12 Cm7
13 Am7b5 14 Ab7 15 Dm7b5 16 G7
17 Cm7 18 Dm7b5 G7 19 Cm7 20 Cm7
21 Bbm7 22 Eb7 23 AbMaj7 24 AbMaj7
25 A 26 EbMaj7 27 Dm7b5 G7 28 Cm7
29 F7 30 Fm7 Bb7 31 EbMaj7 21 Dm7b5 G7
Geber Ramalho – SBCM’03
27
D’Accord Guitar Project(www.daccord.com.br)
2 researchers (Geber Ramalho e Carlos Sandroni)
2 Ph.D. students (Giordano Cabral e Ernesto Lima)
1 M.Sc. Student (Márcio Dahia)
6 undergrad students (Hugo, Bel, Tiago,...)
Geber Ramalho – SBCM’03
29Projeto D’accord
3 Modules• D’Accord Guitar Player
• D’Accord Guitar Editor
• D’Accord Guitar Browser
Geber Ramalho – SBCM’03
30Main Challenges
File Format (synchronize information)
Precise scheduling
Cipher interpretation
Solo transcription
Transposition
Cipher Interpretation
Challenges• various positions for the same note and various fingers for
the same position
=> hundreds of possible positions for the same chord
• Theoretical, cultural, personel and anatomic constraints: how to take them into account?
Geber Ramalho – SBCM’03
32Cipher Interpretation: steps
Valid chords• Identify valid notes in the fret board
• Identify valid positions in the fret board (according to music theory)
• Identify valid fingerings
Evaluate each chord and chord transition to find the best path
• Multi-criteria decision
Cipher Interpretation
For a single chord• Facility (anatomy/rules)
• Flexibility (anatomy/rules): to play other notes
• Frequency (cultural/data base): how classic?
• Sonorority (cultural/data base): aesthetic
For a chord transition• Facility (anatomy/rules)
• Bass line (musical theory/rules): voice leading
• Melodic (soprano) line (musical theory/rules): voice leading
Geber Ramalho – SBCM’03
35Results
More than 40.000 users (beta test)
95% considered D’Accord Guitar “good” or “excellent”
However, the right hand was not that good!!• Too repetitive
Geber Ramalho – SBCM’03
36
D´Accord Guitar: rhythmic accompaniment
From scratch via a computer keyboard
• good quality
• time-consuming
Apply a rhythmic pattern to all chords
• bad quality
• quite fast
Geber Ramalho – SBCM’03
37Cyber-João
Adapt the Impact approach will require• New musical fragments
• New attributes and
• New rules
Due to the changes• Bass to Guitar
• Melody to Rhythm
• Jazz to Bossa Nova
João Gilberto’s records• Manually identified some
rhythmic patterns
Geber Ramalho – SBCM’03
38Cyber-João
Examples• Chega de Saudade
• Desafinado
Question• Is the approach scalable?
Geber Ramalho – SBCM’03
39VexPat
A Pattern Extraction tool• Identify similar recurrent music material (melody, rhythm, …)
Projects
MIDIGuitar
MIDIGuitar
CDsCDs Dataacquisition
Symbolic(MIDI) data
Environment/ToolsPattern Extraction
VisualizationMachine Learning
Pre-processing(meta-data inductionand noise filtering)
New Symbolic data
Basic D’Accord Guitar
(+ Cyber-João)
D’Accord GuitarPersonalization
Automatic MusicAnalysis
Prototypedeveloped
Geber Ramalho – SBCM’03
42Musicological Analysis of Brazilian Guitar
There are lots of knowledge in harmony, melody, counterpoint,…
However, we know not too much about • Rhythm in general and
• (non-classic) guitar in particular
Geber Ramalho – SBCM’03
43Musicological Analysis of Brazilian Guitar
Some open questions• What is a rhythmic pattern, considering short (millisecond)
time variations?
• How performance modifies the standard rhythm?
• Are there rhythmic patterns for a given style and/or performer?
• Is there a way to formalize pattern choices? In which situations are they used?
– Harmonic, melodic, location, tempo…– Previous/next patterns (grammar)…
• How frequently are they used?
• Are there variations of these patterns?