Data Mining and Text Analytics in Music Audi Sugianto and Nicholas Tawonezvi.
Data Mining Music
-
Upload
paul-lamere -
Category
Technology
-
view
13.078 -
download
0
description
Transcript of Data Mining Music
Data Mining Music
Paul LamereMarch 11, 2012
#SXMusicData
#SXMusicData
Digital music revolution continues
I’ve got 10 million
songs in my pocket.
Now what?
#SXMusicData
10 Easy Pieces
• Artist names
• Artist relationships
• Fan Passion
• Better recommendations
• Frog-based playlisting
• Finding a better drummer
• The Loudness War
• Looking for the slow build
• Automatic music visualization
• Data-driven music remixing
Mining Data about music Mining music itself
#SXMusicData
Tale of Two Datasets
#SXMusicData
MusicBrainz
• Basic metadata
• 645K Artists
• 1M Releases
• 10M Recordings
• And more ...
Let’s have some fun with MusicBrainz data
#SXMusicData
Have band names been getting longer?
#SXMusicData
let’s use the some data to find out• Collect the top 500 artists for each 5 year window
(via Echo Nest or Musicbrainz)• Calculate the average name length for period.
Have band names been getting longer?
#SXMusicData
Have artist names been getting longer?
• Collect the top 500 artists for each 5 year window• Calculate the average name length for period.
The code is shorter than Zach’s answer
Let’s find out with big data!
(well, really not that big)
#SXMusicData
Have band names been getting longer?
#SXMusicData
The average length of artist names peaked in the period from 1955-1959.Average name length was 2 characters longer in 1955 than in 2012
Average name length
Year
Have band names been getting longer? NO
#SXMusicData
Band names from 1955-1959
44 Van McCoy & The Soul City Symphony Orchestra35 Academy of St. Martin in the Fields33 The Clancy Brothers & Tommy Makem33 Cliff Bennett & The Rebel Rousers32 Maurice Williams and The Zodiacs31 Herb Alpert & The Tijuana Brass30 Smokey Robinson & The Miracles30 Little Anthony & The Imperials29 Van Morrison & The Chieftains29 Nelson Riddle & His Orchestra29 George Clinton and Parliament29 Frankie Lymon & The Teenagers29 Clancy Brothers & Tommy Makem
#SXMusicData
Longest band names
51 2010 Tim and Sam’s Tim and the Sam Band with Tim and Sam48 1994 ...And You Will Know Us By The Trail Of The Dead47 1967 Charles Wright & The Watts 103rd St Rhythm Band46 1993 The Presidents of the United States of America44 1959 Van McCoy & The Soul City Symphony Orchestra43 2000 Richard Cheese & Lounge Against The Machine43 1950 Benedictine Monks of Santo Domingo de Silos41 1981 Emir Kusturica & The No Smoking Orchestra39 1972 Afrika Bambaataa & The Soul Sonic Force37 2010 Antoine Dodson & The Gregory Brothers37 2005 Edward Sharpe and the Magnetic Zeroes37 1999 Someone Still Loves You Boris Yeltsin36 1977 Gloria Estefan & Miami Sound Machine
#SXMusicData
More fun with MusicBrainzArtist relationship data
#SXMusicData
We can build an Oracle of Kevin Bacon for music
Woah!
#SXMusicData
How many steps from Eric Clapton to Justin Bieber?
Eric Clapton
Phil Collins
Usher Justin BieberPerformed
on trackI wish it
would Rainwith Phil Collins
Composed You’ll be in my Heart
performed by Usher
Who performed
on First Dance with
Justin Bieber
1 2 3
#SXMusicData
Six Degrees of Black Sabbath
http://labs.echonest.com/SixDegrees/?start=eric+clapton&end=justin+bieber
#SXMusicData
The Million Song Data Set
• Core Data: Echo Nest analysis for a million songs
• Complimentary Data
• Second Hand Songs - 20K cover songs
• MusixMatch - 237K bucket-of-words lyric sets
• Last.fm tags - song level tags for 500K tracks. plus 57 million sim. track pairs
• Echo Nest Taste profile subset - 1M users, 48M user/song/play count triples
http://labrosa.ee.columbia.edu/millionsong/
#SXMusicData
Data mining listener data
Who are the most passionate fans?
Dubstep Fan Metal heads
#SXMusicData
Passion Index
High plays per listener
#SXMusicData
Passion Index
Low plays per listener
#SXMusicData
Who has the most passionate fans?
#SXMusicData
Who has the most passionate fans?
Average 115 plays per fan
#SXMusicData
Passion Index
Metal heads are passionate
music fans
passionate
#SXMusicData
Passion Index
Who has the least passionate fans?
#SXMusicData
Passion Index Who has the least passionate fans?
Average only 5 plays per ‘fan’
#SXMusicData
Better RecommendationsPeople who listen to the Beatles also listen to:
That’s pretty boring
#SXMusicData
Better RecommendationsPeople who bought Sgt Pepper’s also bought:
That’s pretty irrelevant
If you like The Beatles you might like
Going beyond user data
• Top ‘terms’ for an artist
• Artist popularity
• Artist familiarity
• Artist relations
• Years of activity
• Fan Passion
Beatles near neighbor graph tuned for discovery
The Beatles
Bob Dylan Emitt Rhodes
The Beau Brummels The Hollies
Pink Floyd Bee Gees
The Who
Elvis Presley
The Byrds
Harry Nilsson
Badfinger
Todd Rundgren
Duncan Browne
The Left Banke
The Zombies
The Lovin' Spoonful
Buffalo Springfield
The Grass Roots Moby Grape
If you like The Beatles you might like
The Beatles
Bob Dylan Emitt Rhodes
The Beau Brummels The Hollies
Pink Floyd Bee Gees
The Who
Elvis Presley
The Byrds
Harry Nilsson
Badfinger
Todd Rundgren
Duncan Browne
The Left Banke
The Zombies
The Lovin' Spoonful
Buffalo Springfield
The Grass Roots Moby Grape
If you like The Beatles you might like
The Beatles
Bob Dylan Emitt Rhodes
The Beau Brummels The Hollies
Pink Floyd Bee Gees
The Who
Elvis Presley
The Byrds
Harry Nilsson
Badfinger
Todd Rundgren
Duncan Browne
The Left Banke
The Zombies
The Lovin' Spoonful
Buffalo Springfield
The Grass Roots Moby Grape
#SXMusicData
More fun with artist similarity graphs
http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.html
#SXMusicData
Finding a path from the smooth jazz of Kenny G
To the brutal death metal
of Nile
#SXMusicData
Kenny G
Amy Lee John Tesh Kim Waters David Sanborn David Foster Gerald Albright Dave Koz George Howard James Vargas
Jan Hammer Jay Graydon Michael Kamen Stephen Bishop
Jennifer Warnes Dan Hill Andrew Gold Art Garfunkel Christopher Cross Dan Fogelberg Barry Manilow
Bread Ambrosia Benny Mardones Little River Band Michael McDonald
Peter Cetera
Kenny Loggins
Captain & Tennille
James IngramPaul Carrack Hall & Oates
Culture Club The Doobie Brothers The Righteous Brothers Phil Collins Billy Ocean Daryl Hall Rick Springfield
Rod Stewart
Dire Straits
Labi Siffre Richard Marx Elton John Peter Gabriel Michael Bolton Sting
Bee Gees Three Dog NightFleetwood Mac Paul McCartney Joe Cocker Neil Sedaka Long John Baldry
Christine McVieThe Rolling Stones Lindsey Buckingham Chicken Shack Peter Frampton Dance Yourself to Death Stevie Nicks Bob Welch
Bob SegerMott The Hoople Pat Travers Band Eric Clapton
J.J. CaleGary Moore Jimmy Page Alvin Lee Jimi Hendrix John Mayall & The Bluesbreakers Michael Burks Mark Knopfler
Jefferson Airplane The Doors Haale Led Zeppelin John Frusciante Janis Joplin Ace Frehley Eric Gales
Dave Navarro Beck Chad Smith The Mars Volta Slash Ataxia
The Prize Fighter Inferno Sparta Free Moral Agents The Dillinger Escape Plan The Stiletto Formal Omar A. Rodriguez-Lopez Schoolyard Heroes Heroes & Zeros
Benea Reach Car Bomb Spitfire Ephel Duath Meshuggah The Number Twelve Looks Like You HORSE the band Look What I Did Fear Before the March of Flames
Synthetic Breed Zero Hour TesseracT Gojira Carcass
Textures
Periphery Forbidden
Septic FleshStrapping Young Lad Lamb of God Nile Mastodon
Finding a path from the smooth jazz of Kenny
To the brutal death metal of Nile
#SXMusicData
Kenny G
Amy Lee John Tesh Kim Waters David Sanborn David Foster Gerald Albright Dave Koz George Howard James Vargas
Jan Hammer Jay Graydon Michael Kamen Stephen Bishop
Jennifer Warnes Dan Hill Andrew Gold Art Garfunkel Christopher Cross Dan Fogelberg Barry Manilow
Bread Ambrosia Benny Mardones Little River Band Michael McDonald
Peter Cetera
Kenny Loggins
Captain & Tennille
James IngramPaul Carrack Hall & Oates
Culture Club The Doobie Brothers The Righteous Brothers Phil Collins Billy Ocean Daryl Hall Rick Springfield
Rod Stewart
Dire Straits
Labi Siffre Richard Marx Elton John Peter Gabriel Michael Bolton Sting
Bee Gees Three Dog NightFleetwood Mac Paul McCartney Joe Cocker Neil Sedaka Long John Baldry
Christine McVieThe Rolling Stones Lindsey Buckingham Chicken Shack Peter Frampton Dance Yourself to Death Stevie Nicks Bob Welch
Bob SegerMott The Hoople Pat Travers Band Eric Clapton
J.J. CaleGary Moore Jimmy Page Alvin Lee Jimi Hendrix John Mayall & The Bluesbreakers Michael Burks Mark Knopfler
Jefferson Airplane The Doors Haale Led Zeppelin John Frusciante Janis Joplin Ace Frehley Eric Gales
Dave Navarro Beck Chad Smith The Mars Volta Slash Ataxia
The Prize Fighter Inferno Sparta Free Moral Agents The Dillinger Escape Plan The Stiletto Formal Omar A. Rodriguez-Lopez Schoolyard Heroes Heroes & Zeros
Benea Reach Car Bomb Spitfire Ephel Duath Meshuggah The Number Twelve Looks Like You HORSE the band Look What I Did Fear Before the March of Flames
Synthetic Breed Zero Hour TesseracT Gojira Carcass
Textures
Periphery Forbidden
Septic FleshStrapping Young Lad Lamb of God Nile Mastodon
#SXMusicData
Kenny G
Amy Lee John Tesh Kim Waters David Sanborn David Foster Gerald Albright Dave Koz George Howard James Vargas
Jan Hammer Jay Graydon Michael Kamen Stephen Bishop
Jennifer Warnes Dan Hill Andrew Gold Art Garfunkel Christopher Cross Dan Fogelberg Barry Manilow
Bread Ambrosia Benny Mardones Little River Band Michael McDonald
Peter Cetera
Kenny Loggins
Captain & Tennille
James IngramPaul Carrack Hall & Oates
Culture Club The Doobie Brothers The Righteous Brothers Phil Collins Billy Ocean Daryl Hall Rick Springfield
Rod Stewart
Dire Straits
Labi Siffre Richard Marx Elton John Peter Gabriel Michael Bolton Sting
Bee Gees Three Dog NightFleetwood Mac Paul McCartney Joe Cocker Neil Sedaka Long John Baldry
Christine McVieThe Rolling Stones Lindsey Buckingham Chicken Shack Peter Frampton Dance Yourself to Death Stevie Nicks Bob Welch
Bob SegerMott The Hoople Pat Travers Band Eric Clapton
J.J. CaleGary Moore Jimmy Page Alvin Lee Jimi Hendrix John Mayall & The Bluesbreakers Michael Burks Mark Knopfler
Jefferson Airplane The Doors Haale Led Zeppelin John Frusciante Janis Joplin Ace Frehley Eric Gales
Dave Navarro Beck Chad Smith The Mars Volta Slash Ataxia
The Prize Fighter Inferno Sparta Free Moral Agents The Dillinger Escape Plan The Stiletto Formal Omar A. Rodriguez-Lopez Schoolyard Heroes Heroes & Zeros
Benea Reach Car Bomb Spitfire Ephel Duath Meshuggah The Number Twelve Looks Like You HORSE the band Look What I Did Fear Before the March of Flames
Synthetic Breed Zero Hour TesseracT Gojira Carcass
Textures
Periphery Forbidden
Septic FleshStrapping Young Lad Lamb of God Nile Mastodon
#SXMusicData
Boil the Frog
#SXMusicData
Music is data too
Sections
Bars
Beats
Segments
The Beat Hierarchy
Tatums
#SXMusicData http://labs.echonest.com/click/
#SXMusicData
#SXMusicData
#SXMusicData
#SXMusicData
#SXMusicData
Fun with Loudness data
#SXMusicData
Why does today’s music sound like crap?
http://musicmachinery.com/2009/03/23/the-loudness-war/
Dave Brubeck’s Take Five - released in 1959
20 dB of dynamic range
#SXMusicData http://musicmachinery.com/2009/03/23/the-loudness-war/
Metallica’s Cyanide - released in 2008
Why does today’s music sound like crap?
3 dB of dynamic range
#SXMusicData http://musicmachinery.com/2009/03/23/the-loudness-war/
The Loudness wars
#SXMusicData
Fun with Loudness Info
http://musicmachinery.com/2011/09/18/looking-for-the-slow-build/
Looking for the slow build
#SXMusicData
Fun with Loudness InfoLooking for the slow build
#SXMusicData
Fun with Loudness InfoLooking for the slow build
#SXMusicData
Fun with Loudness InfoLooking for the slow build
#SXMusicData
Timbre and Pitch Data
#SXMusicData
Fun with Timbre info
http://static.echonest.com/BohemianRhapsichord/index.html
Bohemian Rhapsichord
#SXMusicData
Fun with Pitch info
MIDEM Music Machine
http://static.echonest.com/MidemMusicMachine/index.html
Data-driven music manipulations
Turning music into silly putty
#SXMusicData
Data-driven music remixing
Once you have precise information on all of the musical events in a song, you can use this info to algorithmically
remix the song
#SXMusicData
Tristan’s The SwingerMakes any song swing
#SXMusicData
Tristan’s WaltzifyTurns a 4/4 song into a Waltz
#SXMusicData
I’ve got 10 million songs in my pocket.Now what?
Tools and techniques that help us better understand the world of music
will be increasingly important
#SXMusicData
A cautionary noteTaste is irrational
Music I Like
Music You Like
Music I UsedTo Like
Get this t-shirt at dieselsweeties.com
#SXMusicData
http://musicmachinery.com/
More like this
#SXMusicData
http://blog.last.fm/category/Trends-and-Data/
More like this