Music genre classification using traditional and ...
Transcript of Music genre classification using traditional and ...
Music genre classificationusing traditional and relational approaches
Jorge Valverde-Rebaza, Aurea Soriano, Lilian Berton,Maria Cristina F. Oliveira and Alneu de Andrade Lopes
Laboratory of Computational Intelligence (LABIC)
Laboratory of Visualization, Imaging and Computer Graphics (VICG)
University of Sao Paulo (USP)
Brazil
October 2014
Outline
1 Introduction
2 Method
3 Experimental Evaluation
4 Conclusion
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 2 / 20
Outline
1 Introduction
2 MethodMusic FeaturesGraph constructionGraph construction
3 Experimental EvaluationDatasetsExperimental setupResults
4 Conclusion
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 3 / 20
Introduction
Many music collections, typically very large
Manual music classification: a non-expert person can identify the genre of a music with
72% accuracy after hearing 3 seconds of it [Perrot and Gjerdigen, 1999].Classifying a large collection demands time, effort and expertise
Automatic music classification: Solutions achieve high accuracy (ranging from 63 to
84%) [Shao et al., 2004, Pampalk et al., 2005, Scaringella and Mlynek, 2005,
Yaslan and Cataltepe, 2009, Poria et al., 2013]require music files with the same sizeapplied on controlled environments, i.e. without considering class imbalanceemployed traditional classifiers
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 4 / 20
Introduction
Many music collections, typically very large
Manual music classification: a non-expert person can identify the genre of a music with
72% accuracy after hearing 3 seconds of it [Perrot and Gjerdigen, 1999].Classifying a large collection demands time, effort and expertise
Automatic music classification: Solutions achieve high accuracy (ranging from 63 to
84%) [Shao et al., 2004, Pampalk et al., 2005, Scaringella and Mlynek, 2005,
Yaslan and Cataltepe, 2009, Poria et al., 2013]require music files with the same sizeapplied on controlled environments, i.e. without considering class imbalanceemployed traditional classifiers
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 4 / 20
Introduction
Many music collections, typically very large
Manual music classification: a non-expert person can identify the genre of a music with
72% accuracy after hearing 3 seconds of it [Perrot and Gjerdigen, 1999].Classifying a large collection demands time, effort and expertise
Automatic music classification: Solutions achieve high accuracy (ranging from 63 to
84%) [Shao et al., 2004, Pampalk et al., 2005, Scaringella and Mlynek, 2005,
Yaslan and Cataltepe, 2009, Poria et al., 2013]require music files with the same sizeapplied on controlled environments, i.e. without considering class imbalanceemployed traditional classifiers
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 4 / 20
Outline
1 Introduction
2 MethodMusic FeaturesGraph constructionGraph construction
3 Experimental EvaluationDatasetsExperimental setupResults
4 Conclusion
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 5 / 20
Music feature extraction
Feature extracted from MIDI description
(12)Histograms
Distance: DTW
(8) Moments(4) Melody and (4) Rhythm: themean, standard deviation, entropy
and uniformityDistance: Euclidean
(248)StructureIdentification of patterns from
chord sequences. Generatesvectors of different sizes.
Distance: DTW
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20
Music feature extraction
Feature extracted from MIDI description
(12)Histograms
Distance: DTW
(8) Moments(4) Melody and (4) Rhythm: themean, standard deviation, entropy
and uniformityDistance: Euclidean
(248)StructureIdentification of patterns from
chord sequences. Generatesvectors of different sizes.
Distance: DTW
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20
Music feature extraction
Feature extracted from MIDI description
(12)Histograms
Distance: DTW
(8) Moments(4) Melody and (4) Rhythm: themean, standard deviation, entropy
and uniformityDistance: Euclidean
(248)StructureIdentification of patterns from
chord sequences. Generatesvectors of different sizes.
Distance: DTW
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20
Music feature extraction
Feature extracted from MIDI description
(12)Histograms
Distance: DTW
(8) Moments(4) Melody and (4) Rhythm: themean, standard deviation, entropy
and uniformityDistance: Euclidean
(248)StructureIdentification of patterns from
chord sequences. Generatesvectors of different sizes.
Distance: DTW
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20
Music feature extraction
Feature extracted from MIDI description
(12)Histograms
Distance: DTW
(8) Moments(4) Melody and (4) Rhythm: themean, standard deviation, entropy
and uniformityDistance: Euclidean
(248)StructureIdentification of patterns from
chord sequences. Generatesvectors of different sizes.
Distance: DTW
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 6 / 20
Graph construction
Graph construction techniques
kNN
Connect each vertex onlyto its k nearest neighbors
mutual-kNN
Two vertices areconnected only if theneighborhood pertinencecondition is met by both
regular-kNN
All the vertices have thesame degree
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20
Graph construction
Graph construction techniques
kNN
Connect each vertex onlyto its k nearest neighbors
mutual-kNN
Two vertices areconnected only if theneighborhood pertinencecondition is met by both
regular-kNN
All the vertices have thesame degree
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20
Graph construction
Graph construction techniques
kNN
Connect each vertex onlyto its k nearest neighbors
mutual-kNN
Two vertices areconnected only if theneighborhood pertinencecondition is met by both
regular-kNN
All the vertices have thesame degree
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20
Graph construction
Graph construction techniques
kNN
Connect each vertex onlyto its k nearest neighbors
mutual-kNN
Two vertices areconnected only if theneighborhood pertinencecondition is met by both
regular-kNN
All the vertices have thesame degree
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20
Graph construction
Graph construction techniques
kNN
Connect each vertex onlyto its k nearest neighbors
mutual-kNN
Two vertices areconnected only if theneighborhood pertinencecondition is met by both
regular-kNN
All the vertices have thesame degree
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 7 / 20
Classifiers
Classifiers
Traditional
Map input data to acategory
Decision trees, naıveBayes, neural networks,support vector machine,etc
Relational
Map relational input data to acategory
Weighted vote relational neighbor,network-only Bayes, probabilisticrelational neighbor, network-onlylink based [Macskassy, 2007]
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20
Classifiers
Classifiers
Traditional
Map input data to acategory
Decision trees, naıveBayes, neural networks,support vector machine,etc
Relational
Map relational input data to acategory
Weighted vote relational neighbor,network-only Bayes, probabilisticrelational neighbor, network-onlylink based [Macskassy, 2007]
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20
Classifiers
Classifiers
Traditional
Map input data to acategory
Decision trees, naıveBayes, neural networks,support vector machine,etc
Relational
Map relational input data to acategory
Weighted vote relational neighbor,network-only Bayes, probabilisticrelational neighbor, network-onlylink based [Macskassy, 2007]
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20
Classifiers
Classifiers
Traditional
Map input data to acategory
Decision trees, naıveBayes, neural networks,support vector machine,etc
Relational
Map relational input data to acategory
Weighted vote relational neighbor,network-only Bayes, probabilisticrelational neighbor, network-onlylink based [Macskassy, 2007]
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20
Classifiers
Classifiers
Traditional
Map input data to acategory
Decision trees, naıveBayes, neural networks,support vector machine,etc
Relational
Map relational input data to acategory
Weighted vote relational neighbor,network-only Bayes, probabilisticrelational neighbor, network-onlylink based [Macskassy, 2007]
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 8 / 20
Outline
1 Introduction
2 MethodMusic FeaturesGraph constructionGraph construction
3 Experimental EvaluationDatasetsExperimental setupResults
4 Conclusion
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 9 / 20
Datasets
Tabela: Music genre distribution for the music collection considered
Genre # Tracks
Classical 31Brazilian Backcountry 243Pop/Rock 550Jazz 95
Total 919
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 10 / 20
Experimental setup
Feature vectors
Histogram Moments Structure
Graph construction (1 ≤ k ≤ 15 )
kNN mutual-kNN regular-kN
Classifiers
Traditional Relational
Decision tree (J48)
Naıve Bayes (NB)
Multilayer perceptron withbackpropagation (MLP)
Support vector machine (SMO)
weighted vote relational neighbor (wvrn)
network-only Bayes (no-Bayes)
probabilistic relational neighbor (prn)
network-only link-based
mode-link (no-lb-mode)count-link (no-lb-count)binary-link (no-lb-binary)class-distribution-link (no-lb-distrib)
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 11 / 20
Experimental setup
Feature vectors
Histogram Moments Structure
Graph construction (1 ≤ k ≤ 15 )
kNN mutual-kNN regular-kN
Classifiers
Traditional Relational
Decision tree (J48)
Naıve Bayes (NB)
Multilayer perceptron withbackpropagation (MLP)
Support vector machine (SMO)
weighted vote relational neighbor (wvrn)
network-only Bayes (no-Bayes)
probabilistic relational neighbor (prn)
network-only link-based
mode-link (no-lb-mode)count-link (no-lb-count)binary-link (no-lb-binary)class-distribution-link (no-lb-distrib)
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 11 / 20
Experimental setup
Feature vectors
Histogram Moments Structure
Graph construction (1 ≤ k ≤ 15 )
kNN mutual-kNN regular-kN
Classifiers
Traditional Relational
Decision tree (J48)
Naıve Bayes (NB)
Multilayer perceptron withbackpropagation (MLP)
Support vector machine (SMO)
weighted vote relational neighbor (wvrn)
network-only Bayes (no-Bayes)
probabilistic relational neighbor (prn)
network-only link-based
mode-link (no-lb-mode)count-link (no-lb-count)binary-link (no-lb-binary)class-distribution-link (no-lb-distrib)
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 11 / 20
Traditional classification results
Tabela: Traditional classifiers performance measured by AUC
J48 NB MLP SMO
Histogram 0.619 0.607 0.665 0.506Moments 0.706 0.750 0.771 0.585Structure 0.738 0.920 0.816 0.724
Average rank 2.667 2.000 1.333 4.000
1 2 3 4
MLPNB J48
SMO
CD
Figura: Post-hoc test results for traditional classifiers performance
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 12 / 20
Traditional classification results
Tabela: Traditional classifiers performance measured by AUC
J48 NB MLP SMO
Histogram 0.619 0.607 0.665 0.506Moments 0.706 0.750 0.771 0.585Structure 0.738 0.920 0.816 0.724
Average rank 2.667 2.000 1.333 4.000
1 2 3 4
MLPNB J48
SMO
CD
Figura: Post-hoc test results for traditional classifiers performance
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 12 / 20
Relational classification results: using kNN
Tabela: Relational classifiers performance evaluated by AUC in kNN networks
no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn
Histogram 0.575 (k=11) 0.723 (k=9) 0.622 (k=2) 0.71 (k=8) 0.712 (k=9) 0.515 (k=1) 0.537 (k=1)
Moments 0.547 (k=5) 0.635 (k=13) 0.575 (k=8) 0.644 (k=7) 0.644 (k=9) 0.563 (k=2) 0.571 (k=3)
Structure 0.834 (k=7) 0.939 (k=14) 0.851 (k=4) 0.945 (k=14) 0.931 (k=15) 0.922 (k=15) 0.903 (k=9)
Average rank 6.333 2.000 4.667 1.667 2.333 5.667 5.333
1 2 3 4 5 6 7
no-lb-distribno-lb-count
wvrnno-lb-binary
prnno-Bayesno-lb-mode
CD
Figura: Post-hoc test results for relational classifiers built on kNN networks
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 13 / 20
Relational classification results: using kNN
Tabela: Relational classifiers performance evaluated by AUC in kNN networks
no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn
Histogram 0.575 (k=11) 0.723 (k=9) 0.622 (k=2) 0.71 (k=8) 0.712 (k=9) 0.515 (k=1) 0.537 (k=1)
Moments 0.547 (k=5) 0.635 (k=13) 0.575 (k=8) 0.644 (k=7) 0.644 (k=9) 0.563 (k=2) 0.571 (k=3)
Structure 0.834 (k=7) 0.939 (k=14) 0.851 (k=4) 0.945 (k=14) 0.931 (k=15) 0.922 (k=15) 0.903 (k=9)
Average rank 6.333 2.000 4.667 1.667 2.333 5.667 5.333
1 2 3 4 5 6 7
no-lb-distribno-lb-count
wvrnno-lb-binary
prnno-Bayesno-lb-mode
CD
Figura: Post-hoc test results for relational classifiers built on kNN networks
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 13 / 20
Relational classification results: using mutual-kNN
Tabela: Relational classifiers performance evaluated by AUC in mutual-kNNnetworks
no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn
Histogram 0.621 (k=1) 0.712 (k=10) 0.626 (k=2) 0.735 (k=12) 0.727 (k=13) 0.555 (k=1) 0.571 (k=1)
Moments 0.570 (k=1) 0.657 (k=14) 0.588 (k=2) 0.633 (k=15) 0.630 (k=14) 0.578 (k=1) 0.574 (k=1)
Structure 0.864 (k=1) 0.955 (k=14) 0.818 (k=2) 0.963 (k=15) 0.964 (k=14) 0.913 (k=6) 0.902 (k=2)
Average rank 6.000 2.333 5.000 1.667 2.000 5.333 5.667
1 2 3 4 5 6 7
no-lb-distribwvrn
no-lb-countno-lb-binary
no-Bayesprnno-lb-mode
CD
Figura: Post-hoc test results for relational classifiers built on mutual-kNNnetworks
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 14 / 20
Relational classification results: using mutual-kNN
Tabela: Relational classifiers performance evaluated by AUC in mutual-kNNnetworks
no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn
Histogram 0.621 (k=1) 0.712 (k=10) 0.626 (k=2) 0.735 (k=12) 0.727 (k=13) 0.555 (k=1) 0.571 (k=1)
Moments 0.570 (k=1) 0.657 (k=14) 0.588 (k=2) 0.633 (k=15) 0.630 (k=14) 0.578 (k=1) 0.574 (k=1)
Structure 0.864 (k=1) 0.955 (k=14) 0.818 (k=2) 0.963 (k=15) 0.964 (k=14) 0.913 (k=6) 0.902 (k=2)
Average rank 6.000 2.333 5.000 1.667 2.000 5.333 5.667
1 2 3 4 5 6 7
no-lb-distribwvrn
no-lb-countno-lb-binary
no-Bayesprnno-lb-mode
CD
Figura: Post-hoc test results for relational classifiers built on mutual-kNNnetworks
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 14 / 20
Relational classification results: using regular-kNN
Tabela: Relational classifiers performance evaluated by AUC in regular-kNNnetworks
no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn
Histogram 0.608 (k=1) 0.724 (k=8) 0.611 (k=2) 0.737 (k=12) 0.730 (k=12) 0.544 (k=1) 0.553 (k=1)
Moments 0.569 (k=1) 0.652 (k=13) 0.571 (k=1) 0.620 (k=8) 0.625 (k=6) 0.560 (k=1) 0.565 (k=1)
Structure 0.904 (k=1) 0.948 (k=11) 0.82 (k=1) 0.967 (k=15) 0.966 (k=15) 0.923 (k=3) 0.904 (k=2)
Average rank 5.333 2.333 5.000 1.667 2.000 6.000 5.667
1 2 3 4 5 6 7
no-lb-distribwvrn
no-lb-countno-lb-binary
no-lb-modeprnno-Bayes
CD
Figura: Post-hoc test results for relational classifiers built on regular-kNNnetworks
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 15 / 20
Relational classification results: using regular-kNN
Tabela: Relational classifiers performance evaluated by AUC in regular-kNNnetworks
no-lb-mode no-lb-count no-lb-binary no-lb-distrib wvrn no-Bayes prn
Histogram 0.608 (k=1) 0.724 (k=8) 0.611 (k=2) 0.737 (k=12) 0.730 (k=12) 0.544 (k=1) 0.553 (k=1)
Moments 0.569 (k=1) 0.652 (k=13) 0.571 (k=1) 0.620 (k=8) 0.625 (k=6) 0.560 (k=1) 0.565 (k=1)
Structure 0.904 (k=1) 0.948 (k=11) 0.82 (k=1) 0.967 (k=15) 0.966 (k=15) 0.923 (k=3) 0.904 (k=2)
Average rank 5.333 2.333 5.000 1.667 2.000 6.000 5.667
1 2 3 4 5 6 7
no-lb-distribwvrn
no-lb-countno-lb-binary
no-lb-modeprnno-Bayes
CD
Figura: Post-hoc test results for relational classifiers built on regular-kNNnetworks
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 15 / 20
Best performances
1 2 3
regular-kNNmutual-kNN
kNN
CD
Figura: Post-hoc test results for identifying the influence of network constructiontechniques in relational classifiers
1 2 3 4
no-lb-distribwvrn MLP
NB
CD
Figura: Post-hoc test results for the comparison between the best traditional andrelational classifiers
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 16 / 20
Best performances
1 2 3
regular-kNNmutual-kNN
kNN
CD
Figura: Post-hoc test results for identifying the influence of network constructiontechniques in relational classifiers
1 2 3 4
no-lb-distribwvrn MLP
NB
CD
Figura: Post-hoc test results for the comparison between the best traditional andrelational classifiers
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 16 / 20
Outline
1 Introduction
2 MethodMusic FeaturesGraph constructionGraph construction
3 Experimental EvaluationDatasetsExperimental setupResults
4 Conclusion
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 17 / 20
Conclusion
We introduce a novel feature vector (called Structural) whichcaptures information encoded in the MIDI files
We evaluated traditional and relational classifiers on a musiccollection with an imbalanced distribution of four music genres
The Structural features resulted in improved performance of bothtraditional and relational classifiers
The regular-kNN networks provided the relational model mostsuitable to improve the performance of relational classifiers
Relational classifiers perform better than traditional classifiers onmusic genre classification
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20
Conclusion
We introduce a novel feature vector (called Structural) whichcaptures information encoded in the MIDI files
We evaluated traditional and relational classifiers on a musiccollection with an imbalanced distribution of four music genres
The Structural features resulted in improved performance of bothtraditional and relational classifiers
The regular-kNN networks provided the relational model mostsuitable to improve the performance of relational classifiers
Relational classifiers perform better than traditional classifiers onmusic genre classification
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20
Conclusion
We introduce a novel feature vector (called Structural) whichcaptures information encoded in the MIDI files
We evaluated traditional and relational classifiers on a musiccollection with an imbalanced distribution of four music genres
The Structural features resulted in improved performance of bothtraditional and relational classifiers
The regular-kNN networks provided the relational model mostsuitable to improve the performance of relational classifiers
Relational classifiers perform better than traditional classifiers onmusic genre classification
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20
Conclusion
We introduce a novel feature vector (called Structural) whichcaptures information encoded in the MIDI files
We evaluated traditional and relational classifiers on a musiccollection with an imbalanced distribution of four music genres
The Structural features resulted in improved performance of bothtraditional and relational classifiers
The regular-kNN networks provided the relational model mostsuitable to improve the performance of relational classifiers
Relational classifiers perform better than traditional classifiers onmusic genre classification
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20
Conclusion
We introduce a novel feature vector (called Structural) whichcaptures information encoded in the MIDI files
We evaluated traditional and relational classifiers on a musiccollection with an imbalanced distribution of four music genres
The Structural features resulted in improved performance of bothtraditional and relational classifiers
The regular-kNN networks provided the relational model mostsuitable to improve the performance of relational classifiers
Relational classifiers perform better than traditional classifiers onmusic genre classification
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 18 / 20
References
Macskassy, S. A. (2007).
Improving learning in networked data by combiningexplicit and mined links.In AAAI, pages 590–595.
Pampalk, E., Flexer, A., Widmer, G., et al. (2005).
Improvements of audio-based music similarity andgenre classificaton.In ISMIR, volume 5, pages 634–637.
Perrot, D. and Gjerdigen, R. (1999).
Scanning the dial: An exploration of factors in theidentification of musical style.In Proceedings of the 1999 Society for MusicPerception and Cognition, page 88.
Poria, S., Gelbukh, A., Hussain, A., Bandyopadhyay,
S., and Howard, N. (2013).Music genre classification: A semi-supervisedapproach.
In Pattern Recognition, pages 254–263.
Scaringella, N. and Mlynek, D. (2005).
A mixture of support vector machines for audioclassification.IEEE MIREX, London.
Shao, X., Xu, C., and Kankanhalli, M. S. (2004).
Unsupervised classification of music genre using hiddenmarkov model.In Multimedia and Expo, 2004. ICME’04. 2004 IEEEInternational Conference on, volume 3, pages2023–2026.
Yaslan, Y. and Cataltepe, Z. (2009).
Audio genre classification with semi-supervised featureensemble learning.In Second International Workshop on MachineLearning and Music.
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 19 / 20
Thank you!
Jorge Valverde-Rebaza
Jorge Valverde-Rebaza et al. Music genre classification using traditional and relational . . . 20 / 20