한류세계화와 우리농식품의세계화전략 · 한류세계화와 우리농식품의세계화전략 홍석경(서울대언론정보학과교수) [email protected] 2013년9월4일,
Music Information Retrieval Information Universe Seongmin Lim [email protected] Dept. of Industrial...
-
Upload
baldwin-merritt -
Category
Documents
-
view
213 -
download
0
Transcript of Music Information Retrieval Information Universe Seongmin Lim [email protected] Dept. of Industrial...
![Page 1: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/1.jpg)
Music Information RetrievalInformation Universe
Seongmin Lim
Dept. of Industrial Engineering
Seoul National University
![Page 2: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/2.jpg)
2
contents
![Page 3: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/3.jpg)
3
Brief history of MIR and state of research
Cross media retrieval supporting Natural language queries like mood, melody information.- Contain semantic information taken from community data bases- “A Music Search Engine Built upon Audio-based and Web-based
Similarity Measures”
Query by Example- You have an example query having the same representation in
the database.- For music search: humming, recorded by cell phones,
microphones- “Music Structure Based Vector Space Retrieval”
![Page 4: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/4.jpg)
4
Stages of First Paper
“A Music Search Engine Built upon Audio-based and Web-based Similarity Measures”
![Page 5: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/5.jpg)
5
Stage 1: Preprocessing the Collection
Using information in the ID3 tag- Artist- Album- Title
all duplicates of tracks are excluded to avoid redundancies
Live or instrumentals of the same song removed
![Page 6: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/6.jpg)
6
Stage 2: Web based features addition
Search on the web for- “artist”music- “artist”“album”music review- “artist”“title”music review –lyrics
![Page 7: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/7.jpg)
7
Stage 2: Web based features addition (2)
Every term is weighted according to the term frequency ×inverse document frequency (tf×idf) function. w(t,m) of a term t for music piece m. N is the total number of documents.
![Page 8: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/8.jpg)
8
Stage 3: Audio Based Similarity measures
For each audio track, Mel Frequency Cepstral Coefficients (MFCCs) are computed on short-time audio segments (called frames)
each song is represented as a Gaussian Mixture Model (GMM) of the distribution of MFCCs
Kullback-Leibler divergence can be calculated on the means and covariance matrices
A rank list of similar tracks is found based on this measure corresponding to each track
![Page 9: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/9.jpg)
9
GMM(Gaussian Mixture Model)
a probabilistic model for representing the presence of sub-populations within an overall population
the mixture distribution that represents the probability distribution of observations in the overall population
![Page 10: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/10.jpg)
10
Stage 4: Dimensionality Reduction
chi square test to distinguish the most similar terms using audio similarities
A is the number of documents in s which contain t B is the number of documents in d which contain t C is the number of documents in s without t D is the number of documents in d without t N is the total number of examined documents
![Page 11: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/11.jpg)
11
Stage 5: Vector Adaptation
Smoothing for tracks where no related information
![Page 12: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/12.jpg)
12
Querying the Music Search Engine
method to find those tracks that are most similar to a natural language query
extend queries to the music search engine by the word music and send them to Google
Query vector is constructed in the feature space from the top 10 pages retrieved
Euclidean distances are calculated from the collection tracks and a relevance ranking is got
![Page 13: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/13.jpg)
13
Evaluating the System
to evaluate on “real-world” queries, a source for phrases which are used by people to describe music is needed
Tags provided by AudioScrobbler groundtruth is used
227 tags are used
as test queries
![Page 14: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/14.jpg)
14
Goal of the evaluation
Goals- Effect of dimensionality on the feature space- Retrieving relevant information - Effect of re weighting of the term vectors- Effect of query expansion
Metrics used : precision values for various recall levels
![Page 15: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/15.jpg)
15
Performance Evaluation -I
audio-based term selection has a very positive impact on the retrieval
setting 2/50 yields best results
![Page 16: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/16.jpg)
16
Performance Evaluation -II
Effect of re weighting using various re weighting techniques
the impact of audiobased vector re-weighting is only marginal
![Page 17: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/17.jpg)
17
Performance Evaluation –III (other metrics)
![Page 18: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/18.jpg)
18
Examples
![Page 19: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/19.jpg)
19
System design of Second paper
“Music structure based vector space retrieval”
![Page 20: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/20.jpg)
20
Music Layout : The Pyramid
![Page 21: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/21.jpg)
21
Stage 1: MUSIC INFORMATION MODELING
Music Segmentation by smallest note length
Cord modeling
Music region content modeling
![Page 22: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/22.jpg)
22
Stage 2: MUSIC INDEXING AND RETRIEVAL
Harmony Event and Acoustic Event- each song’s cord and music region information is represented as
a Gaussian Mixture Model (GMM) of the distribution of MFCCs
n-gram Vector- The harmony and acoustic decoders serve as the tokenizers for
music signal- an event is represented in a text-like format
![Page 23: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/23.jpg)
23
Stage 3: Music information retrieval
![Page 24: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/24.jpg)
24
Summary
Natural query vs. query by example Information from web and audio Audio frame segmentation KL divergence vs. vector space modeling Analyzing audio features Data itself vs. metadata domain knowledge of music
![Page 25: Music Information Retrieval Information Universe Seongmin Lim hovern@snu.ac.kr Dept. of Industrial Engineering Seoul National University.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649f0d5503460f94c21a81/html5/thumbnails/25.jpg)
End of Document
Seongmin Lim
Dept. of Industrial Engineering
Seoul National University