Music Recommendation

7/31/2019 Music Recommendation

1/113

Music RecommendationListen to the music You like

Recuperao de Informao 1semestre 2011/2012Ricardo Dias, n55444


2/113

Bibliography

Music Recommendation and Discovery: The Long

Tail, Long Fail, and Long Play

scar Celma, Springer 2010, Ch-1-3,5

Recommender SystemsPrem Melville, Vikas Sindhwani

Encyclopedia of Machine Learning, 2010

Handbook of Multimedia for Digital

Entertainment and Arts

Borko Furht (Ed.), Springer 2009


3/113

MOTIVATION & CONTEXT

What is Recommendation? Do we need it? Why?


4/113

Recommendation in our lives


5/113

Restaurants


6/113

Discos and Bars


7/113

Food


8/113

Books


9/113

And Music!


10/113

Do we need Music recommendation?


11/113

Music Consumption Change

Time

Physical Stores Online Stores


12/113

Music RecommendationDigital Era - Portability

Up to ~20 tracks Up to ~40.000 tracks

Time


13/113

Music RecommendationDigital Era Online Services


14/113

Music RecommendationDigital Era Online Services

Amazon

17 Million Songs


15/113

Music Recommendation Problem

Overwhelming number of choices of whichmusic to listen to

Users feel:

Paralyzed

Doubtful

Need to provide personalized filters and

recommendations to easeusers decisions


16/113


17/113

Music RecommendationDigital Era


18/113

Music Characteristics

Different from other types of media

Tracking users preferences can be implicit

Items can be consumed several times

(even repeatedly and continuously)

Instant feedback

Music consumption depends on context

(morning, work, afternoon, etc.)


19/113

Music Recommendation Specificities

Current music recommendation algorithms tryto accurately predict what people will want tolisten

Making accurate predictions about a user could

listento, or buynext, independently of howusefulthe provided recommendations are tothe user


20/113

THE RECOMMENDATION PROBLEM

Formalization, Use Cases, Profile Generation, Recommendation Methods


21/113

Formalization

Recommendation Problem

Prediction problemestimationof the items

likeliness for a given user Recommend a list ofN items assuming that the

system can predict likeliness for yet unrated items


22/113

Prediction Problem

= , , , the set ofUsers

= , , , items that can be

recommend

list of items a userj expressed his interests

Function ,j

predicted likeness of item

,

for the active user , where Usually represented by a rating < , , >


23/113

Recommendation Problem

Find a list of N items that the user willlike the most

The ones with higher, The resultant list should not contain items from

the users interests

=


24/113

Use Cases

Common usages of a recommender system:

1. Find good items

2. Find all good items3. Recommend sequence (e.g. playlist generation)

4. Just browsing

5. Find credible recommender6. Express self

7. Influence others


25/113

General Model

Users and Items

Two types of

recommendations: Top-N predicted items

Top-N predicted

neighbors


26/113

User Profile Generation

Two key elements:

Generation and Maintenance

Exploitation of the profile using arecommendation system


27/113


28/113


29/113


30/113

Recommendation Methods

Standard classification of recommendersystems:

1. Demographic Filtering2. Collaborative Filtering

3. Content-based Filtering

4. Context-based Filtering5. Hybrid Approaches


31/113

Demographic Filtering

Used to identify the kind of users that like acertain item

Classifies user profiles in clusters based on: Personal data (age, gender, marital status, etc.)

Geographic data (city, country)

Psycographic data (interests, lifestyle, etc.)


32/113

Advantages/Limitations

The simplest recommendation method

But

Recommendations are too general Requires effort from the user to generate the

profile


33/113

Collaborative Filtering

Predict user preferences for items by learningpast user-item relationships

CF methods work by build a matrix M with nitems and m users, that contains theinteraction (e.g. ratings, plays, etc.) of the

users with the items.


34/113


The value , represents the rating of the

user for the item


35/113

Collaborative Filtering Approaches

Item-Based Neighborhood

User-Based Neighborhood

Matrix Factorization


36/113


Only users that rated the items and , are

taken into account in the process


37/113


Only users that rated the items and , are

taken into account in the process


38/113


1. Compute the similarity between two items, iandj

1. Example: Adjusted cosine similarity

2. Predict to the target user, u, a value for theactive item, i

; - set of k neighbors of item I,that the user u has rated


39/113

User-Based Neighborhood

Compute the predicted rating value of item i,for the active user u, taking into account those

users that are similar to u

- average rating for user u()- set of k neighbors for user u (the similar ones)


40/113


Useful when the M user-item matrix is sparse

Reduce dimensionality of the original matrix,

generating matrices U and V that approximate theoriginal one

Example: SVDSingular Value Decomposition

Computes matrices and , for a given

number k, such as:

diagonal matrix containing the singular values of M


41/113


After matrix reduction we can calculate thepredicted rating value for item i for a user u

,


42/113


43/113

Content-based Filtering

Uses information describing the items

Process of characterizing item data set can be:

Manual (annotations by domain experts) Automatic (extracting features by analyzing the

content)

Key component: Similarity Function


44/113

Content-based Filtering

Similarity Functions

1. Euclidean

2. Manhattan

3. Chebychev

4. Mahalanobis


45/113

Limitations

Cold-start problem (only to user)

Gray-sheep problem

Novelty (?)

Limitationof extracted automatic features

Subjective (personal opinions) not taken into

account


46/113

Context-based Filtering

Uses context information to describe andcharacterize the items Context Information any information that can be

used to characterize a situation or an entity

Context != Content

Two main techniques: Web mining

Social Tagging


47/113


48/113

Social Tagging

Aims at annotating web content using tags

Tags are freely chosen keywords, not

constrained to a predefined vocabulary Recommender systems can derive social

tagging data to derive item (or user) similarity


49/113

Social Tagging

When users tag items, we get tuples of :< , , >

These triples conform a 3-order matrix(tensor)


50/113

Social Tagging

Two approaches to compute item (and user)similarity:

1. Unfold the 3-order tensor in threebidimensionalmatrices (user- tag, item-tag and user-itemmatrices)

2. Directly use the 3-order tensor


51/113

Unfolding the 3-order tensor

User-Tag (U matrix) - ,contains the number

of times user i applied the tagj

Item-Tag (I matrix) - , contains the numberof times an item i has been tagged with tag j

User-Item (R binary matrix) - , denotes

whether the user i has tagged the itemj


52/113

Unfolding the 3-order tensor

Item similarity (using I) or user similarity (usingU or I), can be computed using:

Cosine-based distance Dimensionality reduction techniques(SVD, NMF)

Then recommendations can be made by using:

R user-item matrix or, User profile obtained from U or I


53/113

Using the 3-order tensor

The available techniques are (high-order)extensions ofSVD and NMF

HOSVD is a higher order generalization of matrixSVD for tensors,

Non-negative Tensor Factorization (NTF) is ageneralization of NMF


54/113

Limitations

Coverage

Problems with tags:

Polysemy Synonymy

Usefulness of personal tags

Sparsity Attacks / Vandalism


55/113

Hybrid Approaches

Goal

Achieve better recommendations by combining

some of the previous approaches Methods:

Weighted

Switching Mixed

Cascade


56/113

Factors Affecting Recommendation

Novelty and Serendipity

Explainability (transparency)

Cold Start Problem Data Sparsity and High Dimensionality

Coverage


57/113

Factors Affecting Recommendation

Trust

Attacks

Temporal Effects Understanding the Users


58/113

MUSIC RECOMMENDATION

Use cases, User and Item Profiles Representation, Recommendation Examples


59/113

Use Cases

Main task of a music recommendation

system:

Propose interesting music, consisting of a mix ofknown and unknown artists, as well as theavailable tracks, given a user profile


60/113

Use Cases

Artist Recommendation

Playlist Generation

Shuffle, Random Playlists Personalized Playlists

Neighbor Recommendation


61/113

How about other use cases?


62/113

User Profile Representation

Extend user profile with music relatedinformation

Has not been largely investigated

Useful to:

Improve music recommendation Share with others your preferences


63/113

Type of Listeners

Each type of listener needs different type ofrecommendations

User Profile Representation


64/113

User Profile RepresentationProposals

Most relevant proposals are:

User modeling for Information Retrieval (UMIRL)

MPEG-7 standard Friend of a Friend (FOAF) initiative

User Modeling for Information


65/113

User Modeling for InformationRetrieval

Allows one to describe perceptual andqualitative features of the music


66/113

MPEG-7 User Preferences

User preferences in MPEG-7 includes:

Content filtering

Searching and browsingpreferences

Usage history

FOAF: User Profiling in the


67/113

FOAF: User Profiling in theSemantic Web

Provides conventions and a language to tella machine the type of things a user says about

herself in her homepage


68/113

Item Profile Representation

Music items:

Artists

Songs


69/113

Music Information Plane


70/113

Music Information Plane

Music knowledge management categories:

Editorial Metadata

Cultural Metadata Acoustic Metadata


71/113

Editorial Metadata


72/113


73/113

Acoustic Metadata


74/113

Music Description Facets

Low-level Timbre Descriptors

Spectral Centroid/Flateness/Skewness, MFCCs, etc.

Instrumentation

Rhythm

Harmony

Structure

Intensity

Genre

Mood

Recommendation Methods


75/113

Recommendation Methods(examples and specificities)

Collaborative Filtering (CF)

Explicit/Implicit Feedback

Content-Based Filtering Context-Based Filtering

Hybrid Methods


76/113


CF makes use of the editorial and culturalinformation

Explicit feedback based on ratings about songs /artists

Implicit feedback tracking user listening habits


77/113

CF with Explicit Feedback

Examples:

Ringo 1st music recommender based on CF andexplicit feedback

Racofibased on CF and a set of logic rules basedon Horn clauses

Indiscover

Slope One CF method


78/113

CF with Implicit Feedback

Main Drawbacks:

The value that a user assigns to an item is notalways in a predefined range (e.g. from 1..5 or likeit/hate it)

Cannot gather negativefeedback

Recommendations usually performed at artist

level, but listening habits are recorded at songlevelAggregation

d l


79/113

Content-Based Filtering

Uses content extracted from music to providerecommendations

Compute similarity among songs, in order torecommend music to the user

Two ways to describe audio content:

Manually

Automatically

Manually Audio Content


80/113

Manually Audio ContentDescription

Very time consuming

Scalability problems

But Annotations can be more accurate than automatic

Example: Pandora

Analysts annotate 400 parameters per song, usinga ten point scale per attribute

~ 15.000 songs analyzed per month

Automatic Audio Content


81/113

Automatic Audio ContentDescription

Early work on audio similarity is based on low-level descriptors, such as Mel Frequency

Cepstral Coefficients ( MFCC) Foote proposed a music indexing system based on

MFCC histograms

Audio features are usually aggregated together

using mean and variance, or modeling it as aGaussian Mixture Model (GMM)

Automatic Audio Content


82/113

Automatic Audio ContentDescription

Analyses audio signal and automaticallyextracts a set of features: Tzanetakis extracted a set of features representing

the spectrum, rhythm and harmony (chordstructure); then merged into a single vector, andwere used to determine song similarity

Cataltepe et al. presented a music

recommendation system based on audiosimilarity, where users listening history is takeninto account

C B d Fil i T h i


83/113

Context-Based Filtering Techniques

Uses cultural information to compute artist orsong similarity

Mainly based on web mining techniques, ormining data from collaborative tagging

C B d Fil i T h i


84/113

Context-Based Filtering Techniques

Example:

M3 (Music for My Mood), uses context (season,month, day of the week, weather, temperature)and Case-based Reasoning to recommend music

H b id M th d


85/113

Hybrid Methods

Allows a system to minimize the issues that asolely method can have

How cascade approach works: One technique is applied first, obtaining a ranked list of

items. Then, a second technique refines or re-rank theresults obtained in the first step

H b id M th d


86/113

Hybrid Method

Example:

Tiemann et al. investigate ensemble learningmethods for hybrid music recommenderalgorithms. Their approach combines social andcontent-based methods, where each oneproduces a weak learner. Then using a

combination rule, it unifies the output of the weaklearners.


87/113

EVALUATION

System-centric, Network-centric, User-centric

E l ti


88/113

Evaluation

Three different strategies

System-centric

Network-centric User-centric

S t t i E l ti


89/113

System-centric Evaluation

Evaluation measures how accurate the systemcan predict the actual values that user havepreviously assigned

S t t i E l ti


90/113


Most approaches based on the leave-n-outmethod Similar to the classic n-fold cross validation

Dataset divided into two (usually disjunct)sets: Training and Test

Accuracy evaluation based only on a usersdataset

The rest of the items of the catalog are ignored

S t t i E l ti


91/113


Metrics:

Predictive accuracy

Mean Absolute Error, Root Mean Square Error

Decision based

Mean Average Precision, Recall, F-measure, Accuracy,ROC

Rank based Spearmans , Kandall, Half-life Utility, Discounted

Cumulative Gain


92/113


93/113

Network centric Evaluation


94/113

Network-centric Evaluation

The similarity network is the basis to providethe recommendations

Important to analyze and understand theunderlying topology of the similarity network

Measures:

Coverage

Diversity of recommendations



95/113


In terms of: Navigation

Average Shortest Path, Giant Component

Connectivity Degree Distribution, Degree-Degree Correlation, Mixing

Patterns

Clustering

Local/Global Clustering Coefficient

Centrality

Degree, Closeness, Betweeness



96/113


Limitations:

Accuracy of the recommendations cannot bemeasured

Transparency (explainability) and trustworthiness(confidence) of the recommendations cannot bemeasured

The perceived quality (i.e. usefulness andeffectiveness) of the recommendations cannot bemeasured

User centric Evaluation


97/113

User-centric Evaluation

Evaluation focuses on the users perceived

quality and usefulness of therecommendations



98/113


Copes with the limitations of both:

System- and Network-centric approaches

Evaluates: Novelty

Perceived Quality


99/113


100/113

Novelty


101/113

Novelty

Ask users if they recognize the predicted itemsor not

Combining novelty and perceived quality wecan infer if:

User likes to receive and discover unknown items

Prefers more conservative and familiarrecommendations

A/B Testing


102/113

A/B Testing

Present two different versions of an algorithm(or two algorithms)

Evaluate which one performs the best

Performance measured by the impact the

new algorithm has on the visitors behavior,compared to the baseline algorithm



103/113


Limitations:

Need ofuser intervention in the evaluationprocess

Gathering feedback from the user can be tedious forsome users

Time-consuming

Evaluation summary


104/113

Evaluation summary

Combining the three methods we can coverall the facets of a recommender algorithm


105/113


106/113

Last fm Dataset 1K users


107/113

Last.fm Dataset 1K users

Contains tuples

Represents the listening habits for ~1.000 users

Collected from Last.fmAPI

User.getRecentTracks()

Statistics: ~108,000 Artists with MusicBrainz ID

~70.000 Artists without MusicBrainz ID

Last fm Dataset 360K users


108/113

Last.fm Dataset 360K users

Contains tuples from360.000 users

Collected from Last.fm API User.getTopArtists()

Statistics:

~190.000 Artists with MusicBrainz ID ~107.000 Artists without MusicBrainz ID

The Million Song Dataset


109/113

The Million Song Dataset

One Million Songs!!! 280GB of data

~45.000 unique artists

~8.000 unique terms

> 2 Million asymmetric similarity relationships

Acoustic features

Pitch, Timbre, Loudness, etc. Links to other sources to obtain more information

Musicbrainz, 7digital, playme

NEXTONE PLAYER


110/113

NEXTONE PLAYER

NEXTONE PLAYER: A

Music Recommendation

System Based on User

BehaviorYajie Hu and Mitsunori OgiharaISMIR 2011

Session-based CF for Music


111/113

Recommendation

Session-based Collaborative

Filtering for Predicting the

Next SongSung Eun Park, Sangkeun Lee, Sang-goo LeeCNSI 2011


112/113


113/113

Music Recommendation

Documents

Transcript of Music Recommendation