Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee...
Transcript of Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee...
![Page 1: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/1.jpg)
Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee, Shrikanth Narayanan Signal Analysis and Interpretation Laboratory (SAIL) Electrical Engineering Department University of Southern California
![Page 2: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/2.jpg)
• 2009 Interspeech Emotion Challenge
• Classifier Sub-Challenge • Five Emotion Classes
• Emphatic, Angry, Neutral, Positive, Rest
• AIBO Database • Training: 9959 – labeled • Testing: 8257 – unlabeled • 384 Dimension Feature Vector
• Primary Performance Measure
• Unweighted Recall Percentage • Weighted Recall Percentage
![Page 3: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/3.jpg)
We used a combination of binary classifiers instead of one multi-class classifier
We proposed a classification framework composed of a hierarchical tree, where the top level classification was performed on the easiest emotion recognition task
![Page 4: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/4.jpg)
Data (A, E, N, P, R)
Classifier A/E vs. P
Classifier A vs. E
Classifier P vs. N/R Classifier
A vs. N/R Classifier E vs. N/R
Classifier N vs. R
Angry
Emphatic Positive
Neutral Rest
A: Angry E: Emphatic N: Neutral P: Positive R: Rest
![Page 5: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/5.jpg)
• Bayesian Logistic Regression
• Binary Classifier • Gaussian prior on model coefficients β • Threshold tuning based on balanced error rate criterion
• Support Vector Machine
• Binary Classifier • Linear Kernel • Cost of Error, C , was set to approximate the distribution of data
• Feature Selection
• Six sets of features • Forward feature selection using binary logistic regression
• Feature Normalization
• z-normalize – µ , σ2 calculated from the neutral data in the training dataset
![Page 6: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/6.jpg)
Experiment I: Leave one speaker out - 26 fold - cross validation on the training dataset
Experiment II: Evaluate performance on the unlabeled testing dataset
![Page 7: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/7.jpg)
Angry Emphatic Neutral Positive Rest
Angry 504 145 126 53
Emphatic 397 1078 412 101 107
Neutral 506 1020 2703 776 585
Positive 121 439 62
Rest 97 130 185 171 138
21
Bayesian Logistic Regression
Unweighted Recall Weighted Recall 48.27% 48.82%
31
53
94.82 % Recall
![Page 8: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/8.jpg)
Angry Emphatic Neutral Positive Rest
Angry 504 145 126 53 53
Emphatic 397 1078 412 101 107
Neutral 506 1020 2703 776 585
Positive 21 31 121 439 62
Rest 97 130 185 171 138
Bayesian Logistic Regression
Unweighted Recall Weighted Recall 48.27% 48.82%
• Rest is recognized at about chance level • Comparable recall percentage for emotional classes (Angry, Emphatic & Positive)
![Page 9: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/9.jpg)
Support Vector Machine
Unweighted Recall Weighted Recall 47.44% 46.84%
Angry Emphatic Neutral Positive Rest
Angry 463 159 123 57 79
Emphatic 322 1041 424 156 150
Neutral 386 930 2548 958 768
Positive 27 29 103 446 69
Rest 80 123 159 192 167
![Page 10: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/10.jpg)
Submitted Results on Test Dataset
Unweighted Recall Weighted Recall
Baseline 38.20% 39.20%
Bayesian Logistic Regression 41.57% 39.87%
Support Vector Machine 40.84% 38.05%
Angry Emphatic Neutral Positive Rest
Angry 290 171 65 63 22
Emphatic 210 752 325 136 150
Neutral 748 1094 2047 1109 369
Positive 23 13 39 131 9
Rest 95 58 134 197 62
• 3.37% absolute (8.82% relative) improvement (Bayesian Logistic Regression) • 52% recall percentage for emotional classes (Angry, Emphatic & Positive) • 25% recall percentage for non-emotional classes (Neutral, Rest)
![Page 11: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/11.jpg)
• Concluding Summary
• 3.37% absolute (8.82% relative) improvement over baseline • Carefully designed hierarchical structure • Easier classification task as first level
• Future Work
• Other feature selection scheme (large-margin based) • Ensemble learning techniques • Soft decision at every decision level
![Page 12: Chi-Chun Lee, Emily Mower, Carlos Busso, Sungbok Lee ...ecs.utdallas.edu/research/researchlabs/msp-lab/publications/Lee_2009_slide.pdfChi-Chun Lee, Emily Mower, Carlos Busso, Sungbok](https://reader036.fdocuments.net/reader036/viewer/2022071116/5ffe7fc21765a81cb76a725a/html5/thumbnails/12.jpg)
This work was supported in part by funds from NSF, Army, USC Annenberg Fellowship, Intel Foundation PhD Fellowship