University of Missouri, Columbia Slides Adapted from Book...
Transcript of University of Missouri, Columbia Slides Adapted from Book...
![Page 1: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/1.jpg)
Dr. Jianlin Cheng
University of Missouri, Columbia
Slides Adapted from Book and CMU, Stanford Machine Learning Courses
Fall, 2011
![Page 2: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/2.jpg)
Syllabus
• Course web site: http://people.cs.missouri.edu/~chengji/supervised_learning/
• Location: EBW 355; Time: TuTh 4:00 pm - 5:15 pm; Office Hours: TuTh 3:00 pm - 4:00 pm
• Text Book: Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2007
• Grading: assignment (40%), group project report (30%), group project representation (30%); grade scale (A+, A, A-, B+, B, B-, C+, C, C-, and F)
• Questions / Assignment submission: [email protected]
![Page 3: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/3.jpg)
Topics
• Introduction and Bayes optimal learning rule • MLE and MAP (parametric learning) • Generative VS discriminative methods • Nonparametric methods • Model selection • Boosting and bagging • Support vector machines • Graphical models (emphasized) • Semi-supervised learning
![Page 4: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/4.jpg)
![Page 5: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/5.jpg)
![Page 6: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/6.jpg)
![Page 7: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/7.jpg)
![Page 8: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/8.jpg)
![Page 9: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/9.jpg)
![Page 10: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/10.jpg)
![Page 11: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/11.jpg)
![Page 12: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/12.jpg)
![Page 13: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/13.jpg)
![Page 14: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/14.jpg)
![Page 15: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/15.jpg)
Face Recognition
![Page 16: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/16.jpg)
Face Recognition
![Page 17: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/17.jpg)
![Page 18: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/18.jpg)
![Page 19: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/19.jpg)
![Page 20: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/20.jpg)
![Page 21: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/21.jpg)
![Page 22: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/22.jpg)
![Page 23: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/23.jpg)
![Page 24: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/24.jpg)
![Page 25: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/25.jpg)
![Page 26: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/26.jpg)
![Page 27: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/27.jpg)
![Page 28: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/28.jpg)
![Page 29: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/29.jpg)
Probably approximately correct learning
Occam's Razor – When everything is equal, a simple Solution is better.
![Page 30: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/30.jpg)
![Page 31: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/31.jpg)
![Page 32: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/32.jpg)
![Page 33: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/33.jpg)
![Page 34: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/34.jpg)
![Page 35: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/35.jpg)
![Page 36: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/36.jpg)
![Page 37: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/37.jpg)
![Page 38: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/38.jpg)
![Page 39: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/39.jpg)
![Page 40: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/40.jpg)
![Page 41: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/41.jpg)
How to minimize errors?
x
P(x, class) Class 1 Class 2
Where to set a threshold on x to make classification in order to minimize classification errors?
![Page 42: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/42.jpg)
Bayes Error
Calculate the probability of an error – Bayes error
![Page 43: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/43.jpg)
Bayes Error
Calculate the probability of an error – Bayes error
P(x,y) = P(x|y)P(y)
Joint Prob = Con. Prob * Marginal Prob.
![Page 44: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/44.jpg)
![Page 45: University of Missouri, Columbia Slides Adapted from Book ...calla.rnet.missouri.edu/cheng_courses/supervised... · Natural _LoseWeight SuperFood Endorsed by Oprah Winfrey, Free Trial](https://reader035.fdocuments.net/reader035/viewer/2022081612/5f7c575c57adce25ac147e46/html5/thumbnails/45.jpg)