CSTalks - On machine learning - 2 Mar

of 61/61
On Machine Learning at CSTalks by Vlad Hosu
  • date post

    20-Nov-2014
  • Category

    Documents

  • view

    650
  • download

    5

Embed Size (px)

description

 

Transcript of CSTalks - On machine learning - 2 Mar

  • 1. On Machine Learning
    • at CSTalks
      • by Vlad Hosu

2. Introduction 3. Fundamental Questions

  • What are the fundamental laws that govern all learning processes?
  • How can we build computer systems that automatically improve with experience?

4. Learning: Method

    • a process of adaption
    • by which a parametric model is automatically adjusted
    • so that some fitness criteria is more readily met

5. Before Learning Im learning, hence I need adapt! 6. After Result: Liony adjusts his diet. 7. Biological Learning

  • Model: nervous system neuron connectivity, chemical changes etc
  • Fitness: improved behavior skills, memory, knowledge

8. Machine Learning

  • a mathematical model
  • with adjustable parameters
  • optimizing some fitness function

9. Motivation 10. Why?

  • some things are hard to code
  • too much data
  • automatic learning works better
  • is easier to customize/personalize

11. Learning: Purpose

  • estimation
    • function - stock market
    • class - recognition
    • structure - grouping

12. Requirements

  • good learning ability
  • scalability to large problems
  • simple and easy algorithm implementation

13. Things Ahead

  • Problems
    • Clustering
    • Classification
    • Regression
  • Learning issues
    • importance of domain knowledge
    • learning/generalization ability
    • model complexity issues
  • Optimization

14. Important Problems 15. Clustering 16. Classification x1 x2 17. Classification

  • Types
    • discriminative
    • generative

x1 x2 18. Classification

  • Types
    • discriminative
    • generative

x1 x2 1 0 19. Classification 20. Regression 21. Making Connections

  • discrete value regression => generative classification
  • regression on boundary space => discriminative classification
  • clustering + labels => classification

22. Learning Issues 23. Domain Knowledge

  • exploitation of problem structure
    • human abstractions are better
    • important for picking the right model

24. Grouping in Images

  • groups together similar parts of an image
    • select objects
    • find patterns
  • features = pixel values (function of)

25. Segmentation 26. Color Space RGB space RGB space 27. Color Space (cont) 28. Suitable Clustering 29. Generalization Ability

    • training data generalizes to new data
    • important for classification accuracy

30. Support Vector Machines (SVM)

  • linear classifier on distorted space

31. Learning Ability over fitting 32. Problems withOver-fitting 33. SVM vs Decision Trees 34. Complexity Issues

  • models should be
    • as simple as possible
    • but representative of the training data

35. Neural Networks

  • model: weights
  • fitness: output error
  • general function

36. Training a Network 37. Non-trivial Functions 38. Optimization 39. Optimizing Fitness

  • find extrema
  • strategies
    • gradient descent
    • convex optimization

40. Optimization

  • finding extrema
  • local/global

41. Gradient Descent 42. Problem: Local Extrema 43. Problem: Speed 44. Linear Programming x1 x2 lines define aconvex function planes in 3D etc 45. Considerations

  • scaling to large features spaces
    • feature selection
    • dimensionality reduction

46. Open Problems 47. Open Problems

  • unlabeled data for regression
  • exploiting sparsity in high dimensional spaces for non-parametric learning
  • transferring learnt information from one task to simplify learning another

48. Open Problems (cont)

  • algorithms for learning control strategies from delayed rewards and other inputs
  • best active learning strategies for different learning problems
  • degree one can preserve data privacy while obtaining the benefits of data mining

49. The end Questions? 50. Types of Regression

  • parametric
  • non-parametric

51. Linear vs Non-linear

  • linear
    • smooth
    • under-fitting
    • good enough for some processes (biz)
  • non-linear
    • complex
    • over-fitting
    • works on most data-sets

52. Naive Bayes good spam write people free No. Good No. Spam * * 53. Graph Clustering 54. Mean Shift 55. Problems in CV

  • What are the physical and geometric processes that govern (digital) imaging?
  • What are the informative areas of an image and how do we detect them?
  • What portions of an image pertain to one another and to relevant physical phenomena?
  • From one (or more) images, how can we determine the geometry of the scene?

56. Linear Regression

  • model: straight line
  • 2 adjustable parameters
  • fitness function: root mean squared error

57. Solution Stability y-shift slope 58. Some Issues with Model Selection normal outliers wrong model 59. Real Photo inColor Space EM KMeans 60. Conjugate Gradient 61. Newtons Method