# CSTalks - On machine learning - 2 Mar

date post

20-Nov-2014Category

## Documents

view

650download

5

Embed Size (px)

description

### Transcript of CSTalks - On machine learning - 2 Mar

- 1. On Machine Learning
- at CSTalks

- by Vlad Hosu

2. Introduction 3. Fundamental Questions

- What are the fundamental laws that govern all learning processes?

- How can we build computer systems that automatically improve with experience?

4. Learning: Method

- a process of adaption

- by which a parametric model is automatically adjusted

- so that some fitness criteria is more readily met

5. Before Learning Im learning, hence I need adapt! 6. After Result: Liony adjusts his diet. 7. Biological Learning

- Model: nervous system neuron connectivity, chemical changes etc

- Fitness: improved behavior skills, memory, knowledge

8. Machine Learning

- a mathematical model

- with adjustable parameters

- optimizing some fitness function

9. Motivation 10. Why?

- some things are hard to code

- too much data

- automatic learning works better

- is easier to customize/personalize

11. Learning: Purpose

- estimation

- function - stock market

- class - recognition

- structure - grouping

12. Requirements

- good learning ability

- scalability to large problems

- simple and easy algorithm implementation

13. Things Ahead

- Problems

- Clustering

- Classification

- Regression

- Learning issues

- importance of domain knowledge

- learning/generalization ability

- model complexity issues

- Optimization

14. Important Problems 15. Clustering 16. Classification x1 x2 17. Classification

- Types

- discriminative

- generative

x1 x2 18. Classification

- Types

- discriminative

- generative

x1 x2 1 0 19. Classification 20. Regression 21. Making Connections

- discrete value regression => generative classification

- regression on boundary space => discriminative classification

- clustering + labels => classification

22. Learning Issues 23. Domain Knowledge

- exploitation of problem structure

- human abstractions are better

- important for picking the right model

24. Grouping in Images

- groups together similar parts of an image

- select objects

- find patterns

- features = pixel values (function of)

25. Segmentation 26. Color Space RGB space RGB space 27. Color Space (cont) 28. Suitable Clustering 29. Generalization Ability

- training data generalizes to new data

- important for classification accuracy

30. Support Vector Machines (SVM)

- linear classifier on distorted space

31. Learning Ability over fitting 32. Problems withOver-fitting 33. SVM vs Decision Trees 34. Complexity Issues

- models should be

- as simple as possible

- but representative of the training data

35. Neural Networks

- model: weights

- fitness: output error

- general function

36. Training a Network 37. Non-trivial Functions 38. Optimization 39. Optimizing Fitness

- find extrema

- strategies

- gradient descent

- convex optimization

40. Optimization

- finding extrema

- local/global

41. Gradient Descent 42. Problem: Local Extrema 43. Problem: Speed 44. Linear Programming x1 x2 lines define aconvex function planes in 3D etc 45. Considerations

- scaling to large features spaces

- feature selection

- dimensionality reduction

46. Open Problems 47. Open Problems

- unlabeled data for regression

- exploiting sparsity in high dimensional spaces for non-parametric learning

- transferring learnt information from one task to simplify learning another

48. Open Problems (cont)

- algorithms for learning control strategies from delayed rewards and other inputs

- best active learning strategies for different learning problems

- degree one can preserve data privacy while obtaining the benefits of data mining

49. The end Questions? 50. Types of Regression

- parametric

- non-parametric

51. Linear vs Non-linear

- linear

- smooth

- under-fitting

- good enough for some processes (biz)

- non-linear

- complex

- over-fitting

- works on most data-sets

52. Naive Bayes good spam write people free No. Good No. Spam * * 53. Graph Clustering 54. Mean Shift 55. Problems in CV

- What are the physical and geometric processes that govern (digital) imaging?

- What are the informative areas of an image and how do we detect them?

- What portions of an image pertain to one another and to relevant physical phenomena?

- From one (or more) images, how can we determine the geometry of the scene?

56. Linear Regression

- model: straight line

- 2 adjustable parameters

- fitness function: root mean squared error

57. Solution Stability y-shift slope 58. Some Issues with Model Selection normal outliers wrong model 59. Real Photo inColor Space EM KMeans 60. Conjugate Gradient 61. Newtons Method