Introduction to Machine Learning
description
Transcript of Introduction to Machine Learning
![Page 1: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/1.jpg)
Artificial IntelligenceArtificial Intelligence
Lecture 15
Introduction to Machine Learning
![Page 2: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/2.jpg)
OverviewOverview
Machine Learning ID3 Decision Tree Algorithm Discussions
![Page 3: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/3.jpg)
Machine LearningMachine Learning
Supervised Learning• Training examples consist of pairs of input
vectors, and desired outputs
Unsupervised Learning• Training examples do not contain hints about
correct outputs• Usually used to identify unusual structures in
data
![Page 4: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/4.jpg)
Inductive Learning BasicsInductive Learning Basics
Inferring a boolean/real-valued function from training examples
A training example is a pair of (x, f(x))• x is the input• f(x) is the output of the function applied to x
![Page 5: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/5.jpg)
Any function that approximates the given set of examples
(a) (b) (c) (d)
Bias: preference for one hypothesis beyond mere consistency
HypothesisHypothesis
![Page 6: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/6.jpg)
Hypothesis SpaceHypothesis Space
A set of all hypotheses consistent with data denoted by H:{H1, H2, …, Hn}
Inductive learning is searching for a good hypothesis in the hypothesis space
Occam’s razor: prefer the simplest hypothesis consistent with data
Inductive Learning Assumption: Any hypothesis found to approximate the target function well over a sufficiently large set of training examples will also approximate the target function well over other unobserved examples.
![Page 7: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/7.jpg)
ID3 Decision Tree Algorithm ID3 Decision Tree Algorithm
![Page 8: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/8.jpg)
Training ExamplesTraining Examples
![Page 9: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/9.jpg)
Decision Tree for Decision Tree for PlayTennisPlayTennis
Outlook Temperature Humidity Wind PlayTennis
Sunny Hot High Weak ?
Overcast Hot High Weak ?
![Page 10: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/10.jpg)
Decision Tree RepresentationDecision Tree Representation
Decision Tree• Each internal node tests an attribute• Each branch takes an attribute value• Each leaf node predict a class label
Disjunction of conjunctions of a set of attribute values
![Page 11: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/11.jpg)
EntropyEntropy
![Page 12: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/12.jpg)
Information GainInformation Gain
![Page 13: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/13.jpg)
ID3 AlgorithmID3 Algorithm
![Page 14: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/14.jpg)
Training ExamplesTraining Examples
![Page 15: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/15.jpg)
Selecting the Best AttributeSelecting the Best Attribute
![Page 16: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/16.jpg)
![Page 17: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/17.jpg)
Hypothesis Space Search by ID3Hypothesis Space Search by ID3
![Page 18: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/18.jpg)
DiscussionsDiscussions
![Page 19: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/19.jpg)
Hypothesis Space Search by ID3Hypothesis Space Search by ID3
Hypothesis space is complete• Target function surely in there …
Outputs a single hypothesis (Which one?)• Cannot determine how many alternatives
No back tracking• Local minima …
Use statistical properties of all training data at each step in search• Robust to noisy data
![Page 20: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/20.jpg)
Inductive Bias in ID3Inductive Bias in ID3
Note the hypothesis space H is the power set of instances X. Unbiased?
Not really.• Preference for short trees, and for those with
high information gain attributes near the root• Bias is a preference for some hypotheses,
rather than a restriction of hypothesis space H, e.g. target concept is not in H.
![Page 21: Introduction to Machine Learning](https://reader036.fdocuments.net/reader036/viewer/2022081413/546ce153af7959ea368b4814/html5/thumbnails/21.jpg)
SummarySummary
Machine Learning ID3 Decision Tree Algorithm Discussions