Machine Learning A Quick look Sources: Artificial Intelligence – Russell & Norvig Artifical...
-
Upload
ellen-spellman -
Category
Documents
-
view
230 -
download
1
Transcript of Machine Learning A Quick look Sources: Artificial Intelligence – Russell & Norvig Artifical...
Machine Learning
A Quick look
Sources:• Artificial Intelligence – Russell & Norvig• Artifical Intelligence - Luger
By: Héctor Muñoz-Avila
What Is Machine Learning?“Logic is not the end of wisdom, it is just the beginning” --- Spock
System
Knowledge
Environment
Action1
time
Knowledge
Environment
System
changed
Action2
Learning: The Big Picture
•Two forms of learning:
Supervised: the input and output of the learning component can be perceived (for example: friendly teacher)
Unsupervised: there is no hint about the correct answers of the learning component (for example to find clusters of data)
4
• Online – during gameplay– Adapt to player tactics– Avoid repetition of mistakes– Requirements: computationally cheap,
effective, robust, fast learning (Spronck 2004)
• Offline - before the game is released– Devise new tactics– Discover exploits
Offline vs. Online Learning
Classification(According to the language representation)
• Symbolic Version Space Decision Trees Explanation-Based Learning …
• Sub-symbolic Reinforcement Learning Connectionist Evolutionary
Classification(According to the language representation)
• Symbolic Version Space Decision Trees Explanation-Based Learning …
• Sub-symbolic Reinforcement Learning Connectionist Evolutionary
Version Space
Idea: Learn a concept from a group of instances, some positive and some negative
Example: •target: obj(Size,Color,Shape) Size = {large, small} Color = {red, white, blue} Shape = {ball, brick, cube}
•Instances: +:
obj(large,white,ball)obj(small,blue,ball)
−: obj(small,red,brick)
obj(large,blue,cube)
Two extremes (temptative) solutions:
obj(X,Y,Z)
obj(large,white,ball) obj(small,blue,ball) …
too general
too specific
obj(large,Y,ball) obj(small,Y,ball)
obj(X,Y,ball)
…
con
cep
t sp
ace
How Version Space Works
++
+
+
++
+
+− −
If we consider only positives If we consider positive and negatives
++
+
+
++
+
+− −
What is the role of the negative instances?
to help prevent over-generalizations
Classification(According to the language representation)
• Symbolic Version Space Decision Trees Explanation-Based Learning …
• Sub-symbolic Reinforcement Learning Connectionist Evolutionary
Explanation-Based learning
AC
B A B C A CB
CBA
BA
C
BAC
B CA
CAB
ACB
BCA
A BC
AB
C
ABC
Can we avoid making this error again?
?
?
?
Explanation-Based learning (2)
AC
B A B CCBA
ACB
A BC
?
?
?
Possible rule: If the initial state is this and the final state is this, don’t do that
More sensible rule: don’t stack anything above a block, if the block has to be free in the final state
Classification(According to the language representation)
• Symbolic Version Space Decision Trees Explanation-Based Learning …
• Sub-symbolic Reinforcement Learning Connectionist Evolutionary
Motivation # 1: Analysis Tool
•Suppose that a company have a data base of sales data, lots of sales data
•How can that company’s CEO use this data to figure out an effective sales strategy
•Safeway, Giant, etc cards: what is that for?
Motivation # 1: Analysis Tool (cont’d)
Ex’ple Bar Fri Hun Pat Type Res wait
x1 no no yes some french yes yes
x4 no yes yes full thai no yes
x5 no yes no full french yes no
x6
x7
x8
x9
x10
x11
Sales data
“if buyer is male & and age between 24-35 & married then he buys sport magazines”
induction
Decision Tree
The Knowledge Base in Expert Systems
A knowledge base consists of a collection of IF-THEN rules:
if buyer is male & age between 24-50 & married then he buys sport magazines
if buyer is male & age between 18-30then he buys PC games magazines
Knowledge bases of fielded expert systems contain hundreds and sometimes even thousands such rules. Frequently rules are contradictory and/or overlap
Main Drawback of Expert Systems: The Knowledge Acquisition Bottle-NeckThe main problem of expert systems is acquiring knowledge from human specialist is a difficult, cumbersome and long activity.
Name KB #Rules Const. time
(man/years)
Maint. time
(man/years)
MYCIN KA 500 10 N/A
XCON KA 2500 18 3
KB = Knowledge BaseKA = Knowledge Acquisition
Motivation # 2: Avoid Knowledge Acquisition Bottle-Neck
•GASOIL is an expert system for designing gas/oil separation systems stationed of-shore
•The design depends on multiple factors including: proportions of gas, oil and water, flow rate, pressure, density, viscosity, temperature and others
•To build that system by hand would had taken 10 person years
•It took only 3 person-months by using inductive learning!
•GASOIL saved BP millions of dollars
Motivation # 2 : Avoid Knowledge Acquisition Bottle-Neck
Name KB #Rules Const. time
(man/years)
Maint. time
(man/months)
MYCIN KA 500 10 N/A
XCON KA 2500 18 3
GASOIL IDT 2800 1 0.1
BMT KA (IDT)
30000+ 9 (0.3) 2 (0.1)
KB = Knowledge BaseKA = Knowledge AcquisitionIDT = Induced Decision Trees
Example of a Decision TreePatrons?
no yes
nonesome
waitEstimate?
no yes
0-10>60
Full
Alternate?
Reservation?
Yes
30-60
no
yes
No
no
Bar?
Yes
no
yes
Fri/Sat?
No Yes
yes
no yes
Hungry?
yes
No
10-30
Alternate?
yes
Yes
no
Raining?
no yes
yes
no yes
Definition of A Decision Tree
A decision tree is a tree where:
•The leaves are labeled with classifications (if the classification is “yes” or “no”. The tree is called a boolean tree)
•The non-leaves nodes are labeled with attributes
•The arcs out of a node labeled with an attribute A are labeled with the possible values of the attribute A
Induction
Ex’ple Bar Fri Hun Pat Type Res wait
x1 no no yes some french yes yes
x4 no yes yes full thai no yes
x5 no yes no full french yes no
x6
x7
x8
x9
x10
x11
Data
patternpattern
Databases: what are the data that matches this pattern?
database
Induction: what is the pattern that matches these data?
induction
Induction of Decision Trees
•Objective: find a concise decision tree that agrees with the examples
•The guiding principle we are going to use is the Ockham’s razor principle: the most likely hypothesis is the simplest one that is consistent with the examples
•Problem: finding the smallest decision tree is NP-complete
•However, with simple heuristics we can find a small decision tree (approximations)
Induction of Decision Trees: Algorithm
Algorithm:
1. Initially all examples are in the same group
2. Select the attribute that makes the most difference (i.e., for each of the values of the attribute most of the examples are either positive or negative)
3. Group the examples according to each value for the selected attribute
4. Repeat 1 within each group (recursive call)
Example
Ex’ple Bar Fri Hun Pat Alt Type wait x1 no no yes some yes French yes
x4 no yes yes full yes Thai yes
x5 no yes no full yes French no
x6 yes no yes some no Italian yes
x7 yes no no none no Burger no
x8 no no yes some no Thai yes
x9 yes yes no full no Burger no
x10 yes yes yes full yes Italian no
x11 no No no none no Thai no
IDT: Example
Lets compare two candidate attributes: Patrons and Type. Which is a better attribute?
Patrons?
none
X7(-),x11(-)
some
X1(+),x3(+),x6(+),x8(+)
full
X4(+),x12(+), x2(-),x5(-),x9(-),x10(-)
Type?
french
X1(+),x5(-)
italian
X6(+),x10(-)
burger
X3(+),x12(+), x7(-),x9(-)X4(+),x12(+)
x2(-),x11(-)
thai
Example of a Decision TreePatrons?
no yes
nonesome
waitEstimate?
no yes
0-10>60
Full
Alternate?
Reservation?
Yes
30-60
no
yes
No
no
Bar?
Yes
no
yes
Fri/Sat?
No Yes
yes
no yes
Hungry?
yes
No
10-30
Alternate?
yes
Yes
no
Raining?
no yes
yes
no yes
Decision Trees in Gaming
• Black & White, developed by Lionhead Studios, and released in 2001 used ID3
• Used to predict a player’s reaction to a certain creature’s action
• In this model, a greater feedback value means the creature should attack
Decision Trees in Black & White
Example Attributes Target
Allegiance Defense Tribe Feedback
D1 Friendly Weak Celtic -1.0
D2 Enemy Weak Celtic 0.4
D3 Friendly Strong Norse -1.0
D4 Enemy Strong Norse -0.2
D5 Friendly Weak Greek -1.0
D6 Enemy Medium Greek 0.2
D7 Enemy Strong Greek -0.4
D8 Enemy Medium Aztec 0.0
D9 Friendly Weak Aztec -1.0
Decision Trees in Black & White
Allegiance
Defense
Friendly Enemy
0.4 -0.3
-1.0
Weak Strong
0.1
Medium
Note that this decision tree does not even use the tribe attribute
Decision Trees in Black & White
• Now suppose we don’t want the entire decision tree, but we just want the 2 highest feedback values
• We can create a Boolean expressions, such as
((Allegiance = Enemy) ^ (Defense = Weak)) v ((Allegiance = Enemy) ^ (Defense = Medium))