Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... ·...
Transcript of Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... ·...
![Page 1: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/1.jpg)
Automatic Heuristic Construction in aComplete General Game Player
Gregory Kuhlmann Kurt Dresner Peter Stone
Learning Agents Research GroupDepartment of Computer SciencesThe University of Texas at Austin
AAAI 2006
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 2: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/2.jpg)
Computer Game Playing
• One of AI’s biggest success stories• checkers, chess, scrabble, othello, connect-4
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 3: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/3.jpg)
Computer Game Playing
• One of AI’s biggest success stories• checkers, chess, scrabble, othello, connect-4
• Search is universal in game playing
• Bound search for large state spaces• Board evaluation function (heuristic)
• Game analysis• Traditionally performed by human designers• Specific to a single game
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 4: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/4.jpg)
General Game Playing
• Single system plays many games in a class
• Analysis performed by system itself• Player inputs game rules for unknown game
• Game description allows simulation• Expand game tree
Rules + Inference
• If not exhaustively searchable, what to do?• Look for hints in game description
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 5: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/5.jpg)
General Game Playing
• Single system plays many games in a class
• Analysis performed by system itself• Player inputs game rules for unknown game
• Game description allows simulation• Expand game tree
Rules + Inference
• If not exhaustively searchable, what to do?• Look for hints in game description
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 6: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/6.jpg)
AAAI GGP Competition
• Game Players run as servers• Game Manager sends rules to players
• Game Description in GDL• Start clock
• Time to analyze description (1–40 minutes)• Play clock:
• Time to make moves (10–120 seconds)
Play Play
GM
GP
Rules Move Move . . .
Start
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 7: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/7.jpg)
Class of Games
Deterministic, Perfect Information Games
YES NO
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 8: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/8.jpg)
Game Description Language
• First order logic (KIF)
• State: database of provable facts• Constructs
• init: initial state• legal: legal moves• next: state transitions• terminal: termination conditions• goal: value of terminal states
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 9: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/9.jpg)
Game Description Language
(role white) (role black)(init (cell a 1 b)) (init (cell a 2 b))(init (cell a 1 b)) (init (cell a 2 bk))(init (cell a 1 wr)) (init (cell a 2 b))(init (cell a 1 b)) (init (cell a 2 b))(init (control white)) (init (step 1))(<= (legal white (move wk ?u ?v ?x ?y))
(true (control white))(true (cell ?u ?v wk))(kingmove ?u ?v ?x ?y)(true (cell ?x ?y b)))
(<= (next (step ?y))(true (step ?x))(succ ?x ?y))
(succ 1 2) (succ 2 3) (succ 3 4) (succ 4 5)(<= (goal white 100)
checkmate)(<= terminal
(true (step 10)))
• Simulate with theorem prover (Prolog)• How can we do better than just legal play?
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 10: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/10.jpg)
Game Description Language
(role white) (role black)(init (cell a 1 b)) (init (cell a 2 b))(init (cell a 1 b)) (init (cell a 2 bk))(init (cell a 1 wr)) (init (cell a 2 b))(init (cell a 1 b)) (init (cell a 2 b))(init (control white)) (init (step 1))(<= (legal white (move wk ?u ?v ?x ?y))
(true (control white))(true (cell ?u ?v wk))(kingmove ?u ?v ?x ?y)(true (cell ?x ?y b)))
(<= (next (step ?y))(true (step ?x))(succ ?x ?y))
(succ 1 2) (succ 2 3) (succ 3 4) (succ 4 5)(<= (goal white 100)
checkmate)(<= terminal
(true (step 10)))
• Simulate with theorem prover (Prolog)• How can we do better than just legal play?
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 11: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/11.jpg)
Identifying Structures
Identify structures from common game elements
Successor Relations
(succ 1 2) (angel paper table)(succ 2 3) (angel table bottom)(succ 3 4) (angel bottom mellow)(succ 4 5) (angel mellow yard)
Tokens will be scrambled. Based on structure alone.
Bridge between logical and numerical representations
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 12: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/12.jpg)
Identifying Structures (cont.)
Find rules matching templates
Step Counters
(<= (next (step ?x)) (<= (next (foo ?u))(true (step ?y)) (true (foo ?v))(succ ?y ?x)) (bar ?v ?u))
Again no lexical clues used.
• Bounds tree depth
• Remove for longer internal games
• Remove from Transposition Table
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 13: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/13.jpg)
Board Game Structures
Many games have a board of some type
State
(cell 1 1 bk) (cell 1 2 b)(cell 1 3 wk) (cell 1 4 b)(cell 2 1 b) (cell 2 2 b)(cell 2 3 bk) (cell 2 4 b)(cell 3 1 wr) (cell 3 2 b)(cell 3 3 b) (cell 3 4 b)(cell 4 1 b) (cell 4 2 b)(cell 4 3 b) (cell 4 4 b)
Boards and Piecescell:0,1->2 ; [b, wk, wr, bk]cell:0,2->1 ; [1, 2, 3, 4]cell:1,2->0 ; [1, 2, 3, 4]
• Start with all ternary functions
• Divide slots into inputs andoutputs
• Refine through internalsimulation
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 14: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/14.jpg)
Board Game Structures
Many games have a board of some type
State
(cell 1 1 bk) (cell 1 2 b)(cell 1 3 wk) (cell 1 4 b)(cell 2 1 b) (cell 2 2 b)(cell 2 3 bk) (cell 2 4 b)(cell 3 1 wr) (cell 3 2 b)(cell 3 3 b) (cell 3 4 b)(cell 4 1 b) (cell 4 2 b)(cell 4 3 b) (cell 4 4 b)
Boards and Piecescell:0,1->2 ; [b, wk, wr, bk]
• Start with all ternary functions
• Divide slots into inputs andoutputs
• Refine through internalsimulation
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 15: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/15.jpg)
Board Game Structures
Many games have a board of some type
State
(cell 1 1 bk) (cell 1 2 b)(cell 1 3 wk) (cell 1 4 b)(cell 2 1 b) (cell 2 2 b)(cell 2 3 bk) (cell 2 4 b)(cell 3 1 wr) (cell 3 2 b)(cell 3 3 b) (cell 3 4 b)(cell 4 1 b) (cell 4 2 b)(cell 4 3 b) (cell 4 4 b)
Boards and Piecescell:0,1->2 ; [wk, wr, bk]
• Start with all ternary functions
• Divide slots into inputs andoutputs
• Refine through internalsimulation
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 16: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/16.jpg)
Features
Identified Structure Generated Features
Ordered Board w/ Pieces Each piece’s X coordinateEach piece’s Y coordinateManhattan distance between each pair of piecesSum of pair-wise Manhattan distances
Board w/o Pieces Number of markers of each typeQuantity Amount
• Board inputs ordered by successor relation(s)?
• Board has at least one piece?
• Non-board features also identified
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 17: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/17.jpg)
From Features to Heuristics
Maximize single feature:
H(s) = 1 + R− + (R+− R−
− 2) ∗ V (s)
Or minimize single feature:
H(s) = 1 + R− + (R+− R−
− 2) ∗ [1 − V (s)]
• Example: Maximize white rook’s y-coordinate
• Actual win always better than heuristic value
• Actual loss always worse
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 18: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/18.jpg)
Complete System
During Start Clock:• Candidate heuristics constructed from GD• “Best” heuristic is chosen
• Old approach: parallel search• New approach: internal tournament
During Play Clock:• Iterative-deepening Minimax search
• Minimax search w/ αβ pruning• Transposition table and history heuristic• Extensions for > 2 players, simultaneous games
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 19: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/19.jpg)
Experiments
• Goal: Identify impact of game analysis• Three different games
• created by competition organizers
• Heuristic chosen manually• simulates good method to choose heuristic• no experimentation after initial selection
• Opponent: constant heuristic (exhaustive search)
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 20: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/20.jpg)
Nothello
Othello variant
• more corner squares
• opposite goal: finish with fewer markers
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 21: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/21.jpg)
Nothello
Othello variant
• more corner squares
• opposite goal: finish with fewer markers
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 22: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/22.jpg)
Nothello
Othello variant
• more corner squares
• opposite goal: finish with fewer markers
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 23: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/23.jpg)
Nothello
Othello variant
• more corner squares
• opposite goal: finish with fewer markers
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 24: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/24.jpg)
Nothello
Othello variant
• more corner squares
• opposite goal: finish with fewer markers
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 25: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/25.jpg)
Nothello
Heuristic: minimize number of own markers
NumMarkers: 10 NumMarkers: 8 NumMarkers: 8 NumMarkers: 9
H(s): 74.25 H(s): 79.20 H(s): 79.20 H(s): 76.73
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 26: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/26.jpg)
Nothello
Heuristic: minimize number of own markers
NumMarkers: 10 NumMarkers: 8 NumMarkers: 8 NumMarkers: 9
H(s): 74.25 H(s): 79.20 H(s): 79.20 H(s): 76.73
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 27: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/27.jpg)
Nothello
Heuristic: minimize number of own markers
NumMarkers: 10 NumMarkers: 8 NumMarkers: 8 NumMarkers: 9
H(s): 74.25 H(s): 79.20 H(s): 79.20 H(s): 76.73
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 28: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/28.jpg)
Nothello
Heuristic: minimize number of own markers
NumMarkers: 10 NumMarkers: 8 NumMarkers: 8 NumMarkers: 9
H(s): 74.25 H(s): 79.20 H(s): 79.20 H(s): 76.73
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 29: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/29.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 30: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/30.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 31: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/31.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 32: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/32.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 33: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/33.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 34: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/34.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 35: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/35.jpg)
Hallway
Chess board with two pawns
• Actions: move pawn or place wall
• Goal: reach other side first
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 36: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/36.jpg)
Hallway
Heuristic: maximize own pawn’s y-coordinate
25.50
13.25
1.00
37.75
50.00
62.25
74.50
86.75
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 37: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/37.jpg)
Farmers
Commodities trading game w/ three simultaneous players
Commodities
Structures
BarneyAlice Charlie
Heuristic: maximize own money
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 38: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/38.jpg)
Farmers
Commodities trading game w/ three simultaneous players
Commodities
Structures
BarneyAlice Charlie
Heuristic: maximize own money
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 39: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/39.jpg)
Results
• Experimental results
Game Matches Expected Wins Empirical Wins p
Nothello 15 7.5 15 10−5
Hallway 15 3 15 10−11
Farmers 25 8.3 11 0.234
• Competition Results• 2005: competitive but technical difficulties• 2006: very competitive (3rd place)
• after 72 matches, gap with first: ∼ 3 games
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 40: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/40.jpg)
Results
• Experimental results
Game Matches Expected Wins Empirical Wins p
Nothello 15 7.5 15 10−5
Hallway 15 3 15 10−11
Farmers 25 8.3 11 0.234
• Competition Results• 2005: competitive but technical difficulties• 2006: very competitive (3rd place)
• after 72 matches, gap with first: ∼ 3 games
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 41: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/41.jpg)
Conclusion and Future Work
• General Game Playing• Automate game analysis
• Automatic Heuristic Construction• Structures → Features → Heuristics
• Method incorporated into complete agent• Future Work
• Learn more complex evaluation functions• Understand game similarity• Transfer knowledge between games
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP
![Page 42: Automatic Heuristic Construction in a Complete General Game …pstone/Courses/394Rspring13/... · 2013. 3. 21. · Computer Game Playing • One of AI’s biggest success stories](https://reader036.fdocuments.net/reader036/viewer/2022071402/60f041c8833abd61704a1c6b/html5/thumbnails/42.jpg)
Conclusion and Future Work
• General Game Playing• Automate game analysis
• Automatic Heuristic Construction• Structures → Features → Heuristics
• Method incorporated into complete agent• Future Work
• Learn more complex evaluation functions• Understand game similarity• Transfer knowledge between games
G. Kuhlmann, K. Dresner, P. Stone – UT Austin Automatic Heuristic Construction in a Complete GGP