University College Cork (Ireland) Department of Civil and Environmental Engineering Course:...

University College Cork (Ireland) Department of Civil and Environmental Engineering

Course:Engineering Artificial Intelligence

Dr. Radu Marinescu

Lecture 5


Today’s class

o Game-playing

R. Marinescu October-2009 page 2


Overview

o Computer programs which play 2-player games• game-playing as search• with the complication of an opponent

o General principles of game-playing and search• evaluation functions• mini-max principle• alpha-beta-pruning• heuristic techniques

o Status of Game-Playing Systems• in chess, checkers, backgammon, Othello, etc, computers routinely defeat

leading world playerso Applications?

• think of “nature” as an opponent• economics, war-gaming, medical drug treatment



History


1949 – Shannon paper1951 – Turing paper1958 – Bernstein program55-60 – Simon-Newell program (α – β McCarthy?)1961 – Soviet program66-67 – MacHack 6 (MIT AI Lab)70’s – NW Chess 4.580’s – Crazy Blitz90’s – Belle, Hitech, Deep Thought, Deep Blue


Solving 2-player games

o Two players, perfect informationo Examples:

• e.g., chess, checkers, tic-tac-toeo Configuration of the board = unique

arrangement of “pieces”

o Games as a Search Problem• States = board configurations• Operators = legal moves• Initial State = current configuration• Goal = winning configuration• payoff function = gives numerical value of outcome of the game



Game tree search

o Game tree: encodes all possible games

o We are not looking for a path, only the next move to make (that hopefully leads to a winning position)

o Our best move depends on what the other player does



Move generation



Example: partial game tree for Tic-Tac-Toe


MAX (O)

MIN (X)

MAX (O)

MIN (X)


Scoring function


We will use the same scoring function for both players, simply negating the values to represent the opponent's scores.


Example: scoring function for Tic-Tac-Toe


Leaves are either win (+1), loss (-1) or draw (0)


Min-Max: an optimal procedure

o Designed to find the optimal strategy for MAX player and find best move:

• 1. Generate the whole game tree to leaves• 2. Apply scoring (payoff) function to leaves• 3. Back-up values from leaves toward the root:

a MAX node computes the max of its child values a MIN node computes the min of its child values

• 4. When value reaches the root: choose max value and the corresponding move.



Min-Max



Properties of Min-Max

o Complete? Yes (if tree is finite)o Optimal? Yes (against an optimal opponent)o Time complexity? O(bd)o Space complexity? O(bd) (depth-first exploration)

o For chess, b ≈ 35, m ≈100 for "reasonable" games exact solution completely infeasible

• Chess: b ~ 35 (average branching factor) d ~ 100 (depth of game tree for typical game) bd ~ 35100 ~10154 nodes!!

• Tic-Tac-Toe ~5 legal moves, total of 9 moves 59 = 1,953,125 9! = 362,880 (Computer goes first) 8! = 40,320 (Computer goes second)



Min-Max

o Naïve Min-Max assumes that it is possible to search the full tree, where the scoring function of the leaves is either win (+1), loss (-1) or draw (0)

o However: For most games, it is impossible to develop the whole search tree

o Instead develop part of the tree (up to a certain depth or a number of plys) and evaluate promise of leaves using a static evaluation function.



Static evaluation function

o The static evaluation function:• Estimates how good the current board configuration is for a player.• Typically, one figures how good it is for the player, and how good it

is for the opponent, and subtracts the opponents score from the players

• Othello: Number of white pieces - Number of black pieces• Chess: Value of all white pieces - Value of all black pieces

o Typical values from -infinity (loss) to +infinity (win) or [-1, +1].

o If the board evaluation is X for a player, it’s -X for the opponent

o Example: • Evaluating chess boards, Checkers, Tic-tac-toe



Evaluation functions: chess


For Chess, typically a linear weighted sum of features

n

iiinn sfwsfwsfwsfwsEval

12211 )()(...)()()(

e.g.,)queensblack ofnumber ()queens whiteofnumber ()(

9

1

1

sf

w weight of the piece (Q = 9, P=1, K=3, B=3.5, R=5)


Evaluation functions: Tic-Tac-Toe


nnE

nO

nM

nOnMnE

statefor evaluation total theis )(

(O) lines winningpossible sOpponent' of total theis )(

(X) lines winningpossibleMy of total theis )( where

)()()(

Another example

E(n) =


Backup values


Two-ply minimax applied to the opening move of tic-tac-toe


Backup values


Two-ply minimax applied to the X’s second move


Backup values


Two-ply minimax applied to X’s move near end game


Impact of tree depth



Deep Blue

o 32 SP2 processors• Each with 8 dedicated chess processors = 256 CP

o 50 – 100 billion moves in 3 min• 13-30 ply search



α – β pruning

o In Min-Max there is a separation between node generation and evaluation.


Backup Values


α – β pruning

o Idea: • Do depth-first search to generate partial game tree• Give static evaluation function to leaves• Compute bound on internal nodes

o Alpha (α), Beta (β) bounds:• α value for MAX node means that max real value is at least alpha• β for MIN node means that MIN can guarantee a value below beta

o Computation:• Alpha of a MAX node is the maximum value of its seen children• Beta of a MIN node is the lowest value seen of its child node



When to prune

o Pruning

• Below a MIN node whose beta value is lower than or equal to the alpha value of its ancestors.

• Below a MAX node having an alpha value greater than or equal to the beta value of any of its MIN nodes ancestors.



α – β pruning



Example



α – β properties

o Guaranteed same value as Min-Max

o In a perfectly ordered tree, expected work is O(bd/2), vs O(bd) for Min-Max, so can search twice as deep with the same effort

o With good move orderings, the actual running time is close to the optimistic estimate



Game program

o Move generator (ordered moves) 50%o Static evaluation 40%o Search control 10%


OpeningsEnd games

databases

[all in place by late 60’s]


Move generator

o Legal moveso Ordered by

• Most valuable victim• Least valuable aggressor

o Killer heuristics



Static evaluation

o Initially - very complex

o 70’s - very simple (material)

o Now - Deep searchers: moderately complex

(hardware)

PC programs: elaborate, hand tuned



Other games

o Backgammon• Involves randomness – dice rolls• Machine-learning based player was able to draw the world

champion human playero Bridge

• Involves hidden information – other players’ cards – and communication during bidding

• Computer players play well but do not bid wello Go

• No new elements but huge branching factor• No good computer players exist



Observations

o Computers excel in well-defined activities where rules are clear

• Chess• Mathematics

o Success comes after a long period of gradual refinement



Summary

o Game playing is best modeled as a search problem

o Game trees represent alternate computer/opponent moves

o Evaluation functions estimate the quality of a given board configuration for the MAX player.

o Min-Max is a procedure which chooses moves by assuming that the opponent will always choose the move which is best for them

o Alpha-Beta is a procedure which can prune large parts of the search tree and allow search to go deeper

o For many well-known games, computer algorithms based on heuristic search match or out-perform human world experts

o Reading: R&N Chapter 6


University College Cork (Ireland) Department of Civil and Environmental Engineering Course:...

Documents

Transcript of University College Cork (Ireland) Department of Civil and Environmental Engineering Course:...