Game tree searchChapter 6 (6.1 to 6.3 and 6.6) cover games. 6.6 covers state of the art

game players in particular. 6.5 covers games that involve uncertainty.

Now, on to what we learned last time ....

Let’s practice by computing all the game theoretic values for nodes in this

tree.

A

A

A

A

B

B

B

Let’s practice by computing all the game theoretic values for nodes in this

tree.

A

A

A

A

B

B

B

0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2

Let’s practice by computing all the game theoretic values for nodes in this

tree.

A

A

A

A

B

B

B

0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2

0 3 2 0 1 -5 -3 2

A

A

A

A

B

B

B

0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2

0 3 2 0 1 -5 -3 2

0 2 0 1 -5 2

0 2 1 2

0 1

1

Question: if both players play rationally, what path will be followed through this tree?

A

A

A

A

B

B

B

0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2

0 3 2 0 1 -5 -3 2

0 2 0 1 -5 2

0 2 1 2

0 1

1

Question:what happens if

there are loops in the tree?

A

s1 s2s3

14 12 8

β = 8

2 4

α = 2, then 4, then ....

s4

s5

B

9 11

2

Example 1: We are currently expanding possible moves for

player A, from left to right. Which of the node expansions

above could we prune, and why?

A

s1 s2s3

14 12 8

β = 8

2 4

α = 9

s4

s5

B

9 11

2

Once we discover a node with value ‘9’, there is no need to

expand the nodes to the right!

B

s1 s2s3

6 2 7

α = 7

9 3

β = 9, then 3, then ....

s4

s5

A

4 2

8

Example 2: We are currently expanding possible moves for

player B, from left to right. Which of the node expansions

above could we prune, and why?

B

s1 s2s3

6 2 7

α = 7

9 3

β = 3

s4

s5

A

4 2

8

Once we discover a node with value ‘3’, there is no need to

expand the nodes to the right!

Which computations could we have avoided here? Assuming we expand

nodes left to right?

A

A

A

A

B

B

B

0 -3 3 -3 -2 2 -5 0 1 -3 -5 -3 2

0 3 2 0 1 -5 -3 2

0 2 0 1 -5 2

0 2 1 2

0 1

1

A

A

A

A

B

B

B

Effectiveness of alpha beta pruning• With no pruning, you have to explore O(bD) nodes, which makes the run time of a search with pruning the same as plain minimax.

• If, however, the move ordering for the search is optimal (meaning the best moves are searched first), the number of nodes we need to search using alpha beta pruning O(bD/2). That means you can, in theory, search twice as deep!

• In Deep Blue, they found that alpha beta pruning meant the average branching factor at each node was about 6 instead of 35.

This is an example of the best case scenario for alpha beta pruning. The effective branching factor of the first layer is B.

The effective branching of the second is 1. The effective layer of the third is B. And so on ....

A

AB

Rational Opponents• This, however, all assumes that your opponent is rational• e.g., will choose moves that minimize your score

• Storing your strategy is a potential issue:• you must store “decisions” for each node you can

reach by playing optimally• if your opponent has unique rational choices, this is

a single branch through game tree• if there are “ties”, opponent could choose any one of

the “tied” moves: which means you must store a strategy for each subtree

• What if your opponent doesn’t play rationally?• will it affect the quality of the outcome? will your

stored strategies work?

Heuristic evaluation functions in games

Some examples of heuristic evaluation

functions• Example for tic tac toe: h(n) = [# of 3 lengths that are left open for player A] - [# of 3 lengths that are left open for player B].

• Alan Turing’s function for chess: h(n) = A(n)/B(n) where A(n) is the sum of the point value for player A’s pieces and B(n) is the sum for player B.

• Most evaluation functions are specified as a weighted sum of features: h(n) = w1*feat1(n) + w2*feat2(n) + ... wi*feati(n).

• Deep Blue used about 6000 features in its evaluation function.

Heuristic evaluation functions in games

• Think of a few games and suggest some heuristics for estimating the “goodness” of a position• chess?• checkers?• your favorite video game?• “find the last parking spot”?

An example of realtime game tree search

1.We run A* (or our favorite search algorithm) until we are forced to make a move or run out of memory. Note: no leaves are goals yet.

2. We use evaluation function f(n) to decide which path looks best (let’s say it is the red one).

3. We take the first step along the best path (red), by actually making that move.

4. We restart search at the node we reach by making that move. (We may actually cache the results of the relevant part of first search tree if it’s hanging around, as it would with A*).

Some issues in real game searches

• How far should we search in our game tree to determine the value of a node, if we only have a fixed amount of time?

• What if we stop our search at a level in the search tree where subsequent moves dramatically change our evaluation?

• What if our opponent pushes this level off of the search horizon?

• Often, it makes sense to make the depth we search to dynamically decided.

Question: is there an alpha beta version you can use to search this

tree?