11

Artificial Intelligence

CS 165A

Tuesday, October 23, 2007

Adversarial search – games (Ch 6) Knowledge and reasoning (Ch 7)

1

22

Notes

• HW#2 posted tomorrow, due Tuesday

• Multiplayer games and minimax search…

3

Minimax search

• The minimax decision maximizes the utility under the assumption that the opponent seeks to minimize it (and uses the same evaluation function)

• Generate the tree of minimax values– Then choose best (maximum) move

– Don’t need to keep all values around Good memory property

• Depth-first search is used to implement minimax– Expand all the way down to leaf nodes

– Recursive implementation (Fig. 6.3)

3

4

Note: Your opponent is not always optimal

• What’s your best move?

1 7 2 5 2 8 9 4 6 3 49 57

A B C D1 2 4 3

4

Minimax says C but what if opponent isn’t perfectly rational?

5

Minimax properties

• Optimal?

• Complete?

• Time complexity?

• Space complexity?

Yes, against an optimal opponent

Yes, if the tree is finite

Exponential: O( bm )

Exponential: O( bm )

5

6

But this could take forever…

• For chess, b 35 and m 100 for “reasonable” games– So optimality, completeness are kind of irrelevant!

• Rather, cut the search off early and apply a heuristic evaluation function to the leaves– h(n) estimates the expected utility of the game from a given

position (node) n

• The performance of a game-playing program depends on the quality (and speed!) of its evaluation function

6

7

Heuristics (Evaluation function)

• Typical evaluation function for game: weighted linear function– h(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s)– weights · features [dot product]

• For example, in chess– W = { 1, 3, 3, 5, 8 }– F = { # pawns advantage, # bishops advantage, # knights

advantage, # rooks advantage, # queens advantage }– Is this what Deep Blue used?– What are some problems with this?

• More complex evaluation functions may involve learning– Adjusting weights based on outcomes– Perhaps non-linear functions– How to choose the features?

7

8

Cutting off search

• Suppose we have 100 seconds per move (e.g., in chess) and we can explore 104 nodes per second

• Options:– Stop search after 106 nodes

– Choose a depth d that will typically take < 100 sec

– Do iterative deepening search until allocated time runs out

• E.g., in chess:– bm = 106 , b = 35 m = 4 (can look 4 ply ahead)

– 4-ply: Human novice level (1.5M)

– 8-ply: Typical PC, good human (2.2T)

– 12-ply: Deep Blue, Kasparov (?) (3x1018)

8

9

Cutting off search (cont.)

• This strategy of cutting off search and applying a heuristic evaluation function does not always work well in practice

• What’s more, a blind application of the evaluation function can be disastrous– E.g., queen to be captured at ply N+1

• One solution to the cutoff problem: use quiescent search– Evaluation function is only valid for quiescent states (ones that are

likely to be reasonably steady in the near future)

– Have to search further to evaluate non-quiescent states (e.g., right before a capture)

– Watch out for the horizon effect

9

10

Pruning

• What’s really needed is “smarter,” more efficient search– Don’t expand “dead-end” nodes!

• Pruning – eliminating a branch of the search tree from consideration

• Alpha-beta pruning, applied to a minimax tree, returns the same “best” move, while pruning away unnecessary branches– Many fewer nodes might be expanded

– Hence, smaller effective branching factor

– …and deeper search

– …and better performance Remember, minimax is depth-first search

10

11

Alpha-beta pruning

• Consider traversing a minimax game tree in a depth-first manner

• General principle: Prune nodes (drop them and their descendants from consideration) if they cannot do better than previously generated nodes

• Alpha-beta search does a minimax search with pruning, using these parameters to describe bounds: - best choice so far for MAX - best choice so far for MIN

• Update and as search progresses– Prune remaining branches at a node if current value of node is less

than current value for MAX, or more than current value for MIN

11

12

Alpha-beta pruning (cont.)

12

If m is better than n for Player, neither it not its descendents will ever be a candidate solution (so they can be pruned.)

13

Alpha-beta example

13

A

B C D

3 12 8 2 1 13 14 5 2

[ ??, ?? ]

[ ??, ?? ] [ ??, ?? ] [ ??, ?? ]

Worst I could do at this node

Best I could do at this node

14

Alpha-beta example

14

A

B C D

3 12 8 2 1 13 14 5 2

[3, 3]

[3, 3] [-inf, 2] [2, 2]

15

Non-deterministic games

• Games that include an element of chance– E.g., Backgammon, Monopoly, Risk, Scrabble

In backgammon, roll s P(s) of dice determines legal moves– Human (i.e., imperfect!) opponent

• A deterministic minimax value is not possible– You do not know what a “perfect” opponent will do!– Instead of an exact minimax value, calculate an expected value of

a node (“Expectiminimax”) Weighted average of utility over all possible rolls of dice Concept of expected utility

s P(s) u(s), where P(s) is probability of state s u(s) is utility of state s.

Start with actual utilities of terminal nodes

16

Non-deterministic games

• How to represent– Add “chance” nodes to tree with MAX and MIN nodes

– Represent possible paths (e.g., roll of the dice)

– Calculate the average utility over all the possible chance events

– Numerical scale of utility function is important

• Aside: Complex deterministic games can be modeled as deterministic games with chance nodes

17

Schematic diagram of backgammon

MAX

CHANCE

MIN

CHANCE

MAX

Possible moves

Possible dice rolls

Possible opponent moves

Possible dice rolls

18

Expectiminimax calculation

P(s1) = 0.3

P(s2) = 0.2

P(s3) = 0.5

Expectiminimax (A) = 0.3*6 + 0.2*4 + 0.5*5 = 5.1

Expectiminimax (B) = 0.3*2 + 0.2*2 + 0.5*2 = 2.0

Choose move A

A B

6 4 5 2 2 2

0.3 0.2 0.5 0.3 0.2 0.5

19

Expectiminimax calculation

P(s1) = 0.3

P(s2) = 0.2

P(s3) = 0.5

Expectiminimax(A) = 0.3*6 + 0.2*4 + 0.5*5 = 5.1

Expectiminimax(B) = 0.3*2 + 0.2*9 + 0.5*9 = 6.9

Choose move B

A B

6 4 5 2 9 9

0.3 0.2 0.5 0.3 0.2 0.5

20

Examples

• Tomorrow’s discussion sessions will include examples of minimax, alpha-beta pruning, and expectiminimax

20

21

Knowledge and Reasoning

Logic

22

Reminder: Three parts of an AI Program?

• AI programs can generally be thought of as comprising three separated parts– Data / knowledge (“knowledge base”)

– Operations / rules (“production rules”)

– Control

• What are these three parts in M&C, TTT, vacuum world?

• We need to consider how to represent knowledge and how to reason about what we know (with respect to what we want)\– More high-level and flexible representations and reasoning

abilities

23

Knowledge and reasoning

• We want more powerful methods for – Representing Knowledge – more general methods for representing

facts about the world and how to act in world

– Carrying out Reasoning – more general methods for deducing additional information and a course of action to achieve goals

– Focus on knowledge and reasoning rather than states and search Note that search is still critical

• This brings us to the idea of logic

2424

Logic – the basis of reasoning

• Logics are formal languages for representing information such that conclusions can be drawn– I.e., to support reasoning

• Syntax defines the sentences in the language– Allowable symbols– Rules for constructing grammatically correct sentences

• Semantics defines the meaning of sentences – Correspondence (isomorphism) between sentences and facts in the

world– Gives an interpretation to the sentence– Sentences can be true or false with respect to the semantics

• Proof theory specifies the reasoning steps that are sound (the inference procedure)

25

Knowledge-based agents

• A knowledge-based agent uses reasoning based on prior and acquired knowledge in order to achieve its goals

• Two important components:– Knowledge Base (KB)

Represents facts about the world (the agent’s environment)– Fact = “sentence” in a particular knowledge representation

language (KRL) KB = set of sentences in the KRL

– Inference Engine – determines what follows from the knowledge base (what the knowledge base entails)

Inference / deduction– Process for deriving new sentences from old ones

Sound reasoning from facts to conclusions

26

Knowledge base (KB) for agents

• The KB agent must be able to – Represent states, actions, etc.– Incorporate new percepts– Update the internal representation of the world– Deduce hidden properties of the world– Deduce appropriate actions, given the goals

• Can be viewed at different levels of explanation– Knowledge level

Abstract. What does the agent know? What might you TELL or ASK the agent? How does the agent reason?

– Logical level Sentences in the KRL, inference rules

– Implementation level Data structures, implementation efficiency

27

Basic approach to representing KB agents

• Basic activities that must be supported:– KB is consulted and updated when making percepts

about world

– KB is consulted and updated when making choices of actions in world (use of reasoning mechanisms)

– KB is consulted and updated in response to perceived changes following actions

• All represented in logical terms

28

Knowledge Base

Inference engine

Domain specific content; facts

ASK

TELL

Domain independent algorithms; can deduce new facts from the KB

KB Agent