Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory...

73
Computer Science Universite itMaastric ht Institute for Knowledge and Agent Technology Games, Theory and Application Games, Theory and Application Jaap van den Herik and Jeroen Donkers Jaap van den Herik and Jeroen Donkers Institute for Knowledge and Agent Technology, Institute for Knowledge and Agent Technology, IKAT IKAT Department of Computer Science Department of Computer Science Universiteit Maastricht Universiteit Maastricht The Netherlands The Netherlands SOFSEM 2004 SOFSEM 2004 Prague, Hotel VZ Merin Prague, Hotel VZ Merin Sunday, January 25 Sunday, January 25 8.30-10.00h 8.30-10.00h

Transcript of Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory...

Page 1: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

ComputerScience

UniversiteitMaastricht

Institute for Knowledgeand Agent Technology

Games, Theory and ApplicationGames, Theory and Application

Jaap van den Herik and Jeroen DonkersJaap van den Herik and Jeroen Donkers

Institute for Knowledge and Agent Technology, IKATInstitute for Knowledge and Agent Technology, IKAT

Department of Computer ScienceDepartment of Computer Science

Universiteit MaastrichtUniversiteit Maastricht

The NetherlandsThe Netherlands

SOFSEM 2004 SOFSEM 2004 Prague, Hotel VZ MerinPrague, Hotel VZ Merin

Sunday, January 25 Sunday, January 25 8.30-10.00h8.30-10.00h

Page 2: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

2Sofsem'04

ContentsContents

• Games in Artificial-Intelligence ResearchGames in Artificial-Intelligence Research• Game-tree search principlesGame-tree search principles• Using opponent modelsUsing opponent models• Opponent-Model searchOpponent-Model search• Gains and risksGains and risks• Implementing OM search: complexitiesImplementing OM search: complexities• Past future of computer chessPast future of computer chess

Page 3: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

3Sofsem'04

Games, such as Chess, Checkers, and Go Games, such as Chess, Checkers, and Go

are an attractive are an attractive paspastime andtime and

scientifically interestingscientifically interesting

Page 4: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

4Sofsem'04

ChessChess

• Much research has been performed in Computer Chess Much research has been performed in Computer Chess • Deep Blue (IBM) defeated the world champion Deep Blue (IBM) defeated the world champion

Kasparov in 1997Kasparov in 1997• A Micro Computer better than the world champion?A Micro Computer better than the world champion?

Page 5: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

5Sofsem'04

World Champion ProgramsWorld Champion Programs• KAISSAKAISSA 19741974 StockholmStockholm• CHESSCHESS 19771977 TorontoToronto• BELLEBELLE 19801980 LinzLinz• CRAY BLITZCRAY BLITZ 19831983 New YorkNew York• CRAY BLITZ CRAY BLITZ 19861986 KeulenKeulen• DEEP THOUGHTDEEP THOUGHT 19891989 EdmontonEdmonton• REBELREBEL 19921992 MadridMadrid• FRITZFRITZ 19951995 Hong KongHong Kong• SHREDDERSHREDDER 19991999 PaderbornPaderborn• JUNIORJUNIOR 20022002 MaastrichtMaastricht• SHREDDERSHREDDER 20032003 GrazGraz• ?? 20042004 Ramat-GanRamat-Gan

Page 6: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

6Sofsem'04

International DraughtsInternational Draughts

• Buggy best draughts programBuggy best draughts program• Human better than computer, but the margin is smallHuman better than computer, but the margin is small• Challenge: More knowledge in programChallenge: More knowledge in program

Page 7: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

7Sofsem'04

GoGo

• Computer Go programs are weakComputer Go programs are weak• Problem: recognition of patterns Problem: recognition of patterns • Top Go programs: Go4++, Many Faces of Go, Top Go programs: Go4++, Many Faces of Go,

GnuGo, and HandtalkGnuGo, and Handtalk

Page 8: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

8Sofsem'04

AwariAwari

• A Mancala game (pebble-hole game)A Mancala game (pebble-hole game)• The game is a drawThe game is a draw• Solved by John Romein and Henri Bal (VU Amsterdam) 2002 Solved by John Romein and Henri Bal (VU Amsterdam) 2002 • All positions (900 billion) with a cluster computer of 144 1GHz processors and 72Gb All positions (900 billion) with a cluster computer of 144 1GHz processors and 72Gb

ram computed in 51 hourram computed in 51 hour• Proven that even the best computer programs still make many errors in their Proven that even the best computer programs still make many errors in their

gamesgames

Page 9: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

9Sofsem'04

ScrabbleScrabble

• Maven beats every human opponentMaven beats every human opponent• Author Brian SheppardAuthor Brian Sheppard• Ph.D. (UM): Ph.D. (UM): “Towards Perfect Play of Scrabble” (02)“Towards Perfect Play of Scrabble” (02)

• Maven can play scrabble in any languageMaven can play scrabble in any language

Page 10: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

10Sofsem'04

OverviewOverview

Page 11: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

11Sofsem'04

Computer OlympiadComputer Olympiad

• Initiative of David Levy (1989)Initiative of David Levy (1989)• Since 1989 there have been 8 olympiads; 4x Since 1989 there have been 8 olympiads; 4x

Londen, 3x Maastricht, 1x GrazLonden, 3x Maastricht, 1x Graz• Goal: Goal:

• Finding the best computer program for each gameFinding the best computer program for each game• Connecting programmers / researchers of different games Connecting programmers / researchers of different games

• Computers play each other in competition Computers play each other in competition • Demonstrations:Demonstrations:

• Man versus MachineMan versus Machine• Man + Machine versus Man + Machine Man + Machine versus Man + Machine

Page 12: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

12Sofsem'04

Computer versus ComputerComputer versus Computer

Page 13: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

13Sofsem'04

Computer OlympiadComputer Olympiad

• Last time in Graz 2004Last time in Graz 2004• Also World Championship Computer Chess and Also World Championship Computer Chess and

World Championship BackgammonWorld Championship Backgammon• 80 particpants in several categories80 particpants in several categories• Competitions in olympiad’s history:Competitions in olympiad’s history:

Abalone, Awari, Amazons, Backgammon, Bao, Abalone, Awari, Amazons, Backgammon, Bao, Bridge, Checkers, Chess, Chinese Chess, Dots and Bridge, Checkers, Chess, Chinese Chess, Dots and Boxes, Draughts, Gipf, Go-Moku, 19x19 Go, 9x9 Go, Boxes, Draughts, Gipf, Go-Moku, 19x19 Go, 9x9 Go, Hex, Lines of Action, Poker, Renju, Roshambo, Hex, Lines of Action, Poker, Renju, Roshambo, Scrabble, and ShogiScrabble, and Shogi

Page 14: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

14Sofsem'04

UM Programs on the Computer OlympiadsUM Programs on the Computer Olympiads

Bao 1.0

Gipfted - Gipf

Anky - Amazons

MIA - LOA

Magog - Go

Page 15: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

15Sofsem'04

Computer Game-playingComputer Game-playing

• Can computers beat humans in board games Can computers beat humans in board games like Chess, Checkers, Go, Bao?like Chess, Checkers, Go, Bao?

• This is one of the first tasks of Artificial This is one of the first tasks of Artificial Intelligence (Shannon 1950)Intelligence (Shannon 1950)

• Successes obtained in Chess (Deep Blue), Successes obtained in Chess (Deep Blue), Checkers, Othello, Draughts, Backgammon, Checkers, Othello, Draughts, Backgammon, Scrabble...Scrabble...

Page 16: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

16Sofsem'04

Computer Game-PlayingComputer Game-Playing

• Sizes of game trees:Sizes of game trees:• Nim-5: Nim-5: 28 nodes28 nodes• Tic-Tac-Toe:Tic-Tac-Toe: 10 1055 nodes nodes• Checkers: Checkers: 10 103131 nodes nodes• Chess:Chess: 10 10123123 nodes nodes• Go: Go: 10 10360360 nodes nodes

• In practice it is intractable to find a solution In practice it is intractable to find a solution with minimax: so use with minimax: so use heuristicheuristic search search

Page 17: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

17Sofsem'04

Three TechniquesThree Techniques

• Minimax SearchMinimax Search

• α-β Searchα-β Search

• Opponent ModellingOpponent Modelling

Page 18: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

18Sofsem'04

Game-playingGame-playing

• Domain:Domain: two-player zero-sum game with two-player zero-sum game with perfect information (Zermelo, 1913)perfect information (Zermelo, 1913)

• TaskTask: find best response to opponent’s moves, : find best response to opponent’s moves, provided that the opponent does the sameprovided that the opponent does the same

• SolutionSolution: Minimax procedure (von Neumann, : Minimax procedure (von Neumann, 1928)1928)

Page 19: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

19Sofsem'04

Minimax SearchMinimax Search

Players remove on turn 1, 2, or 3 matches. The player who takes the last match wins the game.

Nim-5

Page 20: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

20Sofsem'04

Minimax SearchMinimax Search

+1 +1 +1 +1 +1 +1 +1–1 –1 –1 –1 –1 –1

4 3 2

2 1 0 1 0 0

0

1 0 0 0

1

1

1

1

1

1

1

1 1 1 1

11

2

2 2 2

2 2

2

2

3

3

33

1 1

5

03

1 0 0 0 0

2 1 2 1 0 1

Page 21: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

21Sofsem'04

Minimax SearchMinimax Search

1

11 1 1

+1 –1 –1

–1 –1 +1 –1 +1 +1

+1

–1 +1 +1 +1

1

1

1 1 1 1 1 1

11

2

2 2 2

2 2

2

2

3

3

33

+1 +1 +1 +1 +1 +1 +1–1 –1 –1 –1 –1 –1

+1

–1+1

+1 –1 –1 –1 –1

+1 +1 +1 +1 –1 +1

MINIMAXING

Page 22: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

22Sofsem'04

PruningPruning

• You do not need the total solution to play You do not need the total solution to play (well): only the move at the root is needed(well): only the move at the root is needed

• Methods exist to find this move without a need Methods exist to find this move without a need for a total solution: for a total solution: pruningpruning• Alpha-beta search (Knuth & Moore ‘75)Alpha-beta search (Knuth & Moore ‘75)• Proof-number search (Allis et al. ‘94), etc.Proof-number search (Allis et al. ‘94), etc.

• Playing is solving a sequence of game treesPlaying is solving a sequence of game trees

Page 23: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

23Sofsem'04

Heuristic SearchHeuristic Search

• Truncate the game tree (depth + selection)

• Use a (static heuristic) evaluation function at the leaves to replace pay-offs

• Miminax on the reduced game tree

0.25 –1 –1

1

1 11

2

2 22

3

33

0.25

–10.25 0.33 0.5 0.33 0.5 –1 0.5

Page 24: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

24Sofsem'04

Heuristic SearchHeuristic Search• The approach works very well in Chess, The approach works very well in Chess,

but...but...• Is solving a sequence of reduced games the Is solving a sequence of reduced games the

bestbest way to win a game? way to win a game?• Heuristic values are used instead of pay-offsHeuristic values are used instead of pay-offs• Additional information in the tree is unusedAdditional information in the tree is unused• The opponent is not taken into accountThe opponent is not taken into account

• We aim at the last item: We aim at the last item: opponentsopponents

Page 25: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

25Sofsem'04

MinimaxMinimax

3 4 2 7

3 2

[3]

Page 26: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

26Sofsem'04

α-β algorithmα-β algorithm

3 4 2 7

3

3

2

β-pruning

Page 27: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

27Sofsem'04

The strength of α-β The strength of α-β

3 4 2

More than thousand prunings

Page 28: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

28Sofsem'04

The importance of α-β algorithmThe importance of α-β algorithm

3 4 2

β-pruning

3

3

Page 29: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

29Sofsem'04

THE NUMBER OF DIFFERENT, REACHABLETHE NUMBER OF DIFFERENT, REACHABLE

POSITIONS IN CHESS ISPOSITIONS IN CHESS IS

(CHINCHALKAR): 10(CHINCHALKAR): 104646

The Possibilities Of ChessThe Possibilities Of Chess

Page 30: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

30Sofsem'04

A Clever Algorithm (α-β)A Clever Algorithm (α-β)SAVES THE SQAURE ROOT OF THE NUMBER OF POSSIBILITIES, SAVES THE SQAURE ROOT OF THE NUMBER OF POSSIBILITIES, N, THIS IS MORE N, THIS IS MORE THANTHAN

99,9999999999999999999999999999999999999999999999999999,9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999%999999999999999999999999%

Page 31: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

31Sofsem'04

A CalculationA Calculation

NUMBER OF POSSIBILITIES:NUMBER OF POSSIBILITIES: 10104646

SAVINGS BY α-Β ALGORITHM:SAVINGS BY α-Β ALGORITHM: 10102323

1000 PARALLEL PROCESSORS:1000 PARALLEL PROCESSORS: 101033

POSITIONS PER SECOND:POSITIONS PER SECOND: 101099

LEADS TO: 10LEADS TO: 1023-1223-12 = = 10101111 SECONDS SECONDS

A CENTURY ISA CENTURY IS 101099 SECONDS SECONDS

SOLVING CHESS:SOLVING CHESS: 101022 CENTURIES CENTURIES

SO 100 CENTURIES OR 10,000 YEARSO 100 CENTURIES OR 10,000 YEAR

Page 32: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

32Sofsem'04

Using opponent’s strategy (NIM-5)Using opponent’s strategy (NIM-5)

“Player 1 never takes 3”

1

11 1 1

–1 –1

–1 –1 –1 +1 +1

+1

–1 +1 +1

1

1

1 1 1 1 1 1

11

2

2 2 2

2 2

2

2

3

3

33

–1

–1

+1 –1 –1 –1 –1

+1 +1 +1 +1 –1

Page 33: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

33Sofsem'04

Using opponent’s strategyUsing opponent’s strategy

• Well known Tic-Tac-Toe strategy:Well known Tic-Tac-Toe strategy:R1: make 3-in-a-row if possibleR1: make 3-in-a-row if possible

R2: prevent opponent’s 3-in-a-row R2: prevent opponent’s 3-in-a-row

if possibleif possible

H1: occupy central square if possibleH1: occupy central square if possible

H2: occupy corner square if possibleH2: occupy corner square if possible

knows that uses this strategy

Page 34: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

34Sofsem'04

Opponent-Model searchOpponent-Model searchIida, vd Herik, Uiterwijk, Herschberg (1993)Iida, vd Herik, Uiterwijk, Herschberg (1993)

Carmel and Markovitch (1993)Carmel and Markovitch (1993)

• Opponent’s evaluation function is known Opponent’s evaluation function is known ((unilateralunilateral: the opponent uses minimax): the opponent uses minimax)

• This is the opponent modelThis is the opponent model• It is used to predict opponent’s movesIt is used to predict opponent’s moves• Best response is determined, using the own Best response is determined, using the own

evaluation functionevaluation function

Page 35: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

35Sofsem'04

OM SearchOM Search

• Procedure:Procedure:• At opponent’s nodesAt opponent’s nodes: use minimax (alpha-beta) : use minimax (alpha-beta)

to predict the opponent’s move. Return the own to predict the opponent’s move. Return the own value of the chosen movevalue of the chosen move

• At own nodesAt own nodes: Return (the move with) the highest : Return (the move with) the highest valuevalue

• At leavesAt leaves: Return the own evaluation: Return the own evaluation

• Implementation: Implementation: one-passone-pass or or probingprobing

Page 36: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

36Sofsem'04

OM Search EquationsOM Search Equations

Page 37: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

37Sofsem'04

OM Search ExampleOM Search Example

7

6878

7 6

V0: 6Vop: 7

V0: 8Vop: 6

V0: 7Vop: 6

V0: 8Vop: 9

v0: 7vop: 6

v0: 8vop: 6

v0: 8vop: 6

Page 38: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

38Sofsem'04

OM Search Algorithm (probing)OM Search Algorithm (probing)

Page 39: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

39Sofsem'04

Risks and Gains in OM searchRisks and Gains in OM search

• Gain: difference between the predicted move Gain: difference between the predicted move and the minimax moveand the minimax move

• Risk: difference between the move we expect Risk: difference between the move we expect and the move the opponent really selectsand the move the opponent really selects

• Prediction of the opponent is important, and Prediction of the opponent is important, and depends on the quality of opponent model.depends on the quality of opponent model.

Page 40: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

40Sofsem'04

Additional riskAdditional risk

• Even if the prediction is correct, there are Even if the prediction is correct, there are traps for OM searchtraps for OM search

• Let P be the set of positions that the opponent Let P be the set of positions that the opponent selects, vselects, v00 be our evaluation function and v be our evaluation function and vopop the opponent’s function.the opponent’s function.

Page 41: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

41Sofsem'04

Four types of errorFour types of error

• VV00 overestimates a position in P (bad) overestimates a position in P (bad)

• VV00 underestimates a position in P underestimates a position in P

• VVopop underestimates a position that enters P underestimates a position that enters P (good for us)(good for us)

• VVopop overestimates a position in P overestimates a position in P

Page 42: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

42Sofsem'04

6

26

2876

MAX

MAX

MIN

V0 = 6Vop = 6

V0 = 7Vop = 7

V0 = 8Vop = 8

V0 = 2Vop = 2

v0 = 2vop = 2mmx = 2obt = 2

v0 = 6vop = 6mmx = 6obt = 6

v0 = 6vop = 6mmx = 6obt = 6

Base case

Vo: our evaluationVop: opponent’s evaluationMmx: minimax valueObt: obtained value

Page 43: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

43Sofsem'04

6

26

2876

MAX

MAX

MIN

V0 = 6Vop = 6

V0 = 7Vop = 7

V0 = 8Vop = 8

V0 = 9Vop = 2

v0 = 9vop = 2mmx = 8obt = 2

v0 = 6vop = 6mmx = 6obt = 6

v0 = 9vop = 6mmx = 8obt = 2

Type-I error

Vo: our evaluationVop: opponent’s evaluationMmx: minimax valueObt: obtained value

Page 44: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

44Sofsem'04

6

26

2876

MAX

MAX

MIN

V0 = 6Vop = 6

V0 = 7Vop = 7

V0 = 8Vop = 8

V0 = 9Vop = 9

v0 = 8vop = 8mmx = 8obt = 8

v0 = 6vop = 6mmx = 6obt = 6

v0 = 8vop = 8mmx = 8obt = 8

Type-I errorvanished

Vo: our evaluationVop: opponent’s evaluationMmx: minimax valueObt: obtained value

Page 45: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

45Sofsem'04

6

26

2876

MAX

MAX

MIN

V0 = 1Vop = 6

V0 = 7Vop = 7

V0 = 8Vop = 8

V0 = 2Vop = 2

v0 = 2vop = 2mmx = 2obt = 2

v0 = 1vop = 6mmx = 1obt = 6

v0 = 2vop = 6mmx = 2obt = 2

Type-II error

Vo: our evaluationVop: opponent’s evaluationMmx: minimax valueObt: obtained value

Page 46: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

46Sofsem'04

6

26

2876

MAX

MAX

MIN

V0 = 6Vop = 6

V0 = 7Vop = 7

V0 = 8Vop = 1

V0 = 2Vop = 2

v0 = 8vop = 1mmx = 2obt = 8

v0 = 6vop = 6mmx = 6obt = 6

v0 = 8vop = 6mmx = 6obt = 8

Type-III error

Vo: our evaluationVop: opponent’s evaluationMmx: minimax valueObt: obtained value

Page 47: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

47Sofsem'04

6

26

2876

MAX

MAX

MIN

V0 = 6Vop = 8

V0 = 7Vop = 7

V0 = 8Vop = 8

V0 = 2Vop = 2

v0 = 2vop = 2mmx = 2obt = 2

v0 = 7vop = 7mmx = 6obt = 7

v0 = 7vop = 7mmx = 6obt = 7

Type-IV error

Vo: our evaluationVop: opponent’s evaluationMmx: minimax valueObt: obtained value

Page 48: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

48Sofsem'04

Pruning in OM SearchPruning in OM Search

• Pruning at MAX nodes is not possible in OM Pruning at MAX nodes is not possible in OM search, only pruning at MIN nodes can take search, only pruning at MIN nodes can take place (Iida place (Iida et alet al, 1993)., 1993).

• Analogous to α-β search, the pruning version is Analogous to α-β search, the pruning version is called β-pruning OM searchcalled β-pruning OM search

Page 49: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

49Sofsem'04

Pruning ExamplePruning Example a

gfed

b c57

78

78

on

h

qp

i

sr

j

ut

k

wv

l m

57

68

- 8

78

- 9

-9

74

57

46

57

-8

74

68

-9

-5

-9

-10

46

-8

-9

yx-8

-9

Page 50: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

50Sofsem'04

Two ImplementationsTwo Implementations

• One-pass: One-pass: • visit every node at most oncevisit every node at most once• back-up both your own and the opponent’s value back-up both your own and the opponent’s value

• Probing:Probing:• At MIN nodes use α-β search to predict the At MIN nodes use α-β search to predict the

opponent’s moveopponent’s move• back-up only one valueback-up only one value

Page 51: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

51Sofsem'04

One-Pass β-pruning OM searchOne-Pass β-pruning OM search

Page 52: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

52Sofsem'04

Probing β-pruning OM searchProbing β-pruning OM search

Page 53: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

53Sofsem'04

Best-case time complexityBest-case time complexity

Page 54: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

54Sofsem'04

Best-case time complexityBest-case time complexity

COM = Subtree A

C’OM = Subtree B

Page 55: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

55Sofsem'04

Best-case time complexityBest-case time complexity

• The time complexity for the one-pass version is The time complexity for the one-pass version is equal to that of the theoretical bestequal to that of the theoretical best

• The time complexity for the probing version is The time complexity for the probing version is different:different:• first detect the number of leaf first detect the number of leaf

evaluations needed:evaluations needed:• Then find the number of probes needed:Then find the number of probes needed:

Page 56: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

56Sofsem'04

Best-case time complexityBest-case time complexity

In the best case, pruning in the probes is not changed by β=v+1:all probes take the same number of evaluations as with a fully open window.

Page 57: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

57Sofsem'04

Best-case time complexityBest-case time complexity

Page 58: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

58Sofsem'04

Time complexities comparedTime complexities compared

(Branching factors 4, 8, 12, 16, 20)

Page 59: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

59Sofsem'04

Average-case time complexityAverage-case time complexity

(Measured on random game trees)

Page 60: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

60Sofsem'04

Probabilistic Probabilistic Opponent-model SearchOpponent-model Search

Donkers, vd Herik, Uiterwijk (2000)Donkers, vd Herik, Uiterwijk (2000)

• More elaborate opponent model:More elaborate opponent model:• Opponent uses a Opponent uses a mixedmixed strategy: strategy: nn different different

evaluation functions (opponent types) plus a evaluation functions (opponent types) plus a probability distributionprobability distribution

• It is assumed to It is assumed to approximateapproximate the true opponent’s the true opponent’s strategystrategy

• Own evaluation function is one of the opponent typesOwn evaluation function is one of the opponent types

Page 61: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

61Sofsem'04

PrOM SearchPrOM Search• Procedure:Procedure:

• At opponent’s nodesAt opponent’s nodes: use minimax (alpha-: use minimax (alpha-beta) to predict the opponent’s move beta) to predict the opponent’s move for all for all opponent typesopponent types separately. Return the separately. Return the expected expected own value of the chosen movesown value of the chosen moves

• At own nodesAt own nodes: Return (the move with) the : Return (the move with) the highest valuehighest value

• At leafsAt leafs: Return the own evaluation: Return the own evaluation

• Implementation: Implementation: one-passone-pass or or probingprobing

Page 62: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

62Sofsem'04

PrOM Search EquationsPrOM Search Equations

Page 63: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

63Sofsem'04

PrOM Search ExamplePrOM Search Example

7

6878

7 6

0: 61: 10

0: 81: 8

0: 71: 6

0: 81: 7

v0: 8 v0: 7 v0: 8 v0: 6

0: 61: 8

0: 71: 6

Pr(0) = 0.3Pr(1) = 0.7

0: 71: 8

0.30.7

v0: 0.3 x 6 + 0.7 x 8 = 7.4v0: 7

0.0 1.0

v0: 7.4

Page 64: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

64Sofsem'04

PrOM Search AlgorithmPrOM Search Algorithm

Page 65: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

65Sofsem'04

How did chess players envision the How did chess players envision the future of non-human chess players?future of non-human chess players?

- Euwe- Euwe- Donner- Donner- De Groot- De Groot- Timman- Timman- Sosonko- Sosonko- Böhm- Böhm

Question: Do you think that a computer will be Question: Do you think that a computer will be able to play at grandmaster level?able to play at grandmaster level?

Page 66: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

66Sofsem'04

Euwe (June 10, 1980)Euwe (June 10, 1980)

““I don’t believe so.I don’t believe so.

I am almost convinced of that, too”I am almost convinced of that, too”

Page 67: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

67Sofsem'04

Timman (June 22, 1980):Timman (June 22, 1980):

““If you would express it in ELO-points, than I believe If you would express it in ELO-points, than I believe that a computer will not be able to obtain more thanthat a computer will not be able to obtain more than

...... ......

let’s say let’s say ...... ......

2500 ELO-points.” 2500 ELO-points.”

Page 68: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

68Sofsem'04

Sosonko (August 26, 1980):Sosonko (August 26, 1980):

• ““I haven’t thought about that so much, but I believe I haven’t thought about that so much, but I believe that it will certainly not be harmful for the human that it will certainly not be harmful for the human chess community. It will become more interesting. I chess community. It will become more interesting. I am convinced that computer chess will play a am convinced that computer chess will play a increasingly important role. It will be very increasingly important role. It will be very interesting, I think”interesting, I think”

Page 69: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

69Sofsem'04

Donner (April 13, 1981):Donner (April 13, 1981):

“But the computer cannot play chess at all and will never be able to play the game, at least not the first two thousand years, (...)

What it does now has nothing to do with chess.”

Page 70: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

70Sofsem'04

De Groot (June 19, 1981):De Groot (June 19, 1981):

““No, certainly not. As a matter of fact, I believe it will No, certainly not. As a matter of fact, I believe it will not even possible that it can reach a stable master not even possible that it can reach a stable master level.level.

I believe it is possible that the computer can reach a I believe it is possible that the computer can reach a master level, but not that it is a stable master.”master level, but not that it is a stable master.”

Page 71: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

71Sofsem'04

Contributions from ScienceContributions from Science

• COMPUTERS PLAY STRONGER THAN COMPUTERS PLAY STRONGER THAN HUMANS.HUMANS.

• COMPUTERS CAN NOT SOLVE CHESS.COMPUTERS CAN NOT SOLVE CHESS.

• COMPUTERS ENABLE AN ALTERNATIVE FORM COMPUTERS ENABLE AN ALTERNATIVE FORM OF GAME EXPERIENCE.OF GAME EXPERIENCE.

Page 72: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

72Sofsem'04

ConclusionsConclusions

1. 1. Chess is a frontrunner among the gamesChess is a frontrunner among the games

2. 2. Kasparov’s defeat has become a victory Kasparov’s defeat has become a victory for brute force in combination with for brute force in combination with

knowledge and opponent modellingknowledge and opponent modelling

Page 73: Computer Science Universiteit Maastricht Institute for Knowledge and Agent Technology Games, Theory and Application Jaap van den Herik and Jeroen Donkers.

73Sofsem'04

The Inevitable Final ConclusionThe Inevitable Final Conclusion

TECHNOLOGY, AND IN PARTICULAR OPPONENT MODELLING,

MAKES THE BEST BETTER, BUT DOES NOT LEAVE THE

WEAKER PLAYERS WITHEMPTY HANDS

SINCE THEIR LEVEL WILL INCREASE TOO.