Computer analysis of World Chess Champions
Matej Guid and Ivan Bratko
CG 2006
Introduction
Who was the best chess player of all time?
Chess players of different eras never met across the chess board.
No well founded, objective answer.
I Wilhelm Steinitz, 1886 - 1894
High quality chess programs...
Provide an opportunity of an objective comparisson.
Statistical analysis of results do NOT reflect:
true strengths of the players, quality of play.
Computers...
Were so far mostly used as a tool for statistical analysis of players’ results.
Related work
II Emanuel Lasker, 1894 -1921
Jeff Sonas, 2005:
rating scheme, based on tournament results from 1840 to the present,
ratings are calculated for each month separately, player’s activity is
taken into account.Disadvantages
Playing level has risen dramatically in the recent decades. The ratings in general reflect the players’ success in
competition, but NOT directly their quality of play.
Our approach
III Jose Raul Capablanca, 1921 -1927
computer analysis of individual moves played determine players’ quality of play regardless of the
game score
the differences in players’ style were also taken into account calm positional players vs aggresive tactical players a method to assess the difficulty of positions was
designedAnalysed games
14 World Champions (classical version) from 1886 to 2004 analyses of the matches for the title of “World Chess Champion”
slightly adapted chess program Crafty has been used
The modified Crafty
Instead of time limit, we limited search to fixed search depth. Backed-up evaluations from depth 2 to 12 were obtained for each
move. Quiescence search remained turned on to prevent horizont effects.
IV Alexander Alekhine, 1927 -1935 and 1937 - 1946
Advantages
complex positions automatically get more computation time, the program could be run on computers of different computational
powers.
Obtained data
best move and its evaluation, second best move and its evaluation, move played and its evaluation, material state of each player.
Average error
average difference between moves played and best evaluated moves basic criterion
Formula
∑|Best move evaluation – Move played evaluation|Number of moves
“Best move” = Crafty’s decision resulting from 12 ply search
Constraints
Evaluations started on move 12. Positions, where both the move suggested and the move played
were outside the interval [-2, 2], were discarded.
V Max Euwe, 1935 - 1937
Positional players are expected to commit less errors due to somewhat less complex positions, than tactical players.
Average error
V Max Euwe, 1935 - 1937
Blunders
VI Mikhail Botvinnik, 1948 - 1957, 1958 - 1960, and 1961 - 1963
Big mistakes can be quite reliably detected with a computer. We label a move as a blunder when the numerical error exceeds 1.00.
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
Basic idea
A given position is difficult, when different “best moves”, which considerably alter the evaluation of the root position, are discovered at different search depths.
Assumption
This definition of complexity also applies to humans. This assumption is in agreement with experimental results.
Formula
∑|Best move evaluation – 2nd best move evaluation|
besti ≠ besti - 1
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.00 + (1.30 – 1.16)
Euwe-Alekhine, 16th World Championship 1935
complexity = 0.14
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.14
Euwe-Alekhine, 16th World Championship 1935
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
depth 1st eval 2nd eval
2 Qc2 -0.09 Qc1 -0.17
3 Qc2 +0.24 Qc1 +0.16
4 Qc2 +0.08 Qc1 +0.00
5 Qc2 +0.35 Qc1 +0.30
6 Qc2 +0.07 Qc1 +0.02
7 Qc2 +0.57 Qc1 +0.55
8 Qc2 +0.72 Qc1 +0.60
9 Qc2 +0.96 Qc1 +0.87
10 Qc1 +1.30 Qc2 +1.16
11 Qc1 +1.52 Qc2 +1.26
12 Qd4 +4.46 Qc1 +1.60
complexity = 0.14 + (4.46 – 1.60)
Euwe-Alekhine, 16th World Championship 1935
complexity = 0.14 + 2.86complexity = 3.00
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
Average error in equally complex positions
VIII Mikhail Tal, 1960 - 1961
How would players perform if they faced equally complex
positions? What would be their expected error if they were playing in
another style?
0
10
20
30
40
50
60
0,1 0,3 0,5 0,7 0,9 1,1
complexity
% o
f mov
es
average Capablanca Tal
Percentage of best moves played
It alone does NOT reveal true strength of a
player.
IX Tigran Petrosian, 1963 - 1969
The difference in best move evaluations
X Boris Spassky, 1969 - 1972
Percentage of best moves played...... and the difference in best move
evaluations
XI Robert James Fischer, 1972 - 1975
Material
XII Anatoly Karpov, 1975 - 1985
0
10
20
30
40
50
60
70
80
1 11 21 31 41 51 61 71 81
move no.
mat
eria
l
Kramnik Petrosian Spassky Steinitz
-20
-15
-10
-5
0
5
10
1 11 21 31 41 51 61 71 81 91
move no.
devi
atio
n
Kramnik Petrosian Spassky Steinitz
Credibility of Crafty as an analysis tool
XIII Garry Kasparov, 1985 - 2000
By limiting search depth we achieved automatic adaptation of
time used to the complexity of a given position.
Occasional errors cancel out through statistical averaging
(around 1.400
analyses were applied, altogether over 37.000 positions).Using another program instead of Crafty...
An open source program was required for the modification of the
program.
Analyses of “Man against the machine” matches indicate that
Crafty
competently appreciates the strength of the strongest chess
programs.
Deep Blue 0.0757 New York, 1997
Kasparov
Deep Fritz 0.0617 Bahrain, 2002 Kramnik
Deep Junior
0.0865 New York, 2003
Kasparov
FritzX3D 0.0904 New York, 2003
Kasparov
Hydra 0.0743 London, 2005 Adams
Conclusion
XIV Vladimir Kramnik, 2000 -
Slightly modified chess program Crafty was applied as tool for computer
analysis aiming at an objective comparison of chess players of different eras.
Several criteria for evaluation were designed: average difference between moves played and best evaluated
moves rate of blunders (big errors) expected error in equally complex positions rate of best moves played & difference in best moves evaluations
A method to assess the difficulty of positions was designed, in order to bring
all players to a “common denominator”.
The results might appear quite surprising. Overall, they can be nicely
interpreted by a chess expert.
XIV Vladimir Kramnik, 2000 -
Top Related