Learning to rank moves in mahjong using SVM with tree kernels
description
Transcript of Learning to rank moves in mahjong using SVM with tree kernels
![Page 1: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/1.jpg)
Learning to rank moves in mahjong using SVM with tree
kernelsChikayama & Taura Lab.
M1 Ayato Miki
1
![Page 2: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/2.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
2
![Page 3: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/3.jpg)
Features = elements to evaluate positions in games◦ e.g. Numbers and arrangements of pieces in Shogi
Difficulty in creating features
◦ Require expert knowledge for the game
◦ Simple linear combinations are insufficient e.g. XOR
1. Introduction
3
![Page 4: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/4.jpg)
Use kernels for evaluation features in games
◦ Simple inputs Tree structure
◦ Expect to work as non-linear features Implicit classification in high level feature space
Objective
4
![Page 5: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/5.jpg)
Classification of moves in mahjong using SVM with kernels
◦ “Evaluation functions > search” in mahjong Kernel method is effective
◦ Tree kernels for tree structures of mahjong hands Similarity representation by kernel functions
◦ Use expert game records Learn “expert moves > other moves” with SVM
Method overview
5
![Page 6: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/6.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
6
![Page 7: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/7.jpg)
Machine learning using simple features◦ TD-Gammon [Tesauro, 1992]
Research about mahjong◦ Learning from expert records [Kitagawa, 2007]
2. Related work
7
![Page 8: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/8.jpg)
Machine learning in mahjong [Kitagawa, 2007]
Game Mahjong
Features Manually picked
Method Bonanza method(without search)
Accuracy 56%
9
![Page 9: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/9.jpg)
Mahjong features [Kitagawa, 2007]
Features of player
面前の持ち牌面前の持ち牌 2枚の組み合わせ面前の持ち牌 3枚の組み合わせ
鳴いた牌の構成と状態面子数
リャンメン数カンチャン数とペンチャン数の和
トイツ数テンパイしているかどうか
ドラの枚数面前であるかどうか親であるかどうか
リーチしているかどうか自分が捨てたことのある牌
Features of opponents
鳴いた牌の構成と状態鳴いた回数
鳴いた牌の中で見えているドラの数親であるかどうか
リーチしているかどうかそのプレイヤに対する完全安牌筋や壁などによって安全度が高い牌
自分との点差
Features of field オーラスかどうか見えていない牌の残り枚数
10
![Page 10: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/10.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
11
![Page 11: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/11.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
12
![Page 12: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/12.jpg)
2-class linear classifier
Support Vector Machine (SVM)[Vapnik, 1965]
13
bxwxg )(
0)( xg
1)( xg
1)( xg
w1
w1
Maxmize margin w2
![Page 13: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/13.jpg)
Method for non-linear classification
SVM with kernels[Cortes and Vapnik, 1995]
)(xx
)()( 21 xx ),( 21 xxK
Explicit Replace14
![Page 14: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/14.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
15
![Page 15: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/15.jpg)
Tree structure手牌
面子面子候補孤立牌
暗刻 暗順リャンメン カンチャン トイツ
明刻ペンチャン
…
Specific cards as leaves16
![Page 16: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/16.jpg)
Example
孤立牌 暗刻 暗順リャンメン カンチャン トイツペンチャン
17
![Page 17: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/17.jpg)
Tree kernels [Moschitti, 06]
手牌面子面子候補孤立牌
暗順リャンメン カンチャン
手牌面子面子候補孤立牌
暗順リャンメン カンチャン
Count common subtrees
… … …リャンメン …
リャンメン手牌
面子面子候補暗順リャンメン
… …面子候補リャンメン
SST
18
![Page 18: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/18.jpg)
Deep subtrees are not very important
Subtrees weight [Moschitti, 06]
面子暗順暗順 暗刻
depth )10(
19
![Page 19: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/19.jpg)
Tree kernel function
11 22
),(),( 2121t tNn Nn
t nnttK
F
iii
fl nInInn i
121
)(21 )()(),(
},,,{ 21 FfffF
01
)(nI i
tN
)( ifl
Set of nodes in tree t
Subtree set
Depth of subtree fi
otherwise
If fi is rooted at node n
20
![Page 20: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/20.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
21
![Page 21: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/21.jpg)
SVM is just a 2-class classifier◦ How learn to rank
Learning to rank [Shen et al. 03]
If you want to know ranks of three moves…
Order classifier ( > or < )
),(),(),(
32
31
21
mmmmmm
32
31
21
mmmmmm
213 mmm Rank
Input data -> Pairs of moves
22
![Page 22: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/22.jpg)
One training example has two tree instances
Learn and classify orders
Define kernel function for relative order
ri
lii tte ,
),(),(),(),(),( 2121212121lr
trl
trr
tll
ttr ttKttKttKttKeeK
Classify “tl > tr” and “ tl< tr”
Label +1 when tl > trLabel -1 when tl < tr
24
![Page 23: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/23.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
25
![Page 24: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/24.jpg)
1. Experiment of proposed method
2. Error analysis
3. Comparison with related work
4. Practical player
26
4. Experiment
![Page 25: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/25.jpg)
Machine spec◦ Dual-Core AMD Opteron 2.4GHz◦ 32GB RAM
Implementation◦ SVM-Light-TK [Moschitti, 2004]
Soft margin trade-off parameter C=0.1 Optimization threshold ε=0.1
Environment
27
![Page 26: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/26.jpg)
Learn from tsumo positions in expert records◦ “Offensive” positions only
Nobody declares “li-zhi” Nobody calls 3 or more “chi”, “pon” or “kan”
◦ Using records of Totugeki Tohoku ~285 games (~13,000 training positions)
Evaluation◦ Accuracy rates of trained classifiers
Are expert moves ranked as the bests ?◦ 4-fold cross validation
Method of experiment
28
![Page 27: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/27.jpg)
Accuracy rates
0 2000 4000 6000 8000 10000 12000 1400030%
40%
50%
60%
70%
80%
90%
100%
rank 1rank 1-2rank 1-3rank 1-5
Training positions
Accu
racy
rat
es
29
![Page 28: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/28.jpg)
“Defensive” positions
Typical mistakes (1)
31
![Page 29: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/29.jpg)
Positions require “yaku”(=poker hands) knowledge
Typical mistakes (2)
32
![Page 30: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/30.jpg)
Using features designed in [Kitagawa, 2007]◦ Including board status information
Implementation◦ Ranking SVM in SVM-Light [Joachims, 2002]
Comparison with SVM using linear features
34
![Page 31: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/31.jpg)
Accuracy rates with linear features
0 2000 4000 6000 8000 10000 12000 1400030%
40%
50%
60%
70%
80%
90%
100%
rank 1rank 1-2rank 1-3rank 1-5
Training positions
Accu
racy
rat
es
35
![Page 32: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/32.jpg)
Core2 Duo 1.06GHz 91819 support vectors 700ms for classification of one tree pair
Practical player ?
7 seconds for deciding one move◦ 4 seconds in dual threading◦ Good enough for playing against human players
36
![Page 33: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/33.jpg)
1. Introduction2. Related work3. Proposed method
◦ SVM with kernels◦ Tree kernels in mahjong◦ Learning to rank using SVM
4. Experiment5. Conclusion
Outline
37
![Page 34: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/34.jpg)
Classified ranks of moves with tree kernels◦ Possible with simple input
57% accuracy◦ Despite the lack of information of field and
opponents◦ Increasing…
Fine accuracy with permissible cost
5. Conclusion
38
![Page 35: Learning to rank moves in mahjong using SVM with tree kernels](https://reader035.fdocuments.net/reader035/viewer/2022062323/568165dd550346895dd8f7b6/html5/thumbnails/35.jpg)
Classification analysis◦ Positions that linear combinations cannot classify
Refine tree structure
Other information◦ Hands information with other kernels
String kernels◦ Information of field and opponents
Add as linear combinations or other kernels
Heavy computing cost◦ Classification time increases with a number of training positions◦ Indispensable in other games
Future work
39