Complex Problem Solving With Neural Networks: Learning...
Transcript of Complex Problem Solving With Neural Networks: Learning...
Com
plex
Pro
blem
Sol
ving
With
Neu
ral
Net
wor
ks: L
earn
ing
Che
ss
Mr.
Jack
Sig
anD
r. A
leks
ande
r Mal
inow
ski,
Adv
isor
Dep
t. of
Ele
ctric
al a
nd C
ompu
ter E
ngin
eerin
g
Apr
il 19
, 200
5
Out
line
•In
trodu
ctio
n to
pro
ject
and
neu
ral n
etw
orks
•N
eura
l net
wor
ks a
nd c
hess
•N
eura
l net
wor
k sy
stem
des
ign
•To
ols
and
proc
edur
es–
Pre
proc
essi
ng–
Trai
ning
–In
terfa
ce•
Res
ults
and
con
clus
ions
–In
itial
resu
lts–
Cre
atio
n of
the
eval
uatio
n fu
nctio
n–
Fina
l con
clus
ions
Sys
tem
Blo
ck D
iagr
am
AN
N“S
yste
m”
Inte
rfac
e (G
ener
ates
Bo
ard
D
escr
ipti
on)
Pla
yer
Mo
ve
AN
N M
ove
(s)
AN
N“S
yste
m”
Dat
a P
rep
roce
ssin
g
Bo
ard
Po
siti
on
Ch
ose
n M
ove
s
Pla
yin
g o
r A
dvi
sory
Mo
de
Lea
rnin
g M
od
e
1792
Fee
dfo
rwar
d n
etw
ork
s:2
laye
rs o
f 128
no
des
Gam
e D
ataB
ase
Neu
ral N
etw
orks
W1
W2
Wn
Thre
shol
d (T
)
. . .
X 1 X 2 X n
∑ =n ii
iWX
0)
(O
ut (Y
)
∑ =
−=
n ii
iT
WX
Tanh
Y0
))
((
Figu
re 1
A: N
ode
stru
ctur
e w
ithhy
perb
olic
tang
ent a
ctiv
atio
nfu
nctio
n
Figu
re 1
B: S
impl
e pa
rtial
lyco
nnec
ted
neur
al n
etw
ork
stru
ctur
e fro
m S
tuttg
art
Neu
ral N
etw
ork
Sim
ulat
or(S
NN
S)
Che
ss a
nd N
eura
l Net
wor
ks?
•D
emon
stra
tes
com
plex
dec
isio
n m
akin
g•
Hig
hly
nonl
inea
r pro
blem
•S
chem
as•
Wid
ely
stud
ied
•M
assi
ve a
mou
nts
of a
vaila
ble
data
•S
ucce
ss w
ith c
heck
ers
•M
ixed
resu
lts w
ith c
hess
in th
e pa
st
Sta
ndar
ds a
nd re
sear
ch•
Num
erou
s ap
plic
able
“sta
ndar
ds”
–C
hess
“law
s”•
FID
E (F
édér
atio
n In
tern
atio
nale
des
Éch
ecs)
•ht
tp://
ww
w.fi
de.c
om–
PG
N fi
le s
tand
ard
•re
c.ga
mes
.che
ss, 1
994
–E
PD
file
sta
ndar
d•
Form
at s
uppl
ied
by “E
PD
_Pos
ition
.exe
” (st
anda
rd?)
–C
hess
eng
ine
stan
dard
com
man
d re
fere
nce
•M
ost i
nflu
entia
l res
earc
h–
K. C
hella
pilla
and
D.B
. Fog
el–
C. P
osth
off,
S. S
chaw
elsk
i and
M. S
chlo
sser
Par
alle
l Net
wor
k D
esig
ns
•D
ecre
ase
lear
ning
cyc
le ti
me
•Le
ss d
estru
ctiv
e le
arni
ng p
roce
ss•
Mul
tiple
“sug
gest
ed” m
oves
may
be
retu
rned
•Si
mpl
er n
etw
ork
arch
itect
ures
•2
para
digm
s fo
r dec
onst
ruct
ing
ches
s–
Geo
grap
hica
l “m
ove
base
d”–
Func
tiona
l “pi
ece
base
d”•
Exte
rnal
logi
c is
app
lied
to b
oth
para
digm
s to
filte
r out
ille
gal m
oves
or i
mpo
ssib
le m
oves
Par
alle
l Net
wor
k D
esig
ns
Figu
re 3
: Map
ping
the
lega
l mov
es in
che
ss, u
sing
ove
rlay
cons
istin
g of
que
en +
kni
ght m
oves
Exc
ludi
ng c
astli
ng, t
here
are
185
6 m
oves
pos
sibl
e—
igno
ring
the
piec
e ty
pe w
hich
is m
oved
Exa
mpl
e: M
ovin
g fro
m d
3 to
d4
is c
onsi
dere
d O
NE
pos
sibl
em
ove,
whe
ther
the
piec
e is
a p
awn,
que
en, k
ing,
etc
.
I
I
Par
alle
l Net
wor
k D
esig
ns•
star
ting
posi
tion
i•
final
pos
ition
f•
mif=
f(bi)
repr
esen
ts a
ll le
gal m
oves
for a
boa
rd p
ositi
on b
i•
gam
e g
of n
mov
es m
ay b
e ex
pres
sed
as a
set
of b
oard
posi
tions
bi,
biЄ
g, w
here
i is
the
posi
tion
num
ber 0
to n
.
∑ =
)(
=n i
ibf
L0 ∑∞ =
)=
0
(i
iLM
•In th
is d
esig
n, it
is re
quire
d to
cre
ate
an in
divi
dual
AN
N s
truct
ure
for a
ll m
oves
t, tЄ
M. M
=185
6, ig
norin
g ca
stlin
g
Lega
l mov
es fo
r a g
ame
of n
pos
ition
s
Lega
l mov
es fo
r che
ss (a
ll ga
mes
)
App
roac
h A
: Geo
grap
hica
l
AN
NPo
sitio
n A
8-A7
AN
NPo
sitio
n A
8-B8
AN
NPo
sitio
nH
2-H1
RU
LE L
OG
ICB
inar
y O
utpu
t‘1
’=Le
gal
Mov
e
Dec
isio
n Fi
naliz
atio
n(P
ick
The
Stro
nges
t Out
put F
rom
The
Out
put S
et)
Save
AN
N R
esul
t And
Le
galit
y Fo
r Ada
ptiv
e R
eson
ance
Tra
inin
g . .
.
‘1’ W
ill C
lose
“S
witc
h”‘0
’ Dis
able
s M
ove
Out
put
(Mov
e)
Figu
re 4
: Des
ign
with
“mov
e sp
ecifi
c” n
eura
l net
wor
ks—
the
geog
raph
ical
app
roac
h
Par
alle
l Net
wor
k D
esig
ns•
star
ting
posi
tion
i•
final
pos
ition
f•
piec
e p
•m
if=f(p
,bi)
repr
esen
ts a
ll le
gal m
oves
for a
pie
ce in
bi
•ga
me
g of
n m
oves
may
be
expr
esse
d as
a s
et o
f boa
rdpo
sitio
ns b
i, biЄ
g, w
here
i is
the
mov
e nu
mbe
r 0 to
n.
∑ =
)(
=n i
ibp
fL
0
,
∑∞ =
)=
0
(i
iLM
•In th
is d
esig
n, it
is re
quire
d to
cre
ate
an in
divi
dual
AN
N s
truct
ure
for a
ll pi
eces
p, p
=1 to
16
Lega
l mov
es fo
r a g
ame
of n
mov
es
Lega
l mov
es fo
r a p
iece
(all
gam
es)
App
roac
h B
: Fun
ctio
nal
Bla
ck P
awn
1B
lack
Paw
n 2
Bla
ck K
ing
RU
LE L
OG
ICB
inar
y O
utpu
t‘1
’=Le
gal
Mov
e
Dec
isio
n Fi
naliz
atio
n (P
ick
The
Stro
nges
t Out
put F
rom
The
O
utpu
t Set
)
Sav
e E
ach
AN
N R
esul
t A
nd L
egal
ity F
or U
se
In A
dapt
ive
Res
onan
ce T
rain
ing
. .
.
‘1’ W
ill C
lose
“S
witc
h”‘0
’ Dis
able
s M
ove
Out
put
(Mov
e)
Switc
h fo
r EA
CH
O
utpu
tFi
gure
5: D
esig
n w
ith “p
iece
sp
ecifi
c” n
eura
l net
wor
ks—
the
func
tiona
l app
roac
h
Fina
l AN
N S
yste
m
AN
NPo
sitio
n A
8-A7
AN
NPo
sitio
n A
8-B8
AN
NPo
sitio
nH
2-H1
Dec
isio
n Fi
naliz
atio
n(P
ick
The
Stro
nges
t Out
put F
rom
The
Out
put S
et)
. .
.
‘1’ W
ill C
lose
“S
witc
h”‘0
’ Dis
able
s M
ove
Out
put
(Mov
e)
Eval
uatio
n Fu
nctio
n an
d R
ule
Logi
c
Cur
rent
B
oard
Po
sitio
n
Des
ign
with
“mov
e sp
ecifi
c” n
eura
l net
wor
ks—
the
geog
raph
ical
app
roac
h
Dat
a R
epre
sent
atio
ns
1. e4 d6 2. d4 Nf6 3. Nc3 g6 4. Nf3 Bg7 5. Be2 O-O 6. O-O Bg4
7. Be3 Nc6 8. Qd2 e5 9. d5 Ne7 10. Rad1 Bd7 11. Ne1 Ng4 12. Bxg4 Bxg4
13. f3 Bd7 14. f4Bg4 15. Rb1 c6 16. fxe5 dxe5 17. Bc5 cxd5 18. Qg5 dxe4
19. Bxe7 Qd4+ 20. Kh1f5 21. Bxf8 Rxf8 22. h3 Bf6 23. Qh6 Bh5 24. Rxf5 gxf5
25. Qxh5 Qf2 26. Rd1e3 27. Nd5 Bd8 28. Nd3 Qg3 29. Qf3 Qxf3 30. gxf3 e4
31. Rg1+ Kh8 32. fxe4 fxe4 33. N3f4 Bh4 34. Rg4 Bf2 35. Kg2 Rf5 36. Ne7 1-0
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - pm d4;
rnbqkbnr/pppppppp/8/8/3P4/8/PPP1PPPP/RNBQKBNR b KQkq d3 pm Nf6;
rnbqkb1r/pppppppp/5n2/8/3P4/8/PPP1PPPP/RNBQKBNR w KQkq - pm Nf3;
rnbqkb1r/pppppppp/5n2/8/3P4/5N2/PPP1PPPP/RNBQKB1R b KQkq - pm b6;
Figu
re 6
: Exa
mpl
e of
the
PG
N (a
lgeb
raic
) sta
ndar
d
Figu
re 7
: Exa
mpl
e of
the
EP
D (s
tring
) for
mat
Inpu
t Vec
tor C
reat
ion
0.1,
-0.1
p,P
Paw
n
0.3,
-0.3
b.B
Bis
hop
0.4,
-0.4
n,N
Knig
ht
0.5,
-0.5
r,RR
ook
0.9,
-0.9
q,Q
Que
en
1.0,
-1.0
k,K
Kin
g
Wei
ght
Bla
ck, w
hite
EPD
Cha
ract
erB
lack
, whi
tePi
ece
“Sta
ndar
d” v
alue
s fo
r pie
ces
used
in th
e flo
atin
g po
int
inpu
t vec
tor c
reat
ion
Floa
ting
poin
t inp
ut v
ecto
r for
the
initi
al b
oard
pos
ition
Pre
proc
essi
ng
PGN
File
EPD
File
Floa
ting
Poin
tVe
ctor
Dat
abas
eSe
lect
ions
Star
tEn
d
The
gam
e da
ta p
repr
oces
sing
pro
cedu
re
•Boa
rd p
ositi
onal
dat
a, p
lus
a ‘y
es’ o
r ‘no
’ dec
isio
n in
rega
rds
to m
akin
gth
e sp
ecifi
c m
ove
mus
t be
in th
e flo
atin
g po
int v
ecto
r.
•A m
ix o
f ‘ye
s’ a
nd ‘n
o’ s
ampl
es w
ill b
e us
ed in
all
train
ing
sets
•Tra
inin
g se
ts a
re ra
ndom
ly c
hose
n
•Do
not d
iffer
entia
te b
etw
een
piec
es to
be
mov
ed, o
nly
the
initi
al a
nd fi
nal
posi
tions
are
impo
rtant
—ru
le lo
gic
is s
epar
ate
Trai
ning
Set
up
. .
. .
PC1
PC2
PCn
GD
AN
SK S
ERV
ER Networ
k file
Train
ing
data
Trai
ning
Pro
cess
Sta
rt
Ob
tain
netw
ork
n
am
e f
rom
co
nfi
gu
rati
on
file
Ob
tain
req
uir
ed
raw
d
ata
files
(do
wn
load)
Gen
era
te t
rain
ing
d
ata
set
fro
m r
aw
d
ata
Init
ialize
a n
ew
n
etw
ork
Ch
eck f
or
up
date
s
an
d p
rocess t
hem
Sta
rt S
NN
S, lo
ad
n
etw
ork
an
d d
ata
Tra
in 5
0 e
po
ch
s
Sen
d t
rain
ed
n
etw
ork
to
serv
er
(FT
P)
Mo
re
netw
ork
s t
o
train
?
Gen
era
te s
cri
pts
an
d b
atc
h f
iles—
Sta
rt b
atc
h
execu
tio
n
Setu
p(S
hell
P
rog
ram
)
Data
P
rocessin
gT
rain
ing
Sta
rt c
on
tro
l scri
pt
Exit
Bu
tto
n(K
ill all
pro
cesses)
En
dY
es
Trai
ning
Pro
blem
s•
50 e
poch
s of
abo
ut 5
thou
sand
pat
tern
s•
1792
net
wor
ks w
ith 1
8 m
inut
es p
erne
twor
k•
SN
NS
is N
OT
the
mos
t ele
gant
sol
utio
n!–
Java
ver
sion
lack
s sc
riptin
g ab
ility
–Sl
ow tr
aini
ng–
Varia
ble
exec
utio
n tim
e•
Pro
blem
s w
ith .N
ET
/ lab
mac
hine
s
Inte
rface
Mod
ule
Initi
al F
indi
ngs
and
Res
ults
•B
ackp
rop
does
not
wor
k•
Usi
ng re
silie
nt b
ackp
rop
inst
ead
•N
o si
gnifi
cant
diff
eren
ce b
etw
een
2 or
mor
e hi
dden
laye
rs (i
n tra
inin
g sp
eed)
•C
lose
out
put p
roxi
mity
—ho
w to
dec
ide?
•A
n ev
alua
tion
func
tion
is re
quire
d
Eva
luat
ion
Func
tion
Eva
luat
ion
Func
tion
1
•O
nly
top
NN
out
puts
will
be
cons
ider
ed•
Rat
e th
e m
oves
, ign
ore
NN
out
put
•C
onsi
der
–M
ater
ial (
?M)
–Th
reat
s (?
T)–
Mob
ility
(?O
)–
Vul
nera
bilit
ies
(?V
)•
Scor
e =
a*?
M +
b*?
T +
c*?
O +
d*?
V•
a,b,
c,d
are
wei
ghts
(exp
erim
enta
lly d
eter
min
ed)
Eva
luat
ion
Func
tion
2•
Onl
y to
p N
N o
utpu
ts w
ill b
e co
nsid
ered
•R
ate
the
mov
es w
ith N
N o
utpu
t as
a fa
ctor
•C
onsi
der
–M
ater
ial (
?M)
–Th
reat
s (?
T)–
Mob
ility
(?O
)–
Vul
nera
bilit
ies
(?V
)–
NN
out
put (
Y)
•Sc
ore
= a*
?M
+ b
*?T
+ c*
?O
+ d
*?V
+ Y
•a,
b,c,
d ar
e w
eigh
ts (e
xper
imen
tally
det
erm
ined
)
Fina
l Eva
luat
ion
Func
tion
•A
che
ss e
ngin
e is
util
ized
–Pr
ovid
es a
n ex
perim
enta
l “ba
selin
e”–
Prov
ides
a b
oard
eva
luat
ion
“E” o
nly
•W
hite
sid
e ch
oose
s m
ove
base
d on
“E”
•B
lack
use
s E
+a*Y
•Y
is th
e ne
ural
net
wor
k ou
tput
•Th
e “a
” coe
ffici
ent i
s fo
und
expe
rimen
tally
Find
ings
and
Res
ults
•N
eura
l net
wor
k ou
tput
can
con
tribu
te to
the
eval
uatio
n of
a m
ove
•Pe
rform
ance
of t
he N
N s
yste
m is
sig
nific
antly
bette
r tha
n th
e ch
ess
engi
ne a
lone
•Th
e sy
stem
per
form
s ve
ry p
oorly
in e
nd-g
ame
scen
ario
s, p
ossi
bly
indi
catin
g a
need
for p
iece
-sp
ecifi
c kn
owle
dge
•M
odifi
ed tr
aini
ng m
ay le
ad to
impr
ovem
ents
inpe
rform
ance
…
Find
ings
and
Res
ults
Avera
ge Sc
ore vs
Ply
-4-3-2-1012345
05
1015
2025
3035
4045
Ply
Average Score (Gaviota Engine)
Contr
olNN
x2
Trai
ning
Vec
tor C
odin
g•
May
nee
d to
eva
luat
e ot
her f
orm
ats
–B
inar
y re
pres
enta
tion
(use
d in
end
-gam
e re
sear
ch)
–M
ultip
le s
patia
l rel
atio
nshi
ps?
•Tr
aini
ng m
ust b
e st
arte
d to
kno
w!
Figu
re 1
1: P
ossi
ble
spat
ial r
elat
ions
hips
Add
ition
al q
uest
ions
and
com
men
ts a
rein
vite
d. P
leas
e co
ntac
t:Ja
ck S
igan
, jsi
gan@
brad
ley.
edu
Pro
ject
Web
site
:ht
tp://
cegt
201.
brad
ley.
edu/
proj
ects
/pro
j200
5/nn
ches
s/
Que
stio
ns?