Luan Van Kohonen

7/29/2019 Luan Van Kohonen

1/67

MC LC

MC LC .............................................................................................................................. 1

CC HNH V, BNG BIU TRONG LUN VN .......................................................... 2THUT NG TING ANH .................................................................................................. 2Li cm n ..............................................................................................................................3Chng 1. Gii thiu v mng nron nhn to ...................................................................... 5

1.1 Cu trc v m hnh mng nron ................................................................................ 71.1.1 M hnh mt nron sinh hc ................................................................................. 71.1.2 Cu trc v m hnh ca mt nron nhn to ....................................................... 8

1.2 Cu to v phng thc lm vic ca mng nron ....................................................111.2.1 Mng nron mt lp ............................................................................................131.2.2 Mng nron truyn thng nhiu lp ....................................................................141.2.3 Mng nron phn hi .......................................................................................... 15

1.2.4 Mng nron hi quy ............................................................................................151.2.5 Mng Hopfield .................................................................................................... 151.2.6 Mng BAM ..........................................................................................................17

1.3 Cc lut hc ................................................................................................................181.3.1 Hc c gim st ...................................................................................................191.3.2 Hc cng c .........................................................................................................201.3.3 Hc khng c gim st ........................................................................................20

1.4 Thut ton lan truyn ngc .......................................................................................221.5 Kt lun ...................................................................................................................... 28

Chng 2. Tm hiu mng nron Kohonen ......................................................................... 302.1 Gii thiu ....................................................................................................................30

2.2.1 Mng nron Kohonen nhn dng ................................................................332.2.2 Cu trc ca mng nron Kohonen .....................................................................332.2.3 Chun ha d liu u vo ..................................................................................342.2.4 Tnh ton d liu u ra ca nron ..................................................................... 352.2.5 nh x lng cc ................................................................................................ 352.2.6 Chn nron thng ................................................................................................362.2.7 Qu trnh hc ca mng Kohonen .......................................................................372.2.8 T l (tc ) hc ................................................................................................ 382.2.9 iu chnh cc trng s (cp nht trng s) ........................................................392.2.10 Tnh ton sai s ..................................................................................................39

2.3. Thc thi mng nron Kohonen ................................................................................. 402.3.1 Thc thi mng nron truyn thng .....................................................................402.3.2 Thc thi lan truyn ngc ................................................................................... 452.3.3 Cc tp hun luyn .............................................................................................. 462.3.4 Bo co tin trnh ................................................................................................ 472.3.4.1 Lp mng c s ................................................................................................472.3.4.2 Lp KohonenNetwork ......................................................................................49

2.4 Kt lun ...................................................................................................................... 57Chng 3. Nhn dng k t quang s dng mng nron Kohonen .....................................59

3.1 Gii thiu chung .........................................................................................................593.2 Hun luyn mng ........................................................................................................60

- 1 -


2/67

3.3 Th nghim s dng mng nron Kohonen nhn dng k t quang ...................613.4 Trng hp ngoi l ...................................................................................................643.5 Kt lun ..................................................................................................................... 64

KT LUN ..........................................................................................................................66

Ti tham kho .......................................................................................................................67

CC HNH V, BNG BIU TRONG LUN VN

Hnh 1.1 Mt nron sinh hc ................................................................................................. 8Hnh 1.2. S lin kt cc nron .............................................................................................. 8Hnh 1.3 M hnh mt nron nhn to .................................................................................. 9Hnh 1.4 th cc dng hm truyn ...................................................................................11Hnh 1.5 Mng nron ba lp ................................................................................................12Hnh 1.6 Mt s dng mng nron .......................................................................................14Hnh 1.7 Cu trc ca mng Hopfield ..................................................................................16Hnh 1.8 Cu trc ca BAM ................................................................................................17Hnh 1.9: Cu trc hun luyn mng nron ......................................................................... 18Hnh 1.10: Hc c gim st .................................................................................................20Hnh 1.11: Hc khng c gim st ......................................................................................20Hnh 1.12: S cu trc chung ca qu trnh hc .............................................................21Hnh 1.14: Mng 3 lp lan truyn ngc ............................................................................. 23Hnh 2.1: Mt dng mng nron Kohonen .......................................................................... 34Hnh 2.2: S khi biu din hun luyn mng nron Kohonen.....................................38Hnh 3.1:M hnh chung trong nhn dng ch vit............................................................. 60Hnh 3.2: S hun luyn mng ........................................................................................60

Hnh 3.4: Biu din k t e theo ma trn 5x7 pixcel............................................................62Hnh 3.5: Biu din k t e theo ma trn 5x7 bi cc gi tr .............................................62Hnh3.6: Biu din k t e vector u vo ........................................................................62

Hnh 3.7 V v gn k t .................................................................................................. 63Hnh 3.8 Kt qu mng nron Kohonen nhn dng k t e v k t c ................................63Hnh 3.9 a ch k vo mng v gn tn ..........................................................................63Hnh 3.10 Kt qu mng nron Kohonen nhn dng ch k ..............................................64

THUT NG TING ANH

- 2 -


3/67

ANN Mng nron cn hun luynBAM Mng BAM (Bidirectional Associative Memory)SOM Mng nron t t chc (Self Organizing Maps)PE Phn t x l (Processing Element)OCR Nhn dng k t quang (optical character recognition)

Li cm n

Chng ta u bit rng, b no con ngi l mt sn phm hon ho ca to

ha, n c kh nng t duy v sng to. Hin nay, con ngi ang nghin cu

phng thc hot ng ca b no, sau p dng cho nhng cng ngh hin i.

- 3 -


4/67

tip cn kh nng hc, ngi ta a ra m hnh mng nron gm cc nron lin

kt vi nhau thnh mng phng theo cu trc mng thn kinh ca con ngi.

Mi nron ring l c kh nng x l thng tin yu, nhng khi chng c

ghp vi nhau thnh mng, th kh nng x l thng tin s mnh hn rt nhiu. Mi

cu trc mng u c mt u im c th, chng cho ta mt cng c mnh trong

cc lnh vc k thut iu khin v k thut thng tin. Mt mng nron nhn to l

tp hp mt s ln cc phn t x l (cc nt hay cc khi), thng c t chc

song song v c cu hnh theo kin trc quy. Cch ng s trn mng nron

nhn to ging nh b no con ngi, n chng t kh nng hc, nh li, v tng

qut ha t d liu hun luyn.Mng nron nhn to l cng c tt trong vic gii quyt cc bi ton nh:

hp v phn lp i tng, xp x hm, ti u ha, nh lng vector, phn cm d

liu,...N thay th hiu qu cc cng c tnh ton truyn thng gii quyt cc bi

ton ny.

Nhn dng l mt lnh vc ng vai tr quan trng trong khoa hc k thut.

Trong hu ht cc vn k thut ngy nay, ta u phi xc nh, nhn dng c

cc m hnh v i tng lin quan, t tm ra gii php. Nhn dng m hnh

l bi ton rt quan trong trong l thuyt h thng. L do n gin l v khng th

phn tch, tng hp h thng khi khng c m hnh ton hc m t h thng. Trong

qu trnh xy dng m hnh h thng trn phng din l thuyt, ngi ta thng

khng kho st c mi nh hng ca mi trng n tnh ng hc ca h

thng, cng nh nhng tc ng qua li bn trong h thng mt cch chnh xc

tuyt i. Rt nhiu yu t b b qua, hoc ch c xem xt n nh l mt tc

ng ngu nhin. Bi vy, nu ni mt cch cht ch th nhng hiu bit l thuyt

ban u v h thng, mi ch c th gip ta khoanh c lp cc m hnh thch hp.

c th c c mt m hnh c th c cht lng ph hp vi bi c th ton t

ra trong lp cc m hnh thch hp , th phi s dng phng php nhn dng.

Cn bi ton nhn dng, phn tch phn cm d liu, l cc bi ton rt hay gp

- 4 -


5/67

trong thc t, khi chng ta nhn thy mt vt g , th cu hi thng trc ca mi

ngi l; vt c my loi, v n thuc loi no trong cc loi c th c.

gii quyt cc bi ton nhn dng, ngi ta a vo cc cch tip cn

khc nhau, mi phng php tip cn trong nhng bi ton c th u c nhng u,

nhc im ring. Phng php ng dng mng nron trong nhn dng l mt cch

tip cn mi v hin i. N c th l cng c rt mnh gii quyt cc bi ton

trong lnh vc ny.

Ni dung ca ti i vo tm hiu v xy dng cc phn t nron c bn,

xem xt v nghin cu cu trc mt mng nron, gii thiu v mng nron nhiu

lp vi thut ton lan truyn ngc. Trng tm ca ti i vo tm hiu v mngnron Kohonen (hay mng nron t t chc SOM).

ti gm ba chng

Chng 1, trnh by cu trc mt phn t nron c bn, cc cu trc mng

nron nhn to thng gp, thut ton hc, phn tch u nhc im ca chng, v

gii thiu v thut ton lan truyn ngc.

Chng 2, tm hiu mng nron Kohonen.

Chng 3, nhn dng k t quang s dng mng nron Kohonen.

Cui cng em xin cm n cc thy c gio, c bit l PGS.TSKH Bi Cng

Cng tn tnh ch dn cho em trong sut thi gian lm ti. Xin cm n cc

bn cng lp to iu kin cho ti c hc tp v nghin cu trong mi trng

tt.

H ni, thng 12 nm 2009.

Chng 1. Gii thiu v mng nron nhn to

Hc my l mt ngnh khoa hc nghin cu cc thut ton cho php my

tnh c th hc c cc khi nim.

- 5 -


6/67

Phn loi: C hai loi phng php hc my chnh

Phng php quy np : My hc/phn bit cc khi nim da trn d liu

thu thp c trc . Phng php ny cho php tn dng c ngun d

liu rt nhiu v sn c.

Phng php suy din : My hc/phn bit cc khi nim da vo cc lut.

Phng php ny cho php tn dng c cc kin thc chuyn ngnh h

tr my tnh.

Hin nay, cc thut ton u c gng tn dng c u im ca hai phng php

ny.

Cc ngnh khoa hc lin quan:

L thuyt thng k: cc kt qu trong xc sut thng k l tin cho rt

nhiu phng php hc my. c bit, l thuyt thng k cho php c

lng sai s ca cc phng php hc my.

Cc phng php tnh: cc thut ton hc my thng s dng cc tnh tons thc/s nguyn trn d liu rt ln. Trong , cc bi ton nh: ti u

c/khng rng buc, gii phng trnh tuyn tnh v.v c s dng rt ph

bin.

Khoa hc my tnh: l c s thit k cc thut ton, ng thi nh gi

thi gian chy, b nh ca cc thut ton hc my.

ng dng:Hc my c ng dng rng khp trong cc ngnh khoa hc/sn xut,c bit nhng ngnh cn phn tch khi lng d liu khng l. Mt s ng dng

thng thy nh:

X l ngn ng t nhin: x l vn bn, giao tip ngi my,

Nhn dng: nhn dng ting ni, ch vit tay, vn tay, th gic my

(Computer Vision)

- 6 -


7/67

Tm kim

Chn on trong y t: phn tch nh X-quang, cc h chuyn gia chn on

t ng.

Tin sinh hc: phn loi chui gene, qu trnh hnh thnh gene/protein

Vt l: phn tch nh thin vn, tc ng gia cc ht

Pht hin gian ln ti chnh (financial fraud): gian ln th tn dng

Phn tch th trng chng khon (stock market analysis)

Chi tr chi: t ng chi c, hnh ng ca cc nhn vt o

Rbt: l tng hp ca rt nhiu ngnh khoa hc, trong hc my to nn

h thn kinh/b no ca ngi my.

Cc nhm gii thut hc my: Hc c gim st, hc khng gim st, hc

na gim st, hc tng cng,

1.1 Cu trc v m hnh mng nron

1.1.1 M hnh mt nron sinh hc

Phn t x l c bn ca mt mng nron sinh hc l mt nron, phn t

ny c th chia lm bn thnh phn c bn nh sau: dendrites, soma, axon, v

synapses.

- Dendrites: l phn nhn tn hiu u vo.

- Soma: l ht nhn.

- Axon: l phn dn ra tn hiu x l.

- Synapses: l ng tn hiu in ha giao tip gia cc nron.

Kin trc c s ny ca b no con ngi c mt vi c tnh chung. Mt

cch tng qut, th mt nron sinh hc nhn u vo t cc ngun khc nhau, kt

hp chng ti vi nhau, thc thi t hp phi tuyn chng cho ra kt qu cui cng

u ra. Hnh 1.1 ch ra mi quan h gia bn phn t ca mt nron sinh hc.

- 7 -


8/67

Hnh 1.1 Mt nron sinh hc

Mt nron sinh hc ch c mt s chc nng c bn nh vy, ta nhn thy

kh nng x l thng tin ca n l rt yu. c c kh nng x l thng tin

hon ho nh b no con ngi, th cc nron phi kt hp v trao i thng tin vinhau. Ta hnh dung s lin kt, v trao i thng tin gia hai nron nh hnh 1.2.

Hnh 1.2. S lin kt cc nron

1.1.2 Cu trc v m hnh ca mt nron nhn to

M hnh ton hc ca mng nron sinh hc c xut bi McCulloch v

Pitts, thng c gi l nron M-P, ngoi ra n cn c gi l phn t x l v

c k hiu l PE (Processing Element).

M hnh nron c m u vo x1, x2, ..., xm, v mt u ra yi nh sau:

- 8 -


9/67

Hnh 1.3 M hnh mt nron nhn to

Gii thch cc thnh phn c bn:

- Tp cc u vo: L cc tn hiu vo ca nron, cc tn hiu ny thng

c a vo di dng mt vector m chiu.

- Tp cc lin kt (cc trng s): Mi lin kt c th hin bi mt trng

s (thng c gi l trng s lin kt). Trng s lin kt gia tn hiu

vo th j cho nron i thng c k hiu l wij. Thng thng cc trng

s ny c khi to ngu nhin thi im khi to mng v c cpnht lin tc trong qu trnh hc mng.

- B tng (Hm tng): Thng dng tnh tng ca tch cc u vo vi

trng s lin kt ca n.

- Ngng: Ngng ny thng c a vo nh mt thnh phn ca hm

truyn.

- Hm truyn: Hm ny dng gii hn phm vi u ra ca mi nron.

N nhn u vo l kt qu ca hm tng v ngng cho. Thng

thng, phm vi u ra ca mi nron c gii hn trong on [0,1]

hoc [-1,1]. Cc hm truyn rt a dng, c th l cc hm tuyn tnh

hoc phi tuyn. Vic la chn hm truyn ty thuc vo tng bi ton v

kinh nghim ca ngi thit k mng.

- 9 -


10/67

- u ra: L tn hiu u ra ca mt nron, vi mi nron s c ti a mt

u ra.

V mt ton hc, cu trc ca mt nron i c m t bng cp biu thc

sau:

)( iii netfy = v jn

jiji xwnet

=

=1

trong : x1, x2, xm l cc tn hiu u vo, cn w i1, wi2,,wim l cc trng s

kt ni ca nron th i, neti l hm tng, f l hm truyn, i l mt ngng, yi l

tn hiu u ra ca nron.

Nh vy, tng t nh nron sinh hc, nron nhn to cng nhn cc tnhiu u vo, x l (nhn cc tn hiu ny vi trng s lin kt, tnh tng cc

tch thu c ri gi kt qu n hm truyn), v cho mt tn hiu u ra (l kt

qu ca hm truyn).

Hm truyn c th c cc dng sau:

- Hm bc

0 (1.9)

- Hm ngng hai cc

11

2

+= xe

y vi >0 (1.10)

- 10 -


11/67

th cc dng hm truyn c biu din nh sau:

Hnh 1.4 th cc dng hm truyn

1.2 Cu to v phng thc lm vic ca mng nron

Da trn nhng phng php xy dng nron trnh by mc trn, ta c th

hnh dung mng nron nh l mt h truyn t v x l tn hiu. c tnh truyn

t ca nron phn ln l c tnh truyn t tnh.

Khi lin kt cc u vo/ra ca nhiu nron vi nhau, ta thu c mt mng

nron, vic ghp ni cc nron trong mng vi nhau c th l theo mt nguyn tc

bt k. V mng nron l mt h truyn t v x l tn hiu, nn c th phn bit

cc loi nron khc nhau, cc nron c u vo nhn thng tin t mi trng bn

ngoi khc vi cc nron c u vo c ni vi cc nron khc trong mng,chng c phn bit vi nhau qua vector hm trng s u vo w.

Nguyn l cu to ca mng nron bao gm nhiu lp, mi lp bao gm nhiu

nron c cng chc nng trong mng. Hnh 1.5 l m hnh hot ng ca mt mng

nron 3 lp vi 8 phn t nron. Mng c ba u vo l x 1, x2, x3 v hai u ra y1,

y2. Cc tn hiu u vo c a n 3 nron u vo, 3 nron ny lm thnh lp

u vo ca mng. Cc nron trong lp ny c gi l nron u vo. u ra ca

- 11 -


12/67

cc nron ny c a n u vo ca 3 nron tip theo, 3 nron ny khng trc

tip tip xc vi mi trng bn ngoi m lm thnh lp n, hay cn gi l lp

trung gian. Cc nron trong lp ny c tn l nron ni hay nron n. u ra ca

cc nron ny c a n 2 nron a tn hiu ra mi trng bn ngoi. Cc

nron trong lp u ra ny c gi l nron u ra.

Hnh 1.5 Mng nron ba lp

Mng nron c xy dng nh trn l mng gm 3 lp mc ni tip nhau i t

u vo n u ra. Trong mng khng tn ti bt k mt mch hi tip no. Mt

mng nron c cu trc nh vy gi l mng mt hng hay mng truyn thng

mt hng (Feed forward network), v c cu trc mng ghp ni hon ton (v bt

c mt nron no trong mng cng c ni vi mt hoc vi nron khc). Mng

nron bao gm mt hay nhiu lp trung gian c gi l mng Multilayer

Perceptrons) (MLP-Network).

Mng nron khi mi c hnh thnh th cha c tri thc, tri thc ca mng s

c hnh thnh dn dn sau mt qu trnh hc. Mng nron c hc bng cch

a vo nhng kch thch, v mng hnh thnh nhng p ng tng ng, nhng

p ng tng ng ph hp vi tng loi kch thch s c lu tr. Giai on ny

c gi l giai on hc ca mng. Khi hnh thnh tri thc mng, mng c th

gii quyt cc vn mt cch ng n. c th l vn ng dng rt khc

- 12 -


13/67

nhau, c gii quyt ch yu da trn s t chc hp nht gia cc thng tin u

vo ca mng v cc p ng u ra.

Nu nhim v ca mt mng l hon chnh hoc hiu chnh cc thng tin thu

c khng y hoc b tc ng ca nhiu. Mng nron kiu ny c

ng dng trong lnh vc hon thin mu, trong c mt ng dng c th l

nhn dng ch vit.

Nhim v tng qut ca mt mng nron l lu gi ng cc thng tin. Dng

thng tin lu gi ny chnh l quan h gia cc thng tin u vo v cc p

ng u ra tng ng, khi c mt kch thch bt k tc ng vo mng,

mng c kh nng suy din v a ra mt p ng ph hp. y chnh lchc nng nhn dng theo mu ca mng nron. thc hin chc nng

ny, mng nron ng vai tr nh mt b phn t chc cc nhm thng tin

u vo, v tng ng vi mi nhm l mt p ng u ra ph hp. Nh

vy, mt nhm bao gm mt loi thng tin u vo v mt p ng u ra.

Cc nhm c th c hnh thnh trong qu trnh hc, v cng c th khng

hnh thnh trong qu trnh hc.

Hnh 1.6 l mt s lin kt c th ca mng nron. Nron c v l cc vng

trn xem nh mt t bo thn kinh, chng c cc mi lin h n cc nron khc

nh cc trng s lin kt. Tp hp cc trng s lin kt ny s lp thnh cc ma

trn trng s tng ng.

1.2.1 Mng nron mt lp

Mi mt nron c th phi hp vi cc nron khc to thnh mt lp cc

trng s. Mng mt lp truyn thng nh hnh 1.6a. Mt lp nron l mt nhm cc

nron m chng u c cng trng s, nhn cng mt tn hiu u vo ng thi.

Trong ma trn trng s, cc hng l th hin nron, hng th j c th t

nhn nh mt vector wj ca nron th j gm m trng s wji. Cc trng s trong cng

mt ct th j (j=1,2,...,n) ng thi cng nhn mt tn hiu u vo xj.

wj = [wj1, wj2, ..., wjm]

- 13 -


14/67

Ti cng mt thi im, vector u vo x = [x1, x2,..., xn] c th l mt ngun

bn ngoi l cm bin hoc thit b o lng a ti mng.

(a) Mng truyn thng mt lp (b) Mng hi tip mt lp

(c) Mng truyn thng nhiu lp

(d) Mng nron hi quy

Hnh 1.6 Mt s dng mng nron

1.2.2 Mng nron truyn thng nhiu lp

Mng nron nhiu lp (Hnh 1.6.c) c cc lp c phn chia thnh 3 loi sau

y:

- 14 -


15/67

Lp vo l lp nron u tin nhn tn hiu vo x i (i = 1, 2, ..., n). Mi tn

hiu xi c a n tt c cc nron ca lp u vo. Thng thng, cc

nron u vo khng lm bin i cc tn hiu vo xi, tc l chng khng c

cc trng s hoc khng c cc loi hm chuyn i no, chng ch ng vai

tr phn phi cc tn hiu.

Lp n l lp nron sau lp vo, chng khng trc tip lin h vi th gii

bn ngoi nh cc lp nron vo/ra.

Lp ra l lp nron to ra cc tn hiu ra cui cng.

1.2.3 Mng nron phn hi

Mng nron phn hi l mng m u ra ca mi nron c quay tr li

ni vi u vo ca cc nron cng lp c gi l mng Laeral nh hnh 1.6b

1.2.4 Mng nron hi quy

Mng nron phn hi c th thc hin ng vng c gi l mng nron

hi quy nh hnh 1.6d. Mng nron hi quy c trng s lin kt i xng nh mng

Hopfield, mng lun hi t v trng thi n nh (Hnh 1.6.b). Mng BAM thuc

nhm mng nron hi quy, gm 2 lp lin kt 2 chiu, khng c gn vi tn hiu

vo/ra. Nghin cu mng nron hi quy m c trng s lin kt khng i xng, th

s gp phi vn phc tp nhiu hn so vi mng truyn thng v mng hi quy

c trng s lin kt i xng.

1.2.5 Mng Hopfield

Mng Hopfield l mng phn hi mt lp, c ch ra trong hnh 1.6.b. Cu

trc chi tit ca n c th hin trong hnh 1.7. Khi hot ng vi tn hiu ri rc,

n c gi l mng Hopfield ri rc, v cu trc ca n cng c gi l mng hi

quy.

- 15 -


16/67

Hnh 1.7 Cu trc ca mng Hopfield

Nh mng Hopfield v trn, ta thy nt c mt u vo bn ngoi xj v

mt gi tr ngng j (j = 1,2,...n). Mt iu quan trng cn ni y l mi nt

khng c ng phn hi v chnh n. Nt u ra th j c ni ti mi u vo

ca nt khc qua trng s wij, vi ij, (i = 1,2,...,n), hay ni cch khc wii = 0, (vi

i = 1,2,...,n).

Mt iu quan trng na l trng s ca mng Hopfield l i xng, tc l

wij = wji, (vi i,j = 1,2,...,n). Khi , lut cp nht cho mi nt mng l nh sau:

,sgn1

)()1(

+=

=

+n

ijj

ik

jijk

i xywy

i = 1,2,...,n (1.11)

Lut cp nht trn c tnh ton trong cch thc khng ng b. iu ny

c ngha l, vi mt thi gian cho trc, ch c mt nt mng cp nht c u ra

ca n. S cp nht tip theo trn mt nt s s dng chnh nhng u ra c

cp nht. Ni cch khc, di hnh thc hot ng khng ng b ca mng, mi

u ra c cp nht c lp.

- 16 -


17/67

C s khc bit gia lut cp nht ng b v lut cp nht khng ng b.

Vi lut cp nht khng ng b th s ch c mt trng thi cn bng ca h (vi

gi tr u c xc nh trc). Trong khi , vi lut cp nht ng b th c

th lm mng hi t mi im c nh hoc mt vng gii hn.

1.2.6 Mng BAM

Mng BAM bao gm hai lp v c xem nh l trng hp m rng ca mng

Hopfield. y ta ch xt mng ri rc, v n n gin v d hiu.

Hnh 1.8 Cu trc ca BAM

Khi mng nron c tch cc vi gi tr u vo ca vector ti u vo ca mt

lp, mng s c hai mu trng thi n nh, vi mi mu ti u ra ca n l mt

lp. Tnh ng hc ca mng th hin di dng tc ng qua li gia hai lp. C

th hn, gi s mt vector u vo x c cung cp cho u vo ca lp nron y.

u vo c x l v truyn ti u ra ca lp y nh sau:

y = a(wx) ;

= jiji xway ' ; vi i = 1,2,...,n (1.12)

a(.) l hm truyn, vector y by gi li nui tr li lp nron X v to nn

u ra nh sau:

x = a(wTy);

=

=

n

iiijj ywax

1

; vi j = 1,2,...,m (1.13)

- 17 -


18/67

Sau x nui tr li u vo ca lp y v to ra hm y theo phng trnh

(1.12). Qu trnh ny c tip tc, bao gm cc bc nh sau:

y(1) = a(wx(0)) (truyn thng ln th nht)

x(2) = a(w(T)y(1)) (truyn ngc ln th nht)

y(3) = a(wx(2)) (truyn thng ln th hai)

x(4) = a(w(T)y(3)) (truyn ngc ln th hai) (1.14)

y(k-1) = a(wx(k-2)) (truyn thng ln th k/2)

x(k) = a(w(T)y(k-1)) (truyn ngc ln th k/2)

Ch rng trng thi cp nht trong phng trnh (1.14) l ng b theo phngtrnh (1.12) v (1.13). Trng thi cp nht cng c th khng ng b theo phng

trnh (1.12) v (1.13) vi cc nt i, j c chn t do. Ngi ta ch ra rng, h

thng n nh cho c hai ch ng b v khng ng b. Tuy nhin, ch ng

b s lm cho h thng hi t nhanh hn nhiu.

1.3 Cc lut hc

Thng thng, mng nron c iu chnh hoc c hun luyn hngcc u vo ring bit n ch u ra. Cu trc hun luyn mng c ch ra

hnh di. y, hm trng s ca mng c iu chnh trn c s so snh u ra

vi ch mong mun (taget), cho ti khi u ra ca mng ph hp vi ch. Nhng

cp vo/ch (input/taget) c dng gim st cho s hun luyn mng.

Hnh 1.9: Cu trc hun luyn mng nron

- 18 -

ANNTrng s

wi

Sosnh

ch

Dliuvo

iu chnh


19/67

c c mt s cp vo/ra, mi gi tr vo c gi n mng v gi

tr ra tng ng c thc hin bng mng l s xem xt v so snh vi gi tr

mong mun. Bnh thng, n s tn ti mt sai s v gi tr mong mun khnghon ton ph hp vi gi tr thc. Sau mi ln chy, ta c tng bnh phng ca tt

c cc sai s. Sai s ny c s dng xc nh cc hm trng s mi.

Sau mi ln chy, hm trng s ca mng c sa i vi c tnh tt hn

tng ng vi c tnh mong mun. Tng cp gi tr vo/ra phi c kim tra v

trng s c iu chnh mt vi ln. S thay i cc hm trng s ca mng s

c dng li, nu tng cc bnh phng sai s nh hn mt gi tr t trc, hoc

chy mt s ln chy xc nh (trong trng hp ny, mng c th khng

tho mn yu cu t ra do sai lch cn cao). C hai kiu hc:

Hc tham s: l cc tham s v trng s cp nht kt ni gia cc nron.

Hc cu trc: trng tm l s bin i cu trc ca cc mng nron gm s

lng nt v cc loi lin kt.

Gi s ma trn trng s bao gm tt c cc phn t thch ng ca mng nron.

Nhim v ca vic hc tham s l tm ra c ma trn chnh xc mong mun t matrn gi thit ban u (vi cu trc ca mng nron c sn). lm c iu ny

th mng nron phi s dng cc trng s iu chnh, vi nhiu phng php hc

khc nhau c th tnh ton gn ng ma trn W cn tm c trng cho mng. Sau

y l 3 phng php hc:

1.3.1 Hc c gim st

Hc c gim st: l qu trnh hc c tn hiu ch o bn ngoi d (Hnh 1.10).Trong hc c gim st, th ti mi thi im khi u vo c cung cp ti mng

nron, phn ng u ra mong mun d tng ng ca h thng c a ra. hnh

(1.10), khi mi u vo x(k) c t vo mng, u ra mong mun tng ng d(k)

cng c cung cp ti mng. Hiu gia u ra thc y (k) v u ra mong mun d(k)

c o trong my pht tn hiu li. My ny s to ra tn hiu li cho mng hiu

- 19 -


20/67

chnh cc trng s ca mng, v vi cc hiu chnh ny th u ra thc s tin st

vi u ra mong mun.

Hnh 1.10: Hc c gim st

1.3.2 Hc cng c

Tn hiu ch o d c th ly t mi trng bn ngoi, nhng tn hiu ny

khng c y , m ch c mt vi bit i din c tnh cht kim tra qu trnh tt

hay xu. Hc cng c cng l mt dng ca hc c gim st, bi v mng vn nhn

mt s tn hiu t bn ngoi. Nhng tn hiu phn hi ch mang tnh cht nh gi

hn l mng tnh cht ch dn. N cho bit mc tt hay xu ca mt u ra c

bit. Tn hiu cng c bn ngoi thng c x l bng my pht tn hiu nh gi

to ra nhiu hn na cc thng tin tn hiu nh gi, sau dng iu chnh

cc trng s vi mc ch t c tn hiu nh gi tt hn.

1.3.3 Hc khng c gim st

Hnh 1.11: Hc khng c gim st

Trong phn hc khng c gim st, s khng c thy hng dn, tc l

khng c tn hiu d cung cp ti mch phn hi. iu ny cho thy, ta s khng bit

- 20 -


21/67

u ra t gi tr g. Vi loi ny, th cc nron t xoay x vi cc d liu mu m

n c c, ch khng c thy gi cn luyn theo hng no. Mng phi t

khm ph mu, c tnh, s tng quan hay loi u vo. Trong khi khm ph

nhng c tnh ny, tham s ca mng s b thay i. Qu trnh ny c gi l t

t chc. Mt v d in hnh l qu trnh phn loi i tng khng c thy, nhng

lp thch hp c hnh thnh bng cch khm ph s tng t v khng tng t

trong s cc i tng.

Hnh (1.12) m t cu trc chung ca qu trnh hc ca ba phng php hc

nu trn. Trong tn hiu vo xj (j = 1,2,...,m), c th c ly t u ra ca cc

nron khc hoc c th c ly ra t bn ngoi. Trng s ca nron th i cthay i ty theo tn hiu u vo m n thu nhn gi tr u ra ca n.

Dng tng qut ca lut hc trng s ca mng nron cho bit s gia ca

vector wi l iw t l vi tn hiu hc r v tn hiu u vo x(t).

iw (t) = )(.. txr (1.15)

l mt s dng v c gi l hng s hc dng xc nh tc hc, r l tn

hiu hc v ph thuc:).,,(

iirdxwfr

= (1.16)

Hnh 1.12: S cu trc chung ca qu trnh hc

- 21 -


22/67

T hnh (1.12) ta thy, vector trng s wi = [wi1, wi2,..., wim]T c s gia t l

vi tn hiu vo x v tn hiu hc r. Vector trng s thi im (t+1) c tnh nh

sau:

wi(t+1) = wi(t) + fr(wi(t),x(t),d(t)).x(t). (1.17)

Phng trnh lin quan n s bin i trng s trong mng nron ri rc, v tng

ng vi s thay i trng s trong mng nron lin tc theo biu thc sau:

)(.. txrdt

dwi = (1.18)

Vn quan trng trong vic phn bit lut hc cp nht trng s c gim st

hay khng c gim st, hoc hc cng c l tn hiu hc r. Nh vy, i vi tn hiu

hc r th n lm th no thay i hoc cp nht trng s trong mng nron.

Mng nron nhn to c cc tnh cht sau:

L h phi tuyn

L h x l song song

L h hc v thch nghi: Mng c luyn t s liu qu kh, c kh nng t

chnh u vo khi s liu u vo b mt.

L h nhiu bin, nhiu u vo, nhiu u ra (MISO), rt tin dng khi iu

khin i tng c nhiu bin s.

1.4 Thut ton lan truyn ngc

Thut ton lan truyn ngc c ng dng gii cc bi ton iu khin cc

h phi tuyn phc tp v bt n nh. Lan truyn ngc l mt phng php cho

php xc nh tp trng s tt nht ca mng gii mt bi ton cho. Vic p

dng phng php lan truyn ngc l mt qu trnh lp i lp li nhiu ln hai tin

trnh chnh: lan truyn tin thc hin nh x v lan truyn ngc sai s cp

nht cc trng s. Cc trng s ca mng l cc h s ca m hnh. Phng php

gim gradient c dng cp nht nhng h s ny sao cho gim thiu c sai

s ca m hnh.

Xt lp mng 3 lp nh hnh (1.14)

- 22 -


23/67

Hnh 1.14: Mng 3 lp lan truyn ngc

Thut ton: u tin ta cho lan truyn thng sut trong mng, qua cc phn t

nron v c tip tc vi cc hm kch hot ca phn t nron. Cc mng c

nghin cu cng vi thut ton hc lan truyn ngc c gi l mng lan truyn

ngc.

Hun luyn cc cp vo/ra.

{(x(k), d(k))}, k = 1,2,...,p

Thut ton cung cp mt th tc cho vic thay i cc vector trng s trong

mng, u ra ca mng c lan truyn ngc tr li lp u vo cho ng cc

mu. C s cho vic cp nht cc trng s l phng php dc Gradient.

Vi cp vo ra (x(k), d(k)), thut ton lan truyn ngc thc hin cc bc nh

sau:

u tin, mu x(k)

c lan truyn t lp u vo qua cc lp n i n lp ura c kt qu l y(k). Sau , sai s gia y(k) v d(k) c lan truyn ngc tr li t

lp u ra ti lp u vo cp nht trng s. Hnh (1.14) din gii thut ton lan

truyn ngc. Kt qu c th m rng sang mng nron nhiu lp.

Trn hnh (1.14) c m phn t nron u vo, l phn t nron lp n, v n

phn t nron lp u ra. ng nt lin din t lan truyn thng ca cc tn hiu,

ng nt t din t lan truyn ngc ca cc sai s. u tin hun luyn vo cp

- 23 -


24/67

vo/ra k hiu (x,d) cho n gin ta b ch s k. Khi mt mu u vo x c

a vo th cc phn t trong mng s c tnh nh sau:

u vo phn t q ca lp n s c tnh theo phng trnh:

j

m

jqjq xvnet

=

=1

(1.19)

Phng trnh u ra ca q s l: ()( anetaz qq == jm

jqjxv

=1) (1.20)

u vo phn t th i ca lp u ra s l:

)(111

j

m

jqj

l

qiqq

l

qiqi xvawzwnet

===

== (1.21)

Phng trnh u ra ca phn t nowron th i s l:

))(.()()(111

j

m

jqj

l

qiqq

l

qiqii xvawazwanetay

===

=== (1.22)

Cc ch s trn c tnh ton cho s lan truyn tin ca cc tn hiu u vo

xuyn sut qua cc lp mng nron. Trc khi ta cp n cc tn hiu sai s ca

s lan truyn ngc, ta s nh ngha mt hm mc tiu nh sau:

[ ]

2

1 1

2

11

2

2

1)(

2

1)(

2

1)(

= ===

===

n

i

l

q

qiqi

n

i

ii

n

i

ii zwadnetadydwE (1.23)

Sau , theo phng php dc Gradient, cc trng s ni gia lp n v lp u

ra c cp nht bi iqw , v n c tnh theo cng thc sau:

iqiq w

Ew

= (1.24)

S dng cc cng thc (1.21-1.23) v thay i lut vi iqwE

, ta c:

[ ][ ][ ] qoiqiiiiq

i

i

i

i

i

iiq zznetaydw

net

net

net

net

y

y

Ew ==

= )(' (1.25)

Trong , oi l tn hiu sai s, ch s di th hai l im th i trong lp u ra.

Sai s tn hiu c nh ngha bi:

- 24 -


25/67

[ ][ ])(' iiii

i

iioi netaydnet

y

y

E

net

E=

=

= (1.26)

Trong , neti l u vo ca phn t nron th i trong lp u ra v

i

ii net

netaneta

=)(

)(' .

By gi ta phi tnh u ra zq ca lp n:

Vi trng s ni gia u vo v cc lp n, ta s dng thay i lut cng phng

php dc Gradient, ta cp nht trng s kt ni gia phn t th j ca lp u

vo vi phn t th q ca lp n. Khi :

=

=

=

qj

q

q

q

qqj

q

qqj

qjv

net

net

z

z

E

v

net

net

E

v

Ev (1.27)

T cng thc (1.23), th mi sai s [di-yi], vi i=1,2,...,n l mt hm ca zq.

nh gi thay i lut ta c:

( )[ ] jqn

iiqiiiqj xnetawnetaydv ).('.).('.

1=

= (1.28)

S dng cng thc (1.26), ta c th vit li cng thc (1.28) nh sau:

[ ] jhqjqn

iiqoiqj xxnetawv ==

=

).('..1

(1.29)

y, hq l sai s tn hiu ca phn t th q ca lp n v c nh ngha nh

di y:

iq

n

ioiq

q

q

qqhq wnetanet

z

z

E

net

E =

=

=

=1

)(' (1.30)

Trong , netq l u vo phn t th q ca lp n.Tn hiu sai s ca mt phn t trong lp n khc so vi tn hiu sai s ca

mt phn t trong lp u ra, nh c ch ra trong cng thc (1.30) v (1.26).

Do c s khc nhau ny, nn cc th tc cp nht cc trng s trn c gi l lut

hc delta tng qut. Chng ta xem xt cng thc (1.30), sai s tn hiu hq ca phn

t lp n q c th c xc nh trong cc mu ca cc tn hiu sai s oi ca cc

- 25 -


26/67

phn t lp ra th i (yi) cung ng. Cc h s l cc trng s c s dng cho lan

truyn thng, nhng y chng truyn cc tn hiu sai s ( oi ) ngc tr li,

chnh l cc ng nt t trong hnh (1.13). iu ny chng t c c imquan trng ca thut ton lan truyn ngc lut cp nht cc b, y chnh l tnh

ton trng s thay i da vo s kt ni, v chng ta ch cn gi tr hai u ca

kt ni ny.

S o hm trn c th d dng m rng cho mng c nhiu hn mt lp

n, bng cch s dng chui lut lin tip. Trong trng hp chung, vi s lp ty

, th s lan truyn ngc c cp nht lut dng sau:

jinputioutputjiij xxw == (1.31)

y, (output-i) v (input-j) quy vo hai u ca s kt ni t phn t th j

ti phn t th i, xj l u vo cui cng kch hot t mt phn t lp n, hoc t

mt u vo bn ngoi. Ngoi ra, i l tn hiu hc c nh ngha bi cng thc

(1.26) vi u ra hoc lp cui cng ca cc trng s kt ni, v c nh ngha

bi cng thc (1.30) cho tt c cc lp khc. Khi hm sigmoid lng cc c s

dng lm hm kch hot, ng thi s dng (1.26) v (1.30) ta c hm y c xcnh nh sau:

11

1)(

+== nete

netay

Khi ta c: [ ] )1(2

1)(1

2

1)()(' 22 yneta

net

netaneta ==

=

[ ]iioi ydy = )1(2

1 2 (1.32)

iq

n

ioihq wz

=

=1

2 )1(2

1 (1.33)

Thut ton lan truyn ngc

Xem xt mt mng vi Q lp lan truyn ngc, q=1,2,...,Q; vi qneti v qyi

ln lt l u vo v u ra ca khi trong lp th q. Mng c m nron u vo, l

nron lp n, v n nron u ra. Vi qwij l trng s ni t q-1wj n qyi.

- 26 -


27/67

u vo: cc cp hun luyn {x(k), d(k) | k=1,2,...,p}, gi tr u vo ca

phn t cui cng bng -1, tc l 1)( 1 =+k

mx .

Bc 0 (t gi tr ban u)- La chn bc tnh (Hng s hc) 0


28/67

Bc 4 (lan truyn ngc sai s)

Cc sai s lan truyn ngc vi mc ch cp nht cc trng s v

tnh ton cc tn hiu sai lch iq 1

cho cc lp x l:j

qi

qij

q yw 1.. = ; ijqold

ijqnew

ijq www += (1.38)

j

j

q

ji

q

i

q

i

q wneta = )( 1'1 ; vi q=Q,Q-1,...,2 (1.39)

Trong :

- l sai lch ti thi im tnh ca gi tr trng s lin kt cp nht

mi v c, lin kt t phn t th j ca lp q-1 n phn t i ca lp q.

- l gi tr trng s lin kt cp nht mi t phn t th j ca lp

(q-1) n phn t i ca lp q.

- l gi tr trng s lin kt c t phn t th j ca lp (q-1) n phn

t i ca lp q.

- l tn hiu ra ca phn t j ca lp (q-1).

Bc 5 (Sau mi vng lp)

Kim tra xem lp ht cc gi tr mu hun luyn cha, nu cha quay

vng ht (tc l k


29/67

mng nron nhn dng mt mu, n ging nh qu trnh hc ca lan truyn ngc,

l m t mt qu trnh mng nron s c hun luyn.

Mt mng nron truyn thng l mt mng m cc nron ch c kt

ni ti mt lp tip theo. Khng c kt ni gia cc nron vi cc lp trc hoc

vi chnh nron . Thm vo , cc nron s khng kt ni ti cc nron qu xa

lp tip theo. Mt mu c x l bi s truyn thng, lch v cc trng s kt

ni s c p dng.

Cc mng nron c hun luyn bng cch s dng thut ton lan truyn

ngc. Thut ton lan truyn ngc l mt dng hun luyn c gim st. Mng

nron c a vo d liu hun luyn cho cho ra kt qu, cc kt qu ny c sosnh vi cc kt qu c mong i. S khc nhau gia cc kt qu thc t v cc

kt qu mong i s c mt mc sai s no . gim thiu sai s ny, ngi ta

tin hnh s dng thut ton lan truyn ngc iu chnh cc trng s v lch

ca mng. y l cch m sai s c gim ng k. Trong chng tip theo,

chng ta s tm hiu v mng nron Kohonen. S khc nhau quan trng nht gia

mng nron Kohonen v mng nron lan truyn ngc l phng thc hun luyn.

Phng thc lan truyn ngc s dng phng thc hun luyn c gim st. Cn

mng nron Kohonen s dng phng thc hun luyn khng gim st.

- 29 -


30/67

Chng 2. Tm hiu mng nron Kohonen

2.1 Gii thiu

Trong cc kiu khc nhau ca mng nron, mng nron Kohonen ging vi

mng nron sinh hc hn c v cu to ln c ch hc. Mng nron nhn to ny

ln u tin c gii thiu bi Kohonen vo nm 1982, n c bit n nh l

nh x c trng t t chc (SOM). nh x t t chc ca Kohonen cn c bit

n l mt trong nhng m hnh kh n gin ca mng nron, v ngi ta thng

gi l mng nron Kohonen.

Mng nron Kohonen (hay nron t t chc) m hnh ha cch hot ng bno ca con ngi, mc d n kh n gin. Chng ta c th nhn c mt s

tng cch b no lu tr hnh nh v cch n nhn dng cc hnh nh.

Cch x l thng tin cc mng nron khc thng ch quan tm n gi tr v

du hiu ca thng tin u vo, cha quan tm khai thc cc mi lin h c tnh cht

cu trc trong ln cn ca cc vng d liu mu, hay ton th khng gian. Nhng

trong mng nron Kohonen quan tm n cc yu t ny.

T t chc trong mng nron l mt trong nhng ch cun ht trong

mng nron. Mt mng nron nh vy c th c luyn tm ra cc quy lut v

cc tng quan, cc gi tr nhp vo v d on cc kt qu tip theo. Cc nron

ca mng thng qua qu trnh luyn cnh tranh nhn ra mt nhm cc i tng

u vo tng ng nhau. Mc ch chnh ca vic luyn trong mng nron

Kohonen l nhn dng mt nhm cc vector u vo cng loi.

Trong phn tip theo, chng ta c th ch ra hnh nh c lu tr v nhn

dng vi mng nron Kohonen. Vic thi hnh mng nron Kohonen c th c

thay th bi mt thut ton tng ng m d dng thi hnh, v lun lun c s

dng trong cc ng dng ca mng nron Kohonen. Chng ta gi thut ton l

thut ton mng nron t t chc (Kohonen, 1988) hay nh x t t chc SOM.

tng ng ch ca thut ton ny l nh x cc c trng topo t t chc nhm

bo ton trt t sp xp cc mu trong khng gian biu din nhiu chiu sang mt

- 30 -


31/67

khng gian mi vi cc mng nron c s chiu nh hn, thng l hai chiu. y

l mt php chiu phi tuyn em li mt nh x c trng hai chiu, n c th

c s dng trong vic pht hin v phn tch nhng c trng trong khng gian

u vo. Ta hiu iu ny nh l bo ton cu trc cc c trng. Trong mng

nron Kohonen, cc vector tn hiu u vo gn nhau s c nh x sang cc

nron ln cn trong mng.

K thut SOM c p dng thnh cng trong mt s lnh vc nh nhn

dng, phn cm d liu, d on chui v khai ph d liu,... Mu c nhn dng

c th l nh, m thanh hoc vn bn,... C th xem SOM l mt lp in hnh,

nhng rt n gin ca cc mng nron Kohonen.2.2 Mng nron Kohonen

Mng luyn khng khng c thy hng dn, l mt kiu luyn m cc

nron t xoay x vi cc d liu mu m n c c ch khng c ng thy gi

cn luyn theo hng no.

T mnh khm ph nhng quan h ang c quan tm, v d v cc dng

( patterns), cc c trng (features ) t d liu vo (input data) sau chuyn

thnh ci ra (outputs). Nh vy thc cht : l cc mng t t chc (hay mng

nron Kohonen) .

nh ngha: Mng noron Kohonen l mng c kh nng s dng nhng kinh

nghim ca qu kh thch ng vi nhng bin i ca mi trng (khng d

bo trc). Loi mng ny thuc nhm h hc, thch nghi khng cn c tn hiu ch

o t bn ngoi.

Trong phn ny chng ta s trnh by mt s quy trnh luyn tham s ca

luyn khng c thy nh sau:

M hnh

Mng c n nron PEi , i=1,2,..., n.

Ci ra ca chng l iy , i=1,...,n.

C m tn hiu vo {x1, x2,..., xm},wij l trng s lin kt t xj vi PEi .

- 31 -


32/67

Gi s(x), s(y) l hm chuyn tn hiu, gi thit l hm n iu khng

gim lin tc nh dng hm Sigmoid.

Phng thc bin i trng s c gi l lut luyn Hebb, quy tc luyn

n gin theo dng Hebb cho bi phng trnh :

)().()(

'jjiiij

ijij xsyswdt

tdww +== (2.40)

By gi ta xt mt s trng hp ring:

- Quy tc luyn cnh tranh (Competitive Learning Rule Grossberg 1969,

Rumelhart 1986)

))()((

'

ijjjiiij wxsysw = (2.41)

y1

( ) , 01 i

i i cys y c

e

= >

+

(2.42)

- Nu dng ( )j j js x x= ta thu c :

Quy tc luyn cnh tranh tuyn tnh (the Linear competitive learning rule)

))((' ijjiiij wxysw = (2.43)

Trng hp ring quan trng l quy tc thng ly tt c - the winner-take-alllearning rule ca Kohonen , gii thch hiu qua bi ton phn cm tp mu

{ }1 2, ,..., pX x x x= thnh n cm , vi n cho.

K hiu 1 2, w (w ,w ,...,w )m m

i i i imx R R = , l h s hc.

Ti mi vng lp k, quy tc luyn gm 2 bc :

1/ Bc tm ci khp nht (matching) tm nron tm gi l nron thng (theo

ngha gn mu nht), s dng cng thc sau:

x

wxyyy m

.)...max( 1 == , (2.44) (Phng php tnh tch v hng).

2/ Tnh ton sai s v iu chnh trng s

K hiu ej l sai s ct th j, w ij l biu din cho ct th j ca ma trn trng s w,

sai s ny c tnh theo cng thc sau:

ej = ||x-wij|| (2.45)

- 32 -


33/67

Nu tng sai s cha nm di mc cho php, ta iu chnh trng s theo cng

thc:

)(1 kiji

k

ij

k

ij

wxww +=+ vi k =1,,m (2.46)

Khi mt mu c a ti mt mng Kohonen, nhng nron c chn l

nron thng (winner) (nron thch hp nht theo ngha m ta t ra). Nron thng

ny l d liu u ra t mng Kohonen. Thng thng, cc nron thng ny tng

ng vi cc nhm trong d liu a vo mng Kohonen.

Mng Kohonen c hun luyn trong mt ch khng c gim st. S dng

mng Kohonen ny, d liu c th c phn loi thnh tng cm. Chng ta s xem

xt mng Kohonen qua qu trnh hun luyn.

2.2.1 Mng nron Kohonen nhn dng

Ta s xem vic hun luyn mng nron Kohonen nh th no nhn dng

ng cc mu m ta mong mun. Chng ta bt u bng cch xem xt cu trc ca

mng nron Kohonen.

2.2.2 Cu trc ca mng nron KohonenMng nron Kohonen ch bao gm mt lp d liu u vo v mt lp d

liu u ra ca cc nron v n khng cha lp n.

Lp d liu u vo i vi mng nron Kohonen l cc nron u vo. Cc

nron u vo ny to thnh mu d liu u vo ca mng. i vi mng nron

Kohonen, ta nn chn d liu u vo chun ha trong khong gia -1 v 1. Khi

thc thi mu d liu u vo, mng s to ra cc nron u ra.

Lp u ra ca mng nron Kohonen rt khc vi lp u ra ca mng nron

truyn thng. i vi mng truyn thng, nu chng ta c mt mng nron vi 5

nron u ra, chng s c th cho kt qu bao gm 5 gi tr. Cn trong mng nron

Kohonen ch c mt nron u ra cho ra mt gi tr. Gi tr duy nht ny c th l

ng hoc sai. D liu u ra t mng nron Kohonen thng l cc ch s ca

nron (V d nron s 5,). Cu trc c trng ca mng nron Kohonen c ch

ra trong hnh 2.1.

- 33 -


34/67

Hnh 2.1: Mt dng mng nron Kohonen

By gi, chng ta xem xt mng nron Kohonen x l thng tin nh th no.

kim tra qu trnh ny, chng ta xem xt mt v d sau:

V d

Chng ta s xm xt mt mng nron Kohonen n gin. Mng ny s ch c

2 nron u vo, v 2 nron u ra. D liu u vo c cho l 2 nron c ch

ra trong bng 2.1 v cc trng s kt ni gia cc nron trong bng 2.2.

Bng 2.1 Bng 2.2

S dng cc gi tr ny, chng ta xem xt nron c th thng v cung cp d

liu u ra. Chng ta bt u bng cch chun ha d liu u vo.

2.2.3 Chun ha d liu u vo

Mng nron Kohonen i hi d liu u vo phi c chun ha. Yu cu

ca mng nron Kohonen l d liu u vo ca n phi c phn hoch trn min

xc nh gia -1 v 1. Mi d liu u vo nn s dng hon ton min xc nh, v

nu cc nron u vo ch s dng cc s gia 0 v 1, th thnh qu ca mng nron

s khng cao (b tn tht).

- 34 -


35/67

chun ha d liu u vo, chng ta s tnh ton di vector (vector

length) ca cc d liu u vo, hoc vector u vo. Trong trng hp ny di

vector s l: (0.5 * 0.5) + (0.75 * 0.75) = 0.8125.

Vy trng hp trn s c di vector l 0.8125. S dng di ny,

chng ta c th xc nh c h s chun ha. H s chun ha l s nghch o

ca cn bc hai di vector u vo. Trong trng hp trn th h s chun ha l:

Tnh ton gi tr trn cho kt qu h s chun ha l 1.1094. H

s chun ha ny s s dng cho bc tip theo, l tnh ton u ra

cho nron.

2.2.4 Tnh ton d liu u ra ca nron

tnh ton d liu u ra, vector d liu u vo v cc trng s k ni

nron u phi c xem xt. Th nht, tnh tch v hng ca hai vector vector

u vo v vector cc trng s lin kt. tnh ton tch v hng gia hai vector

ny, ta thc hin nh sau.

Ta bt u tnh ton cho nron thc nht. Thut ton Kohonen cho vic lytch v hng hai vector ca vector u vo v vector trng s c tnh nh sau:

Php tnh ton trn "dot product" cho kt qu l 0.395. Tnh ton ny s c

thc hin cho nron u ra u tin, v n s c thc hin nh vy i vi mi

nron u ra tip theo.

By gi, d liu u ra ny phi c chun ha bng nhn kt qu ca "dotproduct" trn (0.395) vi h s chun ha l 1.1094, v cho kt qu l 0.438213. D

liu u ra c tnh ton v chun ha, n c nh x ti mt s lng cc?.

2 .2.5 nh x lng cc

Khi to bn lng cc, s 0 s l -1, v cc s cn li l 1. Tc l to bn

lng cc t [-1,1] ti [0,1].

- 35 -


36/67

V d, bin i chui 0101 v s lng cc ta lm nh sau:

Do d liu u vo c chun ha trong khong [-1,1] nn ta phi thc hin

chun ha tng t i vi nron u ra. lp c bn , ta cng thm 1 vo

kt qu ri ly kt qu . Theo kt qu trn c kt qu u ra l 0.438213, v kt

qu trong d liu u ra cui cng l :

(0.438213+1)/2 = 0.7191065.

Gi tr 0.7191065 l d liu u ra ca nron u tin. Gi tr ny s c sosnh vi cc d liu u ra ca nron khc. Bng cch so snh cc gi tr ny,

chng ta c th xc nh c nron thng.

2 .2.6 Chn nron thng

Nu chng ta mun xc nh c mt nron u ra thng, chng ta phi

tnh ton gi tr cho nron u ra th hai. By gi, chng ta s xem nhanh qu trnh

tnh ton cho nron u ra th hai.Nron d liu u ra th hai s s dng h s chun ha ging nh s

dng tnh ton cho nron u ra u tin. Theo trn ta c h s chun ha l

1.1094. Chng ta p dng phng php tnh tch v hng cho trng s ca nron

u ra th hai v vector d liu u vo, chng ta s nhn c mt gi tr l :

|0.5 0.75|*|0.3 0.4| = (0.5*0.75)+(0.3*0.4) = 0.45.

Gi tr ny c nhn vi h s chun ha 1.1094, chng ta nhn c kt

qu l 0.49923.

By gi chng ta c gi tr u ra cho mi nron. Nron u tin c gi tr

u ra l 0.7191065 v nron th hai c mt gi tr u ra l 0.49923. chn

nron thng, chng ta chn gi tr ca d liu u ra l ln nht. Trong trng hp

ny th nron thng l nron u ra u tin vi gi tr u ra l 0.7191065.

Nh vy ta xc nh c gi tr u ra ca mng nron Kohonen. Ta

thy cc trng s gia nron u vo v nron u ra xc nh d liu u ra. Trong

- 36 -


37/67

phn sau, chng ta s iu chnh cc trng s ny n cung cp d liu u ra ph

hp vi ci m ta mong mun.

2 .2.7 Qu trnh hc ca mng Kohonen

Ton b qu trnh hun luyn cho mt mng nron Kohonen cn phi lp li

qua vi cng on. Nu sai s tnh ton ca mng nron Kohonen mc thp

hn mc c th chp nhn c th s hon tt qu trnh hun luyn. tnh ton t

l sai s cho mng nron Kohonen, ta s iu chnh cc trng s cho mi cng

on.

Qu trnh hun luyn cho mng nron Kohonen l luyn cnh tranh nn mi

tp hun luyn s c mt nron thng. Nron thng ny s c trng s c iu

chnh sao cho ngay lp tc n s tc ng tr li mnh m hn trong d liu u

vo ln tip theo. S khc nhau gia cc nron thng s dn ti s khc nhau gia

cc mu u vo tip theo.

Chng ta xem xt ton b qu trnh lin quan n vic hun luyn mng

nron Kohonen. Cc bc c tm tt trong hnh 2.2.

T hnh 2.2 ta thy, mng nron Kohonen c hun luyn bi cc cngon tun hon cho n khi mt trong hai vn sau xy ra:

- Nu tnh ton cc sai s mc thp hn mc c th chp nhn c th

nhim v ca chu k s hon thnh qu trnh hun luyn.

- Nu tt c cc t l sai s ch thay i bi a s cn bin, th chu k ring

l ny s b loi bm v cc trng s li c khi to li vi cc gi tr

ngu nhin, ng thi mt chu k hun luyn mi li bt u. Chu k

hun luyn ny s tip tc chu k hun luyn trc v n s phn tch cc

cng on a ra kt qu; hoc l chu k b loi b hoc to ra mt tp

cc trng s m c mc sai s c th chp nhn c.

- 37 -

Dng

Bt u

Khi to ngu nhin ma trn trngs

Tnh ton sai s

Kim tra sai s

xem c mcchp nhn ckhng?

Thc hin hun luyn, iuchnh trng s da vo nron

thng

Tnh ton li t l sai s, ghili nhng gi tr c ci

thin

Ci thin t lsai s l tm

thng?

Ma trn trng sny tt nht

cha?

Nu c ma trntrng s tt hn

ca ma trn trngs tt nht?

Gn ma trn trng s ny thnhma trn trng s tt nht

qu s chuk xc nh?

Yes

No

YesYes

No

No

No

No

Yes

Yes


38/67

Hnh 2.2: S khi biu din hun luyn mng nron Kohonen.

2 .2.8 T l (tc ) hc

Tham s hc l mt hng v n l mt s nh hn 1, trn k hiu l .

- 38 -


39/67

Thng thng, ta t cho tham s hc mt gi tr t 0.4 n 0.5, nu t l hc

ln th qu trnh hun luyn nhanh hn. Tuy nhin, vic t cho t l hc mt gi tr

qu ln c th s lm cho mng khng bao gi hi t. iu ny lm thay i cc

vector trng s s qu ln gy nh hng ti vic phn loi cc mu nn ta ch cn

chn t l hc va , v s dng n iu chnh cc trng s ca cc nron.

Trong phn tip theo, chng ta s xem cc trng s ny c iu chnh s dng

tham s hc nh th no.

2 .2.9 iu chnh cc trng s (cp nht trng s)

Mt cng on xut hin khi d liu hun luyn c a vo mng nron

Kohonen, v cc trng s c iu chnh da trn cc kt qu ca d liu hun

luyn. S iu chnh cc trng s s lm cho mng c thun li hn ngay sau khi n

c a tr li mng. Ma trn trng s cui cng c s dng s l ma trn tt

nht c xc nh t mi chu k. By gi, chng ta s xem xt cc trng s ny

c thay i nh th no.

Phng php cho vic tnh ton thay i cc trng s s dng phng trnh

sau:)(1 ttt wxww +=+ (2.48)

Bin x l vector hun luyn c a vo mng. Bin wt l trng s ca

nron thng, v bin wt+1 l trng s mi, l h s hc.

2 .2.10 Tnh ton sai s

Khi mng hun luyn khng gim st bao gi cng c sai s, l s khc

nhau gia kt qu mong i v kt qu thc t ca mng nron. Sai s m chng ta

tnh ton l nhng ci m khng ng gia kt qu mong i v kt qu thc t.

ej = ||x-wj||

Mc ch ca mng nron Kohonen l phn loi d liu u vo vo trong

cc tp khc nhau, cho nn sai s ca mng nron Kohonen phi c o lng. Sai

s ny s c tnh ton trong qu trnh hun luyn mng.

- 39 -


40/67

2 .3. Thc thi mng nron Kohonen

C vi lp c th c s dng cng nhau to ra mt mng nron

Kohonen. Chng sau ta s ch ra cch xy dng mt ng dng da trn mng

nron Kohonen, l ng dng nhn dng k t quang (c th l nhn dng ch

vit tay). Cc lp c m t nh sau:

KohonenNetwork Thc thi cc phng thc thuc v mng nron

Kohonen. y l ni m mng nron Kohonen c hun luyn v ly cc

mu.

Network Cha ng cc phng php khng thuc v mng nron

Kohonen. Cc lp ny cha cc phng php tnh ton tch v hng, vchiu di vector.

NeuralReportable Mt giao din n gin cho php mng nron

Kohonen tr v thng tin tin b sau khi mng nron c hun luyn.

TrainingSet Mt tp hun luyn cha i tng, l c th cha cc

mng ca cc ln hun luyn ring l. Tp hun luyn c th cha c cc

phn t d liu u vo v d liu u ra.

Cc lp ny hot ng cng nhau cung cp cc chc nng cho mng

Kohonen. Trc tin, ta xem vic thc thi mng nron truyn thng xem chng hot

ng nh th no.

2.3.1 Thc thi mng nron truyn thng

Khi thc thi mng nron truyn thng, d liu c truyn n i tng

synapse. Phng thc run chy mt vng lp lun lun i mu d liu mi v

sau chuyn chng thnh d liu u ra, danh sch 2.1 ch ra phng thc run

hot ng nh mu ca lp Layer.

Danh sch 2.1: Phng thc Layer.run

public void run() {

while ( running ) {

int dimI = getRows();

int dimO = getDimension();

// Nh mu

- 40 -


41/67

inps = new double[dimI];

this.fireFwdGet();

if ( m_pattern != null ) {

forward(inps);

m_pattern.setArray(outs);

fireFwdPut(m_pattern);

}

if ( step != -1 )

// Kim tra nu bc tip theo ang c hc

m_learning = monitor.isLearningCicle(step);

else

// Dng mng

running = false;//Nu ((m_learning) && (m_batch != 1))

if ( (m_learning) && (running) )

{ // Hc

gradientInps = new double[dimO];

fireRevGet();

backward(gradientInps);

m_pattern = new Pattern(gradientOuts);

m_pattern.setCount(step);

fireRevPut(m_pattern);

}

} // Kt thc while (running = false)

myThread = null;

}

Phng thc fireFwdPut c gi truyn mu vo trong synapse u ra.

forward(inps);

m_pattern.setArray(outs);

fireFwdPut(m_pattern);Khi cc phng thc forward v fireFwdPut c gi, c ba bin lin quan

l:

Cc trng s kt ni

lch trng s

Hm ngng

- 41 -


42/67

Phng thc u tin c gi l phng thc fireFwdGet. Cng vic chnh

ca phng thc ny l nhn mu t synapse u vo, v i cho n khi khng

c mu no c a vo.

Lp d liu u vo ch nhn cc gi tr {0,1} ca mu nh. Sau , lp

u vo s p dng hm ngng, a chng n lp tip theo. Chng ta s dng

kiu hm sigmoid trong lp d liu u vo, thc thi hm sigmoid qua lp

SigmoidLayer. Danh sch 2.2 ch ra phng thc SigmoidLayer.forward().

Danh sch 2.2: Phng thc SigmoidLayer.forward

public void forward(double[] pattern)

{

int x;double in;

int n = getRows();

try {

for ( x = 0; x < n; ++x ) {

in = pattern[x] + bias.value[x][0];

outs[x] = 1 / (1 + Math.exp(-in));

}

} catch ( Exception aioobe ) {

aioobe.printStackTrace();

}

}

T danh sch 2.2 ta thy, phng thc SigmoidLayer.layer() p dng hm

sigmoid cho mi nron trong lp ny. Hm sigmoid c cp trn.

Phng thc Layer.run x l d liu u vo bng cch s dng hm

ngng sigmoid, lp ny sn sng a mu ti lp k tip. Khi mu c a vo

lp tip theo, th cc trng s thch hp c cp nht.By gi, phng thc Layer.run p dng hm ngng cho mi gi tr nron,

lp phi truyn mu ln trn synapse. Synapse ny p cc trng s kt ni v gi

mu ti lp tip theo, chng c th hin phng thc fireFwdPut(). Phng

thc fireFwdPut c ch ra trong danh sch 2.3.

Danh sch 2.3: Phng thc Layer.fireFwdPut

protected void fireFwdPut(Pattern pattern) {

- 42 -


43/67

if ( aOutputPatternListener == null ) {

return;

};

int currentSize = aOutputPatternListener.size();

OutputPatternListener tempListener = null;

for ( int index = 0; index < currentSize; index++ ){

tempListener =

(OutputPatternListener)aOutputPatternListener.elementAt(index);

if ( tempListener != null ) {

tempListener.fwdPut((Pattern)pattern.clone());

};

};

}Phng thc Layer.fireFwdPut chu hai trch nhim. Th nht, n p cc

trng s kt ni gia cc nron lp hin thi cho cc nron lp tip theo. Th

hai, n chuyn mu ny n synapse. Phng thc ny chuyn mu n synapse

bng cch gi phng thc Synapse.fireFwdPut. Phng thc Synapse.fireFwdPut

c ch ra trong danh sch 2.4.

Danh sch 2.4: Phng thc Synapse.fireFwdPut

public synchronized void fwdPut(Pattern pattern) {if ( isEnabled() ) {

count = pattern.getCount();

while ( items > 0 ) {

try {

wait();

} catch ( InterruptedException e ) {

//e.printStackTrace();

return;

}}

m_pattern = pattern;

inps = (double[])pattern.getArray();

forward(inps);

++items;

notifyAll();

} }

- 43 -


44/67

Khi vng lp ch hon thnh synapse, n s x l d liu u vo, v sau

truyn mu ti lp tip theo. Vic x l ch l synapse s thc hin trn mu p

lch. Theo cc quy trnh ca phng thc Synapse.fwdPut, n s sao chp mu

m n truyn vo bin lp m_pattern.

m_pattern = pattern;

Mng mu sau c sao chp ti mt mng hai gi tr x l.

inps = (double[])pattern.getArray();

Sau , mng mu hai gi tr c truyn vo phng thc forward. Trong

tt c cc lp, phng thc forward lun lun c s dng p lch.

forward(inps);

Khi lch c p, mu c sn sng c x l lp tip

theo.

C nhiu loi synapse c to c th x l lch theo cc cch khc

nhau. By gi, chng ta s xem xt phng thc FullSynapse.forwardp dng

lch c ch ra trong danh snh 2.5.

Danh sch 2.5: Phng thc FullSynapse.forward

public void forward(double[] pattern) {

int x;double in;

int n = getRows();

try {

for ( x = 0; x < n; ++x ) {

in = pattern[x] + bias.value[x][0];

outs[x] = 1 / (1 + Math.exp(-in));

}

} catch ( Exception aioobe ) {

aioobe.printStackTrace();

}

}

Ta c th thy lch c p cho mi phn t ca mu, v mu s c

thm vo mt lch. Gi tr nghch o ca tng ny s c tr v phng thc

calling c truyn ti lp tip theo. Cc qu trnh ny s c lp li cho mi

- 44 -


45/67

lp ca mng nron. Sau y ta xem chng s thc thi lan truyn ngc tr li nh

th no.

2.3.2 Thc thi lan truyn ngc

Khi mng nron hun luyn, n c a vo vi cc tp hun luyn. Sau

, kt qu thu c t mng nron s c so snh vi kt qu trc . Phn

c thm vo lm cho d liu u ra hin ti ph hp vi d liu u ra trc

c gi l sai s.

C vi cch gim hm sai s ny v mc ti thiu . Ph bin nht l s

dng phng thc gim theo gradient. Thut ton c lng o hm ca hm

sai s c bit n nh l thut ton lan truyn ngc, l n lan truyn cc sai

s ny ngc li trong mng.

Trong thc t, phng thc lan truyn ngc hot ng bng cch; u tin

n chy mt chng trnh nhn dng i vi d liu hun luyn thu c mt ma

trn trng s, v sau iu chnh cc trng s v lch ci thin sai s.

trn ta thy rng phng thc Layer.run c ch ra trong danh dch 5.1,

chu trch nhim cung cp d liu hun luyn cho mng nron. N s c chy ivi tng tp hun luyn, v d liu hun luyn s c chy lp i lp li cho n

khi sai s ca mng nron nm trong mc cho php.

Phng thc Synapse.revPut s gi phng thc Synapse.backward iu

chnh cc lch cn thit ca nron bt k. Hot ng ca phng thc

Synapse.backward c ch ra trong danh sch 2.6.

Danh sch 2.6: Phng thc Synapse.backward

protected void backward(double[] pattern) {int x;

int y;

double s, dw;

int m_rows = getInputDimension();

int m_cols = getOutputDimension();

// iu chnh cc trng s

for ( x=0; x < m_rows; ++x ) {

double absv;

- 45 -


46/67

s = 0;

for ( y=0; y < m_cols; ++y ) {

s += pattern[y] * array.value[x][y];

if ( getMomentum() < 0 ) {

if ( pattern[y] < 0 )

absv = -pattern[y];

else

absv = pattern[y];

dw = getLearningRate() * pattern[y] * inps[x] + absv *

array.delta[x][y];

} else

dw = getLearningRate() * pattern[y] * inps[x] + getMomentum() *

array.delta[x][y];array.value[x][y] += dw;

array.delta[x][y] = dw;

}

bouts[x] = s;

}

}

2.3.3 Cc tp hun luyn

hun luyn mng nron Kohonen th cc tp hun luyn phi c cung

cp. D liu hun luyn ny s c lu tr trong lp TrainingSet, lp ny c

thit k n l mt lp cha ng d liu. Lp TrainingSetqun l hai bin mng

di ca u vo v u ra.

Trong lp TrainingSet, n lu tr cc bin v cch s dng chng c tm

tt nh sau:

inputCount - S lng cc phn t u vo s c cho tng mu hun luyn. outputCount - S lng cc phn t u ra s c cho tng mu hun luyn.

input[][] - Cc mu u vo hun luyn.

output[][] - Cc mu u ra hun luyn.

trainingSetCount - S lng mu hun luyn.

Do qu trnh hot ng ca mng nron Kohonen l khng c gim st, ch

c cc phn t d liu u vo c cung cp, nn i tng TrainingSetc xy

- 46 -


47/67

dng hun luyn cho ra d liu u ra, n s c chuyn ti i tng

KohonenNetwork hun luyn trong ln tip theo.

2.3.4 Bo co tin trnh

chun b cc tp hun luyn v thu nhn tnh trng thng tin t qu trnh

hun luyn, ta phi hiu r cc lp mng Kohonen hot ng nh th no. Chng ta

s bt u bng vic xem xt lp mng c s.

2.3.4.1 Lp mng c s

By gi chng ta s xem xt lp Network. Lp ny l lp c s cho lp

KohonenNetwork, n l lp cui cng cung cp cho mng nron Kohonen.

Vic tnh ton chiu di mt vector l mt phn quan trng ca mng nron

Kohonen. Lp Network cha mt phng thc tnh ton di vector ca vector

cho, v n c biu din di dng mng. Phng thc ny c ch ra trong

danh sch 2.7.

Danh sch 2.7: Tnh di ca mt vector (Network.java)

/**

* @Tham s v vector* @Kt qu tr v di vector.

*/

static double vectorLength( double v[] )

{

double rtn = 0.0 ;

for ( int i=0;i


48/67

* @Tham s vec1 l vector th nht

* @Tham s vec2 l vector cn li

* @Kt qu tr v l tch v hng.

*/

double dotProduct(double vec1[] , double vec2[] )

{

int k,v;

double rtn;

rtn = 0.0;

k = vec1.length;

v = 0;

while ( (k--)>0 ) {

rtn += vec1[v] * vec2[v];v++;

}

return rtn;

}

u tin, cc trng s gia nron c khi to vi cc gi tr ngu nhin.

Sau cc gi tr ngu nhin ny c hun luyn cho ra cc kt qu tt hn.

Ti thi im bt u ca mi chu k hun luyn, cc trng s cng c khi to

vi cc gi tr ngu nhin. Lp Network cung cp mt phng thc thc thi vn

ny. Danh sch 2.9 ch ra phng thc sinh cc trng s ngu nhin.

Danh sch 2.9: Khi to cc trng s ngu nhin (Network.java)

/**

* @Tham s weight l mt ma trn trng s.

*/

void randomizeWeights( double weight[][] )

{

double r ;int temp = (int)(3.464101615 / (2. * Math.random() ));

for ( int y=0;y


49/67

}

}

}

}

2.3.4.2 Lp KohonenNetwork

Chng ta s xem xt lp KohonenNetwork. Lp ny l lp thc thi mng

nron Kohonen. Lp KohonenNetwork c mt s tnh cht c ch ra trong danh

sch 2.10.

Danh sch 2.10: Cc tnh cht ca lp KohonenNetwork

public class KohonenNetwork extends Network {

double outputWeights[][];protected int learnMethod = 1;

protected double learnRate = 0.5;

protected double quitError = 0.1;

protected int retries = 10000;

protected double reduction = .99;

protected NeuralReportable owner;

public boolean halt = false;

protected TrainingSet train;

Cc tnh cht c m t nh sau:

halt Thit lp ny l xc thc hy b qu trnh hun luyn.

learnMethod T l hc, t bng 1.

learnRate T l hc ban u.

outputWeights[][] Cc trng s ca cc nron u ra da trn u vo.

owner Lp owner, lp ny thc thi giao din NeuralReportable.

quitError Khi t l sai s t n mc nh hn 10% th dng hun luyn. reduction Lng gim t l hc ban u (learnRate) bi mi cng on.

retries - Tng s chu k cho php, n t mt mc trn (a ceiling) s lng

cc chu k hun luyn c th xy ra.

train Tp hun luyn.

cho mng nron Kohonen hot ng tt, ta khng ch chun ha vector

u vo, m ta cn phi chun ha c ma trn trng s. Danh sch 2.13 ch ra mt

- 49 -


50/67

phng thc chun ha cc d liu u vo a ti mng nron Kohonen, v

danh sch 2.14 ch ra s chun ha ma trn trng s.

Danh sch 2.13: Chun h d liu u vo (KohonenNetwork.java)

/**

* @Tham s input l mu d liu vo

* @Tham s normfac l nhn t chun ha

* @Tham s synth l gi tr u vo cui cng

*/

void normalizeInput(

final double input[] ,

double normfac[] ,

double synth[]

)

{

double length, d ;

length = vectorLength ( input ) ;

// iu chnh trong trng hp di qu nh

if ( length < 1.E-30 )

length = 1.E-30 ;

normfac[0] = 1.0 / Math.sqrt ( length ) ;

synth[0] = 0.0 ;}

Danh sch 2.14: Chun ha trng s (KohonenNetwork.java)

/**

* @Tham s w l cc trng s u vo

*/

void normalizeWeight( double w[] )

{int i ;

double len ;

len = vectorLength ( w ) ;

// iu chnh trong trng hp di qu nh

if ( len < 1.E-30 )

len = 1.E-30 ;

len = 1.0 / Math.sqrt ( len ) ;

for ( i=0 ; i


51/67

w[i] *= len ;

w[inputNeuronCount] = 0;

}

By gi ta kim tra phng thc th mu, c s dng a mu d liuu vo ti mng nron Kohonen. Phng php ny c gi l phng php th

trial, c ch ra trong danh sch 2.15.

Danh sch 2.15: Th mu vo (KohonenNetwork.java)

/**

* Phng thc ny c th c s dng khi a mt mu ti mng.

* Thng thng, n hay dng gi nron thng

* @Tham s input l mu vo.

*/void trial ( double input[] )

{

int i ;

double normfac[]=new double[1], synth[]=new double[1], optr[];

normalizeInput(input,normfac,synth) ;

for ( i=0 ; i 1.0 )

output[i] = 1.0 ;

if ( output[i] < 0.0 )

output[i] = 0.0 ;

}

}

Vy qu trnh tnh ton gi tr cho mi nron u ra c tnh ton bng

cch ly tch v hng c chun ha ca d liu u vo v cc trng s. Do

d liu u ra cui cng c th ln hn 1 hoc nh hn 0, ta phi a n v khong

[0,1]. a d liu u ra v khong [0,1] th cc kt qu nh hn 0 th ta a n

v 0, v cc kt qu ln hn 1 c a v 1. D liu u ra cui cng ca mi

nron c lu tr trong mng d liu u ra.

- 51 -


52/67

Chng ta ch quan tm n nron thng v chng c a li vo mu

hun luyn. Danh sch 2.16 ch ra phng thc a mt mu d liu u vo ti

mng Kohonen, v thu nhn nron thng. Phng thc ny chnh l phng thc

dng phn loi mu trong mng Kohonen.

Danh sch 2.16: a ra mt mu vo v thu nhn nron thng

/**

* @Tham s input l mu vo

* @Tham s normfac l nhn t chun ha

* @Tham s synth l gi u vo cui cng (synthetic last

input)

* @Kt qu tr v l s nron thng.

*/

public int winner(double input[] ,double normfac[] ,double

synth[])

{

int i, win=0;

double biggest, optr[];

normalizeInput( input , normfac , synth ) ; // Chun ha d

liu u vo

biggest = -1.E30;

for ( i=0 ; i biggest ) {

biggest = output[i] ;

win = i ;

}

// account for rounding

if ( output[i] > 1.0 )

output[i] = 1.0 ;

if ( output[i] < 0.0 )

output[i] = 0.0 ;

}

return win ; }

- 52 -


53/67

Phng thc ny s thng xuyn c s dng khi ta mun a mt mu

ti mt mng nron phn loi. Cn phng thc th m chng ta va xem xt

trn ch c s dng trong khi hun luyn mng. Khi hun luyn, chng ta quan

tm n d liu u ra hin ti ca mi nron. Tri li, khi phn loi mu th chng

ta ch quan tm n nron thng.

Phng thc winnerlp qua mi nron u ra v tnh ton d liu u ra cho

mi nron ring bit. Trong qu trnh lp ny th cc ch s ca nron c lu li.

Ch s ny l ca nron c gi tr u ra cao nht. Nron c ch s cao nht c

gi l nron thng. Nron thng ny c tr v tp mu tip tc tham gia vo

qu trnh hun luyn.By gi chng ta bt u xem xt qu trnh hun luyn. Phng thc

trainingc ch ra trong danh sch 2.17.

Danh sch 2.17: Hun luyn mng nron (KohonenNetwork.java)/*** @exception java.lang.RuntimeException*/public void learn ()throws RuntimeException{int i, key, tset,iter,n_retry,nwts;int won[],winners ;double work[],correc[][],rate,best_err,dptr[];double bigerr[] = new double[1] ;double bigcorr[] = new double[1];KohonenNetwork bestnet; // Preserve best heretotalError = 1.0 ;bestnet = new

KohonenNetwork(inputNeuronCount,outputNeuronCount,owner) ;won = new int[outputNeuronCount];correc = new double[outputNeuronCount][inputNeuronCount+1];if ( learnMethod==0 )work = new double[inputNeuronCount+1];

elsework = null ;

rate = learnRate;initialize () ;best_err = 1.e30 ;

// Vng lp chnh:n_retry = 0 ;for ( iter=0 ; ; iter++ ) {

evaluateErrors ( rate , learnMethod , won , bigerr , correc , work) ;

totalError = bigerr[0] ;if ( totalError < best_err ) {best_err = totalError ;copyWeights ( bestnet , this ) ;

- 53 -


54/67

}winners = 0 ;for ( i=0;i retries )break ;

initialize () ;iter = -1 ;rate = learnRate ;continue ;

}if ( rate > 0.01 )rate *= reduction ;

}// Hon thnh

copyWeights( this , bestnet ) ;for ( i=0 ; i


55/67

V chu k hun luyn cc nhanh nn chng ta ch theo di chu k c t l sai

s tt nht. Khi chng ta tm thy mt ma trn trng s c sai s nm di mc sai

s cho php, th s hun luyn c hon thnh. Ngc li, th chng ta s ly ma

trn tt nht c xc nh chu k trc.

By gi, chng ta s bt u xem xt xem cc sai s c c lng nh th

no. Danh sch 2.18 ch ra c lng cc sai s.

Danh sch 2.18: c lng cc sai s (KohonenNetwork.java)/*** Phng thc ny dng trong qu trnh hc. N dng c lng

cc trng s da vo tp hun luyn.

* @Tham s rate l t l hc* @Tham s learn_method l dng phng thc method(0=additive,1=subtractive)

* @Tham s won l qun l s ln nron thng* @Tham s bigerr l tr v sai s* @Tham s correc l tr v mng hiu chnh* @Tham s work l phm vi hot ng* @exception java.lang.RuntimeException*/*/void evaluateErrors (

double rate ,int learn_method ,int won[],double bigerr[] ,double correc[][] ,double work[])

throws RuntimeException{int best, size,tset ;double dptr[], normfac[] = new double[1];double synth[]=new double[1], cptr[], wptr[], length, diff ;

// Hiu chnh v t li s ln thngfor ( int y=0;y


56/67

diff = dptr[i] * normfac[0] - wptr[i] ;length += diff * diff ;if ( learn_method!=0 )cptr[i] += diff ;

else

work[i] = rate * dptr[i] * normfac[0] + wptr[i] ;}diff = synth[0] - wptr[inputNeuronCount] ;length += diff * diff ;if ( learn_method!=0 )cptr[inputNeuronCount] += diff ;

elsework[inputNeuronCount] = rate * synth[0] +

wptr[inputNeuronCount] ;if ( length > bigerr[0] )bigerr[0] = length ;

if ( learn_method==0 ) {normalizeWeight( work ) ;

for ( int i=0 ; i


57/67

cptr = correc[i];f = 1.0 / (double) won[i] ;if ( learn_method!=0 )f *= rate ;

length = 0.0 ;

for ( int j=0 ; j bigcorr[0] )bigcorr[0] = length ;

}// Tnh ton nhn t hiu chnhbigcorr[0] = Math.sqrt ( bigcorr[0] ) / rate ;

}

2.4 Kt lun

Trong chng ny, chng ta tm hiu v mng nron Kohonen. Mng

nron Kohonen khc vi mng lan truyn ngc vi im. Mng nron Kohonen

l luyn khng gim st. iu ny c ngha rng, mng nron Kohonen c cho

d liu u vo nhng khng bit trc c ci ra. Sau , trong khi hun luyn

th mng nron Kohonen bt u v ln bn ca mi nron trong mu hun

luyn, mu ny bao gm cc nron u ra.

Mt mng nron Kohonen ch bao gm hai lp. Mng c a vo mtmu d liu u vo v coi y l lp d liu vo. Mu d liu u vo ny phi

c chun ha thnh cc gi tr nm trong khong [-1,1]. D liu u ra t mng

nron ny l nron u ra thng ring l. Cc nron u ra c th c a li vo

trong cc nhm, tng ng vi cc nhm m mng nron Kohonen phn loi i

vi u vo.

S hun luyn mt mng nron Kohonen khc ng k so vi thut ton lan

truyn ngc c gii thiu. hun luyn mt mng nron Kohonen, chng

ta phi a vo cho n cc phn t hun luyn v xem nron u ra l nron thng.

cho cc nron thng th cc trng s ca nron thng ny c sa i sao cho

n s hot ng cao hn trn mu.

Cng c trng hp c th c mt hoc vi nron khng bao gi thng. Nh

vy, s c cc nron c trng s cht c a n mng nron. iu ny s gy

- 57 -


58/67

cho mng lm vic qu ti khi nhn dng, v thng tht bi khi gi tr trn cc

nron u ra ngang nhau.

Trong chng tip theo chng ta s p dng mng nron Kohonen vo ng

dng thc t, l ng dng nhn dng k t quang (c th l ch vit tay bng

chut v ch k a v di dng nh).

- 58 -


59/67

Chng 3. Nhn dng k t quang s dng mng nron

Kohonen

Nhn dng lun l mt phn quan trng v th v trong cc ng dng tin hc

hin nay. Nhn dng ch vit cng nh nhn dng ni chung l dng thut ton

tm ra cc c im ring ca tng c th trong qun th m chng ta phi nhn

dng. i vi nhn dng k t quang th vi mi d liu nhp vo (c th l k t

vit bng chut hay ch k c scan), ta s phi tm ra c u ra tng ng vi

n.

My Scanner l mt loi my scan phng (Flatbed Scanner) dng scan

nhng ti liu n l nh vn bn, hnh nh, th, CMND

3.1 Gii thiu chung

Trong chng trc, ta tm hiu v mng nron Kohonen. Ta bit rng,

mt mng nron Kohonen c th c s dng phn loi cc mu vo trong cc

nhm khc nhau. Trong chng ny, chng ta s xem xt k hn mt ng dng

ring bit ca mng nron Kohonen. l, mng nron Kohonen s c p dng

nhn dng k t quang, c th l ch vit tay bng chut hoc ch k c scan.

Cc chng trnh nhn dng quang cng c kh nng c vn bn in. Vn

bn ny c th l vn bn c qut t ti liu, hoc vn bn vit tay l v bng

thit b cm tay,

Trong chng ny, chng ta s pht trin mt v d, l n c th c

hun luyn nhn dng ch vit tay ca con ngi.

Trong :

Tin x l: l qu trnh chun ha d liu u vo.

Trch chn c trng: l qu trnh tm ra cc thng tin hu ch v c trng

nht cho mu u vo s dng trong qu trnh nhn dng.

- 59 -


60/67


61/67

Model: l m hnh to ra sau khi hun luyn mng vi tp d liu hun

luyn.

Evaluator: l phn nh gi cht lng m hnh, thng ly t l phn loi

ng trn tp kim tra lm tiu ch nh gi.

Hun luyn mng vi tp d liu hun luyn, kim tra m hnh thu c trn tp

d liu kim tra. Chng no sai s trn tp kim tra cn gim xung c th ta tip

tc hun luyn li trn tp d liu hun luyn. Khi no sai s trn tp kim tra

khng gim c na th dng.

3.3 Th nghim s dng mng nron Kohonen nhn dng k t quang

ng dng nhn dng quang s hin th mt giao din ngi-my bng ha

n gin, l s cho php ta hun luyn v s dng mng nron.

Nh ni, chng trnh ny khng trc tip c nhn dng cc k t m n

phi c hun luyn t cc k t c a vo bng vic v trc tip bng chut,

hoc a vo di dng nh c scan, trc khi n c th nhn dng d liu u

vo. Cc file hun luyn k t c lu tr trong quyn t in mu.

D liu ta a vo phi c ti mu trc khi chng c nhn dng. K t(nh) c ti mu bng cch; a k t (nh) vo trong mt li nh 5x7 pixel.

Khi ta ti mu k t (nh), mt khung hnh ch nht c v quanh k t (nh).

Khung ny gi l khung ct xn. Mc ch ca khung ct xn l ct ra khong trng

khng cn thit cho k t (nh). V vy, ta c th v k t (ly nh) v tr bt k

trong khu vc v, th chng trnh vn nhn dng c k t (nh).

Theo cch phn chia trn th mng c 5x7=35 nron u vo. S nron u ra

c chn cho mng nron Kohonen c s dng trong chng trnh ny tng

ng vi s k t (nh) c trong tp mu. Nu c 26 k t (nh) trong tp mu, th s

c 26 nron u ra. Th nghim chng trnh vi tp mu nh (khong 150 mu)

th chnh xc khong 80%.

*Hun luyn mng nron: Mi kt ni nron c gn mt trng s. xc nh

tng s cc kt ni, ta cn phi nhn s nron u vo v s cc nron u ra. Nu

- 61 -


62/67

mt mng nron vi 26 nron u ra v 35 nron u vo s c kt qu l 910 trng

s kt ni. Qu trnh hun luyn l qu trnh iu chnh cc trng s thu c ma

trn trng s m ta mong mun.

Do u vo ca mng l 35 nron nn cc nron ny l nron thng", n

c lu gi vi s nguyn "tt nht" chnh l s 1. Hnh minh ha pha di.

Hnh 3.4: Biu din k t e theo ma trn 5x7 pixcel.

Hnh 3.5: Biu din k t e theo ma trn 5x7 bi cc gi tr

Hnh3.6: Biu din k t e vector u vo

Ta bt u khi to ma trn trng s bng cch la chn ngu nhin. Sau khi

khi to ma trn th qu trnh luyn s bt u. Ban u, ma trn trong s c nh

gi xc nh xem nhng g hin ti ca n l sai s (li) cp. Sai s ny c xcnh bi vic hun luyn u vo nh th no (cc k t m ta to ra) a ra cc

nron u ra. Nu sai s tnh ton mc di 10% th qu trnh hon thnh.

Sau y l mt s kt qu thu c trong qu trnh chy th nghim chng

trnh.

- 62 -


63/67

a. Chn v trc tip, ti mu v gn k t cho mu (hnh 3.7)

Hnh 3.7 V v gn k t

Hnh 3.8 Kt qu mng nron Kohonen nhn dng k t e v k t c

f. Chn nh, ly ch k (hnh 3.9)

Hnh 3.9 a ch k vo mng v gn tn

- 63 -


64/67

Hnh 3.10 Kt qu mng nron Kohonen nhn dng ch k

3.4 Trng hp ngoi l

Chng trnh c trnh by y ch c kh nng trong nhn dng cc k t

ring bit.

Khi chy chng trnh, ta thy c vn xy ra. V d nh: chng trnh kh c

th xc nh phn bit gia ch o thng v ch O hoa, hay ch s 0 (s

khng). Vy ta thy, gia cc k t ny th khng th vn dng bng mng nron

Kohonen.

3.5 Kt lun

Trong chng ny, ch ra mt ng dng thc t ca mng nron Kohonen. Tiu

im ca chng ny l to ra mt ng dng nh l kh nng nhn dng ch vit

tay.

Cc k t (ch k c scan) c ngi s dng a vo vi phn gii cao.

biu din c ti mng nron, ta gim bt phn gii ny bng cch s dng

k thut ct xn v ti mu, k t (ch k) c chuyn sang mt nh th hai vi

phn gii thp hn nhiu.

Khi nh c a vo, n phi c ct xn loi b cc khong trng. V th

n khng ph thuc vo v tr v c vit ca k t (nh).

- 64 -


65/67

S ti mu l qu trnh a nh c phn gii cao c chuyn i v nh c

phn gii thp hn. Mi pixel trong k t (ch k) ti mu c n nh cho mt

mu trung bnh ca vng tng ng trong nh c phn gii cao.

Kt qu nh ti mu sau c a ti tp hun luyn hoc ti qu trnh nh

ca mng nron Kohonen. Mng nron Kohonen c s dng trong v d ny c

s nron u vo bng s pixel ca k t (hnh nh) c ti mu, v s nron u

ra bng s k t m ta to trong tp mu.

- 65 -


66/67

KT LUN Kin thc tm hiu c

- Nm c kin thc c bn ca mng nron nhn to v mng nron

Kohonen.

- Nm cquy trnh chung trong vic xy dng h thng nhn dng ch

vit.

- S dng phng php tnh tch v hng hai vector tm nron thng.

- a ra phng php trch chn c trng cho nhn dng k t quang (c

th l ch vit tay bng chut v ch k a v di dng nh).

- Pht trin thnh cng phn mm nhn dng k t quang (c th l chvit tay bng chut v ch k a vo di dng nh).

- Chng trnh ng dng mi ang bc u th nghim, v chnh xc

trn tp mu (150 mu) khong 80%.

Hng pht trin nghin cu

Lun vn c th pht trin cho nhn dng vn bn vit tay. Tuy nhin lm

c iu ny phi l c mt qu trnh tm hiu v nghin cu lu di. Do thi gian

c hn nn lun vn mi ch trnh by mc n gin l nhn dng ch k v

tng k t ring bit.

T nh gi

Mc d em c gng hon thin ti, nhng chc chn khng th trnh

c nhng thiu st, em rt mong nhn c s ch bo v gip ca cc thy c

gio, cng vi s gp kin ca nhng ai quan tm.

- 66 -


67/67

Ti tham kho1. Teuvo Kohonen,Self-Organizing Maps, Third Edition, Springer, Heidelberg,

2001

2. V. Rao and H. Rao, C++ Neural Networks and Fuzzy Logic, Second Edition,

MIS Press, NewYork, 1995

Luan Van Kohonen

Documents

Transcript of Luan Van Kohonen