Bai Giang Mang Noron Va Ung Dung p1

Phm c Long HCNTT&TT Thi Nguyn

Thi Nguyn 2-2014 1

MNG N RON V NG DNG

BI GING MNG NRON V NG DNG

Ti liu tham kho: [1]. L Minh Trung (bin dch), Gio trnh mng n ron nhn to, Nh XB

Thng k, 1999. [2]. Bi Cng Cng v Nguyn Don Phc, H m, mng nron v ng

dng , NXB Khoa hc v k thut, 2006. [3]. Nguyn nh Thc, Tr tu nhn to - Mng nron - Phng php v

ng dng, NXB GD, 2000. [4]. Lng Mnh B, Nguyn Thanh Thy, Nhp mn x l nh s, NXB

KHKT, 1999. [5]. Chin- Teng Lin and C.S.George lee, Neural Fuzzy Systems, Prentice - Hall

International Editions. 1996. [6]. Simon Haykin, Neural networks A Comprehensive Foundation, Prentice

Hall. 1999. [7]. Y. Takefuji, Neural Network Parallel Computing, Kluwer Acad. Publ.,

1992. [8]. Warren McCulloch and Walter Pitts, "A Logical Calculus of Ideas

Immanent in Nervous Activity", 1943, Bulletin of Mathematical Biophysics 5:115-133.

[9]. Walter Pitts, "Some observations on the simple neuron circuit", Bulletin of Mathematical Biology, Volume 4, Number 3, 121-129, 1942.

[10]. 1945, "A Heterarchy of Values Determined by the Topology of Nervous Nets". In: Bulletin of Mathematical Biophysics, 7, 1945, 8993.

[11]. T.M Mitchell, Machine learning, McGraw-Hill, 1997. [12]. William M.K. Trochim, Measurement Error,

http://www.socialresearchmethods.net/kb/measerr.php. [13]. Ripley B.D., Pattern Recognition and Neural Networks, Cambridge

university Press, 1996. [14]. http://home.agh.edu.pl/~vlsi/AI/hamming_en/ [15]. http://www.cs.ucla.edu/~rosen/161/notes/hopfield.html


Thi Nguyn 2-2014 2

MNG N RON V NG DNG

CHNG 1. GII THIU CHUNG V MNG NRON

1.1. N ron sinh hc v mng n ron sinh hc

Theo cc nh nghin cu sinh hc v b no, h thng thn kinh ca con ngi bao gm khong 100 t t bo thn kinh, thng gi l cc n-ron. Mi t bo nron gm ba phn:

- Thn n-ron vi nhn bn trong (gi l soma), l ni tip nhn hay pht ra cc xung ng thn kinh.

- Mt h thng dng cy cc dy thn kinh vo (gi l dendrite) a tn hiu ti nhn n-ron. Cc dy thn kinh vo to thnh mt li dy c xung quanh thn n-ron, chim din tch khong 0,25 mm2

- u dy thn kinh ra (gi l si trc axon) phn nhnh dng hnh cy, c th di t mt cm n hng mt. Chng ni vi cc dy thn kinh vo hoc trc tip vi nhn t bo ca cc n-ron khc thng qua cc khp ni (gi l synapse). Thng thng mi n-ron c th c t vi chc cho ti hng trm ngn khp ni ni vi cc n-ron khc. C hai loi khp ni, khp ni kch thch (excitatory) s cho tn hiu qua n ti n-ron cn khp ni c ch (inhibitory) c tc dng lm cn tn hiu ti n-ron. Ngi ta c tnh mi n-ron trong b no ca con ngi c khong 104 khp ni (hnh 1.1) Chc nng c bn ca cc t bo n-ron l lin kt vi nhau to nn h

thng thn kinh iu khin hot ng ca c th sng. Cc t bo n-ron truyn tn hiu cho nhau thng qua cc dy thn kinh vo v ra, cc tn hiu c dng xung in v c to ra t cc qu trnh phn ng ho hc phc tp. Ti nhn t bo, khi in th ca tn hiu vo t ti mt ngng no th n s to ra mt xung in dn ti trc dy thn kinh ra. Xung ny truyn theo trc ra ti cc nhnh r v tip tc truyn ti cc n-ron khc.


Thi Nguyn 2-2014 3

MNG N RON V NG DNG

HHnh 1.1 Cu to ca t bo n-ron sinh hc

1.2. M hnh n ron nhn to v mng n ron nhn to, hm tch hp

v cc hm kch hot Vi mc ch to ra mt m hnh tnh ton phng theo cch lm vic ca n-ron trong b no con ngi, vo nm 1943, cc tc gi McCulloch v Pitts (Warren Sturgis McCulloch (November 16, 1898 September 24, 1969) was an American neurophysiologist and cybernetician, known for his work on the foundation for certain brain theories and his contribution to the cybernetics movement. Walter Harry Pitts, Jr. (23 April 1923 14 May 1969) was a logician who worked in the field of cognitive psychology) [15] xut mt m hnh ton cho mt n-ron nh sau:


Thi Nguyn 2-2014 4

MNG N RON V NG DNG

Hnh N ron nhn to

Cc thnh phn ca mt n ron nhn to


Thi Nguyn 2-2014 5

MNG N RON V NG DNG

Mt rron nhn to l mt n v x l

Hnh 1.2 n v x l (Processing unit) trong : xi : cc u vo wji : cc trng s tng ng vi cc u vo


Thi Nguyn 2-2014 6

MNG N RON V NG DNG

j : ngng (bias) aj : u vo mng (net-input) zj : u ra ca nron g(x): hm chuyn (hm kch hot). Trong m hnh ny, mt n-ron th j s nhn cc tn hiu vo xj vi cc trng s

tng ng l wji , tng cc thng tin vo c trng s l =

n

iiji xw

1

Thng tin u ra thi im t+1 c tnh t cc thng tin u vo nh sau:

( )jiji txwgtout +=+ )()1( (1.1)

Trong g l hm kch hot (cn gi l hm chuyn) c dng l hm bc nhy, n ng vai tr bin i t thng tin u vo thnh tn hiu u ra

>=

0001

)(xifxif

xg (1.2)

Nh vy, out = 1 (ng vi vic n-ron to tn u ra) khi tng cc tn hiu vo ln hn ngng j , cn out = 0 (n-ron khng to tn hiu u ra) khi tng cc tn hiu vo nh hn ngng j. Trong m hnh n-ron ca McCulloch v Pitts, cc trng s wji th hin nh hng ca khp ni trong lin kt gia n-ron i (n-ron gi tn hiu) v n-ron j (n-ron nhn tn hiu). Trng s wji dng ng vi khp ni kch thch, trng s m ng vi khp ni c ch cn wji bng 0 khi khng c lin kt gia hai n-ron. Hm chuyn g ngoi dng hm bc nhy cn c th chn nhiu dng khc nhau v s c cp cc phn sau. Thng qua cch m hnh ho n gin mt n-ron sinh hc nh trn, McCulloch v Pitts a ra mt m hnh n-ron nhn to c tim nng tnh ton quan trng. N c th thc hin cc php ton l-gc c bn nh AND, OR v NOT khi cc trng s v ngng c chn ph hp. S lin kt gia cc n-ron nhn to vi cc cch thc khc nhau s to nn cc loi mng n-ron nhn to (Artificial Neural Network - ANN) vi nhng tnh cht v kh nng lm vic khc nhau. Nh c gii thiu, mng n-ron nhn to l mt h thng x l thng tin c xy dng trn c s tng qut ho m hnh ton hc ca n-ron sinh hc


Thi Nguyn 2-2014 7

MNG N RON V NG DNG

v phng theo c ch lm vic ca b no con ngi. Mng n-ron nhn to c th hin thng qua ba thnh phn c bn:

+ m hnh ca n-ron, + cu trc v s lin kt gia cc n-ron, + phng php hc c p dng cho mng n-ron.

1.2.1. Cc phn t x l Th no l x l?

Mt cng c x l c u vo v u ra. Khi a 1 (hoc 1 s) u vo sau bin i u vo theo 1 phng thc nht nh (chnh l chng trnh x l theo mt thut ton no ) s thu c chc chn 1 u ra.

Chng trnh khng c l cc dng lnh my tnh nh thi thng suy ngh ca chng ta m n c th l mt mch phn cng hot ng n nh. Xt cho cng v mt tn hiu th x l l bin i tn hiu vt l (in p hoc dng in) t mt gi tr ny sang gi tr khc. Trong mt mng nron c ba kiu n v: 1) Cc n v u vo (Input units), nhn tn hiu t bn ngoi; 2) Cc n v u ra (Output units), gi d liu ra bn ngoi; 3) Cc n v n (Hidden units), tn hiu vo (input) v ra (output) ca n nm trong mng. Mi n v j c th c mt hoc nhiu u vo: x0, x1, x2, xn, nhng ch c mt u ra zj. Mt u vo ti mt n v c th l d liu t bn ngoi mng, hoc u ra ca mt n v khc, hoc l u ra ca chnh n. Hm x l Hm kt hp Mi mt n v trong mt mng kt hp cc gi tr a vo n thng qua cc lin kt vi cc n v khc, sinh ra mt gi tr gi l net input. Hm thc hin nhim v ny gi l hm kt hp (combination function), c nh ngha bi mt lut lan truyn c th. Trong phn ln cc mng nron, chng ta gi s rng mi mt n v cung cp mt b cng nh l u vo cho n v m n c lin kt. Tng u vo n v j n gin ch l tng trng s ca cc u ra ring l t cc n v kt ni cng thm ngng hay lch (bias) j :


Thi Nguyn 2-2014 8

MNG N RON V NG DNG

Trng hp wji > 0, nron c coi l ang trong trng thi kch thch. Tng t, nu nh wji < 0, nron trng thi kim ch. Chng ta gi cc n v vi lut lan truyn nh trn l cc sigma units. Trong mt vi trng hp ngi ta cng c th s dng cc lut lan truyn phc tp hn. Mt trong s l lut sigma-pi, c dng nh sau:

Rt nhiu hm kt hp s dng mt " lch" hay "ngng" tnh net input ti n v. i vi mt n v u ra tuyn tnh, thng thng, j c chn l hng s v trong bi ton xp x a thc j = 1. Hm kch hot (hm chuyn) Phn ln cc n v trong mng nron chuyn net input bng cch s dng mt hm v hng (scalar-to-scalar function) gi l hm kch hot, kt qu ca hm ny l mt gi tr gi l mc kch hot ca n v (unit's activation). Loi tr kh nng n v thuc lp ra, gi tr kch hot c a vo mt hay nhiu n v khc. Cc hm kch hot thng b p vo mt khong gi tr xc nh, do thng c gi l cc hm bp (squashing). Cc hm kch hot hay c s dng l:

+ Hm ng nht (Linear function, Identity function ) g(x)=x Nu coi cc u vo l mt n v th chng s s dng hm ny. i khi mt hng s c nhn vi net-input to ra mt hm ng nht.

Hm ng nht (Identity function)

+ Hm bc nh phn (Binary step function, Hard limit function) Hm ny cng c bit n vi tn "Hm ngng" (Threshold function hay Heaviside function hoc Hm McCuloch-Pitts). u ra ca hm ny c gii hn vo mt trong hai gi tr:


Thi Nguyn 2-2014 9

MNG N RON V NG DNG

Dng hm ny c s dng trong cc mng ch c mt lp. Trong hnh v sau, c chn bng 1.

Hm bc nh phn (Binary step function)

+ Hm sigmoid (Sigmoid function (logsig))

Hm ny c bit thun li khi s dng cho cc mng c hun luyn (trained) bi thut ton Lan truyn ngc (back-propagation), bi v n d ly o hm, do c th gim ng k tnh ton trong qu trnh hun luyn. Hm ny c ng dng cho cc chng trnh ng dng m cc u ra mong mun ri vo khong [0,1].

Hm Sigmoid

+ Hm sigmoid lng cc (Bipolar sigmoid function (tansig))

Hm sigmoid lng cc

Cc hm chuyn ca cc n v n (hidden units) l cn thit biu din s phi tuyn vo trong mng. L do l hp thnh ca cc hm ng nht l mt hm ng nht. Mc d vy nhng n mang tnh cht phi tuyn (ngha l, kh nng biu din cc hm phi tuyn) lm cho cc mng nhiu tng c kh nng rt tt


Thi Nguyn 2-2014 10

MNG N RON V NG DNG

trong biu din cc nh x phi tuyn. Tuy nhin, i vi lut hc lan truyn ngc, hm phi kh vi (differentiable) v s c ch nu nh hm c gn trong mt khong no . Do vy, hm sigmoid l la chn thng dng nht. i vi cc n v u ra (output units), cc hm chuyn cn c chn sao cho ph hp vi s phn phi ca cc gi tr ch mong mun. Chng ta thy rng i vi cc gi tr ra trong khong [0,1], hm sigmoid l c ch; i vi cc gi tr ch mong mun l lin tc trong khong th hm ny cng vn c ch, n c th cho ta cc gi tr ra hay gi tr ch c cn trong mt khong ca hm kch hot u ra. Nhng nu cc gi tr ch khng c bit trc khong xc nh th hm hay c s dng nht l hm ng nht (identity function). Nu gi tr mong mun l dng nhng khng bit cn trn th nn s dng mt hm kch hot dng m (exponential output activation function). Mt s hm khc:

+ Saturating Linear (satlin)

+ Symmetrical Hard Limit (hardlims)

+ Log-Sigmoid (logsig)


Thi Nguyn 2-2014 11

MNG N RON V NG DNG

+ Hm tuyn tnh tng on

+

Cu hi: Ti sao li l cc hm kiu ny m khng phi l cc hm khc? Tr li: V phi c s chuyn trng thi mi kch hot c n ron tip theo. Kh nng biu din ca n ron B vi x l my tnh da trn tch hp cc mch logic c s. C th thy rng cc n ron hon ton m phng kh nng tnh ton ca cc mch c s AND, OR, NOT.

1.2.2 Lin kt trong mng n-ron nhn to

Mng n-ron nhn to gm cc n-ron v lin kt c trng s gia chng. ANN to nn mt h thng x l thng tin lm vic trn c s phng theo cch lm vic ca h thng cc n-ron trong b no con ngi. Tuy nhin, trong b

=0=1,5 w =1

w =1

w =1

w =1=-0,5 w=-1

Z = X and Y Z = X or Y Y = not X

X

Y

X

Y

ZZX Y


Thi Nguyn 2-2014 12

MNG N RON V NG DNG

no ca con ngi, cc t bo n-ron lin kt vi nhau chng cht v to nn mt mng li v cng phc tp.

Cc loi mng n-ron nhn to c xc nh bi cch lin kt gia cc n-ron, trng s ca cc lin kt v hm chuyn ti mi n-ron. Cc hnh v di y th hin cc cch kt ni khc nhau.

Hnh .... Mng n-ron nhn to ch c mt nt v c s phn hi

Hnh .... Mng n-ron truyn thng mt lp(Single-layer feedforward network)

Mng n-ron truyn thng mt lp l loi mng ch c lp n-ron u vo v mt lp n-ron u ra (thc cht lp n-ron u vo khng c vai tr x l, do ta ni mng ch c mt lp). Loi mng ny cn c gi l mng perceptron mt lp. Mi n-ron u ra c th nhn tn hiu t cc u vo x1, x2, , xm to ra tn hiu u ra tng ng.

Hnh Mng n-ron truyn thng nhiu lp (Multil-layer feedforward network).


Thi Nguyn 2-2014 13

MNG N RON V NG DNG

Trong mng n-ron truyn thng nhiu lp, lp nhn tn hiu vo ca mng gi l lp vo (input layer), n thng khng thc hin vic chuyn i thng tin m ch lm chc nng nhn tn hiu. Tn hiu ra ca mng c a ra t lp ra (output layer). Cc lp gia lp vo v lp ra gi l cc lp n. Trong mng truyn thng (feedforward network) khng c nt no m u ra ca n l u vo ca mt nt khc trn cng lp vi n hoc lp trc.

Mng c phn hi (feedback network) l mng m u ra ca mt n-ron c th tr thnh u vo ca n-ron trn cng mt lp hoc ca lp trc . Mng feedback c chu trnh khp kn gi l mng hi quy (recurrent network)

Hnh .... Mng n-ron hi quy mt lp

Theo quan im xem xt kt ni cc nron trong mng

Cch thc kt ni cc nron trong mng xc nh kin trc (topology) ca mng. Cc nron trong mng c th kt ni y (fully connected) tc l mi nron u c kt ni vi tt c cc nron khc, hoc kt ni cc b (partially connected) chng hn ch kt ni gia cc nron trong cc tng khc nhau. Ngi ta chia ra hai loi kin trc mng chnh:

T kt hp (autoassociative): l mng c cc nron u vo cng l cc nron u ra. Mng Hopfield l mt kiu mng t kt hp.

Mng t kt hp


Thi Nguyn 2-2014 14

MNG N RON V NG DNG

Kt hp khc kiu (heteroassociative): l mng c tp nron u vo v

u ra ring bit. Perceptron, cc mng Perceptron nhiu tng (MLP: MultiLayer Perceptron), mng Kohonen, thuc loi ny.

Mng kt hp khc kiu

Theo quan im xem xt c kt ni lin h ngc hay khng Ty thuc vo mng c cc kt ni ngc (feedback connections) t cc nron u ra ti cc nron u vo hay khng, ngi ta chia ra lm 2 loi kin trc mng.

Kin trc truyn thng (feedforward architechture): l kiu kin trc mng khng c cc kt ni ngc tr li t cc nron u ra v cc nron u vo; mng khng lu li cc gi tr output trc v cc trng thi kch hot ca nron. Cc mng nron truyn thng cho php tn hiu di chuyn theo mt ng duy nht; t u vo ti u ra, u ra ca mt tng bt k s khng nh hng ti tng . Cc mng kiu Perceptron l mng truyn thng.

Mng truyn thng

Kin trc phn hi (Feedback architecture): l kiu kin trc mng c cc kt ni t nron u ra ti nron u vo. Mng lu li cc trng thi trc , v trng thi tip theo khng ch ph thuc vo cc tn hiu u vo m cn ph thuc vo cc trng thi trc ca mng. Mng Hopfield thuc loi ny.


Thi Nguyn 2-2014 15

MNG N RON V NG DNG

Mng phn hi

d h thng ta c th phn loi theo cc tiu ch sau: Theo kiu lin kt n ron: Ta c mng n ron truyn thng (feel-forward

Neural Network) v mng n ron hi qui (recurrent NN).

+ Trong mng n ron truyn thng, cc lin kt n ron i theo mt hng nht nh, khng to thnh th khng c chu trnh (Directed Acyclic Graph) vi cc nh l cc n ron, cc cung l cc lin kt gia chng.

+ Ngc li, cc mng hi qui cho php cc lin kt n ron to thnh chu trnh. V cc thng tin ra ca cc n ron c truyn li cho cc n ron gp phn kch hot chng, nn mng hi qui cn c kh nng lu gi trng thi trong ca n di dng cc ngng kch hot ngoi cc trng s lin kt n ron.

Theo s lp: Cc n ron c th t chc li thnh cc lp sao cho mi n ron ca lp ny ch c ni vi cc n ron lp tip theo, khng cho php cc lin kt gia cc n ron trong cng mt lp, hoc t n ron lp di ln n ron lp trn. y cng khng cho php cc lin kt n ron nhy qua mt lp.

D dng nhn thy rng cc n ron trong cng mt lp nhn c tn hiu t lp trn cng mt lc, do vy v nguyn tc chng c th x l song song. Thng thng, lp n ron vo ch chu trch nhim truyn a tn hiu vo, khng thc hin mt tnh ton no nn khi tnh s lp ca mng, ngi ta khng tnh lp ny. 1.2.3 Kh nng ng dng ca mng n-ron nhn to c trng ca mng n-ron nhn to l kh nng hc v x l song song. N c th gn ng mi quan h tng quan phc tp gia cc yu t u vo v u ra ca cc qu trnh cn nghin cu v khi hc c th vic kim tra c lp thng cho kt qu tt. Sau khi hc xong, mng n-ron nhn to c th tnh ton kt qu u ra tng ng vi b s liu u vo mi. V mt cu trc, mng n-ron nhn to l mt h thng gm nhiu phn t x l


Thi Nguyn 2-2014 16

MNG N RON V NG DNG

n gin cng hot ng song song. Tnh nng ny ca ANN cho php n c th c p dng gii cc bi ton ln. V kha cnh ton hc, theo nh l Kolmogorov, mt hm lin tc bt k f( x1, x2,.. , xn ) xc nh trn khong In ( vi I =[0,1]) c th c biu din di dng:

trong : j , ij l cc hm lin tc mt bin. ij l hm n iu, khng ph thuc vo hm f. Mt khc, m hnh mng n-ron nhn to cho php lin kt c trng s cc phn t phi tuyn (cc n-ron n l) to nn dng hm tng hp t cc hm thnh phn. Do vy, sau mt qu trnh iu chnh s lin kt cho ph hp (qu trnh hc), cc phn t phi tuyn s to nn mt hm phi tuyn phc tp c kh nng xp x hm biu din qu trnh cn nghin cu. Kt qu l u ra ca n s tng t vi kt qu u ra ca tp d liu dng luyn mng. Khi ta ni mng n-ron nhn to hc c mi quan h tng quan u vo - u ra ca qu trnh v lu li mi quan h tng quan ny thng qua b trng s lin kt gia cc nron. Do , mng n-ron nhn to c th tnh ton trn b s liu u vo mi a ra kt qu u ra tng ng.

Hi qui tuyn tnh Xp x bng mng nron

Y = a1X + a2 Y=f(X, a1, . , an) Hnh S khc nhau gia hi quy tuyn tnh v mng n-ron

Vi nhng c im , mng nron nhn to c s dng gii quyt nhiu bi ton thuc nhiu lnh vc ca cc ngnh khc nhau. Cc nhm ng dng m mng n-ron nhn to c p dng rt c hiu qu l: Bi ton phn lp: Loi bi ton ny i hi gii quyt vn phn loi cc i tng quan st c thnh cc nhm da trn cc c im ca cc nhm i tng . y l dng bi ton c s ca rt nhiu bi ton trong thc t: nhn dng ch vit, ting ni, phn loi gen, phn loi cht lng sn phm,


Thi Nguyn 2-2014 17

MNG N RON V NG DNG

Bi ton d bo: Mng n-ron nhn to c ng dng thnh cng trong vic xy dng cc m hnh d bo s dng tp d liu trong qu kh d on s liu trong tng lai. y l nhm bi ton kh v rt quan trng trong nhiu ngnh khoa hc. Bi ton iu khin v ti u ho: Nh kh nng hc v xp x hm m mng n-ron nhn to c s dng trong nhiu h thng iu khin t ng cng nh gp phn gii quyt nhng bi ton ti u trong thc t.

Tm li, mng n-ron nhn to c xem nh l mt cch tip cn y tim nng gii quyt cc bi ton c tnh phi tuyn, phc tp v c bit l trong tnh hung mi quan h bn cht vt l ca qu trnh cn nghin cu khng d thit lp tng minh.

1.3. Cc quy tc hc Chc nng ca mt mng nron c quyt nh bi cc nhn t nh: hnh trng mng (s lp, s n v trn mi tng, v cch m cc lp c lin kt vi nhau) v cc trng s ca cc lin kt bn trong mng. Hnh trng ca mng thng l c nh, v cc trng s c quyt nh bi mt thut ton hun luyn (training algorithm). Tin trnh iu chnh cc trng s mng nhn bit c quan h gia u vo v ch mong mun c gi l hc (learning) hay hun luyn (training). Rt nhiu thut ton hc c pht minh tm ra tp trng s ti u lm gii php cho cc bi ton. Cc lut hc ca mng n-ron nhn to Nh c cp phn u lut hc l mt trong ba yu t quan trng to nn mt mng n-ron nhn to. C hai vn cn hc i vi mi mng n-ron nhn to l hc tham s (parameter learning) v hc cu trc (structure learning). Hc tham s l vic thay i trng s ca cc lin kt gia cc n-ron trong mt mng, cn hc cu trc l vic iu chnh cu trc ca mng bao gm thay i s lp n-ron, s n-ron ca mi lp v cch lin kt gia chng. Hai vn ny c th c thc hin ng thi hoc tch bit. V mt phng php hc, c th chia ra lm ba loi:

- hc c gim st hay cn gi l hc c thy (supervised learning), - hc tng cng (reinforcement learning) th no l hc tng cng? - hc khng c gim st hay cn gi l hc khng c thy

(unsupperviced learning).


Thi Nguyn 2-2014 18

MNG N RON V NG DNG

1.3.1. Hc c thy Mng c hun luyn bng cch cung cp cho n cc cp mu u vo v cc u ra mong mun (target values). Cc cp c cung cp bi "thy gio", hay bi h thng trn mng hot ng. S khc bit gia cc u ra thc t so vi cc u ra mong mun c thut ton s dng thch ng cc trng s trong mng. iu ny thng c a ra nh mt bi ton xp x hm s - cho d liu hun luyn bao gm cc cp mu u vo x, v mt ch tng ng t, mc ch l tm ra hm f(x) tho mn tt c cc mu hc u vo.

M hnh Hc c thy (Supervised learning model)

Trong hc c gim st, ta c cho trc mt tp v d gm cc cp

v mc tiu l tm mt hm f (trong lp cc hm c php) khp vi cc v d. Ni cch khc, ta mun tm nh x m d liu u vo hm , vi hm chi ph o khng khp gia nh x ca ta v d liu.


Thi Nguyn 2-2014 19

MNG N RON V NG DNG

1.3.2. Hc khng c thy Vi cch hc khng c thy, khng c phn hi t mi trng ch ra rng u ra ca mng l ng. Mng s phi khm ph cc c trng, cc iu chnh, cc mi tng quan, hay cc lp trong d liu vo mt cch t ng. Trong thc t, i vi phn ln cc bin th ca hc khng c thy, cc ch trng vi u vo. Ni mt cch khc, hc khng c thy lun thc hin mt cng vic tng t nh mt mng t lin hp, c ng thng tin t d liu vo. Trong hc khng c gim st, ta c cho trc mt s d liu , v hm chi ph cn c cc tiu ha c th l mt hm bt k ca d liu v u ra ca mng, . Hm chi ph c quyt nh bi pht biu ca bi ton. Phn ln ng dng nm trong vng cc bi ton c lng nh m hnh ha thng k, nn, lc (filtering), blind source seperation v phn mnh (clustering). 1.3.3 Hc tng cng Trong hc tng cng, d liu thng khng c cho trc m c to ra trong qu trnh mt agent tng tc vi mi trng. Ti mi thi im , agent thc hin hnh ng v mi trng to mt quan st v mt chi ph tc thi

, theo mt quy trnh ng no (thng l khng c bit). Mc tiu l tm mt sch lc la chn hnh ng cc tiu ha mt chi ph di hn no , ngha l chi ph tch ly mong i. Quy trnh ng ca mi trng v chi ph di


Thi Nguyn 2-2014 20

MNG N RON V NG DNG

hn cho mi sch lc thng khng c bit, nhng c th c lng c. Mng n-ron nhn to thng c dng trong hc tng cng nh l mt phn ca thut ton ton cc. Cc bi ton thng c gii quyt bng hc tng cng l cc bi ton iu khin, tr chi, v cc nhim v quyt nh tun t (sequential decision making) khc. 1.3.4 Hm mc tiu hun luyn mt mng v xt xem n thc hin tt n u, ta cn xy dng mt hm mc tiu (hay hm gi) cung cp cch thc nh gi kh nng h thng mt cch khng nhp nhng. Vic chn hm mc tiu l rt quan trng bi v hm ny th hin cc mc tiu thit k v quyt nh thut ton hun luyn no c th c p dng. pht trin mt hm mc tiu o c chnh xc ci chng ta mun khng phi l vic d dng. Mt vi hm c bn c s dng rt rng ri. Mt trong s chng l hm tng bnh phng li (sum of squares error function),

trong : p: s th t mu trong tp hun luyn i : s th t ca n v u ra tpi v ypi : tng ng l u ra mong mun v u ra thc t ca mng cho n v u ra th i trn mu th p. Trong cc ng dng thc t, nu cn thit c th lm phc tp hm s vi mt vi yu t khc c th kim sot c s phc tp ca m hnh. CHNG 2. MNG NRON HOPFIELD V NG DNG

2.1. Mng Hopfield ri rc v lin tc: hm nng lng v quy tc cp nht trng thi.


Thi Nguyn 2-2014 21

MNG N RON V NG DNG

2.1.1 Cu trc mng Hopfield

mng Hopfield ch nhn cc inputs l -1 hay 1. Hm truyn vo ca mng l hm satlins:

th hm satlins


Thi Nguyn 2-2014 22

MNG N RON V NG DNG

Wii = 0, i (khng co nu t nao lin k t vi chinh no ) Wij = Wji i, j (cac lin kt la i xng)

Rang bu c rng cac tro ng s phai i xng thng c s du ng, vi no am bao rng ham nng lng se giam m t cach n iu trong khi lam theo ca c lut kich hoat, va mang co th xut hin hanh vi tun hoan hoc h n loan nu du ng cac tro ng s khng i xng. Tuy nhin, Hopfield nhn thy rng hanh vi h n loan nay chi han ch nhng phn tng i nho cu a khng gian pha, va khng lam giam i kha nng thc hin vai tro lam h th ng b nh co th anh ia chi n i dung cu a mang. Ta nh ngha hm nng lng ca mng l:

( )1 1 1 1

1,..., y = - - + 2

n n n n

i n ij i j i i i ii j i i

E E y w y y x y y= = = =

= (2.2) Ty theo phng thc hot ng ca mng m ngi ta phn mng Hopfield ra thnh mng Hopfield ri rc v mng Hopfield lin tc. 2.1.2 Mng Hopfield ri rc Mng Hopfield ri rc l mng c tnh ri rc (u ra ri rc) lm vic ch khng ng b. Trng hp mng nhn cc gi tr nh phn {0,1}: Hm kch hot c xc nh nh sau:

( ) 1 net > 0 = 0 net 0

f net

(2.3)

Vic cho hm kch hot (2.3) tng ng vi qui tc chuyn trng thi ca mng: yi (t + 1)= yi (t) +yi (2.4) Trong : yi = yik+1 yik (2.5) nh l: Gi s vi wii = 0. Khi vi qui tc chuyn trng thi nh trn v cp nht khng ng b th nng lng ca mng khng tng (tc l gim hoc gi nguyn). Chng minh: Gi s n-ron k thay i trng thi t thi im t n t+1. Khi mng s thay i nng lng v: E = E(t+1) E(t)


Thi Nguyn 2-2014 23

MNG N RON V NG DNG

1 + - - =- + - 1 1

n nk kk kij iji i i ii i kj jj j

w y w yy y yx x + = = = Vi:

yi = 1 nu ( ) + - 1

0n

tij i ijjw y x

= v yi(t) = 0

yi = -1 nu ( ) + - 1

0n

tij i ijjw y x

= v yi(t) = 1

yi = -1 trong cc trng hp khc. V th ta lun c E 0, tc l nng lng ca mng khng tng, V th hm nng lng s t ti gi tr cc tiu do hm gii ni. 2.1.3 Mng Hopfield lin tc Mt mng Hopfield c th no c th bin i vi mt m hnh lin tc theo thi gian gi thit l mt bin lin tc v nhng nt c u ra lin tc, b chia (phn bc) thay cho u ra nh phn hai trng thi. T y, nng lng ca mng b gim bt lin tc. Nu dng mt mch in t thc thi mng Hopfield lin tc s s dng nhng b khuych i v nhng in tr phi tuyn( hnh v 2.0). iu ny gi kh nng ca vic xy dng mt mng bc nhy ngn s dng cng ngh tng t VLSI.

Mng Hopfield lin tc l mng m trng thi ca n c m t theo

phng trnh.

( )1 1

n ni

i ij i i i i i ij i i i ij j

duC w y u g u x w y G u xdt = =

= + = + (2.5)


Thi Nguyn 2-2014 24

MNG N RON V NG DNG

Vi: 1

n

i ij ij

G w g=

= + (2.6) Hun luyn mng

Mng Hopfield HF cho php to nh x d liu t tn hiu vo sang tn hiu ra theo kiu t kt hp (auto - association) tc l nu tn hiu vo l X thuc min gi tr D no th kt qu ra Y:

Y = Tinh(X,NN) cng thuc vo min D . Nh vy, mt vect tn hiu vo X b thiu thng tin hoc bin dng c th

c phc hi dng nguyn bn ca mnh. Trong ng dng, mng Hopfield m phng c kh nng t kt hp

(hi tng) ca b no ngi, nhn ra ngi quen sau khi nhn thy nhng nt quen thuc trn khun mt. Ngoi ra, vi mt s ci bin mng Hopfield cn c dng gii quyt cc bi ton ti u, bi ton x l d liu trong iu khin t ng.

Qu trnh hun luyn mng Hopfield thc cht l qu trnh xc nh b trng s lin kt ca mng sau ny khi s dng vi b trng s mng nhn ra i tng cn nhn dng. Vic ny ging nh vic ghi nhn cc c im ca i tng cn nh ca b no ngi.

Mng Hopfield HF hc da trn nguyn tc c gim st. Gi s c p mu hc tng ng vi cc vect tn hiu vo Xs , s=1,p. Mng s xc nh b trng s W sao cho

Xs = Tinh ( Xs , W) vi mi s=1,p (4) Ta xy dng ma trn trng s W nh sau : W = (w ji) vi

y Xs = (xs1,...,xsm). V d cho mng hc (nhn mt) 24 ch ci th ta s c 24 mu tn hiu vo

mi tn hiu c ln ca n ty thuc font ch ta s dng. Mt cch trc quan, trng s lin kt ji s tng thm mt lng l 1 (tng ng vi s hng xsj.xsi ) nu c hai thnh phn th i v th j ca mu hc Xs bng nhau. Khi c mu hc mi Xp+1 ta ch cn xt cc thnh phn th i v th j ca n cp nht gi tr cho wji (6). C th chng minh c vi ma trn W c xc nh nh trong (5), ta s c c (4). Hay l vi cc c im ghi nh c (qua ma trn trng s) mng c th s thc hin nhn ra c i

=

p

ssisj xx

1 Nu i jwji =

0 Nu i=j(5)


Thi Nguyn 2-2014 25

MNG N RON V NG DNG

tng quen thuc m n ghi nh. Ni cch khc, mng "hc thuc" cc v d mu {Xs}.

S dng mng. C th so snh y l qu trnh nhn ra i tng quen. Gi s a vo

mng vect tn hiu ____X . S dng mng tnh u ra tng ng vi tn hiu vo

____X l mt qu trnh lp bao gm cc bc:

1. Ban u , t X(0) = ____X . Gi Y(t) l vect tn hiu ra tng ng vi mt

ln cho X(t) lan truyn trong mng. Y(t) = out(t) = Tinh ( HF, X(t)).

2. Nu Y(t) X(t) th tip tc bc lp vi t=t+1 v X(t+1) = Y(t) = out(t) Nu Y(t) = X(t) th dng v khi X

(t) c coi l kt qu x l ca mng

khi c tn hiu vo ____X .

im ch quan trng l ma trn W khng thay i trong qu trnh s dng mng.

Mt vi tnh hung ny sinh 1) Mng khng hi t.

2) Mng hi t v X(t) = ____X

3) Mng hi t v X(t) = Xs vi Xs l mu no hc.

4) Mng hi t vi X(t) Xs vi mi mu hc Xs 5) Mng hi t vi X(t) no nh trong 2) 3) 4) nhng l nh

ngc ( 1 thnh -1, -1 thnh 1). 6) Mng c th a ra lun phin mt vi mu hc (hoc nh ngc ca chng).

Trng hp 2) c ngha rng vect ____X c on nhn ng da trn

mu hc {Xs} hay ni cch khc, ____X c th suy ra t mu hc.

Trng hp 3) chng t rng mng phc hi dng nguyn bn Xs ca ____X .

Trng hp 4) ch ra mt vect mi, c th xem l mu hc v s c dng cp nht ma trn trng s (xem (6)).

Th nghim mng trong phc hi nh Xt bi ton phc hi nh en trng kch c MxN. Nh vy mi nh c

MxN im nh. V d mun phc hi cc ch t tp 24 ch ci. Font ch kiu 8x7


Thi Nguyn 2-2014 26

MNG N RON V NG DNG

nh ch T vi font 8x7

Trong khi phn tch di y n gin chng ta s s dng trng hp nh en trng kch c 4 x 4. Nh vy mi nh c 16 im nh. Ta thit k mt mng HF vi 16 u vo v 16 n ron ra. Vect u vo ca mng nhn c t ma trn nh, ly tng dng mt, sau khi bin i nh s dng hm x'=2x-1.

Ban u ta c 4 mu

X1=(0,0,1,1,0,0,1,1,1,1,0,0,1,1,0,0) X2=(0,0,0,0,0,0,0,0,1,1,1,1,1,0,0,0) X3=(1,1,1,1,0,0,0,1,0,0,0,1,1,1,1,1) X4=(1,0,0,0,1,0,0,0,1,0,0,0,1,1,1,1)

Hnh . Mu hc

X1 0 0 1 1 0 0 1 1 1 1 0 0 1 1 0 0 X1'=2X-1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1

... ...

O O O O O O O O O O O O O O O O

Y1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1 Hnh Mng Hopfield khi phc nh.


Thi Nguyn 2-2014 27

MNG N RON V NG DNG

Ma trn W c tnh theo cng thc (5) cho kt qu sau: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0 2 0 0 2 0 -2 0 -2 -4 -2 0 0 2 4 4 1 2 0 2 2 0 2 0 2 -4 -2 0 2 -2 0 2 2 2 0 2 0 4 -2 0 2 4 -2 0 -2 0 0 2 0 0 3 0 2 4 0 -2 0 2 4 -2 0 -2 0 0 2 0 0 4 2 0 -2 -2 0 2 0 -2 0 -2 0 -2 -2 0 2 2 5 0 2 0 0 2 0 2 0 -2 0 2 0 -4 -2 0 0 6

-2 0 2 2 0 2 0 2 0 2 0 -2 -2 0 -2 -2 7 0 2 4 4 -2 0 2 0 -2 0 -2 0 0 2 0 0 8

-2 -4 -2 -2 0 -2 0 -2 0 2 0 -2 2 0 -2 -2 9 -4 -2 0 0 -2 0 2 0 2 0 2 0 0 -2 -4 -4 10 -2 0 -2 -2 0 2 0 -2 0 2 0 2 -2 -4 -2 -2 11 0 2 0 0 -2 0 -2 0 -2 0 2 0 0 -2 0 0 12 0 -2 0 0 -2 -4 -2 0 2 0 -2 0 0 2 0 0 13 2 0 2 2 0 -2 0 2 0 -2 -4 -2 2 0 2 2 14 4 2 0 0 2 0 -2 0 -2 -4 -2 0 0 2 0 4 15 4 2 0 0 2 0 -2 0 -2 -4 -2 0 0 2 4 0 16

Kt qu th nghim vi cc nh c nhiu ti 2,5,13 im nh (tng ng

vi 13, 31 v 81%) c cho trn hnh 7.11. Hn na, vi nh u vo c cng s im nh bin dng c th dn ti nhng hnh vi khc nhau (khng hi t ging nhau, s vng lp khc nhau ...). Nu c hn 50% im nh bin dng th nh c ti to u ra l m bn ca nh gc. E . Kh nng nh mu ca mng Hopfield

Kt qu thc nghim ch ra rng s n ron Nn ron ni chunggp 7 ln s nh mu N anh cn phi nh ( khi phc) trong mng:

Nn ron = 7. N anh (7).

T cng thc ny rt ra hai iu: Th nht, phn gii r x r ca nh ph thuc vo cn phi nh bao nhiu

nh mu. Chng hn, nu cn nh 100 nh mu th cn phi c 700 n ron, mi n ron tng ng vi mt im nh. Do vy,

r 2 = Nn ron = 7. N anh = 700, do nh phi c phn gii 27 x 27.


Thi Nguyn 2-2014 28

MNG N RON V NG DNG

Mu X1 Mu X2 Mu X3 Mu X4

Hnh Th nghim mng vi nh nhiu. Th hai, kch thc ma trn trong s s l m2 = (Nn ron)2 = 49 (N anh)2 Nu cn nh 100 nh mu, cn phi lu gi 490.000 trng s, mi trng s

cn 2 hoc 4 byte; Do vy, lu giu thng tin v mng cn phi mt c 1Mbyte hoc 2Mbyte. y chnh l phc tp ca mng Hopfield.

c lng chi ph thi gian, ta lm nh sau: Mi ln lp tnh out(t) t X(t) ta cn chi ph c 2.106 php nhn v 2.106

php cng. Mt c lng th l phc tp thi gian ca mng Hopfield tng theo lu tha bc 2 ca kch c bi ton.

H thc (7) ch ng khi cc mu hc phn b ngu nhin trong khng gian mu. Nu phn b hoc la chn mu hc tt, c th tng kh nng nh mu ca mng t 0,14 mu/1 nron ln ti 0,25 mu / 1 n ron. Trong v d xt, ta ch c 4 mu (N anh=4) dng cho mng vi Nn ron = 4x4 = 16 n ron. Kh nng nh mu ca n l 4/16 = 0,25.

Mt s im lu v mng Hopfield + Mng Hopfield cho php to nh x t kt hp trn cc tp d liu + D liu vo, ra c gi tr lng cc + Hc c gim st + Mng cho php phc hi d liu

Kh nng nh mu ph thuc vo s n ron ca mng 2.2. B nh kt hp

Application of Hopfield Network Mng Hopfield c nhiu ng dng khc nhau:

1 2 1 2

2 2 2 3

3 544

nh gc

nh nhiu ti 2 im nh kt qu s ln lp

nh nhiu ti 5 im

nh nhiu ti 13 im


Thi Nguyn 2-2014 29

MNG N RON V NG DNG

+ B nh lin hp ( Associative memories): Mng c th nh mt s trng thi, cc mu (patterns).

+ Ti u ha t hp: nu vn c m hnh ha trc tip mng c th cho mt s cc tiu v mt s li gii nhng t khi tm c li gii ti u.

Mng Hopfield c th c s dng nh hp en tnh ton kt qu u ra. Mt trong nhng ng dng ph bin nht l Hebbian learning. Associative Memory Hebbian Learning Trong Hebbian Learning, trng s ca cc kt qu ma trn t mt hc khng gim st, loi hc tp ny l mt hc tng cng. Chng ti cho rng chng ta c th "khc su" mu vo mng nhn ra mt trong s chng ngay c khi chng c cho trong u vo vi rt nhiu nhiu. Giai on hc bao gm hc cc mu khc nhau thit lp trng s ca ng bin. i vi mi muchng ta c th tip tc giai on hc nh: + nu hai khi (unit) c trng thi kch hot ging nhau ({1,1} {-1,-1}) th

cng ca mi quan h (connexion) s tng (wij tng). + nu hai khi (unit) c trng thi kch hot ngc nhau {-1,1} th cng

ca mi quan h s suy gim (decline). those Vi nhng loi quy tc , W s lu tr cc mu v sau cho php mng hi t. Tuy nhin trong trng hp chng ta khng th d on s hi t cc mu ng (mng vn s hi t nhng chng ta khng th d on trng thi n nh). Learning Ma trn trng s c iu chnh theo m mu chng ta mun lu tr. nhng mu i din cho trng thi n nh chng ta hy vng s c c. Cc trng s c sa i, v d:

(6) c lp thi gian khc nhau cho mi mu. xi v xj biu din trng thi ca cc n v i v j ca mu c hc. l mt hng s ngn nga cc kt ni tr thnh ln hay nh (v d: =1/m). Khi m=1 chng ta lun hi t v mu gy n tng. Khi m> 1, chng ta ch c th hy vng s hi t v mt mu c bit n nu cc mu khc nhau nh trc giao cng tt (cc mu khc nhau phi thc s khc nhau). Nu chng kh lin quan c th c mt kt hp ca m hnh khc nhau.


Thi Nguyn 2-2014 30

MNG N RON V NG DNG

2.3. ng dng mng n ron Hopfield gii cc bi ton ti u ha: 2.3.1. Bi ton ngi bn hng rong,

Hy xt bi ton kinh in sau: Thm tt c cc thnh ph mi thnh ph thm t nht mt ln, khi hnh v kt thc Bucaret. iu ny rt ging vi tm kim ng i, bi v cc ton t vn tng ng vi cc chuyn i gia cc thnh ph lin k. Nhng i vi bi ton ny, khng gian trng thi phi ghi nhn nhiu thng tin hn. Ngoi v tr ca agent, mi trng thi phi lu li c cc thnh ph m agent i qua. Nh vy, trng thi ban u s l Bucaret: thm{Bucaret}., mt trng thi trung gian in hnh s l Vaslui: thm {Bucaret, Urziceni, Vaslui}. v vic kim tra mc tiu s kim tra xem agent Bucaret cha v tt c 20 thnh ph c ving thm xong ton b cha. Bi ton ngi bn hng rong (TSP) l mt bi ton du lch ni ting trong mi thnh ph phi c ving thm ng chnh xc mt ln. Mc ch l tm hnh trnh ngn nht. 6 Bi ton c phc tp NP(Karp,1972), nhng c mt s c gng rt ln nhm ci thin kh nng ca cc thut ton TSP. Ngoi cc chuyn i c lp k hoch cho ngi bn hng rong, nhng thut ton ny c s dng cho cc nhim v nh lp k hoch cho s dch chuyn ca cc mi khoan trn trn bng mch mt cch t ng. ng dng mng Hopfild gii bi ton ngi bn hng T mu th, phng ha th, Mng Hopfield c th s dng gii bi ton tha mn cc rng buc. Chng ny trnh by ng dng ca mng Hopfield gii bi ton bn mu, v c p dng t mu bn cho mt bn bt ky. 1. Gii thiu bi ton bn mu Chung ta bit rng mt th phng bt ky c th t bi bn mu sao cho hai nh k nhau c hai mu khc nhau. p dng vo mng Hopfield thc hin bi ton ny. Bi ton t ra l: Cho mt bn trong c N vng ( cc tnh thnh hoc cc nc gn nhau ). Cn t mu cho cc vng sao cho hai nc k nhau c t bi hai mu khc nhau, nh c th phn bit c cc vng d dng. Thc hin: Trc ht ta s m hnh ha bi ton ny nh mt bi ton tha mn rng buc. K hiu: x : l ch s min cn t mu (x = 1, 2, , n). i :l ch s mu (i;1, 2, 3, 4).


Thi Nguyn 2-2014 31

MNG N RON V NG DNG


Thi Nguyn 2-2014 32

MNG N RON V NG DNG


Thi Nguyn 2-2014 33

MNG N RON V NG DNG

2.3.2. Lp lch biu,

Gii hn ca mng Hopfield (The limits of Hopfield networks trang 29 K13) Mng Hopfield rt ph hp vi cc bi ton ti u t hp, c bit l i vi mt s bi ton thuc lp bi ton NP y nh bi ton ngi bn hng, bi ton snh cp c trng .

Tuy nhin, khi s dng mng Hopfield vo vic gii cc bi ton ti u trong thc t cn gp mt s hn ch sau:

Mng Hopfield khng m bo cho hi t ton cc. khc phc , ngi ta kt hp mng Hopfield vi mt s gii thut khc, v d nh gii thut di truyn , hoc a vo phng trnh ng hc mt s hng c bit gi l s hng leo i. T hm nng lng ca mng c th thay i n mt trng thi cao hn, trnh c hi t a phng v tin ti hi t ton cc. Vic chn h s ca hm mc tiu v h s ca hm rng buc nhn mt li gii t l ht sc kh khn. Cho n nay, vic chn n vn ch yu da vo kinh nghim.

2.4. Bi tp


Thi Nguyn 2-2014 34

MNG N RON V NG DNG

Bai Giang Mang Noron Va Ung Dung p1

Documents

Transcript of Bai Giang Mang Noron Va Ung Dung p1