Inteligenta computationala

7/28/2019 Inteligenta computationala

1/148

Chonbuk National University 2008


2/148



3/148



4/148


n

iii

n

iijij

n

iii

xwy

xwy

xwy

122

1

111

..

jiji yxw ~

1y

2y


5/148



6/148



7/148



8/148



9/148



10/148



11/148



12/148



13/148



14/148



15/148



16/148



17/148



18/148



19/148



20/148



21/148


kk

m

k

kk owoy

0


22/148


xo

x o

xxx mj ,...,,...1


23/148


jj xxn

i

ijij xw1

tanhx

R

d

dRBF

ddRBFjj

exp)(:exemplude

;),( wxx

)1(1 jxtrix iijji x

altfel,0

;daca,1 xx

xtrinq

1numberquantanq

12,..1 nqj


24/148


2exp,, kk

defk

K xxxxxx


25/148


n

k

kkk Kdby

1

,xx

2

2

2exp),(

r

drdg

2

2exp,

kkK xxxx

n

i

jiij

e cxd1

2

2cx

kj xc2/1

2r


26/148



27/148



28/148



29/148



30/148



31/148



32/148



33/148



34/148



35/148



36/148



37/148



38/148



39/148


n

i

jiij

mcxd

11

cx

n

i

jiij

ecxd

1

2

2cx

else1

if0),(

rk

drkd

rdm

2k

2

2

2exp),(

r

drd

g


40/148


else1

if0),(

r

drd

rdm


41/148



42/148



43/148



44/148



45/148



46/148



47/148



48/148



49/148



50/148



51/148



52/148



53/148



54/148



55/148



56/148



57/148


5.


58/148



59/148



60/148



61/148



62/148



63/148



64/148


2

1

22

1

2

)(m

jjjo

m

jjh

wodydeJ

eJ

jj

j

jj

o

jj

h

o

ye

e

o

o

ye

e

J

ee

J

22

2

2

1

22

1

2

)(m

jjjo

m

jjh

wodydeJ

eJ

jj

j ewo

yee


65/148


K

k

m

jkjkjk

K

kko wodeJ

1

2

11

2

K

kkjk

K

k kjkj weo

yee

11


66/148



67/148



68/148



69/148



70/148



71/148



72/148



73/148



74/148



75/148



76/148


rule 1: IF temperature IS cool AND pressure IS weak


77/148


rule 1: IF temperature IS cool AND pressure IS weak,THEN throttle is P3.

rule 2: IF temperature IS cool AND pressure IS low,THEN throttle is P2.

1,0xA


78/148



79/148



80/148



81/148



82/148



83/148


xA


84/148


xA


85/148



86/148



87/148



88/148



89/148



90/148



91/148



92/148



93/148



94/148



95/148



96/148



97/148



98/148



99/148



100/148


Classic solution:

Alternative:

Compact Image Compression for Mobile Equipment


101/148



102/148


71x71 micro meters

per cell in a 2 um techn.


103/148



104/148


xxg )( )( 0123 xzzzzxf

3 absolute value nestsThe absolute value nest


105/148


0 00

zf


106/148


0

0

zf1k

k

k fzf

mk ,...,1

m=0

m=1

m=2

0f

)(011 fzf

)(122

fzf

0

0

0 z

10

1

0 zz

10

1

1 zz

2

1

0

2

0

z

zz

2

1

0

2

0

z

z

z2

1

0

2

0

z

z

z

2

1

0

2

0

z

z

z

3

1i

iibu

expander.nonlinearaasacting

input whensignalabecanitor

Adalineanofoutputthebe

maywhichuescalar vala-

,nests""ofnumberthe-

exponent,positiveais-

,parametersbiasare,..,0

m

zz m

1k

k

kfgzf


107/148


)(xgRTD

A

g

pV VV


108/148


IP

IVVV VPPVP

x

g

g

g

V

2gg

P

P

V

Ig

Pv

PV

VV

IIg

projection


109/148



110/148


n

i

iiub1

msfy

The architecture of the multi-nested cell

Example:

PARITY-8: 1,2,4,0,1,1,1,1,1,1,1,1,1,1 zb s ,


111/148


Digital MLP

Multi-nested PWL Cell

The M-nested Cell Equation


112/148


Any Boolean function with n inputs has an associatedgene G. The main problem is to find (learn) the gene.


113/148



114/148


0pZ LZ 216384214

L


115/148


tDiscID

n

tanh2

2

0z

pZ

BpB b

zLZ 2

bLB

16384214

L

Found IDs 6 03 22 5 91 11 61228 61174 60074 58124 60190 524 54 46388

b 1 4 6 7 8 10 75 75 0 1500

z1 16 16 16 16 16 20 0 2 00 0 4 000

z

b

After 106

iterations


116/148


30000

99995.0


117/148


trials


118/148


The MNEST cell f or classif icat ion problems


119/148


B1=[-0.2689 0.7069 -0.8141 0.3913]Z1=[0.2501]

B2=[0.9122 0.2739 -2.6800 -1.6449]Z2=[-0.1138 0.6395 -2.0810 0.6826 -1.0501 -0.3774]

B3=[-0.7208 -0.1217 0.8324 1.1242]Z3=[-0.2136 0.1033 -0.8857 0.8326 -0.4945 -0.4378 -0.1670]

0%

1.33%

1.33%

MNEST

cells trainedto solve theIRIS

problem

The IRISproblem

Cell2

Cell1

Cell3

Missclassification error on test data

Using a digital multiplexer

16 to 1

Multiplexer

More than 120 CMOS transistors

are needed for each CNN cell


120/148


p

Using an RTD-CNN cell

Iref0

Iref1

Irefm


121/148


IIN2 IINm+1

IR1

I1

In

IRTD1

I1

I0

IIN1

Legend

IRTD2


122/148


)( xx UsynI

inout IIII 12


123/148


n

i

biibin UsynuUsynI1

0 )()(

inout 12


124/148



125/148



126/148



127/148


uj

uj


128/148



129/148


n

i

iinn vbzzzzsignc1

121 ...

lii icc ll


130/148


A filtering problem given by training samples

Test samples used to evaluate the quality

The DRAM content (512 bits)

The result of the simplicial filtering

The simplicial filter with continuous

coefficients; error evolution during training


131/148



132/148

Chonbuk National University 2008 1 bit quantization


133/148


The binary gene

can be realized with

a simple m-nest cell instead

of the RAM ----->

323221432 ,,,,,, uuuuuuuuuu1u432 ,,, uuuu1u


134/148


4241323221432 ,,,,,,,, uuuuuuuuuuuuuu1u

C1 C2 C3 N C1Q C2Q C3QClass 1: 19.2% 20.13% 19.95% 17% 29.35% 22.61% 20.91%

5432 ,,,, uuuuu1u

514131215432 ,,,,,,,, uuuuuuuuuuuuu1u

32514131215432 ,,,,,,,,, uuuuuuuuuuuuuuu1u


135/148


,1211kk

uu mk ,...,2


136/148


Objective: A c om pac t , VLSI f r ien dly neuralarchitecture for classification tasks

Precursors: Thesimplicial neural net (S-net)


137/148


Problems with the S-nets : Requires hugememory O(2n) for n inputs, impractical forn>20

Main feature of the SORT net : Requires much

less memory, O(n2

) at most for n inputs, it isbased on the observation that sorting of the

input vector can provide a very effectivemean to generate a partition of the inputspace. The result is a kernel-based neural net


138/148



139/148


M

k

jkcy

1

For a given number of inputs provides an

expanded vector to the sorting processor. The

effect is equivalent to having amuch finer

subdivisionof the inputspace(morebasis)suchth t h d li bl b l d


140/148


subdivisionof the input space (more basis) suchthat hard nonlinear problems can be solved.

ii uu 1,iml ,2FOR

121 1,, lili uu

END

How it w ork s (2 inpu t s c ase / z=1)

No expander

[1 1]Expander

[1 2]

1u

2u

1U

2U

1u

2u

1U

2U

3U

Sorter outputs: sequence of coefficientsjSorter outputs: sequence of coefficientsj

0 2 0 4 8


141/148


Sorter outputs:

1 0 1 2

2 0 2 1

3 1 2 0

4 1 0 2

5 2 0 16 2 1 0

1u1

1

sequence of coefficientskj

0 5 10

0 6 9

1 6 8

1 4 10

2 4 92 5 8

Sorter outputs:

1 0 1

2 1 0

sequence of coefficientskj

0 3

1 2

21 UU

12 UU

942)5( cccy u

2u

e.t.c

0 11 -12 -1

3 X1

1 Class +2 1 Cl

If the memory stores:

kjkcj

uu y

Fast learning: all coefficients selected

when u is in 3 and 6 take value 1


142/148


3 X4 15 -16 -17 X

8 -19 110 1

1 Class +2 1 Class +

3 -3 Class -4 1 Class +

5 1 Class +

6 -3 Class -

Using larger size expanders more

complicated class separation boundaries

can be learned.

Controlling the generalization performanceThe z parameter

Sorter outputs: sequence of coefficientskj5

0 4 8Sorter outputs: sequence of coefficients

kj5

0 0 4Z = 1 Z=2


143/148


1 0 1 2

2 0 2 1

3 1 2 0

4 1 0 2

5 2 0 16 2 1 0

j0 5 10

0 6 9

1 6 8

1 4 10

2 4 92 5 8

1 0 1 2

2 0 2 1

3 1 2 0

4 1 0 2

5 2 0 16 2 1 0

j0 5 6

0 2 5

1 2 4

1 0 6

2 0 52 1 4

No more

than one

coefficientoverlap

Thissegment

is a

separationfrontier

3 overlapping

Coefficients

- the same

output for

either 3 or 6

This segment

is notanymore a

separation

frontier

Low numbernof inputs

M=[1 ,1 , 1 , 1 , 1 ] M=[1 ,2 , 1 , 1 , 1 ] M=[3,1,3,3,1] M=[3,1,3,3,2]

Z = 1 27% (25) 23% (36) 18% (120) 18% (143)

Problem: Phonem e 5 input s, hard (best k now n resul t : 14.2%)


144/148


( )

Z= 2 29% (15) 23% (18) 20% (65) 24% (72)

Z=3 28% (10) 27% (12) 27% (33) 24.5% (36)

Z= 4 31% (5) 31% (6) 31% (22) 28.5% (24)

For low dimensional inputs, the best performance (lowestpercentage of incorrect classified patterns) is usuallyoptimized by trying various expansion schemes (M) for z=1

(no truncation in the bits representing the sequence)

Large number n of inputs

Problem: OPTD64 64 input s, 10 c lasse s (best know n resul t : 2.3%)

High memorization Less accurate separation frontier

Z= 1 2 3 4 5 6 7


145/148


In general no expansion is needed or when is needed it

should be applied up to 3 times for a few of the inputs.

The best performance (lowest percentage of incorrect

classified patterns) is now optimized by trying varioustruncation values z>1.

The number in parenthesis indicates the number of

coefficients to be stored

Misclerror

%

9.69

(26330)

8.86

(13760)

7.75

(7280)

8.08

(3980)

9.58

(2240)

15.6

(1250)

90

(640)


146/148


The main advantage is implementation simplicity for good

performance, compared with more sophisticated classifiers


147/148


performance, compared with more sophisticated classifiers

Using standard RAMs to store weights is very convenient and

can lead to very compact designs in the absence of

multipliers. The most critical component is the sorter but

some good VLSI implementations were already reported.

Problem: Choosing the right expansion vector M (GA or other

optimization techniques may be used) -> may take time,

cannot be determined using a simple method.


148/148


Inteligenta computationala

Documents

Transcript of Inteligenta computationala