The Improved Localized Generalization Error Model and Its Applications to

Feature Selection for RBFNN

报告人：马艳东

指导教师：王熙照教授

目录

阿Wing的L-GEM

基于范数的模型 Old model VS New model Algorithm for RBFNN Feature selection

Simulation Conclusion Future Work

阿 Wing 的 L-GEM

Wing et al. ignore the unseen samples located far from the training samples and compute the generalization error within the Q-neighborhood.

2

( )1

2

( )1

1 1( ) ( ( ) ( ))

(2 )

( ) ( ) ( )1 1

( ) ( ) ( ) (2 )

Q b

Q b

N

SM nS xb

Nb b

nS xb b b

R Q f x F x dxN Q

f x f x f xdx

F x F x F xN Q

2 2

( ) ( )1 1

2

( )1

1 1 1 1( ) ( ( ))

(2 ) (2 )

1 1( ( ) ( ))

(2 )

Q b Q b

Q b

N N

bn nS x S xb b

N

b nS xb

y dx err x dxN Q N Q

F x F x dxN Q

2 2

( ) ( )1 1

2 2

( ) ( )1 1

2

( )1

1 1 1 12 ( ) ( ( ))

(2 ) (2 )

1 1 1 12 ( ( )) ( ( ) ( ))

(2 ) (2 )

1 1 12 ( ) ( ( )

(2 )

Q b Q b

Q b Q b

Q b

N N

bn nS x S xb b

N N

b bn nS x S xb b

N

bnS xb

y dx err x dxN Q N Q

err x dx F x F x dxN Q N Q

y dx F xN Q N

＋

＋

＋ 2

( )1

1( ))

(2 )Q b

N

nS xb

F x dxQ

＋

22

*

(( ) ) A

( )SM

y

R Q

Qemp SR ＋ E ＋ ＋

Here,

( ) ( )b berr f x F x

2

1

(1/ ) ( ( ))N

emp bb

R N err x

2 2

( )1

1 1(( ) ) ( )

(2 )Q b

N

SQ nS xb

E y y dxN Q

NL-GEM

( )1

( ) ( ) ( ) ( )

1( ) ( ) ( )

Q

Q b

SM S

N

S xb

R Q f x F x p x dx

f x F x p x dxN

( )1

( ( ) ( ))1

( ( ) ( )) ( )

( ( ) ( ))Q b

bN

b bS xb

b

f x f x

f x F x p x dxN

F x F x

( )1

( )1

( )1

1( ( ) ( )) ( )

1( ) ( ) ( )

1( ) ( ) ( )

Q b

Q b

Q b

N

bS xb

N

b bS xb

N

bS xb

f x f x P x dxN

f x F x P x dxN

F x F x P x dxN

＋

( )QS

E y empR*SMR

Const

Old model VS New model

2* 2( )= (( ) ) ASMR Q y Qemp SR ＋ E ＋ ＋

* ( )QSM S empR E y R Const

Algorithm for RBFNN Feature selection based on

Step1. Initialize the IFS to be the full set of features; Step2. Train the classifier using the dataset with features in

IFS; Step3. For each i: Compute the for the classifier trained in Step2 witho

ut the i-th feature; Step4. Delete the z-th feature from IFS if the for the z-th fe

ature is the smallest among all choices; Step5. If the stopping criterion is not satisfied, go to Step2.

*SMR

Simulation

Conclusion

NL-GEM becomes simpler and easier to derivate, understand and realize while the performance is satisfying .

But the time complexity is not low

Future Work

to do more experiments to test the validity of this model

to make use of MSE to realize the model

亟待解决的问题 to improve the performance of time complexity 方法一：另起炉灶，重新推导模型 方法二：用其他方法计算 (1) 按范数计算。 两种最常用的方法， Mente-Calro 和网格法在 time co

mplexity 上都是不太好； (2) 统计的方法。 的 不好去掉。

( )QS

E y

y

Holder 不等式

1

1

2 2 2

1 11<p,q< + =1 ( , ),

p q

( , ),

2

fg

p

p q

f L u

L u fg f g

p q

du f du g du

设 ， ，若

则fg 且 。

当 时，上述不等式为Cauchy不等式：