Plasma cells

Non-plasma cells

Optimal hyperplane

• Naïve Bayes

• Support Vector Machine

• Fisher’s discriminant

• Logistic

• Mahalanobis

Making classifiers by supervised learning

ii

n

nn

n

n

n

CFp

CFpCFpCFp

CFFFpCFFFpCFFpCFp

CFFFFpCFFpCFp

CFFFpCFp

CFFFpCp

)|(

)|()...|()|(

),,...,|()...,,|(),|()|(

),,|,...,(),|()|(

),|,...,()|(

)|,...,,()|(

21

11213121

213121

121

21F

i

i CFpCP )|()|(F

where, C and ¬C represent plasma cell and non-plasma cell, and Fi represent i-th different discrete fluorescence data. Using Bayes’ theorem,

Similarly, for the non-plasma cell, we can calculate its probability by the following equation,

i i

i

i i

i

CFp

CFp

Cp

Cp

CFp

CFp

Cp

Cp

CpCp

CpCp

Cp

Cp

)|(

)|(ln

)(

)(ln

)|(

)|(

)(

)(ln

)()|(

)()|(ln

)|(

)|(ln

F

F

F

F

),...,,|()|Cp( ),,...,,|()|( 2121 nn FFFCpFFFCpCp FF

),...,,(

)()|,...,,(),...,,|(

21

2121

n

nn FFFp

CpCFFFpFFFCp

)()|(

)()|(

)()(),(

CpFCp

FpCFp

CpFpCFp

)|(),|(

)|(),|(

)|()|()|,(

212

121

2121

CFpCFFp

CFpCFFp

CFpCFpCFFp

Naïve Bayes Classifier

Our model makes the simplifying assumption that, conditional on the state of the cell (i.e. C/¬C), the fluorescence data are independent: i.e.,

Finally, log-likelihood ratio can be written as following,

Statistical independence

Conditional independence

http://en.wikipedia.org/wiki/Naive_Bayes_classifier

Nibtyt

ty

ty

byhyperplane

iT

iii

ii

ii

T

,...,1 ,0)()(

1 ,0)(

1 ,0)(

)(:

xwx

x

x

xwx

1,1 ),,...,,(:

,...,,:

21

21

iN

N

ttttclass

input

t

xxx

b)(t|||| i

Ti

i,bxw

wwmin

1maxarg

Distance between hyper plane and xi : ||||

)(

||||

)(

w

xw

w

x btyt iT

iii

Scaling: 1min b)(t iT

ii

xw

||||

2

w

Maximize margin:

Nibt iT

i ,...,1 ,1)(:subject to ,|||| : minimize xww

SVM (Hard margin)

http://en.wikipedia.org/wiki/Support_vector_machine, Pattern Recognition And Machine Learning (Christopher M. Bishop)

Nibt iT

i ,...,1 ,1)(:subject to ,||||2

1 : minimize 2 xww

Quadratic programming (Primal and dual form)

Lagrangian:

N

ii

Tii b)(ta-|||| ,b,a)L(

1

2 12

1xwww

N

iiiitaw

L

1

0 xw

N

iiita

b

L

1

00

N

iii

i

N

i

N

jjijiji

N

ii

ta

,...,Nia

ttaaa (a)L

1

1 11

0

1 ,0

,2

1~xx

N

iiiitaw

1

x

Only a few ai will be greater than 0 (support vectors), which lie on the margin and satisfy

Nsv

iii

T

iiT

iT

i

tNsv

b

tb

b)(t

1

1

1

xw

xw

xw 01 b)(ta iT

ii xw

),...,( 21 Naaaa

By SMO

)()(),(

,...,1 ,0)( subject to

),( : minimize

1

i

xxx

x

x

i

N

ii gfL

Lagrangian

Nig

f

activeiifxg

xg

xg

gf

i

ii

i

i

i

N

ii

0 0)(

0)(

0

0)(

0)()(1

xx

QP:

KKT conditions

,...,Niyt iii 1 ,1)( x

NibtξC|||| iiT

i

N

ii ,...,1 ,1)(:subject to ,

2

1 : minimize

1

2

xww

Lagrangian:

N

i

N

iiiii

Tii

N

ii b)(ta-C|||| )a,b,L(

1 11

2 12

1,, xwww

N

iiiita

L

1

0 xww

N

iiita

b

L

1

00

iii

CaL

0

QP:

),...,( 21 Naaaa

N

iii

i

i

N

i

N

jjijiji

N

ii

ta

Ca

,...,Nia

ttaaa (a)L

1

i

1 11

0

0

1 ,0 ,0

,2

1~

xx

SVM (Soft margin)

||||

2

w

0

1

1

)( ii xx

N

i

N

jjijiji

N

ii

N

i

N

jjijiji

N

ii Kttaaattaaa (a)L

1 111 11

),(2

1)()(

2

1~xxxx

)()(

),2,)(,2,(

2

)()(),(

2221

21

2221

21

22

222211

21

21

22211

2

zx

zxzx

T

T

T

zzzzxxxx

zxzxzxzx

zxzxk

Txxxxx ),2,()( 2221

21

Kernel trick (non-linear classification)

が存在する．を満たす写像

　 　

が半正定値　 カーネル行列〉〈

Fxxxxk

RcxxkccKcc

K

jiji

nn

i

n

jjiji

T

:,,

,0,

KernelMercer

1 1

jixxkxxk ijji ,),,(),( 　

Ex.

),2,(:),,(),( 2221

2132121

32

xxxxzzzxx

(4)

(3) ,)( 22

nT

n

Cnknk

y

mysk

xw

wSwwSw

w

w

WT

BT

J

ssmm

J

)(

)()( 2

221

212

21

))(())((

))((

2211

1212

Cn

Tnn

Cn

TnnW

TB

mxmxmxmxS

mmmmS

Maximize J(w)

)(

))()(()()(

121

1212

mmSw

wmmmmwSwwSwSwwSwSw

W

TW

TBW

TWB

T

2

'''

,0g

fggf

g

fJ

w

scalar

21 2

21

1

1,

1

Cnn

Cnn NN

xmxm

(2)

(1) ),( 1212

kT

k

T

m

mm

mw

mmw

xwTy

Within class variance with label k

射影されたクラスの平均の分離度

(1), (2), (3), (4)を代入

Fisher discriminant analysis

Cluster 16 = plasma cells

Sensitivity = 91.63 %Specificity = 99.90 %

Naïve Bayes ClassificationTrue

Plasma Non-plasma

predictionPlasma 1477 99

Non-Plasma 135 98289

Naïve Bayes Classification

Plasma cells

Non-plasma cells

SVM classification (Radial kernel)

Sensitivity = 97.02 %Specificity = 99.97 %

Plasma cells

Non-plasma cells

SVM ClassificationTrue

Plasma Non-plasma

predictionPlasma 1564 26

Non-Plasma 48 98362

Summary of classification

by T xwx)(

0)( ||||

)(

||||)()(

||||

||||

||||

xw

x

wxx

w

wwxwxw

w

wwxwxw

w

wxx

yy

r

ryy

rbb

r

r

TTT

TTT

x

x

w

||||

)(

w

xy

Distance from a point to a plane