Plasma cells
Non-plasma cells
Optimal hyperplane
• Naïve Bayes
• Support Vector Machine
• Fisher’s discriminant
• Logistic
• Mahalanobis
Making classifiers by supervised learning
ii
n
nn
n
n
n
CFp
CFpCFpCFp
CFFFpCFFFpCFFpCFp
CFFFFpCFFpCFp
CFFFpCFp
CFFFpCp
)|(
)|()...|()|(
),,...,|()...,,|(),|()|(
),,|,...,(),|()|(
),|,...,()|(
)|,...,,()|(
21
11213121
213121
121
21F
i
i CFpCP )|()|(F
where, C and ¬C represent plasma cell and non-plasma cell, and Fi represent i-th different discrete fluorescence data. Using Bayes’ theorem,
Similarly, for the non-plasma cell, we can calculate its probability by the following equation,
i i
i
i i
i
CFp
CFp
Cp
Cp
CFp
CFp
Cp
Cp
CpCp
CpCp
Cp
Cp
)|(
)|(ln
)(
)(ln
)|(
)|(
)(
)(ln
)()|(
)()|(ln
)|(
)|(ln
F
F
F
F
),...,,|()|Cp( ),,...,,|()|( 2121 nn FFFCpFFFCpCp FF
),...,,(
)()|,...,,(),...,,|(
21
2121
n
nn FFFp
CpCFFFpFFFCp
)()|(
)()|(
)()(),(
CpFCp
FpCFp
CpFpCFp
)|(),|(
)|(),|(
)|()|()|,(
212
121
2121
CFpCFFp
CFpCFFp
CFpCFpCFFp
Naïve Bayes Classifier
Our model makes the simplifying assumption that, conditional on the state of the cell (i.e. C/¬C), the fluorescence data are independent: i.e.,
Finally, log-likelihood ratio can be written as following,
Statistical independence
Conditional independence
http://en.wikipedia.org/wiki/Naive_Bayes_classifier
Nibtyt
ty
ty
byhyperplane
iT
iii
ii
ii
T
,...,1 ,0)()(
1 ,0)(
1 ,0)(
)(:
xwx
x
x
xwx
1,1 ),,...,,(:
,...,,:
21
21
iN
N
ttttclass
input
t
xxx
b)(t|||| i
Ti
i,bxw
wwmin
1maxarg
Distance between hyper plane and xi : ||||
)(
||||
)(
w
xw
w
x btyt iT
iii
Scaling: 1min b)(t iT
ii
xw
||||
2
w
Maximize margin:
Nibt iT
i ,...,1 ,1)(:subject to ,|||| : minimize xww
SVM (Hard margin)
http://en.wikipedia.org/wiki/Support_vector_machine, Pattern Recognition And Machine Learning (Christopher M. Bishop)
Nibt iT
i ,...,1 ,1)(:subject to ,||||2
1 : minimize 2 xww
Quadratic programming (Primal and dual form)
Lagrangian:
N
ii
Tii b)(ta-|||| ,b,a)L(
1
2 12
1xwww
N
iiiitaw
L
1
0 xw
N
iiita
b
L
1
00
N
iii
i
N
i
N
jjijiji
N
ii
ta
,...,Nia
ttaaa (a)L
1
1 11
0
1 ,0
,2
1~xx
N
iiiitaw
1
x
Only a few ai will be greater than 0 (support vectors), which lie on the margin and satisfy
Nsv
iii
T
iiT
iT
i
tNsv
b
tb
b)(t
1
1
1
xw
xw
xw 01 b)(ta iT
ii xw
),...,( 21 Naaaa
By SMO
)()(),(
,...,1 ,0)( subject to
),( : minimize
1
i
xxx
x
x
i
N
ii gfL
Lagrangian
Nig
f
activeiifxg
xg
xg
gf
i
ii
i
i
i
N
ii
0 0)(
0)(
0
0)(
0)()(1
xx
QP:
KKT conditions
,...,Niyt iii 1 ,1)( x
NibtξC|||| iiT
i
N
ii ,...,1 ,1)(:subject to ,
2
1 : minimize
1
2
xww
Lagrangian:
N
i
N
iiiii
Tii
N
ii b)(ta-C|||| )a,b,L(
1 11
2 12
1,, xwww
N
iiiita
L
1
0 xww
N
iiita
b
L
1
00
iii
CaL
0
QP:
),...,( 21 Naaaa
N
iii
i
i
N
i
N
jjijiji
N
ii
ta
Ca
,...,Nia
ttaaa (a)L
1
i
1 11
0
0
1 ,0 ,0
,2
1~
xx
SVM (Soft margin)
||||
2
w
0
1
1
)( ii xx
N
i
N
jjijiji
N
ii
N
i
N
jjijiji
N
ii Kttaaattaaa (a)L
1 111 11
),(2
1)()(
2
1~xxxx
)()(
),2,)(,2,(
2
)()(),(
2221
21
2221
21
22
222211
21
21
22211
2
zx
zxzx
T
T
T
zzzzxxxx
zxzxzxzx
zxzxk
Txxxxx ),2,()( 2221
21
Kernel trick (non-linear classification)
が存在する.を満たす写像
が半正定値 カーネル行列〉〈
Fxxxxk
RcxxkccKcc
K
jiji
nn
i
n
jjiji
T
:,,
,0,
KernelMercer
1 1
jixxkxxk ijji ,),,(),(
Ex.
),2,(:),,(),( 2221
2132121
32
xxxxzzzxx
(4)
(3) ,)( 22
nT
n
Cnknk
y
mysk
xw
wSwwSw
w
w
WT
BT
J
ssmm
J
)(
)()( 2
221
212
21
))(())((
))((
2211
1212
Cn
Tnn
Cn
TnnW
TB
mxmxmxmxS
mmmmS
Maximize J(w)
)(
))()(()()(
121
1212
mmSw
wmmmmwSwwSwSwwSwSw
W
TW
TBW
TWB
T
2
'''
,0g
fggf
g
fJ
w
scalar
21 2
21
1
1,
1
Cnn
Cnn NN
xmxm
(2)
(1) ),( 1212
kT
k
T
m
mm
mw
mmw
xwTy
Within class variance with label k
射影されたクラスの平均の分離度
(1), (2), (3), (4)を代入
Fisher discriminant analysis
Cluster 16 = plasma cells
Sensitivity = 91.63 %Specificity = 99.90 %
Naïve Bayes ClassificationTrue
Plasma Non-plasma
predictionPlasma 1477 99
Non-Plasma 135 98289
Naïve Bayes Classification
Plasma cells
Non-plasma cells
SVM classification (Radial kernel)
Sensitivity = 97.02 %Specificity = 99.97 %
Plasma cells
Non-plasma cells
SVM ClassificationTrue
Plasma Non-plasma
predictionPlasma 1564 26
Non-Plasma 48 98362
Summary of classification
by T xwx)(
0)( ||||
)(
||||)()(
||||
||||
||||
xw
x
wxx
w
wwxwxw
w
wwxwxw
w
wxx
yy
r
ryy
rbb
r
r
TTT
TTT
x
x
w
||||
)(
w
xy
Distance from a point to a plane
Top Related