![arXiv:1406.1078v3 [cs.CL] 3 Sep 2014raywong/comp5331/References/Lea...In this formulation, when the reset gate is close to 0, the hidden state is forced to ignore the pre-vious hidden](https://static.fdocuments.net/doc/165x107/5e2649a2037fa55f33571e2a/arxiv14061078v3-cscl-3-sep-raywongcomp5331referenceslea-in-this-formulation.jpg)
COMP5331
22
COMP5331 1 COMP5331 Other Classification Models: Support Vector Machine (SVM) Prepared by Raymond Wong Presented by Raymond Wong raywong@cse
-
Upload
germaine-vega -
Category
Documents
-
view
23 -
download
2
description
COMP5331. Other Classification Models: Support Vector Machine (SVM). Prepared by Raymond Wong Presented by Raymond Wong raywong@cse. What we learnt for Classification. Decision Tree Bayesian Classifier Nearest Neighbor Classifier. Other Classification Models. - PowerPoint PPT Presentation
Transcript of COMP5331
COMP5331 1
COMP5331
Other Classification Models:Support Vector Machine (SVM)
Prepared by Raymond WongPresented by Raymond Wong
raywong@cse
COMP5331 2
What we learnt for Classification
Decision Tree Bayesian Classifier Nearest Neighbor Classifier
COMP5331 3
Other Classification Models
Support Vector Machine (SVM) Neural Network
COMP5331 4
Support Vector Machine
Support Vector Machine (SVM) Linear Support Vector Machine Non-linear Support Vector Machine
COMP5331 5
Support Vector Machine
Advantages: Can be visualized Accurate when the data is well
partitioned
COMP5331 6
Linear Support Vector Machine
x1
x2
w1x1 + w2x2 + b = 0
w1x1 + w2x2 + b > 0
w1x1 + w2x2 + b < 0
COMP5331 7
Linear Support Vector Machine
COMP5331 8
Linear Support Vector Machine
COMP5331 9
Linear Support Vector Machine
COMP5331 10
Linear Support Vector Machine
Margin
We want to maximize the margin Why?
x1
x2
Support Vector
COMP5331 11
Linear Support Vector Machine
x1
x2
w1x1 + w2x2 + b = 0
w1x1 + w2x2 + b - D = 0
w1x1 + w2x2 + b + D = 0
COMP5331 12
Linear Support Vector Machine
x1
x2
w1x1 + w2x2 + b = 0
w1x1 + w2x2 + b - 1 = 0
w1x1 + w2x2 + b + 1 = 0
w1x1 + w2x2 + b - 1 0
w1x1 + w2x2 + b + 1 0
+1+1
+1 +1-1
-1 -1
-1
Let y be the label of a point
COMP5331 13
Linear Support Vector Machine
x1
x2
w1x1 + w2x2 + b = 0
w1x1 + w2x2 + b - 1 = 0
w1x1 + w2x2 + b + 1 = 0
w1x1 + w2x2 + b - 1 0
w1x1 + w2x2 + b + 1 0
+1+1
+1 +1-1
-1 -1
-1
Let y be the label of a pointy(w1x1 + w2x2 + b) 1
y(w1x1 + w2x2 + b) 1
COMP5331 14
Linear Support Vector Machine
x1
x2
+1+1
+1 +1-1
-1 -1
-1
Let y be the label of a pointy(w1x1 + w2x2 + b) 1
y(w1x1 + w2x2 + b) 1
Margin
We want to maximize the margin
w1x1 + w2x2 + b - 1 = 0
w1x1 + w2x2 + b + 1 = 0
Margin
=|(b+1) – (b-1)|
22
21 ww
=2
22
21 ww
COMP5331 15
Linear Support Vector Machine Maximize
Subject to
for each data point (x1, x2, y)where y is the label of the point (+1/-1)
=2
22
21 ww
y(w1x1 + w2x2 + b) 1
Margin
COMP5331 16
Linear Support Vector Machine Minimize
Subject to
for each data point (x1, x2, y)where y is the label of the point (+1/-1)
2
22
21 ww
y(w1x1 + w2x2 + b) 1
COMP5331 17
Linear Support Vector Machine Minimize
Subject to
for each data point (x1, x2, y)where y is the label of the point (+1/-1)
22
21 ww
y(w1x1 + w2x2 + b) 1
Quadratic objective
Linear constraints
Quadratic programming
COMP5331 18
Linear Support Vector Machine
We have just described 2-dimensional space
We can divide the space into two parts by a line
For n-dimensional space where n >=2, We use a hyperplane to divide the
space into two parts
COMP5331 19
Support Vector Machine
Support Vector Machine (SVM) Linear Support Vector Machine Non-linear Support Vector Machine
COMP5331 20
Non-linear Support Vector Machine
x1
x2
COMP5331 21
Non-linear Support Vector Machine
Two Steps Step 1: Transform the data into a
higher dimensional space using a “nonlinear” mapping
Step 2: Use the Linear Support Vector Machine in this high-dimensional space
COMP5331 22
Non-linear Support Vector Machine
x1
x2
