Scholastic Book SupportVectorM Part01 2014-01-26
-
Upload
ephifannia -
Category
Documents
-
view
219 -
download
0
Transcript of Scholastic Book SupportVectorM Part01 2014-01-26
-
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
1/17
Scholastic Video Book Series
Part 1
Linear Support Vector Machines (LSVM)
(with English Narrations)
http://scholastictutors.webs.com
(http://scholastictutors.webs.com/Scholastic-Book-SupportVectorM-Part01-2014-01-26.pdf)
1
Scholastic Tutors (Jan, 2014)ISVT 911-0-20-140126-1
SUPPORT VECTOR
MACHINES
http://scholastictutors.webs.com/http://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
2/17
International Baccalaureate (IB)
2
Support Vector Machines - #1Liner Support Vector Machines (LSVM)
http://scholastictutors.webs.com
(SVM-001)
http://youtu.be/LXGaYVXkGtg
Click here to see the video
http://scholastictutors.webs.com/http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtghttp://scholastictutors.webs.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
3/17
Support Vector Machine (SVM) algorithms are used in
Classification.
Classification can be viewed as the task of separating classes
in feature space.
http://scholastictutors.webs.com
Support Vector Machines
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
http://scholastictutors.webs.com/http://scholastictutors.webs.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
4/17
Here we select 3 Support Vectors to start with.
They are S1, S2and S3.
Support Vector Machines
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
S1
S2
S3
=
2
1
= 21
= 40
http://youtu.be/LXGaYVXkGtg
Click here to see the video
http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtg -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
5/17
Here we will use vectors augmented with a 1 as a bias input,
and for clarity we will differentiate these with an over-tilde.
That is:
http://scholastic-videos.com
Support Vector Machines
=211
=
2
1 1
=40
1
= 21
= 2
1
= 40
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
6/17
Now we need to find 3 parameters , ,and based onthe following 3 linear equations:
http://scholastic-videos.com
Support Vector Machines
. + . + . = 1 ( )
. + . + . = 1 ( )
. + . + . = +1 (+ )
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
7/17
Lets substitute the values for S1 , S2 and S3 in the aboveequations.
http://scholastictutors.webs.com
Support Vector Machines
211
.211
+ 21 1
.211
+ 401
.211
=1
=211
= 21 1
=401
211 . 211 + 21 1 . 211 + 401 . 211 =1
211
.401
+ 21 1
.401
+ 401
.401
=+1
http://scholastictutors.webs.com/http://scholastictutors.webs.com/http://scholastictutors.webs.com/http://scholastictutors.webs.com/http://scholastictutors.webs.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
8/17
After simplification we get:
Simplifying the above 3 simultaneous equations we
get: == -3.25 and = 3.5.http://scholastic-videos.com
Support Vector Machines
6 + 4 + 9 =14 + 6 + 9 =1
9 + 9 +17 =+1
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
9/17
The hyper plane that discriminates the
positive class from the negative class is give
by:
Substituting the values we get:
http://scholastic-videos.com
Support Vector Machines
=
= 2
11 + 2
1 1 + 4
01
= 3.25 .211
+ 3.25 . 21 1
+ 3.5 .401
=10
3
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
10/17
Our vectors are augmented with a bias.
Hence we can equate the entry in as thehyper plane with an offset b.
Therefore the separating hyper plane equation
= + with = 1
0and offset = 3.
Support Vector Machines
http://youtu.be/LXGaYVXkGtgClick here to see the video
http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtg -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
11/17http://scholastic-videos.com
Support Vector Machines
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
S1
S2
S3
= + with = 10 and offset = 3 .
This is the expected decision surface of the LSVM.
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
12/17http://scholastic-videos.com
Support Vector Machines
Programme Code Example 1
% 3 support vector version
s1 = [ 0 -1 1 ];
s2 = [ 0 1 1 ];
s3 = [ 2 0 1 ];
A = [ sum(s1.*s1) sum(s2.*s1) sum(s3.*s1) ;
sum(s1.*s2) sum(s2.*s2) sum(s3.*s2) ;
sum(s1.*s3) sum(s2.*s3) sum(s3.*s3) ]
Y = [ -1 -1 +1 ]
X = Y/A
p = X(1)
q = X(2)
r = X(3)
W = [ p*s1 + q*s2 + r*s3 ]
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
When you run you should get:
= [ 1 0 -1]. This is a vertical line passing through x1=1.
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
13/17
http://scholastic-videos.com
Support Vector Machines -
Programme Code Examples 2
% 4 support vector version
s1 = [ 1 1 1 ];
s2 = [ 1 1 1 ];
s3 = [ 3 1 1 ];
s4 = [ 3 1 0 ];
A = [ sum(s1.s1) sum(s2.s1) sum(s3.s1) sum(s4.s1); sum(s1.s2) sum(s2.s2) sum(s3.s2) sum(s4.s1); sum(s1.s3) sum(s2.s3) sum(s3.s3) sum(s4.s3); sum(s1.s4) sum(s2.s4) sum(s3.s4) sum(s4.s4);] Y = [ 1 1 +1 +1 ]
X = Y/A
p = X(1) q = X(2)
r = X(3)
s = X(4)
W = [ ps1 + qs2 + rs3 + ss4 ]
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
When you run you should get:
= [ 1 0 -2]. This is a vertical line passing through x1=2.
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
14/17
http://scholastic-videos.com
Support Vector Machines -
Programme Code Example 3
% 5 support vector version
s1 = [ 1 0 1 ];
s2 = [ 2 0 1 ];
s3 = [ 3 0 1 ];
s4 = [ 2 2 1 ];
s5 = [ 3 2 1 ];
A = [ sum(s1.s1) sum(s2.s1) sum(s3.s1) sum(s4.s1)sum(s5.s1);
sum(s1.s2) sum(s2.s2) sum(s3.s2) sum(s4.s2) sum(s5.s2); sum(s1.s3) sum(s2.s3) sum(s3.s3) sum(s4.s3) sum(s5.s3); sum(s1.s4) sum(s2.s4) sum(s3.s4) sum(s4.s4) sum(s5.s4); sum(s1.s5) sum(s2.s5) sum(s3.s5) sum(s4.s5) sum(s5.s5)] Y = [ 1 1 1 +1 +1 ]
X = Y/A
p = X(1) q = X(2)
r = X(3)
s = X(4)
t = X(5)
W = [ ps1 + qs2 + rs3 + ss4 + ts5 ]
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
When you run you should get: = [ 0 1 -1]. This is a horizontal line passing through x2=1.
http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
15/17
Support Vector Machines
Classification Examples
Lets take the 5 support vector version = [ 0 1 -1]. This is a horizontal line
passing through x2=1.
Lets classify the point (x1,x2)=(4,2).
. = 01 . 42 = 2 > 1 Hence this point belongs to the red
class
Lets classify the point (x1,x2)=(2,-2).
. = 01 . 22 = 2 < 1 Hence this point belongs to the blue
class
We can do the same for any new point.
x1
x2
0
-1
-2
1
2
1 2 3 4 5 6
http://youtu.be/LXGaYVXkGtgClick here to see the video
http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtg -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
16/17
International Baccalaureate (IB)
16
Support Vector Machines - #1Linear Support Vector Machines (LSVM)
http://scholastic-videos.com
(SVM-001)
END of the Book
If you like to see similar solutions to any Mathematics problems please
contact us at: [email protected] your request.
http://youtu.be/LXGaYVXkGtgClick here to see the video
http://scholastic-videos.com/http://[email protected]/http://youtu.be/LXGaYVXkGtghttp://youtu.be/LXGaYVXkGtghttp://[email protected]/http://scholastic-videos.com/http://scholastic-videos.com/http://scholastic-videos.com/ -
8/9/2019 Scholastic Book SupportVectorM Part01 2014-01-26
17/17
Videos at: http://www.youtube.com/user/homevideotutor
17
(http://scholastictutors.webs.com/Scholastic-Book-SupportVectorM-Part01-2014-01-26.pdf)
Scholastic Video Book Series
Support Vector Machines (SVM)
Part 1
(LSVM)
(with English Narrations)
(END)
Scholastic Tutors (Jan, 2014)ISVT 911-0-20-140126-1
http://www.youtube.com/user/homevideotutorhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://scholastictutors.webs.com/Scholastic-Book-NeuralNetworks-Part02-2013-09-23.pdfhttp://www.youtube.com/user/homevideotutorhttp://www.youtube.com/user/homevideotutor