CS558 Project
description
Transcript of CS558 Project
![Page 1: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/1.jpg)
CS558 Project
Local SVM Classification based on triangulation
(on the plane)
Glenn Fung
![Page 2: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/2.jpg)
Outline of Talk
Classification problem on the plane All of the recommended stages were applied:
Sampling Ordering:
Clustering Triangulation
Interpolation (Classification)SVM: Support vector Machines
Optimization: Number of training points increased Evaluation:
Checkerboard datasetSpiral dataset
![Page 3: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/3.jpg)
Classification Problem in Given m points in 2 dimensional space Represented by an m-by-2 matrix A Membership of each in class +1 or –1A i
R 2
![Page 4: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/4.jpg)
SAMPLING:1000 randomly sampled points
![Page 5: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/5.jpg)
ORDERING:Clustering
A Fuzzy-logic based clustering algorithm was used. 32 cluster centers were obtained
-50 0 50 100 150 200 250-50
0
50
100
150
200
250
![Page 6: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/6.jpg)
0 50 100 150 200
0
50
100
150
200
ORDERING:Delaunay Triangulation
Algorithms to triangulate and to get the Delaunay triangulation from HWKs 3 and 4 were used. Given a point,the random point approach is used to localize the triangle that contains it.
![Page 7: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/7.jpg)
Interpolation:SVM
SVM : Support Vector Machine Classifiers A different nonlinear Classifier is used for each triangle
The triangle structure is efficiently used for both training and testing phases and for defining a “simple” and fast nonlinear classifier.
![Page 8: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/8.jpg)
What is a Support Vector Machine?
An optimally defined surface Typically nonlinear in the input space Linear in a higher dimensional space Implicitly defined by a kernel function
![Page 9: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/9.jpg)
What are Support Vector Machines Used For?
Classification Regression & Data Fitting Supervised & Unsupervised Learning
(Will concentrate on classification)
![Page 10: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/10.jpg)
Support Vector MachinesMaximizing the Margin between Bounding
Planes
x0w= í +1
x0w= í à 1
A+
A-
jjwjj22
w
![Page 11: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/11.jpg)
The Nonlinear Classifier
K (A;A0) : Rmân â Rnâm7à! RmâmK (x0;A0)Du = í
The nonlinear classifier:
Where K is a nonlinear kernel, e.g.: Gaussian (Radial Basis) Kernel :
"àökA iàA jk22; i; j = 1;. . .;mK (A;A0)ij = The ij -entry of K (A;A0) represents the “similarity”
of data points A i A jand
![Page 12: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/12.jpg)
Reduced Support Vector Machine AlgorithmNonlinear Separating Surface: K (x0;Aö0)Döuö= í
(i) Choose a random subset matrix ofA 2 Rmânentire data matrix A 2 Rmâ n
(ii) Solve the following problem by the Newtonmethod with corresponding D ú D :
2÷kp(eà D(K (A;A0)Döuöà eí );ë)k22+ 2
1kuö; í k22min(u; í ) 2 Rm+1
K (x0;Aö0)Döuö= í
(iii) The separating surface is defined by the optimal(u;í )solution in step (ii):
![Page 13: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/13.jpg)
How to Choose in RSVM?A A is a representative sample of the entire dataset
Need not be a subset of A A good selection of A may generate a classifier using
very small m Possible ways to chooseA :
Choose random rows from the entire datasetm A Choose such that the distance between its rows A
exceeds a certain tolerance Use k cluster centers of Aas AàA+ and
![Page 14: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/14.jpg)
![Page 15: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/15.jpg)
Obtained Bizarre “Checkerboard”
![Page 16: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/16.jpg)
Optimization: More sampled pointsTraining parameters adjusted
![Page 17: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/17.jpg)
Result: Improved Checkerboard
![Page 18: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/18.jpg)
Nonlinear PSVM: Spiral Dataset94 Red Dots & 94 White Dots
![Page 19: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/19.jpg)
Next:Bascom Hill
![Page 20: CS558 Project](https://reader033.fdocuments.net/reader033/viewer/2022051518/56816028550346895dcf304f/html5/thumbnails/20.jpg)
Some Questions Would it work for B&W pictures (regression instead of classification?
Aplications?