Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction
description
Transcript of Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction
![Page 1: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/1.jpg)
Local Fisher Discriminant Analysis for Supervised
Dimensionality Reduction
Presented by Xianwang WangPresented by Xianwang Wang
Masashi Sugiyama
![Page 2: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/2.jpg)
Dimensionality Reduction Goal
Embed high-dimensional data to low-dimensional space Preserve intrinsic information
Example
High dimension
3-dimension
![Page 3: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/3.jpg)
Categories Nonlinear
ISOMAP Locally Linear Embedding (LLE) Laplacian Eigenmap (LE)
Linear Principal Components Analysis (PCA) Locality-Preserving Projection (LPP) Fisher Discriminant Analysis (FDA)
Unsupervised S-ISOMAP, S-LLE, PCA
Supervised LPP, FDA
![Page 4: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/4.jpg)
Formulation Number of samples:
d-dimensional samples:
Class labels :
Number of samples in the class :
Data matrix :
Embedded samples:
![Page 5: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/5.jpg)
Goal for linear dimensionality Reduction Find a transformation matrix
Use Iris data for demos (http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data) Attribute Information:
sepal length in cm sepal width in cm petal length in cm petal width in cm
class: Iris Setosa; Iris Versicolour; Iris Virginica
![Page 6: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/6.jpg)
FDA(1) Mean of samples in the class
Mean of all samples
Within-class scatter matrix
Between-class scatter matrix
![Page 7: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/7.jpg)
FDA(2) Maximize the following objective
Maximize the following constrained optimization problem equivalently
Use the lagrangian,
Apply KKT conditions
Demo
![Page 8: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/8.jpg)
LPP Minimize
Equivalently
We can get
Demo
![Page 9: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/9.jpg)
Local Fisher Discriminant Analysis(LFDA) FDA can perform poorly if samples in some class form
several separate clusters
LPP can make samples of different classes overlapped if they are close in the original high dimensional space
LFDA combines the idea of FDA and LPP
![Page 10: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/10.jpg)
LFDA(1) Reformulating FDA
![Page 11: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/11.jpg)
LFDA(2) Definition of LFDA
![Page 12: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/12.jpg)
LFDA(3) Maximize the following objective
Equivalently,
Similarly, we can get
Demo
![Page 13: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/13.jpg)
Conclusion
LFDA provided more separate embedding than FDA and LPP
FDA (globally), while LFDA(locally)
More discussion about efficiently computing of LFDA transformation matrix and Kernel LFDA in the paper
![Page 14: Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction](https://reader033.fdocuments.net/reader033/viewer/2022061602/56814548550346895db21726/html5/thumbnails/14.jpg)
Questions?