A Generalization of PCA to the Exponential Family Collins, Dasgupta and Schapire Presented by Guy...
-
Upload
martin-gibson -
Category
Documents
-
view
218 -
download
5
Transcript of A Generalization of PCA to the Exponential Family Collins, Dasgupta and Schapire Presented by Guy...
A Generalization of PCA to the Exponential Family
Collins, Dasgupta and Schapire
Presented by Guy Lebanon
Two Viewpoints of PCA
• Algebraic Given data find a linear
transformation such that the sum of squared distances is minimized (over all linear transformation )
• Statistical Given data assume that each point
is a random variable. Find the maximum likelihood estimator under the constraint that are in a K dimensional subspace and are linearly related to the data.
Nin RxxxD },,,{ 1
KN RRT :
KN RR
i ii Txx2
Nin RxxxD },,,{ 1 ix
),( IN i i }ˆ{ i
• The Gaussian assumption may be inappropriate – especially if the data is binary valued or non-negative for example.
• Suggestion: replace the Gaussian distribution by any exponential distribution.
Given data such that each point comes from an exponential family distribution , find the MLE for
under the assumption that it lies in a low dimensional subspace.
Nin RxxxD },,,{ 1
ix)()(, iiii cxtxe
i
• The new algorithm finds a linear transformation in the parameter space but a nonlinear subspace in the original coordinates .
• The loss functions may be cast in terms of Bregman distances.
• The loss function is not convex in the general case.
• The authors use the alternating minimization algorithm (Csiszar and Tsunadi) to compute the transformation.
i
x