O UT - OF -S AMPLE E XTENSION AND R ECONSTRUCTION ON M ANIFOLDS Bhuwan Dhingra Final Year (Dual...

23
OUT-OF-SAMPLE EXTENSION AND RECONSTRUCTION ON MANIFOLDS Bhuwan Dhingra Final Year (Dual Degree) Dept of Electrical Engg.

Transcript of O UT - OF -S AMPLE E XTENSION AND R ECONSTRUCTION ON M ANIFOLDS Bhuwan Dhingra Final Year (Dual...

OUT-OF-SAMPLE EXTENSION AND RECONSTRUCTION ON MANIFOLDSBhuwan DhingraFinal Year (Dual Degree)Dept of Electrical Engg.

INTRODUCTION

An m-dimensional manifold is a topological space which is locally homeomorphic to the m-dimensional Euclidean space

In this work we consider manifolds which are: Differentiable Embedded in a Euclidean space Generated from a set of m latent variables via a

smooth function f

INTRODUCTION

n >> m

NON-LINEAR DIMENSIONALITY REDUCTION

In practice we only have a sampling on the manifold

Y is estimated using a Non-Linear Dimensionality Reduction (NLDR) method

Examples of NLDR methods โ€“ISOMAP, LLE, KPCA etc.

However most non-linear methods only provide the embedding Y and not the mappings f and g

PROBLEM STATEMENT

x*y*

g

f

OUTLINE

p is the nearest neighbor of x* Only the points in are used for extension and

reconstruction

OUTLINE

The tangent plane is estimated from the k-nearest neighbors of p using PCA

๐‘ฅ๐‘–โˆˆ๐‘๐‘๐‘˜

OUT-OF-SAMPLE EXTENSION

A linear transformation Ae is learnt s.t Y = AeZ

Embedding for new point y* = Aez*

๏ฟฝฬ‚๏ฟฝ๐‘โˆˆ๐‘ ๐‘ฆ ๐‘โˆˆ๐‘ŒAe

z* y*

OUT-OF-SAMPLE RECONSTRUCTION

A linear transformation Ar is learnt s.t Z = ArY

Projection of reconstruction on tangent plane z* = Ary*

๏ฟฝฬ‚๏ฟฝ๐‘โˆˆ๐‘ ๐‘ฆ ๐‘โˆˆ๐‘Œ

z* y*Ar

PRINCIPAL COMPONENTS ANALYSIS

Covariance matrix of neighborhood:

Let be the eigenvector and eigenvalue matrixes of Mk

Then

Denote then the projection of a point x onto the tangent plane is given by:

LINEAR TRANSFORMATION

Y and Z are both centered around and Then Ae =BeRe where Be and Re are scale and

rotation matrices respectively If is the singular value decomposition

of ZTY, then

FINAL ESTIMATES

ERROR ANALYSIS

We donโ€™t know the true form of f or g to compare our estimates against

Reconstruction Error: For a new point x* its reconstruction is computed as , and the reconstruction error is

SAMPLING DENSITY

To show: As the sampling density of points on the manifold increases, reconstruction error of a new point goes to 0

In a k-NN framework, the sampling density can increase in two ways: k remains fixed and the sampling width

decereases remains fixed and

We consider the second case

NEIGHBORHOOD PARAMETERIZATION

Assume that the first m-canonical vectors of are along

RECONSTRUCTION ERROR

But ArAe = I, hence

RECONSTRUCTION ERROR

Tyagi, Vural and Frossard (2012) derive conditions on k s.t the angle between and is bounded

They show that as

Equivalently, where Rm is an aribitrary m-dimensional rotation matrix

and

RECONSTRUCTION ERROR

Hence the reconstruction approaches the projection of x* onto

SMOOTHNESS OF MANIFOLD

If the manifold is smooth then all will be smooth

Taylor series of :

As because x* will move closer to p

RESULTS - EXTENSION

Out of sample extension on the Swiss-Roll dataset

Neighborhood size = 10

RESULTS - EXTENSION

Out of sample extension on the Japanese flag dataset

Neighborhood size = 10

RESULTS - RECONSTRUCTION

Reconstructions of ISOMAP faces dataset (698 images)

n = 4096, m = 3 Neighborhood size = 8

RECONSTRUCTION ERROR V NUMBER OF POINTS ON MANIFOLD

ISOMAP Faces dataset Number of cross validation sets = 5 Neighborhood size = [6, 7, 8, 9]