Enhancing Tensor Subspace
Learning by Element Rearrangement
1
Dong XU
School of Computer Engineering
Nanyang Technological Universityhttp://www.ntu.edu.sg/home/dongxu
Outline
• Summary of our recent works on Tensor (or Bilinear) Subspace Learning
• Element Rearrangement for Tensor (or Bilinear) Subspace Learning
What is Tensor?
Tensors are arrays of numbers which transform in certain ways under coordinate transformations.
1m
2m3m
1m
Vector Matrix 3rd-order Tensor
2m
1m
1 2 3m m m X R1 2m mX R1mxR
Bilinear (or 2D) Subspace Learning: each image is represented as a 2nd-order tensor (i.e., a matrix)
Tensor Subspace Learning (more general case): each image is represented as a higher order tensor
2
1
m
ij ik kjk
Y X U
100
100
100
100
100
100
.
.
.
100
10
100
10
100
10
100
10
100
10
100
10
=
=
=
.
.
.
Definition of Mode-k Product
(100)
2 )'(10m2 (100)m
2m(100)
1m(100)
1m
2 )'(10m
1m
2 (100)m
1(100)m
3 (40)m
2 )'(10m
2 (100)m
k U Y XNotation:
Product for two Matrices
Original Matrix
New Matrix
= 1(100)m
3 (40)m
2 )'(10m
Y XUProjection
Matrix
Original Tensor
New Tensor
Projection Matrix
Projection: high-dimensional space -> low-dimensional space
Reconstruction: low-dimensional space -> high-dimensional space
Definition of Mode-k Flattening
...1(100)m
2 3 (100*40)m m
Tensor Matrix
1m
2 (100)m
1(100)m
3 (40)m
Potential Assumption in Previous Tensor-based Subspace Learning:
Intra-tensor correlations: Correlations along column vectors of mode-k flattened matrices.
Data Representation in Dimensionality Reduction
Vector Matrix 3rd-order Tensor
High Dimension
Low Dimension
Examples
PCA, LDA Rank-1 Decomposition, 2001
A. Shashua and A. Levin,Tensorface, 2002
M. Vasilescu andD. Terzopoulos,
Our WorkXu et al., 2005
Yan et al., 2005…
...
...
...
Filtered Image Video SequenceGray-level Image
What is Gabor Features?Gabor features can improve recognition performance in comparison to grayscale features. Chengjun Liu T-IP, 2002
Gabor Wavelet Kernels
Eight Orientations
Five
Scales
Input: Grayscale
Image Output: 40 Gabor-filtered
Images
…
Why Represent Image Objects as Tensors instead of Vectors?
Natural Representation Gray-level Images (2D structure) Videos (3D structure) Gabor-filtered Images (3D structure)
Enhance Learnability in Real Application Curse of Dimensionality (Gabor-filtered image: 100*100*40 -> Vector: 400,000)
Small sample size problem (less than 5,000 images in common face databases)
Reduce Computation Cost
... ...
Concurrent Subspace Analysis (CSA) as an Example(Criterion: Optimal Reconstruction)
1m
100
40
100
1U
2U
The reconstructed sample
Input sample
Projection Matrices?
Sample in Low- dimensional space
3U
1m
10
10
10
Dimensionality Reduction
1m
100
40
100
Reconstruction
D. Xu, S. Yan, Lei Zhang, H. Zhang et al., CVPR 2005 and T-CSVT 2008 3
1
* 31
21 1 1 3 3 3
|
( | )
arg min || ... ||k k
k k
i iiU
U
U U U U
X X
Objective Function:
Tensorization - New Research Direction: Other Related Works
• Discriminant Analysis with Tensor Representation (DATER): CVPR 2005 and T-IP 2007
• Coupled Kernel-based Subspace Learning (CKDA): CVPR 2005 • Rank-one Projections with Adaptive Margins (RPAM): CVPR 2006 and T-SMC-B
2007• Enhancing Tensor Subspace Learning by Element Rearrangement: CVPR 2007
and T-PAMI 2009• Discriminant Locally Linear Embedding with High Order Tensor Data (DLLE/T):
T-SMC-B 2008• Convergent 2D Subspace Learning with Null Space Analysis (NS2DLDA): T-
CSVT 2008• Semi-supervised Bilinear Subspace Learning: T-IP 2009• Applications in Human Gait Recognition
– CSA+DATER: T-CSVT 2006– Tensor Marginal Fisher Analysis (TMFA): T-IP 2007
Note: Other researchers also published several papers along this direction!!!
Tensorization - New Research Direction Tensorization in Graph Embedding Framework
Direct Graph Embedding
1minT
T
y B yy Ly
Original PCA & LDA,ISOMAP, LLE,
Laplacian Eigenmap
Linearization
PCA, LDA, LPP, MFA
wXy T
Kernelization
KPCA, KDA, KMFA
)( iii xw
Tensorization
CSA, DATER, TMFA
nnii wwwy 2
21
1X
Type
Formulation
Example
S. Yan, D. Xu, H. Zhang et al., CVPR, 2005 and T-PAMI,2007
Google Citation: 174 (until 15-Sep-2009)
Element Rearrangement: Motivations
• The success of tensor-based subspace learning relies on the redundancy among the unfolded vector
• However, such correlation/redundancy is usually not strong for real data
• Our Solution: Element rearrangement is employed as a preprocessing step to increase the intra-tensor correlations for existing tensor subspace learning methods
12
Motivations-Continued
Sets of highlycorrelated pixels
Columns of highlycorrelated pixels
Element Rearrangement
Low correlation High correlation
Intra-tensor correlations: Correlations among the features within certain tensor dimensions, such as rows, columns and Gabor features…
Problem Definition
• The task of enhancing correlation/redundancy among 2nd–order tensor is to search for a pixel rearrangement operator R, such that
14
* 2
,1
arg min{ min || || }N
R T R Ti iR U V
i
R X UU X VV
1. is the rearranged matrix from sample2. The column numbers of U and V are predefined
iXRiX
After the pixel rearrangement, we can use the rearranged tensors as input for tensor subspace learning algorithms!
Solution to Pixel Rearrangement Problem
15
Compute reconstructed matrices
1, 1 1 1 1
nRRec T Ti n n n i n nX U U X V V
Optimize operator R
2,
1
arg min || ||N
R Recn i i nR
i
R X X
Optimize U and V
n=n+1
Initialize U0, V0
2
,1
( , ) arg min || ||n n
NR RT T
n n i iU Vi
U V X UU X VV
2 2, 1 1 1 1
1 1
: || || || ||n n n
N NR R RRec T Ti i n i n n i n n
i i
Note X X X U U X V V
• It is Integer Programming problem
Step for Optimizing R
16
* 2,
1
arg min || ||N
R Reci i nR
i
R X X
,
min .
1: 0 1; 2 : 1; 3 : 1
pq pqR
p q
pq pq pqp q
c R st
R R R
2,
1
| ( ) ( ) |N
Recpq i i n
i
where c X p X q
1. Linear programming problem in Earth Mover’s Distance (EMD) has integer solution.
2. We constrain the rearrangement within spatially local neighborhood or feature-based neighborhood for speedup.
p
Original matrix
Reconstructed matrix
q
pqc
Sender
Receiver
Convergence Speed
17
Rearrangement Results
18
Reconstruction Visualization
19
Reconstruction Visualization
20
Classification Accuracy
Summary
.
• Our papers published in CVPR 2005 are the first works to address dimensionality reduction with the image objects represented as high-order tensors of arbitrary order.
• Our papers published in CVPR 2005 opens a new research direction. We also published a series of works along this direction.
• Element arrangement can further improve data compression performance and classification accuracy.
Top Related