High-resolution Hyperspectral Imaging via Matrix Factorization
Rei Kawakami1 John Wright2 Yu-Wing Tai3 Yasuyuki Matsushita2 Moshe Ben-Ezra2 Katsushi Ikeuchi3
1University of Tokyo, 2Microsoft Research Asia (MSRA),
3Korea Advanced Institute of Science and Technology (KAIST)
CVPR 11
Giga-pixel Camera
M. Ezra et al.Giga-pixel Camera
@ Microsoft research
Large-format lens CCD
Spectral cameras
LCTF filter35,000 $
Hyper-spectral camera55,000 $
Line spectral scanner25,000 $
• Expensive• Low resolution
Approach
Low-reshyperspectral
high-resRGB
High-reshyperspectral image
Combine
Problem formulation
W(Image width)
H(Image height)
S
Goal:
Given:
A1: Limited number of materials
•
Sparse vector
For all pixel (i,j)
Sparse matrix
W (Image width)
H (Image height)
S
= …
00.40…
0.6
Sampling of each camera
• Low-res camera • RGB camera
Spectrum
Wavelength
Intensity
Sparse signal recovery
• •
Filter SignalObservation
t
tm
n
Sparsity
Signal Basis Weights
Signal Basis Weights
0
S
S-sparse
Sparse signal recovery
Observation
Need not to know which bases are important
Sparsity and Incoherence matters
A2: Sparsity in high-res image
W (Image width)
H (Image height)
S
Sparse coefficients
Sparse vector
Simulation experiments
460 nm 550 nm 620 nm 460 nm 550 nm 620 nm
430 nm 490 nm 550 nm 610 nm 670 nm
Error images of Global PCA with back-projection
Error images of local window with back-projection
Error images of RGB clustering with back-projection
Estimated430 nm
Groundtruth
RGBimage
Errorimage
HS camera
Filter
CMOSLensAperture
Focus
Translational stage
Real data experiment
Input RGB Input (550nm) Input (620nm)Estimated (550nm) Estimated (620nm)
Summary
• Method to reconstruct high-resolution hyperspectral image from – Low-res hyperspectral camera– High-res RGB camera
• Spatial sparsity of hyperspectral input– Search for a factorization of the input into
• basis • set of maximally sparse coefficients.
Top Related