High-resolution Hyperspectral Imaging via Matrix Factorization

Rei Kawakami1 John Wright2 Yu-Wing Tai3 Yasuyuki Matsushita2 Moshe Ben-Ezra2 Katsushi Ikeuchi3

1University of Tokyo, 2Microsoft Research Asia (MSRA),

3Korea Advanced Institute of Science and Technology (KAIST)

CVPR 11

Giga-pixel Camera

M. Ezra et al.Giga-pixel Camera

@ Microsoft research

Large-format lens CCD

Spectral cameras

LCTF filter35,000 $

Hyper-spectral camera55,000 $

Line spectral scanner25,000 $

• Expensive• Low resolution

Approach

Low-reshyperspectral

high-resRGB

High-reshyperspectral image

Combine

Problem formulation

W(Image width)

H(Image height)

S

Goal:

Given:

A1: Limited number of materials

•

Sparse vector

For all pixel (i,j)

Sparse matrix

W (Image width)

H (Image height)

S

= …

00.40…

0.6

Sampling of each camera

• Low-res camera • RGB camera

Spectrum

Wavelength

Intensity

Sparse signal recovery

• •

Filter SignalObservation

t

tm

n

Sparsity

Signal Basis Weights

Signal Basis Weights

0

S

S-sparse

Sparse signal recovery

Observation

Need not to know which bases are important

Sparsity and Incoherence matters

A2: Sparsity in high-res image

W (Image width)

H (Image height)

S

Sparse coefficients

Sparse vector

Simulation experiments

460 nm 550 nm 620 nm 460 nm 550 nm 620 nm

430 nm 490 nm 550 nm 610 nm 670 nm

Error images of Global PCA with back-projection

Error images of local window with back-projection

Error images of RGB clustering with back-projection

Estimated430 nm

Groundtruth

RGBimage

Errorimage

HS camera

Filter

CMOSLensAperture

Focus

Translational stage

Real data experiment

Input RGB Input (550nm) Input (620nm)Estimated (550nm) Estimated (620nm)

Summary

• Method to reconstruct high-resolution hyperspectral image from – Low-res hyperspectral camera– High-res RGB camera

• Spatial sparsity of hyperspectral input– Search for a factorization of the input into

• basis • set of maximally sparse coefficients.