Hyperspectral Unmixing: Geometrical, Statistical,

and

Sparse Regression-Based Approaches

José M. Bioucas-Dias

Instituto de Telecomunicações

Instituto Superior Técnico

Universidade de Lisboa

Portugal

SUPELEC, SONDRA - 2014

Hyperspectral imaging (and mixing)

c:

o ----·

Mixed pixel

( oil + rocks)

Pure pi.xel

(water)

4 ----------------

3000 J /'-

j::i: -r --

300 eoo 900 t 200 1500 ·aoc 210024DO

Wa\'elength (om) u 4000

c<J

5 l000 <J

2..0.00

c:: 1000

0 ... ' t

300 eoo soo ·200 1500 ·aoo 2HlO 21&00

Wa\·el ngth (nm)

Mixed pi.xel

(·'egetation + soil)

5000

... 4000 Col

5 30!)() Col

4.:.0. 00

c:: 1000

lOO eoo 900 1200 1500 taoo 2100 2400

Wavelengtb (om)

Hyperspectral unmixing

AVIRIS of Cuprite,

Nevada, USA

R – ch. 183 (2.10 m)

G – ch. 193 (2.20 m)

B – ch. 207 (2.34 m)

Alunite Kaolinite Kaolinite #2

VCA [Nascimento, B-D, 2005]

Outline

Mixing models

Linear

Nonlinear

Signal subspace identification

Unmixing

Geometrical-based

Statistical-based

Sparse regression-based

J. M. Bioucas-Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot,

"Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches",

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,

vol. 5, no. 2, pp. 354-379, 2012.

W.-K. Ma, J. Bioucas-Dias, T.-H. Chan, N. Gillis, P. Gader, A. Plaza, A. Ambikapathi and C.-Y. Chi,

"A signal processing perspective on hyperspectral unmixing", IEEE Signal Processing Magazine, Jan.,

2014.

4

Linear mixing model (LMM)

6 i .

7

2

m = 6

6 4

½1i 3

½2i 7

7 5

½Li

Incident radiation interacts only

with one component

(checkerboard type scenes)

Hyperspectral linear

unmixing Estimate

5

Nonlinear mixing model

Intimate mixture (particulate media) Two-layers: canopies+ground

Radiative transfer theory

material fractions media parameters single scattering double scattering

6

CUHK - 2012 7

Schematic view of the unmixing process

Spectral linear unmixing (SLU)

Given N spectral vectors of dimension L:

Subject to the LMM:

ANC: abundance

nonnegative

constraint

Determine:

The mixing matrix M (endmember spectra)

The fractional abundance vectors

ASC: abundance

sum-to-one

constraint

SLU is a blind source separation problem (BSS)

8

Subspace identification

Problem: Identify

the subspace generated by the columns of

Reasoning underlying DR

1. Lightens the computational

complexity

2. Attenuates the noise power

by a factor of

Subspace identification

11

Subspace identification algorithms

refe

cta

nce

Exact ML solution [Scharf, 91] (known p, i.i.d. Gaussian noise)

PCA - Principal component analysis (unknown p, i.i.d. noise)

NAPC - Noise adjusted principal components [Lee et al., 90]

MNF - Maximum noise fraction [Green et al., 88]

HFC - Harsanyi-Farrand-Chang [Harsanyi et al., 93]

NWHFC - [Chang, Du, 94]

HySime - Hyperspectral signal identification by minimum error [B-D, Nascimento, 08]

Hypothesis testing and random matrix theory [Kritchman, Nadler, 2009],

[Cawse, Robin, Sears, 11]

1

0.8

x

y

yA Example (HySime)

0.6

0.4

0.2

0 0 50 100 150 200 250

band 12

Unsupervised unmixing frameworks

Geometrical

Exploits parallelisms between the linear mixing

model and properties of convex sets

Statistical

Approaches linear unmixing as a statistical

inference problem

Sparse regression

Approaches linear unmixing as a sparse

regression problem

13

Minimum volume simplex

Methodology: Find the simplex of

minimum volume containing the data ([Craig, 1990, 1994])

Equivalent formulation: find such that

is minimized and all data points are inside the

simplex defined by the columns of

14

Minimum volume simplex (MVS)

Pure pixels No pure pixels No pure pixels

MVS works MVS works MVS does not work

15

Minimum volume simplex algorithms

Hard assumption

The data set contains at least one pure pixel of each material

Search endmembers in the data set

Computationally lighter

PPI - [Boardman, 93]; N-FINDR - [Winter, 99]; IEA - [Neville et al., 99];

AMEE – [Plaza et al, 02]; SMACC – [Gruninger et al., 04]

VCA - [Nascimento, B-D, 03, 05]; SGA - [Chang et al., 06]

AVMAX, SVMAX - [Chan, Ma, Ambikapathi, Chi, 2011];

RNMF- [Gillis, 2013];

SD-SOMP, SD-ReOMP - [Fu, Ma, Chan, B-D, Iordche, 2013];

14

Pure pixel based algorithms

N-FINDR

AVMAX

VCA

SVMAX

iteratively increase Idet( m1, ... , mp) 1

rn 1, . . . ,mp E {y1, ... ,y N}

max

st: m1, ... , mp E conv{y1, ... , YN}

for k == 1 : p

rn:== argmax II(Pifç)TYi li Yi

M :== [M, rn]

for k == 1 : p

rn:== argmax II(Pifyi ll Yi

M :== [M, rn]

15

Unmixing example

HYDICE sensor M E 21o x 6 N == 500 x 307 resolution == O.75m

• Data

0 N-FINDR

VCA

16

Minimum volume simplex algorithms

Minimum-volume constrained nonnegative matrix

factorization (MVC-NMF) (inspired by NMF [Lee, Seung, 01])

volume regularizer

DPFT - [Craig, 90]; CCA - [Perczel et al., 89] (seminal works on MVC)

ICE - [Breman et al., 2004] ( );

MVC-NMF - [Miao,Qi, 07] ( );

SPICE - [Zare, Gader, 2007] ( )

L1/2 - NMF - [Qian, Jia, Zhou, Robles-Kelly, 11] ( )

CoNMF - [Li, B-D, Plaza, 12] ( )

17

Minimum volume simplex algorithms

MVSA – Minimum volume simplex analysis [Li, B-D, 08]

Optimization variable

ASC ANC

MVSA solves a sequence of quadratic programs

MVES [Chan, Chi, Huang, Ma, 2009]

Solves a sequence of linear programs by exploiting the cofactor

expansion of det (Q)

The existence of pure pixels is a sufficient condition for exact

identification of the true endmembers

18

Robust minimum volume simplex algorithms: outliers

SISAL – Simplex identification via split augmented Lagrangian [B-D,09]

ASC

soft ANC

SISAL solves a sequence of convex subproblems using ADMM

19

Robust minimum volume simplex algorithms: noise

RMVES – Robust MVES [Ambikapathi, Chan, Ma, Chi, 12]

ANC - chance constraint

probit function

Solves a sequence of quadratic programs

exploiting the cofactor expansion of det(Q)

20

Example: data set contains pure pixels

a)

Y(1

202,:

)

0.8

data points 1

true sisal

0.7

0.6

0.5

0.4

0.3

0.2

0.1

estimated

true

initial (vc

(sisal)

0.8

0.6

0.4

0.2

0

vca

0 0.2 0.4 0.6 0.8 Y(100,:)

0 50 100 150 200 250

Time:

VCA 0.5 sec

SISAL 2 sec

IWMSDF, SYSU- 2014 21

Example: Data set does not contain pure pixels

(mvca)

Y(1

202,:

)

0.8

0.7

0.6

0.5

data points

estimated - (sisal)

true

initial - (vca)

1

0.8

0.6

true sisal vca

0.4 0.4

0.3

0.2

0.1 0 0.2 0.4 0.6 0.8

Y(100,:)

0.2

0 0 50 100 150 200 250

22

No pure pixels and outliers

v2'*

Y

4

3.5

3

2.5

2

1.5

1

Endmembers and data points (2D projection)

data points

true

SISAL

VCA

NFINDR

MVC

ERROR(mse):

SISAL = 0.03

VCA = 0.88

NFINDR = 0.88

MVC-NMF = 0.90

TIMES (sec):

SISAL = 0.61,

VCA = 0.20

NFINDR = 0.25

MVC-NMF = 25

0.5 -6 -5 -4 -3 -2 -1 0

v1'*Y

23

Real data: ArtImageDataA (converted into absorbance)

oil

Tiles 1,2 (‘Prussian blue’ + oil) 2

Tiles 15,16 (‘Ultramarine blue’ + oil) 1

Projection on [e , e ] 1 2

0 Estimate endmember signatures by SISAL

2

-1

U. blue

1.5 -2

1 -3 -20 -15 -10 -5 0

0.5

0

P. blue

SISAL

solution

-0.5 0 50 100 150 200

24

Geometrical Approaches: Limitations

PPI, N FINDR VCA, SGA AMEE, SVMAX AVMAX

depend on the existence of pure pixels in the data

Do not work in highly mixed scenarios

v2'*

Y

Limitations of geometrical methods Inference determined by a small number of pixels;

some may be outliers

MVS – Computationally heavy

- , , ,

3.5

3

2.5

Endmembers and data points (2D projection)

data points

true

SISAL

VCA

NFINDR

MVC-NMF

2

25 1.5

Statistical approaches

observation model

prior (Bayesian framework)

posterior density

inference

26

Spectral linear unmixing and ICA/IFA

Formally, SLU is a linear source separation problem

Independent Component Analysis (ICA) come to mind

ICA Fastica, [Hyvarinen & Oja, 2000]

Jade, [Cardoso, 1997]

Bell and Sejnowski, [Bell and Sejnowski , 1995]

IFA IFA, [Moulines et al., 1997], [Attias, 1999]

27

Statistical approaches: ICA

Assumptions

1. Fractional abundances (sources) are independent

2. Non-Gaussian sources

Endmembers compete for the same area

Sources are Dependent

ICA does not apply [Nascimento, B-D, 2005]

IWMSDF, SYSU- 2014 28

Representative Bayesian approaches

[Parra et al., 00]

[Moussaoui et al., 06,a,b], [Dobigeon et al., 09,a,b], [Dobigeon et al., 09,b],

[Arngreen, 11]

data term associated with the LMM

Uniform on the simplex conjugate prior distributions for some unknown parameters

Infer MMSE estimates by Markov chain Monte Carlo algorithms

DECA - [Nascimento, B-D 09, 12]

data term associated with a noiseless LMM

Dirichlet mixture model

MDL based inference of the number of Dirichlet modes

MAP inference (GEM - algorithm)

29

DECA – Dependent component analysis

parameters of the Dirichlet mixture

simulated data

minimum volume term

30

DECA – Results on Cuprite

[Mittelman, Dobigeon, Hero, 12]

data term associated with the LMM

Latent label process enforcing

adjacent pixels to have the same label

spatial prior: tree-structured sticky

hierarchical Dirichlet process (SHDP)

MMSE inference by MCMC

Model order inference

(number of endmembers)

Sparse regression-based SLU

Spectral vectors can be expressed as linear combinations

of a few pure spectral signatures obtained from a

(potentially very large) spectral library

[Iordache, B-D, Plaza, 11, 12]

0

0

0 0

0

0

Unmixing: given y and A, find the sparsest solution of

Advantage: sidesteps endmember estimation

Disadvantage: Combinatorial problem !!!

32

Sparse regression-based SLU

Problem – P0

(library, , undetermined system)

Very difficult (NP-hard)

Approximations to P0:

OMP – orthogonal matching pursuit [Pati et al., 2003]

BP – basis pursuit [Chen et al., 2003]

BPDN – basis pursuit denoising

IHT (see [Blumensath, Davis, 11], [Kyrillidis, Cevher, 12])

33

Convex approximations to P0

CBPDN – Constrained basis pursuit denoising

Equivalent problem

Striking result: In given circumstances, related with the

coherence among the columns of matrix A, BP(DN)

yields the sparsest solution ([Donoho 06], [Candès et al. 06]).

Efficient solvers for CBPDN: SUNSAL, CSUNSAL [B-D, Figueiredo, 10]

35

Real data- AVIRIS Cuprite

50

100

150

.200

250

300

"

SELECTED "'- ... • IMAGE [ \t'

[Iordache, B-D, Plaza, 11, 12] 35

Real data- AVIRIS Cuprite

200 400 600 800 1000 1200 1400 1600 1800 2000

[Iordache, B-D, Plaza, 11, 12] 36

Sparse reconstruction of hyperspectral data: Summary

Bad news: Hyperspectral libraries have poor RI constants

Good news: Hyperspectral mixtures are highly sparse, very often p · 5

Surprising fact: Convex programs (BP, BPDN, LASSO, …) yield much

better empirical performance than non-convex state-of-the-art

competitors

Directions to improve hyperspectral sparse reconstruction

Structured sparsity + subspace structure

(pixels in a give data set share the same support)

Spatial contextual information

(pixels belong to an image)

37

Constrained total variation sparse regression (CTVSR)

[Iordache, B-D, Plaza, 11]

Total Variation of

i-th band

Related work [Zhao, Wang, Huang, Ng, Plemmons, 12]

Other Regularizers:

vector total variation (VTV)! promotes piecewise smoo vectors [Bresson, Chan, 02], [Goldluecke et al., 12], [Yuan, Zhang, Shen, 12]

convex generalizations of Total Variation based on the Structure Tensor [Lefkimmiatis et al., 13]

sparse representation (2D, 3D) in the wavelet domain

38

Ilustrative examples with simulated data : SUnSAL-TV

Original data cube

Original abundance of EM5

SUnSAL estimate

SUnSAL-TV estimate

41

Constrained colaborative sparse regression (CCSR)

0 0

0 0

0 0

0 0

0 0

0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 0

0

0

0 0

0 0

0 0

0 0

0 0

0 0

0

0

0

0

0

0 0 0

0 0 0 0

0 0 0 0

0

[Iordache, B-D, Plaza, 11, 12] [Turlach, Venables, Wright, 2004]

=

multiple measurements

share the same support

Theoretical guaranties (superiority of multichanel) : the probability of recovery

failure decays exponentially in the number of channels. [Eldar, Rauhut, 11]

40

Ilustrative examples with simulated data : CSUnSAL

Original x

50

Without collaborative sparseness

50

With collaborative sparseness

50

100 100 100

150 150 150

200 200 200

250 250 250

300 300 300

20 40 60 80 100

20 40 60 80 100

43 20 40 60 80 100

MUSIC – Colaborative SR algorithm

MUSIC-CSR algorithm [Iordahe, B-D, Plaza, 2013]

1) Estimate the signal subspace using, e.g.

the HySime algorithm.

2) Compute and define

the index set

3) Solve the colaborative sparse regression optimization

[B-D, Figueiredo, 2012]

Related work: CS-MUSIC

(N < k and iid noise)

[Kim, Lee, Ye, 2012]

44

MUSIC – CSR results

A – USGS , Gaussian shaped noise, SNR = 25 dB, k = 5, m = 300,

acummulated

abundances

50

100

150

200

250

300

500 1000 1500 2000 2500 3000 3500 4000 4500 5000

MUSIC-CSR SNR = 11.7 dB

computation time ' 10 sec true

endmembers

CSR SNR = 0 dB

computation time ' 600 sec 45

Brief Concluding remarks

HU is a hard inverse problem (noise, bad-conditioned direct

operators, nonlinear mixing phenomena)

HU calls for sophisticated math tools and framework (statistical

inference, optimization, machine learning)

The research efforts devoted to non-linear mixing models are

increasing

Linear mixing

Apply geometrical approaches when there are data vectors near or

over the simplex facets

Apply statistical methods in highly mixed data sets

Apply sparse regression methods, if there exits a spectral library

for the problem in hand

44

Spectral nonlinear unmixing (SNLU). Just a few topics

Detecting nonlinear mixtures in polynomial post-nonlinear mixing

model, [Altmann, Dobigeon, Tourneret, 11,13]

hypothesis test

Bilinear unmixing model, [Fan, Hu, Miller, Li, 09], [Nascimento, B-D, 09],

[Halimi, Altmann, Dobigeon, Tourneret, 11,11]

Kernel-based unmixing algorithms to specifically account for intimate

mixtures [Broadwater, Chellappa, Burlina, 07], [Broadwater, Banerjee, 09,10, 11], [Chen,

Richard, Ferrari, Honeine, 13]

N. Dobigeon, J.-Y. Tourneret, C. Richard, J. C. M. Bermudez, S. McLaughlin and A. O. Hero, "Nonlinear

unmixing of hyperspectral images: models and algorithms," IEEE Signal Process. Magazine, vol. 31, no 1,

pp. 82-94, 2014

R. Heylen, M.Parente, and P. Gader, "A review of nonlinear hyperspectral unmixing methods,"

Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of , vol.7, no.6,

pp.1844-1868, 2014 45