Residual Component Analysis of Hyperspectral Images for ...RCA of Hyperspectral Images for Joint...

RCA of Hyperspectral Images for Joint Nonlinear Unmixing and Nonlinearity Detection

Residual Component Analysis ofHyperspectral Images for Joint NonlinearUnmixing and Nonlinearity Detection

Steve McLaughlin(1)

Joint work with Y. Altmann(1), N. Dobigeon(2), J.-Y. Tourneret(2) and M.Pereyra(3)

(1)Heriot-Watt University – School of Engineering of Physical Sciences, U.K.(2)University of Toulouse – IRIT/INP-ENSEEIHT/Tı¿ 1

2 SA, France(2)University of Bristol – School of Mathematics,U.K.

MAHI 2014 workshop, Nice, 15/12/2014

1 /44


Spectral unmixing

Features of interest

I pure spectral signatures:endmembers

I proportions of eachcomponent: abundances

Unsupervised unmixing

I Endmembers andabundances unknown

I Blind source separation

Supervised unmixing

I Endmembers assumed to beknown

I Abundance estimation:inverse problem

2 /44


Linear spectral unmixing

Linear mixing model (LMM)

yn =

R∑r=1

ar,nmr + en = Man + en

I yn nth observed pixel in L bandsI M = [m1, . . . ,mR] endmember matrix containing the spectra of

the R pure components of the sceneI an = [a1,n, . . . , aR,n]T abundance vector of the nth pixel

Abundance constraints

Abundance positivity: ar,n ≥ 0, r = 1, . . . , R n = 1, . . . , N.

Abundance sum-to-one:∑Rr=1 ar,n = 1 n = 1, . . . , N.

3 /44


Nonlinear spectral unmixing

yn = g (M,an) + en

I Possible interactions between the components of the scene

I Nonlinear terms included in the mixing model

I Several models depending on the nature of the scene (intimatemixtures, bilinear models, kernel-based models,...)

4 /44



Endmember variability

I Changes in observation conditionsI Intrinsic variability of the materials

Endmember Spectra measured using a handheld ASD spectrometer: Asphalt

(blue), Yellow Curb (black), Grass (red), Oak Leaves (green). (source: Master

Thesis of Xiaoxiao Du, University of Missouri-Columbia).

5 /44



Nonlinear unmixing to account for general deviations from the LMM

I Deformation of the classical data simplex

PCA-based representation of the real Villelongue image pixels.

6 /44


Outline

Kernel-based mixing models

Residual Component AnalysisBayesian modelResults

Generalised RCABayesian modelResults

Conclusion

7 /44


RCA-based model

Outline




Conclusion

8 /44


RCA-based model


Motivations

I Various sources of nonlinear effects

I Several possible parametric models to be considered (modelselection,. . . )

I Kernel-based models general and flexible tools to handle differentnonlinearities

I Often rely on few hyperparameters

9 /44


RCA-based model


Two main model classes

I Discriminative models (RBF1, Kernel-FCLS2,...)

an ≈ f(M,yn)

I Generative modelsyn ≈ f(M,an)

⇒ Generative models related to the actual acquisition process, moreintuitive

1Y. Altmann et al., “Non- linear unmixing of hyperspectral images using radial basis functions

and orthogonal least squares,” in Proc. IEEE IGARSS , Vancouver, Canada, 2011.2J. Broadwater et al., “Kernel fully constrained least squares abundance estimates,” in Proc.

IEEE IGARSS, Barcelona, Spain, 2007.

10 /44


RCA-based model

Gaussian Processes for SU

Unsupervised unmixing problem

yn ≈ f(M,an)

I Blind nonlinear source separation problem: Estimation of M,anand f(·) unknown

I Challenging problem (even if f(·) is known)

Idea: Modeling f(·) by a Gaussian Process (GP)

11 /44


RCA-based model


First Idea

I Consider a set of N pixels observed at L spectral bands gatheredin Y

I Abundance matrix: A

I Vectorized observation matrix Y: (NL× 1 vector)

Y: ∼ N (µM,A,KM,A)

I KM,A models correlations between pixels (through A) andspectral bands (through M)

⇒ Computationally intractable without particular structure for KM,A

Solution: Consideration of correlated pixels or bands only

12 /44


RCA-based model


Second Idea: GP-LVM

I L spectral bands assumed to be i.i.d.

y:,` ∼ N (µA,KA) , ∀`

I M not explicit in the model

I Choice for KA: Gaussian Kernel3, Polynomial Kernel4

I Kernels possibly too general: regularization required (LLE,ISOMAP,. . . )

I More promising results obtained with the polynomial kernel(more suited for bilinear mixtures)

3Y. Altmann et al.,“Nonlinear unmixing of hyperspectral images using Gaussian processes,” in

Proc. ICASSP, Kyoto, Japan, 2012.4Y. Altmann et al., “Nonlinear spectral unmixing of hyperspectral images using Gaussian

processes,” IEEE Trans. Signal Processing, May 2013.

13 /44


RCA-based model


Conclusion on GP-based unsupervised unmixing

I GPs flexible enough to handle correlations between pixels andbetween spectral bands

I Interesting results although the problem is highly ill-posed

I Model complexity possible prohibitive for large data sets

I Results highly dependent on the kernel choice (withoutadditional regularization)

I Limited to one mixing model per image...

⇒ GPs more adapted for more constrained problems, e.g., supervisedunmixing

14 /44


RCA-based model

Outline




Conclusion

15 /44


RCA-based model

Residual Component Analysis (RCA)

RCA-based model

yn = Man + φn + en, n = 1, . . . , N

I φn: additive nonlinear effects in the nth pixel

I en ∼ N (0,Σ0) with Σ0 diagonal

Contraints

ar,n ≥ 0, ∀r, n and

(R∑r=1

ar,n = 1 ∀n

)

16 /44


RCA-based model


On the sum-to-one constraint

I Widely acknowledged for linear SU

I Accurate when illumination changes occur?

I Abundance interpretation in nonlinear mixture?I Relief: projected areas?I Multilayer scene: visible surfaces, volume?

⇒ Both cases investigated: with/without enforced sum-to-one

17 /44


RCA-based model


General nonlinear model

yn = Man + φn + en, n = 1, . . . , N

I φn = 0→ LMM

I φn =∑Ri=1

∑Rj=1 βi,j,nmi �mj → polynomial models5 6 7

(� denotes the Hadamard (termwise) product)

I φn = ψ (M)→ K-Hype8

I ...

5J. M. P. Nascimento and J. M. Bioucas-Dias, “Nonlinear mixture model for hyperspectral

unmixing,” Proc. SPIE Image and Signal Processing for Remote Sensing XV, 2009.6A. Halimi et al., “Nonlinear unmixing of hyperspectral images using a generalized bilinear

model,” IEEE Trans. Geoscience and Remote Sensing, vol. 49, no. 11, pp. 4153-4162, Nov. 2011.7Y. Altmann et al., “Supervised nonlinear spectral unmixing using a post-nonlinear mixing model

for hyperspectral imagery,” IEEE Trans. Image Process., 2012.8J. Chen et al., “Nonlinear unmixing of hyperspectral data based on a

linear-mixture/nonlinear-fluctuation model,” IEEE Trans. Signal Process., 2013.

18 /44


RCA-based model

Bayesian model

Modeling nonlinearities

Nonlinearity-based image partition

I K classes {Ck}k=0,...,(K−1) corresponding to K levels ofnonlinearity

I zn: class label of the nth pixel

I yn ∈ Ck ⇔ zn = k

Label priorPotts Markov model for z = [z1, . . . , zN ]T

f(z) =1

G(β)exp[β

∑Nn=1

∑n′∈V(n) δ(zn−zn′ )]

I Spatial dependencies between nonlinearity levels

I β > 0: fixed granularity parameter

19 /44


RCA-based model

Bayesian model

Modeling nonlinearities

Linearly mixed pixels (k = 0)

f(φn|zn = 0) =

L∏`=1

δ(φ`,n)

Nonlinearly mixed pixels (k 6= 0)

φn|M, zn = k, s2k ∼ N

(0L, s

2kKM

)I [KM]i,j =

(mTi,:mj,:

)2, i, j ∈ {1, . . . , L}

I s2k: controls the nonlinearity level of the kth class→ included in the Bayesian model: f(s2

k) non-informativeinverse-gamma distribution

I Tractable marginalization of φn

20 /44


RCA-based model

Bayesian model

Bayesian model

Other model elements

I Likelihood: Gaussian distribution (zero-mean coloured noise)

I Other model parameters/hyperparameters: classical priordistributions

Joint posterior distributionAfter marginalization of Φ

f(θ|Y,M) ∝ f(Y|M,θ)f(θ)

with θ ={A, z,σ2, s2

}⇒ MCMC method (Metropolis-within-Gibbs) to sample from theposterior distribution...

21 /44


RCA-based model

Results

Real data

Data set

I 180× 250 pixels image composed of R = 5 endmembers (L = 160)

I Endmembers extracted from the data using VCA

I K = 5 levels of nonlinearity, β = 1.6

22 /44


RCA-based model

Results

Real data

Estimated Abundances (with sum-to-one constraint)

23 /44


RCA-based model

Results

Real data

Estimated Abundances (without sum-to-one constraint)

Sum of the estimated abundances for each pixel of the image.

I Abundance maps similar to those obtained with sum-to-oneconstraint

I Sums of the estimated abundance “naturally” close to 1

24 /44


RCA-based model

Results

Real data

Nonlinearity detection

25 /44


RCA-based model

Results

Residual component analysis model

Summary

I RCA: flexible nonlinear model able to approximate differentmixing models

I Joint nonlinearity detection and abundance estimation

I Spatially correlated nonlinearities

I Image segmentation based on nonlinearity levels

I marginalisation of the nonlinearity spectra

Limitations

I Estimation of the Potts parameter β: efficiently solved bymarginal maximum likelihood estimation

I Positivity of the interaction spectra? (zero-mean GPs)

I Selecting the number of classes K...

26 /44


RCA-based model

Results

Residual component analysis

Limitations

I Number of nonlinear classes K

I Often depending on the application (possibly infinite number ofclasses)

I Estimation of one nonlinearity level per pixel + spatialcorrelation...

27 /44


Generalised RCA

Outline




Conclusion

28 /44


Generalised RCA

Bayesian model

Generalised Residual Component Analysis (GRCA)

RCA reformulation

yn = Man + φn + en, n = 1, . . . , N

yn = Man + φ(γn) + en, n = 1, . . . , N

φ(γn) =

R−1∑k=1

R∑k′=k+1

γ(k,k′)i,j

√2mk �mk′ +

R∑k=1

γ(k)i,j mk �mk.

I γn = [γ(1,2)n , . . . , γ

(R−1,R)n , γ

(1)n , . . . , γ

(R)n ]T of length

P = R(R+ 1)/2

I γn|s2n ∼ N

(0, s2

nIP)→ RCA

I Nonlinearity positivity: γn|s2n ∼ N(R+)P

(0, s2

nIP)

29 /44


Generalised RCA

Bayesian model

Generalised Residual Component Analysis (GRCA)

Introduction of spatial correlation

{γn|s2

n ∼ N(R+)P(0, s2

nIP)

s2n ∼ IG(α3, α3α4,n).

I α4,n: controls the mean/mode of the prior

I α3: controls the shape of the tails

I Correlation of the {s2n}: α4,n depending on the nonlinearity levels

in neighbouring pixels

⇒ Hidden gamma Markov random field (GMRF): α3 controls thecorrelation level

30 /44


Generalised RCA

Bayesian model

Bayesian model

Other model elements

I Likelihood: Gaussian distribution (zero-mean coloured noise)

I Other model parameters/hyperparameters: classical priordistributions

Bayesian inference

I MCMC method (stochastic gradient MCMC) to sample from theposterior distribution

I GMRF parameter estimation: marginal maximum likelihoodestimation

I More computationally efficient than samplingI Provide accurate unmixing results in practice

I Other model parameters estimated via Monte Carlo integration

31 /44


Generalised RCA

Bayesian model

Bayesian estimation

Parameter estimation

AMMSE = E [A|Y, α3](‖φn‖

2

2

)MMSE

= E[‖φn‖22 |Y, α3

]Bayesian nonlinearity detection

Tn(φn,an) =‖φn‖22∥∥∥yn −Man − φ(t)

n

∥∥∥2

2

Pn = E [Tn(φn,an) > η|Y, α3]

I η: user-defined threshold (η ∈ {2, 3} in practice)I Nonlinearity detection via probability thresholding

32 /44


Generalised RCA

Results

Synthetic data

Data set

I 100× 100 pixels image composed of R = 3 endmembers

I 6 mixing modelsI LMM (without sum-to-one constraint)I LMM (with sum-to-one constraint)I Generalized bilinear model (GBM)I Polynomial post-nonlinear mixing model (PPNMM) (b = 0.2)I Nascimento’s bilinear modelI RCA model (s2 = 0.1)

I SNR ' 30dB

33 /44


Generalised RCA

Results

Synthetic data

34 /44


Generalised RCA

Results

Unmixing performance

Abundance estimation

RNMSEk =

√1

NkR

∑n∈Ik

‖an − an‖2

I an: nth estimated abundance vector

C1 C2 C3 C4 C5 C6(LMM-WSTO) (LMM-STO) (GBM) (PPNM) (NM) (RCA)

NLCS 0.98 0.96 5.11 5.10 10.38 26.35FLCS 81.45 0.59 11.57 9.90 32.51 30.40GBM 80.68 0.60 4.64 5.01 32.54 29.25PPNM 70.33 1.11 1.85 0.97 28.13 23.08NM 81.06 0.98 11.53 9.77 2.73 29.43RCA 1.12 1.09 2.69 2.62 3.63 6.85

G-RCA 1.34 1.29 2.69 2.65 3.56 6.96G-RCA+ 1.21 1.14 2.11 2.86 2.88 19.63

Table: Abundance RNMSEs (×10−2).35 /44


Generalised RCA

Results


Reconstruction

REk =

√1

NkL

∑n∈Ik

‖yn − yn‖2

I yn: nth reconstructed pixel

C1 C2 C3 C4 C5 C6(LMM-WSTO) (LMM-STO) (GBM) (PPNM) (NM) (RCA)

NLCS 1.72 1.72 1.81 1.78 2.07 4.43FLCS 55.12 1.72 2.61 2.49 7.65 11.69GBM 54.54 1.72 1.92 2.01 7.65 11.28PPNM 9.47 1.72 1.73 1.72 3.26 4.00NM 55.09 1.73 2.62 2.62 1.71 9.97RCA 1.72 1.71 1.70 1.71 1.70 1.70

G-RCA 1.72 1.71 1.70 1.71 1.70 1.70G-RCA+ 1.72 1.72 1.72 1.72 1.72 3.60

Table: Reconstruction errors (×10−2).

36 /44


Generalised RCA

Results

Real data

Estimated abundances

I Similar reconstruction errors

I Model assessment?

37 /44


Generalised RCA

Results

Real data

Nonlinearity analysis

38 /44


Conclusion

Outline




Conclusion

39 /44


Conclusion

Conclusion

Residual component analysis model

I RCA: flexible nonlinear model able to approximate differentmixing models

I Joint nonlinearity detection and abundance estimation

I Spatially correlated nonlinearitiesI Piece-wise constant: Potts model (finite number of classes)I Smoothness: GMRF (more flexible)

I Image segmentation based on nonlinearity levelsI RCA: via Potts modelI G-RCA: Bayesian hypothesis testing

I Estimation of the regularisation parameters (Potts, GMRF)

I GRCA+: positive nonlinear interactions

40 /44


Conclusion

Conclusion and future work

On the nonlinearity characterization

I Nonlinearity prior based on polynomial kernel KM well adaptedfrom polynomial mixtures

I The GP/prior could also depend on the abundance vectors

φn|M, zn = k, s2k,an ∼ N(0L, s

2kKM,a

)I The GP/prior covariance could also be directly learned from the

data

Unsupervised unmixing

I Extension of the proposed Bayesian model to estimate/correctthe endmember signatures

41 /44


Conclusion

Conclusion and future work

On the Endmember variability

I Bundles of possible spectral signatures

I Endmember distribution: e.g., Gaussian9 or Beta prior10

I Machine learning approaches (GPs, volatility models)

9O.Eches et al., “Bayesian Estimation of Linear Mixtures Using the Normal Compositional Model.

Application to Hyperspectral Imagery,” IEEE Trans. Image Processing, 2010.10

X. Du et al., “ Spatial and Spectral Unmixing Using the Beta Compositional Model ,” J. of Sel.Topics in Applied Earth Observ. and Rem. Sens., 2014.

42 /44


Conclusion

Thank you for your attention...

43 /44


Conclusion

Residual Component Analysis ofHyperspectral Images for Joint NonlinearUnmixing and Nonlinearity Detection

Steve McLaughlin(1)

Joint work with Y. Altmann(1), N. Dobigeon(2), J.-Y. Tourneret(2) and M.Pereyra(3)

(1)Heriot-Watt University – School of Engineering of Physical Sciences, U.K.(2)University of Toulouse – IRIT/INP-ENSEEIHT/Tı¿ 1

2 SA, France(2)University of Bristol – School of Mathematics,U.K.

MAHI 2014 workshop, Nice, 15/12/2014

44 /44


Conclusion

Synthetic data

Data set

I 60× 60 pixels image composed of R = 3 endmembersI 4 mixing models ⇒ K = 4 levels of nonlinearity

I LMMI Generalized bilinear model (GBM)I Polynomial post-nonlinear mixing model (PPNMM)I RCA model (s2 = 0.1)

I β = 1.6, SNR ' 30dB

Actual label map. log(‖φn‖2

). Detection map.

45 /44


Conclusion


Abundance estimation

RNMSEk =

√1

NkR

∑n∈Ik

‖an − an‖2

I an: nth estimated abundance vector

Unmixing algo.Class #0 Class #1 Class #2 Class #3

(LMM) (GBM) (PPNMM) (RCA)

FCLS 0.35 9.02 20.45 29.23

GBM 0.36 3.29 15.51 28.16

PPNMM 0.66 1.38 0.47 24.03

K-HYPE 3.36 3.26 3.13 3.57

RCA 0.44 1.55 2.13 3.52

Table: Abundance RNMSEs (×10−2).

46 /44


Conclusion


Reconstruction

REk =

√1

NkL

∑n∈Ik

‖yn − yn‖2

I yn: nth reconstructed pixel

Unmixing algo.Class #0 Class #1 Class #2 Class #3

(LMM) (GBM) (PPNMM) (RCA)

FCLS 0.99 2.17 1.33 3.10

GBM 1.00 1.15 4.38 11.94

PPNMM 1.00 1.01 0.99 3.79

K-HYPE 0.98 0.98 0.98 0.98

RCA-SU 1.00 0.99 0.98 0.98

Table: REs (×10−2).

47 /44

Residual Component Analysis of Hyperspectral Images for ...RCA of Hyperspectral Images for Joint...

Documents

Transcript of Residual Component Analysis of Hyperspectral Images for ...RCA of Hyperspectral Images for Joint...