Hyperspectral Imagery (HSI) Dimensionality Reduction

79
Know the Earth…Show the Way NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY proved for Public Release 05-269 Hyperspectral Imagery (HSI) Dimensionality Reduction Ronald G. Resmini, Ph.D. Ronald G. Resmini, Ph.D. 18 July 2005 18 July 2005 Institute for Pure and Applied Mathematics Institute for Pure and Applied Mathematics (IPAM) (IPAM) v: 703-735-3899 v: 703-735-3899 • [email protected] [email protected]

description

Hyperspectral Imagery (HSI) Dimensionality Reduction. Ronald G. Resmini, Ph.D. 18 July 2005 Institute for Pure and Applied Mathematics (IPAM) v: 703-735-3899 • [email protected]. Outline. Introduction to HSI HSI Dimensionality HSI Dimensionality Reduction (DR) - PowerPoint PPT Presentation

Transcript of Hyperspectral Imagery (HSI) Dimensionality Reduction

Page 1: Hyperspectral Imagery (HSI) Dimensionality Reduction

Know the Earth…Show the Way

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCYApproved for Public Release 05-269

Hyperspectral Imagery (HSI)

Dimensionality Reduction

Hyperspectral Imagery (HSI)

Dimensionality Reduction

Ronald G. Resmini, Ph.D.Ronald G. Resmini, Ph.D.18 July 200518 July 2005

Institute for Pure and Applied Mathematics (IPAM)Institute for Pure and Applied Mathematics (IPAM)

v: 703-735-3899 v: 703-735-3899 •• [email protected] [email protected]

Page 2: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 2Approved for Public Release 05-269

OutlineOutline

• Introduction to HSI

• HSI Dimensionality

• HSI Dimensionality Reduction (DR)

Do We Need DR?

• HSI Algorithms

• What Can You Do? How ShouldYou Do It?

• Introduction to HSI

• HSI Dimensionality

• HSI Dimensionality Reduction (DR)

Do We Need DR?

• HSI Algorithms

• What Can You Do? How ShouldYou Do It?

Page 3: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 3Approved for Public Release 05-269

writ large...the phenomenology of spectra;remote material detection, identification, characterization

and quantification

Introduction to Hyperspectral Imagery (HSI)

Remote Sensing

HSI is, fundamentally:

Page 4: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 4Approved for Public Release 05-269

HSI Remote Sensing:Frame of Reference...• Remote sensing of the earth

airbornespaceborneground (portables)

• But bear in mind other apps:medicalindustrialmany, many others

Page 5: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 5Approved for Public Release 05-269

Electromagnetic EnergyElectromagnetic Spectrum

Electromagnetic EnergyElectromagnetic Spectrum

Electromagnetic SpectrumElectromagnetic Spectrum

WavelengthWavelength

(nm)(nm)

Cosmic Cosmic RaysRays

Gamma Gamma RaysRays

X X RaysRays

Microwaves Microwaves (Radar)(Radar)

Radio & Radio & Television WavesTelevision WavesUVUV

101055 101066 101077 101088 101099 10101010 10101111 1010121210101110101010-1-11010-2-21010-3-31010-4-41010-5-5

Shorter WavelengthsHigh Energy

Shorter WavelengthsHigh Energy

Longer WavelengthsLow Energy

Longer WavelengthsLow Energy

V / NIR / SWIR / V / NIR / SWIR / MWIR / LWIRMWIR / LWIR

Optical RegionOptical Region

400400 1400014000

400

0.4

400

0.4

14000

14.0

14000

14.0

1500

1.5

1500

1.5

3000

3.0

3000

3.0

5000

5.0

5000

5.0

700

0.7

700

0.7

NIRNIR MWIRMWIRSWIRSWIRRRGG LWIR LWIR BB LWIRLWIRWavelength

(nm)(m)

Emitted Emitted EnergyEnergy

Reflected Reflected EnergyEnergy

Page 6: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 6Approved for Public Release 05-269

Reflected vs. Emitted EnergyReflected vs. Emitted Energy

1

104

1000

100

10

0.1 1 1053 7

Irra

dia

nc

e (W

-m-2-u

m-1)

Wavelength (µm)

Earth Emission

(100%)

EarthReflectance

(100%)

radian

t exitance (W

-m-2-u

m-1)

MWIR

Assumes no atmosphere

.4 .7

Page 7: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 7Approved for Public Release 05-269

Sampling the SpectrumSampling the Spectrum

NIR SWIR MWIR LWIR

400 nm400 nm 700700 15001500 30003000

RRBB

50005000 14000 nm

GG

Panchromatic: one very wide bandPanchromatic: one very wide bandLOW

Multispectral: several to tens of bandsMultispectral: several to tens of bandsMED

Hyperspectral: hundreds of narrow bandsHyperspectral: hundreds of narrow bandsHIGH

Page 8: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 8Approved for Public Release 05-269

Interaction of energy and objectsInteraction of energy and objects

Transmitted EnergyTransmitted Energy

Absorbed EnergyAbsorbed Energy

Reflected EnergyReflected EnergyV-MWIRV-MWIR

Emitted EnergyEmitted EnergyMW-LWIRMW-LWIR

Energy Balance Equation: EI () = ER() + EA() + ET() Energy Balance Equation: EI () = ER() + EA() + ET()

Incident EnergyIncident Energy

Page 9: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 9Approved for Public Release 05-269

NASA AVIRIS Cuprite, NV, HSI Data, (1995)

Page 10: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 10Approved for Public Release 05-269

An AVIRIS (NASA) HSI Image Cube

Page 11: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 11Approved for Public Release 05-269

The Spectrum is the Fundamental Datum of HSI RS

Page 12: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 12Approved for Public Release 05-269

Applications of HSI RSApplications of HSI RS

• Geology• Forestry• Agriculture• Mapping/land use, land cover analysis• Atmospheric analysis• Environmental monitoring• Littoral zone RS• Many, many others

Page 13: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 13Approved for Public Release 05-269

Levels of Spectral Information Levels of Spectral Information

Quantification: Determines the abundance of materials.

Characterization: Determines variability of identified material (e.g. wet/dry sand, soil particle size effects).

Identification: Determines the unique identity of the foregoing generic categories (i.e. material identification).

Discrimination: Determines generic categories of the foregoing classes.

Classification: Separates materials into spectrally similar groups.

Detection: Determines the presence of materials, objects, activities, or events.

Quantification: Determines the abundance of materials.

Characterization: Determines variability of identified material (e.g. wet/dry sand, soil particle size effects).

Identification: Determines the unique identity of the foregoing generic categories (i.e. material identification).

Discrimination: Determines generic categories of the foregoing classes.

Classification: Separates materials into spectrally similar groups.

Detection: Determines the presence of materials, objects, activities, or events.

Panchromatic Panchromatic

Low Spectral ResolutionLow Spectral Resolution

High Spectral ResolutionHigh Spectral Resolution

Hyperspectral

(100’s of bands)

Hyperspectral

(100’s of bands)

MultispectralMultispectral

(10’s of bands)

MultispectralMultispectral

(10’s of bands)

Page 14: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 14Approved for Public Release 05-269

Image from the NASA Langley Research Center, Atmospheric Sciences Division.http://asd-www.larc.nasa.gov/erbe/ASDerbe.html

Page 15: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 15Approved for Public Release 05-269

Electromagnetic EnergyAtmospheric Absorption

Electromagnetic EnergyAtmospheric Absorption

Page 16: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 16Approved for Public Release 05-269

Reflectance: Is the ratio of reflected energy to incident energy. Varies with wavelength Function of the molecular properties of the material.

Reflectance Signature: A plot of the reflectance of a material as a function of wavelength.

Reflectance: Is the ratio of reflected energy to incident energy. Varies with wavelength Function of the molecular properties of the material.

Reflectance Signature: A plot of the reflectance of a material as a function of wavelength.

Reflected EnergyReflected Energy

Red brick KaoliniteSandy loamConcreteGrass

All solids and liquids have reflectance signatures that

potentially can be used to identify

them.

All solids and liquids have reflectance signatures that

potentially can be used to identify

them.

Page 17: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 17Approved for Public Release 05-269

Emissive EnergyBasic Concepts

Emissive EnergyBasic Concepts

• Blackbody – A theoretical material that absorbs and radiates 100% of the energy incident upon it. BB curve is a function of temperature and wavelength.

• Planck’s Law – gives shape of blackbody curve at a specific temperature.

• Wien’s Displacement Law – determines wavelength of peak emittance.

• Blackbody – A theoretical material that absorbs and radiates 100% of the energy incident upon it. BB curve is a function of temperature and wavelength.

• Planck’s Law – gives shape of blackbody curve at a specific temperature.

• Wien’s Displacement Law – determines wavelength of peak emittance.

Wavelength (µm) 0.2 0.4 0.7 1 2 3 5 8 10 30

Sp

ectr

al R

adia

nt

Em

itta

nce

PeakEmittance

300KAmbient

250K

500K

800K

373KBoilingWater

6000KSun

3000KLight Bulb

1500KHot Coals

Page 18: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 18Approved for Public Release 05-269

1

52 12

kT

hc

BB ehcM

The Planck or Blackbody Radiation Equation:

mm

W2Units:

TM

TM

BB

Page 19: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 19Approved for Public Release 05-269

Emissive EnergyEmissive Energy• Emissivity - is a measure of how efficiently an object radiates

energy compared to a blackbody at the same temperature. Varies with wavelength Function of the molecular properties of the material.

• Emissivity Signature - A plot of emissivity as a function of wavelength. All materials have emissivity signatures that potentially can be used to identify them.

• Emissivity - is a measure of how efficiently an object radiates energy compared to a blackbody at the same temperature. Varies with wavelength Function of the molecular properties of the material.

• Emissivity Signature - A plot of emissivity as a function of wavelength. All materials have emissivity signatures that potentially can be used to identify them.

Blackbody

Graybody

Selective emitter(emissivity signature)

Selective emitter(emissivity signature)

Em

issi

vity

0

0.5

1.0

Wavelength

Red brick KaoliniteGrass Water

Black paint Concrete

Page 20: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 20Approved for Public Release 05-269

Spectral Signature LibrariesSpectral Signature Libraries

• Spectral signatures of thousands of materials (solid, liquid, gas) have been measured in the laboratory and gathered into “libraries”.

• Library signatures are used as the basis for identification of materials in HSI data.

(...beyond scope for a discussion on DR; but...)

Page 21: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 21Approved for Public Release 05-269

Understanding Spectral Data: Signature Variability Factors

Understanding Spectral Data: Signature Variability Factors

Brightness BRDF Target morphology

• shape

• orientation Particle size Moisture Spectral mixing

Composition

• original

• change over time Surface quality

• roughness

• weathering Shade & Shadow Temperature

Page 22: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 22Approved for Public Release 05-269

Reflected EnergyReflected Energy

• The manner in which a material reflects energy is primarily a function of the optical properties and surface roughness of the feature.

• Most objects are diffuse reflectors

• The manner in which a material reflects energy is primarily a function of the optical properties and surface roughness of the feature.

• Most objects are diffuse reflectors

Specular Reflectance

Specular Reflectance

Diffuse Reflectance

Diffuse Reflectance

Angle of Incidence = Angle of ReflectanceAngle of Incidence = Angle of Reflectance

Smooth Surface

Rough Surface

(Microscopic)

Energy Scattered in

All Directions

Page 23: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 23Approved for Public Release 05-269

Emissive EnergyIdentification of GasesEmissive EnergyIdentification of Gases

DetectedSignature

Plume

WavelengthWavelength

Emission

Background (Cool)

Gas (Warm)

Gases appear in either emission or absorption depending on the temperature contrast between the gas and the background.

Same Temperature

Same Temperature

WavelengthWavelength

No Detection

Background

Gas

WavelengthWavelength

Absorption

Background (Warm)

Gas (Cool)

Page 24: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 24Approved for Public Release 05-269

d1ehc2dMM

1

kT

hc52

0

432

445

Thc15

Tk2

Stefan-Boltzmann Law:

Two surfaces radiating at each other:

View Factor Algebra and Radiant Exchange...

1

11TTA

q

21

42

41

12

...from Welty, Wicks, and Wilson (1984)

Page 25: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 25Approved for Public Release 05-269

Spectral Mixture Analysis (SMA)

• An area of ground of, say 1.5 m by 1.5 m may contain 3 materials: A, B, and C.

• An HSI sensor with a GSD of 1.5 m would measure the ‘Mixture’ spectrum

• SMA is an inversion technique to determine the quantities of A, B, and C

in the ‘Mixture’ spectrum

• SMA is physically-based on the spectral interaction of photons of light and matter

• SMA is in widespread use today in all sectors utilizing spectral remote sensing

• Variations include different constraints on the inversion; linear SMA; nonlinear SMA

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40

Wavelength (micrometers)

Re

flect

an

ce

A

B

C

Mixture

‘Mixture’ = 25%A + 35%B + 40%C

Page 26: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 26Approved for Public Release 05-269

• Spatial Resolution

• Radiometric Resolution

• Temporal Resolution

• Spatial Resolution

• Radiometric Resolution

• Temporal Resolution

ResolutionsResolutions

(...beyond scope for a discussion on DR; but...)

Page 27: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 27Approved for Public Release 05-269

HSI Fundamentals SummaryHSI Fundamentals Summary

• Hyperspectral remote sensing involves measuring energy in the Visible – LWIR portions of the electromagnetic spectrum.

• Some of the measured energy is reflected from objects while some energy is emitted from objects.

• Every material has a unique spectral signature.

• Spectral image data are collected such that signatures can be extracted for material detection, classification, identification, characterization, and quantification.

• Spectral, spatial, radiometric, and temporal resolution determine the capabilities of the remote sensing sensor/system.

• Hyperspectral remote sensing involves measuring energy in the Visible – LWIR portions of the electromagnetic spectrum.

• Some of the measured energy is reflected from objects while some energy is emitted from objects.

• Every material has a unique spectral signature.

• Spectral image data are collected such that signatures can be extracted for material detection, classification, identification, characterization, and quantification.

• Spectral, spatial, radiometric, and temporal resolution determine the capabilities of the remote sensing sensor/system.

Page 28: Hyperspectral Imagery (HSI) Dimensionality Reduction

The

Gen

eral

Dat

a A

naly

sis/

Exp

loita

tion

Flo

w DN

Calibration

Fixes/Corrections

Data Ingest

Look At/Inspect the Data!!

Atmospheric Compensation

Algorithms for Information Extraction

Information Fusion

Geometric/Geospatial

Product/Report Generation

Distribution

Archive/Dissemination

Planning for Additional Collections

Spectral Library Access

Iteration

DR?

Approved for Public Release 05-269

Page 29: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 29Approved for Public Release 05-269

HSI Remote Sensing:Frame of Reference...

• A Scientist’s Approach to the Data: look at the data(!)observables have a physical,

chemical, biological, etc. basismust understand nature of observablesbumps and wiggles have real,

physical (spectroscopic) significanceapplication of tools comes last!

Page 30: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 30Approved for Public Release 05-269

Defining HSI Dimensionality• Hundreds of bands of data in an HSI data cube• An HSI pixel (a spectrum) is an n-D vector

n = number of bandsa spectrum is a point in an n-D space

• “Redundancy” of information• Embedding or spanning dimension• Intrinsic dimension/virtual dimension• A distinction

large volume of datadimensionality

Page 31: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 31Approved for Public Release 05-269

The n-D Space — Where Many Algorithms Operate

Each HSI spectrum (or pixel) is an n-D vector that

can be represented as a single point in n-D space.

n-D space is actually where many of our algorithms

operate.

Tn7654321 ,...,,,,,,,)pixelor(Spectrum

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.4 0.8 1.2 1.6 2.0 2.4

Wavelength (m)

Ref

lect

ivity

,

Page 32: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 32Approved for Public Release 05-269

Four (A-D) Equivalent Notations/Representations

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.50 0.75 1.00 1.25 1.50 1.75 2.00 2.25 2.50

Wavelength (micrometers)

Re

flect

an

ce,

(0.11, 0.23, 0.30, 0.25, 0.16, 0.27, 0.31, 0.37,...,)

...p.o.n.m.l.k.j.i. 370310270160250300230110

, Band a

, B

and

b

Spectrum s1

...imagine an n-Dhyperspace...

A B

C

D

Page 33: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 33Approved for Public Release 05-269

Some HSI Scatter Plots; Spectra as Points in ‘Hyperspace’

Page 34: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 34Approved for Public Release 05-269

Defining HSI Dimensionality• “Curse of dimensionality”

for Gaussian distribution......for a given classification accuracy# of training samples grows quadraticallybased on exploitation methodology; e.g.:

Mahalanobis Distance:

i1Ti

2i mxmxm,xd

Maximum Likelihood:

i1i

Tii

2i mxmxlnm,xd

Page 35: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 35Approved for Public Release 05-269

HSI or MSI• 100’s of bands vs. 10’s of bands• Maybe all you need is 6 bands but...

you need six; and you need six; and so on• Atmospheric compensation...• HSI is spectroscopy writ large

its about resolving spectral informationfine spectral featuresbroad spectral features

• Today’s FPAs make HSI a breeze anyway...

Page 36: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 36Approved for Public Release 05-269

Multispectral - Hyperspectral Signature Comparison

Multispectral - Hyperspectral Signature Comparison

Multispectral Hyperspectral

Resampled to Landsat TM7 Bands

Page 37: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 37Approved for Public Release 05-269

400

0.40

400

0.40

1500

1.50

1500

1.50

3000

3.00

3000

3.00

700

0.70

700

0.70

NIRNIR SWIRSWIRRRGGBB

Wavelength (nm)

(m)

Minerals/Geology

SoilsBathymetry

Vegetation

FuelsAerosols

Atmos. Comp.

Plastics

Fabrics

Paints

O2 CO2

Chlorophyll

DOM/CDOM Cirrus

Iron oxides

Similar figures may be constructed for M/LWIR regions.

Page 38: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 38Approved for Public Release 05-269

Estimating HSI Dimensionality

• Eigenvalues of the covariance principal components analysis (PCA)/aka KL optimal, least squares sense

• Eigenvalues of the correlation matrix

• Visual—based on eigenvalues

• Continuous significant linear dimensionality CSD; eigenvalues (next slide...)

Umaña-Díaz, A., and Vélez-Reyes, M., (2003). Determining the dimensionality of hyperspectral imagery for unsupervised band selection. Proceedings of the SPIE, S.S Shen and P.E. Lewis, eds., v. 5093, pp. 70-81. (...and references cited therein.)

Page 39: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 39Approved for Public Release 05-269

n

ii,minCSD

1

1

m

ii

p

ii

Var%

1

1100

where: i are the eigenvalues

i

Tyy mymyECovariance Matrix:

Find the eigenvalues of the covariance(or correlation) matrix and then...

or:

Page 40: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 40Approved for Public Release 05-269

0

5000000

10000000

15000000

20000000

25000000

30000000

35000000

40000000

45000000

50000000

0 20 40 60 80 100 120 140

Band Number

Eig

enva

lue

Virginia City Probe-1 HSI Data

Eigenvalues from a PCA

116 bands out of 128

Page 41: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 41Approved for Public Release 05-269

0

200000

400000

600000

800000

1000000

1200000

0 50 100 150 200

Band Number

Eig

enva

lue

Eigenvalues from a PCA

162 bands out of 210

Urban Scene HYDICE HSI Data

Page 42: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 42Approved for Public Release 05-269

0

10000

20000

30000

40000

50000

60000

70000

80000

0 20 40 60 80 100 120 140

Band Number

Eig

enva

lue

Eigenvalues from a PCA

128 bands out of 128

Mormon Mesa SEBASS HSI Data

Page 43: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 43Approved for Public Release 05-269

Estimating HSI Dimensionality

• Wavelet basis

• Nonlinear dimension estimationNear neighbor method of PettisFukunaga and Olsen’s KL-related methodFractal dimension

o Hausdorff dimensiono Box-counting methodo Correlation integral/dimension (next slide...)

Page 44: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 44Approved for Public Release 05-269

rln

rvlnlimDr

B 10

rln

rClnlimD m

r 0

Box Counting:

Correlation Dimension:

where: r is box size (DB) or radius of a hypersphere (D)

N

j,iji

Nm xxrH

NlimrC

12

1

Correlation Function:

Page 45: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 45Approved for Public Release 05-269

HSI Dimensionality Reduction• Techniques

Principal components analysis (PCA)Minimum noise fraction (MNF)Vector quantization (VQ)Projection pursuit (PP)The universe of data compression

o lossless/lossy (when/why?)o discrete cosine transformation (DCT)o wavelets-based compression

Best bands selection/band averaging

Page 46: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 46Approved for Public Release 05-269

Best-Bands SelectionBest-Bands Selection

2.0 m to 2.5 m – SWIR, Only

Page 47: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 47Approved for Public Release 05-269

Other Means of DR

• Spectral mixture analysis (SMA)basis vectors

• Analysis of filter vectors (OSP algorithms...)• Wavelet-based feature selection• On-board processing

transmit productadvanced computationquantum computation?

Page 48: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 48Approved for Public Release 05-269

Other Means of DR

• Transmit only bands of interestbest-bands selection/band averaging......perhaps after atmos. comp.

• Spectral parameterizations• Derivative spectroscopy• Binary encoding

Page 49: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 49Approved for Public Release 05-269

0.6

0.7

0.8

0.9

1.0

3 4 5 6 7 8 9 10 11 12 13 14

Wavelength (micrometers)

Quartz Primary Lobe 2Band Depth

(QP2)Quartz Primary Lobe 1 Slope

(QP1 Slope)

Primary Carbonate AbsorptionBand Depth(MWIR CO3)

Spectral Parameterization:Spectral Metrics (1 of 2)

Soil Spectrum

Page 50: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 50Approved for Public Release 05-269

Desert Soil (Malpais)

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.00 0.10 0.20 0.30 0.40 0.50

3.5 - 4.17 microns Band 2 FWHM

3.5

- 4.

17

mic

rons

Ban

d 1

Dep

th

Disturbed Soil

Pristine Soil

Vehicle Treads

Spectral Parameterization:Spectral Metrics (2 of 2)

Page 51: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 51Approved for Public Release 05-269

The Need For DR• Is there a need for DR?• Is there a “curse of dimensionality”?• Well...it depends...• Not with today’s (and tomorrow’s) computers• Not with many capable HSI algorithms• Structure of HSI in n-D space

linear mixing trends mixed pixels; spectral endmembers are these clusters?

• Yes...if using traditional MSI classification

techniques...

Page 52: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 52Approved for Public Release 05-269

Mahalanobis Distance

i1Ti

2i mxmxm,xd

Maximum Likelihood

i1i

Tii

2i mxmxlnm,xd

Page 53: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 53Approved for Public Release 05-269

dxw|xpw|xpJ

2

x

jiij

Jeffries-Matusita (JM) Distance

Bij e12J

21

j21

i

ji

ji

1

jiTji

2

2

1mm

2mm

8

1B

Where B is the Bhattacharyya distance

Page 54: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 54Approved for Public Release 05-269

HSI Algorithms

Page 55: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 55Approved for Public Release 05-269

Euclidean Distance: n-D Geometry

A 2D scatterplot with 2 spectra:

Band a

Ban

d b

Spectrum s1

Spectrum s2

Whole-Pixel Distance Metric in n-D Hyperspace

Assume a two band spectral remote sensing system. Each two point‘spectrum’ is a point in Band b vs. Band a space.

Euclidean Distance

Page 56: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 56Approved for Public Release 05-269

SAM: n-D Geometry

A 2D scatterplot with 2 spectra:

Band a

Ban

d b

Spectrum s1

Spectrum s2

Angular Distance Metric (Spectral Angle Mapper or SAM)

Assume a two band spectral remote sensing system. Each two point‘spectrum’ is a point in Band b vs. Band a space.

The angle, , between the two

lines connecting each spectrum

(point) to the origin is the angular

separation of the two spectra.

Smaller angular separations in-

dicate more similar spectra.

Page 57: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 57Approved for Public Release 05-269

SAM: The Math

• Chang (2003), ch. 2, pp. 20-21; and...• Assume two 5-band spectra as shown:

21

2T11

ss

sscos T1s

2s

Page 58: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 58Approved for Public Release 05-269

Linear Spectral Unmixing

The reflectance of an image pixel is a linear combination of

reflectances from (typically) several “pure” substances (or

endmembers) contained within the ground-spot sampled by the

remote sensing system:

n

jii,jji rMfR

1

where: Ri is the reflectance of a pixel in band i,

fj is the fractional abundance of endmember j in the pixel,

Mj,i is the reflectance of endmember substance j in band

i,

ri is the unmodeled reflectance for the pixel in band i, and

n is the number of endmembers.

Page 59: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 59Approved for Public Release 05-269

A linear equation...

7 x 5 5 x

1

7 x

1

5 endmembers in a 7-band spectral data set

A

x

b

bAAAx TT 1bAx

Page 60: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 60Approved for Public Release 05-269

y,xny,xMy,xr

d,uuuM 1pi1

1pi1 uuuU

nUdr p

OSP/LPD/DSR: Scene-Derived Endmembers

(Harsanyi et al., 1994; see also ch. 3 of Chang, 2003)

Page 61: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 61Approved for Public Release 05-269

#UUIP

T1T# UUUU

PnPdPr p

PnxPdxPrx Tp

TT

Page 62: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 62Approved for Public Release 05-269

xPPx

xPPddxmax

xPnnPEx

xdPdxmax

TT

TTT

x2

2p

TTT

T2p

T

xmax

xAxB 1

TTPPddA

TPPB

Page 63: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 63Approved for Public Release 05-269

Pdq TT

This is equivalent to Unconstrained SMA

The value of xT which maximizes is given by xT = dT

Pdd

PxdT

T

p

scalary,xrqT

Page 64: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 64Approved for Public Release 05-269

Statistical Characterization of the Background(LPD/DSR)

0

nUr

q

1i

Tiir rr

q

1

(Harsanyi et al., 1994)

Page 65: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 65Approved for Public Release 05-269

VV rT

r

#VVIP~

P~

dw TT

scalary,xrwT

Page 66: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 66Approved for Public Release 05-269

Constrained Energy Minimization (CEM)• The description of CEM is similar to that of OSP/DSR (previous slides)• Like OSP and DSR, CEM is an Orthogonal Subspace Projection (OSP)

family algorithm• CEM differs from OSP/DSR in the following, important ways:

CEM does not simply project away the first n eigenvectors The CEM operator is built using a weighted combination of the

eigenvectors (all or a subset)• Though an OSP algorithm, the structure of CEM is equally readily observed by

a formal derivation using a Lagrange multiplier

• CEM is a commonly used statistical spectral matched filter

• CEM for spectral remote sensing has been published on for over 10 years

• CEM has a much longer history in the multi-dimensional/array signal

processing literature

• Just about all HSI tools today contain CEM or a variant of CEM

• If an algorithm is using M-1d as the heart of its filter kernel (where M is the

data covariance matrix and d is the spectrum of the target of interest), then

that algorithm is simply a CEM variant

Page 67: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 67Approved for Public Release 05-269

Hº: pº(x)= xMx2

1exp(M2 1T2

12J

J = # of Bands

H1: p1(x)= bxMbx2

1exp(M2 1T2

12J

Form the log-likelihood ratio test of Hº and H1:

Xp

xpln)x(l

0

1

Stocker, A.D., Reed, I.S., and Yu, X., (1990). Multi-dimensional signal processing for electro-Optical target detection. In: Signal and Data Processing of Small Targets 1990, Proceedingsof the SPIE, v. 1305, pp. 218-231.

Derivation taken from:

Page 68: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 68Approved for Public Release 05-269

xMx2

1exp(M2

bxMbx2

1exp(M2

ln)x(l1T2

12J

1T212J

xMx2

1exp(

bxMbx21

exp(ln

1T

1T

xMx

bxMbx1T

1T

Some algebra...

Page 69: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 69Approved for Public Release 05-269

A trick...recast as a univariable problem:

2

2

2

2

1T

1T

xbx

2

1exp

xMx21

exp(

bxMbx21

exp(

After lots of simple algebra applied to the r.h.s:

2

2

2 2

bbxexp

Now, go back to matrix-vector notation:

0

1T1T

2

bMbxMbexp

a scalar threshold

Page 70: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 70Approved for Public Release 05-269

Take the natural log:

sxMb 1T ...a scalar for each pixel

{ {

FilterKernel

Pixel

01T

1T ln2

bMbxMb

Threshold, T

{

>T for H1; <T for H0

Page 71: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 71Approved for Public Release 05-269

xQQbxMb TTT 11

“The vector: QTx is a projection of the original spectral

data onto the eigenvectors of the covariance matrix, M,

which corresponds to the principal axes of clutter

distribution.” Stocker et al., 1990.

Page 72: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 72Approved for Public Release 05-269

“Further SCR gain is obtained by forming the optimum

weighted combination of principal components using

the weight vector:”

1QbT

From Stocker et al., 1990.

Page 73: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 73Approved for Public Release 05-269

Today’s HSI algorithms can also benefit from

1) spatial and spectral subsetting; 2) hierarchical

application of techniques; 3) other...

Its Important to Note That...

Page 74: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 74Approved for Public Release 05-269

What Should You Do?How Should You Do It?

• Join the fray!...HSI is a big tent• Dump Lena/girl...

the spectrum is the fundamental datum

• Conduct honest, rigorous comparisons with

existing, current best-practices HSI techniques• Apply techniques to multiple, large, diverse data sets• Team with HSI expert(s) and subject-matter expert(s)• Seek peer-reviews from HSI experts

I’m happy to be a reviewer...

• Learn about/care about the field; be relevant be more than buzz words we’re more than an opportunity for statistical analyses

Page 75: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 75Approved for Public Release 05-269

Contact Information

Ron Resmini

The Boeing Company (Associate Technical Fellow);

The National Geospatial-Intelligence Agency (NGA); and

School of Computational Sciences, George Mason University

v: 703-735-3899

e-mail(1): [email protected]

e-mail(2): [email protected]

Page 76: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 76Approved for Public Release 05-269

Backup SlidesBackup Slides

Page 77: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 77Approved for Public Release 05-269

• Remote Sensing of Environment

• International Journal of Remote Sensing

• IEEE Transactions on Geoscience and Remote Sensing

• Journal of Geophysical Research• Solid Earth, Planets, Oceans and Atmospheres

• Icarus

• Remote Sensing Reviews

• Photogrammetric Engineering and Remote Sensing

• Applied Optics

• Journal of the Optical Society of America

• Many others, too!

Resources: Peer-Reviewed Journals

Page 78: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 78Approved for Public Release 05-269

Resources: Non-Reviewed Journals

• Published conference proceedings:

• SPIE

• your library may subscribe to SPIE

• abstract services

• AVIRIS (NASA) conference

• IEEE/IGARSS

• Proceedings of the ASPRS

• Many others, too!

Page 79: Hyperspectral Imagery (HSI) Dimensionality Reduction

NATIONAL GEOSPATIAL-INTELLIGENCE AGENCY

Know the Earth…Show the Way 79Approved for Public Release 05-269

Linear Spectral Unmixing Theory

Spectral unmixing theory states that the reflectance of an image pixel is a

linear combination of reflectances from the (typically) several “pure”

substances (or endmembers) contained within the ground-spot sampled by

the remote sensing system. This is indicated below:

n

jii,jji rMfR

1

where: Ri is the reflectance of a pixel in band i, f j is the fractional abundance of substance (or

endmember) j in the pixel, and Mj,i is the reflectance of endmember substance j in band i. r i is

the band-residual or unmodeled reflectance for the pixel in band i, and n is the number of

endmembers. A spectral unmixing analysis results in n fraction-plane images showing the

quantitative areal distribution of each of the endmember substances and one root mean

squared (RMS) image showing an overall or global goodness of fit of the suite of

endmembers for each pixel. The RMS image is formed, on a pixel-by-pixel basis, by:

n

j

in

rRMS1

2Objects may also be detected as

anomalies in the RMS image.