MacArthur_Introduction to Field Spectroscopy

7/27/2019 MacArthur_Introduction to Field Spectroscopy

1/63

Introduction to Field Spectroscopy

NERC Field Spectroscopy FacilityCourse Handbook

COST Action ES0903Monte Bondone, Italy

7th to 9th July 2011

1


2/63

NERC FSF

Introduction to Field Spectroscopy Course

Alasdair Mac Arthur

School of GeoSciences, University of Edinburgh

7th

to 9th

July 2011

Programme

Thursday, 7th

of July 2011

14:30 15:00 Coffee and introductions -

15.00-16:15 - Introduction - The role of field spectroscopy in research

16.15 16:30 Break

16:30 18:00 The principles of field spectroscopy

19.00 Dinner at Hotel Montana

Friday, 8th

of July 2011

8:00 9:30 The design and calibration of spectroradiometers

9:30 10:15 Measurement in the laboratory and in the field (theory)

10:15 10.30 Break

10:30 12:30 Practical introduction to spectroradiometers (demo of bench top/laboratory

spectroscopy measurements) (1 x assistant required)

12:30 Lunch at Hotel Montana

14:00 15:15 Sampling design and measurement uncertainty

15:15 16:00 Introduction to Measurements in the environment spectroradiometers and sun

photometers. Outside but adjacent to training facility (1 x assistant required)

16:00 16:15 Break

16:15 18:00 The processing and analysis of spectral datasets Part 1

2


3/63

Saturday, 9

thof July 2011

8:00 10:15 The processing and analysis of spectral datasets Part 2 & Conclusions

10:15- 10:30 Break

3


4/63

1

Introduction to FieldSpectroscopy

Session 1: The role of field

spectroscopy in research

Alasdair Mac Arthur

Field Spectroscopy Facility

Iain Woodhouse, Director Chris MacLellan, Equipment Manager

Alasdair Mac Arthur, Operations Manager

Who are you?

Time for introductions

Structure of the course

Two day introduction to field spectroscopy

Assumes you already have a background knowledge ofthe physical principles of remote sensing

Lecture sessions, demonstrations, and a bit of hands on

Not designed to be majorly practical

Weather permitting field work Sunday and Monday

Timetable for the course Day 115.00-16:15 Introduction - The role of field spectroscopy in

research

16.15 16:30 Coffee break

16:30 18:00 The principles of field spectroscopy

19.00 Dinner at Hotel Montana

Timetable for the course Day 28:00 9:30 The design and calibration of spectroradiometers

9:30 10:15 Measurement in the laboratory and in the field (theory)

10:15 10.30 Coffee break

10:30 12:30 Practical introduction to spectroradiometers

12:30 Lunch at Hotel Montana

14:00 15:15 Sampling design and measurement uncertainty

15:15 16:00 Introduction to Measurements in the environment

Outside if possible

16:00 16:15 Coffee break

16:15 18:00 The processing and analysis of spectral datasets Part 1

4


5/63

2

Timetable for the course Day 38:00 10:15 The processing and analysis of spectral

datasets Part 2 & Conclusions

10:15- 10:30 Coffee break Session 1 The role of field

spectroscopy in research

Structure of the lecture

What is field spectroscopy?

Whats it used for?

What kind of research questions can itaddress?

What its basis?

Why a Field Spectroscopy Facility and whatdoes FSF do?

The case for and the challenges of thehyperspectral domain

Conclusions

What is field spectroscopy?

Definition: the quantitative measurement of

radiance, irradiance, reflectance or

transmission in the field

Definitions Hyper: meaning many, over-many

Contiguous: Next in space, immediatelysuccessive, neighbouring, situated in closeproximity

Continuous: having no breaks, unbroken,uninterrupted in sequence

Continuous spectrum: a spectrum not brokenby bands or lines

Definitions Hyperspectral sensing therefore represents an

extension and natural evolution of the conceptof multispectral sensing, to sensing in anincreased number of discrete contiguous bands(representing contiguous measurement of theoptical spectrum)

But, when does a sensor cease being MS and

become HS?

5


6/63

3

What is spectroscopy used for?

To make direct measurements of reflectances in the field To convert image-based measurements of reflected

radiance to calibrated radiance or reflectance(calibration of air- and spaceborne images)

For validation of satellite/airborne measurements

To better understand the nature of the interaction of ERwith earth surface objects

To improve the quantitative determination of earthsurface objects

To provide data for input into radiative transfer models

To build spectral libraries

Applications

0

5

10

15

20

25

30

35

400 900 1400 1900 2400

Wavelength (nm)

Reflectance(%)

p1

p4

p5

p8

p9

Applications Applications

Atmospheric science (spatial and temporal

variation in atmospheric constituents)

Water quality (chlorophyll estimation,

CDOM)

Ecology (vegetation biomass, physiology,

productivity) Geology (surface mineral identification,

mapping, geomorphic mapping)

What research questions can FS

address? What is the optimum spectral resolution required

for detection and characterisation of a target?

What are the optimum number of bands requiredto characterise a target given a range inbiophysical parameters?

What are the optimum algorithms whatmethods of analysis can be applied?

What spatial resolution is required?

To what extent does the signal vary with time?What is the best time of year/day?

What signal to noise ratio is required?

The basis of field spectroscopy

Fundamentally physically based

quantitative

objective

Replicable or at least it should be!

The topic of the next lecture

6


7/63

4

Spectroradiometers

Portable instruments for the measurement ofthe optical characteristics of earth surfacetargets

For use in laboratory or the field

Data are recorded for a single sample unit

Ability to control acquisition parameters,particularly in an experimental design moreso than for airborne or spacebornehyperspectral imaging

Sunphotometers

Portable instruments for the measurement of

atmospheric optical properties

For use in the field

For:

Atmospheric correction of image datasets

Characterisation of atmospheric particulate

components, for modelling

Instrumentation @ FSF

4 ASD FieldSpec Pros

1 SVC HR-1024

3 FSF VSWIRs

4 GER 1500s

6 MicroTops sunphotometers

2 CIMEL sunphotometer

Calibration facilities and

maintenance of standards to

National Physical Laboratory

standards

GRASS System for estimating BRDF with bespoke VSWIR

spectroradiometer

FTIR Fourier Transform infrared (FTIR) instrument,

new instrument now deployed

Measures over 0.2 to 15 m

Accessories A range of fore optics for different fields of

view radiance; irradiance and reflectance

Contact probes

A range of field reference panels

Tripods and mounts

Light meters

Stabilised power supplies and lamps

GPSs

Laptop and Toughbook computers

7


8/63

5

The case for and the challenges

of the hyperspectral domain

Discrete spectral bands

Landsat TM data 6 optical, 1 thermal ~ limiting?

1

2

4

5

7

6

3

(thermal)

Multispectral sensorTM Spectral Bands

0

10

20

30

40

50

60

70

80

400 700 1000 1300 1600 1900 2200 2500

Wavelenth (nm)

Reflectance

1 5432 7

Discrete numbers of spectral bands

Loss (or overlooking) of potentially useful

information

The spectral signature

MS is limiting when we want to exploit the more subtledifferences between (often spectrally similar) earthsurface objects

0

10

20

30

40

50

400 600 800 1000 1200 1400 1600 1800 2000 2200 2400

Wavelength (nm)

Reflectance(%)

90% cover vegetation

50% cover vegetation

Bare light soil

Bare dark peak

Clear lake water

Limitations of the MS approach Crude spectral characterization of the

reflectance properties of earth surface objectsusing multispectral approaches

Loss of potentially useful information inregions overlooked by the bands

Loss of more subtle information contained inthe fluctuations in the signature curve

Doesnt allow us to better define somethingabout the object of interest

The limited information in multispectralimaging systems potentially limitsclassification

The argument for hyperspectral Improves characterisation of the Earth's

surface

Provides more detailed information

Can filter data to match any lower spectral

resolution sensor

Identify spectral features too small to be

detected by multispectral sensors

Hypothesized that higher resolution spectral

information will lead to better characterization

or classification of Earth surfaces

8


9/63

6

The argument for hyperspectral Analysis

Use of alternative techniques for data analysis

(beyond the spectral ratio)

Alternative processing techniques which

identify features and materials through

measurement of spectral absorption features

Draws on concepts developed in laboratory

spectral analysis (from chemistry), e.g.:

Spectroscopy

Gas chromatography

Liquid chromatography

The argument for hyperspectral

Improved understanding of:

Earth surface reflectances / spectral

signatures

Interaction of light with earth surface

objects

Use of radiative transfer models can help us

better understand interaction of ER & Earth

surfaces and develop improved techniques for

analysis

The argument for hyperspectral

Pushes us away from traditional approaches,

E.g. image analysis limitations in 3-band

colour display

But complexity of the data complicates the

issue of analysis Data rich

A rich dataset

Requires techniques

for data reduction

Allows new methods

of data exploration not

available in datasets

covering fewer bands0

5

10

15

20

25

30

35

400 900 1400 1900 2400

Wavelength (nm)

Reflectance(%)

p1

p4

p5

p8

p9

Conclusions

Field spectroscopy is a key component inhyperspectral (and MS) remote sensing underpinsRS from aircraft and satellites

Advantages

Full exploitation of the spectral domain

RT modelling

Enhanced ability for spectral discrimination

Chemical and molecular analyses

Physiological and biogeochemical analyses ofvegetation

Chemical and mineralogic analyses of water

Geological and mineralogical analyses

Conclusions

Disadvantages

In the field may rely on the Sun

Difficult to do well- easy to do badly!

Processing requirements

Cost

Difficulty of analysis analysis in its

infancy!

9


10/63


11/63

GeometryGeometry TheoryTheory BRDF - In direct illumination, incident and

reflected can be regarded as confined to twoslender elongated cones. If the solid angles ofthe cones (measured in steradians, sr), areinfinitesimally small, the reflectance of thetarget can be defined as a function:

dL is the reflected radiance per unit solidangle

dEis the irradiance per unit solid angle

i and rdenote incident and reflected rays

( )( )

( )ii

rrrrii

dE

dLf

,

,,,, =

BRDFBRDF To specify completely the reflectance must be measured

at all possible source/sensor positions =BidirectionalReflectance Distribution Function

The fundamental physical property governingreflectance behaviour from the surface (Nicodemus1982)

A theoretical concept: not possible to measure inpractice - we estimate it

Commonly simplified to bidirectional reflectance factor(BRF) but BRF describes reflectance for parallel beamsof radiance and irradiance

To relate BRF to BRDF involves a number ofassumptions not well understood

BRDF & BRFBRDF & BRF

This is for a pointsource

HDRFHDRF We need to consider an irradiance hemisphere (direct +

diffuse)

Also need to consider the direction of reflectance or radiance

Leads to HDRF (R ) hemispherical directional reflectancefactor

L is the reflected radiance per unit solid angle

Eo is the direct and Ed diffuse irradiance per unit solid angle

Note introduction of

should have been included in allprevious functions

Measurement of light always has wavelength dependencies

but not there yet .

(sometimes referred to as HCRF)(sometimes referred to as HCRF) iHDRFiHDRF We measure an integrated reflectance or radiance from

a surface area defined by the directional responsefunction (DRF) of the spectroradiometer

DRF commonly considered to be FOV = erroneous

The response of spectroradiometer to photons has bothdirectional and wavelength dependencies

Caveat emptorif you purchase a spectroradiometer andbelieve the manufacturers specified FOV!

integral - areax1 to x2 by y1 to y2 is the spectrometers directional and wavelength dependent

response function (DRF)

((x,yx,y))

11


12/63

iHDRFiHDRF

Is determined by the DRF of thespectroradiometer/fore optic and the structural

and optical properties of the surface

shadow-casting

multiple scattering

transmission, absorption and emission by surface

elements

transmission, absorption and emission by surface

body

facet orientation distribution and facet density

Why is (i)HDRF important?Why is (i)HDRF important?

Needed for: Correction of view and i llumination angle effects on

images

Deriving albedo

Land cover classification

Atmospheric correction

To attribute reflecting components to gross reflectancerecorded

Gives a boundary condition for any radiative transferproblem and hence its relevance for climatemodelling and energy budget investigations

Directional reflectance illustratedDirectional reflectance illustrated

Black spruce forward and

back scatteringBare soil forward and back

scattering

Why is (i)HDRF important?Why is (i)HDRF important?

Measurement of iHDRFMeasurement of iHDRF

FIGOS

Visualising iHDRFVisualising iHDRF

Polar plots

In principal plane Theprincipal planeis the plane

of illumination

12


13/63

FSF GRASS instrumentFSF GRASS instrument Gonio Radiometric

Spectrometer System Currently in development at

NPL

30 degree intervals around

hemisphere on up to six arms,

each with 5-6 collecting optics

Fibre optics based, feeding

through a multisplexer to a

spectroradiometer

Very rapid measurement

(~30min per hemisphere!)

Two practical measurement geometries

in the field

Two practical measurement geometries

in the field

Cos-conical method

Bi-conical method

Cos-conical methodCos-conical method

Upward-looking

spectrometer with a cosine-

corrected receptor is used

(measures irradiance)

Is not dependent upon the

zenith or azimuth angle ofthe incident flux

Often used in dual mode

Cos-conical methodCos-conical method

Reflectance

where dEis the irradiation as measured by theupward-looking cosine sensor

kis correction factor relating cosine receptor toa perfectly diffuse white panel

N.B. All other iHDRF functions omitted forbrevity

( )( )

( )rrii

rrt

rriik

dE

dLR

,,,

,,,, =

What does a cosine head do?What does a cosine head do? The flux of a beam of light at an oblique angle

delivers fewer photons per m2 than a beamperpendicular to the surface:

The ratio of the flux densities of the two beams isthe cosine of the angle of the oblique beam

A sensor should respond to oblique beams withthis ratio

One that does is said to give a cosine response

White diffuserWhite diffuser

At low angles some light is reflected, causing a

lower reading than reality

To correct for this, sensors are enclosed in a

black cylinder with a raised, small plastic

diffuser on top

This is called a cosine corrected head or

remote cosine receptor (RCR)

13


14/63


15/63

Spectralon panelsSpectralon panels

Do have non-Lambertian reflectanceproperties with respect to global radiation at

very large solar zenith angles (above 60)

Sun-angle correction factors available from

panel manufacturers but !

FSF measured cosine responsesFSF measured cosine responses

Results from work by C. MacLellan of FSFResults from work by C. MacLellan of FSF

Spectralon correction factorsSpectralon correction factorsAssumptions in reflectance field

spectroscopy

Assumptions in reflectance field

spectroscopy Sensor field-of-view < ~20 degrees

The panel must fill the sensor field-of-view

There is no change in incident irradiation amount or itsspectral distribution

Direct solar flux dominates the irradiation field

The sensor responds in a linear fashion to changes in

radiant flux The reflectance properties of the standard panel are known

and invariant over the course of the measurements

The sensor is sufficiently distant from the target

What does it all mean?What does it all mean? Two measurements required:

Target radiance

Incident irradiance (cosine or panel)

Measurements must be made simultaneouslyor as close in time as practical

FOV < 20

Made under clear sky conditions

The sensor is calibrated

The panel is calibrated

Height above target is important

ReflectanceReflectance

Reflectance = Target/Reference

Calculation of reflectance cancels out multiplicative

effects such as:

Spectral irradiance of the illumination source

Optical throughput of the field spectrometer

But assumes characteristics of the illumination were

the same for the target and reference measurements

if they were made at different times (sequentially)

15


16/63


17/63

Session 3: Instrument design and

calibration

Session 3: Instrument design and

calibration

Alasdair Mac Arthur

Structure of the lectureStructure of the lecture

Inside the black box

Key instrument components

Key issues and specifications

Examples of field spectroradiometers

Spectroradiometric calibration

Conclusion

Whats inside a spectroradiometer?Whats inside a spectroradiometer?

GER 3700

{ Diffraction gratings

{ Detector arrays

{ Entrance slits

Together, these components can makeup whats known as a spectrograph, an

instrument used to separate and measurethe relative amounts of radiation in eachwavelength in electromagnetic radiation

+ Beam splitters

Key components inside the boxKey components inside the box

Diffraction gratingsDiffraction gratings

An optical element, which separates

(disperses) polychromatic light into its

constituent wavelengths (colours)

Realized as fine parallel and equally spaced

grooves on a surface

When light hits a diffraction grating,

diffractive and mutual interference effects

occur, and light is reflected or transmitted in

discrete directions, called diffraction orders

Diffraction GratingsDiffraction Gratings

Grating equation:

d(sin + sin ) = m , where

dis distance between adjacent grooves

is angle of incidence

is angle of diffraction

m is order (-3rd to +3rd)

is wavelength

Entrance Slit

Exit Slit

17


18/63

Diffraction GratingsDiffraction Gratings

Zero, first & second order spectra

First OrderS e c o ndO r de r

Overlapping1 st & 2nd Order

Zero OrderInput Slit

DetectorsDetectors

Made from a variety of different substances

Silicon photodiodes (200 1100 nm)

Indium gallium arsenide photodiodes (~900

~2500 nm)

Germanium photodiodes (650 1800 nm)

Lead sulphide detectors (1000 3300 nm at minus

45

C, require cooling)

Detector arraysDetector arrays

Individual detectors arranged in a line

(1-D) or 2-D matrix

Silicon photodiode arrays

Charge Couple Devices (CCD)

PbS & InGaAs arrays Need order blocking requirements

SpectrographsSpectrographs

Flat field gratings image is projected

onto a flat plane, focussed

Linear array means acquires

whole spectrum

simultaneously

Spectrographs key benefitsSpectrographs key benefits High speed (~ 10 ms) - means very short measuring

times

Photodiode array or CCD technology

Parallel spectral channel acquisition by bolting

several spectrographs together

No moving parts (excluding shutter)

High sensitivity with long integration times

Needs a shutter to measure dark current, set

exposure to ambient light conditions

Beam splitterBeam splitter

Used to split a beam of light in two

Means the same light beam can be used

to feed a number of spectrographs

Transmitted Light

18


19/63

GER 3700 full wavelength deviceGER 3700 full wavelength device

350 2500 nm Essentially has three spectrographs, from the

one input optic, to cover: Visible and near infrared (Vis/NIR 512

elements)

Short wave infrared (SWIR1 128 elements)

Short wave infrared (SWIR2 64 elements)

That means it has: Two beam splitters

{ Three diffraction gratings

{ Three detector arrays

Three spectrographs

DetectorsDetectors

Vis/NIR

SWIR1

SWIR2

Diffraction gratingsDiffraction gratings Beam splittersBeam splitters

Optical pathOptical path

Vis/NIR

SWIR1

SWIR2

SWIR1 & 2Optical Chopper

Shutter

GER 3700 in diagrammatic formGER 3700 in diagrammatic form

SWIR

1

SWIR2

Diffraction Gratings

Detector Arrays

Lenses

Beam Splitters

Apertures & Slits

Shutter

Visible & NIR

10 Fore Optic

19


20/63

Inside the ASD FieldSpec ProInside the ASD FieldSpec Pro

Three spectrometers but the SWIR unitsare oscillating gratings

VNIR

SWIR 1

SWIR 2

Fixed Diffraction Grating

Oscillating Diffraction Grating

Detectors & Detector Array

Lenses & Mirrors

Fibre Optic Light Guide

MonochromatorsMonochromators

Oscillating grating, oscillates over 20

Passes different wavelengths across detector

Single high quality/sensitive detector, reduces

cost

Input Slit

Exit Slit

Detector

Other key issuesOther key issues

Stray light

Order effects

Input optics

Detector noise and linearity

Harmonics

Stray light and order effectsStray light and order effects

Radiation of the wrong wavelength activates a

signal at a detector element. Comes from:

Ambient light

Scattering from imperfect optical components

Reflections of non-optical components

Order overlap from diffraction gratings

Encasing the spectrometer in a light tight

housing eliminates ambient stray light.

Filters can be used to eliminate order effects

Input opticsInput optics

Field-of-view lenses

Fibre optic light guides

Cosine corrected diffusers

Ideally, these should have spatially

uniform properties

ASD Non uniform input opticsASD Non uniform input optics

With 10 lens

fore-optic:

FOV extends

up to 270 mm

beyond 10

limit at 1m

Nominal FOVActual area of measurement

support per detector

20


21/63

GER 3700 with 10 fore-optic DRFGER 3700 with 10 fore-optic DRF

450nm 700nm 950nm

1500nm 2200nm

Detector noise and linearityDetector noise and linearity

There will always be responses in the system unrelated to scenebrightness (noise)

Instruments must be designed so that the noise levels are smallrelative to the signal (target brightness)

Measured as the Signal to Noise ratio (SNR) = ratio of usablesignal to non-interpretable portion of the signal (i.e. noise) at agiven input signal level

SNR needs to be large over the full dynamic range

Noise Equivalent Spectral Radiance (NESR), alsocalledNoise Equivalent Power(NEP): minimumresolvable change in input. The lowest brightness that can bereliably measured, i.e. when SNR = 1

tDarkCurren

tDarkCurrenDNSNR

=

Dark current, linearity and dynamic

range

Dark current, linearity and dynamic

range

Ideal linear response

Dark current

signal

Imagebrightness

Scene brightness

Actual sensorresponse

SaturationDynamic range

A

B

Field spectroradiometersField spectroradiometers

Key specifications

Spectral range & sensitivity over full range

Optical bandwidth & wavelength resolution

Speed

Size & weight

Signal to noise

Dynamic range

Non-linearity

Resolution and optical bandwidthResolution and optical bandwidth

Sampling resolution - set by number of pixels indetector array

Optical bandwidth is defined by:

Dispersion of diffraction grating

Focal length

Width of the entrance & exit slits

Pixel width

Oversampling - Most instruments oversample thespectrum e.g. optical bandwidth of ~10 nm andwavelength sampling interval of ~2-5 nm

Resolution and optical bandwidthResolution and optical bandwidth

Full Width at HalfMaximum

Ideal response

Achievableresponse

Bandwidth

50%Detectorresponse(%)

Wavelength (nm)

50

100

Definition of resolution

21


22/63

Calibration of spectroradiometersCalibration of spectroradiometers

Spectroradiometric calibrationSpectroradiometric calibration

Calibrations performed by FSF

typically includes:

Absolute spectral radiance calibration (300

2500nm)

Absolute spectral irradiance calibration

(300 2500nm)

Wavelength calibration

Needs standards against which to

perform the calibration

Calibration facilityCalibration facility Radiance calibrationRadiance calibration

Uses integrating sphere

White coated, provides uniform light

output

Calibrated by the National Physical

Laboratories (NPL)

Irradiance calibrationIrradiance calibration Uses FEL lamp

Calibrated by NPL

Good output over all wavelengths

Wavelength calibrationWavelength calibration Erbium and holmium oxide panels, lamps

(mercury, sodium, etc)

Major Peaks

Mercury Emission Lines

3 65 .0 nm 4 91 .6 nm404.7nm 546.1nm

435.8nm 578.1nm*

Argon Emission Lines

6 96 .5 nm 8 26 .5 nm

706.7nm 841.8nm*7 38 .4 nm 8 52 .1 nm

750 .9nm* 866 .8nm

7 63 .5 nm 9 12 .3 nm

7 72 .4 nm 9 22 .4 nm801 .1nm* 965 .8nm

811.1nm*

* denotes doublet, (a1+ b2)where a & bare relative intensities

Mercury-argon lamp

22


23/63

Calibration meansCalibration means

Quality assurance (NER, SNR radiometric andwavelength accuracies)

Traceability (quantification of uncertainty)

Stability monitoring of instruments

Confidence in the accuracy of the data

Credible comparison to historical/international

data sets

User reassurance

Sources of errorSources of error

Input optics

Stray light

Spectroradiometric calibration

Equipment instability

These are only the errors related to

the instruments themselves. It ignores

errors associated with actual

measurements in the field

ConclusionsConclusions

Get to know your equipment (intimately)

Know how it works

Question the suitability of the instrument

for your application

Trial the instrument before you take themeasurements that count

23


24/63

1

Session 4: Measurement in the

laboratory and field

Alasdair Mac Arthur


Introduction

Laboratory measurements

Outdoor measurements:

Conclusions

Making laboratory measurements The Hardy tent mobile lab

Designed to fit over boxes Advantages

More stable illumination compared to outdoors

No atmospheric interference (full spectrum)

Controlled geometry, no variations inillumination angle

Can make measurements during non-optimalconditions

Potential increase in signal:noise ratio

Lends itself to an experimental set up andapproach

Can control the background

24


25/63

2

Disadvantages

Is artificial illumination (notsunlight), difficult

to set up

Direct or direct and scattered illumination

Variable spatial light field

Less real

Influence of backgrounds if not controlled??

Heat produced by lamps can affect target and

spectroradiometer

Key issues

Lamps: Want something that simulates sunlight

Not many do

Tungsten lamps are probably best or combinationof tungsten and halogen

500 Watt security lamps are cheap Power supplies:

Always check the stability of the power supplypowering the illumination lamps Real issue withASDs

Often surprising amounts of variation in voltages,especially in laboratories

Use a stabilised power supply required

Other issues

Backgrounds best kept black

The lamps usually require a warm up time

determine what it is

Check the heat output of the lamps:

Determine likely impact on the target (if

vegetation it might be significant) Put lamps further away or think about providing

cooling for the target (e.g. fan?)

Measure the spatial distribution of the

illumination over the target

Spatial distribution of illumination

Overcoming spatial illumination problems

Consider adding more lights, but

Altering shading patterns

Confounding the heating problems

Always measure in the same

configuration to minimise its impact

(e.g. orientation of samples)

Making measurements in the field

25


26/63


27/63

4

Single field-of-view

Single sensor head

Alternated between target and

reference (panel usually)

Necessarily imparts a time delay

between the two measurements

Assumes the target and reference are

viewed under the same irradiation

conditions and geometry

The greater the difference in time

between the two measurements the

greater the risk of error

Measurement geometry

Target viewing geometry:

Appropriate field of view (choice of instrument

foreoptic).

Appropriate sampling unit (area of target

sampled) - Related to instrument field of view

and height above the canopy

View angle nadir/off nadir

Illumination geometry:

In direction of incident irradiance

Beware shading of target and panel

Height above canopy

Depend on what your wish to achieve

Daughtry et al. (1982) for row crops

Number of measurements required for a givenlevel of precision decreases with increasingsensor altitude

Higher the sensor the greater the areaintegrated into the spectrum recorded

Ultimately, a compromise between the theoryand what is practical

Suspending a spectroradiometer

Sun angles and timing Time of day:

At midday?

At time of sensor overpass?

How cope with illumination effects

induced by changing sun angle if

measurements made through the day

Timing through year:

How frequent?

What are you trying to characterise?

Other influences in the field Clouds (always take sunglasses)

Cirrus clouds are difficult to spot but cansignificantly alter absolute intensity and spectralshape

Partial cloud cover also contributes significantlyto diffuse skylight illumination

Need to minimise the time between target andirradiance (reference) measurement

Wind movement of vegetation canopies and waterin particular can significantly alter reflectance

Dew affects and wet canopies

27


28/63

5

Influence of atmosphere

0

10

20

30

40

50

400 900 1400 1900 2400

Wavelength (nm)

Reflectance(%)

Overcast sky

Clear Sky

Atmospheric influences

Possible reasons: Direct radiation (i.e. clear skies) more

shading of canopy components

Diffuse less shading, greater reflectance

in regions where scattering is high.

Lower reflectance in certain areas where

greater chance of absorption (by water or

pigments)

Recording variations in incident

light

PAR sensor records PAR radiationPyranometer records totalsolar radiation

Direct and diffuse

Ratio of direct to diffuse conditions Use a

shade ring over a pyranometer to measure

diffuse, or

Use a sun photometer to measure aerosol

optical thickness and water vapour

DeltaT BF3 sunshine sensor

records diffuse & total solarradiation (PAR) and

Sunshine Hours

Important factors to be recorded Spectrometer set-up:

Field-of-view

Raw output versus reflectance output

File naming conventions

Viewing geometry

Atmospheric conditions:

Clouds, humidity, haze, wind, pyranometerirradiance, horizontal line of sight

Physical conditions of target:

Biophysical parameters, slope, aspect, moistureconditions, GPS location, soil moisture

Take digital photos Record the condition of the target at the time

of measurement

Take a photo of the sky

Record the set up

28


29/63

6

Recording forms

Example logsheets

Considerations of the very practical

kind

Warming up of instruments Charging batteries

Logistics

Check everything is there and everythingworks BEFORE you go into the field!

Allow for the fact that your first visit wontgive you useful data

Transport

You will need help very rarely is it a oneperson show

Sequence of events: ideal scenario

Choice of Instrument

Instrument configuration

Experiment design

Fieldwork plan

Training

Fieldwork Post processing

Publications

29


30/63

1

Session 5: Sampling design and

measurement uncertainty

Alasdair Mac Arthur


Introduction

Definitions accuracy and precision

Design of field programs

Site selection

Experimental design / sampling design

Uncertainties

Vicarious calibration

Conclusions

Definitions

Accuracy confidence in the relation of one set ofmeasurements with another

Getting it right

Precision careful measurement under controlledconditions

Confidence in successive measurements with

the same equipment and operating conditions The repetitiveness of measurement of the same

target

Accuracy is telling the truth . . . Precision istelling the same story over and over again

Approaches to research

Inductive phenomena observed, generalisationsmade, conclusions drawn

Deductive hypotheses posed, tested byobservation, experiment

Much of the spectroscopy literature is inductive,indicating that we are still evaluating differentapproaches, strengths and limitations of the data

:K\GRZHQHHGGHVLJQ"

A little planning saves a lot of time

To gather information as efficiently andaccurately as possible

Statistics cannot rescue a badly designedmethod

Poorly collected data lead to poor conclusions

If you must break the rules, do so consciously

Design of field programs

Dependent on objectives of study, e.g.:

Spectral libraries

Vicarious calibration of image data,

empirical line targets

Establishing relationships

But sampling strategy is key to all

30


31/63

2

Strategies

Dependent on approach, e.g.:

Correlation vs manipulative study

Field or laboratory or both (my preferred

approach!)

Being able to control what you can, and

randomize the remaining effects

Control tends to decrease the experimental

error

Key considerations

Selecting the site/sample

Relation to the objectives of the investigation

Being practical and achievable

Cost-effective in time and equipment

Providing estimates of population parameters

that are truly representative and unbiased

How you plan to process the data

Site selection considerations

Uniformity versus heterogeneity

Scale of several space sensor pixels

Sampling sites span the radiometric range thatoccurs across the landscape

Other considerations: Slope and aspect

Time Accessibility (some of the kit is not light!)

Matching the radiometers characteristics to airor space sensor (spectral bands, viewing angle)

How many samples?

How many observations or measurements must beacquired to be confident of:

Detecting differences between surface types ordifferent conditions

Sufficiently characterising the surfacebidirectional reflectance properties?

Required number of sample sites and number ofmeasurement replicates

Dependent on scale of study and objectives

Do a pilot study the only way to have statisticalvalidity!

How many samples?

Ideally, representative samples should

be:

Large enough to give sufficient precision

Unbiased by the sampling procedure or

equipment (i.e. accurate)

Sampling design Approaches:

Point sampling

Transects

Plots

Approach should be established before theinvestigation proceeds

Key factors:

The dimensions and shape of the sampling unit(e.g. field of view (DRF!) versus height)

The number of sampling units in each sample

The location of sampling units within the samplingarea

31


32/63

3

Point sampling Plots

Sampling a plot How not to do it! Transects

ASD on GPS enabled quad-bike

Acquiring spectral transect data Grid sampling to acquire spectra

32


33/63

4

Grid sampling to acquire spectraReplication Replication within a treatment shows how variable the

response can be Provides an estimate of experimental error

Improves precision by reducing the standard error of themean

Increases the scope of inference of the results

Required number dependent on statistical analysis to beapplied (to establish significance)

If in doubt, start with n = 3 but in the naturalenvironment you will need many many more

Be consistent in your data recording

Beware pseudoreplication

Pseudoreplication in spectroscopy would be

taking three scans from the same point and

assuming they are replicates

i.e. intra-sample variation versus inter-sample

variation

Result is an unrealistically small SE, which

may invalidate conclusions Can use intra-sample measurements to

increase signal to noise but cant use them as

the basis of a statistical analysis

Point sampling strategies

Random? Stratified? Systematic?

Webster et al. (1989):

Relate sampling effort to the variation

present

The more variable a region the larger the

sample should be

Determine required sampling by pilot

survey

Bog pool approx 10x20m

Heterogeneous landscapes

May Building June Building

Heterogeneous areas of interest

33


34/63

5

Point sampling strategies

Many targets are spatially autocorrelated

Random sampling is less optimal in this

situation

Thus, other approaches are better

Webster et al. propose use of the semi-

variogram to define the spatial variation and to

design an efficient sampling

Vicarious calibration

Includes empirical line correction

Use earth surface sites (pseudo-invariant) to

verify calibrations or calibrate sensors

Needs in-situ radiometer measurements

Vicarious calibration - sites with long-term

stability to enable temporal calibration drifts to

be determined

Sites need to be large and homogeneous

Desert Calibration Targets

Sonora - MexicoLibyan (Western) Desert - Egypt

Makhtesh Ramon Israel, Negev desert for

wavelength calibration

Vicarious calibration Allows radiometric closure between sensor top-of-

atmosphere radiance values and ground measurements(bottom-of-atmosphere)

Measurements on ground must be synchronous withsatellite overpass (due to changes in atmosphere andtarget properties)

Scaling problems: satellite integrates over a larger areaon the surface. How respresentative is the point

measurement on the ground? Needs sufficient spectroradiometer measurements to

characterise variability within the area of the target/pixel

All measurements are usually normalised to a groundbased, laboratory reference standard

Empirical line method To establish empirical relationships between

sensor radiance and ground reflectance

Requires coincident measurement of two ormore ground sites of contrasting brightnessthat are discernable on imagery

Again, requires sufficient spectralmeasurements to characterise the variability inthe targets

Smith and Milton area 4 x image pixel size

20+ measurements

Empirical line measurements

Karpouzli, E., Malthus, T. (2003). The empirical line method for

the atmospheric correction of IKONOS imagery.International

Journal of Remote Sensing, 24(5):1143-1150.

34


35/63

6

Sufficient number of targets

IKONOS NIRed Band 1

4

9

5

6

8

3

7

2

y = 0.0559x- 4.5544

R2

= 0.9609

0

10

20

30

40

50

60

70

80

0 200 400 600 800 1000 1200 1400

Radiance recorded by the sensor (DN values)

Groundmeasurementsreflectance(%)

Ensure range of brightness

NI R Waveband

Lawn

Grass

Cement Sand

Tarmac

y = 0.096x- 4 .3262

R2

= 0.9308

0

10

20

30

40

50

60

70

80

0 200 4 00 6 00 800

IKONOS Radiance

Ground

reflectance(%)

Bl ue Waveband

Cement

Sand

Grass

Lawn

Tarmac y = 0.1229x- 29.264

R2

= 0.8731

0

10

20

30

40

50

0 2 00 400 600 8 00

IKONOS Radiance

roun

re

ectanc

e

GreenWaveband

Sand

Cement

Lawn Grass

Tarmac y= 0.0903x- 15.687

R2

=0.878

0

10

20

30

40

50

0 200 400 6 00 8 00

IKONOS Radiance

Groundreflectan

ce(%)

Red Waveband

Sand

Cement

GrassLawnTarmac y= 0.0899x- 7.4236

R2

=0.886

0

10

20

30

40

50

0 2 00 400 600 8 00

IKONOS Radiance

Ground

reflectance(%

)

Malthus, T.J., Karpouzli, E.

(2003). Integrating field and high

spatial resolution satellite based

methods for monitoring shallow

submersed aquatic habitats in the

Sound of Eriskay, Scotland, UK.

International Journal of Remote

Sensing, 24(13):2585-2593.

Conclusions

Careful consideration must be given to the

design of your sampling approach and the

propagation of errors in measurement

A careful approach maximises your chance of

Obtaining meaningful data

Obtaining reproducible data Reliably detecting differences or

phenomena not previously explained

35


36/63


37/63

Other considerationsOther considerations

Correction for non-Lambertian panel

reflectance at very large solar zenith angles

(above 60)

Using the published corrections for Spectralon

Requires calculation of sun-angle of course

(time and position dependent)

UncertaintiesUncertainties

Inherent

Instrument errors

Calibration errors

Measurement uncertainty

Influences of the atmosphere

Variable illumination

Influence of temperature

Real variation

Influence of the atmosphereInfluence of the atmosphere

0

10

20

30

40

50

400 900 1400 1900 2400

Wavelength (nm)

R

eflectance(%)

Overcast sky

Clear Sky

Influence of temperatureInfluence of temperature

Change in radiance measurement over reference panel

in NIR inside the Hardy tent

0

500

1000

1500

2000

2500

3000

3500

900 1000 1100 1200

Wavelength (nm)

Radiance(RawD

N)

Software demonstrationsSoftware demonstrations

The FSF spreadsheet

ASD Viewspec software

Better still FSF Matlab Toolbox

SAMSSAMS

Spectral Analysis and Management System

Version 3.2 recently available

From Centre for Spatial Technologies and

Remote Sensing (CSTARS) at UC, Davis

http://sams.casil.ucdavis.edu/

37


38/63

SAMSSAMS

Accepts ASD and GER formats

Imports ASCII (plain text) files intelligently

Save spectra as groups and then save in SAMS

database

Can link spectra to metadata

Common spectral functions

Other tools PARLesOther tools PARLes

Merging data into a single file Transformations (smoothing, detrending,

derivatives, wavelets)

Principal components analysis (PCA)

Partial least squares regression (PLSR)

Bagging PLSR (bootstrap aggregation)

Viscarra-Rossel (2008), available on request from

[email protected]

Calculating broad band reflectancesCalculating broad band reflectances

Calculating broad band reflectancesCalculating broad band reflectances

To simulate reflectances from different

airborne and satellite borne sensors

AVHRR Spectral Bands

0

10

20

30

40

50

60

70

80

4 0 0 7 0 0 1 0 00 1 3 0 0 1 6 0 0 1 9 0 0 2 2 00 2 5 0 0

Wavelenth (nm)

Reflectance

1 2

TM Spectral Bands

0

10

20

30

40

50

60

70

80

4 0 0 7 0 0 1 0 00 1 3 0 0 1 6 0 0 1 9 0 0 2 2 0 0 2 5 00

Wavelenth (nm)

Reflectance

1 5432 7

Broad band reflectancesBroad band reflectances

Needs spectral response functions for the

specific sensor(s) you will simulate

Why?

Because they are all different!

They are not uniformly sensitive

Set of filter functions incorporated into the

FSF Matlab Toolbox

Landsat ETM+ bands 1 to 7Landsat ETM+ bands 1 to 7

0

0.2

0.4

0.6

0.8

1

400 900 1400 1900 2400

Wavelength (nm)

Spectralsensitivity

Not particularly symmetrical, best approximation using available filters

38


39/63


40/63

Comparison of smoothing methodsComparison of smoothing methodsInterpolationInterpolation

Dont unless you have to! Can introduce artefacts

Talk to a mathematician

If you use an ASD FS Pro it will interpolate foryou whether you like it or not!

Spectral signaturesSpectral signatures

Spectral librariesSpectral libraries

Sets of measured spectra for components likely to beencountered in the study area

For spectral matching, spectral mixture analysis, etc.

Several available, particularly for rocks and minerals

Easy to compile for some objects (e.g. minerals)

For others (e.g. plants, water targets) the key is thecoincident access to the metadata which describes the

key characteristics of the target

Find out how they were generated before you use themand consider the implications

Spectral librariesSpectral libraries Increasing availability of spectral data (more

instrumentation), more widespread

Increased focus on cal / val

Integration with IP softwares / data analysis

Moves to continuous monitoring of reflectances

Initiatives and calls to develop archives of spectra use- wider community initiatives

Incompatible, often internal, data formats, fromdifferent instruments, separated from metadata

Needs for data preservation, legacy value, lineage

Uses of spectral librariesUses of spectral libraries Calibration e.g. potential use as a target for

atmospheric correction

Validation e.g. to validate a model-basedatmospheric correction, model simulation

End members for incorporation into imageprocessing routines

Global algorithms / analysis

Retrospective analysis for test a new researchhypothesis / analysis technique

40


41/63

Spectral librariesSpectral libraries

Spectral libraries:

USGS spectral library - ~500 spectra of mineralsover 200 3,000 nm range(http://speclab.cr.usgs.gov)

Johns Hopkins University

JPL spectral library (160 spectra)

Mainly mineral, obtained under controlled conditions,

few biophysical targets

Conditions of measurement and subsequent

processing less well documented

Spectral databasesSpectral databases

Spectral databases

ASTER spectral library ~ 2000 spectra of rocks,soils, water, snow, man-made materials(http://speclib.jpl.nasa.gov)

Hyperspectral.info

SPECCHIO (www.specchio.ch)

Again, metadata components are minimal

IssuesIssues

Make goodmeasurements in the field

Spectral data collections are most often project(campaign) based, obtained for different purposes(unique?)

Different methods, different instruments

Of highly variable (unknown) quality

How to store and easily exchange such data

Implications for data quality and assessment

Coping with single spectra, nested data from projects,replicates, related targets, campaigns

Efficient in metadata entry

The challenge of field measurementThe challenge of field measurement

What determines quality?What determines quality? Quality of conditions under which it was obtained

Quality of the instrument and its calibration

Design of the study

Experience of the user

Visual quality of the spectrum obtained

Quality of the documentation obtained with the

measurement, including information on the properties

of the target (even a photograph)

Quality flags?

MetadataMetadata Quality relies on spectral data themselves but

associated metadata is fundamentally necessary

The existence of extensively documented metadataultimately determines long-term usability andquality

Assists searching and selection

Assists assessment of suitability for other researchprojects

Critical if data are obtained in field data

A current hot topic with NERC

Workshop on FS metadata and spectral libraries beingplanned for Oct 2011 by Tim Malthus, CSIRO,Canberra.

41


42/63

Metadata requirementsMetadata requirements

Should document:

Instrument characteristics

Conditions of measurement (meteorological,

physical, geometrical)

Target properties

Subsequent processing

Data characteristics

A spectral dataset of high quality is one where these

characteristics are extremely well documented it

indicates that care has been taken in data acquisition

MetadataMetadataInstrument

make and modelManufacturer

Serial numberOwnerDetector types

Spectral wavelength rangeSpectral bandwidthSpectral resolution

OperatorDarkSignal correction

Signal to Noise

Scan durationOptic Field-of-view dimension X

Optic field-of-view dimension YGain settings (Automatic/Manual)Signal averaging (instrumental)

Integration timeSetup (single beam, dual beam)

Mode (cos-conincal, biconical)

Calibration

Date

IrradianceRadiance

Dark noise

Signal to Noise

Linearity

Stray light

Calibration data

Traceability (e.g. Yes, No)

Standard (e.g. NIST, NPL)

Reference standards

None

Cosine receptorType

Reference standard

Reference (panel, cosine)

Serial number

Reference material

Time of measurement

Calibration standard

MetadataMetadataMeasurement and configuration

General:

Name of experiment/ProjectDate of experiment

Relevant publicationRelevant websitesProject participants

Acknowledgement Text

Plot numberDescription of target/sampleType of measurement (field, lab, etc)

Target type vegetation, water, rock, air,Target ID

Target treatmentSky conditions

Clear sky, % cloud, cloud type

Cloud coverHorizontal sight

Wind speed

Optical measure of ambient conditions (direct,diffuse)

Source of illumination (e.g. sun, lamp)Location

Referencing DatumMap projection

Base unitCoordinate sourceN-S Coordinates

E-W CoordinatesLongitudeLatitude

AltitudeViewing geometry

Time of measurement

Distance from targetDistance from ground/background

Area of target in field of view

Illumination zenith angleIllumination azimuth angle

Sensor zenith angleSensor azimuth angle

Optical measure of ambient conditionsAmbient temperatureInstrument temperature

MetadataMetadataData (post)processing and manipulation

Software used, version numberInterpolation

NoneAlgorithm applied

Number of interpolated points per datapoint

Atmospheric band removal (yes, no)

Averaging (yes, no, if yes, how many?)Averaging (mean or median, or closest

spectra?, Standard deviation reported?)

SmoothingNone

Algorithm appliedFilter sizePolynomial order

Number of times appliedOther

Difference spectraDerivative spectra

Supplementary Target data:

VegetationWater

Rock/soilAtmosphere etc.

Vegetation:Common nameSpecies

TypeClass

SubclassLeaf / Canopy

LAIChlorophyll contentBiomass

Moisture contentLeaf angle distributionTime ofyear

Background (soil / other?)Soil typeSoil moisture content

Comment, etc.

MetadataMetadata The data itselfData precisionData type (Reflectance, Radiance)First X valueLast Y value

First Y valueLast Y value

Min X valueMax X valueMin Y value

Max Y valueNumber of X values

Wavelength intervalXTitleYTitle

XUnitsYUnits

Scaling factors

XfactorYfactor

Wavelength dataSpectrum

WaterLocation

WaveheightWind conditions

Depth of measurementSuspended sediment concentrationChlorophyll concentration

Secchi disk transparency, etc.

Underwater substratum TargetSubstrate descriptionType (hard, soft, vegetation, animal)

Specifications?Density of growthPresence of epiphytes

Water typeSpectrum type (in situ/on boat/in lab)

Upwelling/downwelling radiance

Spatial resolutionWater surface conditions

Wind conditions

SolutionSolution

Research project which is developing an XML

exchange format, not a database

PhD research into metadata requirement for FS

currently being conducted by

Being rigorous promotes best practice in field

measurement

42


43/63

ConclusionsConclusions

Visual assessment of individual spectra is important Software tools to enable visual analysis and to allow

for batch processing

Watch broad band calculations, not as easy as it

sounds

Smoothing is more complicated than it looks! As is

interpolation and extrapolation is fraught!

Spectral libraries require work and use critically!

43


44/63

Session 7: The analysis of

spectral datasets

Session 7: The analysis of

spectral datasets

Alasdair Mac Arthur

Structure of the lectureStructure of the lecture

Introduction - Approaches

Linear methods

Multivariate methods

Non-linear methods

Conclusions

IntroductionIntroduction

The data richness of hyperspectral data presents aconsiderable challenge to the analyst

They provide fine spectral detail, but at anoverwhelming data volume

There is much apparent redundancy betweenneighbouring bands

Much of the analysis is an exercise in data reduction,but such that the differences in reflectances betweenground surface objects can be retained (featureextraction / reduction)

The curse of dimensionality

IntroductionIntroduction

The development of appropriate tools and approaches

for visualising and analysing hyperspectral data is still

very much ongoing

Much of that is focussed on the spectral rather than the

spatial, let alone directional the next frontier!

Feature reduction transforms original data to datasetof lower dimensionality but retaining original

information

Deductive approaches to analysisDeductive approaches to analysis

Empirical approaches

Correlation

Derivative analysis

Spectral decomposition

Feature reduction

Discriminant analysis

Neural networks

Many of the techniques are methods of datareduction

SoftwareSoftware Specific to field spectra:

ViewSpec, SAMS package, PARLes

Bespoke programs

Spreadsheets (e.g. Excel)

Software processing and visulaisationpackages (e.g. Matlab, IDL)

Matlab possibly preferred by fieldspectroscopists, IDL by image analysts

A plead - Please learn a scripting languagethat can manipulate 3-D numerical arrays

44


45/63


46/63

IndicesIndices

Subsurface reflectance CASI data

More complex indicesMore complex indices

E.g. Normalised difference

Three-band reflectance models:

Using reciprocal reflectances

Pigmentconcentration R1

1( )R1

2( )[ ]R 3( )

3-band algorithm - Water3-band algorithm - Water

Chlorophyll in water. Gitelson et al. (2008) works across a

range of waters

Chl mg m3( ) R1 665( )R1 715( )[ ]R 750( )

n-band algorithm - Vegetationn-band algorithm - Vegetation

Chlorophyll in crops. Gitelson et al. (2005) works across

different crops

Correlation matricesCorrelation matrices

Welsh lake subsurfacereflectances and all possible

ratios versus chlorophyll

concentration

George and Malthus (2001)

NIRSNIRS Near infrared reflectance spectroscopy

Extensively used in agro-chemistry

E.g. composition of grains and foods

Require calibrations that relate spectral information to

analyte being determined (e.g. C or N content)

Translated to foliar biochemistry

Often based on stepwise multivariate regression, PCA

or PLSR, derivatives

E.g. Workman (1992), Curran et al. (1992), Peterson et

al. (1988)

46


47/63

NIRSNIRS

Problems:

Overfitted (more wavelengths than samples

selected)

Selection of non-causal wavelengths

Intercorrelation of variates

Generality

E.g. Curran et al. (2002)

Continuum removalContinuum removal


Fitting the continuum Continuum removed

Huang et al. (2004)

Continuum removal / band depthContinuum removal / band depth

Band depth can be depth of the

feature or area of the feature

Analysis on basis of continuum

removal tends to better

relationships (i.e. R2) than

normal data Note subtle difference between

approach used in these graphics


Removes irrelevant background reflectance

Allows isolation and enhancement of

absorption features

Tends to give higher R2 values

Combined with stepwise multiple regression

and PLSR

Often used by geologists of much more limited

use to others

Stepwise regressionStepwise regression

Leaf biochemistry from reflectances, from Curran etal. (2002)

Widely used in NIRS, suffers from problemshighlighted above

47


48/63

Partial least squares regressionPartial least squares regression An extension of multiple regression

Similar to principal component analysis, but fits linear model

Tries to find multidimensional direction inx-space that explains

maximum multidimensional variance direction iny-space

Suited to hyperspectral data when no. of predictors exceeds

number of observations

Overcomes overfitting problems and multicollinearity among X

values

Requires correlograms of reflectance to help identify optimal

bands

LimitationLimitation

Stepwise multiple regression and PLSR both based on

the assumption that linearrelationships exist between

variable and reflectance

But those relationships may be non-linear

Spectral features selected tend to be site-specific or be

non-causal, therefore non-general in application

(requires recalibration)

Derivative analysisDerivative analysis

Derivatives give an indication of rate of change, or

slope of the original spectrum

Increasing reflectance = positive first derivative

Decreasing reflectance = negative first derivative

Can be of any order (e.g. 1 st, 2nd, 3rd, 4th) but noise is

amplified with every level smoothing often required

Derivative analysisDerivative analysis Commonly applied in analytical chemistry where

peaks in 4th derivative can be shown to be related topeaks in absorbing compounds

i.e. useful for the resolution of overlapping spectralfeatures

Allows for elimination of background signals (e.g.soils from vegetation spectra)

Derivatives of second or higher should be relatively

insensitive to variations in illumination intensitywhether caused by changes in sun angle, cloud cover,or topography

Examples: Demetriades-Shah et al. (1990), Malthusand Dekker (1995), Tsai and Philpot (1998)

Derivative analysisDerivative analysis

Derivative correlationDerivative correlation

Subsurface reflectance CASI data

48


49/63

Derivative approachesDerivative approaches

Shaw et al. (1998) Scots Pine regeneration study

Neural networksNeural networks

Caters from non-linear relationships

Evidence that they give stronger relationships (higher

R2) than linear methods (Huang et al. 2004)

Does not need continuum removal

Somewhat black box in operation

Binary encodingBinary encoding

Goetz 1991, Goetz et al. 1985.

Binary encodingBinary encoding

Considerably reduces dimensions of dataset

while preserving main features

Field spectra can then be compared to a binary

encoded spectral library for spectral matching

Spectral matchingSpectral matching Matching image spectra to

those derived from spectrallibraries

Needs accurate conversion ofimage radiance to reflectance

Scene should contain mainlypure pixels (of materials)

Complicated by mixed pixelproblem

Uses goodness of fit todetermine best match

Can use continuum removal

Critique of spectral matchingCritique of spectral matching

Though an attractive and straightforward idea,

spectral matching and [spectral libraries] have not

proven to be very powerful in terms of their ability to

extract information in a robust and practical sense

Landgrebe (1999)

Criticism in that it relies on the notion of the pure

pixel is there such a thing?

49


50/63

Discriminant analysisDiscriminant analysis

Multivariate analysis technique

A form of statistical classification on a priori

defined groups

Stepwise methods can be used as a tool for

data reduction to identify the key

wavelengths leading to target separation

Does have its faults

WaveletsWavelets Mathematical functions that can be

convolved and passed across spectra Analyse spectra across wavelengths

for frequency content

Decompose spectra into decreasing

sets of coefficients

Can be used for denoising data

Can be used as a method of data

reduction

Can be used for analysis (Blackburn

2008)

RT models forward and inverse

modelling

RT models forward and inverse

modelling PROSPECT (Jacquemoud 1996)

Models the optical properties of leaves

LIBERTY (Dawson 198)

Models the optical properties of needles

ACRM (Kuusk 2001)

Models canopy reflectance and can account for

layers incorporates PROSPECT and LIBERTY

4Sail2 (Verhoef 2003)

Models canopy reflectance and can account for

layers can be coupled to PROSPECT

Other methodsOther methods

Support vector machines (e.g. Melgani and Bruzzone

2004) shown to be more effective than other non-

parametric classifiers (e.g. ANNs, K-nn)

Spectral un-mixing (linear and non-linear)

Spectral angle mapper (SAM)

Spectral deconvolution

Principal components analysis (PCA)

??????

We just keep nicking them for others!

ConclusionsConclusions Power of high spectral resolution allows the

investigation of alternative analytical

approaches

Largely techniques of feature reduction

Often adopt empirical approaches to algorithm

development

Algorithm development / analysis still in its

infancy

RT modelling offers a virtual laboratory

50


51/63


52/63

Method IV ( Dual System)

Reflectance =

Method V (Dual System)

Reflectance = x Cal Inter-Cal

Notes:i) All DN values must be normalised for integration times and amplifier gains

settingsii) Inter calibration methods do not require the system response calibration files

(CalIrradiance, CalRadiance) but a uniquely generated for the systems used and theambient lighting conditions

DN#2Upwelling x Cal#2 Radiance

DN#1Downwelling x Cal#1 Irradiance x 2

Spectro #1with cosinereceptor

Spectro #2with FOVor barefibre

Ref Panel


Spectro #1with FOVor barefibre

Target

SimultaneousMeasurements

Inter-Calibration


DN#2 Upwelling

DN#1 Downwelling


Spectro #2with FOV

or barefibre

Target


52


53/63

Logsheets

53


54/63

54


55/63

55


56/63

56


57/63


58/63


59/63

Introduction to Field Spectroscopy

Selected reference list

Asrar, G. (1989). Theory and applications of optical remote sensing. New York, John Wiley and

Sons. (Chapter 2 summarises a few relevant approaches)

Curran, P. J., Dungan, J. L., Macler, B. A., Plummer, S. E., & Peterson, D. L. (1992).

Reflectance spectroscopy of fresh whole leaves for the estimation of chemical composition.

Remote Sensing of Environment, 39: 153-166. (Foliar biochemistry, regression analysis)

Curran, P.J., Dungan, J.L., Peterson, D.L. (2002). Estimating the foliar biochemical

concentration of leaves with reflectance spectrometry: Testing the Kokaly and Clark

methodologies. Remote Sensing of Environment, 76: 349-359. (Continuum removal,

multivariate regression, foliar biochemistry)

Daughtry, C. S. T., K. P. Gallo, et al. (1983). Spectral estimates of solar radiation intercepted by

corn canopies.Agronomy Journal 75: 527-31. (Effects of variations in measurement height)

Daughtry, C. S. T., V. C. Vanderbilt, et al. (1982). Variability of reflectance measurements with

sensor altitude and canopy type. Agronomy Journal 74: 744-51. (Effects of variations in

measurement height)

Demetriades-Shah, T. H., M. D. Steven, et al. (1990). High resolution derivative spectra in

remote sensing.Remote Sensing of Environment33: 55-64. (The case for derivative analysis)

Duggin, M. J. (1981). Simultaneous measurement of irradiance and reflected radiance in field

determination of spectral reflectance. Applied Optics 20(22): 3816-3818. (Approaches to

measurement)

Duggin, M. J. (1983). The effect of irradiation and reflectance variability on vegetation condition

assessment.International Journal of Remote Sensing 4(3): 601-608. (Influence of extraneousfactors)

Duggin, M. J. and T. Cunia (1983). Ground reflectance measurement techniques - a Comparison.

Applied Optics 22(23): 3771-3777. (Comparison of methods)

George, D.G., Malthus, T.J. (2001). Using a compact airborne spectrographic imager to monitor

phytoplankton biomass in a series of lakes in north Wales. Science of the Total Environment,

268:215-226. (Empirical approach to analysis)

Gitelson A.A., Vina, A., Ciganda, V., Rundquist, D.C., Arkebauer, T.J. (2005). Remote

estimation of canopy chlorophyll content in crops. Geophysical Research Letters, 32:

L08403 (3 band algorithms - vegetation)

Gitelson, A.A., DallOlmo, G., Moses, W., et al. (2008). A simple semi-analytical model for

remote estimation of chlorophyll-a in turbid waters: Validation. Remote Sensing ofEnvironment, 112: 3582-3593. (3 band algorithms - water)

Goel, N.S., and N.E. Reynolds (1989). Bidirectional canopy reflectance and its relationship to

vegetation characteristics.International Journal of Remote Sensing, 10:107-132. (on BRDF).

Goetz, A.F.H. (1991). Imaging Spectrometry for studying Earth, Air, Fire and Water. EARSeL

Advances in Remote Sensing 1: 3-15. (binary encoding)

Goetz, A.F.H., Vane, G., Salomon, J. and Rock, B.N. (1985). Imaging spectrometry for Earth

remote sensing. Science, 228: 1147-1153. (binary encoding)

Huang, Z., Turner, B.J., Dury, S.J., Wallis, I.R., Foley, W.J. (2004). Estimating foliage nitrogen

concentration from HYMAP data using continuum removal analysis. Remote Sensing of

Environment, 93: 18-29. (Continuum removal, stepwise regression, partial least squares,

neural networks)

59


60/63

Karpouzli, E., Malthus, T. (2003). The empirical line method for the atmospheric correction of

IKONOS imagery. International Journal of Remote Sensing, 24(5):1143-1150. (Example of

spectroradiometry for atmospheric correction)

Karpouzli, E., Malthus, T.J., Place, C.J. (2004). Hyperspectral discrimination of coral reef

benthic communities in western Caribbean. Coral Reefs, 23:141-151. (Example of use of

disriminant analysis of both reflectance and derivative data)Kimes, D. S., J. A. Kirchner, et al. (1983). Spectral radiance errors in remote sensing ground

studies due to nearby objects. Applied Optics 22: 8-10. (Influence of variations in

background on measurements obtained)

MacArthur, A.A., C. MacLellan, T. J. Malthus (2006). What does a spectroradiometer see?

Proceedings of the Remote Sensing and Photogrammetric Society Annual Conference,

University of Cambridge, September 2006. (highlights deficiencies in uniformity of fields-

of-view of GER and ASD spectroradiometers).

Malthus, T.J., Dekker, A.G. (1995). First derivative indices for the remote sensing of inland

water quality using high spectral resolution reflectance. Environment International, 23:221-

232. (Example of analysis based on derivatives, correlation analysis)Malthus, T.J., George, DG, (1997). Airborne remote sensing of aquatic macrophytes in Cefni

Reservoir, Anglesey, UK. Aquatic Botany, 58:317-332. (Example of use of discriminantanalysis)

Malthus, T.J., Karpouzli, E. (2003). Integrating field and high spatial resolution satellite based

methods for monitoring shallow submersed aquatic habitats in the Sound of Eriskay,

Scotland, UK. International Journal of Remote Sensing, 24(13):2585-2593. (Example of

empirical line method applied)

Melgani, F., Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with

support vector machines. IEEE Transactions In Geoscience and Remote Sensing, 42: 1778-

1790. (Support vector machines)

Meroni, M., Colombo, R. (2009). 3S: A novel program for field spectroscopy. Computers and

Geosciences, 35:1491-1496. (Software package for processing spectroscopic data)Milton, E. J. (1987). Principles of field spectroscopy.International Journal of Remote Sensing 8:

1807-27. (Lays down the basic principles of the approach well worth a read)

Milton, E.J., Schaepman, M.E., Anderson, K., Kneubuehler, M., & Fox, N. (2009). Progress in

field spectroscopy.Remote Sensing of Environment, 113:S92-S109. (opinion paper).

Milton, E. J., Rollin, E.M., Emery, D.R (1995). Advances in field spectroscopy. Advances in

environmental remote sensing. F. M. Danson, Plummer, S.E. Chichester, Wiley: 9-32.

(Recent advances to 1995)

Nicodemus, F.E. (1970). Reflectance nomenclature and directional reflectance emissivity.

Applied Optics, 9:1474-1475. (defining paper on BRDF)

Nicodemus, F.E., C.J., Richmond, et al. (1977). Geometrical considerations and nomenclature

for reflectance. US Government Printing Office, Washington DC 20402. (defining paper on

BRDF)

Peddle, D. R., H. P. White, et al. (2001). Reflectance processing of remote sensing

spectroradiometer data. Computers & Geosciences 27(2): 203-213. (describes spreadsheet

basesd approach to processing of spectral data)

Pegrum, H., N. Fox, M. Chapman, and E Milton. (2006). Design and testing a new

instrumentation to measure the angular reflectance of terrestrial surfaces. Proceedings of

IGARSS06, Denver, Colorado (IEEE). (Describes basis and design of the GRASS

goniometer).

Peterson, D. L., Aber, J. D., Matson, P. A., Card, D. H., Swanberg, N. A., Wessman, C. A., &

Spanner, M. A. (1988). Remote sensing of forest canopy leaf biochemical contents. RemoteSensing of Environment, 24: 85-108. (multivariate regression, foliar biochemistry)

60


61/63

Reeves, J., McCarty, G., Mimmo, T. (2002). The potential of diuse reflectance spectroscopy

for the determination of carbon inventories in soils. Environmental Pollution, 116: S277-

S284. (Near infrared reflectance spectroscopy)

Richardson, A. J. (1981). Measurement of reflectance factors under daily and intermittent

irradiance variations. Applied Optics 20(19): 3336-3340. (Influence of changes in incidentirradiance)

Robinson, F. B. and L. L. Behl (1979). Calibration procedures for measurements of reflectance

factors in remote sensing field research. Society of Photo-Optical Instrumentation

Engineering 196: 16-26. (Importance of calibration)

Savitzky, A. and M. J. E. Golay (1964). Smoothing and differentiation of data by simplified least

squares procedures. Analytical Chemistry 36: 1627-39. (Paper outlining least squares

polynomial smoothing, but errors contained in tables of coefficients)

Schaepman-Strub, G., M.E. Schaepman, T.H. Painter, S. Dangel, J.V. Martonchik (2006).

Reflectance quantities in optical remote sensing definitions and case studies. Remote

Sensing of Environment, 103:27-42. (lays out the mathematics for different reflectance

measurements in an attempt to standardise the terminology in field spectroscopy)Shaw, D. T., T. J. Malthus, et al. (1998). High-spectral resolution data for monitoring Scots pine

(Pinus sylvestris L.) regeneration. International Journal of Remote Sensing 19(13): 2601-

2608. (Example of use of derivatives)

Smith, G. M., and Milton, E. J., (1999). The use of the empirical line method to calibrate

remotely sensed data to reflectance. International Journal of Remote Sensing, 20, 2653

2662.

Steven, M.D, Malthus, T.J., Baret, F., Xu, H., Chopping, M.J. (2003). Intercalibration of

vegetation indices from different sensor systems.Remote Sensing of Environment, 88(4):412-

422. (Example of application of sensor broad band filters to spectroradiometer data)

Tsai, F. and W. Philpot (1998). Derivative analysis of hyperspectral data. Remote Sensing of

Environment66: 41-51. (Use of derivatives)

Viscarra Rossel RA. 2008. ParLeS: Software for chemometric analysis of spectroscopic data.

Chemometrics and Intelligent Laboratory Systems, 90: 72-83. (more software)

Webster, R., P. J. Curran, et al. (1989). Spatial correlation in reflected radiation from the ground

and its implications for sampling and mapping by ground-based radiometry. Remote Sensing

of Environment29: 67-78. (One approach to the choosing sample sites, based on geostatistics

and the semi-variogram).

Workman, J.J. (1992). NIR spectroscopy calibration basics. In: Burns, D.A., Ciurczak, E.W.,

(Eds.), Handbook of near infrared analysis. Marcel Dekker, New York, pp. 247-280. (Near

infrared reflectance spectroscopy)

61


62/63

Underwater field spectroscopy references

Bukata et al. (1995) Optical properties and remote sensing of inland and coastal waters. CRC.

Dekker, A. G., Brando, V.E., Anstee, J.M., Fyfe, S., Malthus, T.J.M. & Karpouzli, E. (2006)

Remote sensing of seagrass ecosystems: use of spaceborne and airborne sensors, Chapter 15in : Larkum, A,., Orth, B and Duarte, C. (eds) Seagrass Biology, Ecology and Conservation ,

Springer Verlag, Germany: pp 630.

Dekker, A. G., V. E. Brando, J. M. Anstee, N. Pinnel, T. Kutser, H. J. Hoogenboom, R.

Pasterkamp, S. W. M. Peters, R. J. Vos, C. Olbert, and T. J. Malthus, (2001), Imaging

spectrometry of water, Ch. 11 in:Imaging Spectrometry: Basic principles and prospective

applications: Remote Sensing and Digital Image Processing, v. IV: Dordrecht, Kluwer

Academic Publishers, p. 307 - 359.

Green E.P., Mumby P.J., Edwards A.J., Clark C.D. (Ed. A.J. Edwards) (2000).Remote Sensing

Handbook for Tropical Coastal Management. Coastal Management Sourcebooks 3.

UNESCO, Paris. 316 pp.

Hooker, S.B. et al. (1994). Editor of SeaWIFS Technical Report series (available on the

SeaWIFS web site)

Jerlov (1976).Marine optics. Elsevier.

Kirk (1994).Light and photosynthesis in aquatic ecosystems. CUP.

Kutser, T., Dekker, A. G., Skirving, W. (2003) Modeling spectral discrimination of Great Barrier

Reef benthic communities by remote sensing instruments.Limnology & Oceanography

48:497-510

Mobley, C. (1994).Light and water: Radiative transfer in natural waters. Academic Press.

Mobley C., et al. (1993). Comparison of numerical models for computing underwater light

fields. Appl. Optics, 32:7484-7504.

Robinson, I.S. (1994) Satellite oceanography. Wiley

62


63/63

Note Pages

MacArthur_Introduction to Field Spectroscopy

Documents

Transcript of MacArthur_Introduction to Field Spectroscopy