Biomedical Imaging

Hyperspectral Imaging

Charles A. DiMarzio & Eric M. Kercher

EECE–4649

Northeastern University

Universidad de los Andes

June 2019

Why Spectral?

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–1

Why Hyperspectral

Right ≈ North, Down = [400 to 2400 nm] (not Linear)

South Bay of California; 101 curves down on the left.

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–2

Why Hyperspectral

Right ≈ North, Down = [400 to 2400 nm] (not Linear)

Where are the Shrimp?

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–3

Hardware

• Tunable Filter (Lyot Filter, Pronounced “Leo”)

– x, y on camera

– λ with time

• Grating spectrometer with Pinhole

– λ on Camera

– x, y with time (whiskbroom: Slow)

∗ Slit, Grating and 2D Camera

· x, λ on camera

· y with time (pushbroom)

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–4

Applications

• Military (Where’s the Tank in the Trees?)

• Law Enforcement (Which crop is illegal?)

• Environmental (e.g. Deep Horizon)

• Biomedical

– Fluorescence Spectroscopy (Multiple, Overlapping Fluo-

rophores)

– Hemoglobin Spectroscopy

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–5

Forward Models

• Monte–Carlo

• Diffusion

• FDTD

• Linear Spectral Prediction

Yλ =N∑

n=1

Mλ,nXn

• Backscatter

Y = µa M = κn (λ)

• Fluorescence

Y = Eλ M = Eλ,n

• Intuitive Backscatter

1

W=

µs + µa

µs

1

W= 1+

µa

µs

• or µaℓeff = ln (1/T )

• Linear Superposition

µa (λ) =N∑

n=1

κn (λ)Xn

• Fluorescence

Source Powers Add

Eλ (λ) =N∑

n=1

Eλ,n (λ)Xn

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–6

Some CommonAbsorbers

200 300 400 500 600 700 800 900 1000

,Wavelength, nm

10-4

10-3

10-2

10-1

100

101

102

103

104

a,

Ab

sorp

tio

n C

oef

f, /

cm

Oxy in Skin

Deoxy

Oxy in Blood

Deoxy

Water

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–7

Some CommonFluorophores

200 300 400 500 600 700

, Wavelength, nm

-2

0

2

4

6

8

10

12

a,

Sp

ecif

ic A

bso

rpti

on

(cm

M)

- 1

104 Dye Spectra (file m10254.m)

488 514 633

Hoechst 33258

FITC Isomer 1

Mito Green FM

Mito CMXRos

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–8

Matrix Methods

• Forward Problem

Y = MX

• Typical Example

Y (λ1)

Y (λ2)

Y (λ3)

Y (λ4)...

Y (λ200)

=

M1,1 M1,2 M1,3

M2,1 M2,2 M2,3

M3,1 M3,2 M3,3

M4,1 M4,2 M4,3... ... ...

M200,1 M200,2 M200,3

X1

X2

X3

• Inverse Problem

X = M−1Y Oh No!!!

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–9

Variance, Covariance

• Variance:

Mean of Squared Difference from Mean

• Covariance:

Mean of Squared Difference

• Each Pixel (k) is a Column

Y =

Y1 (λ1) Y2 (λ1) . . .

Y1 (λ2) Y2 (λ1) . . .

Y1 (λ3) Y2 (λ1) . . .... . . .

Y1 (λ200) Y2 (λ200) . . .

COV (Y) = YY†

• We can visualize this in 3D space

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–10

Variance, Covariance,Principal Components

• Covariance

COV (Y) = YY†

• Find Eigenvalues and Eigenvectors

• Arrange in Eigenvalue Order Decending

• Remember Y = MX for a pixel or Y = MX for an image

• It’s Possible from this to Guess

– M, the basis comonent spectra in columns

– X , the strength of each component in each pixel

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–11

Non–Negative LeastSquares

Real–Time NLSS

Eric M. Kercher

NU Physics Ph.D. Candidate

DOC Teaching Assistant 2018

Application to Skin Imaging

Jaime Prieto & Matias Rivera

(UAndes)

Everett O’Malley (NU, UCSB)

June 2019 Chuck DiMarzio, Northeastern University 12286..slides7–12