Light field microscopy Marc Levoy, Ren Ng, Andrew Adams Matthew Footer, Mark Horowitz Stanford...
-
date post
21-Dec-2015 -
Category
Documents
-
view
240 -
download
4
Transcript of Light field microscopy Marc Levoy, Ren Ng, Andrew Adams Matthew Footer, Mark Horowitz Stanford...
Light field microscopy
Marc Levoy, Ren Ng, Andrew Adams
Matthew Footer, Mark Horowitz
Stanford ComputerGraphics Laboratory
Marc Levoy
Executive summary
• captures the 4D light field inside a microscope
• yields perspective flyarounds and focal stacks from a single snapshot, but at lower spatial resolution
• focal stack → deconvolution microscopy → volume data
Marc Levoy
Devices for recording light fields
smallscenes
bigscenes
• handheld camera [Buehler 2001]
• camera gantry [Stanford 2002]
• array of cameras [Wilburn 2005]
• plenoptic camera [Ng 2005]
• light field microscope (this paper)
(using geometrical optics)
Marc Levoy
Light fields at micron scales
• wave optics must be considered– diffraction limits the spatial × angular resolution
• most objects are no longer opaque– each pixel is a line integral through the object
» of attenuation
» or emission
– can reconstruct 3D structure from these integrals» tomography
» 3D deconvolution
Marc Levoy
Digitally moving the observer
• moving the observer = moving the window we extract from the microlenses
Σ
Σ
Marc Levoy
A light field microscope (LFM)
• 40x / 0.95NA objective
↓
0.26μ spot on specimen× 40x = 10.4μ on sensor
↓
2400 spots over 25mm field
• 1252-micron microlenses
↓
200 × 200 microlenses with12 × 12 spots per microlens
objective
specimen
intermediateimage plane
eyepiecesensor
→ reduced lateral resolution on specimen= 0.26μ × 12 spots = 3.1μ
Marc Levoy
Example light field micrograph
• orange fluorescent crayon
• mercury-arc source + blue dichroic filter
• 16x / 0.5NA (dry) objective
• f/20 microlens array
• 65mm f/2.8 macro lens at 1:1
• Canon 20D digital camera
ordinary microscope light field microscope
200μ
Marc Levoy
The geometry of the light fieldin a microscope
• microscopes make orthographic views
• translating the stage in X or Y provides no parallax on the specimen
• out-of-plane features don’t shift position when they come into focus
f
objective lensesare telecentric
Marc Levoy
Axial resolution(a.k.a. depth of field)
• wave term + geometrical optics term
• ordinary microscope (16x/0.4NA (dry), e = 0)
• with microlens array (e = 125μ)
• stopped down to one pixel per microlens
eNAM
n
NA
nλDOFtot
2
3.34.0
1535.02
8.225.193.31254.016
1
4.0
1535.02
237spots125.193.3
→ number of slicesin focal stack
= 12
(geometrical optics dominates)
(wave optics dominates)
Marc Levoy
3D reconstruction
• confocal scanning [Minsky 1957]
• shape-from-focus [Nayar 1990]
• deconvolution microscopy [Agard 1984]
– 4D light field → digital refocusing →3D focal stack → deconvolution microscopy →3D volume data
(UMIC SUNY/Stonybrook) (Noguchi) (DeltaVision)
Marc Levoy
3D deconvolution
• object * PSF → focus stack {object} × {PSF} → {focus stack} {focus stack} {PSF} → {object}
• spectrum contains zeros, due to missing rays
• imaging noise is amplified by division by ~zeros
• reduce by regularization, e.g. smoothing
focus stack of a point in 3-space is the 3D PSF of that imaging system
[McNally 1999]
{PSF}
Marc Levoy
Silkworm mouth(40x / 1.3NA oil immersion)
slice of focal stack slice of volume volume rendering
100μ
Marc Levoy
3D reconstruction (revisited)
• 4D light field → digital refocusing →3D focal stack → deconvolution microscopy →3D volume data
• 4D light field → tomographic reconstruction →3D volume data
(from Kak & Slaney)
(DeltaVision)
Marc Levoy
Implications of this equivalence
• light fields of minimally scattering volumes contain only 3D worth of information, not 4D
• the extra dimension serves to reduce noise, but could be re-purposed?
OpticalProjectionTomography[Sharpe 2002]
Marc Levoy
Conclusions
• captures 3D structure of microscopic objects in a single snapshot, and at a single instant in time
Calcium fluorescent imagingof zebrafish larvae optic tectumduring changing visual stimula
Marc Levoy
Conclusions
• captures 3D structure of microscopic objects in a single snapshot, and at a single instant in time
but...
• sacrifices spatial resolution to obtain control over viewpoint and focus
• 3D reconstruction fails if specimen is too thick or too opaque
Marc Levoy
Future work
• extending the field of view by correcting digitally for objective aberrations
Nikon 40x 0.95NA (dry) Plan-Apo
Marc Levoy
Future work
• extending the field of view by correcting digitally for objective aberrations
• microlenses in the illumination path→ an imaging microscope scatterometer
200μ
angular dependenceof reflection fromsingle squid iridophore