Australian Indigenous Culture. Excursion update Review holiday homework Intro activities booklet.
MIT Camera Culture Group Update July 2009
-
Upload
cameraculture-mit-media-lab -
Category
Technology
-
view
6.570 -
download
1
Transcript of MIT Camera Culture Group Update July 2009
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureAssociate Professor, MIT Media Lab http://raskar.info
How can we create an entirely new class of imaging platforms
that have an understanding of the world that far exceeds human ability
and produce meaningful abstractions that are well within human comprehensibility ?
Ramesh Raskar http://raskar.info
Can you look around a corner ?
Can you decode a 5 micron feature Can you decode a 5 micron feature from 3 meters away from 3 meters away
with an ordinary camera ?with an ordinary camera ?
Beyond Multi-touch: Mobile 3D Interfaces?
6D DisplayLight sensitive 4D display
One Pixel of a 6D Display = 4D Display Raskar, Saakes, Fuchs, Siedel, Lensch, 2008
A. Introduction 5 minutes ‐‐
B. Cameras of the future (Raskar, 30 minutes) * Form factors, Modalities and Interaction * Enabling Visual Social Computing
C. Reconstruction the World (Seitz, 30 minutes) * Photo tourism and beyond * Image based modeling and rendering on a massive scale ‐* Scene summarization
D. Understanding a Billion Photos (Efros, 30 minutes) * What will the photos depict? * Photos as visual content for computer graphics * Solving computer vision
E. Discussion 10 minutes ‐‐
Course: Next Billion CamerasWedn at 3:30pm
Looking around a corner via Transient Imaging
Coded AperturePost-capture Focus
Control
Bokode: Long Distance Barcode
Camera Culture Group, MIT Media Lab Ramesh Raskar http://raskar.info
BiDi Screen
Light Field
Augmented Light Field
WDF
Theory of Light Propagation
International Conference on Computational Photography
Papers due November 2,
2009
http://cameraculture.media.mit.edu/iccp10
Traditional PhotographyTraditional Photography
Lens
Detector
Pixels
Image
Mimics Human Eye for a Single Snapshot:
Single View, Single Instant, Fixed Dynamic range and Depth of field for given Illumination in a Static world Courtesy: Shree
Nayar
Computational Illumination
Computational Camera
Scene: 8D Ray Modulator
Display
GeneralizedSensor
Generalized OpticsProcessing
4D Ray BenderUpto 4D
Ray Sampler
Ray Reconstruction
Generalized Optics
Recreate 4D Lightfield
Light Sources
Modulators
4D Incident Lighting
4D Light Field
Computational PhotographyComputational Photography
Computational Photography [Raskar and Tumblin]
1. Epsilon Photography– Low-level vision: Pixels– Multi-photos by perturbing camera parameters– HDR, panorama, …– ‘Ultimate camera’
2. Coded Photography– Mid-Level Cues:
• Regions, Edges, Motion, Direct/global– Single/few snapshot
• Reversible encoding of data– Additional sensors/optics/illum– ‘Scene analysis’
3. Essence Photography– High-level understanding
• Not mimic human eye• Beyond single view/illum
– ‘New artform’
Digital
Epsilon
Coded
Essence
Goal and Experience
Low Level Mid Level HighLevel
Hyper realism
Raw
Angle, spectrum
aware
Non-visual Data, GPS
Metadata
Priors
Comprehensive
Computational Photography aims to make progress on
both axis
Camera ArrayHDR, FoV Focal stack
Decomposition problems
Depth
Spectrum
LightFields
Human Stereo Vision
8D reflectance field
Dylan’s bicycle
Transient Imaging
Virtual Object Insertion
Relighting
Augmented Human
Experience
Material editing from single photo
Scene completion from photos
Motion Magnification
• Ramesh Raskar and Jack Tumblin
• Book Publishers: A K Peters• Siggraph 2009 booth: 20% off • Booth #2527
• ComputationalPhotography.org
• Meet the Authors• Thursday at 2pm-2:30pm
Can you look around a corner ?
Can you look around a corner ?
Kirmani, Hutchinson, Davis, Raskar 2009Accepted for ICCV’2009, Oct 2009 in Kyoto
Impulse Response of a Scene
Femtosecond Laser as Light SourcePico-second detector array as
Camera
Coded Aperture CameraCoded Aperture Camera
The aperture of a 100 mm lens is modified
Rest of the camera is unmodifiedInsert a coded mask with chosen binary pattern
Captured Blurred Photo
Refocused on Person
MIT media lab camera culture
Bokode:
ankit mohan, grace woo, shinsaku hiura,quinn smithwick, ramesh raskar
camera culture group, MIT media lab
imperceptible visual tags for camera based interaction from a distance
MIT media lab camera culture
Barcodesmarkers that assist machines in
understanding the real world
• Smart Barcode size : 3mm x 3mm• Ordinary Camera: Distance 3 meter
Computational Probes: Computational Probes: Long Distance Bar-codesLong Distance Bar-codes
Mohan, Woo,Smithwick, Hiura, RaskarAccepted as Siggraph 2009 paper
MIT Media Lab Camera Culture
Bokode
MIT Media Lab Camera Culture
Defocus blur of Bokode
MIT Media Lab Camera Culture
Image greatly magnified.
Simplified Ray Diagram
MIT Media Lab Camera Culture
Our Prototypes
MIT media lab camera culture
street-view tagging
MIT media lab camera culture
tabletop/surface interaction
MIT media lab camera culture
multi-user interaction
MIT media lab camera culture
Varying Exposure VideoAmit Agrawal MERL, Yi Xu Purdue, Ramesh Raskar, MIT
Deblurred Result
Blurred Photo
Varying Exposure Video
DFT
Exposure Time
Exposure Time
Exposure Time
Can we deal with particle-wave duality of light with modern Lightfield theory ?
45
Young’s Double Slit Expt
first null (OPD = λ/2)
Diffraction and Interferences modeled using Ray representation
Light Fields
•Radiance per ray•Ray parameterization:•Position : x •Direction : θ Reference
plane
positiondirection
Goal: Representing propagation, interaction and image formation of light using purely position and angle
parameters
Light Fields for Wave Optics EffectsLight Fields for Wave Optics Effects
Wigner Distribution Function
Light Field
LF < WDF
Lacks phase propertiesIgnores diffraction, phase masks
Radiance = Positive
Light Field
Augmented Light Field
WDF
ALF ~ WDF
Supports coherent/incoherent
Radiance = Positive/NegativeVirtual light sources
Limitations of Traditional Lightfields
Wigner Distribution Function
TraditionaTraditional Light l Light FieldField
ray optics basedsimple and powerful
rigorous but cumbersomewave optics based
limited in diffraction & interference
holograms
beam shaping
rotational PSF
Example: New Representations Augmented Lightfields
Wigner Distribution Function
TraditionaTraditional Light l Light FieldField
WDF
TraditionaTraditional Light l Light FieldField
Augmented LF
Interference & DiffractionInteraction w/ optical elements
ray optics basedsimple and powerfullimited in diffraction & interference
rigorous but cumbersomewave optics based
Non-paraxial propagation
http://raskar.scripts.mit.edu/~raskar/lightfields/
(ii) Augmented Light Field with LF Transformer
50
WDF
Light Light FieldField
Augmented LF
Interaction at the optical elements
LF propagatio
n
(diffractive)
optical element
LF LF LF LF
LF propagatio
n
light field transformer
negative radiance
Augmenting Light Field to Model Wave Optics Effects , [Oh, Barbastathis, Raskar]
Virtual light projector with real valued (possibly negative radiance) along a ray
51
real projector
real projector
first null (OPD = λ/2)
virtual light projector
(ii) ALF with LF Transformer
52
Converting LCD Screen = large Camera for 3D Interactive HCI and Video Conferencing
Matthew Hirsch, Henry HoltzmanDoug Lanman, Ramesh Raskar
BiDi Screen*
Light Sensing Pixels in LCD
Dis
play
with
em
bedd
ed o
ptic
al s
enso
rs
Sharp Microelectronics Optical Multi-touch Prototype
Beyond Multi-touch: Hover Interaction
• Seamless transition of multitouch to gesture
• Thin package, LCD
Beyond Multi-touch: Mobile
Laptops
Mobile
Design Vision
Object Collocated Captureand Display
Bare Sensor
Spa
tial L
ight
Mod
ulat
or
Touch + Hover using Depth Sensing LCD Sensor
Overview: Sensing Depth from Array of Virtual Cameras in
LCD
Design Overview
Dis
play
with
em
bedd
ed o
ptic
al s
enso
rs
LCD , displaying mask
Opt
ical
sen
sor a
rray
~2.5 cm~50 cm
International Conference on Computational Photography
Papers due November 2,
2009
http://cameraculture.media.mit.edu/iccp10
Looking around a corner via Transient Imaging
Coded AperturePost-capture Focus
Control
Bokode: Long Distance Barcode
Camera Culture Group, MIT Media Lab Ramesh Raskar http://raskar.info
BiDi Screen
Light Field
Augmented Light Field
WDF
Theory of Light Propagation