OUTLINEOUTLINE What is Photography? What is Photography? What is ‘The Photographic Signal’? What...

of 23 /23
OUTLINE OUTLINE What is Photography? What is Photography? What is ‘The Photographic Signal’? What is ‘The Photographic Signal’? Perfecting Film-Like Photography: Perfecting Film-Like Photography: Old Problems, New Approaches Old Problems, New Approaches Photography Beyond Film: Photography Beyond Film: New Goals, Methods, Expressions New Goals, Methods, Expressions

Embed Size (px)

Transcript of OUTLINEOUTLINE What is Photography? What is Photography? What is ‘The Photographic Signal’? What...

  • Slide 1
  • OUTLINEOUTLINE What is Photography? What is Photography? What is The Photographic Signal? What is The Photographic Signal? Perfecting Film-Like Photography: Old Problems, New Approaches Perfecting Film-Like Photography: Old Problems, New Approaches Photography Beyond Film: New Goals, Methods, Expressions Photography Beyond Film: New Goals, Methods, Expressions
  • Slide 2
  • Course 15: Computational Photography Course 15: Computational Photography
  • Slide 3
  • Jack Tumblin Northwestern University A2: Core Concepts (15 minutes) Computational Photography
  • Slide 4
  • Film-Like Photography Film Camera design assumptions: Instantaneous light measurement Of focal plane image behind a lens. Reproduce those amounts of light. Implied: What we see is focal-plane intensities. well, nowe see much more! (seeing is deeply cognitive) What we see is focal-plane intensities. well, nowe see much more! (seeing is deeply cognitive)
  • Slide 5
  • Our Definitions Film-like Photography: Film-like Photography: Static instantaneous record of the 2D image formed by a lens Display image sensor image Computational Photography: Computational Photography: displayed image sensor image A more expressive, controllable displayed result, from transformed, merged, decoded sensor data from transformed, merged, decoded sensor data
  • Slide 6
  • What is Photography? A bucket word: a neat container for messy notions (e.g. aviation, music, comprehension) A bucket word: a neat container for messy notions (e.g. aviation, music, comprehension) A record of what we see, or would like to see, in tangible form. A record of what we see, or would like to see, in tangible form. Does film photography always capture it? Does film photography always capture it? no. So, what do we see? So, what do we see? Harold Doc Edgerton 1936
  • Slide 7
  • Display RGB(x,y,t n ) Image I(x,y,,t) Light & Optics 3D Scene light sources, BRDFs, shapes, positions, movements, Eyepoint position, movement, projection, PHYSICALPERCEIVED What is Photography? Exposure Control, tone map Scene light sources, BRDFs, shapes, positions, movements, Eyepoint position, movement, projection, Vision Photo: A Tangible Record Editable, storable as Film or Pixels
  • Slide 8
  • 3D Scene? light sources, BRDFs, shapes, positions, movements, Eyepoint? position, movement, projection, Meaning VisualStimulus 3D Scene light sources, BRDFs, shapes, positions, movements, Eyepoint position, movement, projection, PHYSICAL PERCEIVED or UNDERSTOOD Ultimate Photographic Goals Vision Sensor(s) Computing Light & Optics Photo: A Tangible Record Scene estimates we can capture, edit, store, display
  • Slide 9
  • Missing: Viewpoint Freedom Multiple-Center-of-Projection Images Rademacher, P, Bishop, G., SIGGRAPH '98
  • Slide 10
  • Missing: Reliable Visual Boundaries 5 ray sets explicit geometric occlusion boundaries Ramesh Raskar, MERL, 2004
  • Slide 11
  • Missing: Expressive Time Manipulations What other ways better reveal appearance to human viewers? (Without direct shape measurement? ) Time for space wiggle. Time for space wiggle. Gasparini, 1998. Can you understand this shape better? this shape better?
  • Slide 12
  • Photographic Signal: Pixels Rays Core ideas are ancient, simple, seem obvious: Core ideas are ancient, simple, seem obvious: Lighting: ray sources Optics: ray bending/folding devices Sensor: measure light Processing: assess it Display: reproduce it Ancient Greeks: eye rays wipe the world to feel its contents Ancient Greeks: eye rays wipe the world to feel its contents http://www.mlahanas.de/Greeks/Optics.htm
  • Slide 13
  • The Photographic Signal Path Computing can improve every component: Light Sources Sensors Data Types, Processing Display Rays OpticsOptics Scene Rays
  • Slide 14
  • Review: How many Rays in a 3-D Scene? A 4-D set of infinitesimal members. Imagine: Convex Enclosure of a 3D scene Inward-facing ray camera at every surface point Pick the rays you need for ANY camera outside. 2D surface of cameras, 2D ray set for each camera, 4D set of rays. 4D set of rays. (Levoy et al. SIGG96) (Gortler et al. 96) +
  • Slide 15
  • 4-D Light Field / Lumigraph Measure all the outgoing light rays.
  • Slide 16
  • 4-D Illumination Field Same Idea: Measure all the incoming light rays
  • Slide 17
  • 4D x 4D = 8-D Reflectance Field Ratio: R ij = (outgoing ray i ) / (incoming ray j )
  • Slide 18
  • Future Photography: Novel Illuminators Novel Cameras Scene : 8D Ray Modulator Generalized Sensor Generalized Processing 4D Ray Sampler Ray Reconstructor General Optics: 4D Ray Benders Recreated 4D Light field Lights Modulators 4D Incident Lighting Viewed 4D Light Field General Optics: 4D Ray Benders Generalized Display Generalized Display Novel Displays
  • Slide 19
  • Expand Optics Into Software Programmable Optical Devices enable new forms of: Programmable Optical Devices enable new forms of: [ Nayar ] Omni-Directional Lens Systems Omni-Directional Lens Systems (Hi-Def 360 o video) Assorted Pixels Sensors (Robust HDR, multispectral) Assorted Pixels Sensors (Robust HDR, multispectral) Lensless Adaptive-Aperture Cameras (tracking without panning) Lensless Adaptive-Aperture Cameras (tracking without panning)
  • Slide 20
  • Expand Scientific / Medical Imaging 4D light sources + 4D cameras enable new forms of: 4D light sources + 4D cameras enable new forms of: [ Levoy ] Synthetic Aperture Imaging (see through trees) Synthetic Aperture Imaging (see through trees) Tomography (3-D volumetric imaging) Tomography (3-D volumetric imaging) Confocal Scanning (look inside muddy water) Confocal Scanning (look inside muddy water)
  • Slide 21
  • The Ideal Photographic Signal I CLAIM IT IS: All Rays? Some Rays? Changes in Some Rays All Rays? Some Rays? Changes in Some Rays Photographic ray space is vast and redundant >8 dimensions: 4D view, 4D light, time,, ? Gather only visually significant ray changes ? ? What rays should we measure ? ? How should we combine them ? ? How should we display them ?
  • Slide 22
  • Beyond Film-Like Photography Call it Computational Photography: To make meaningful ray changes tangible, To make meaningful ray changes tangible, Optics can do more Optics can do more Sensors can do more Sensors can do more Light Sources can do more Light Sources can do more Processing can do more Processing can do more by applying low-cost storage, computation, and control. by applying low-cost storage, computation, and control.
  • Slide 23
  • Course 15: Computational Photography Course 15: Computational Photography A.2: Concepts Tumblin