Introduction to Computational Photography. Computational Photography Digital Camera What is...


of 25

  • date post

  • Category

  • view

  • download


Embed Size (px)


  • Introduction toComputational Photography

  • What isComputational Photography?Second breakthrough by ITFirst : electronic image sensor (digital camera)Digital representation of image formed by lensSecond : Re-definition of whole camera (optics, usage)Image is reconstructed by computationImageprocessingImageImage sensorOpticsWhole part of camera is affected by computational photographyFilm cameraDigital Camera

  • Light field(Light space)What is camera?Camera is a machine to record the distribution of the light in a sceneHow to represent the distribution of the light in the scene?3-D coordinate of the point where the light passing through : X, Y, ZDirection of the light : Wavelength of the light (color of the light) : Time : tThe 7 parameters function P which represent the distribution of the light is called Plenoptic functionLight sourceObject

  • Light field (light ray)Optics (lens)Image sensor (pixel)Integration of cameraCamera integrates the light for all 7 parametersPositionrange of X, Y, Z : aperture size should not be zero)Directionrange of : pixel size is not zero)Wavelength range of : No single wavelength filter)Exposure time ( range of t : shutter speed should not be too fast)Multiple samples - ,:number of pixel, RGBt : burst shotSo, what is multiple sampling for X, Y, Z?

  • Camera arrayMeasuring the distribution of the light at multiple position

    ProFUSION25 (ViewPlus, Inc.)The Stanford Multi-Camera Array(Marc Levoy @Stanford University)

  • Use of camera arrayFree-viewpoint imageDefocus generation by synthetic aperture3-D video (Matsuyama lab, Kyoto Univ.)Synthetic aperture (Vaish@Stanford)

  • Defocus control by Uncalibrated Synthetic ApertureNatsumi Kusumoto, Shinsaku Hiura and Kosuke Sato, Uncalibrated Synthetic Aperture for Defocus Control, CVPR2009 (Jun. 2009)

  • Reviewing integrationSome part of information is lost by integrationSine wave which period is just as same as the integration duration is lost

    Blur of object within an exposure timeDefocus by misfocus= 0

  • Coded ExposureCoded exposure : exposure is coded in time axisFlutter Shutter Camera (Raskar@MERL)

  • This TraditionalCoded ExposureImage of Static ObjectDeblurred ImageDeblurred ImageSlide by R. Raskar

  • Coded ExposureTemporal 1-D broadband code: Motion DeblurringSlide by R. Raskar

  • Captured Blurred PhotoSlide by R. Raskar

  • Refocused on PersonSlide by R. Raskar

  • Coded ApertureLevin@MIT(2007)Depth estimation by single image (manual operation is necessary)

  • Coded ApertureLevin@MIT(2007)

  • Coded ApertureLevin@MIT(2007)

  • Multi-focus camera withCoded ApertureStabilizing the depth estimation and deblur by coded apertureSimultaneous capture of 3 images with different focused distanceHiura et al, CVPR(1998),SSII(1999)

  • Multi-Focus Range Sensorusing Coded Aperture

  • Invariant integrationDefocus : changed according to the distance

    Blur : changed according to the speed of the object

    Reconstruction is not easy because the estimation of the speed or distance is necessary Is it possible to make defocus or blur invariant to the distance or speed?

  • Invariant integrationDefocusSpecial optics : Wavefront Coding

    Motion of the image sensor while exposureBlurReciprocal motion of the cameraCDM Optics, Inc.

  • Motion of the image sensorfor invariant defocusH. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008

  • H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008Motion of the image sensor for invariant defocus

  • Deblur by reciprocal motion of the cameraA. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008. Input imageDeblurred image

  • A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008. EquipmentConceptual figure forLight sources withdifferent speedDeblur by reciprocal motion of the camera

  • More..Resources on wwwWikipedia : computational photography Conference on Computational PhotographySIGGRAPH, CVPR, .. Session about computational photography

    *Comparisons**Reversibly encode all the information in this otherwise blurred photo*The glint out of focus shows the unusual pattern.