Introduction toComputational Photography
What isComputational Photography?Second breakthrough by ITFirst : electronic image sensor (digital camera)Digital representation of image formed by lensSecond : Re-definition of whole camera (optics, usage)Image is reconstructed by computationImageprocessingImageImage sensorOpticsWhole part of camera is affected by computational photographyFilm cameraDigital Camera
Light field(Light space)What is camera?Camera is a machine to record the distribution of the light in a sceneHow to represent the distribution of the light in the scene?3-D coordinate of the point where the light passing through : X, Y, ZDirection of the light : Wavelength of the light (color of the light) : Time : tThe 7 parameters function P which represent the distribution of the light is called Plenoptic functionLight sourceObject
Light field (light ray)Optics (lens)Image sensor (pixel)Integration of cameraCamera integrates the light for all 7 parametersPositionrange of X, Y, Z : aperture size should not be zero)Directionrange of : pixel size is not zero)Wavelength range of : No single wavelength filter)Exposure time ( range of t : shutter speed should not be too fast)Multiple samples - ,:number of pixel, RGBt : burst shotSo, what is multiple sampling for X, Y, Z?
Camera arrayMeasuring the distribution of the light at multiple position
ProFUSION25 (ViewPlus, Inc.)The Stanford Multi-Camera Array(Marc Levoy @Stanford University)
Use of camera arrayFree-viewpoint imageDefocus generation by synthetic aperture3-D video (Matsuyama lab, Kyoto Univ.)Synthetic aperture (Vaish@Stanford)
Defocus control by Uncalibrated Synthetic ApertureNatsumi Kusumoto, Shinsaku Hiura and Kosuke Sato, Uncalibrated Synthetic Aperture for Defocus Control, CVPR2009 (Jun. 2009)
Reviewing integrationSome part of information is lost by integrationSine wave which period is just as same as the integration duration is lost
Blur of object within an exposure timeDefocus by misfocus= 0
Coded ExposureCoded exposure : exposure is coded in time axisFlutter Shutter Camera (Raskar@MERL)
This TraditionalCoded ExposureImage of Static ObjectDeblurred ImageDeblurred ImageSlide by R. Raskar
Coded ExposureTemporal 1-D broadband code: Motion DeblurringSlide by R. Raskar
Captured Blurred PhotoSlide by R. Raskar
Refocused on PersonSlide by R. Raskar
Coded ApertureLevin@MIT(2007)Depth estimation by single image (manual operation is necessary)
Multi-focus camera withCoded ApertureStabilizing the depth estimation and deblur by coded apertureSimultaneous capture of 3 images with different focused distanceHiura et al, CVPR(1998),SSII(1999)
Multi-Focus Range Sensorusing Coded Aperture
Invariant integrationDefocus : changed according to the distance
Blur : changed according to the speed of the object
Reconstruction is not easy because the estimation of the speed or distance is necessary Is it possible to make defocus or blur invariant to the distance or speed?
Invariant integrationDefocusSpecial optics : Wavefront Coding
Motion of the image sensor while exposureBlurReciprocal motion of the cameraCDM Optics, Inc.
Motion of the image sensorfor invariant defocusH. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008
H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, Flexible Depth of Field Photography, ECCV2008Motion of the image sensor for invariant defocus
Deblur by reciprocal motion of the cameraA. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008. Input imageDeblurred image
A. Levin, P. Sand, T. S. Cho, F. Durand, W. T. Freeman. Motion-Invariant Photography. SIGGRAPH2008. EquipmentConceptual figure forLight sources withdifferent speedDeblur by reciprocal motion of the camera
More..Resources on wwwWikipedia : computational photographyhttp://computationalphotography.org/http://www1.cs.columbia.edu/CAVE/projects/what_is/http://projects.csail.mit.edu/photo/ConferencesInternational Conference on Computational PhotographySIGGRAPH, CVPR, .. Session about computational photography
*Comparisons**Reversibly encode all the information in this otherwise blurred photo*The glint out of focus shows the unusual pattern.