Introduction and course overview -...
Transcript of Introduction and course overview -...
![Page 1: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/1.jpg)
Coded photography
15-463, 15-663, 15-862Computational Photography
Fall 2018, Lecture 14http://graphics.cs.cmu.edu/courses/15-463
![Page 2: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/2.jpg)
Overview of today’s lecture
• The coded photography paradigm.
• Dealing with depth blur: coded aperture.
• Dealing with depth blur: focal sweep.
• Dealing with depth blur: generalized optics.
• Dealing with motion blur: coded exposure.
• Dealing with motion blur: parabolic sweep.
![Page 3: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/3.jpg)
Slide credits
Most of these slides were adapted from:
• Fredo Durand (MIT).• Anat Levin (Technion).• Gordon Wetzstein (Stanford).
![Page 4: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/4.jpg)
The coded photography paradigm
![Page 5: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/5.jpg)
Conventional photography
• Optics capture something that is (close to) the final image.• Computation mostly “enhances” captured image (e.g., deblur).
optics computationcaptured image enhanced imagereal world
![Page 6: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/6.jpg)
Coded photography
• Generalized optics encode world into intermediate representation.• Generalized computation decodes representation into multiple images.
generalized optics
generalized computation
coded representation of real world
final image(s)real world
? ??
Can you think of any examples?
![Page 7: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/7.jpg)
Early example: mosaicing
• Color filter array encodes color into a mosaic.• Demosaicing decodes color into RGB image.
generalized optics
generalized computation
coded representation of real world
final image(s)real world
CFA
dem
osa
icin
g
![Page 8: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/8.jpg)
Recent example: plenoptic camera
• Plenoptic camera encodes world into lightfield.• Lightfield rendering decodes lightfield into refocused or multi-viewpoint images.
generalized optics
generalized computation
coded representation of real world
final image(s)real world
Ligh
tfie
ldre
nd
erin
g
![Page 9: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/9.jpg)
Why are our images blurry?
• Lens imperfections.
• Camera shake.
• Scene motion.
• Depth defocus.
last lecture: deconvolution
last lecture: blind deconvolution
conventional photography
flutter shutter, motion-invariant photo
coded aperture, focal sweep, lattice lens
coded photography
![Page 10: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/10.jpg)
Why are our images blurry?
• Lens imperfections.
• Camera shake.
• Scene motion.
• Depth defocus.
last lecture: deconvolution
last lecture: blind deconvolution
conventional photography
flutter shutter, motion-invariant photo
coded aperture, focal sweep, lattice lens
coded photography
![Page 11: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/11.jpg)
Dealing with depth blur: coded aperture
![Page 12: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/12.jpg)
Defocus blur
object distance D focus distance D’
Point spread function (PSF): The blur kernel of a (perfect) lens at some out-of-focus depth.
blur kernel
What does the blur kernel depend on?
![Page 13: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/13.jpg)
Defocus blur
object distance D focus distance D’
Point spread function (PSF): The blur kernel of a (perfect) lens at some out-of-focus depth.
• Aperture determines shape of kernel.• Depth determines scale of blur kernel.
blur kernel
![Page 14: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/14.jpg)
Depth determines scale of blur kernel
object distance D focus distance D’
PSF
![Page 15: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/15.jpg)
Depth determines scale of blur kernel
object distance D focus distance D’
PSF
![Page 16: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/16.jpg)
Depth determines scale of blur kernel
object distance D focus distance D’
PSF
![Page 17: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/17.jpg)
Depth determines scale of blur kernel
object distance D focus distance D’
PSF
![Page 18: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/18.jpg)
Depth determines scale of blur kernel
object distance D focus distance D’
PSF
![Page 19: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/19.jpg)
Aperture determines shape of blur kernel
object distance D focus distance D’
PSF
![Page 20: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/20.jpg)
Aperture determines shape of blur kernel
PSF
photo of aperture shape of aperture (optical transfer function, OTF)
blur kernel(point spread function, PSF)
How do the OTF and PSF relate to each other?
What causes these lines?
![Page 21: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/21.jpg)
Removing depth defocus
input defocused imagemeasured PSFs at different depths
How would you create an all in-focus image given the above?
![Page 22: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/22.jpg)
Removing depth defocus
input defocused image
measured PSFs at different depths
How would you create an all in-focus image given the above?
Defocus is local convolution with a depth-dependent kernel
*
*
*
=
=
=
depth 3
depth 2
depth 1
![Page 23: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/23.jpg)
Removing depth defocus
input defocused image
measured PSFs at different depths
How would you create an all in-focus image given the above?
Defocus is local convolution with a depth-dependent kernel
*
*
*
=
=
=
depth 3
depth 2
depth 1
![Page 24: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/24.jpg)
Removing depth defocus• Deconvolve each image patch with all kernels• Select the right scale by evaluating the deconvolution results
*
*
*
=
=
=
-1
-1
-1
How do we select the
correct scale?
![Page 25: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/25.jpg)
Removing depth defocusProblem: With standard aperture, results at different scales look very similar.
*
*
*
=
=
=
-1
-1
-1
?correct scale
wrong scale
?correct scale
![Page 26: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/26.jpg)
Coded apertureSolution: Change aperture so that it is easier to pick the correct scale
*
*
*
=
=
=
-1
-1
-1
correct scale
wrong scale
wrong scale
![Page 27: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/27.jpg)
![Page 28: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/28.jpg)
![Page 29: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/29.jpg)
Coded aperture changes shape of kernel
object distance D focus distance D’
PSF
![Page 30: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/30.jpg)
Coded aperture changes shape of kernel
object distance D focus distance D’
PSF
![Page 31: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/31.jpg)
Coded aperture changes shape of PSF
![Page 32: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/32.jpg)
Coded aperture changes shape of PSF
New PSF preserves high frequencies• More content available to help us
determine correct depth
![Page 33: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/33.jpg)
Input
![Page 34: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/34.jpg)
All-focused (deconvolved)
![Page 35: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/35.jpg)
Comparison between standard and coded apertureRinging due to wrong scale estimation
![Page 36: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/36.jpg)
Comparison between standard and coded aperture
![Page 37: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/37.jpg)
Refocusing
![Page 38: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/38.jpg)
Refocusing
![Page 39: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/39.jpg)
Refocusing
![Page 40: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/40.jpg)
Depth estimation
![Page 41: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/41.jpg)
Input
![Page 42: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/42.jpg)
All-focused (deconvolved)
![Page 43: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/43.jpg)
Refocusing
![Page 44: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/44.jpg)
Refocusing
![Page 45: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/45.jpg)
Refocusing
![Page 46: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/46.jpg)
Depth estimation
![Page 47: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/47.jpg)
Any problems with using a coded aperture?
![Page 48: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/48.jpg)
Any problems with using a coded aperture?• We lose a lot of light due to blocking.
• The deconvolution becomes harder due to more diffraction/zeros in frequency domain.
• We still need to select correct scale.
![Page 49: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/49.jpg)
Dealing with depth blur: focal sweep
![Page 50: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/50.jpg)
The difficulty of dealing with depth defocusvarying in-focus distance
At every focus setting, objects at different depths are blurred by different PSF
![Page 51: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/51.jpg)
The difficulty of dealing with depth defocusvarying in-focus distance
PSFs for object at depth 1
At every focus setting, objects at different depths are blurred by different PSF
![Page 52: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/52.jpg)
The difficulty of dealing with depth defocusvarying in-focus distance
PSFs for object at depth 1
PSFs for object at depth 2
At every focus setting, objects at different depths are blurred by different PSF
![Page 53: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/53.jpg)
The difficulty of dealing with depth defocusvarying in-focus distance
PSFs for object at depth 1
PSFs for object at depth 2
At every focus setting, objects at different depths are blurred by different PSF
As we sweep through focus settings, each point every object
is blurred by all possible PSFs
![Page 54: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/54.jpg)
Focal sweepvarying in-focus distance
PSFs for object at depth 1
PSFs for object at depth 2
What is the effective PSF in this case?
Go through all focus settings during a single exposure
![Page 55: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/55.jpg)
Focal sweepvarying in-focus distance
Anything special about these effective PSFs?
⎰ dt =
⎰ dt =
effective PSF for object at depth 1
effective PSF for object at depth 2
Go through all focus settings during a single exposure
![Page 56: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/56.jpg)
Focal sweep
The effective PSF is:1. Depth-invariant – all points are blurred the same way regardless of depth.2. Never sharp – all points will be blurry regardless of depth.
What are the implications of this?1. The image we capture will be sharp nowhere; but2. We can use simple (global) deconvolution to sharpen parts we want
1. Can we estimate depth from this?2. Can we do refocusing from this?
![Page 57: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/57.jpg)
Focal sweep
The effective PSF is:1. Depth-invariant – all points are blurred the same way regardless of depth.2. Never sharp – all points will be blurry regardless of depth.
What are the implications of this?
![Page 58: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/58.jpg)
Focal sweep
The effective PSF is:1. Depth-invariant – all points are blurred the same way regardless of depth.2. Never sharp – all points will be blurry regardless of depth.
What are the implications of this?1. The image we capture will not be sharp anywhere; but2. We can use simple (global) deconvolution to sharpen parts we want
1. Can we estimate depth from this?2. Can we do refocusing from this?
![Page 59: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/59.jpg)
Focal sweep
The effective PSF is:1. Depth-invariant – all points are blurred the same way regardless of depth.2. Never sharp – all points will be blurry regardless of depth.
What are the implications of this?1. The image we capture will not be sharp anywhere; but2. We can use simple (global) deconvolution to sharpen parts we want
1. Can we estimate depth from this?2. Can we do refocusing from this?
Depth-invariance of the PSF means that we have lost all depth information
![Page 60: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/60.jpg)
How can you implement focal sweep?
![Page 61: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/61.jpg)
How can you implement focal sweep?
Use translation stage to move sensor relative to fixed lens during exposure
Rotate focusing ring to move lens relative to fixed sensor during exposure
![Page 62: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/62.jpg)
Comparison of different PSFs
![Page 63: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/63.jpg)
Depth of field comparisons
![Page 64: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/64.jpg)
Any problems with using focal sweep?
![Page 65: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/65.jpg)
Any problems with using focal sweep?• We have moving parts (vibrations, motion blur).
• Perfect depth invariance requires very constant speed.
• We lose depth information.
![Page 66: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/66.jpg)
Dealing with depth blur: generalized optics
![Page 67: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/67.jpg)
Change optics, not aperture
object distance D focus distance D’
PSF
![Page 68: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/68.jpg)
Wavefront coding
object distance D focus distance D’
Replace lens with a cubic phase plate
![Page 69: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/69.jpg)
Wavefront coding
• Rays no longer converge.• Approximately depth-invariant PSF for certain range of depths.
standard lens
wavefront coding
![Page 70: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/70.jpg)
Lattice lens
object distance D focus distance D’
Add lenslet array with varying focal length in front of lens
![Page 71: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/71.jpg)
Lattice lens
Does this remind you of something?
![Page 72: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/72.jpg)
Lattice lens
• Effectively captures only the “useful” subset of the 4D lightfield.
• PSF is not depth-invariant, so local deconvolution as in coded aperture.
Light field spectrum: 4DImage spectrum: 2DDepth: 1D
→ Dimensionality gap (Ng 05)
Only the 3D manifold corresponding to physical focusing distance is useful
3D
PSFs at different depths
![Page 73: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/73.jpg)
ResultsStandard lens
![Page 74: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/74.jpg)
ResultsLattice lens
![Page 75: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/75.jpg)
ResultsStandard lens
![Page 76: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/76.jpg)
ResultsLattice lens
![Page 77: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/77.jpg)
ResultsStandard lens
![Page 78: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/78.jpg)
ResultsLattice lens
![Page 79: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/79.jpg)
Refocusing example
![Page 80: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/80.jpg)
Refocusing example
![Page 81: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/81.jpg)
Refocusing example
![Page 82: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/82.jpg)
Comparison of different techniques
Object at in-focus depth
standardlens
codedaperture
focalsweep
wavefront coding
latticelens
< << < <
Object at extreme depth
Depth of field comparison:
![Page 83: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/83.jpg)
Diffusion coded photography
Can you think of any issues?
![Page 84: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/84.jpg)
Dealing with motion blur
![Page 85: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/85.jpg)
Why are our images blurry?
• Lens imperfections.
• Camera shake.
• Scene motion.
• Depth defocus.
last lecture: deconvolution
last lecture: blind deconvolution
conventional photography
flutter shutter, motion-invariant photo
coded aperture, focal sweep, lattice lens
coded photography
![Page 86: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/86.jpg)
Motion blur
Most scene is static
Can moving linearly from left to right
![Page 87: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/87.jpg)
Motion blur
motion blur kernelblurry image of moving object
sharp image of static object
*=
What does the motion blur kernel depend on?
![Page 88: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/88.jpg)
Motion blur
motion blur kernelblurry image of moving object
sharp image of static object
*=
What does the motion blur kernel depend on?• Motion velocity determines direction of kernel.• Shutter speed determines width of kernel.
Can we use deconvolution to remove motion blur?
![Page 89: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/89.jpg)
Challenges of motion deblurring
• Blur kernel is not invertible.
• Blur kernel is unknown.
• Blur kernel is different for different objects.
![Page 90: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/90.jpg)
Challenges of motion deblurring
• Blur kernel is not invertible.
• Blur kernel is unknown.
• Blur kernel is different for different objects.
How would you deal with this?
![Page 91: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/91.jpg)
Dealing with motion blur: coded exposure
![Page 92: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/92.jpg)
Coded exposure a.k.a. flutter shutter
Code exposure (i.e., shutter speed) to make motion blur kernel better conditioned.
motion blur kernelblurry image of moving object
sharp image of static object
*=
motion blur kernelblurry image of moving object
sharp image of static object
*=
traditional camera
flutter-shutter camera
![Page 93: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/93.jpg)
How would you implement coded exposure?
![Page 94: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/94.jpg)
How would you implement coded exposure?
very fast external shutter
electronics for external shutter control
![Page 95: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/95.jpg)
Coded exposure a.k.a. flutter shuttermotion blur kernel
in time domain
motion blur kernel in Fourier domain
Why is flutter shutter better?
![Page 96: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/96.jpg)
Coded exposure a.k.a. flutter shuttermotion blur kernel
in time domain
motion blur kernel in Fourier domain
Why is flutter shutter better?
zeros make inverse filter unstable
inverse filter is stable
![Page 97: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/97.jpg)
Motion deblurring comparison
blurry input
deconvolved output
conventional photography flutter-shutter photography
![Page 98: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/98.jpg)
![Page 99: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/99.jpg)
![Page 100: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/100.jpg)
Challenges of motion deblurring
• Blur kernel is not invertible.
• Blur kernel is unknown.
• Blur kernel is different for different objects.
How would you deal with these two?
![Page 101: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/101.jpg)
Dealing with motion blur: parabolic sweep
![Page 102: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/102.jpg)
Motion-invariant photography
Introduce extra motion so that:• Everything is blurry; and• The blur kernel is motion invariant (same for all objects).
How would you achieve this?
![Page 103: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/103.jpg)
Parabolic sweep
![Page 104: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/104.jpg)
Hardware implementation
Approximate small translation by small rotation
variable radius cam
Rotating platform
Lever
![Page 105: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/105.jpg)
Some results
parabolic input - blur is invariant to velocity
static camera input - unknown and variable blur
![Page 106: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/106.jpg)
Some results
output after deconvolution
Is this blind or non-blind deconvolution?
static camera input - unknown and variable blur
![Page 107: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/107.jpg)
Some results
static camera input parabolic camera input deconvolution output
![Page 108: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/108.jpg)
Some results
output after deconvolutionWhy does it fail in this case?
static camera input
![Page 109: Introduction and course overview - graphics.cs.cmu.edugraphics.cs.cmu.edu/courses/15-463/lectures/lecture14b.pdf · Point spread function (PSF): The blur kernel of a (perfect) lens](https://reader031.fdocuments.net/reader031/viewer/2022022718/5c41c83293f3c338d75522c1/html5/thumbnails/109.jpg)
ReferencesBasic reading:• Levin et al., “Image and depth from a conventional camera with a coded aperture,” SIGGRAPH 2007.• Veeraraghavan et al., “Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing,”
SIGGRAPH 2007.The two papers introducing coded aperture for depth and refocusing, the first covers deblurring in more detail, whereas the second deals with optimal mask selection and includes very interesting lightfield analysis.
• Nagahara et al., “Flexible depth of field photography,” ECCV 2008 and PAMI 2010.The focal sweep paper.
• Dowski and Cathey, “Extended depth of field through wave-front coding,” Applied Optics 1995.The wavefront coding paper.
• Levin et al., “4D Frequency Analysis of Computational Cameras for Depth of Field Extension,” SIGGRAPH 2009.The lattice focal lens paper, which also includes a discussion of wavefront coding.
• Cossairt et al., “Diffusion Coded Photography for Extended Depth of Field,” SIGGRAPH 2010.The diffusion coded photography paper.
• Raskar et al., “Coded Exposure Photography: Motion Deblurring using Fluttered Shutter,” SIGGRAPH 2006.The flutter shutter paper.
• Levin et al., “Motion-Invariant Photography,” SIGGRAPH 2008.The motion-invariant photography paper.
Additional reading:• Zhang and Levoy, “Wigner distributions and how they relate to the light field,” ICCP 2009.
This paper has a nice discussion of wavefront coding, in addition to analysis of lightfields and their relationship to wave optics concepts.
• Gehm et al., “Single-shot compressive spectral imaging with a dual-disperser architecture,” Optics Express 2007.This paper introduces the use of coded apertures for hyperspectral imaging, instead of depth and refocusing.