The Camera Computational ?· Computational Photography ... 19th century camera Sony’s smile...

download The Camera Computational ?· Computational Photography ... 19th century camera Sony’s smile recognition…

of 28

  • date post

    15-Dec-2018
  • Category

    Documents

  • view

    217
  • download

    0

Embed Size (px)

Transcript of The Camera Computational ?· Computational Photography ... 19th century camera Sony’s smile...

1

1

Computational PhotographyComputational PhotographyComputer Science 203.4790Semester B 2009-2010Lecture: Sunday 12:00-15:00 Room: 303

Dr. Hagit Hel-Orhagit@cs.haifa.ac.ilOffice: 415Office Hours: by appointment

Course Internet Site: http://cs.haifa.ac.il/courses/compPhoto

2

The CameraThe Camera

A camera is a device that takes photos of images Camera Obscura (Latin = "dark chamber")

19th century camera Sonys smile recognition camera

3

Camera DevelopmentsCamera Developments

1825 19901913

permanent capturing

(wet plates)

1850

exposuretime and

motion capture(dry plates)

digital sensors

quality and size (35 mm)

1885

portability(film-Kodak)

1933

optics (SLR)

1950

instancy(polaroid)

2000

computationalphotography

1826 - Earliest surviving photograph. This image required an eight-hour exposure.

4

Computational PhotographyComputational Photography

Wikipedia:

Computational photography refers broadly to computational imaging techniques that enhance or extend the capabilities of digital photography. The output of these techniques is an ordinary photograph, but one that could not have been taken by a traditional camera.

Steve Mann: The Cyberman

2

5

Goal: Record a richer, multiGoal: Record a richer, multi--layered visual experiencelayered visual experience1.1. Overcome limitations of todayOvercome limitations of todays camerass cameras

2.2. Support better postSupport better post--capture processingcapture processing

3.3. Enables new classes of recording the visual signal Enables new classes of recording the visual signal

4.4. Synthesize Synthesize impossibleimpossible photosphotos

6

TopicsTopics

Project presentation

Light Field

Single View Modeling

Segmentation and Matting

Data Driven Synthesis

Multi exposure enhancement

Multi exposure enhancement

Appearance-based registration

Blending and Composition

Panoramas and feature-based registration

Single exposure enhancement

Acquisition and camera model

Intro and image formation

7

AdministrationAdministration

Pre-requisites / prior knowledge Course Home Page:

http://cs.haifa.ac.il/courses/compPhoto Messages Lecture slides and handouts Matlab guides Homework, Grades

Exercises: Programming in Matlab, ~3 Assignments Final project

8

Administration (Cont.)Administration (Cont.)

Matlab software: Available in PC labs Student version

Grading policy: Final Grade will be based on:

Exercises (40%) , Final project (60%) Exercises will be weighted Exercises can be submitted in pairs

Office Hours: by email appointment

3

9

Further ReadingsFurther Readings

Related papers New book: Computational

Photography by R. Raskarand J. Tumblin

10

SyllabusSyllabus Image Formation

Image formation HVS pathwayColor models

Acquisition and camera modelCamera model + perspective projectionsSensorsNoise models & DistortionsSampling (spatial+temporal) and quantizationCamera parametersCamera Parameters trade-offs.

Single exposure enhancementWhite BalancingDe-mosaicingDe-noisingDe-blurringGeometrical distortion correction

Panoramas and feature based registrationImage featuresSIFTPanoramasFeature based registrationPanoramasHomographyRANSAC Image stitching

11

Syllabus Syllabus cont.cont. Blending and Composition

Pyramid blendingOptimal cutSeam CarvingGraph-cutGradient domain editing

Appearance based registrationSimilarity measures Lucas Kanade optical flowMulti-modal registrationApplications

Multi exposure enhancement (2 weeks)HDRSuper-resolutionmulti-exposure fusion

Data Driven SynthesisTexture synthesisVideo textureQuiltingImage analogiesSuper-ResolutionImage Completion

12

Syllabus Syllabus cont.cont. Segmentation and Matting

Segmentation using Graph cut.mean-shiftSpectral clusteringInteractive and semi-automatic Matting

Single View ModelingCamera CalibrationMeasurements in affine camera3D reconstruction

Light FieldPlenoptic function and the LumiographRe-sampling the plenoptic function

4

13

1. Image Formation1. Image Formation

Taking a picture HVS pathway Color models

Optic NerveFovea

Vitreous

Optic Disc

Lens

Pupil

Cornea

Ocular MuscleRetina

Humor

Iris

14

2. Camera Model and Acquisition2. Camera Model and Acquisition

Perspective projections Camera pipeline and parameters Sensors and optics Sampling and quantization Noise models & Distortions Camera Parameters trade-offs.

15

3. Single Exposure Enhancement3. Single Exposure Enhancement

White Balancing De-mosaicing De-noising De-blurring Geometrical distortion correction

Difference in white point

16

4. Panoramas and Feature Based Registration4. Panoramas and Feature Based Registration

Image features SIFT Feature based registration Panoramas Homography RANSAC Image stitching

5

17

5. Blending and Composition5. Blending and Composition

Pyramid blending Gradient domain editing Optimal cut Graph-cut

18

6. Appearance Based Registration (warping?)6. Appearance Based Registration (warping?)

Similarity measures Lucas Kanade optical flow Multi-modal registration Applications

19

7. Multi Exposure Enhancement 7. Multi Exposure Enhancement

HDR Super-resolution Different-exposures fusion

20

8. Data Driven Synthesis 8. Data Driven Synthesis Texture synthesis Video texture Quilting Image analogies Super-Resolution Image Completion

6

21

9. Segmentation and Matting 9. Segmentation and Matting

Segmentation using Graph cut. Mean-shift Spectral clustering Interactive and semi-automatic Matting

22

10. Single View Modeling10. Single View Modeling Camera Calibration 3D reconstruction Metrology

Flagellation by Pietro della Francesca (1416-92, Italian Renaissance period)Animation by Criminisi et al., ICCV 99

23

11. Light Field11. Light Field

Plenoptic function and the Lumiograph Re-sampling the plenoptic function

24

Computational PhotographyComputational PhotographyTodayTodays Topic s Topic -- Image FormationImage Formation

What is an image ? What is color ?

7

25

Computer Vision

Rendering

Image/video Processing

Model3D ObjectGeometric Modeling

2D Images

The Visual SciencesThe Visual Sciences

26

What is an Image ?What is an Image ? An image is a projection of a 3D scene into a 2D projection plane. An image can be defined as a 2 variable function I(x,y) , where for

each position (x,y) in the projection plane, I(x,y) defines the light intensity at this point.

27

The Pinhole Camera ModelThe Pinhole Camera Model

Pinhole model: Captures pencil of rays all rays through a single point The point is called Center of Projection (COP) The image is formed on the Image Plane Effective focal length f - distance from COP to Image Plane

Slide by Steve Seitz

COP

Image plane

Focal length

28

Projection Model (where)Projection Model (where)

The coordinate system We will use the pin-hole model as an approximation Put the optical center (Center Of Projection) at the origin Put the image plane (Projection Plane) in front of the COP The camera looks down the negative z axis

Slide by Steve Seitz

8

29

Funny things happenFunny things happen

30

Parallel lines arenParallel lines arentt

31

Lengths canLengths cant be trusted...t be trusted...

32

The Shading Model (what)The Shading Model (what)

Shading Model: Given the illumination incident at a point on a surface, what is reflected?

9

33

ShadingShading Model ParametersModel Parameters

The factors determining the shading effects are:

The light source properties: Positions, Electromagnetic Spectrum, Shape.

The surface properties: Position, orientation, Reflectance properties.

The eye (camera) properties: Position, orientation, Sensor spectrum sensitivities.

34

Newtons Experiment, 1665 Cambridge.Discovering the fundamental spectral components of light.

Light and the Visible SpectrumLight and the Visible Spectrum

35 36

The light SpectrumThe light Spectrum

Electromagnetic Radiation - Spectrum

Gamma X rays Infrared Radar FM TV AMUltra-violet

10-12

10-8

10-4

104

1 108

electricityACShort-

wave

400 nm 500 nm 600 nm 700 nmWavelength in nanometers (nm)

Wavelength in meters (m)

Visible light

10

37

MonochromatorsMonochromators

Monochromators measure the power or energy at different wavelengths

38

The Spectral Power Distribution (SPD) of a light is a function e() which defines the energy at each wavelength.

Wavelength ()

400 500 600 7000

0.5

1

Rel

ativ

e P

ower

Light ParametersLight Parameters

39

ExamplesExamples of Spectral Power Distributionsof Spectral Power Dis