• date post

26-Apr-2020
• Category

Documents

• view

4

0

Embed Size (px)

Transcript of CS6640 Computational Photography - Cornell CS6640 Computational Photography 7. Digital camera...

• © 2012 Steve Marschner

CS6640 Computational Photography

7. Digital camera processing pipeline

1

• Cornell CS6640 Fall 2012

Camera processing pipeline • read image out from sensor —see Sensors lecture • optional: HDR assembly —see Homework 2 • color balance —see Color lecture • demosaic • noise processing • color matrix —see Color lecture • tone map

2

• Cornell CS6640 Fall 2012

Demosaicking • First question: how to spell it

I cite “picnicking” • Each photosite

senses only one color

• We need three measurements at each pixel

3 http://www.currentprotocols.com/WileyCDA/CPUnit/refId-ns0204.html

http://www.currentprotocols.com/WileyCDA/CPUnit/refId-ns0204.html http://www.currentprotocols.com/WileyCDA/CPUnit/refId-ns0204.html

• Cornell CS6640 Fall 2012

Bayer array • Simple solution: make a half-resolution image

• Want sensor-resolution image? Make up 2/3 of the data!

4

? ? ? ? ? ?? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ? ?? ? ?

? ?

? ?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

? ? ? ? ? ?

?

?

?

? ? ? ? ? ?

?

?

?

? ? ? ? ? ?

?

?

?

• Cornell CS6640 Fall 2012 5

• Cornell CS6640 Fall 2012 6

• Cornell CS6640 Fall 2012 6

bayer

• Cornell CS6640 Fall 2012 6

bayer bayer

• Cornell CS6640 Fall 2012 7

• Cornell CS6640 Fall 2012 8

• Cornell CS6640 Fall 2012 8

bayer

• Cornell CS6640 Fall 2012 8

bayer bayer

• Cornell CS6640 Fall 2012

Half-resolution demosaic • Idea 1: treat each block of four pixels as a pixel

Easy to code up in one line of Matlab. But what is wrong with this?

9

• Cornell CS6640 Fall 2012

Half-resolution demosaic • Idea 1: treat each block of four pixels as a pixel

Easy to code up in one line of Matlab. But what is wrong with this? 1. throws away too much resolution

9

• Cornell CS6640 Fall 2012

Half-resolution demosaic • Idea 1: treat each block of four pixels as a pixel

Easy to code up in one line of Matlab. But what is wrong with this? 1. throws away too much resolution 2. produces subpixel shifts in color planes!

9

• Cornell CS6640 Fall 2012 10

• Cornell CS6640 Fall 2012 10

bayer

• Cornell CS6640 Fall 2012 10

bayerblock

• Cornell CS6640 Fall 2012 10

bayerblock bayer

• Cornell CS6640 Fall 2012 10

bayerblock bayerblock

• Cornell CS6640 Fall 2012 11

• Cornell CS6640 Fall 2012 11

bayer

• Cornell CS6640 Fall 2012 11

bayerblock

• Cornell CS6640 Fall 2012 11

bayerblock bayer

• Cornell CS6640 Fall 2012 11

bayerblock bayerblock

• Cornell CS6640 Fall 2012

Centered half-resolution • Average pixels in groups that all have the same

“center of gravity” avoids major color fringing

12

? ? ? ? ? ?? ? ? ? ? ? ? ?? ? ? ? ? ? ? ? ?? ? ?

? ?

? ?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

?

? ? ? ?

?

?

?

? ? ? ? ? ?

?

?

?

? ? ? ? ? ?

?

?

?

• Cornell CS6640 Fall 2012 13

• Cornell CS6640 Fall 2012 13

bayer

• Cornell CS6640 Fall 2012 13

bayerblock

• Cornell CS6640 Fall 2012 13

bayerblockcentered

• Cornell CS6640 Fall 2012 13

bayerblockcentered bayer

• Cornell CS6640 Fall 2012 13

bayerblockcentered bayerblock

• Cornell CS6640 Fall 2012 13

bayerblockcentered bayerblockcentered

• Cornell CS6640 Fall 2012 14

naïve full-resdcraw naïve full-resdcraw

• Cornell CS6640 Fall 2012 14

bayernaïve full-resdcraw naïve full-resdcraw

• Cornell CS6640 Fall 2012 14

bayerblocknaïve full-resdcraw naïve full-resdcraw

• Cornell CS6640 Fall 2012 14

bayerblockcenterednaïve full-resdcraw naïve full-resdcraw

• Cornell CS6640 Fall 2012 14

bayerblockcenterednaïve full-resdcraw bayernaïve full-resdcraw

• Cornell CS6640 Fall 2012 14

bayerblockcenterednaïve full-resdcraw bayerblocknaïve full-resdcraw

• Cornell CS6640 Fall 2012 14

bayerblockcenterednaïve full-resdcraw bayerblockcenterednaïve full-resdcraw

• Cornell CS6640 Fall 2012

Naïve full-resolution interpolation • What if we don’t want to throw away so much sharpness?

15

? ? ? ? ?? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ?? ? ?

? ?

?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

? ?

? ? ?

?

?

?

? ? ? ? ? ?

?

?

?

? ? ? ? ? ?

?

?

?

• Cornell CS6640 Fall 2012 16

dcraw dcraw

• Cornell CS6640 Fall 2012 16

bayerdcraw dcraw

• Cornell CS6640 Fall 2012 16

bayerblockdcraw dcraw

• Cornell CS6640 Fall 2012 16

bayerblockcentereddcraw dcraw

• Cornell CS6640 Fall 2012 16

bayerblockcenterednaïve full-resdcraw dcraw

• Cornell CS6640 Fall 2012 16

bayerblockcenterednaïve full-resdcraw bayerdcraw

• Cornell CS6640 Fall 2012 16

bayerblockcenterednaïve full-resdcraw bayerblockdcraw

• Cornell CS6640 Fall 2012 16

bayerblockcenterednaïve full-resdcraw bayerblockcentereddcraw

• Cornell CS6640 Fall 2012 16

bayerblockcenterednaïve full-resdcraw bayerblockcenterednaïve full-resdcraw

• Cornell CS6640 Fall 2012 17

• Cornell CS6640 Fall 2012 17

bayer

• Cornell CS6640 Fall 2012 17

bayerblock

• Cornell CS6640 Fall 2012 17

bayerblockcentered

• Cornell CS6640 Fall 2012 17

bayerblockcenterednaïve full-res

• Cornell CS6640 Fall 2012 17

bayerblockcenterednaïve full-res bayer

• Cornell CS6640 Fall 2012 17

bayerblockcenterednaïve full-res bayerblock

• Cornell CS6640 Fall 2012 17

bayerblockcenterednaïve full-res bayerblockcentered

• Cornell CS6640 Fall 2012 17

bayerblockcenterednaïve full-res bayerblockcenterednaïve full-res

• slide by Frédo D urand, M

IT

Results of simple linear

• slide by Frédo D urand, M

IT

Results - not perfect

• slide by Frédo D urand, M

IT

Questions?

• slide by Frédo D urand, M

IT

The problem • Imagine a black-on-white corner • Let’s focus on the green channel for now

? ?

? ?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

• slide by Frédo D urand, M

IT

The problem • Imagine a black-on-white corner

? ?

? ?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

0 0 1 0 0 1

0 0 1 0 0 1

1 1 1 1 1 1

• slide by Frédo D urand, M

IT

The problem • Imagine a black-on-white corner

? ?

? ?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

0 0 1 0 0 1

0 0 1 0 0 1

1 1 1 1 1 1

0 0

0

1

.25 .75

.25

.75 1

0

.25

1

1

.1

1

0 0 1 0 0 1

0 0 1 0 0 1

1 1 1 1 1 1

.25 .75 .75

• slide by Frédo D urand, M

IT

The problem • Imagine a black-on-white corner

? ?

? ?

? ?

? ?

? ?

? ?

?

?

?

?

?

?

0 0 1 0 0 1

0 0 1 0 0 1

1 1 1 1 1 1

0 0

0

1

.25 .75

.25

.75 1

0

.25

1

1

.1

1

0 0 1 0 0 1

0 0 1 0 0 1

1 1 1 1 1 1

.25 .75 .75

• slide by Frédo D urand, M

IT

Yep, that’s what we saw

• slide by Frédo D urand, M

IT

Green channel

• slide by Frédo D urand, M

IT

Edge-based Demosaicking

• slide by Frédo D urand, M

IT

Idea • Take into account structure