# Compositing & Blending

date post

31-Jan-2017Category

## Documents

view

221download

0

Embed Size (px)

### Transcript of Compositing & Blending

Compositing & Blending

Tamara Berg Computational Photography

Many slides from: Alexei Efros & Derek Hoiem

Image Compositing & Blending

NASA

This Class How do I put an object from one image into another?

Slide from: Derek Hoiem

Image Compositing

Recent news

Original

Enhanced Version

http://www.guardian.co.uk/world/2010/sep/16/mubarak-doctored-red-carpet-picture

Slide from: Derek Hoiem

Compositing Procedure 1. Extract Sprites (e.g using Intelligent Scissors in Photoshop)

Composite by David Dewey

2. Blend them into the composite (in the right order)

Three methods

1. Cut and paste

2. Laplacian pyramid blending

3. Poisson blending

Slide from: Derek Hoiem

Method 1: Cut and Paste

Slide from: Derek Hoiem

Method 1: Cut and Paste Method: Segment using intelligent scissors Paste foreground pixels onto target region

Slide from: Derek Hoiem

Method 1: Cut and Paste

Problems: Small segmentation errors noticeable Pixels are too blocky Wont work for semi-transparent materials

Slide from: Derek Hoiem

Feathering Near object boundary pixel values come partly from foreground and partly from background

Slide from: Derek Hoiem

Method 1: Cut and Paste (with feathering)

Slide from: Derek Hoiem

Alpha compositing

+

Output = foreground*mask + background*(1-mask) Slide from: Derek Hoiem

Alpha compositing with feathering

Output = foreground*mask + background*(1-mask) Slide from: Derek Hoiem

Proper blending is key

Proper blending is key

Alpha Blending / Feathering

0 1

0 1

+

= Iblend = Ileft + (1-)Iright

Setting alpha: simple averaging

Alpha = .5 in overlap region

Setting alpha: center seam

Alpha = logical(dtrans1>dtrans2)

Distance Transform bwdist!

Setting alpha: blurred seam

Alpha = blurred

Distance transform

Setting alpha: center weighting

Alpha = dtrans1 / (dtrans1+dtrans2)

Distance transform

Ghost!

Alpha Blending / Feathering

0 1

0 1

+

= Iblend = Ileft + (1-)Iright

Affect of Window Size

0

1 left

right 0

1

Large windows -> lots of ghosting

alpha

Affect of Window Size

0

1

0

1

Small window > seams are too obvious

Good Window Size

0

1

Optimal Window: smooth but not ghosted

Medium window -> just right!

What is the Optimal Window? To avoid seams

window = size of largest prominent feature

What is the Optimal Window? To avoid seams

window = size of largest prominent feature

To avoid ghosting window

What is the Optimal Window? To avoid seams

window = size of largest prominent feature

To avoid ghosting window

Fourier domain blending

FFT

Fourier domain blending

Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright)

FFT

Fourier domain blending

Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

Fleft = Fleft1 + Fleft2 +

FFT

Fourier domain blending

Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti

FFT

Fourier domain blending

Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti Sum feathered octave images in frequency domain

FFT

Fourier domain blending

Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti Sum feathered octave images in frequency domain Compute IFFT to get final blended result.

FFT

Fourier domain blending

Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti Sum feathered octave images in frequency domain Compute IFFT to get final blended result.

FFT

Actually better implemented in the spatial domain

Image Pyramids

Known as a Gaussian Pyramid [Burt and Adelson, 1983] In computer graphics, a mip map [Williams, 1983]

- Reminder

A bar in the big images is a hair on the zebras nose; in smaller images, a stripe; in the smallest, the animals nose

Figure from David Forsyth

Gaussian Pyramid

Laplacian Pyramid

Lowpass Images

Laplacian Pyramid

Bandpass Images

Lowpass Images

Laplacian Pyramid

Lowpass Images

Laplacian Pyramid

Lowpass Images

Laplacian Pyramid

Lowpass Images

Bandpass Images

Pyramid Blending

0

1

0

1

0

1

Left pyramid Right pyramid blend

At low frequencies, blend slowly At high frequencies, blend quickly

Method 2: Laplacian Pyramid Blending

Implementation: 1.Build Laplacian pyramids for each image 2.Build a Gaussian pyramid of region mask 3.Blend each level of pyramid using region mask

from the same level

4.Collapse the pyramid to get the final blended image

)1(2112iiiii RLRLL +=

Burt and Adelson 1983

Region mask at level i of Gaussian pyramid

Image 1 at level i of Laplacian pyramid

laplacian level

4

laplacian level

2

laplacian level

0

left pyramid right pyramid blended pyramid

Pyramid Blending

Blending Regions

Horror Photo

david dmartin (Boston College)

Season Blending (St. Petersburg)

Season Blending (St. Petersburg)

Simplification: Two-band Blending Brown & Lowe, 2003

Only use two bands: high freq. and low freq. Blends low freq. smoothly Blend high freq. with no smoothing: use binary alpha

Low frequency ( > 2 pixels)

High frequency ( < 2 pixels)

2-band Blending

Linear Blending

2-band Blending

Practice Quiz Poisson Image Editing: 1) Solving the Poisson equation computes the function whose gradient is the closest, in the L2 norm, to some prescribed vector field under given boundary conditions In blending: What does function refer to? What is an example prescribed vector field?

2) How might you implement texture flattening using Poisson techniques? Photomontage: 3) What are the 2 main techniques used in this paper?

4) What is an image objective? Name 3 possible objectives. Seam Carving: 5) What is the definition of a seam?

6) What algorithm would you use to efficiently find the optimal seam for removal in an image? Describe generally how to implement this.

Break

Related idea: Poisson Blending A good blend should preserve gradients of source region without changing the background

Perez et al. 2003

Related idea: Poisson Blending A good blend should preserve gradients of source region without changing the background

Perez et al. 2003

HW 2!

Whats with Poisson?

Solving Poisson computes the function whose gradient is the closest, in the L2 norm, to some prescribed vector field under given boundary conditions

Whats that mean?

Given a guidance field (e.g. gradient field from source image, or from a composite image)

Calculate the image that best fits this vector field

In general no image exists whose gradient exactly matches input. Best-fit image in least squares sense can be calculated by solving Poisson equation.

Perez et al. 2003

HW2: Gradient Domain Editing

General concept: Solve for pixels of new image that satisfy constraints on the gradient and the intensity Constraints can be from one image (for filtering) or more (for

blending)

HW2: Poisson Blending

A good blend should preserve gradients of source region without changing the background

Treat pixels as variables to be solved

Minimize squared difference between gradients of foreground region and gradients of target region

Keep background pixels constant

Perez et al. 2003

Example

Source: Evan Wallace

Gradient Visualization

Source: Evan Wallace

+ Specify object region

Other results

Perez et al. 2003

What do we lose?

Perez et al. 2003

Foreground color changes Background pixels in target region are replaced

Blending with Mixed Gradients Use foreground or background gradient with

larger magnitude as the guiding gradient

Perez et al. 2003

Concealment

Transparent Objects

NPR

Things t

*View more*