Compositing & Blending

download Compositing & Blending

of 157

  • date post

    31-Jan-2017
  • Category

    Documents

  • view

    221
  • download

    0

Embed Size (px)

Transcript of Compositing & Blending

  • Compositing & Blending

    Tamara Berg Computational Photography

    Many slides from: Alexei Efros & Derek Hoiem

  • Image Compositing & Blending

    NASA

  • This Class How do I put an object from one image into another?

    Slide from: Derek Hoiem

  • Image Compositing

  • Recent news

    Original

    Enhanced Version

    http://www.guardian.co.uk/world/2010/sep/16/mubarak-doctored-red-carpet-picture

    Slide from: Derek Hoiem

  • Compositing Procedure 1. Extract Sprites (e.g using Intelligent Scissors in Photoshop)

    Composite by David Dewey

    2. Blend them into the composite (in the right order)

  • Three methods

    1. Cut and paste

    2. Laplacian pyramid blending

    3. Poisson blending

    Slide from: Derek Hoiem

  • Method 1: Cut and Paste

    Slide from: Derek Hoiem

  • Method 1: Cut and Paste Method: Segment using intelligent scissors Paste foreground pixels onto target region

    Slide from: Derek Hoiem

  • Method 1: Cut and Paste

    Problems: Small segmentation errors noticeable Pixels are too blocky Wont work for semi-transparent materials

    Slide from: Derek Hoiem

  • Feathering Near object boundary pixel values come partly from foreground and partly from background

    Slide from: Derek Hoiem

  • Method 1: Cut and Paste (with feathering)

    Slide from: Derek Hoiem

  • Alpha compositing

    +

    Output = foreground*mask + background*(1-mask) Slide from: Derek Hoiem

  • Alpha compositing with feathering

    Output = foreground*mask + background*(1-mask) Slide from: Derek Hoiem

  • Proper blending is key

  • Proper blending is key

  • Alpha Blending / Feathering

    0 1

    0 1

    +

    = Iblend = Ileft + (1-)Iright

  • Setting alpha: simple averaging

    Alpha = .5 in overlap region

  • Setting alpha: center seam

    Alpha = logical(dtrans1>dtrans2)

    Distance Transform bwdist!

  • Setting alpha: blurred seam

    Alpha = blurred

    Distance transform

  • Setting alpha: center weighting

    Alpha = dtrans1 / (dtrans1+dtrans2)

    Distance transform

    Ghost!

  • Alpha Blending / Feathering

    0 1

    0 1

    +

    = Iblend = Ileft + (1-)Iright

  • Affect of Window Size

    0

    1 left

    right 0

    1

    Large windows -> lots of ghosting

    alpha

  • Affect of Window Size

    0

    1

    0

    1

    Small window > seams are too obvious

  • Good Window Size

    0

    1

    Optimal Window: smooth but not ghosted

    Medium window -> just right!

  • What is the Optimal Window? To avoid seams

    window = size of largest prominent feature

  • What is the Optimal Window? To avoid seams

    window = size of largest prominent feature

    To avoid ghosting window

  • What is the Optimal Window? To avoid seams

    window = size of largest prominent feature

    To avoid ghosting window

  • Fourier domain blending

    FFT

  • Fourier domain blending

    Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright)

    FFT

  • Fourier domain blending

    Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

    Fleft = Fleft1 + Fleft2 +

    FFT

  • Fourier domain blending

    Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

    Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti

    FFT

  • Fourier domain blending

    Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

    Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti Sum feathered octave images in frequency domain

    FFT

  • Fourier domain blending

    Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

    Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti Sum feathered octave images in frequency domain Compute IFFT to get final blended result.

    FFT

  • Fourier domain blending

    Idea (Burt and Adelson) Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands)

    Fleft = Fleft1 + Fleft2 + Feather corresponding octaves Flefti with Frighti Sum feathered octave images in frequency domain Compute IFFT to get final blended result.

    FFT

    Actually better implemented in the spatial domain

  • Image Pyramids

    Known as a Gaussian Pyramid [Burt and Adelson, 1983] In computer graphics, a mip map [Williams, 1983]

    - Reminder

  • A bar in the big images is a hair on the zebras nose; in smaller images, a stripe; in the smallest, the animals nose

    Figure from David Forsyth

    Gaussian Pyramid

  • Laplacian Pyramid

    Lowpass Images

  • Laplacian Pyramid

    Bandpass Images

    Lowpass Images

  • Laplacian Pyramid

    Lowpass Images

  • Laplacian Pyramid

    Lowpass Images

  • Laplacian Pyramid

    Lowpass Images

    Bandpass Images

  • Pyramid Blending

    0

    1

    0

    1

    0

    1

    Left pyramid Right pyramid blend

    At low frequencies, blend slowly At high frequencies, blend quickly

  • Method 2: Laplacian Pyramid Blending

    Implementation: 1.Build Laplacian pyramids for each image 2.Build a Gaussian pyramid of region mask 3.Blend each level of pyramid using region mask

    from the same level

    4.Collapse the pyramid to get the final blended image

    )1(2112iiiii RLRLL +=

    Burt and Adelson 1983

    Region mask at level i of Gaussian pyramid

    Image 1 at level i of Laplacian pyramid

  • laplacian level

    4

    laplacian level

    2

    laplacian level

    0

    left pyramid right pyramid blended pyramid

  • Pyramid Blending

  • Blending Regions

  • Horror Photo

    david dmartin (Boston College)

  • Season Blending (St. Petersburg)

  • Season Blending (St. Petersburg)

  • Simplification: Two-band Blending Brown & Lowe, 2003

    Only use two bands: high freq. and low freq. Blends low freq. smoothly Blend high freq. with no smoothing: use binary alpha

  • Low frequency ( > 2 pixels)

    High frequency ( < 2 pixels)

    2-band Blending

  • Linear Blending

  • 2-band Blending

  • Practice Quiz Poisson Image Editing: 1) Solving the Poisson equation computes the function whose gradient is the closest, in the L2 norm, to some prescribed vector field under given boundary conditions In blending: What does function refer to? What is an example prescribed vector field?

    2) How might you implement texture flattening using Poisson techniques? Photomontage: 3) What are the 2 main techniques used in this paper?

    4) What is an image objective? Name 3 possible objectives. Seam Carving: 5) What is the definition of a seam?

    6) What algorithm would you use to efficiently find the optimal seam for removal in an image? Describe generally how to implement this.

  • Break

  • Related idea: Poisson Blending A good blend should preserve gradients of source region without changing the background

    Perez et al. 2003

  • Related idea: Poisson Blending A good blend should preserve gradients of source region without changing the background

    Perez et al. 2003

    HW 2!

  • Whats with Poisson?

    Solving Poisson computes the function whose gradient is the closest, in the L2 norm, to some prescribed vector field under given boundary conditions

    Whats that mean?

    Given a guidance field (e.g. gradient field from source image, or from a composite image)

    Calculate the image that best fits this vector field

    In general no image exists whose gradient exactly matches input. Best-fit image in least squares sense can be calculated by solving Poisson equation.

    Perez et al. 2003

  • HW2: Gradient Domain Editing

    General concept: Solve for pixels of new image that satisfy constraints on the gradient and the intensity Constraints can be from one image (for filtering) or more (for

    blending)

  • HW2: Poisson Blending

    A good blend should preserve gradients of source region without changing the background

    Treat pixels as variables to be solved

    Minimize squared difference between gradients of foreground region and gradients of target region

    Keep background pixels constant

    Perez et al. 2003

  • Example

    Source: Evan Wallace

    Gradient Visualization

  • Source: Evan Wallace

    + Specify object region

  • Other results

    Perez et al. 2003

  • What do we lose?

    Perez et al. 2003

    Foreground color changes Background pixels in target region are replaced

  • Blending with Mixed Gradients Use foreground or background gradient with

    larger magnitude as the guiding gradient

    Perez et al. 2003

  • Concealment

  • Transparent Objects

  • NPR

  • Things t