Environment Matting and Compositing

CS 395 – Advanced Graphics Presented by Jay Hetler

Douglas E. Zongker ~ Dawn M. Werner ~ Brian Curless ~ David H. Salsin

What is Matting?•Extracting the foreground image from the background•Describing the opacity of the foreground at every point

What is Compositing?•Holding back the part of the new background that theforeground blocks.•Placing the foreground element in a new background

Original Uses

Weather Reports Cinema Special Effects

Matting Equation

C = F + (1- )B

• C ~ Color• F ~ Foreground color• B ~ Background color• ~ Amount of light that passes through the

foreground

How difficult is Matting?

Matte Image

Good Results?

Problems with Matting

Images do not look realistic

• Lack of Refracted Light

• Lack of Reflected Light

Solution:Modify the Matting Equation

Environment Matting Equation

C = F + (1- )B + • C ~ Color• F ~ Foreground color• B ~ Background color• ~ Amount of light that passes through the

foreground• ~ Contribution of light from Environment

that travels through the object

Explanation of

= R M(T , A )R ~ Reflectance coefficient

M ~ Texture mapping operator

for axis-aligned rectangle (A)

of texture (T)

i = 1i i i

m

Environment Mattes

Performance

• Calibration

• Matting: 10-20 minutes extraction time for each texture map (Pentium II 400Mhz)

• Compositing: 4-40 frames per second

• Real-Time?

How much better is Environment Matting?

Alpha Matte Environment Matte Photograph

How much better is Environment Matting?

Alpha Matte Environment Matte Photograph

Movies!

Importance of Sidedrops

Light comes from other parts of the environment

Sidedrop Example

Sidedrop x2

Failure Cases

Box-filter Gaussian-weighted Photograph

Failure Cases

Composite Photograph

Failure of Engineering

Failure of Theory

Next Steps

• Depth

• Accuracy

• Real-Time Capture

Progress?Depth Accuracy

No Real-Time