A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… ·...
Transcript of A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… ·...
![Page 1: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/1.jpg)
A Gentle Introductionto Bilateral Filteringand its Applications
A Gentle Introductionto Bilateral Filteringand its Applications
Sylvain Paris – MIT CSAIL
Pierre Kornprobst – INRIA Odyssée
Jack Tumblin – Northwestern University
Frédo Durand – MIT CSAIL
![Page 2: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/2.jpg)
• The bilateral filter is becoming in computational photography.
• Many applications with high quality results.
![Page 3: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/3.jpg)
Photographic Style Transfer[Bae 06]Photographic Style Transfer[Bae 06]
input
![Page 4: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/4.jpg)
Photographic Style Transfer[Bae 06]Photographic Style Transfer[Bae 06]
output
![Page 5: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/5.jpg)
Tone Mapping[Durand 02]Tone Mapping[Durand 02]
HDR input
![Page 6: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/6.jpg)
Tone Mapping[Durand 02]Tone Mapping[Durand 02]
output
![Page 7: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/7.jpg)
input
Cartoon Rendition[Winnemöller 06]Cartoon Rendition[Winnemöller 06]
![Page 8: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/8.jpg)
Cartoon Rendition[Winnemöller 06]Cartoon Rendition[Winnemöller 06]
output
6 papers atSIGGRAPH’07
6 papers atSIGGRAPH’07
![Page 9: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/9.jpg)
Goal: Image SmoothingGoal: Image Smoothing
Split an image into:
• large-scale features, structure
• small-scale features, texture
![Page 10: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/10.jpg)
input smoothed(structure, large scale)
residual(texture, small scale)
Gaussian Convolution
BLUR HALOS
Naïve Approach: Gaussian BlurNaïve Approach: Gaussian Blur
![Page 11: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/11.jpg)
Impact of Blur and HalosImpact of Blur and Halos• If the decomposition introduces blur and
halos, the final result is corrupted.
Sample manipulation:increasing texture
(residual × 3)
![Page 12: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/12.jpg)
input smoothed(structure, large scale)
residual(texture, small scale)
edge-preserving: Bilateral Filter
Bilateral Filter: no Blur, no HalosBilateral Filter: no Blur, no Halos
![Page 13: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/13.jpg)
input
![Page 14: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/14.jpg)
increasing texturewith Gaussian convolution
H A L O S
![Page 15: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/15.jpg)
increasing texturewith bilateral filter
N O H A L O S
![Page 16: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/16.jpg)
Notation and DefinitionsNotation and Definitions
• Image = 2D array of pixels
• Pixel = intensity (scalar) or color (3D vector)
• Ip = value of image I at position: p = ( px , py )
• F [ I ] = output of filter F applied to image I
x
y
![Page 17: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/17.jpg)
Strategy for Smoothing ImagesStrategy for Smoothing Images
• Images are not smooth because adjacent pixels are different.
• Smoothing = making adjacent pixelslook more similar.
• Smoothing strategypixel → average of its neighbors
![Page 18: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/18.jpg)
Box AverageBox Average
average
input
square neighborhood
output
![Page 19: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/19.jpg)
sum overall pixels q
normalizedbox function
intensity atpixel q
result atpixel p
Equation of Box AverageEquation of Box Average
∑∈
−=S
IBIBAq
qp qp )(][ σ
0
![Page 20: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/20.jpg)
Square Box Generates Defects Square Box Generates Defects
• Axis-aligned streaks
• Blocky results
input
output
![Page 21: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/21.jpg)
unrelatedpixels
unrelatedpixels
relatedpixels
Box ProfileBox Profile
pixelposition
pixelweight
![Page 22: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/22.jpg)
Strategy to Solve these ProblemsStrategy to Solve these Problems
• Use an isotropic (i.e. circular) window.
• Use a window with a smooth falloff.
box window Gaussian window
![Page 23: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/23.jpg)
Gaussian BlurGaussian Blur
average
input
per-pixel multiplication
output*
![Page 24: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/24.jpg)
input
![Page 25: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/25.jpg)
box average
![Page 26: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/26.jpg)
Gaussian blur
![Page 27: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/27.jpg)
normalizedGaussian function
Equation of Gaussian BlurEquation of Gaussian Blur
( )∑∈
−=S
IGIGBq
qp qp ||||][ σ
Same idea: weighted average of pixels.
0
1
![Page 28: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/28.jpg)
unrelatedpixels
unrelatedpixels
uncertainpixels
uncertainpixels
relatedpixels
Gaussian ProfileGaussian Profile
pixelposition
pixelweight
⎟⎟⎠
⎞⎜⎜⎝
⎛−= 2
2
2exp
21)(
σπσσxxG
![Page 29: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/29.jpg)
size of the window
Spatial ParameterSpatial Parameter
( )∑∈
−=S
IGIGBq
qp qp ||||][ σ
small σ large σ
input
limited smoothing strong smoothing
![Page 30: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/30.jpg)
How to set σHow to set σ
• Depends on the application.
• Common strategy: proportional to image size– e.g. 2% of the image diagonal
– property: independent of image resolution
![Page 31: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/31.jpg)
Properties of Gaussian BlurProperties of Gaussian Blur
• Weights independent of spatial location
– linear convolution
– well-known operation
– efficient computation (recursive algorithm, FFT…)
![Page 32: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/32.jpg)
Properties of Gaussian BlurProperties of Gaussian Blur
• Does smooth images
• But smoothes too much:edges are blurred.– Only spatial distance matters
– No edge term
input
output
( )∑∈
−=S
IGIGBq
qp qp ||||][ σspace
![Page 33: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/33.jpg)
Blur Comes from Averaging across EdgesBlur Comes from Averaging across Edges
*
*
*
input output
Same Gaussian kernel everywhere.
![Page 34: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/34.jpg)
Bilateral FilterNo Averaging across EdgesBilateral FilterNo Averaging across Edges
*
*
*
input output
The kernel shape depends on the image content.
[Aurich 95, Smith 97, Tomasi 98]
![Page 35: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/35.jpg)
space weight
not new
range weight
I
new
normalizationfactor
new
Bilateral Filter Definition:an Additional Edge TermBilateral Filter Definition:an Additional Edge Term
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
Same idea: weighted average of pixels.
![Page 36: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/36.jpg)
Illustration a 1D ImageIllustration a 1D Image
• 1D image = line of pixels
• Better visualized as a plot
pixelintensity
pixel position
![Page 37: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/37.jpg)
space
Gaussian Blur and Bilateral FilterGaussian Blur and Bilateral Filter
space rangenormalization
Gaussian blur
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
Bilateral filter[Aurich 95, Smith 97, Tomasi 98]
space
spacerange
p
p
q
q
( )∑∈
−=S
IGIGBq
qp qp ||||][ σ
![Page 38: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/38.jpg)
qqpp
Bilateral Filter on a Height FieldBilateral Filter on a Height Field
output input
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
pp
reproducedfrom [Durand 02]
![Page 39: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/39.jpg)
Space and Range ParametersSpace and Range Parameters
• space σs : spatial extent of the kernel, size of the considered neighborhood.
• range σr : “minimum” amplitude of an edge
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
![Page 40: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/40.jpg)
Influence of PixelsInfluence of Pixels
pp
Only pixels close in space and in range are considered.
space
range
![Page 41: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/41.jpg)
σs = 2
σs = 6
σs = 18
σr = 0.1 σr = 0.25σr = ∞
(Gaussian blur)
input
Exploring the Parameter SpaceExploring the Parameter Space
![Page 42: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/42.jpg)
σs = 2
σs = 6
σs = 18
σr = 0.1 σr = 0.25σr = ∞
(Gaussian blur)
input
Varying the Range ParameterVarying the Range Parameter
![Page 43: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/43.jpg)
input
![Page 44: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/44.jpg)
σr = 0.1
![Page 45: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/45.jpg)
σr = 0.25
![Page 46: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/46.jpg)
σr = ∞(Gaussian blur)
![Page 47: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/47.jpg)
σs = 2
σs = 6
σs = 18
σr = 0.1 σr = 0.25σr = ∞
(Gaussian blur)
input
Varying the Space ParameterVarying the Space Parameter
![Page 48: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/48.jpg)
input
![Page 49: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/49.jpg)
σs = 2
![Page 50: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/50.jpg)
σs = 6
![Page 51: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/51.jpg)
σs = 18
![Page 52: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/52.jpg)
How to Set the ParametersHow to Set the Parameters
Depends on the application. For instance:
• space parameter: proportional to image size– e.g., 2% of image diagonal
• range parameter: proportional to edge amplitude– e.g., mean or median of image gradients
• independent of resolution and exposure
![Page 53: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/53.jpg)
A FewMore Advanced
Remarks
A FewMore Advanced
Remarks
![Page 54: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/54.jpg)
Bilateral Filter Crosses Thin LinesBilateral Filter Crosses Thin Lines• Bilateral filter averages across
features thinner than ~2σs
• Desirable for smoothing: more pixels = more robust• Different from diffusion that stops at thin lines
close-up kernel
![Page 55: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/55.jpg)
Iterating the Bilateral FilterIterating the Bilateral Filter
• Generate more piecewise-flat images
• Often not needed in computational photo.
][ )()1( nn IBFI =+
![Page 56: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/56.jpg)
input
![Page 57: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/57.jpg)
1 iteration
![Page 58: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/58.jpg)
2 iterations
![Page 59: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/59.jpg)
4 iterations
![Page 60: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/60.jpg)
Bilateral Filtering Color ImagesBilateral Filtering Color Images
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
( ) ( )∑∈
−−=S
GGW
IBFq
qqpp
p CCCqp ||||||||1][rs σσ
For gray-level images
For color images
intensity difference
color difference
The bilateral filter isThe bilateral filter isextremely easy to adapt to your need.extremely easy to adapt to your need.
scalar
3D vector (RGB, Lab)
input
output
![Page 61: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/61.jpg)
Hard to ComputeHard to Compute
• Nonlinear
• Complex, spatially varying kernels– Cannot be precomputed, no FFT…
• Brute-force implementation is slow > 10min
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
![Page 62: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/62.jpg)
input smoothed(structure, large scale)
residual(texture, small scale)
edge-preserving: Bilateral Filter
Decomposition into Large-scale and Small-scale LayersDecomposition into Large-scale and Small-scale Layers
![Page 63: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/63.jpg)
Weighted Average of PixelsWeighted Average of Pixels
space rangenormalization
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
space
range
p
q
• Depends on spatial distanceand intensity difference– Pixels across edges have almost influence
![Page 64: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/64.jpg)
A Gentle Introductionto Bilateral Filteringand its Applications
A Gentle Introductionto Bilateral Filteringand its Applications
Efficient Implementationsof the Bilateral Filter
Sylvain Paris – MIT CSAIL
![Page 65: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/65.jpg)
OutlineOutline
• Brute-force Implementation
• Separable Kernel [Pham and Van Vliet 05]
• Box Kernel [Weiss 06]
• 3D Kernel [Paris and Durand 06]
![Page 66: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/66.jpg)
Brute-force ImplementationBrute-force Implementation
For each pixel pFor each pixel q
Compute
8 megapixel photo: 64,000,000,000,000 iterations!
V E R Y S L O W !V E R Y S L O W !More than 10 minutes per imageMore than 10 minutes per image
( ) ( )∑∈
−−=S
IIIGGW
IBFq
qqpp
p qp ||||||1][rs σσ
( ) ( ) qqpqp IIIGG ||||||rs
−− σσ
![Page 67: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/67.jpg)
ComplexityComplexity
• Complexity = “how many operations are needed, how this number varies”
• S = space domain = set of pixel positions
• | S | = cardinality of S = number of pixels– In the order of 1 to 10 millions
• Brute-force implementation: )|(| 2SΟ
![Page 68: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/68.jpg)
Better Brute-force ImplementationBetter Brute-force Implementation
Idea: Far away pixels are negligible
For each pixel pa. For each pixel q such that || p – q || < cte × σs
looking at all pixels looking at neighbors only
![Page 69: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/69.jpg)
DiscussionDiscussion
• Complexity:
• Fast for small kernels: σs ~ 1 or 2 pixels
• BUT: slow for larger kernels
)|(| 2sσ×SΟ
neighborhood area
![Page 70: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/70.jpg)
OutlineOutline
• Brute-force Implementation
• Separable Kernel [Pham and Van Vliet 05]
• Box Kernel [Weiss 06]
• 3D Kernel [Paris and Durand 06]
![Page 71: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/71.jpg)
Separable KernelSeparable Kernel
• Strategy: filter the rows then the columns
• Two “cheap” 1D filters instead of an “expensive” 2D filter
[Pham and Van Vliet 05]
![Page 72: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/72.jpg)
DiscussionDiscussion
• Complexity: – Fast for small kernels (<10 pixels)
• Approximation: BF kernel not separable– Satisfying at strong edges and uniform areas
– Can introduce visible streaks on textured regions
)|(| sσ×SΟ
![Page 73: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/73.jpg)
input
![Page 74: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/74.jpg)
brute-forceimplementation
![Page 75: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/75.jpg)
separable kernelmostly OK,
some visible artifacts(streaks)
![Page 76: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/76.jpg)
OutlineOutline
• Brute-force Implementation
• Separable Kernel [Pham and Van Vliet 05]
• Box Kernel [Weiss 06]
• 3D Kernel [Paris and Durand 06]
![Page 77: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/77.jpg)
Box KernelBox Kernel
• Bilateral filter with a square box window
• The bilateral filter can be computed only from the list of pixels in a square neighborhood.
[Weiss 06]
( ) ( )∑∈
−−=S
IIIGBW
IYq
qqpp
p qp ||||||1][rs σσ
[Yarovlasky 85]
box window
( )∑∈
−=s
r||1][
σ
σB
IIIGW
IYq
qqpp
p
restrict the sum
independent of position q
![Page 78: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/78.jpg)
Box KernelBox Kernel• Idea: fast histograms of square windows
[Weiss 06]
input:full histogram is known
update:add one line, remove one line
Tracking one window
![Page 79: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/79.jpg)
Box KernelBox Kernel• Idea: fast histograms of square windows
[Weiss 06]
input:full histograms are known
update:add one line, remove one line,
add two pixels, remove two pixels
Tracking two windows at the same time
![Page 80: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/80.jpg)
DiscussionDiscussion
• Complexity: – always fast
• Only single-channel images
• Exploit vector instructions of CPU
• Visually satisfying results (no artifacts)– 3 passes to remove artifacts due to
box windows (Mach bands)
)log|(| sσ×SΟ
1 iteration
3 iterations
![Page 81: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/81.jpg)
input
![Page 82: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/82.jpg)
brute-forceimplementation
![Page 83: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/83.jpg)
box kernelvisually different,
yet no artifacts
![Page 84: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/84.jpg)
OutlineOutline
• Brute-force Implementation
• Separable Kernel [Pham and Van Vliet 05]
• Box Kernel [Weiss 06]
• 3D Kernel [Paris and Durand 06]
![Page 85: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/85.jpg)
3D Kernel3D Kernel
• Idea: represent image data such that the weights depend only on the distance between points
[Paris and Durand 06]
pixelintensity
pixel position
1D image
PlotI = f ( x )
far in range
close in space
![Page 86: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/86.jpg)
1st Step: Re-arranging Symbols1st Step: Re-arranging Symbols
( ) ( )
( ) ( )∑
∑
∈
∈
−−=
−−=
S
S
IIGGW
IIIGGW
IBF
qqpp
qqqp
pp
qp
qp
||||||
||||||1][
rs
rs
σσ
σσ
( ) ( )
( ) ( ) 1||||||
||||||][
rs
rs
∑
∑
∈
∈
−−=
−−=
S
S
IIGGW
IIIGGIBFW
qqpp
qqqppp
qp
qp
σσ
σσ
Multiply first equation by Wp
![Page 87: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/87.jpg)
1st Step: Summary1st Step: Summary
• Similar equations
• No normalization factor anymore
• Don’t forget to divide at the end
( ) ( )
( ) ( ) 1||||||
||||||][
rs
rs
∑
∑
∈
∈
−−=
−−=
S
S
IIGGW
IIIGGIBFW
qqpp
qqqppp
qp
qp
σσ
σσ
![Page 88: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/88.jpg)
2nd Step: Higher-dimensional Space2nd Step: Higher-dimensional Space
pp
space
range
• “Product of two Gaussians” = higher dim. Gaussian
![Page 89: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/89.jpg)
2nd Step: Higher-dimensional Space2nd Step: Higher-dimensional Space
pp
space
range
• 0 almost everywhere, I at “plot location”
![Page 90: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/90.jpg)
2nd Step: Higher-dimensional Space2nd Step: Higher-dimensional Space
pp
• 0 almost everywhere, I at “plot location”
• Weighted average at each point = Gaussian blur
![Page 91: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/91.jpg)
2nd Step: Higher-dimensional Space2nd Step: Higher-dimensional Space
pp
• 0 almost everywhere, I at “plot location”
• Weighted average at each point = Gaussian blur
• Result is at “plot location”
![Page 92: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/92.jpg)
higher dimensional functions
Gaussian blur
division
slicing
Higherdimensional
Homogeneousintensity
Higherdimensional
Homogeneousintensity
New num. scheme:• simple operations• complex space
![Page 93: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/93.jpg)
higher dimensional functions
Gaussian convolution
division
slicing
D O W N S A M P L E
U P S A M P L E
Heavilydownsampled
Heavilydownsampled
Strategy:downsampledconvolution
Conceptual view,not exactly the actual algorithm
![Page 94: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/94.jpg)
Actual AlgorithmActual Algorithm
• Never compute full resolution– On-the-fly downsampling
– On-the-fly upsampling
• 3D sampling rate = ),,( rss σσσ
![Page 95: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/95.jpg)
Pseudo-code: StartPseudo-code: Start
• Input– image I
– Gaussian parameters σs and σr
• Output: BF [ I ]
• Data structure: 3D arrays wi and w (init. to 0)
![Page 96: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/96.jpg)
Pseudo-code: On-the-fly DownsamplingPseudo-code: On-the-fly Downsampling
• For each pixel
– Downsample:
– Update:
SYX ∈),(
⎟⎟⎠
⎞⎜⎜⎝
⎛⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=
rss
),(,,),,(σσσ
YXIYXzyx
1),,(),(),,(
=+=+
zyxwYXIzyxwi
D O W N S A M P L E
U P S A M P L E
[ ] = closest int.
![Page 97: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/97.jpg)
Pseudo-code: ConvolvingPseudo-code: Convolving• For each axis , , and
– For each 3D point
• Apply a Gaussian mask ( 1 , 4 , 6 , 4 , 1 ) to wi and we.g., for the x axis:
wi’(x) = wi(x-2) + 4.wi(x-1) + 6.wi(x) + 4.wi(x+1) + wi(x+2)
xr yr zr
),,( zyx
D O W N S A M P L E
U P S A M P L E
![Page 98: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/98.jpg)
Pseudo-code: On-the-fly UpsamplingPseudo-code: On-the-fly Upsampling
• For each pixel
– Linearly interpolate the values in the 3D arrays
SYX ∈),(
( )( )),(,,,einterpolat
),(,,,einterpolat),(][YXIYXwYXIYXwiYXIBF =
D O W N S A M P L E
U P S A M P L E
![Page 99: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/99.jpg)
DiscussionDiscussion
• Complexity:
• Fast for medium and large kernels– Can be ported on GPU [Chen 07]: always very fast
• Can be extended to color images but slower
• Visually similar to brute-force computation
⎟⎟⎠
⎞⎜⎜⎝
⎛+
r2s
||||||σσRSSΟ
numberof pixels
numberof 3D cells
| R | : number of gray levels
![Page 100: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/100.jpg)
input
![Page 101: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/101.jpg)
brute-forceimplementation
![Page 102: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/102.jpg)
3D kernelvisually similar
![Page 103: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/103.jpg)
Running TimesRunning Times
separable kernel
3D kernel box kernel
brut
e fo
rce
![Page 104: A Gentle Introduction to Bilateral Filtering and its ...yu/Teaching/CISC849_S15/handouts/bil… · increasing texture with bilateral filter N O H A L O S. Notation and Definitions](https://reader036.fdocuments.net/reader036/viewer/2022090608/605e7f18bb920923a307e30d/html5/thumbnails/104.jpg)
How to Choose an Implementation?How to Choose an Implementation?
Depends a lot on the application. A few guidelines:
• Brute-force: tiny kernels or if accuracy is paramount
• Box Kernel: for short running times on CPU with any kernel size, e.g. editing package
• 3D kernel: – if GPU available
– if only CPU available: large kernels, color images, cross BF(e.g., good for computational photography)