Accelerating Spatially Varying Gaussian Filters
description
Transcript of Accelerating Spatially Varying Gaussian Filters
![Page 1: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/1.jpg)
![Page 2: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/2.jpg)
Accelerating Spatially Varying Gaussian Filters
Jongmin Baek and David E. JacobsStanford University
![Page 3: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/3.jpg)
Motivation
Input
GaussianFilter
SpatiallyVaryingGaussianFilter
![Page 4: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/4.jpg)
1) Accelerating Spatially Varying Gaussian Filters
2) Accelerating Spatially Varying Gaussian Filters
3) Accelerating Spatially Varying Gaussian Filters
4) Applications
Roadmap
![Page 5: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/5.jpg)
Gaussian Filters
Given pairs as input,
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅ exp (− ∥𝑝𝑖−𝑝 𝑗∥2
2 σ2 )
Position Value
![Page 6: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/6.jpg)
Gaussian Filters
Each output value …
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅ exp (− ∥𝑝𝑖−𝑝 𝑗∥2
2 σ2 )
![Page 7: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/7.jpg)
Gaussian Filters
… is a weighted sum of input values …
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅ exp (− ∥𝑝𝑖−𝑝 𝑗∥2
2 σ2 )
![Page 8: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/8.jpg)
Gaussian Filters
… whose weight is a Gaussian …
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅ exp (− ∥𝑝𝑖−𝑝 𝑗∥2
2 σ2 )
![Page 9: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/9.jpg)
Gaussian Filters
… in the space of the associated positions.
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅ exp (− ∥𝑝𝑖−𝑝 𝑗∥2
2 σ2 )
![Page 10: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/10.jpg)
Gaussian Blur
Gaussian Filters: Uses
𝒑 𝒊=(𝒙 𝒊 , 𝒚 𝒊 ) ,𝒗 𝒊=(𝒓 𝒊 ,𝒈𝒊 ,𝒃𝒊)
![Page 11: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/11.jpg)
Bilateral Filter
Gaussian Filters: Uses
𝒑 𝒊=(𝒙 𝒊 , 𝒚 𝒊 ,𝒓 𝒊 ,𝒈𝒊 ,𝒃𝒊 ) ,𝒗 𝒊=(𝒓 𝒊 ,𝒈𝒊 ,𝒃𝒊)
![Page 12: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/12.jpg)
Non-local Means Filter
Gaussian Filters: Uses
𝒑 𝒊=(𝒙 𝒊 , 𝒚 𝒊 , )𝒗 𝒊=(𝒓 𝒊 ,𝒈𝒊 ,𝒃𝒊)
![Page 13: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/13.jpg)
Applications Denoising images and meshes Data fusion and upsampling Abstraction / Stylization Tone-mapping ...
Gaussian Filters: Summary
Previous work on fast Gaussian Filters Bilateral Grid (Chen, Paris, Durand; 2007) Gaussian KD-Tree (Adams et al.; 2009) Permutohedral Lattice (Adams, Baek, Davis; 2010)
![Page 14: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/14.jpg)
Summary of Previous Implementations:
A separable blur flanked by resampling operations. Exploit the separability of the Gaussian kernel.
Gaussian Filters: Implementations
![Page 15: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/15.jpg)
Spatially Varying Gaussian Filters
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅N (𝑝𝑖−𝑝 𝑗 ;𝜎
2 𝐼 )
𝑣ˆ
𝑖=∑𝑗𝑣 𝑗⋅N (𝑝𝑖−𝑝 𝑗 ;𝐾 𝑖 )
Spatially varying covariance matrix
Spatially Invariant
![Page 16: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/16.jpg)
Trilateral Filter (Choudhury and Tumblin, 2003)
Tilt the kernel of a bilateral filter along the image gradient.
“Piecewise linear”instead of“Piecewise constant”model.
Spatial Variance in Previous Work
![Page 17: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/17.jpg)
Spatially Varying Gaussian Filters: Tradeoff
Benefits: Can adapt the kernel spatially. Better filtering performance.
Cost: No longer separable. No existing acceleration
schemes.
Input Bilateral-filtered Trilateral-filtered
![Page 18: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/18.jpg)
Problem: Spatially varying (thus non-separable) Gaussian filter
Existing Tool: Fast algorithms for spatially invariant Gaussian filters
Solution: Re-formulate the problem to fit the tool. Need to obey the “piecewise-constant” assumption
Acceleration
![Page 19: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/19.jpg)
Naïve Approach (Toy Example)I LOST THE GAME
Input Signal
Desired Kernel1 1 12 3 4
filtered w/ 1filtered w/ 2filtered w/ 3filtered w/ 4
1 1 12
3Output Signal4
![Page 20: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/20.jpg)
In practice, the # of kernels can be very large.
Challenge #1
Pixel Location x
Desired Kernel K(x)
Range ofKernels needed
![Page 21: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/21.jpg)
Sample a few kernels and interpolate.
Solution #1
Desired Kernel K(x)
Sampledkernels
Interpolate result!
Pixel Location x
K1
K2
K3
![Page 22: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/22.jpg)
Interpolation needs an extra assumption to work:
The covariance matrix Ʃi is either piecewise-constant, or smoothly varying.
Kernel is spatially varying, but locally spatially invariant.
Assumptions
![Page 23: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/23.jpg)
Runtime scales with the # of sampled kernels.
Challenge #2
Desired Kernel K(x)
Filter only some regions of the image with each kernel.
(“support”)
Pixel Location x
Sampledkernels
K1
K2
K3
![Page 24: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/24.jpg)
In this example, x needs to be in the support of K1 & K2.
Defining the Support
Desired Kernel K(x)
Pixel Location x
K1
K2
K3
![Page 25: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/25.jpg)
Dilating the Support
Desired Kernel K(x)
Pixel Location x
K1
K2
K3
![Page 26: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/26.jpg)
Algorithm
1) Identify kernels to sample.2) For each kernel, compute the support needed.3) Dilate each support.4) Filter each dilated support with its kernel.5) Interpolate from the filtered results.
![Page 27: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/27.jpg)
Algorithm
1) Identify kernels to sample.2) For each kernel, compute the support needed.3) Dilate each support.4) Filter each dilated support with its kernel.5) Interpolate from the filtered results.
K1
K2
K3
![Page 28: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/28.jpg)
Algorithm
1) Identify kernels to sample.2) For each kernel, compute the support needed.3) Dilate each support.4) Filter each dilated support with its kernel.5) Interpolate from the filtered results.
K1
K2
K3
![Page 29: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/29.jpg)
Algorithm
1) Identify kernels to sample.2) For each kernel, compute the support needed.3) Dilate each support.4) Filter each dilated support with its kernel.5) Interpolate from the filtered results.
K1
K2
K3
![Page 30: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/30.jpg)
Algorithm
1) Identify kernels to sample.2) For each kernel, compute the support needed.3) Dilate each support.4) Filter each dilated support with its kernel.5) Interpolate from the filtered results.
K1
K2
K3
![Page 31: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/31.jpg)
Algorithm
1) Identify kernels to sample.2) For each kernel, compute the support needed.3) Dilate each support.4) Filter each dilated support with its kernel.5) Interpolate from the filtered results.
K1
K2
K3
![Page 32: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/32.jpg)
Applications
HDR Tone-mapping
Joint Range Data Upsampling
![Page 33: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/33.jpg)
Application #1: HDR Tone-mapping
Input HDR
Detail
BaseFil
ter Output
Attenuate
![Page 34: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/34.jpg)
Tone-mapping Example
Bilateral Filter Kernel Sampling
![Page 35: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/35.jpg)
Application #2: Joint Range Data Upsampling
Range Finder Data
Sparse Unstructured Noisy
Scene Image
Output
Filter
![Page 36: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/36.jpg)
Synthetic Example
Scene Image Ground Truth Depth
![Page 37: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/37.jpg)
Synthetic Example
Scene Image Simulated Sensor Data
![Page 38: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/38.jpg)
Synthetic Example : Result
Kernel SamplingBilateral Filter
![Page 39: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/39.jpg)
Synthetic Example : Relative Error
Bilateral Filter Kernel Sampling
2.41% Mean Relative Error 0.95% Mean Relative Error
![Page 40: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/40.jpg)
Real-World Example
Scene Image Range Finder Data
*Dataset courtesy of Jennifer Dolson, Stanford University
![Page 41: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/41.jpg)
Real-World Example: Result
Input
Bilateral
Naive
KernelSampling
![Page 42: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/42.jpg)
Performance
Kernel Sampling
Choudhury and Tumblin (2003) Naïve
Tonemap1
5.10 s 41.54 s 312.70 s
Tonemap2
6.30 s 88.08 s 528.99 sKernel
SamplingKernel Sampling
(No segmentation)Depth1 3.71 s 57.90 sDepth2 9.18 s 131.68 s
![Page 43: Accelerating Spatially Varying Gaussian Filters](https://reader035.fdocuments.net/reader035/viewer/2022062401/56815c1c550346895dc9f1bf/html5/thumbnails/43.jpg)
1. A generalization of Gaussian filters• Spatially varying kernels• Lose the piecewise-constant assumption.
2. Acceleration via Kernel Sampling• Filter only necessary pixels (and their support)
and interpolate.
3. Applications
Conclusion