Noise Estimation from a Single Image
description
Transcript of Noise Estimation from a Single Image
Noise Estimation from a Single Image
Ce Liu William T. Freeman
Richard Szeliski Sing Bing Kang
Parameter Tweaking in Computer Vision
Computer vision algorithms suffer from hand tuning parameters for particular images or image sequences
We want vision algorithms that behave properly under varying lighting conditions, blur levels and noise levels
Our work is one step in that direction
Given an image, estimate the noise level
Modify vision algorithms to be independent of noise
Image Noise Is Important in Vision
In image denoising the noise is assumed to be known as Additive Gaussian White Noise (AWGN)
However, in real applications the noise is unknown and non-additive
Many other computer vision algorithms also explicitly or implicitly assume the type and level of image noise
Hard to make vision algorithms fully automatic without knowing noise
Noise Level Function (NLF)
The standard deviation of noise is a function of image brightness I
Measurable by fixing the camera and taking multiple shots of a static scene
For each pixel: Mean: I Standard deviation:
NLF depends on camera, ISO, shutter speed, aperture
Our goal is to estimate NLF from a single image How to estimate noise without separating noise and signal?
I
An Example Image
Piecewise Smooth Image Prior
Patch
= +
Signal Residual
For each RGB channel:
Brightness mean I
Standard deviation
Red Green Blue
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
Brightness
Sta
nd
ard
d
evi
ati
on
Affine model
Piecewise Smooth Image Prior
Patch
= +
Signal Residual
Red Green Blue
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
Brightness
Sta
nd
ard
d
evi
ati
on
Piecewise Smooth Image Prior
Patch
= +
Signal Residual
Red Green Blue
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
0 0.5 1
0.2
0.1
0 I
Brightness
Sta
nd
ard
d
evi
ati
on
Segmentation-based Approach
Observed image
Segmentation-based Approach
Over-segmentation
Segmentation-based Approach
Signal
Segmentation-based Approach
Residual=
noise + unmodelled
image variation
Estimate NLFs
Assume brightness mean I is accurate estimate
Standard deviation is an over-estimate: (may contain signal)
The lower envelope is the upper bound of NLF
III
Brightness
Resi
du
al st
d.
dev.
Issues
Should the curve be strictly and tightly below the points?
III
Brightness
Resi
du
al st
d.
dev.
Issues
Should the curve be strictly and tightly below the points?
How to handle the missing data?
III
Brightness
Resi
du
al st
d.
dev.
Issues
Should the curve be strictly and tightly below the points?
How to handle the missing data?
Correlation between RGB channels?
III
Brightness
Resi
du
al st
d.
dev.
Solutions
Formulate the inference problem in a probabilistic framework
Learn the prior of noise level functions
III
Brightness
Resi
du
al st
d.
dev.
Outline
Over-segmentation and per-segment variance analysis
Learning the priors of noise level functions (NLF) Synthesize CCD noise Sample noise level functions Learn the prior of noise level functions
Inference: estimate the upper bound of NLF Bayesian MAP to estimate NLFs for RGB channels
Applications Adaptive bilateral filtering Canny edge detection
Noise model
Camera response function (CRF) f: download from Columbia camera response function database (used 196 typical CRFs)
Tsin et. al. Statistical calibration of CCD image process. ICCV, 2001
Camera Noise
Atmospheric Attenuation
Lens/geometricDistortion
CCD Imaging/Bayer Pattern
Fixed PatternNoise
Shot Noise
Thermal Noise
Interpolation/Demosaic
WhiteBalancing
GammaCorrection
A/D Converter
Dark Current Noise
t
QuantizationNoise
SceneRadiance
DigitalImage
CameraIrradiance
)( cs nnLfI
I
L
2)(Var,0)(E sss Lnn Dependent noise:Independent noise: 2)(Var,0)(E ccc nn
Synthesize CCD Noise
Estimate NLF )(),,;( IIEfI Ncs
I NI
Camera response function: f
Dependent noise:
Independent noise:s
c
Sample NLFs by Varying the ParametersCamera response function (CRF) f s
c
Dependent noise:
Independent noise:s
c
0.02
0.18
0.02
0.18
0.02 0.04 0.06
The Prior of NLFs
Likelihood Function
The estimated standard deviation should be probabilistically bigger than and close to the true value
Bayesian MAP inference
nh (n+1)h
)ˆ,ˆ( nnI
I
)|ˆ( p
Validation (1): Synthetic Noise
Add synthetic CCD noise, estimate, compare to the ground truth
— ground truth
estimated———
Validation (2): Measure NLF of a Real Camera
29 images were taken under the same settings (the camera is not in the database for training)
The real NLF is obtained by computing mean and variance per pixel
Validation (3): Robustness Test
Verify that different images from the same camera give the same estimated NLF (camera not in the database for training)
Application (1): Adaptive Bilateral Filtering
Bilateral filter is an edge-preserving low-pass filter Spatial sigma and range sigma
Adaptive bilateral filter Down-weigh RGB values by signal and noise covariance matrices The range sigma is set to be a function of the estimated standard
deviation of the noise
From Durand and Dorsey, SIGGRAPH 02
Input noisy image Smoothing kernel Denoised image
Test on Low and High Noise
low noise high noise
Red Green Blue Red Green Blue
Results—Adaptive Bilateral Filtering
Standard bilateral filtering
Adaptive bilateral filtering
low noise high noise
Results—Adaptive Bilateral Filtering
Zoom in high noise
Standard bilateral filtering
Adaptive bilateral filtering
Application (2): Canny Edge Detection
low noise high noise
Red Green Blue Red Green Blue
Results—Canny Edge Detection
low noise high noise
Parameters adapted in MATLAB
Parameters adapted by estimated noise
Conclusion
Piecewise-smooth image prior model to estimate the upper bound of noise level function (NLF)
Estimate the space of NLF by simulating CCD camera on the existing CRF database
Upper bounds are verified by both synthetic and real experiments
An important step to automate vision algorithms independent of noise
Thank you!
Ce Liu William T. Freeman
CSAIL MIT
Rick Szeliski Sing Bing Kang
Microsoft Research
Noise Estimation from a Single Image