Digital Image Filters
description
Transcript of Digital Image Filters
Computer Vision Group
Digital Image Filters26th November 2012
Computer Vision Group
Outline
Definitions
Linear, shift invariant filters: convolution.
Examples of convolutional filters• Smoothing Filters (denoising)• Differentiating Filters (edge detector)
Examples of Nonlinear Filtering
Computer Vision Group
Systems
A System H is considered as a black box that processes some input signal (f) and gives the output (Hf)
In our case, f is a digital image or a 1-D digital signal, but in principle could be analogic and n-D signal as well as a scalar
Computer Vision Group
Linearity
A system is said to be linear i.i.f.
and this hold for all and for aribrary signals
it is the classical definition of linearity
)()()()( tHgtHftgtfH
Computer Vision Group
Signal and System - Property
A system is said to be time (or shift) – invariant i.i.f.
Systems that are Linear and Translation Invariant are represented by a convolutoin.• LTI systems are characterized entirely by a single
function called the system's impulse response.• The impulse response corresponds to the filter
associated to the system.
))(()( 00 ttHfttfH
Computer Vision Group
Convolution - LTI System
Let us consider a signal and a filter • Their convolution is a signal .• For continuous-domain 1D signals and filters
i.e.,
• For discrete signals and filters
Computer Vision Group
Convolution LIT Systems on Images
are discrete 2D signals
h7 h8 h9
h4 h5 h6
h1 h2 h3
h9 h8 h7
h6 h5 h4
h3 h2 h1
h1 h2 h3
h4 h5 h6
h7 h8 h9
flipX
y1 y2 y3
y4 y5 y6
y7 y8 y9
=flipY
h9 h8 h7
h6 h5 h4
h3 h2 h1
*
Point-wise product h9y1 h8y2 h7y3
h6y4 h5y5 h4y6
h3y7 h2y8 h1y9
sum
z5 = h9y1+ h8y2 + h7y3 + h6y4 + h5y5 + h5y6 + h3y7 + h2y8 + h1y9
Computer Vision Group
Convolution LIT Systems on Images
are discrete 2D signals
h7 h8 h9
h4 h5 h6
h1 h2 h3
h9 h8 h7
h6 h5 h4
h3 h2 h1
h1 h2 h3
h4 h5 h6
h7 h8 h9
flipX
y1 y2 y3
y4 y5 y6
y7 y8 y9
=flipY
h9 h8 h7
h6 h5 h4
h3 h2 h1
*
Point-wise product h9y1 h8y2 h7y3
h6y4 h5y5 h4y6
h3y7 h2y8 h1y9
sum
z5 = h9y1+ h8y2 + h7y3 + h6y4 + h5y5 + h5y6 + h3y7 + h2y8 + h1y9
Computer Vision Group
Convolution LIT Systems on Images
are discrete 2D signals
h7 h8 h9
h4 h5 h6
h1 h2 h3
h9 h8 h7
h6 h5 h4
h3 h2 h1
h1 h2 h3
h4 h5 h6
h7 h8 h9
flipX
y1 y2 y3
y4 y5 y6
y7 y8 y9
=flipY
h9 h8 h7
h6 h5 h4
h3 h2 h1
*
Point-wise product h9y1 h8y2 h7y3
h6y4 h5y5 h4y6
h3y7 h2y8 h1y9
sum
z5 = h9y1+ h8y2 + h7y3 + h6y4 + h5y5 + h5y6 + h3y7 + h2y8 + h1y9
Computer Vision Group
Convolution LIT Systems on Images
are discrete 2D signals
h7 h8 h9
h4 h5 h6
h1 h2 h3
h9 h8 h7
h6 h5 h4
h3 h2 h1
h1 h2 h3
h4 h5 h6
h7 h8 h9
flipX
y1 y2 y3
y4 y5 y6
y7 y8 y9
=flipY
h9 h8 h7
h6 h5 h4
h3 h2 h1
*
Point-wise product h9y1 h8y2 h7y3
h6y4 h5y5 h4y6
h3y7 h2y8 h1y9
sum
z5 = h9y1+ h8y2 + h7y3 + h6y4 + h5y5 + h5y6 + h3y7 + h2y8 + h1y9
Computer Vision Group
Linear Filters for Digital Images
Linear Filters are particular systems where the output at each pixel (sample) is given by a linear combination of neighboring pixels (samples)
10 30 10
20 11 20
11 9 1
* * *
* 5.7 *
* * *
H
Original Signal Filtered Output
Computer Vision Group
Filters or Kernels
The coefficients used in the linear combination are given by the kernel
1 3 0
2 10 2
4 1 1
Image
1 0 -1
1 0.1 -1
1 0 -1
Kernel
= - 3
Filter Output
*
Computer Vision Group
A well-known Test Image - Lena
Computer Vision Group
Trivial example
0 0 0
0 1 0
0 0 0
*
Computer Vision Group
0 0 0
1 0 0
0 0 0
*
Transalation
Computer Vision Group
Linear Filtering
1 1 1
1 1 1
1 1 191* ?
-local averaging
Computer Vision Group
The original Lena
Computer Vision Group
Filtered Lena
Computer Vision Group
Linear Filtering
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
251*
- local averages
Computer Vision Group
Computer Vision Group
Computer Vision Group
What about normalization?
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
252*
Computer Vision Group
… convolution is linear
Computer Vision Group
…what about
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
252 *
Computer Vision Group
Computer Vision Group
Convolution Properties
It is a linear System; it enjoys every LTI properties
Is it Commutative ?
but it depends on the patting used. In continuous domain it holds as well as on periodic signals
It is also associative
and dissociative
)()()()( tHgtHftgtfH
Computer Vision Group
2D Gaussian Filter
2
22
2 2exp
21,
yxyxG
2
22
2 211exp
21,
kjkijiH
array 1212 is , where kkjiH
Continuous Function
Discrete Function
Computer Vision Group
Linear Filtering(Gaussian Filter)-weighted local averages-
*
12
34
56
7
12
34
56
7
0
0.05
0.1
0.15
0.2
Computer Vision Group
Linear Filtering(Gaussian Filter)-weighted local averages-
Computer Vision Group
Gaussian Vs Average
Gaussian Smoothing Support 7x7
Smoothing by Averaging On 7x7 window
Computer Vision Group
31Denoising: The Issue
A Noisy Detail in Camera Raw Data.
Computer Vision Group
32Denoising: The Issue
Denoised
Computer Vision Group
33Denoising: The Issue
A Noisy Detail in Camera Raw Data.
Computer Vision Group
34Denoising: The Issue
Denoised
Computer Vision Group
Observation model is
The goal is to obtain , a reliable estimate of , given and the distribution of .
For the sake of simplicity it is often assumed and independent .
The noise standard deviation is assumed as known.
Image Formation Model
Sensed imagePixel index
Original (unknown) imagenoise
July 23, 2010
35
Computer Vision Group
Observation model is
Thus we can pursue a “regression-approach”,
Image Formation Model
July 23, 2010
36
Computer Vision Group
Observation model is
Thus we can pursue a “regression-approach”, but on images it may not be convenient to assume a parametric expression of on
Image Formation Model
July 23, 2010
37
Computer Vision Group
Observation model is
Thus we can pursue a “regression-approach”, but on images it may not be convenient to assume a parametric expression of on
Image Formation Model
July 23, 2010
38
Computer Vision Group
Denoising Approaches
Parametric Approaches• Transform Domain Filtering, they assume the noisy-free signal is
somehow sparse in a suitable domain (e.g Fourier, DCT, Wavelet) or w.r.t. some dictionary based decomposition)
27th October 2009
39
Computer Vision Group
Denoising Approaches
Parametric Approaches• Transform Domain Filtering, they assume the noisy-free signal is
somehow sparse in a suitable domain (e.g Fourier, DCT, Wavelet) or w.r.t. some dictionary based decomposition)
Non Parametric Approaches• Local Smoothing / Local Approximation• Non Local Methods
27th October 2009
40
Computer Vision Group
Denoising Approaches
Parametric Approaches• Transform Domain Filtering, they assume the noisy-free signal is
somehow sparse in a suitable domain (e.g Fourier, DCT, Wavelet) or w.r.t. some dictionary based decomposition)
Non Parametric Approaches• Local Smoothing / Local Approximation• Non Local Methods
27th October 2009
41
Computer Vision Group
Denoising Approaches
Parametric Approaches• Transform Domain Filtering, they assume the noisy-free signal is
somehow sparse in a suitable domain (e.g Fourier, DCT, Wavelet) or w.r.t. some dictionary based decomposition)
Non Parametric Approaches• Local Smoothing / Local Approximation• Non Local Methods
Estimating from can be statistically treated as regression of on
27th October 2009
42
Computer Vision Group
Fitting and Convolution
One can prove that the least square fit of polynomial of 0-th order (i.e constant) is given by
where
and thus
03/12/2009
Computer Vision Group
Denoising Approaches
Non Parametric Approaches: there are no global model for and basically • Local Methods: define, in each image pixel, the “best
neighborhood” where a simple parametric model can be enforced to perform regression
is described by a (polynomial) model
July 23, 2010
44
Computer Vision Group
Ideal neighborhood – an illustrative example
Ideal in the sense that it defines the support of a pointwise Least Square Estimator of the reference point.
Typically, even in simple images, every point has its own different ideal neighborhood.
For practical reasons, the ideal neighborhood is assumed starshaped
Further details at LASIP c/o Tampere University of Technology http://www.cs.tut.fi/~lasip/
July 23, 2010
45
Computer Vision Group
Neighborhood discretization
A suitable discretization of this neighborhood is obtained by using a set of directional LPA kernels
where determines the orientation of the kernel support, andwhere controls by the scale of kernel support.
Ideal Neighborhood
Directional kernels
Discrete Adaptive Neighborhood
27th October 2009
46
Computer Vision Group
Neighborhood discretization
A suitable discretization of this neighborhood is obtained by using a set of directional LPA kernels
where determines the orientation of the kernel support, andwhere controls by the scale of kernel support.
The initial shape optimization problem can be solved by using standard easy-to-implement varying-scale kernel techniques, such as the ICI rule.
Ideal Neighborhood
Directional kernels
Discrete Adaptive Neighborhood
27th October 2009
47
Computer Vision Group
Ideal neighborhood – an illustrative example
Ideal in the sense that it defines the support of pointwise Least Square Estimator of the reference point.
July 23, 2010
48
Computer Vision Group
Examples of Adaptively Selected Neighorhoods
Define, , the “ideal” neigborhood
Compute the denoised estimate at , “using”
July 23, 2010
49
Computer Vision Group
Examples of adaptively selected neighorhoods
Adaptively selected neighborhoods selected using the LPA-ICI rule
July 23, 2010
50
Computer Vision Group
Denoising
Additive Gaussian White Noise
After Gaussian Smoothing
After Averaging
),( N
Computer Vision Group
Salt & Pepper Noise Additive Noise, 7% outliers
After Gaussian Smoothing
After Averaging
Denoising
Computer Vision Group
Correlation
k l
ljkiHlkIjiHIjiI ,,,*,
A Target can be used as a filter.
Correlation between target and image assumes maximum at image pixels that match the target
Computer Vision Group
Correlation for BINARY target matching
*
• Easier with binary images• Target used as a filter
?
Computer Vision Group
Correlation function Maximum value line profile
Correlation for target matching
Computer Vision Group
Examples of Non Linear Filters
• Median Filter (Weighted Median)• Ordered Statistics based Filters• Threshold
There are many others, such as data adaptive filtering procedures (e.g ICI)
Computer Vision Group
Blockwise Median
Substititute to each pixel the median of its neighbors
),( jiN
),( jif1 3 0
2 10 2
4 1 1
2
2)1,1,4,2,10,2,0,3,1( medianm
med
Computer Vision Group
Denoising with median 3x3
Salt and Pepper noise
Computer Vision Group
Denoising with median 3x3
Additive Gaussian White Noise (AGWN)
Computer Vision Group
What is an Edge
Lets define an edge to be a discontinuity in image intensity function.
Several Models• Step Edge• Ramp Edge• Roof Edge• Spike Edge
They can bethus detected asdiscontinuitiesof image Derivatives
03/12/2009
Computer Vision Group
Differentiation and convolution
Recall
Now this is linear and shift invariant. Therefore, in discrete domain, it will be represented as a convolution
xfxfxf
0lim
03/12/2009
Computer Vision Group
Differentiation and convolution
Recall
Now this is linear and shift invariant. Therefore, in discrete domain, it will be represented as a convolution
We could approximate this as
(which is obviously a convolution with Kernel
; it’s not a very good way to do things, as we shall see)
xfxfxf
0lim
xxfxf
xf n
1
11
03/12/2009
Computer Vision Group
Finite Difference in 2D
yxfyxfxyxf ,,lim,
0
yxfyxfyyxf ,,lim,
0
x
yxfyxfxyxf mnmn
,,, 1
x
yxfyxfyyxf mnmn
,,, 1
11
11
Discrete Approximation
Horizontal
Convolution Kernels
Vertical
03/12/2009
Computer Vision Group
A 1D Example
Take a line on a grayscale image
03/12/2009
Computer Vision Group
A 1D Example (II)
Filter the discrete image values, convolution against [1 -1]
03/12/2009
Computer Vision Group
Differentiating Filters
)1()1()( xfxfxf x1D Discrete derivatives
1 0 -1*ff x
2D Discrete derivatives (separable)
*ff y 1 0 -1t
03/12/2009
Computer Vision Group
Classical Operators : Prewitt
111111
11
11
Smooth Differentiate
101101101
111000111
111111
Horizontal
Vertical
03/12/2009
Computer Vision Group
Classical Operators: Sobel
112211
11
11
Smooth Differentiate
101202101
121000121
121121
Horizontal
Vertical
03/12/2009
Computer Vision Group
Another famous test image - cameraman
03/12/2009
Computer Vision Group
Horizontal Derivatives using Sobel
1 * ty y h
03/12/2009
Computer Vision Group
Vertical Derivatives using Sobel
2 * ty y h
03/12/2009
Computer Vision Group
Gradient Magnitude and edge detectors
2 2| | * *ty y h y h
03/12/2009
Computer Vision Group
A brief overview on Morphological Operators in Image Processing
Giacomo Boracchi24/11/2010
home.dei.polimi.it/boracchi/teaching/IAS.htm
Computer Vision Group
An overview on morphological operations
Erosion, Dilation
Open, Closure
We assume the image being processed is binary, as these operators are typically meant for refining “mask” images.
03/12/2009
Computer Vision Group
AND operator in Binary images
03/12/2009
Computer Vision Group
OR in Binary Images
03/12/2009
Computer Vision Group
Erosion
General definition: Nonlinear Filtering procedure that replace to each pixel value the minimum on a given neighbor
As a consequence on binary imagesE(x)=1 iff the image in the neighbor is constantly 1
This operation reduces thus the boundaries of binary images
It can be interpreted as an AND operation of the image and the neighbor overlapped at each pixel
03/12/2009
Computer Vision Group
Erosion Example
03/12/2009
Computer Vision Group
Erosion Example
03/12/2009
Computer Vision Group
Dilation
General definition: Nonlinear Filtering procedure that replace to each pixel value the maximum on a given neighbor
As a consequence on binary images
E(x)=1 iff at least a pixel in the neighbor is 1
This operation grows fat the boundaries of binary images
It can be interpreted as an OR operation of the image and the neighbor overlapped at each pixel
03/12/2009
Computer Vision Group
Dilation Example
03/12/2009
Computer Vision Group
Dilation Example
03/12/2009
Computer Vision Group
In matlab
They are performed using the
bwmorph.m script passing the name of the operation as a parameter
Examples…
03/12/2009
Computer Vision Group
Open and Closure
Open Erosion followed by a Dilation
Closure Dilation followed by an Erosion
03/12/2009
Computer Vision Group
Open
Open Erosion followed by a Dilation• Smoothes the contours of an object• Typically eliminate thin protrusions
03/12/2009
Computer Vision Group
Open
Figure to Open,
Structuring element a Disk
03/12/2009
Computer Vision Group
Open
First Erode,
03/12/2009
Computer Vision Group
Open
Then Dilate,
Open results, the bridge has been eliminate from the first erosion and cannot be replaced by the dilation.
Edges are smoothed
Corners are rounded
03/12/2009
Computer Vision Group
Closure
Closure Dilation followed by an Erosion• Smoothes the contours of an object, typically creates• Generally fuses narrow breaks
03/12/2009
Computer Vision Group
Open
Figure to Open,
Structuring element a Disk
03/12/2009
Computer Vision Group
Close
First Dilate,
03/12/2009
Computer Vision Group
Close
Then Erode,
Close results, the bridge has been preserved
Edges are smoothed
Corners are rounded
03/12/2009
Computer Vision Group
There are several other Non Linear Filters
Ordered Statistic based• Median Filter • Weight Ordered Statistic Filter• Trimmed Mean• Hybrid Median
Thresholding
03/12/2009