Selected Topics in Computer Engineering (0907779)

Selected Topics in Computer

Engineering (0907779)Engineering (0907779)

Image Enhancement –Part III

Spatial Filtering

Dr. Iyad Jafar

Spatial Filtering

Outline

� Introduction

� Mechanics of Spatial Filtering � Mechanics of Spatial Filtering

� Correlation and Convolution

� Linear Spatial Filtering

�Spatial filters for Smoothing

�Spatial filters for Sharpening�Spatial filters for Sharpening

� Nonlinear Spatial Filtering

� Combining Spatial Enhancement Techniques 2

Background

� Filtering is borrowed from the frequency domainprocessing and refers to the process of passing orrejecting certain frequency components

�Highpass, lowpass, band-reject , and bandpassfilters

� Filtering is achieved in the frequency domain bydesigning the proper filter (Chapter 4)

� Filtering can be done in the spatial domain also byusing filter masks (kernels, templates, or windows)

�Unlike frequency domain filters, spatial filters canbe nonlinear !

3

Spatial Filtering Mechanics

� A spatial filter is characterized by � A rectangular neighborhood of size mxn (usually m and n are odd)

� A predefined operation that is specified by the mask values at each position. position.

3x3 filter mask example

w(-1,-1) w(-1,0) w(-1,1)

w(0,-1) w(0,0) w(0,1)

w(1,-1) w(1,0) w(1,1)

Origin y

(x, y)Neighbourhood

3x3 filter mask example

� Spatial filtering Operation� The filter mask is centered at each pixel in the image and the output

pixel value is computed based on the operation specified by themask

4

x Image f (x, y)

Spatial Filtering MechanicsOrigin

y

xImage f (x, y)

3x3 Filter

xw(-1,-1) w(-1,0) w(-1,1)

w(0,-1) w(0,0) w(0,1)

w(1,-1) w(1,0) w(1,1)

f(x-1,y-1) f(x-1,y) f(x-1,y+1)

f(x,y-1) f(x,y) f(x,y+1)

f(x+1,y-1) f(x+1,y) f(x+1,y+11)

5

Filter Original Image Pixels

a b

s a t b

g( x, y ) w( s,t ) f ( x s, y t )=− =−

= + +∑∑ ∑

Vector Representation of

Spatial Filtering

� The previous filtering equation can be written as

R w z w z ... w z= + + +

� If we represent the coefficients of the filter mask as a row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and

w1 w2 w3

w4 w5 w6

w7 w8 w9

1 1 2 2

1

mn mn

mn

k k

k

R w z w z ... w z

= w z =

= + + +

∑

z1 z2 z3

z4 z5 z6

z7 z8 z9

row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and the image pixels under the mask by z = [z1 z2 z3 z4 z5

z6 z7 z8 z9], then the filtering equation can be written as

6

TR wz=

Spatial Correlation� The filtering operation just described is called

correlation

� In 1-DOrigin

0 0 0 1 0 0 0 0

1 2 3 2 8

f wOrigin

0 0 0 1 0 0 0 0

1 2 3 2 81) Function and mask

2) Initial Alignment

7

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

3) Zero Padding for f by m-1

4) Position after one shift

Spatial Correlation

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts

.6) Repeat

.

.

.

1 2 3 2 8

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position

8

Full correlation result

Cropped correlation result

0 0 0 8 2 3 2 1 0 0 0 0

0 8 2 3 2 1 0 0

Spatial Convolution

� A strongly related operation to correlation isconvolution (an important tool in linear systemstheory)theory)

� The mechanics of convolution is similar to thoseof correlation, except that the filter mask isrotated by 180o before sliding

9

Spatial Convolution

0 0 0 1 0 0 0 0

f wOrigin

1 2 3 2 81) Function and mask

8 2 3 2 1

0 0 0 1 0 0 0 0

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0

2) Initial Alignmentafter rotation ofmask

3) Zero Padding for f by m-1

10

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 04) Position after one shift

Spatial Convolution

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 05) Position after 4 shifts

.6) Repeat

.

.

.

8 2 3 2 1

0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0Final Position

11

Full convolution result

Cropped convolutionresult

0 0 0 1 2 3 2 8 0 0 0 0

0 1 2 3 2 8 0 0

Extension to 2D� Extension to 2D is straight forward

� Zero padding is done in both directions by m-1 and n-1

� In convolution, the mask is flipped vertically and horizontally horizontally

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 1 0 0

0 0 0 0 0

0 0 0 0 0

f

Zero padding by m-1 and n-1

12

0 0 0 0 1 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0

1 2 3

4 5 6

7 8 9

9 8 7

6 5 4

3 2 1

w Mask rotation

Extension to 2D

9 8 7 0 0 0 0 0 01 2 3 0 0 0 0 0 0

6 5 4 0 0 0 0 0 0

3 2 1 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

4 5 6 0 0 0 0 0 0

7 8 9 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

13

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

Initial position in convolution

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

Initial position in correlation

Extension to 2D

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 9 8 7 0 0 0

0 0 0 6 5 4 0 0 0

0 0 0 3 2 1 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 1 2 3 0 0 0

0 0 0 4 5 6 0 0 0

0 0 0 7 8 9 0 0 0

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

14

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

Full convolution result

0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0

Full correlation result

Extension to 2D

0 0 0 0 0

0 1 2 3 0

0 4 5 6 0

0 0 0 0 0

0 9 8 7 0

0 6 5 4 0

Cropped convolution result

0 7 8 9 0

0 0 0 0 0

Cropped correlation result

0 6 5 4 0

0 3 2 1 0

0 0 0 0 0

Correaltion Convolution

15

a b

s a t b

Correaltion

g( x, y ) w( s,t ) f ( x s, y t )=− =−

= + +∑∑a b

s a t b

Convolution

g( x,y ) w( s,t ) f ( x s, y t )=− =−

= − −∑∑

Treatment of Pixels at Edges

� In the previous slides, we padded the image with zerosin both directions in order to compensate forunavailable values.

xunavailable values.

� Other approaches

� Replicate edge pixels

� Consider only available

pixels that fall under the

mask in the computation

xe

e

e

e

mask in the computation

of the new values

� Allow pixels to wrap

around16

yImage f (x, y)

ee e

Treatment of Pixels at Edges

� The result depends on the used approach

� Example 3.6

Filtered Image:

OriginalImage

Filtered Image: Zero Padding

Filtered Image: Replicate Edge Pixels

17

Filtered Image: Wrap Around Edge Pixels

Smoothing Spatial Filters

� A smoothing (averaging, blurring) filter replaces eachpixel with the average value of all pixels under themaskmask

� Uses of smoothing filters

� The smoothed image corresponds to the gross detailsof the image or the low frequency content of the image

� Can be used to reduce noise as it is characterized withsharp transitionssharp transitions

� However, smoothing usually results in degradation ofedge quality (sharpness of the image)

18


� Common smoothing masks

1 1 1 1 2 1

� Notes

1 1 1

1 1 1

1 1 1

X 1/9

1 2 1

2 4 2

1 2 1

X 1/16

Standard averaging mask Weighted average mask

� Notes

� The weighted average filter gives more weight to pixelsnear the center

� It is hard to see the visual difference between theprocessing results of the two filters; however, the weightedaverage filter coefficients sum up to 16, which makes itmore attractive for computers19

� Example: 3.7 Smoothing with different mask sizes. Notice the loss of details as the mask size increases


Original Smoothing by 3x3 Mask Smoothing by 5x5 Mask

20Smoothing by 9x9 Mask Smoothing by 15x15 Mask Smoothing by 35x35 Mask

� Example 3.8: smoothing highlights gross details. Could be useful in providing better segmentation results


results

21

Original Image Smoothed Image Thresholded Image

� Example 3.9: Noise Reduction


22

Original Smoothed

Sharpening Spatial Filters

� The principle objective of sharpening is to highlighttransitions in intensity which usually correspond toedges in images; thus sharpening is the opposite ofsmoothingedges in images; thus sharpening is the opposite ofsmoothing

� If we examine the smoothing operation we can thinkof it as integration

� Thus to perform sharpening in the spatial domain, it isintuitive to use differentiationintuitive to use differentiation

� In the following few slides we examine thefundamental properties of the first and secondderivatives when applied on different intensitytransitions

23

Sharpening Spatial Filters� Lets discuss differentiation for the1-D case

� We are concerned about the behavior of 1st and 2nd

derivatives in the following areas � Constant intensity� Constant intensity

� Onset and end of discontinuities (ramps and steps discontinuities)

� Intensity ramps

� Properties of 1st derivative� Zero in areas of constant intensity

� Nonzero at the onset of a step and intensity rampNonzero at the onset of a step and intensity ramp

� Nonzero along intensity ramp

� Properties 2nd derivative� Zero in areas of constant intensity

� Nonzero at the onset and end of a step and intensity ramp

� Zero along intensity ramp 24


� Derivatives can be approximated as differences

� 1st derivative at x � 1st derivative at x

2nd derivative at x

1f

f ( x ) f ( x )x

∂ = + −∂

� 2nd derivative at x

25

2

21 1 2

ff ( x ) f ( x ) f ( x )

x

∂ = + + − −∂

Sharpening Spatial Filters� Investigation of the derivatives behavior

26


� Notes

� Examining the 1st and 2nd derivatives plots showsthat all of their properties are satisfiedthat all of their properties are satisfied

� 1st derivative produces thicker edges than 2nd

derivatives

� 2nd derivative produces double edge separated by azero crossingzero crossing

� 2nd derivative is commonly used in sharpening sinceit has simpler implementation and finer edges

27

Sharpening Using 2nd

Derivative� The second derivative (Laplacian) in 2-D is defined as

2 22 f f

f∂ ∂∇ = +

If we define

Then the discrete second derivative can be approximated

22 2

f ff

x y

∂ ∂∇ = +∂ ∂

2

21 1 2

ff ( x , y ) f ( x , y ) f ( x, y )

x

∂ = + + − −∂

2

21 1 2

ff ( x, y ) f ( x, y ) f ( x, y )

y

∂ = + + − −∂

Then the discrete second derivative can be approximated by

2

1 1 1 1 4

f

f ( x , y ) f ( x , y ) f ( x, y ) f ( x, y ) f ( x, y )

∇ =+ + − + + + − −

28


Derivative� The Laplacian can be implemented as a filter mask

0 1 0

� Or

0 1 0

1 -4 1

0 1 0

1 1 1

1 -8 1

1 1 1

29


Derivative� Computing the Laplacian doesn’t produce a sharpened

image. However, grayish edge lines and discontinuitiessuperimposed on a dark backgroundsuperimposed on a dark background

� It is common practice to scale the Laplacian image to [0,255] for better display30


Derivative� Alternatively, to obtain a sharpened image g(x,y),

subtract the Laplacian image from the original image

2g( x, y ) f ( x, y ) f ( x, y )= − ∇

- =

31

- =

OriginalImage

LaplacianFiltered Image

SharpenedImage


Derivative�The two steps required to achieve sharpening can becombined into a single filtering operation

),1(),1([),( yxfyxfyxf −++−=)1,()1,( −+++ yxfyxf

)],(4 yxf−

fyxfyxg 2),(),( ∇−=

)],(4 yxf−),1(),1(),(5 yxfyxfyxf −−+−=)1,()1,( −−+− yxfyxf

32


Derivative

�As a filter mask, the previous equation can be represented as

0 -1 0as

0 -1 0-1 5 -10 -1 0

33

Sharpening Using 1st Derivative

�The first derivative in 2-D is implemented as a gradientwhich is defined as a vector

∂f

� The gradient points in the direction of greatest rate ofchange at location (x,y)

∂∂∂∂

=

=∇

y

fx

f

G

G

y

xf

change at location (x,y)

� The magnitude of the gradient is defined as

Or, approximately

34

12 22

f fM( x, y )

x y

∂ ∂ = + ∂ ∂ | | x yf G G∇ ≈ +


� Computation of the gradient using Roberts cross-gradient operators

� Roberts operators for computing Gx and Gy are shownbelow. The upper-left pixel in the operator is overlaid overbelow. The upper-left pixel in the operator is overlaid overthe pixel (x,y) or z5 in the original image

-1 0

0 1

0 -1

1 0

Horizontal Operator

Z1 Z2 Z3

Z4 Z5 Z6

Z7 Z8 Z9Vertical Operator

35

[ ][ ]

29 5 9 5

28 6 8 6

9 5 8 6

x

y

G ( x, y ) z z z z

G ( x, y ) z z z z

M( x, y ) z z z z

= − ≈ −

= − ≈ −

≈ − + −

Horizontal Operator

Pixel z5 and its neighbours

Vertical Operator

� Computation of the gradient using Sobel operators

� Sobel operators for computing Gx and Gy are shownbelow. The center pixel in the operator is overlaid over the


below. The center pixel in the operator is overlaid over thepixel (x,y) or z5

-1 0 1

-2 0 2

-1 0 1

-1 -2 -1

0 0 0

1 2 1

Z1 Z2 Z3

Z4 Z5 Z6

Z7 Z8 Z9

36

7 8 9 1 2 3

3 6 9 1 4 7

2 2

2 2

x

y

x y

G ( x, y ) ( z z z ) ( z z z )

G ( x, y ) ( z z z ) ( z z z )

M( x, y ) G ( x, y ) G ( x, y )

≈ + + − + +

= + + − + +

= +

Mask to Compute GxPixel z5 and its neighbours Mask to Compute Gy

� Example 3.10

� Gradient is widely used in industrial inspection as itproduce thicker edges in the result, which make it easierfor machines to detect artifacts


for machines to detect artifacts

37

Optical Image for a contact lens Gradient Image

Nonlinear (Order-statistic)

Spatial Filters

Order-statistic filters are nonlinear filters whoseresponse is based on ordering the pixels under the maskand then replacing the centre pixel with the valueand then replacing the centre pixel with the valuedetermined by the ranking result

Z1 Z2 Z3

Z4 Z5 Z6

Z7 Z8 Z9

Rank

� Examples

� minimum filter, maximum filter, median filter � popular and useful in removing impulse or salt-and-pepper noise with less blurring than linear smoothing38

New value

Nonlinear (Order-statistic)

Spatial Filters

� Example 3.11

Note how the median filter has reduced the noise significantly with less blurring than smoothing filter

39

Combining Spatial

Enhancements

� It is very common to combine different enhancement techniques in order to achieve different enhancement techniques in order to achieve the desired goal

� Example 3.12

� We want to enhance/sharpen the bones in this imagebones in this image

� Direct application of the Laplacian may result in noise amplification

40

Combining Spatial

EnhancementsCompare to enhancement by single method

41

Processed by a combination of

methods

Sharpened by Laplacian

Histogram Equalization

Power-Law with gamma = 0.5

Combining Spatial

Enhancement

Enhancement by multiple methods

Laplacian filter of

(a)

Laplacian filter of bone scan (a)

Sharpened version of bone scan achieved by subtracting (a) and (b) Sobel filter of bone

scan (a)

(b)

(c)

(d)42

Combining Spatial

Enhancement

Sharpened image which is sum of (a)

Result of applying a power-law trans. to (g)

(g)

(h)

The product of (c) and (e) which will be used as a mask

which is sum of (a) and (f)

(e)

(f)

(g)

Image (d) smoothed witha 5*5 averaging filter

43

Combining Spatial

Enhancement

Compare the original and final images

44

Readings

� Read section 3.6.3 titled “Unsharp Masking and Highboost filtering”.

45

Related Matlab Functions

� Check Matlab documentation for the following functions

�conv2

�corr2

� fillter2

� imfilter� imfilter

�padarray

�median

�medfilt246

Selected Topics in Computer Engineering (0907779)

Documents

Transcript of Selected Topics in Computer Engineering (0907779)