Methods for Digital Image ProcessingMethods for Digital Image Processing

G ray-level Histogram

Spatial

DFT DCT

Spectral

Digital Im age Characteristics

Point Processing M asking Filtering

Enhancem ent

Degradation M odels Inverse Filtering W iener Filtering

Restoration

Pre-Processing

Inform ation Theory

LZW (gif)

Lossless

T ransform -based (jpeg)

Lossy

Com pression

Edge Detection

Segm entation

Shape Descriptors T exture M orphology

Description

Digital Im age Processing

Basic ideasBasic ideas of Image of Image

TransformsTransforms

Spatial FrequencySpatial Frequencyoror

Fourier TransformFourier Transform

Jean Baptiste Joseph Fourier

Why are Spatial Frequencies Why are Spatial Frequencies important?important?

• Efficient data representation

• Provides a means for modeling and removing noise

• Physical processes are often best described in “frequency domain”

• Provides a powerful means of image analysis

What is spatial frequency?What is spatial frequency?

• Instead of describing a function (i.e., a shape) by a series of positions

• It is described by a series of cosines

What is spatial frequency?What is spatial frequency?

A

g(x) = A cos(x)

2

x

g(x)

What is spatial frequency?What is spatial frequency?

Period (L)Wavelength ()Frequency f=(1/ )

Amplitude (A)Magnitude (A)

A cos(x 2/L)g(x) = A cos(x 2/) A cos(x 2f)

x

g(x)

What is spatial frequency?What is spatial frequency?

A

g(x) = A cos(x 2f)

x

g(x)

(1/f)(1/f)

period

But what if cosine is shifted in phase?But what if cosine is shifted in phase?

g(x) = A cos(x 2f + )

x

g(x)

What is spatial frequency?What is spatial frequency?

g(x) = A cos(x 2f + )

A=2 mf = 0.5 m-1

= 0.25 = 45g(x) = 2 cos(x 2(0.5) + 0.25) 2 cos(x + 0.25)

x g(x)0.00 2 cos(0.25) = 0.707106...0.25 2 cos(0.50) = 0.00.50 2 cos(0.75) = -0.707106...0.75 2 cos(1.00) = -1.01.00 2 cos(1.25) = -0.707106…1.25 2 cos(1.50) = 01.50 2 cos(1.75) = 0.707106...1.75 2 cos(2.00) = 1.02.00 2 cos(2.25) = 0.707106...

Let us take arbitrary g(x)

We substitute values of A, f and

We calculate discrete values of g(x) for various values of x

What is spatial frequency?What is spatial frequency?

g(x) = A cos(x 2f + )

x

g(x)We calculate discrete values of g(x) for various values of x

Now we take discrete values of Now we take discrete values of AAii , f , fii

and and i

gi(x) = Ai cos(x 2fi + i), i = 0,1,2,3,...

x

Now we substitute fNow we substitute fi i = i/N= i/N

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N-1

f=i/N

0 N

gi(x) = Ai cos(x 2fi + i), i = 0,1,2,3,...

N = time interval

Values for various values of Values for various values of i

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N-1

f=i/N

0 N

We calculate values of function for various values of ii

Substituting various values of Substituting various values of ii to the to the formula we get various cosinusoidesformula we get various cosinusoides

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N-1

i=0 i=1 i=2

A0A1

A2

Changing Changing NN to to N/2N/2

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N/2 - 1

i=N/2 - 1i=0

If N equals the number of pixel in a line, then...

Lowest frequency Highest frequency

What is spatial frequency?What is spatial frequency?

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N/2-1

i=N/2-1i=0

If N equals the number of pixels in a line, then...

Lowest frequency Highest frequency

What will happen if we take What will happen if we take N/2N/2??

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N/2-1

i=N/2i=0

If N equals the number of pixel in a line, then...

Lowest frequency Too high Redundant frequency

What is spatial frequency?What is spatial frequency?

12/

0

12/

0

/2cos)(Ni

iii

Ni

ii NixAxgxg

g(x) = A cos(x 2f + )

gi(x) = Ai cos(x 2i/N + i), i = 0,1,2,3,…,N/2-1

We try to approximate a periodic We try to approximate a periodic function with standard trivial function with standard trivial (orthogonal, base) functions(orthogonal, base) functions

+

+=

Low frequency

Medium frequency

High frequency

We add values from component We add values from component functions functions point by pointpoint by point

+

+=

g(x)

i=1

i=2

i=3

i=4

i=5

i=63

0 127

xExample of periodic function created by summing standard trivial functions

g(x)

i=1

i=2

i=3

i=4

i=5

i=10

0 127x

Example of periodic function created by summing standard trivial functions

g(x)

g(x)

64 terms

10 terms

Example of periodic function created by summing standard trivial functions

g(x)

i=1

i=2

i=3

i=4

i=5

i=630 127

x

Fourier Decomposition of a step function (64 terms)

Example of periodic function created by summing standard trivial functions

g(x)

i=1

i=2

i=3

i=4

i=5

i=100 63

x

Fourier Decomposition of a step function (11 terms)

Example of periodic function created by summing standard trivial functions

Main concept – summation of base Main concept – summation of base functionsfunctions

12/

0

/2cos)(Ni

iii NixAxg

Any function of x (any shape) that can be represented by g(x) can also be represented by the summation of cosine functions

Observe two numbers for every i

Information is not lost when we Information is not lost when we change the domainchange the domain

gi(x) = 1.3, 2.1, 1.4, 5.7, …., i=0,1,2…N-1

N pieces of information

12/

0

/2cos)(Ni

iiii NixAxg

N pieces of informationN/2 amplitudes (Ai, i=0,1,…,N/2-1) andN/2 phases (i, i=0,1,…,N/2-1) and

SpatialSpatial Domain

Frequency Domain

What is spatial frequency?What is spatial frequency?

gi(x)

Are equivalentThey contain the same amount of information

12/

0

/2cosNi

iii NixA and

The sequence of amplitudes squared is the SPECTRUM

Information is not lost when we Information is not lost when we change the domainchange the domain

EXAMPLE

A cos(x2i/N)frequency (f) = i/Nwavelength (p) = N/I

N=512i f p0 0 infinite1 1/512 51216 1/32 32256 1/2 2

Substitute values

Assuming N we get this table which relates frequency and wavelength of component functions

More examples to give you some intuition….

Fourier Transform NotationFourier Transform Notation• g(x) denotes an spatial domain function of real numbers

– (1.2, 0.0), (2.1, 0.0), (3.1,0.0), …

• G() denotes the Fourier transform

• G() is a symmetric complex function(-3.1,0.0), (4.1, -2.1), (-3.1, 2.1), …(1.2,0.0) …, (-3.1,-2.1), (4.1, 2.1), (-3.1,0.0)

• G[g(x)] = G(f) is the Fourier transform of g(x)

• G-1() denotes the inverse Fourier transform

• G-1(G(f)) = g(x)

Power Spectrum and Phase SpectrumPower Spectrum and Phase Spectrum

• |G(f)|2 = G(f)G(f)* is the power spectrum of G(f)– (-3.1,0.0), (4.1, -2.1), (-3.1, 2.1), … (1.2,0.0),…, (-3.1,-2.1), (4.1, 2.1)

– 9.61, 21.22, 14.02, …, 1.44,…, 14.02, 21.22

• tan-1[Im(G(f))/Re(G(f))] is the phase spectrum of G(f)– 0.0, -27.12, 145.89, …, 0.0, -145.89, 27.12

complex

Complex conjugate

1-D DFT and IDFT1-D DFT and IDFT• Discrete Domains

– Discrete Time: k = 0, 1, 2, 3, …………, N-1– Discrete Frequency: n = 0, 1, 2, 3, …………, N-1

• Discrete Fourier Transform

• Inverse DFT

Equal time intervals

Equal frequency intervals

1N

0k

nkN2

j;e ]k[x]n[X

1N

0n

nkN2

j;e ]n[X

N1

]k[x

n = 0, 1, 2,….., N-1

k = 0, 1, 2,….., N-1

Fourier 2D Image Fourier 2D Image TransformTransform

Another formula for Two-Dimensional Another formula for Two-Dimensional FourierFourier

A cos(x2i/N) B cos(y2j/M)fx = u = i/N, fy = v =j/M

Image is function of x and y

Now we need two cosinusoids for each point, one for x and one for y

Lines in the figure correspond to real value 1

Now we have waves in two directions and they have frequencies and amplitudes

Fourier Transform of a Fourier Transform of a spotspot

Original image Fourier Transform

Transform Results

image

spectrum

transform

Two Dimensional Fast Fourier in Two Dimensional Fast Fourier in MatlabMatlab

Filtering in Frequency Filtering in Frequency DomainDomain

… will be covered in a separate lecture on spectral approaches…..

•H(u,v) for various values of u and v

•These are standard trivial functions to compose the image from

< < image

..and its spectrum

Image and its spectrum

Image and its spectrum

Image and its spectrum

Let g(u,v) be the kernelLet h(u,v) be the imageG(k,l) = DFT[g(u,v)]H(k,l) = DFT[h(u,v)]

Then DFT 1 G H g h

where means multiplicationand means convolution.

This means that an image can be filtered in the Spatial Domain or the Frequency Domain.

Convolution TheoremConvolution Theorem

This is a very important result

Let g(u,v) be the kernelLet h(u,v) be the imageG(k,l) = DFT[g(u,v)]H(k,l) = DFT[h(u,v)]

Then

DFT 1 G H g h

where means multiplicationand means convolution.

Convolution TheoremConvolution Theorem

Instead of doing convolution in spatial domain we can do multiplication

In frequency domain

Convolution in spatial domain

Multiplication in spectral domain

v

u

Image

Spectrum Noise and its spectrum

Noise filtering

Image

v

u

Spectrum

Image x(u,v)

v

u

Spectrum log(X(k,l))

l

k

Spectrum log(X(k,l))

k

lv

u

Image x(u,v)

Image of cow with noise

white noise white noise spectrum

kernel spectrum (low pass filter)

red noise red noise spectrum

Filtering is done in spectral domain. Can be very complicated

Image TransformsImage Transforms• Fast Fourier

– 2-D Discrete Fourier Transform

• Fast Cosine– 2-D Discrete Cosine Transform

• Radon Transform• Slant• Walsh, Hadamard, Paley, Karczmarz• Haar• Chrestenson• Reed-Muller

Discrete Cosine Transform Discrete Cosine Transform (DCT)(DCT)

•Used in JPEG and Used in JPEG and MPEGMPEG

•Another Frequency Another Frequency Transform, with Transform, with Different Set of Basis Different Set of Basis FunctionsFunctions

Discrete Cosine Discrete Cosine Transform in MatlabTransform in Matlab

““Statistical” FiltersStatistical” Filters

•Median Filter also eliminates noise•preserves edges better than blurring

•Sorts values in a region and finds the median

•region size and shape

•how define the median for color values?

““Statistical” Filters Statistical” Filters ContinuedContinued

•Minimum Filter Minimum Filter (Thinning)(Thinning)

•Maximum Filter Maximum Filter (Growing)(Growing)

•““Pixellate” FunctionsPixellate” FunctionsNow we can do this quickly in spectral domain

•ThinninThinningg

•GrowinGrowingg

Pixellate ExamplesPixellate Examples

DCT FeaturesDCT Features

Fringe Pattern

DCT

DCT Coefficients

Zonal Mask

1 2

3

4

5

1 2 3 4 5

(1,1)(1,2)(2,1)(2,2)

.

.

.

FeatureVector

ArtificialNeuralNetwork

Can be used for face recognition, tell my story from Japan.

Noise RemovalNoise Removal

Image with Noise Transform Image reconstructed

Transforms for Noise RemovalTransforms for Noise Removal

Image Segmentation Recall: Image Segmentation Recall: Edge DetectionEdge Detection

f(x,y) Gradient Mask

fe(x,y)

-1 -2 -10 0 01 2 1

-1 0 1-2 0 2-1 0 1

Now we do this in spectral domain!!

Image MomentsImage Moments2-D continuous function f(x,y), the moment of order (p+q) is:

....2 ,1 ,0,

),(

qp

dydxyxfyxm qppq

Central moment of order (p+q) is:

00

01

00

10 ;

where

),()()(

m

my

m

mx

dydxyxfyyxx qppq

Image Moments (contd.)Image Moments (contd.)

Normalized central moment of order (p+q) is:

,.....3 ,2,for

;12

where

;00

qp

qp

pqpq

A set of seven invariant moments can be derived from pq

Now we do this in spectral domain!!

Image TexturesImage Textures

The USC-SIPI Image Databasehttp://sipi.usc.edu/

Grass Sand Brick wall

Now we do this in spectral domain!!

ProblemsProblems• There is a lot of Fourier and Cosine Transform

software on the web, find one and apply it to remove some kind of noise from robot images from FAB building.

• Read about Walsh transform and think what kind of advantages it may have over Fourier

• Read about Haar and Reed-Muller transform and implement them. Experiment

SourcesSources• Howard Schultz, Umass

• Herculano De Biasi• Shreekanth Mandayam• ECE Department, Rowan University• http://engineering.rowan.edu/~shreek/fall01/dip/

http://engineering.rowan.edu/~shreek/fall01/dip/lab4.html

Image CompressionImage CompressionPlease visit the website

http://www.cs.sfu.ca/CourseCentral/365/li/material/notes/Chap4/Chap4.html