Medical Image Analaysis

Atam P. Dhawan

Image Enhancement: Spatial Domain

Histogram Modification

1-L0,1,...,ifor)( ii nrh

where ir is the ith gray-level in the image for a total of L gray values and in is the

number of occurrences of gray-level ir in the image.

n

nrp i

i )(

Medical Images and Histograms

Histogram Equalization

1-L0,1,...,ifor

)()(

0

0

i

j

j

i

jjrii

n

n

rprTs

Image Averaging Masks

K

ii yxg

Kyxg

1

),(1

),(

 

  f(-1,0)  

f(0,-1) f(0,0) f(0,1)

  f(1,0)  

f(-1,-1) f(-1,0) f(-1,0)

f(0,-1) f(0,0) f(0,1)

f(0,-1) f(1,0) f(1,1)

 

Image Averaging

p

px

p

pyp

px

p

py

yyxxfyxw

yxw

yxg ),(),(

),(

1),(

 

1 2 1

2 4 2

1 2 1

Median Filter

),(),(

),( jigNji

medianyxf

Laplacian: Second Order Gradient for Edge Detection

)],(4)1,()1,(),1(),1([

),(

),(),(

2

2

2

22

yxfyxfyxfyxfyxf

y

yxf

x

yxfyxf

 

-1 -1 -1

-1 8 -1

-1 -1 -1

Image Sharpening with Laplacian

 

-1 -1 -1

-1 9 -1

-1 -1 -1

Feature Adaptive Neighborhood

Xc Xc

Center Region

Surround Region

)3(),( cxyxf

)3(),()3( cc xyxfx

)3(),( cxyxf

Feature Enhancement

),(),,(max

),(),(),(

yxPyxP

yxPyxPyxC

sc

sc

),(),( if)),(1)(,(),(

),(),( if),(1

),(),(

yxPyxPyxCyxPyxg

yxPyxPyxC

yxPyxg

scs

scs

C’(x,y)=F{C(x,y)}

Micro-calcification Enhancement

Frequency-Domain Methods

),(),(),(),( yxnyxfyxhyxg

),(),(),(),( vuNvuFvuHvuG

),(

),(

),(

),(),(ˆ

vuH

vuN

vuH

vuGvuF

),(

),(

),(),(

),(

),(

1),(ˆ

2

2

vuG

vuS

vuSvuH

vuH

vuHvuF

f

n

Low-Pass Filtering

High Pass Filtering

Wavelet Transform

Fourier Transform only provides frequency information.

Windowed Fourier Transform can provide time-frequency localization limited by the window size.

Wavelet Transform is a method for complete time-frequency localization for signal analysis and characterization.

Wavelet Transform..

Wavelet Transform : works like a microscope focusing on finer time resolution as the scale becomes small to see how the impulse gets better localized at higher frequency permitting a local characterization

Provides Orthonormal bases while STFT does not.

Provides a multi-resolution signal analysis approach.

Wavelet Transform…

Using scales and shifts of a prototype wavelet, a linear expansion of a signal is obtained.

Lower frequencies, where the bandwidth is narrow (corresponding to a longer basis function) are sampled with a large time step.

Higher frequencies corresponding to a short basis function are sampled with a smaller time step.

Continuous Wavelet Transform

Shifting and scaling of a prototype wavelet function can provide both time and frequency localization.

Let us define a real bandpass filter with impulse response (t) and zero mean:

This function now has changing time-frequency tiles because of scaling. a<1: (a,b) will be short and of high frequency a>1: (a,b) will be long and of low frequency

a

bt

at

tft

dttfa

bt

abaCWT

dtt

ba

ba

R

f

1)( where

)(),(

)(*1

),(

as defined is (CWT) Transform Wavelet contnuousA

0)0()(

,

,

Wavelet Decomposition

T h e w a v e l e t t r a n s f o r m o f a s i g n a l i s i t s d e c o m p o s i t i o n o n a f a m i l y o fr e a l o r t h o n o r m a l b a s e s

m n ( x ) o b t a i n e d t h r o u g h t r a n s l a t i o n a n dd i l a t i o n o f a k e r n e l f u n c t i o n ( x ) k n o w n a s t h e m o t h e r w a v e l e t .

W h e r e m , n Z , a s e t o f i n t e g e r s

)2(2)( 2/, nxx mm

nm

Wavelet Coefficients

Using orthonormal property of the basis functions, wavelet coefficients of a signal f(x) can be computed as

The signal can be reconstructed from the coefficients as

)()()( ,, xdxxfd nmnm

)()( ,, xdxf nmm n

nm

Wavelet Transform with Filters The mother wavelet can be constructed using a scaling

function (x) which satisfies the two-scale equation

Coefficients h(k) have to meet several conditions for the set of basis functions to be unique, orthonormal and have a certain degree of regularity.

For filtering operations, h(k) and g(k) coefficients can be used as the impulse responses correspond to the low and high pass operations.

)()1()(

)2()(2)(

)2()(2)(

klk

where

nxnx

nxnx

k

n

n

hg

g

h

Decomposition

H H

G

H

G

G

2 2

2

2

2

Data

Wavelet Decomposition Space

V0 data

V1 W1

V2 W2

V3 W3

Image Decomposition

h g

sub-sample

Level 0 Level 1

h- h

h-g

g-h

g-g

horizontally vertically

sub-sample

g

gh

h

XImage

Wavelet and Scaling Functions

Image Processing and Enhancement

Image Segmentation

Edge-Based Segmentation Gray-level Thresholding Pixel Clustering Region Growing and Spiliting Artificial Neural Network Model-Based Estimation

Gray-Level Thesholding

Tyxf

Tyxfyxg

),( if 0

),( if 1),(

Region Growing

Center Pixel

Pixels satisfying the

similarity criterion

Pixels not satisfying the similarity criterion

3x3 neighborhood

5x5 neighborhood

7x7 neighborhood

Segmented region

Neural Network Element

x 1

x n

x 2

1

N o n - L i n e a r A c t i v a t i o n F u n c t i o n F

n

inii wxwFy

11

w n + 1

w 1

w 2

w n

n

inii wxwFy

11

Artificial Neural Network: Backpropagation

Hidden Layer Neurons

Output Layer Neurons

Ly1

x1 x2 x3 xn 1

Ly2 Lny

RBF Network

RBF Unit 1

RBF Unit 2

RBF Unit n

Input ImageSliding

Image Window

Output

Linear Combiner

RBF Layer

RBF NN Based Segmentation

Image Representation

Bottom-Up

Scenario

Scene-1 Scene-I

Object-1 Object-J

S-Region-1 S-Region-K

Region-1 Region-L

Pixel (i,j)

Edge-MEdge-1

Pixel (k,l)

Top-Down

Image Analysis: Feature Extraction Statistical Features

Histogram Moments Energy Entropy Contrast Edges

Shape Features Boundary encoding Moments Hough Transform Region Representation Morphological Features

Texture Features Spatio Frequency Features Relational Features

Image Classification

Feature Based Pattern Classifiers Statistical Pattern Recognition

Unsupervised Learning Supervised Learning

Sytntactical Pattern Recognition Logical predicates

Rule-Based Classifers Model-Based Classifiers Artificial Neural Networks

Morphological Features

A

B

BA

BA

Some Shape Features

A

EH

D

B

C

FG

O

•Longest axis GE.•Shortest axis HF.•Perimeter and area of the minimum bounded rectangle ABCD.•Elongation ratio: GE/HF•Perimeter p and area A of the segmented region.

•Circularity

•Compactness2

4

p

AC

A

pC p

2

Relational Features

A

C

B

D

F

I

E

B

C

A

I

ED

F

Nearest Neighbor ClassifierA d i s t a n c e m e a s u r e )( fjD i s d e f i n e d b y t h e E u c l i d e a n d i s t a n c e i n t h e f e a t u r e s p a c e a s

jjD uff )(

w h e r e CjN

jcfj

jj ,...2,1

1

fu

i s t h e m e a n o f t h e f e a t u r e v e c t o r s f o r t h e c l a s s jc a n d N j i s t h e t o t a l n u m b e r o f f e a t u r e

v e c t o r s i n t h e c l a s s jc .

T h e u n k n o w n f e a t u r e v e c t o r i s a s s i g n e d t o t h e c l a s s ic i f

)]([min)( 1 ff jC

ji DD

Rule Based Systems

Strategy RulesA priori knowledge

or models

Focus of Attention Rules

Knowledge Rules

Activity

Center

InputDatabase

OutputDatabase

Strategy RulesStrategy Rule SR1:

If NONE REGION is ACTIVE NONE REGION is ANALYZED Then ACTIVATE FOCUS in SPINAL_CORD AREA Strategy Rule SR2:

If ANALYZED REGION is in SPINAL_CORD AREA ALL REGIONS in SPINAL_CORD AREA are NOT ANALYZED Then ACTIVATE FOCUS in SPINAL_CORD AREA Strategy Rule SR3:

If ALL REGIONS in SPINAL_CORD AREA are ANALYZED ALL REGION in LEFT_LUNG AREA are NOT ANALYZED

Then ACTIVATE FOCUS in LEFT_LUNG AREA

FOA Rules

Focus of Attention Rule FR1:

If REGION-X is in FOCUS AREA REGION-X is LARGEST REGION-X is NOT ANALYZED

Then ACTIVATE REGION-X

Focus of Attention Rule FR2:

If REGION-X is in ACTIVE MODEL is NOT ACTIVE

Then ACTIVATE KNOWLEDGE_MERGE rules

Knowledge Rules

Knowledge Rule: Merge_Region_KR1 If

REGION-1 is SMALL REGION-1 has GIGH ADJACENCY with REGION-2 DIFFERENCE between AVERAGE VALUE of REGION-1 and

REGION-2 is LOW or VERY LOW REGION-2 is LARGE or VERY LARGE

Then MERGE REGION-1 in REGION-2 PUT_STATUS ANALYZED in REGION-1 and REGION-2

Neuro-Fuzzy Classifiers

M1

winner-take-alloutput layer

L

1

fuzzy membershipfunction layer

x1

xi

xd

hyperplanelayer

inputlayer

max

M2

MK

C

Extraction of Ventricles

Composite 3D Ventricle Model

Extraction of Lesions

Extraction of Sulci

Segmented Regions

Center for Intelligent Vision System

Structural Signatures: Volume Measurements of Ventricular Size and Cortical Atrophy in Alcoholic and Normal Populations from MRI

Ventricular Volume Alcoholics

Ventricular Volume Normal

Sulcus Volume Alcoholics

Sulcus Volume Normal

0 0.05 0.1 0.15 0.2 0.25

Multi-Parameter Measurements

Do = f{T1, T2, HD, T1+Gd, pMRI, MRA, 1H-MRS, ADC, MTC, BOLD}where,

T1 = NMR spin-lattice relaxation timeT2 = NMR spin-spin relaxation timeHD = Proton densityGd+T1 = Gadolinium enhanced T1

pMRI = Dynamic T2* images during Gd bolus injectionMRA = Time of flight MR angiographyMRS = Magnetic Resonance SpectroscopyADC= Apparent Diffusion CoefficientMTC= Magnetization Transfer ContrastBOLD = Blood Oxygenation Level Dependent

Regional Classification & Characterization

1. White matter 2. Corpus callosum 3. Superficial gray

4. Caudate 5. Thalamus 6. Putamen

7. Globus pallidus 8. Internal capsule 9. Blood vessel

10. Ventricle 11. Choroid plexus 12. Septum pellucidium

13. Fornices 14. Extraaxial fluid 15. Zona granularis

16. Undefined

Adaptive Multi-Level Multi-Dimensional Analysis

Database ofTissue Signatures

Selection of classesand cluster analysis

New classformation

SignatureSelection

Markov RandomField

BasedClassification

Adaption tospatial domain

Evaluate statisticaldistribution of classes

and probabilities

AcceptableNo more classes

Pixels withlow prob (classified)

Relax class selectioncriteria

No

Yes

Yes

No

Not

Acceptable

All pixels classified ?

Building Signatures

Analysis of 15 classes (normal group)

Stroke Effect on 12-Years Old Subject

Center for Intelligent Vision and Information System

Typical Function of Interest Analysis: Dhawan et al. (1992)Typical Function of Interest Analysis: Dhawan et al. (1992)

FVOI Signature

Anatomical Reference

(S.C.A.)

Functional Reference

(F.C.A.)

ReferenceSignatures

MR Image(New Subject)

PET Image(New Subject)

MR-PETRegistration

Principal Axes Registration

= 1 if (x,y,z) is in the object = 0 if (x,y,z) is not in the objectB x y z( , , ) 

x

xB x y z

B x y zg

x y z

x y z

( , , )

( , , )

, ,

, ,

y

yB x y z

B x y zg

x y z

x y z

( , , )

( , , )

, ,

, ,

z

zB x y z

B x y zg

x y z

x y z

( , , )

( , , )

, ,

, ,

Binary Volume

Centroids

PAR

1. Translate the centroid of V1 to the origin. 2. Rotate the principal axes of V1 to coincide

with the x, y and z axes. 3. Rotate the x, y and z axes to coincide with

the principal axes of V2. 4. Translate the origin to the centroid of V2. 5. Scale V2 volume to match V1 volume.

Iterative PAR for MR-PET Images(Dhawan et al, 1992)

1. Threshold the PET data.

2. Extract binary cerebrum and cerebellum areas from MR scans.

3. Obtain a three-dimensional representation for both MR and PET data: rescale and interpolate. 4. Construct a parallelepiped from the slices of the interpolated PET data that contains the binary PET brain volume. This volume will be referred to as the "FOV box" of the PET data. 5. Compute the centroid and principal axes of the binary PET brain volume.

Iterative PAR…

6. Add n slices to the FOV box on the top and the bottom such that the augmented FOV(n) box will have the same number of slices as the binary MR brain. Gradually shrink this FOV(n) box back to its original size, FOV(0) box, recomputing the centroid and principal axes of the trimmed binary MR brain at each step iteratively.

7. Interpolate the gray-level PET data (rescaled to match the MR data) to obtain the PET volume.

8. Transform the PET volume into the space of the original MR slices using the last set of MR and PET centroids and principal axes.. Extract from the PET volume the slices which match the original MR slices.

IPARIteration 1

Iteration 2

Iteration 3

Center for Intelligent Vision and Information Systems

Multi-Modality MR-PET Brain Image Image Multi-Modality MR-PET Brain Image Image RegistrationRegistration

Center for Intelligent Vision and Information Systems

Multi-Modality MR-PET Brain Image RegistrationMulti-Modality MR-PET Brain Image Registration

Center for Intelligent Vision and Information Systems

Multi-Modality MR-PET Brain Image RegistrationMulti-Modality MR-PET Brain Image Registration

MR Volume Signatures