Automated Medical Image Registration using Global and Conditional Mutual

Information

Dirk LoeckxFrederik Maes, Dirk Vandermeulen, Paul Suetens

Group of Science, Engineeringand Technology

Group of BiomedicalSciences

Radiology, Nuclear Medicine, Cardiology,

Radiotherapy

Processing of Speech & Images

Medical Image Computing

Medical ImagingResearch Center

Image RegistrationFind geometrical relationship between

images

Why?• Wealth of images• Registration

– Integrate information from different images– Different modality, time, patient, pose, contrast– Automatic ↔ implicit

• Clinical workflow– Fully digital

• Acquisition, transmission, storage, retrieval, reading– Processing time, visualisation– Validation

Building BlocksSimilarity Measure

Rr Ffs fprp

frpfrpE

μ

μμ

;

;,log;, xμxg RF ;

x

yji ,

xu

v

djdi ,

Transformation Penalties

xμxxμ dPwEV ;

SmoothnessVolumeRigidity

R

Input data

F

Deformed Image

F’

OptimizerMultiresolutionAnalytic DerivativesLBFGSB (quasi Newton)

Visualization

Similarity Measure

Cross Correlation(CC)

Linear Relation

Mutual Information(MI)

Statistical Relation

Sum of Squared Differences (SSD)

Constant Relation

Rx

xFxRn

SSD 21g

2

22

1

1

1

RxFR

FxFRxRn

CC g

frRxfr fprp

frpfrpMIxFfwxRrw

nfrp

,

,ln,,

1, g

Mutual Information• ‘Information’ R carries about F and vice

versa• Based on joint histogram

– Combine intensity R(x) with F(g(x;m)) in discrete bins

– If I know p(r), how good can I predict p(f)?– Joint and marginal probability p(r;m), p(f;m),

p(r,f;m)– F(g(x;m)) unknown (digital images, partial

volume effect)• No spatial information!

Mutual Information

RH FRH ,

FH

+ – =

H R H F

, ,I R F H R H F H R F

logx

H X p x p xF : Reference image (histogram) : Floating image (histogram) : Entropy

R

Mutual Information• Excellent results for rigid

registration–Only translation/rotation (6

parameters)–Parameters influence whole image

• Sometimes works for nonrigid registration–100’s to 1000’s of parameters–Local influence

Some Examples

Rigid CT/MR

F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imag., vol. 16, no. 2, pp. 187–198, Apr 1997.

Nonrigid Cardiac US• Left ventricle– Manual delineation– Triangulation

• Propagate delineation– Mutual information

image registration– Backward and

forward– Constant

connectivity• 4D path of each

vector• Visual validation:

ok

Subtraction CT angiography

• 3D acquisitions of region of interest– Without / with contrast

• Registration– Nonrigid mutual information– Compensate for movement artefacts

• Subtraction– Only contrast remains– Clear view of vasculature

sCTA: Examples

MI for nonrigid registration• Ongoing field of research• Global Histogram

– Multimodal registration• Minimise minor in favour of major peaks• Reduce smaller image details

– Bias field• Register bias fields, not image features (also

rigid)• Local Histogram (image subdivision)

– Limited ‘hits’: statistical power?– Solution: overlapping subregions

Transformation model• -

– Reference position– Floating position

• Tensor-product B-splines– Rectangular mesh– Displacement vectors– Weighted sum– B-splines

• Multiresolution, limited span, analytical derivative

zkR

yjR

xiR

ijkijkR kzkykx

zyx 222; μμrg

Local Mutual Information• Overlapping subregions

– – : Spatial label, spatial bins

• Multidimensional mutual information?– Total correlation, 3 channel regional MI

• ‘Amount of redundancy in a set of variables’•

– Conditional MI• ‘MI between R and F when X is known’•

– …

XFRHXHFHRHXFRC ,,)(,,

x,,, frpfrp

xp

XFRHXFHXRHXFRI ,,

Conditional MI (cMI) XRH XFH

XFRH ,

, ,I R F X H R X H F X H R F X

+ – =

Locally, if I know p(r), can I better predict p(f)?

H R H F

H X

,, , log

,

x r f

x

p r fI R F X p p r f

p r p f

p I R F

x x

xx x

x x

x

Validation

In theory

R F

Global MI:whole imagelocal optimum

31

2

0

00~

AA

AH

3

2

00

0~

0~

A

AH

Conditional MI:central regionglobal optimum

Multimodal registration• 200 2D image pairs

– ‘CT’, ‘MR’– 256x256 pixels– I = 0, 200, 400, noise s

=50– Mesh spacing 32 voxels– 32 bins, PW, PV

• Initial transformation– m uniform, < 30 pixels

• Validation– Intensity difference– Warping index– ROI: 10% outside polygon

CT (

ori

gin

al)

MR

(warp

ed)

PV

, co

ndit

ional

MI

PW

,conditio

nal

MI

PV

, glo

bal M

I

PW

, glo

bal M

I

Multimodal registration

CT (

ori

gin

al)

MR

(warp

ed)

PV

, co

ndit

ional

MI

PW

,conditio

nal

MI

PV

, glo

bal M

I

PW

, glo

bal M

I

Bias field registration

• 200 2D image pairs– Lena image– 8 bit, 256x256 pixels– Floating: Bias field

• 2nd degree multiplicative• •

• Initial transformation– m uniform, < 30 pixels

• Validation– Intensity difference– Warping index

edycxbxyax 22

75.0,25.0,25.0,75.0

CT (

ori

gin

al)

MR

(warp

ed)

PV

, co

ndit

ional

MI

PW

,conditio

nal

MI

PV

, glo

bal M

I

PW

, glo

bal M

I

Bias field registration

CT (

ori

gin

al)

MR

(warp

ed)

PV

, co

ndit

ional

MI

PW

,conditio

nal

MI

PV

, glo

bal M

I

PW

, glo

bal M

I

Conclusion• Small structures

– Global joint entropy has two local optima• Bias towards minimal floating entropy

– Local joint entropy has single optimum• Combination of joint and marginal entropy

• Bias fields– Global MI

• Combines contributions from all over the image• Bias field: widening of tissue histogram peaks

– Local MI• Local histograms only• Less widening

Conclusion

• Mesh size?– Smaller mesh = fewer voxels/bin = less statistical

power– Compensated by overlapping B-spline windows (?)– 2D: 9216 voxels/bin (~ 3D: 7x7x7 voxels)

• Calculation time – 10x for 2D (200s), 20x for 3D (days)

• Other conditional similarity measures– Conditional Cross Correlation

Questions?