Ms thesis-final-defense-presentation

248
Computer Assisted Screening of Microcalcifications in Digitized Mammogram for Early Detection of Breast Cancer Thesis Presentation Nashid Alam Registration No: 2012321028 [email protected] Supervisor: Prof. Dr. Mohammed Jahirul Islam Department of Computer Science and Engineering Shahjalal University of Science and Technology Driving research for better breast cancer treatment The best protection is early detection0 10 20 30 0 20 40 -0.1 0 0.1 5; 0.5; 0.7854 5; 0.5; 0.7854 Thursday, December 23, 2015

Transcript of Ms thesis-final-defense-presentation

Page 1: Ms thesis-final-defense-presentation

Computer Assisted Screening of Microcalcifications in Digitized Mammogram for Early Detection of Breast CancerThesis Presentation

Nashid AlamRegistration No: [email protected]

Supervisor: Prof. Dr. Mohammed Jahirul Islam

Department of Computer Science and EngineeringShahjalal University of Science and Technology

Driving research for better breast cancer treatment “The best protection is early detection”

010

2030

020

40-0.1

0

0.1

5; 0.5; 0.7854

5; 0.5; 0.7854

Thursday, December 23, 2015

Page 2: Ms thesis-final-defense-presentation

Introduction

Breast cancer:The most devastating and deadly diseases for women.

o Computer aided detection (CADe) o Computer aided diagnosis (CADx) systems

We will emphasis on :

Page 3: Ms thesis-final-defense-presentation

Background Interest

Page 4: Ms thesis-final-defense-presentation

Background Interest

Interest comes from two primary backgrounds

1. Improvement of pictorial information- - For Human Perception

How can an image/video be made more aesthetically pleasing

How can an image/video be enhanced to facilitate:extraction of useful information

Page 5: Ms thesis-final-defense-presentation

Background Interest

Interest comes from two primary backgrounds

2. Processing of data for:Autonomous machine perception- Machine Vision

Page 6: Ms thesis-final-defense-presentation

Micro-calcification

Mammography

Mammogram

Page 7: Ms thesis-final-defense-presentation

Micro-calcification

Background knowledge

Page 8: Ms thesis-final-defense-presentation

Micro-calcification

Micro-calcifications :- Tiny deposits of calcium- May be benign or malignant- A first cue of cancer.

Position:1. Can be scattered throughout the mammary gland, or 2. Occur in clusters.(diameters from some µm up to approximately 200 µm.)3. Considered regions of high frequency.

Page 9: Ms thesis-final-defense-presentation

Micro-calcification

They are caused by a number of reasons:

1. Aging –The majority of diagnoses are made in women over 50

2. Genetic –Involving the BRCA1 (breast cancer 1, early onset) and

BRCA2 (breast cancer 2, early onset) genes

Micro-calcifications Pattern Determines :The future course of the action-

I. Whether it be further investigatory techniques (as part of the triple assessment), or

II. More regular screening

Page 10: Ms thesis-final-defense-presentation

Mammography

Background knowledge

Mammography Machine

Page 11: Ms thesis-final-defense-presentation

Mammography

USE:I. Viewing x-ray imageII. Manipulate X-ray image on a computer screen

Mammography :

Process of using low-energyx-rays to examine the human breast

Used as a diagnostic and a screening tool.

The goal of mammography :The early detection of breast cancer

Mammography Machine

Page 12: Ms thesis-final-defense-presentation

Mammogram

Background knowledge

mdb226.jpg

Page 13: Ms thesis-final-defense-presentation

Mammogram

Mammogram:An x-ray picture of the breast

Use:To look for changes that are not normal.

Result Archive:The results are recorded:

1. On x-ray film or 2.Directly into a computer

mdb226.jpg

Page 14: Ms thesis-final-defense-presentation
Page 15: Ms thesis-final-defense-presentation

Literature Review

To detect micro-calcifications:-A number of methods have been proposed

These include:

Global and local thresholding

Statistical approaches

Neural networks

Fuzzy logic

Thresholding of wavelet coefficients and related techniques.

Literature Review

Page 16: Ms thesis-final-defense-presentation

Focus:Preprocessing Techniques of Mammogram:

Goals:- Pectoral mussel identification- Noise removal- Image enhancement

-No method gives full satisfaction andclinically acceptable results.

Drawbacks:

Literature Review

local range modification algorithmIntegrated wavelets form MC modelStein's thresholding [7]for denoisingUse Contourlets

Method used:Watershed transformationBoundary based methodHybrid techniquesThresholding techniques

Heinlein et.al(2003) [1]Zhibo et.al.(2007) [2]Papadopoulus et al. (2008) [3]

Razzi et al.(2009) [4]Pronoj et al.(2011) [5]Camilus et al.(2011) [6]

Page 17: Ms thesis-final-defense-presentation

Focus:Local feature extraction of Mammogram:

Pal et al.(2008) [8]Yu et al. (2010) [9]

Oliver et al.(2010) [10]

Goals:

- Detect microcalcification and MC cluster

- Only deals with MC morphology

-Position of microcalcifications(Take into account)

-To segment mammogram:Only salient fracture are computed

Drawbacks:

Literature Review

Method used:Inspect local neighborhood of each MCWeighted density functionFuzzy Shell Clustering

Training stage: Pixel-based boosting classifierMulti-layered perception networkBack propagated neural network

Balakumaran et.al.(2010) [11] Oliver et al.(2012) [12]Zhang et.al.(2013)[13]

Page 18: Ms thesis-final-defense-presentation

Literature Review

Focus:Wavelet based Techniques

Wang et.al.(1989) [14]Daubechies I.(1992) [15]Strickland et.at (1996) [16]Papadopoulus et al. (2008) [3]

Goals:

Method used:two-stage decomposition wavelet filteringdiscrete wavelet linear stretching and shrinkage algorithm.low-frequency subbands are discardedbiorthogonal filter bank used

Drawbacks:

-Cluster was considered:if more then 3 microcalcifications

were detected in a 1cm2 area

- Detect microcalcification and MC cluster

Razzi et al.(2009) [4]Yu et al.(2010) [9]Balakumaran et.al.(2010) [11]Zhang et.al.(2013) [13]

Page 19: Ms thesis-final-defense-presentation

Literature Review

Focus:Analysis of large massesinstead of microcalcifications

Zhibo et.al.(2007)[2]Lu et.al.(2013) [17]

Goals:Drawbacks:

Method used:

Mass Detection

Multiscale regularized reconstructionHybrid Image Filtering MethodNoise regularization in DBT reconstructionUse Contourlets

- Detecting subtle mass lesionsin Digital breast tomosynthesis (DBT)

- Only detect large mass

Digital BreastTomosynthesis captures

PHOTO COURTESY :http://www.itnonline.com/article/trends-breast-imaginghttp://www.hoag.org/Specialty/Breast-Program/Pages/breast-screening/screening-types/Tomosynthesis.aspx

Page 20: Ms thesis-final-defense-presentation

Literature Review

Focus:Detect /Classify mammograms

Fatemeh et.al.(2007) [18]

Goals:

Drawbacks:

Method used:

Automatic mass classification

Contourlets Transform

Does not give full satisfaction andclinically acceptable results.

PHOTO COURTESY :https://www.youtube.com/watch?v=kRwKO5k6pi

Mammogram

Page 21: Ms thesis-final-defense-presentation

Literature Review

Focus:

Template matching algorithm

Leeuw et.al.(2014) [7]

Goals: Drawbacks:

Method used:

Detect microcalcifications in breast specimens

Phase derivative to detect microcalcifications

Used MRI instead of mammogram

Breast MRIBreast MRI Machine

PHOTO COURTESY :http://www.leememorial.org/mainlanding/Breast_mri.asp

Page 22: Ms thesis-final-defense-presentation

Literature Review

Focus:

Goals:Insertion of simulated microcalcification clusters:

- In a software breast phantom

PHOTO COURTESY :http://www.math.umaine.edu/~compumaine/index.html

Left: Cluster microcalcification in breast tissue. Right: Simulated cluster microcalcification.

-Algorithm developed as part ofa virtual clinical trial (VCT) :

-Simulation of breast anatomy, - Mechanical compression- Image acquisition- Image processing- Image displaying and interpretation.

Shankla et.al.(2014)[19]

Page 23: Ms thesis-final-defense-presentation

Problem Statement

Page 24: Ms thesis-final-defense-presentation

Burdensome Task Of Radiologist : Eye fatigue:

-Huge volume of images-Detection accuracy rate tends to decrease

Non-systematic search patterns of humansPerformance gap between :

Specialized breast imagers andgeneral radiologists

Interpretational Errors:Similar characteristics:

Abnormal and normal microcalcification

Problem Statement

Reason behind the problem( In real life):

Page 25: Ms thesis-final-defense-presentation

The signs of breast cancer are:

Masses CalcificationsTumorLesionLump

Individual Research Areas

Problem Statement

Page 26: Ms thesis-final-defense-presentation

Motivation to the Research

Page 27: Ms thesis-final-defense-presentation

Motivation to the research: Goal

Better Cancer Survival Rates(Facilitate Early Detection ).

Provide “second opinion” : Computerized decisionsupport systems

Fast,Reliable, andCost-effective

Overcome:The development of breast cancer

Page 28: Ms thesis-final-defense-presentation

Challenges

Page 29: Ms thesis-final-defense-presentation

Develop a logistic model:

Feature extraction Challenge:

-To determine the likelihood of CANCEROUS AREA -- From the image values of mammograms

Challenge:Occur in clusters

The clusters may vary in size from 0.05mm to 1mm in diameter.

Variation in signal intensity and contrast.May located in dense tissue

Difficult to detect.

Challenges

Page 30: Ms thesis-final-defense-presentation

Materials and Tools

Matlab 2014

Database: mini-MIAS

Page 31: Ms thesis-final-defense-presentation

Database: mini-MIAS databasehttp://peipa.essex.ac.uk/pix/mias/

Page 32: Ms thesis-final-defense-presentation

Class of Abnormality

Severity of Abnormality

The Location of The

Center of The

Abnormality and It’s

Diameter.

1 Calcification(25)

1.Benign(Calc-12)

2 Circumscribed Masses

3 Speculated Masses

4 Ill-defined Masses

5 Architectural Distortion

2.Malignant(Cancerous)

(Calc-13)

6 Asymmetry

7Normal

mdb223.jpg mdb226.jpg

mdb239.jpg mdb249.jpg

Figure01:X-ray image form mini-MIAS database

Database: Mini-MIAS Databasehttp://peipa.essex.ac.uk/pix/mias/

Mammography Image Analysis Society (MIAS) -An organization of UK research groups

Page 33: Ms thesis-final-defense-presentation

• Consists of 322 images-- Contains left and right breast images for 161 patients

• Every image is 1024 X 1024 pixels in size

• Represents each pixel with an 8-bit word

• Reduced in resolution(Is not good enough for MC to be detectable)

•Very Poor Quality with .jpg compression effects(Original MIAS doesn’t have such artifacts)

Mini-MIAS Database

Mammography Image Analysis Society (MIAS) -An organization of UK research groups

Database: http://peipa.essex.ac.uk/pix/mias/

http://see.xidian.edu.cn/vipsl/database_Mammo.html

Page 34: Ms thesis-final-defense-presentation

Plan of Action

Where Are We? Our Current Research Stage

Thesis SemesterM-3

Page 35: Ms thesis-final-defense-presentation

Chart 01: Gantt Chart of this M.Sc thesis Showing the duration of task against the progression of time

Where Are We? Our Current Research Stage

Thesis SemesterM-3

Page 36: Ms thesis-final-defense-presentation

Schematic representation of the system

Page 37: Ms thesis-final-defense-presentation

Sche

mat

ic r

epre

sent

atio

n of

the

syst

em

Page 38: Ms thesis-final-defense-presentation

Removing Pectoral MuscleAnd

X-ray Label

Page 39: Ms thesis-final-defense-presentation
Page 40: Ms thesis-final-defense-presentation

X-ray Label Removing Finding The Big BLOB

The types X-ray Label:High Intensity Rectangular LabelLow Intensity LabelTape Artifacts

Page 41: Ms thesis-final-defense-presentation

X-ray Label Removing

1. Histogram equalization of the original X-ray image

2. Adjust image contrast

3. Apply Otsu's Thresholding Method [20] and

find bi-level the image which has several blobs in it.

4. Finding the Largest blob (Bwlargest.bolb)

5. Hole filling within the blob region

6. Keep the true pixel value covering only the area of largest blob and discard other features from the original image

7. X-ray label is successfully removed

Plan of Action

[20] Otsu, N., "A Threshold Selection Method from Gray-Level Histograms," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, 1979, pp. 62-66.

To Achieve The Desired Final Result:Apply:

A Range Of Techniques on original image

Page 42: Ms thesis-final-defense-presentation

1.Original image

2.HistogramEqualization

3.Contrast Image

4.Binary Image

mdb239.jpg

Combining Range of techniques

J = histeq(I); %histogram equalization

contrast_image = imadjust(J, stretchlim(J), [0 1]); %high contrast image

%Apply Thresholding to the Image level = graythresh(contrast_image);

%GRAYTHRESH Global image threshold using %Otsu's methodbw_image = im2bw(contrast_image, level);%getting binary image

X-ray Label Removing

Page 43: Ms thesis-final-defense-presentation

5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

Combining Range of techniquesX-ray Label Removing

Page 44: Ms thesis-final-defense-presentation

Result image(Label Removed)

Original image

Compare the original and final image

X-ray Label Removing

Page 45: Ms thesis-final-defense-presentation

Experimental results

X-ray Label Removing

Page 46: Ms thesis-final-defense-presentation

X-ray Label Removing1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

mdb212.jpg

mdb214.jpg

mdb214.jpg

mdb218.jpg

mdb219.jpg

Benign

Page 47: Ms thesis-final-defense-presentation

X-ray Label Removing Benign1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

mdb222.jpg

mdb223.jpg

mdb226jpg

mdb227jpg

Page 48: Ms thesis-final-defense-presentation

X-ray Label Removing Benign1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

mdb226.jpg

mdb240.jpg

mdb248.jpg

mdb252.jpg

Page 49: Ms thesis-final-defense-presentation

X-ray Label Removing Malignant1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

mdb209.jpg

mdb211.jpg

mdb213.jpg

mdb216.jpg

mdb231.jpg

Page 50: Ms thesis-final-defense-presentation

X-ray Label Removing Malignant1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

mdb233.jpg

mdb238.jpg

mdb239.jpg

mdb241.jpg

Page 51: Ms thesis-final-defense-presentation

X-ray Label Removing Malignant1.Original image

2.HistogramEqualization 3.Contrast Image 4.Binary Image 5.Finding biggest blob

6.Hole fillingInside the blob

7.Result image(Label Removed)

mdb245.jpg

mdb249.jpg

mdb253.jpg

mdb256.jpg

Page 52: Ms thesis-final-defense-presentation

Successful

X-ray Label Removing

Finally!

Page 53: Ms thesis-final-defense-presentation

Removing pectoral muscleKeeping fatty tissues and ligaments

mdb212.jpg(a)Main Image (b)Result Image

mdb213.jpg(a)Main Image (b)Pectoral Muscle

mdb214.jpg

Main Image

Result Image

Page 54: Ms thesis-final-defense-presentation

o Fat t y t i s s ue are ao Duc to Lobul e so Si nuso l i gam e nt s

Extraction of ROIRemoving pectoral muscle

Why removing pectoral muscle?

o Pectoral muscle will never contain micro-calcification

o Less Computational Time And Cost-Operation on small image area

Existence of micro-calcification:

ROI

Page 55: Ms thesis-final-defense-presentation

Edge Detection of pectoral muscleRemoving pectoral muscle

Points to be noted :

-Pectoral muscle a Triangular areamdb212.jpg

mdb214.jpg

Based on this point: Moving on towards solution

mdb209.jpg

(2)Binary Image(1)Original Image

Page 56: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

1. Find the triangular area of the pectoral muscle region

I. Finding white seeding pointII. Finding the 1st black point of 1st row after getting a white seeding pointIII. Draw a horizontal line in these two points.IV. finding the 1st black point of 1st column after getting a white seeding pointV. Draw a vertical line and angular line.

2. Making all the pixels black(zero)resides in the pectoral muscle area

Triangle Detection of pectoral muscle

Visualization in next slide

Page 57: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

Approach-03(Triangle Detection of pectoral muscle):

mdb212.jpg1.Original image

2.Contrast stretching

3.Binary of contrast image

stratching_in_range=uint8(imadjust(I,[0.01 0.7],[1 0]));

BW=~stratching_in_range;

Page 58: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

Approach-03(Triangle Detection of pectoral muscle):

4.Triangle

5.Triangle Filled

6.muscle removed

Page 59: Ms thesis-final-defense-presentation

Experimental results

Removing pectoral muscleApproach-03(Triangle Detection of pectoral muscle):

Triangle Detection of pectoral muscle

Page 60: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

mdb212.jpg

mdb214.jpg

1.Original image 2.Contrast stretching 3.Binary of contrast image 4.Triangle

mdb240.jpg

mdb248.jpg

5.Triangle Filled 6.muscle removed

Class: Benign

Page 61: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

mdb222.jpg

mdb226.jpg

mdb227.jpg

2.Contrast stretching1.Original image 3.Binary of contrast image 4.Triangle

Problems faced

5.Triangle Filled 6.muscle removed

The triangle does not always indicates the proper pectoral muscle area.Reason:

Class: Benign

Artifacts in mammogram

Page 62: Ms thesis-final-defense-presentation

2.Contrast stretching1.Original image

3.Binary of contrast image 4.Triangle

5.Triangle Filled 6.muscle removed

Triangle Detection of pectoral muscleRemoving pectoral muscle

Problems faced:

Defects in mammogram (Vertical Stripe Missing)

mdb227.jpg

Class: Benign

Page 63: Ms thesis-final-defense-presentation

mdb223.jpg

2.Contrast stretching1.Original image 3.Binary of contrast image 4.Triangle 5.Triangle Filled 6.muscle removed

Triangle Detection of pectoral muscleRemoving pectoral muscle

Problems faced:

Defects in mammogram (Horizontal Stripe Missing)Solution: Replicate the 2nd and 3rd row)

Class: Benign

Page 64: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

1.Original image

2.Contrast stretching

3.Binary of contrast image

4.Triangle

5.Triangle Filled

6.muscle removed

Class: Malignant

mdb256.pg

Page 65: Ms thesis-final-defense-presentation

Triangle Detection of pectoral muscleRemoving pectoral muscle

mdb212.jpg

mdb214.jpg

1.Original image 2.Contrast stretching 3.Binary of contrast image 4.Triangle

mdb240.jpg

mdb248.jpg

5.Triangle Filled 6.muscle removed

Class: Benign

Page 66: Ms thesis-final-defense-presentation

Successful

Pectoral Muscle Removing

Finally!

Page 67: Ms thesis-final-defense-presentation

Improved Computer Assisted Screening

Enhancement of digitized mammogram

Goal

Page 68: Ms thesis-final-defense-presentation

MAIN NOVELTY

Input image

BandpassDirectionalsubbands

BandpassDirectionalsubbands

Page 69: Ms thesis-final-defense-presentation

Based on the classical approach used in transform methods for image processing.

1. Input mammogram

2. Forward CT

3. Subband Processing

4. Inverse CT

5. Enhanced Mammogram

Schematic representation of the system

Page 70: Ms thesis-final-defense-presentation

Contourlet transformation

Implementation Based On :

• A Laplacian Pyramid decomposition followed by -

• Directional filter banks applied on each band pass sub-band.

The Result Extracts:-Geometric information of images.

Details in upcoming slides

Main Novelty

Input image

BandpassDirectionalsubbands

BandpassDirectionalsubbands

Page 71: Ms thesis-final-defense-presentation

Why Contourlet?

Page 72: Ms thesis-final-defense-presentation

Why Contourlet?

•Decompose the mammographic image:-Into directional components:

To easily capture the geometry of the image features.

Details in upcoming slides

Target

Page 73: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

DecompositionFrequency partitioning of a directional filter bank

Decomposition level l=3

The real wedge-shape frequency band is 23=8.

horizontal directions are corresponded by sub-bands 0-3

Vertical directions are represented by sub-bands 4-7

Details in upcoming slides

Page 74: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

Decomposition

Laplacian Pyramid Level-1

Laplacian Pyramid Level-2

Laplacian Pyramid Level-3

8 Direction

4 Direction

4 Direction

(mdb252.jpg)

Page 75: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

Decomposition

Wedge-shape frequency band is 23=8.

Horizontal directions are corresponded by sub-bands 0-3

(1) sub-band 0

(2) sub-band 1

(3) sub-band 2(4) sub-band 3

Contourlet coefficient at level 4

Page 76: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

DecompositionContourlet coefficient at level 4

Wedge-shape frequency band is 23=8.

Vertical directions are represented by sub-bands 4-7

(5) sub-band 4

(6) sub-band 5

(7) sub-band 6

(8) sub-band 7

Page 77: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet TransformLaplacian Pyramid: 3 level

Decomposition

(a) Main Image(mdb252.jpg)

(b) Enhanced Image(Average in all 8 direction)

Page 78: Ms thesis-final-defense-presentation

(a) Main image(Toy Image)

Contourlet Transform Example

(b) Horizontal Direction

(c) Vertical Direction

Directional filter banks: Horizontal and Vertical

Page 79: Ms thesis-final-defense-presentation

Contourlet Transform ExampleDirectional filter banks

Horizontal directions are corresponded by sub-bands 0-3

(1) sub-band 0

(2) sub-band 1

(3) sub-band 2

(4) sub-band 3

Page 80: Ms thesis-final-defense-presentation

Contourlet Transform ExampleDirectional filter banks

Vertical directions are represented by sub-bands 4-7

(5) sub-band 4

(6) sub-band 5

(7) sub-band 6

(8) sub-band 7

Page 81: Ms thesis-final-defense-presentation

Input image

BandpassDirectionalsubbands

BandpassDirectionalsubbands

Plan-of-Action

For microcalcifications enhancement :

We use-The Contourlet Transform(CT) [21]

The Prewitt Filter.

21. Da Cunha A. L., Zhou J. and Do M. N,: The Nonsubsampled Contourlet Transform: Theory, Design, and Applications, IEEE Transactions on Image Processing,vol. 15, (2006) pp. 3089-3101

Page 82: Ms thesis-final-defense-presentation

Art-of-Action

An edge Prewitt filter to enhance the directional structures

in the image.

Contourlet transform allows decomposing the image in

multidirectional and multiscale subbands[22].

22. Laine A.F., Schuler S., Fan J., Huda W.: Mammographic feature enhancement by multiscale analysis, IEEE Transactions on Medical Imaging, 1994, vol. 13, no. 4,(1994) pp. 7250-7260

This allows finding • A better set of edges,• Recovering an enhanced mammogram with better visual characteristics.

Microcalcifications have a very small size a denoising stage is not implemented

in order to preserve the integrity of the injuries.

Decompose the digital mammogram

Using Contourlet transform

(b) Enhanced image(mdb238.jpg)

(a) Original image (mdb238.jpg)

Page 83: Ms thesis-final-defense-presentation

The Contourlet Transform

The CT is implemented by:Laplacian pyramid followed by directional filter banks (Fig-01)

Input image

BandpassDirectionalsubbands

BandpassDirectionalsubbands

Figure 01: Structure of the Laplacian pyramid together with the directional filter bank

The concept of wavelet:University of Heidelburg

The CASCADE STRUCTURE allows:- The multiscale and

directional decomposition to be independent

- Makes possible to:Decompose each scale into

any arbitrary power of two's number of directions(4,8,16…)

Figure 01

Details ………….

Decomposes The Image Into Several Directional Subbands And Multiple Scales

Page 84: Ms thesis-final-defense-presentation

Figure 02: (a)Structure of the Laplacian pyramid together with the directional filter bank(b) frequency partitioning by the contourlet transform(c) Decomposition levels and directions.

(a) (b)

Input image

BandpassDirectionalsubbands

BandpassDirectionalsubbands

Details….

(c)

DenoteEach subband by yi,j

Wherei =decomposition level and J=direction

The Contourlet TransformDecomposes The Image Into Several Directional Subbands And Multiple Scales

Page 85: Ms thesis-final-defense-presentation

The processing of an image consists on:-Applying a function to enhance the regions of

interest.

In multiscale analysis:

Calculating function f for each subband :-To emphasize the features of interest-In order to get a new set y' of enhanced subbands:

Each of the resulting enhanced subbands can be expressed using equation 1.

)(', , jiyfjiy = ………………..(1)

-After the enhanced subbands are obtained, the inverse transform is performed to obtain an enhanced image.

Enhancement of the Directional Subbands

The Contourlet Transform

Denote

Each subband by yi,jWherei =decomposition level and J=direction Details….

Page 86: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet Transform

Details….

The directional subbands are enhanced using equation 2.

=)( , jiyf)2,1(

,1 nnWjiy

)2,1(,2 nnWjiy

If bi,j(n1,n2)=0

If bi,j(n1,n2)=1………..(2)

Denote

Each subband by yi,jWherei =decomposition level and J=direction

W1= weight factors for detecting the surrounding tissueW2= weight factors for detecting microcalcifications

(n1,n2) are the spatial coordinates.

bi;j = a binary image containing the edges of the subband

Weight and threshold selection techniques are presented on upcoming slides

Page 87: Ms thesis-final-defense-presentation

Enhancement of the Directional Subbands

The Contourlet Transform

The directional subbands are enhanced using equation 2.

=)( , jiyf)2,1(

,1 nnWjiy

)2,1(,2 nnWjiy

If bi,j(n1,n2)=0

If bi,j(n1,n2)=1………..(2)

Binary edge image bi,j is obtained :-by applying : Prewitt edge detector

-To detect edges on each directional subband.

In order to obtain a binary image:A threshold Ti,j for each subband is calculated.

Details….

Weight and threshold selection techniques are presented on upcoming slides

Page 88: Ms thesis-final-defense-presentation

Threshold Selection

The Contourlet Transform

Details….

The microcalcifications appear :

On each subband Over a very

homogeneous background.

Most of the transform coefficients:

-The coefficients corresponding to theinjuries are far from background value.

A conservative threshold of 3σi;j is selected:where σi;j is the standard deviation of the corresponding subband y I,j .

Page 89: Ms thesis-final-defense-presentation

Weight Selection

The Contourlet Transform

Exhaustive tests:-Consist on evaluating subjectively a set of 322 different mammograms

-With Different combinations of values,

The weights W1, and W2 are determined:- as W1 = 3 σi;j and W2 = 4 σi;j

These weights are chosen to:keep the relationship W1 < W2:

-Because the W factor is a gain -More gain at the edges are wanted.

Page 90: Ms thesis-final-defense-presentation

Experimental Results

Page 91: Ms thesis-final-defense-presentation

Applying Contourlet Transformation Benign

Original image Enhanced image

Goal: Microcalcification Enhancement

mdb222.jpg

mdb223.jpg

Original image Enhanced image

mdb248.jpg

mdb252.jpg

Page 92: Ms thesis-final-defense-presentation

Applying Contourlet Transformation Benign

Original image Enhanced image

mdb226.jpg

mdb227.jpg

Original image Enhanced image

mdb236.jpg

mdb240.jpg

Goal: Microcalcification Enhancement

Page 93: Ms thesis-final-defense-presentation

Applying Contourlet Transformation Benign

Original image Enhanced image Original image Enhanced image

mdb218.jpgmdb219.jpg

Goal: Microcalcification Enhancement

Page 94: Ms thesis-final-defense-presentation

Applying Contourlet Transformation MalignantGoal: Microcalcification Enhancement

Original image Enhanced image

mdb209.jpg

mdb211.jpg

Original image Enhanced image

mdb213.jpg

mdb231.jpg

Page 95: Ms thesis-final-defense-presentation

Applying Contourlet Transformation MalignantGoal: Microcalcification Enhancement

Original image Enhanced image

mdb238.jpg

mdb239.jpg

Original image Enhanced image

mdb241.jpg

mdb249.jpg

Page 96: Ms thesis-final-defense-presentation

Original image Enhanced image

mdb253.jpg

Original image Enhanced image

Applying Contourlet Transformation MalignantGoal: Microcalcification Enhancement

mdb256.jpg

Page 97: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb003.jpg

mdb004.jpg

Original image Enhanced image

mdb006.jpg

mdb007.jpg

Page 98: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb009.jpg

mdb018.jpg

Original image Enhanced image

mdb027.jpg

mdb033.jpg

Page 99: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb046.jpg

mdb056.jpg

Original image Enhanced image

mdb060.jpg

mdb066.jpg

Page 100: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb070.jpg

mdb073.jpg

Original image Enhanced image

mdb074.jpg

mdb076.jpg

Page 101: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb093.jpg

mdb096.jpg

Original image Enhanced image

mdb101.jpg

mdb012.jpg

Page 102: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb128.jpg

mdb137.jpg

Original image Enhanced image

mdb146.jpg

mdb154.jpg

Page 103: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb166.jpg

mdb169.jpg

Original image Enhanced image

mdb224.jpg

mdb225.jpg

Page 104: Ms thesis-final-defense-presentation

Applying Contourlet Transformation NormalGoal: Microcalcification Enhancement

Original image Enhanced image

mdb263.jpg

mdb294.jpg

Original image Enhanced image

mdb316.jpg

mdb320.jpg

Page 105: Ms thesis-final-defense-presentation

Wavelet Transformation

Page 106: Ms thesis-final-defense-presentation

Use Separable Transform

2D Wavelet Transform

Visualization

Label ofapproximation

HorizontalDetails

HorizontalDetails

VerticalDetails

DiagonalDetails

VerticalDetails

DiagonalDetails

Page 107: Ms thesis-final-defense-presentation

Use Separable Transform

2D Wavelet Transform

Decomposition at Label 4

Original image(with diagonal details areas indicated)

Diagonal Details

Page 108: Ms thesis-final-defense-presentation

Use Separable Transform

2D Wavelet Transform

Vertical Details

Decomposition at Label 4

Original image(with Vertical details areas indicated)

Page 109: Ms thesis-final-defense-presentation

Experimental Results

Page 110: Ms thesis-final-defense-presentation

Experimental Results

DWT

1.Original Image(Malignent_mdb238) 2.Decomposition at Label 4

2.Decomposition at Label 1 3.Decomposition at Label 2 3.Decomposition at Label 3

Page 111: Ms thesis-final-defense-presentation

Experimental Results

DWT

1.Original Image(Malignent_mdb238) 2.Decomposition at Label 4

Page 112: Ms thesis-final-defense-presentation

Experimental Results

1.Original Image(Benign_mdb252) 2.Decomposition at Label 4

2.Decomposition at Label 1 3.Decomposition at Label 2 3.Decomposition at Label 3

DWT

Page 113: Ms thesis-final-defense-presentation

Experimental Results

1.Original Image(Malignent_mdb253.jpg) 2.Decomposition at Label 4

2.Decomposition at Label 1 3.Decomposition at Label 2 3.Decomposition at Label 3

Page 114: Ms thesis-final-defense-presentation

Metrics: Quantitive Measurement

Page 115: Ms thesis-final-defense-presentation

Metrics

To compare the ability of :Enhancement achieved by the proposed method

Why?

1. Measurement of distributed separation (MDS)2. Contrast enhancement of background against target (CEBT) and3. Entropy-based contrast enhancement of background against target (ECEBT) [23].

Measures used to compare:

23. Sameer S. and Keit B.: An Evaluation on Contrast Enhancement Techniques for Mammographic Breast Masses, IEEETransactions on Information Technology in Biomedicine, vol. 9, (2005) pp. 109-119

Page 116: Ms thesis-final-defense-presentation

Metrics

1. Measurement of Distributed Separation (MDS)

Measures used to compare:

The MDS represents :How separated are the distributions of each mammogram

…………………………(3)MDS = |µucalcE -µtissueE |- |µucalc0 -µtissue0 |

µucalcE = Mean of the microcalcification region of the enhanced imageµucalc0 = Mean of the microcalcification region of the original image

µtissueE = Mean of the surrounding tissue of the enhanced imageµtissue0 = Mean of the surrounding tissue of the enhanced image

Defined by:

Where:

Page 117: Ms thesis-final-defense-presentation

Metrics

2. Contrast enhancement of background against target (CEBT) Measures used to compare:

The CEBT Quantifies :The improvement in difference between the background and the target(MC).

…………………………(4)

0µucalcEµucalc

0µtissue0µucalc

EµtissueEµucalc

CEBT

σσ

−=

Defined by:

Where:

Eµucalcσ

0µucalcσ

= Standard deviations of the microcalcifications region in the enhanced image

= Standard deviations of the microcalcifications region in the original image

Page 118: Ms thesis-final-defense-presentation

Metrics

3. Entropy-based contrast enhancement of background against target (ECEBT)Measures used to compare:

The ECEBT Measures :- An extension of the TBC metric- Based on the entropy of the regions rather

than in the standard deviations

Defined by:

Where:

…………………………(5)

0µucalcEµucalc

0µtissue0µucalc

EµtissueEµucalc

ECEBT

εζ

−=

= Entropy of the microcalcifications region in the enhanced image

= Entropy of the microcalcifications region in the original image

Eµucalcζ

0µucalcε

Page 119: Ms thesis-final-defense-presentation

Experimental Results

Page 120: Ms thesis-final-defense-presentation

MDS, CEBT and ECEBT metrics on the enhanced mammograms

Experimental Results

CT Method DWT Method

MDS CEBT ECEBT MDS CEBT ECEBT0.853 0.477 0.852 0.153 0.078 0.555

0.818 0.330 0.810 0.094 0.052 0.382

1.000 1.000 1.000 0.210 0.092 0.512

0.905 0.322 0.920 1.000 0.077 1.000

0.936 0.380 0.935 0.038 0.074 0.473

0.948 0.293 0.947 0.469 0.075 0.847

0.665 0.410 0.639 0.369 0.082 0.823

0.740 0.352 0.730 0.340 0.074 0.726

0.944 0.469 0.494 0.479 0.095 0.834

0.931 0.691 0.936 0.479 0.000 0.000

0.693 0.500 0.718 0.258 0.081 0.682

0.916 0.395 0.914 0.796 0.079 0.900

Table 1. Decomposition levels and directions.

Page 121: Ms thesis-final-defense-presentation

0

0.2

0.4

0.6

0.8

1

1.2

TBC

Mammogram

MDS Matrix

CT DWT

The proposed method gives higher results than the wavelet-based method.

MDS, CEBT and ECEBT metrics on the enhanced mammograms

Experimental Results Analysis

Page 122: Ms thesis-final-defense-presentation

0

0.2

0.4

0.6

0.8

1

1.2

TBCE

Mammogram

CEBT Matrix

CT DWT

The proposed method gives higher results than the wavelet-based method.

MDS, CEBT and ECEBT metrics on the enhanced mammograms

Experimental Results Analysis

Page 123: Ms thesis-final-defense-presentation

0

0.2

0.4

0.6

0.8

1

1.2

DSM

Mammogram

ECEBT Matrix

CT DWT

The proposed method gives higher results than the wavelet-based method.

MDS, CEBT and ECEBT metrics on the enhanced mammograms

Experimental Results Analysis

Page 124: Ms thesis-final-defense-presentation

Experimental Results AnalysisMesh plot of a ROI containing microcalcifications

(a)The original mammogram

(mdb252.bmp)

(b) The enhanced mammogram

using CT

Page 125: Ms thesis-final-defense-presentation

Experimental Results Analysis

(a)The original mammogram

(mdb238.bmp)

(b) The enhanced mammogram

using CT

Page 126: Ms thesis-final-defense-presentation

Experimental Results Analysis

(a)The original mammogram

(mdb253.bmp)

(b) The enhanced mammogram

using CT

Page 127: Ms thesis-final-defense-presentation

More peaks corresponding to microcalcifications are enhanced

The background has a less magnitude with respect to the peaks:

-The microcalcifications are more visible.

Observation:

Experimental Results Analysis

Page 128: Ms thesis-final-defense-presentation

Experimental Results

(a)Original image (b)CT method (c)The DWT Method

These regions contain :• Clusters of microcalcifications (target)• surrounding tissue (background).

For visualization purposes :The ROI in the original mammogram are marked with a square.

ACHIEVEMENT

Improved Computer Assisted screen of mammogram

Page 129: Ms thesis-final-defense-presentation

Achievements!

Enhancement of MC in digitized mammogramfor diagnostic support system

Figure: Diagnostic support system

MC

Suspected

Digital mammography systems :Presents images to the Radiologist with properly image processing applied.

Page 130: Ms thesis-final-defense-presentation

Achievements!

(b) Enhanced image(mdb238.jpg)

(a) Original imageROI

(mdb238.jpg)

(a) Original imageWHOLE IMAGE (mdb238.jpg)

Digital mammography systems :Presents images to the Radiologist with properly image processing applied.

Hard to find MC Easy to find MC

Whilephysicians

interact with

The information in an image During interpretation process

Page 131: Ms thesis-final-defense-presentation

Achievements!!

Enhancement of MC in digitized mammogram

With improved visual understanding, we can develop :

ways to further improve :o Decision making ando Provide better patient care

Improved Computer Assisted Screening

Goal Accomplished

Page 132: Ms thesis-final-defense-presentation

Another Step Ahead..how about training a machine?

Page 133: Ms thesis-final-defense-presentation

Dealing with Features

Page 134: Ms thesis-final-defense-presentation

Why Feature Extraction?

Finding a feature:That has the most

discriminative information

The objective of feature selection:

Differs from its immediate surroundings by texture colorintensity

Fig: MC features (Extracted Using Human Visual Perception)

Page 135: Ms thesis-final-defense-presentation

Why Feature Extraction?

Finding a feature:That has the most

discriminative information

The objective of feature selection:

Differs from its immediate surroundings by texture colorintensity

Fig: MC (Irregular in shape and size)(Extracted Using Human Visual Perception)

MoreFeatures: Shape Size

Page 136: Ms thesis-final-defense-presentation

Why Feature Extraction?

Problems With MC Features:Irregular in shape and sizeNo definite patternLow Contrast -

Located in dense tissueHardly any color intensity variation

MC Feature

Fig: MC (Irregular in shape and size)(Extracted Using Human Visual Perception)

Page 137: Ms thesis-final-defense-presentation

Why Feature Extraction? MC Feature

How radiologist deals with feature Detection/Recognition issue ?

Using Human Visual Perception

Page 138: Ms thesis-final-defense-presentation

Why Feature Extraction? MC Feature

How Radiologist (Using Human Eye) deals with feature detection/Recognition issue ?

Using Human Visual Perception

Humans are equipped with sense organs e.g. eye-Eye receives sensory inputs and -Transmits sensory information to the brain

http://www.simplypsychology.org/perception-theories.html

Page 139: Ms thesis-final-defense-presentation

Why Feature Extraction? MC Feature

Teach the machine to see like just we doObjective:

Irregular in shape and sizeNo definite patternLow Contrast -

Located in dense tissueHardly any color intensity variation

Machine Vision Challenges:-To make sense of what it sees

In Real:MC is Extracted Using Human Visual Perception

Page 140: Ms thesis-final-defense-presentation

SURF Point Algorithm

Speeded-Up Robust Features (SURF) Algorithm

Point feature algorithm (SURF)Approach:

Page 141: Ms thesis-final-defense-presentation

Improving the prediction performance of CAD Providing a faster, reliable and cost-effective prediction

Features will facilitate:

Fig: MC Point features (Extracted Using SURF point feature algorithm)

Point feature algorithm (SURF)Approach:

Page 142: Ms thesis-final-defense-presentation

SURF point algorithm

Detect a specific object

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Objective

based on Finding point correspondences

between . The reference and the target image

Reference Image Target Image

Page 143: Ms thesis-final-defense-presentation

Context in using the features:

Feature ExtractionSURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

I. Finding Key pointsII. Matching key pointsIII. Classification

Fig. Putatively Matched Points (Including Outliers )

Page 144: Ms thesis-final-defense-presentation

Context in using the features:

Feature ExtractionSURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

I. Finding Key pointsII. Matching key pointsIII. Classification

Estimate Geometric Transformation and Eliminate Outliers

Page 145: Ms thesis-final-defense-presentation

Context in using the features:

Feature ExtractionSURF point algorithmSpeeded-Up Robust Features (SURF) algorithm to find blob features.

I. Finding Key pointsII. Matching key pointsIII. Classification

Page 146: Ms thesis-final-defense-presentation

Moving Towards MC Feature DetectionUsing

SURF Point Algorithm

Page 147: Ms thesis-final-defense-presentation

Local feature

Details In Next slide

To keep in mind

Page 148: Ms thesis-final-defense-presentation

Local Feature Detection and Extraction

Local features :

A pattern or structure :Point, edge, or small image patch.

- A pattern or structure found in an image,

Differs from its immediate surroundings bytexture colorintensity

- Associated with an image patch that:

Fig.1 : Some Image Patch We used for Feature Point Detection Purpose

Page 149: Ms thesis-final-defense-presentation

Local Feature Detection and Extraction

Applications: Image registration Object detection and classification TrackingMotion estimation

Using local features facilitates: handle scale changes rotation occlusion

Detectors /Methods :• FAST• Harris• Shi & Tomasi• MSER

• SURF

Feature Descriptors:SURFFREAKBRISKHOG descriptors

Detecting corner features

detecting blob/point features.

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Page 150: Ms thesis-final-defense-presentation

Detector Feature Type Scale IndependentFAST [24] Corner No

Minimum eigen value algorithm[25]

Corner No

Corner detector [26] Corner NoSURF [27] Blob/ Point YesBRISK [28] Corner YesMSER [29] Region with uniform

intensityYes

Local Feature Detection and Extraction

Why Using SURF Feature?Trying to identify MC cluster Blob

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Page 151: Ms thesis-final-defense-presentation

detectSURFFeatures(boxImage);

selectStrongest(boxPoints, 100)

extractFeatures(boxImage, boxPoints)

matchFeatures(boxFeatures, sceneFeatures);

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Read the reference image containing the object of interest

Read the target image containing a cluttered scene.

Detect feature points in both images.

Select the strongest feature points found in the reference image.

Select the strongest feature points found in the target image.

Extract feature descriptors at the interest points in both images.

Find Putative Point Matches using their descriptors

Display putatively matched features.

Locate the Object in the Scene Using Putative Matches

Start

End

Page 152: Ms thesis-final-defense-presentation

SURF Point Detection

1.Read the reference image

containing MC cluster

2.Target image containing MC.

2.Strongest feature point

in MC cluster

2. Strongest Feature point in Target Image

3. No match point Found

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Page 153: Ms thesis-final-defense-presentation

Are we getting less feature points?

Figure: No match point Found

Page 154: Ms thesis-final-defense-presentation

No. of SURF feature points: 2 No. of SURF feature points: 47

Image Size256*256

Image Size 549*623

Image mdb238.jpg

More features from the image extracted(most points are mismatched)

To extract relevant feature point from the image

Case 1: Consider Big Reference Image

To get more feature points

Page 155: Ms thesis-final-defense-presentation

To extract relevant feature point from the image

Case 2: Consider A bigger Reference Image andWhole mammogram as Target Image

1. Image of MC Cluster(mdb238.jpg) (256*256)

2. Main mammogram (mdb238.jpg) 1024*1024

3. 100 strongest point of ROI) (256*256) 4. 300 strongest point of Main mammogram (mdb238.jpg) 1024*1024

To get more feature points

Page 156: Ms thesis-final-defense-presentation

What we finally have? No putative match Point

To extract relevant feature point from the image

Case 2: Consider A bigger Reference Image andWhole mammogram as Target Image To get more feature points

Page 157: Ms thesis-final-defense-presentation

1. Image of an Microcalcification Cluster

Too small ROI will cause less feature points to match

2. 23 strongest pointsAmong 100 Strongest Feature Points

from reference image

Reference image: mdb248.jpgImage size: 256 *256

detectSURFFeatures(mc_cluster);

Problem 1: less number of feature points to matchSURF Feature Point

Page 158: Ms thesis-final-defense-presentation

4. Only 1 strongest pointsAmong 300 Strongest Feature Points

from Scene Image

Too small ROI will cause less feature points to match

3. Image of a Cluttered Scene

Scene image: mdb248.jpgImage size: 427*588

detectSURFFeatures(sceneImage)

Problem 1: less number of feature points to matchSURF Feature Point

Page 159: Ms thesis-final-defense-presentation

Result of small ROI (256*256):No Putative Point Matches

[mcFeatures, mc_Points] = extractFeatures(mc_cluster, mc_Points);[sceneFeatures, scenePoints] = extractFeatures(sceneImage, scenePoints);mcPairs = matchFeatures(mcFeatures, sceneFeatures);matchedmcPoints = mc_Points(mcPairs(:, 1), :);matchedScenePoints = scenePoints(mcPairs(:, 2), :);showMatchedFeatures(mc_cluster, sceneImage, matchedmcPoints, ... matchedScenePoints, 'montage');

Problem 1: less number of feature points to matchSURF Feature Point

Page 160: Ms thesis-final-defense-presentation

Image Image Size Number of feature points

1190*589 15

588*427 23

256*256 1

541*520 86

Varying image size to see the effect to get SURF feature points

Page 161: Ms thesis-final-defense-presentation

Approach-01 to solve: Considering the Whole image(Label and Pectoral Muscle)

Image size No. of SURF feature points

1024*1024 63

Target:To acquire more feature

Page 162: Ms thesis-final-defense-presentation

2. Irrelevant Feature Points

Image size No. of SURF feature points

1024*1024 63

1. Less Feature points

Approach-01 to solve: Considering the Whole image(Label and Pectoral Muscle)

Target:To acquire more feature

Result:

Page 163: Ms thesis-final-defense-presentation

Image size No. of SURF feature points

255*256 2

Approach-02 : Detect feature from the cropped image

Target:To acquire more feature

Page 164: Ms thesis-final-defense-presentation

Image size No. of SURF feature points

256*256 2

Target:To acquire more feature

2. Relevant Feature Points

1. Less Feature pointsResult:

Approach-02 : Detect feature from the cropped image

Page 165: Ms thesis-final-defense-presentation

Observation from approach 1 and 2

1. Image Size does not affectThe number of Feature Points

2. Zooming an image mayhelp to extract relevant featuresfrom the image(very few points to match)

mdb238.jpgImage Size: 1024*1024

mdb238.jpgImage Size: 256*256

Page 166: Ms thesis-final-defense-presentation

Observation:Varying image size is not helping to get feature points

Image of an Microcalcification Cluster

23 strongest pointsAmong 100 Strongest Feature Points

from reference image

Reference image: mdb248.jpgImage size: 256 *256

Only 1 strongest pointsAmong 300 Strongest Feature Points

from Scene Image

Scene image: mdb248.jpgImage size: 427*588

Page 167: Ms thesis-final-defense-presentation

Observing SURF Drawback

This method works best for :-- Detecting a specific object

(for example, the elephant in the reference image,rather than any elephant.)

-- Non-repeating texture patterns-- Unique feature

This technique is not likely to work well for:-- Uniformly-colored objects-- Objects containing repeating patterns.

detecting blob /point features. AIM Failed

Speeded-Up Robust Features (SURF) algorithm to find blob features.

Page 168: Ms thesis-final-defense-presentation

Image Correlation Technique

Page 169: Ms thesis-final-defense-presentation

Alternate ApproachImage Correlation Technique

Correlation

∑∑ ++=⊗k l

kjkihlkfhf ))((),(=f Image

=h Kernel/Mask

f1 f2 f3

f4 f5 f6

f7 f8 f9

h1 h2 h3

h4 h5 h6

h7 h8 h9

f1h1 f2h2 f3h3

f4h4 f5h5 f6h6

f7h7 f8h8 f9h9

=⊗ hf⊗

Page 170: Ms thesis-final-defense-presentation

Experimental ResultsImage Correlation Technique

Page 171: Ms thesis-final-defense-presentation

Image no: Benign mdb218.jpg

1. Original image

2. Kernel/ Mask/Template

3. Correlation Output

4. Identified MC(High value of sum.)

Page 172: Ms thesis-final-defense-presentation

Image no: Benign mdb219.jpg

Page 173: Ms thesis-final-defense-presentation

Image no: Benign mdb223.jpg

Page 174: Ms thesis-final-defense-presentation

Image no: Benign mdb226.jpg

Page 175: Ms thesis-final-defense-presentation

Image no: Benign mdb227.jpg

Page 176: Ms thesis-final-defense-presentation

Image no: Benign mdb236.jpg

Page 177: Ms thesis-final-defense-presentation

Image no: Benign mdb248.jpg

Page 178: Ms thesis-final-defense-presentation

Image no: Benign mdb252.jpg

Page 179: Ms thesis-final-defense-presentation

(Fixed Template Problem)..

Page 180: Ms thesis-final-defense-presentation

Image no: Benign mdb222.jpg(Fixed Template Problem) Cont….

Page 181: Ms thesis-final-defense-presentation

(Fixed Template Problem)..

Page 182: Ms thesis-final-defense-presentation

Using Gabor Filter

Page 183: Ms thesis-final-defense-presentation

Using Gabor Filter

• Make Gabor patch:

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

5; 2; 1.5708

• Correlate the patch with image-To extract features of MC

⊗ =

Page 184: Ms thesis-final-defense-presentation

0 10 20 30 40 50 60 70 80 90 100-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Creating Gabor Mask

1. Linear RAMP

2. Linear RAMP values across: Columns Xm (left) and Rows Ym (Right)

3. Linear RAMP values across - Columns(Xm)

The result in the spatial domain

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5Xm (Across Columns) Ym- (Across rows)

Page 185: Ms thesis-final-defense-presentation

4. Across Columns, Xm :a) Increase frequencyb )Use gray color map

6. Adding Xm and Ymtogether in

different proportions5. Across Rows, Ym :

a) Increase frequencyb )Use gray color map

Creating Gabor Mask

Page 186: Ms thesis-final-defense-presentation

7. Create Gaussian Mask

8. Multiply Grating and Gaussian

Grating Gaussian Mask

Creating Gabor Mask

Page 187: Ms thesis-final-defense-presentation

7. GABOR Mask

Creating Gabor Mask

Page 188: Ms thesis-final-defense-presentation

Alternate ApproachUsing Gabor Filter Gabor kernel

2; 2; 0.7854

2; 0.5; 0.7854

2; 2; 1.5708

5; 0.5; 1.5708

5; 2; 0.7854

2; 0.5; 1.5708

5; 0.5; 0.7854

Scale , frequency, orientation

5; 2; 1.5708

MatrixSize = 26; %always scalar!

Scales = [2, 5];Orientations = [pi/4, pi/2];Frequencies = [0.5, 2];

CenterPoints = [13 13]; %int type (eg. [5 5; 13 13])

CreateMethod = FilterBank.CREATE_CROSSPRODUCT;

Page 189: Ms thesis-final-defense-presentation

010

2030

020

40-0.5

0

0.5

2; 2; 0.7854

010

2030

020

40-0.2

0

0.2

2; 0.5; 0.7854

010

2030

020

40-0.2

0

0.2

2; 0.5; 1.5708

010

203

020

40-0.2

0

0.2

5; 2; 0.7854

010

2030

020

40-0.2

0

0.2

5; 2; 1.5708

010

2030

020

40-0.1

0

0.1

5; 0.5; 0.7854

010

2030

020

40-0.1

0

0.1

5; 0.5; 1.5708

010

2030

020

40-0.5

0

0.5

2; 2; 1.5708

Using Gabor Filter Gabor kernel

Page 190: Ms thesis-final-defense-presentation

; 0 5; 5 08

Using Gabor Filter

=

=

=

Page 191: Ms thesis-final-defense-presentation

Using Gabor Filter

=

=

=

⊗ =

Page 192: Ms thesis-final-defense-presentation

Image In Spatial DomainUsing Gabor Filter Final Scenario

Page 193: Ms thesis-final-defense-presentation

mini-MIAS drawbacksExperimental Realization

Page 194: Ms thesis-final-defense-presentation

mini-MIAS drawbacksBenign mdb218

Original Enhanced

Gabor Effects

Page 195: Ms thesis-final-defense-presentation

mini-MIAS drawbacksBenign mdb218

Original

Enhanced

Gabor Effects

Page 196: Ms thesis-final-defense-presentation

Observation 1

Page 197: Ms thesis-final-defense-presentation

mini-MIAS drawbacksBenign mdb218

Original Enhanced

- NO definite Feature found

Gabor Effects

OBSERVATION-1:

Page 198: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb222.jpgBenign

OBSERVATION-1:

-NO definite feature of MC

Page 199: Ms thesis-final-defense-presentation

mini-MIAS drawbacksBenign mdb218

Original Enhanced

Are these really enhanced?

-There is more detail, but could be noise.

Question Arise?

Gabor Effects

Page 200: Ms thesis-final-defense-presentation

mini-MIAS drawbacks

Enhanced version can contain Noise

Experimental Realization

1.Very Poor Quality with .jpg compression effects

a) Original image b) Enhanced image b) Enhanced imagea) Original image

mdb209

mdb213

mdb219

mdb249

Page 201: Ms thesis-final-defense-presentation

mini-MIAS drawbacks

Not good enough for MC to be detectable

Experimental Realization

2. Reduced in resolution

Benign mdb218

Original Enhanced

Page 202: Ms thesis-final-defense-presentation

Observation 2

Page 203: Ms thesis-final-defense-presentation

mini-MIAS drawbacks

Not good enough for MC to be detectable

Experimental Realization

2. Reduced in resolution

Benign mdb218

Original

Enhanced

Where is MC?

OBSERVATION-2:

-There is more detail, but could be noise.

-Enhanced versionseems to contain compression artifacts.

Page 204: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb226.jpgBenign

OBSERVATION-2:

- Bad resolution- Noise dominant- No definite feature of MC

Page 205: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb227.jpgBenign

OBSERVATION-2:

- Bad resolution/Poor quality image

- No definite feature of MC

Page 206: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb236.jpgBenign

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 207: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb240.jpgBenign

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 208: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb209.jpgMalignant

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 209: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb211.jpgMalignant

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 210: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb213.jpgMalignant

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 211: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb231.jpg

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 212: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb238.jpg

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 213: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb253.jpg

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 214: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb256.jpg

OBSERVATION-2:

- Bad resolution-No definite feature of MC- Noise dominant

Page 215: Ms thesis-final-defense-presentation

Observation 3

Page 216: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb219.jpgBenign

OBSERVATION-3:

-Image Smoothing to remove edge will

Vanish the existenceof MC

-No definite feature of MC- Noise dominant

Page 217: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb239.jpg

OBSERVATION-3:

-Image Smoothing to remove edge will

Vanish the existenceof MC

-No definite feature of MC- Noise dominant

Page 218: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb241.jpg

OBSERVATION-3:

-Image Smoothing to remove edge will

Vanish the existenceof MC

-No definite feature of MC- Noise dominant

Page 219: Ms thesis-final-defense-presentation

More Evaluation (Gabor)Malignant mdb249.jpg

OBSERVATION-3:

-Image Smoothing to remove edge will

Vanish the existenceof MC

-No definite feature of MC- Noise dominant

Page 220: Ms thesis-final-defense-presentation

Observation 4,5,6

Page 221: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb223.jpgBenign

OBSERVATION-4:

-NO definite feature of MCFalse contour

Page 222: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb223.jpgBenign

OBSERVATION-5:

-NO definite feature of MCFalse contour

No feature

Page 223: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb223.jpgBenign

OBSERVATION-6:

-NO definite feature of MCFalse contour

No feature

Several similar area false positive o/p

Page 224: Ms thesis-final-defense-presentation

Observation 7

Page 225: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb248.jpgBenign

OBSERVATION-7:

-feature of MC-But MC has different

orientationin different image

Page 226: Ms thesis-final-defense-presentation

More Evaluation (Gabor)mdb252.jpgBenign

OBSERVATION-7:

-feature of MC-But MC has different

orientationin different image

Page 227: Ms thesis-final-defense-presentation

Observation &

Drawing Conclusion

Future detection

Page 228: Ms thesis-final-defense-presentation

Observation & Drawing Conclusion Feature Detection

• Reduced in resolution(Is not good enough for MC to be detectable)

• Very Poor Quality with .jpeg compression effects(Original MIAS doesn’t have such artifacts)

Limitations of mini-MIAS:

What can be done using mini-MIAS ?

• Can be used for big object detection(Pectoral Muscle, X-ray Label, Tumor, Mass detection)

Conclusion: mini-MIAS is not a good choice for:MC feature extraction

Page 229: Ms thesis-final-defense-presentation

Observation & Drawing Conclusion Feature Detection

Any alternative to mini-MIAS?

Page 230: Ms thesis-final-defense-presentation

Observation & Drawing Conclusion Feature Detection

Database Name Authority

MIAS ( Mammographic Image Analysis Society Digital Mammogram Database)

Mammography Image Analysis Society- an

organization of UK research groups

DDSM (Digital Database for Screening Mammogram) University Of South Florida, USA

NDM (National Mammography Database) American College Of Radiology, USA

LLNL/UCSF Database

Lawrence Livermore National Laboratories

(LLNL), University of California at San Fransisco (UCSF)

Radiology Dept.

Page 231: Ms thesis-final-defense-presentation

Observation & Drawing Conclusion Feature Detection

Database Name Authority

Washington University Digital Mammography Database Department of Radiology at the

University of Washington, USA

Nijmegen Database Department of Radiology at the

University of Nijmegen, the

Netherlands

Málaga mammographic database University of Malaga Central Research

Service (SCAI) ,Spain

BancoWeb LAPIMO Database Electrical Engineering Department at

Universidad de São Paulo, Brazil

Page 232: Ms thesis-final-defense-presentation

Observation & Drawing Conclusion Feature Detection

These databases are NOT FREE

Page 233: Ms thesis-final-defense-presentation

Research Findings

5; 0.5; 0.7854

Page 234: Ms thesis-final-defense-presentation

Research FindingsImproved computer assisted

screening of mammogram

Detection and removal of big objects:- Pectoral Muscle - X-ray level

MC

Suspected

Page 235: Ms thesis-final-defense-presentation

Observation & Drawing Conclusion On

Feature Detection

• Reduced in resolution(Is not good enough for MC to be detectable)

• Very Poor Quality with .jpeg compression effects(Original MIAS doesn’t have such artifacts)

Limitations of mini-MIAS:

What can be done using mini-MIAS ?

• Can be used for big object detection(Pectoral Muscle, X-ray Label, Tumor, Mass detection)

Conclusion: mini-MIAS is not a good choice for:MC feature extraction

BesideResearch Findings…

Page 236: Ms thesis-final-defense-presentation

Published PaperAvailable Online:

http://cennser.org/IJCVSP/paper.html

Page 237: Ms thesis-final-defense-presentation

Published PaperAvailable Online:

http://cennser.org/IJCVSP/paper.html

Page 238: Ms thesis-final-defense-presentation

Published PaperAvailable Online:

http://cennser.org/IJCVSP/paper.html

Page 239: Ms thesis-final-defense-presentation

Submitted Paperhttp://www.journals.elsevier.com/image-and-vision-computing/

Page 240: Ms thesis-final-defense-presentation

Further Research ScopeThere is always more to work on..In Research:

Page 241: Ms thesis-final-defense-presentation

Future Plan

1. Segment the image

2. Find out the feature from the segmented image

3. Train the machine with features:

-ANN (Artificial Neural Network)-SVM (Support Vector Machine)

- GentleBoost Classifier [30]

4. Identify the MC5. Classify the MC

Available options

Page 242: Ms thesis-final-defense-presentation

[1]Heinlein P., Drexl J. and Schneider Wilfried: Integrated Wavelets for Enhancement ofMicrocalcifications in Digital Mammography, IEEE Transactions on Medical Imaging, Vol.22, (2003) pp. 402-413

[2]. Zhibo Lu, Tianzi Jiang, Guoen Hu, Xin Wang: Contourlet based mammographicimage enhancement, Proc. of SPIE, vol. 6534, (2007) pp. 65340M-1 - 65340M-8

[3]A.Papadopoulos, D.I . Fotiadis, L.Costrrido,” Improvement of microcalcification clusterdetection in mammogaphy utilizing image enhancement techniques”.Comput.Bio.Med.10,Vol 38,Issue 38,pp.1045-1055,2008

[4]M.Rizzi, M.D’Aloia, B.Castagnolo,” Computer aided detection of microcalcification in digitalMammograms adopting a wavelet decomposition ”,Integr.Comput.-Aided Eng.,Vol 16,Issue 2,pp.91-103,2009

[5]D.Narain Ponraj, M.Evangelin Jenifer, P. Poongodi, J.Samuel Manoharan “A Survey on thePreprocessing Techniques of Mammogram for the Detection of Breast Cancer”, Journal ofEmerging Trends in Computing and Information Sciences, Volume 2, Issue 12, pp. 656-664,2011

Reference

Page 243: Ms thesis-final-defense-presentation

[6]K. Santle Camilus , V. K. Govindan, P.S. Sathidevi,” Pectoral muscle identification inmammograms”, Journal of Applied Clinical Medical Physics , Vol. 12 , Issue No. 3 , 2011

[7] Leeuw H.D., Stehouwer BL, Bakker CJ, Klomp DW, Diest PV, Luijten PR, Seevinck PR,Bosch MA, Viergever MA, Veldhuis WB:Detecting breast microcalcifications with high-field MRI, NMR in Biomedicine,Vol 27, Issue 5, pages 539–546,2014

[8]N.R.Pal,B.Bhowmik, S.K.Patel, S.Pal, J.Das,”A multi-stage nural network aided system fordetection of microcalcification in digitized mammogeams”,Neurocomputing, Vol 11,pp.2625-2634,2008

[9]S.N.Yu, Y.K. Huang,” Detection of microcalcifications on digital mammograms using combined Model-based and statistical textural features”, Expert Syst.Appl. , Vol 37,Issue 7,pp.5461-5469, 2010

[10]Arnau Oliver, Albert Torrent, Meritxell Tortajada, Xavier Llad´o,Marta Peracaula,Lidia Tortajada, Melcior Sent´ıs, and Jordi Freixenet,” Automatic microcalcification andcluster detection for digital and digitised mammograms”, Springer-Verlag BerlinHeidelberg, 36, pp. 251–258, 2010

Reference

Page 244: Ms thesis-final-defense-presentation

[11] Balakumaran T., Vennila ILA, Shankar C.G: Detection of Microcalcification in Mammograms Using Wavelet Transform and Fuzzy Shell Clustering, International Journal of Computer Science and Information Security, Vol 7,Issue 1,pp.121-125,2010

[12] Arnau Olivera, Albert Torrenta , Xavier Lladóa , Meritxell Tortajada, LidiaTortajadab, Melcior Sentísb, Jordi Freixeneta, Reyer Zwiggelaarc,” Automaticmicrocalcification and cluster detection for digital and digitisedmammograms”, Elsevier:Knowledge-Based Systems, Volume 28, pp. 68–75, April 2012.

[13] Zhang X., Homma N., Goto S.,Kawasumi Y., Ihibashi T.,Abe M.,Sugita N.,Yoshizawa M:A Hybrid Image Filtering Method for Computer-Aided Detection of Microcalcification Clusters in Mammograms, Journal of Medical Engineering, Vol 3,Issue 1,pp.111-119,2013

[14]Wang T. C and Karayiannis N. B.: Detection of Microcalci¯cations in Digital Mammograms Using Wavelets, IEEE Transaction on Medical Imaging, vol. 17, no. 4,(1989) pp. 498-509

[15]. Daubechies I.: Ten Lectures on Wavelets, Philadelphia, PA, SIAM, (1992)

Reference

Page 245: Ms thesis-final-defense-presentation

Reference

[16] Strickland R.N. and Hahn H.: Wavelet transforms for detecting microcalcificationsin mammograms, IEEE Transactions on Medical Imaging, vol. 15, (1996) pp. 218-229

[17] Lu J., Ikehara T., Zhang Y,Mihara T., Itoh T.,Maeda R:High quality factor silicon cantilever driven by piezoelectric thin film actuator for resonant based mass detection, Micro system Technologies , Vol 15, Issue 8, pp:1163-1169., 2009

[18]Fatemeh Moayedi, Zohreh Azimifar, Reza Boostani, and Serajodin Katebi: Contourlet-based mammography mass classification, ICIAR 2007, LNCS 4633,(2007) pp. 923-934

[19] Shankla V, David D. P, Susan P. Weinstein; Michael D., Tuite C, Roth R., Emily F:Automatic insertion of simulated microcalcification clusters in a software breast phantom, , Proc. SPIE 9033, Medical Imaging 2014: Physics of Medical Imaging, 2014

[20] Otsu, N., "A Threshold Selection Method from Gray-Level Histograms," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 9, No. 1, 1979, pp. 62-66.

Page 246: Ms thesis-final-defense-presentation

Reference[21] Da Cunha A. L., Zhou J. and Do M. N,: The Nonsubsampled Contourlet Transform:Theory, Design, and Applications, IEEE Transactions on Image Processing,vol. 15, (2006)pp. 3089-3101

[22]Laine A.F., Schuler S., Fan J., Huda W.: Mammographic feature enhancement bymultiscale analysis, IEEE Transactions on Medical Imaging, 1994, vol. 13, no. 4,(1994)pp. 7250-7260

[23] Sameer S. and Keit B.: An Evaluation on Contrast Enhancement Techniques forMammographic Breast Masses, IEEE Transactions on Information Technology inBiomedicine, vol. 9, (2005) pp. 109-119

[24] Rosten, E., and T. Drummond. "Machine Learning for High-Speed Corner Detection." 9th European Conference on Computer Vision. Vol. 1, 2006, pp. 430–443.

[25] Shi, J., and C. Tomasi. "Good Features to Track." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. June 1994, pp. 593–600.

Page 247: Ms thesis-final-defense-presentation

[26] Harris, C., and M. J. Stephens. "A Combined Corner and Edge Detector." Proceedings ofthe 4th Alvey Vision Conference. August 1988, pp. 147–152.

[27] Bay, H., A. Ess, T. Tuytelaars, and L. Van Gool. "SURF: Speeded Up RobustFeatures." Computer Vision and Image Understanding (CVIU). Vol. 110, No. 3, 2008, pp.346–359.

[28] Leutenegger, S., M. Chli, and R. Siegwart. "BRISK: Binary Robust Invariant Scalable

[29] Matas, J., O. Chum, M. Urba, and T. Pajdla. "Robust wide-baseline stereo frommaximally stable extremal regions."Proceedings of British Machine Vision Conference.2002, pp. 384–396.

[30] Oliver A.; Torrent A. , Tortajada M, Liado X, R., Preacaula M , Tortajada L., Srntis M.,Ferixenet J: A Boosting based approach for automatic Microcalcification Detection,Springer-Verlag Berlin Heldelberg,Lecture notes on Computer Science (LNCS 6136), (2010)pp. 251- 258

Reference

Page 248: Ms thesis-final-defense-presentation

Thank you foryour time and attention