Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin
description
Transcript of Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin
![Page 1: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/1.jpg)
![Page 2: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/2.jpg)
2
Outline
• Introduction– Motivation and Goals– Grayscale Chromosome Images– Multi-spectral Chromosome Images
• Contributions• Results • Conclusions
![Page 3: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/3.jpg)
3
Motivation and Goals
• Chromosomes store genetic information• Chromosome images can indicate genetic disease,
cancer, radiation damage, etc.• 325 clinical cytogenetic US labs perform over
250,000 diagnostic studies per year involving chromosome analysis
• Research goals:– Locate and classify each chromosome in an image– Locate chromosome abnormalities
![Page 4: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/4.jpg)
4
Karyotyping
• 46 human chromosomes form 24 types– 22 different pairs– 2 sex chromosomes, X and Y
• Grouped and ordered by length except X and Y
Banding Patterns Karyotype
![Page 5: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/5.jpg)
5
Chromosome Abnormalities
• Abnormal number– Turner’s Syndrome (1 X, no Y chromosome)– Down’s Syndrome (3 of type 21)
• Translocations: Chronic myelogenous leukemia (type 9 and type 22)
• Deletions of genetic material: William’s Syndrome (gene missing in type 7)
• Research goals:– Locate and classify each chromosome in an image– Locate chromosome abnormalities
![Page 6: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/6.jpg)
6
Denver Classifications
• Florescence microscopy• Single dye, even stain• Features
– Length (2-10m)– Relative centromere
position• Disadvantage: Only 7
distinguishable types [1960]
centromere
![Page 7: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/7.jpg)
7
Banding Patterns
• Single dye, bandingpattern staining [1969]
• Features– Length– Relative centromere
position– Banding pattern
• All 24 types distinguishable• Greatly improved manual chromosome analysis• Disadvantage: Computer analysis difficult
![Page 8: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/8.jpg)
8
Multi-spectral Chromosome Imaging
• Multiplex Fluorescence In-Situ Hybridization (M-FISH) [1996]
• Five color dyes (fluorophores)• Each human chromosome type
absorbs a unique combination of the dyes
• 32 (25) possible combinations of dyes distinguish 24 human chromosome types
Healthy Male
![Page 9: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/9.jpg)
9
M-FISH Images
• Images of each dye obtained with appropriate optical filter
• Each pixel a five dimensional vector• Each vector element gives contribution of a dye
at pixel• Chromosomal origin distinguishable at single
pixel (unless overlapping)• Unnecessary to estimate length, relative
centromere position, or banding pattern
![Page 10: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/10.jpg)
10
M-FISH Images
• 6th dye (DAPI) binds to all chromosomes
DAPI Channel6th Dye
M-FISH Image5 Dyes
![Page 11: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/11.jpg)
11
Traditional Chromosome Image Analysis
• Sequential: Segmentation then classification• Segmentation
– Background/foreground– Connected components– Cluster: groups of touching chromosomes– Cluster recognition/decomposition
• Touching chromosomes• Overlapping chromosomes
• Classification
![Page 12: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/12.jpg)
12
Boundary
M-FISH Segmentation
• Multi-spectral data has more segmentation information than boundary data alone
• Previous M-FISH methods1. Segment with DAPI
(grayscale) channel2. Classify multi-spectral
information
Raw Image
![Page 13: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/13.jpg)
13
Translocations
• Exchange of material between two types• More visible in M-FISH images• Appear as objects with two different colors
Grayscale
M-FISH
![Page 14: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/14.jpg)
14
Outline
• Introduction• Contributions
1. Chromosome segmentation using multi-spectral information
2. Joint segmentation and classification3. Aberration scoring
• Results • Conclusions
![Page 15: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/15.jpg)
15
Problem Formulation
• Ci: the set of all pixels belonging to class i
• Ain: the set of pixels belonging to the nth chromosome of class i
• Ain Ci• Segmentation-classification
– Estimating each set Ain
– Segmentation: Proposing a set of pixels, A´– Classification: Proposing a class, i
![Page 16: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/16.jpg)
16
Proposed Approach
• Develop a measure of quality for segmentation-classification possibilities– Must be a function of both segmentation and
classification– Measure is also a likelihood
• Choose a reasonable set of segmentation-classification possibilities
• Maximize the measure over the set of possibilities
![Page 17: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/17.jpg)
17
Maximum Likelihood Formulation
• Proposed likelihood function for single candidate chromosome is a combination of several functions
• Lmulti(·,·): multi-spectral likelihood function• Lsize(·,·): size likelihood function• w(·): weighting function• i: class (classification)• A´: candidate chromosome (segmentation)
AwiALiALiAL sizemulti ,,,
Contribution #2
![Page 18: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/18.jpg)
18
Multi-spectral Information
• Average of individual pixel probabilities• Bayesian pixel classifier returns probabilities for
each class [Sampat, Castleman, and Bovik, 2002]
• Ci: set of all pixels of class i• m: pixel• x(m): multi-spectral image data at pixel m
Am
imulti CpA
iAL mxm1,
Contribution #1
![Page 19: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/19.jpg)
19
Size Information
• Likelihood function is a Gaussian that peaks at mean size of class i
• The size means and variances of the class i are i and i
• Ain: the set of pixels belonging to the nth chromosome of class
2
2
2, i
i
n j
njA
A
size eiAL
![Page 20: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/20.jpg)
20
Weighting Function
• Measures the certainty of the likelihoods• w(·) is the percentage of visible, or non-
overlapped, pixels in the candidate chromosome • Forces more certain non-overlapped chromosomes
segments to be combined first• Precludes possibility of a segment being left out of
the middle of a chromosome
![Page 21: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/21.jpg)
21
Estimating Area of Overlap
Overlapping Chromosomes
15 X X
Chromosome Ends
Border Pixels(in black)
X X
Overlapped area estimated (in black)
![Page 22: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/22.jpg)
22
Weighting Function Example
• Yellow areas represent two possible segmentations for a single chromosome
• The function w(·) gives more weight to b
a) Incorrect Segmentation
b) Correct Segmentation
![Page 23: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/23.jpg)
23
Segmentation Implementation
• Use multi-spectral information to determine segmentation possibilities
• Strategy: Oversegment and merge segments– Use pixel classification and post-processing to
determine initial segments– Merge segments as long as the merging increases the
proposed likelihood function
![Page 24: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/24.jpg)
24
Example
Cluster to be decomposed
![Page 25: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/25.jpg)
25
Determining Initial Segments
Pixel Classification
Majority Filtering
Small Segment Reclassification
![Page 26: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/26.jpg)
26
Merge Segments
0.37
0.05
0.41
Merged Segments
0.03 0.05
0.00
0.57
0.10
Initial Likelihood Values
12
15
22
Classification
![Page 27: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/27.jpg)
27
Aberration Scoring
• Aberration scoring: assigning a value to the likelihood of abnormality
• Design of likelihood function has allowed for straightforward aberration scoring– Segments with low likelihood can be flagged as likely
abnormalities– Low likelihood values also identify incorrect
segmentation and classification– Likelihood values help direct user
Contribution #3
![Page 28: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/28.jpg)
28
Outline
• Introduction• Contributions• Results
– Segmentation– Classification– Aberration Scoring
• Conclusions
![Page 29: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/29.jpg)
29
Comparison
• Data: Advanced Digital Imaging ResearchM-FISH Chromosome Image Dataset– 200 M-FISH images– Wide variety of images– Many difficult images– Includes hand segmented ground-truth images– Freely available on web: http://www.adires.com
• Competition: Compare against user-guided Cytovision grayscale segmentation software [2001]
![Page 30: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/30.jpg)
30
Segmentation Results
Proposed Method
Cytovision(Grayscale)
Touches 77% 58%
Overlaps 34% 44%
Singles Oversegmented
0.8% 0.2%
![Page 31: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/31.jpg)
31
More ResultsRecognized as Clusters
Proposed Method
Cytovision
Clusters 95% 69%Singles 6% 0.4%
Proposed Method
Only Pixel Classification
Misclassified 8.1% 15%
![Page 32: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/32.jpg)
32
Aberration Scoring with Proposed Method
Normal Chromosomes
AbnormalitiesTranslocations Fragments
Likelihood 0.44 0.12 0.02Likelihood 0.24 0.10 0.02< 0.1 likelihood 4.9% 49% 100%< 0.3 likelihood 34% 96% 100%V29 subset of ADIR dataset
15 images with 5 translocations each(Cytovision does not perform aberration scoring)
![Page 33: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/33.jpg)
33
Error Detection withProposed Method
< 0.1 likelihoodAbnormal 49.1%Incorrect Segmentation
52.6%
Incorrect Classification
48.6%
Correct Segments 6.4%
![Page 34: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/34.jpg)
34
Contributions
• Derived single, unified maximum likelihood hypothesis test framework
• Decomposed chromosome clusters using M-FISH multi-spectral data
• Combined segmentation and classification for increased accuracy in both
• Demonstrated effective aberration scoring• Implemented joint segmentation-classification
algorithm in C (2-3 minutes/image on 167MHz Unix machine)
![Page 35: Ph.D. Defense Department of Electrical and Computer Engineering The University of Texas at Austin](https://reader036.fdocuments.net/reader036/viewer/2022062810/56815bb1550346895dc9b226/html5/thumbnails/35.jpg)
35
Future Work
• Improvements in likelihood function– Shape– Number
• Pixel classification• Overcome “greedy” algorithm difficulties• Combine geometric, grayscale, and multi-spectral
information for complete algorithm