Post on 17-Dec-2015
ProbExplorer: Uncertainty-guided Exploration and Editing of Probabilistic Medical Image Segmentation
Ahmed Saad1,2, Torsten Möller1, and Ghassan Hamarneh2
1Graphics, Usability, and Visualization (GrUVi) Lab,2 Medical Image Analysis Lab (MIAL),
School of Computing Science, Simon Fraser University, Canada
2Ahmed Saad ProbExplorer
Outline
• Medical image segmentation challenges• ProbExplorer framework• Case studies
– Highlight suspicious regions (e.g. tumors)– Correct misclassification results
• Uncertainty visualization using shape and appearance prior information
• Conclusion and future work
3Ahmed Saad ProbExplorer
Medical image segmentation
• Partitioning the image into disjoint regions of homogeneous properties
• Useful for statistical analysis, diagnosis, and treatment evaluation
Medical Image Segmentation
4Ahmed Saad ProbExplorer
Segmentation challenges
• Low signal-to-noise ratio• Partial volume effect• Anatomical shape variability• Multi-dimensional data
Magnetic Resonance Imaging Positron Emission Tomography
5Ahmed Saad ProbExplorer
Segmentation challenges
• Low signal-to-noise ratio• Partial volume effect• Anatomical shape variability• Multi-dimensional data
6Ahmed Saad ProbExplorer
Segmentation challenges
• Low signal-to-noise ratio• Partial volume effect• Anatomical shape variability• Multi-dimensional data
Patient 1 Patient 2 Patient 3 Patient 4
Ahmed Saad ProbExplorer
Segmentation challenges
• Low signal-to-noise ratio• Partial volume effect• Anatomical shape variability• Multi-dimensional data
4D CT dPET DTMRI
7
Ahmed Saad ProbExplorer
Segmentation output
Crisp Probabilistic (Fuzzy)
70%
20%10%
Putamen
White matterGrey matter
Putamen
8
Max
9Ahmed Saad ProbExplorer
Outline
• Medical image segmentation challenges• ProbExplorer framework• Case studies
– Highlight suspicious regions (e.g. tumors)– Correct misclassification results
• Uncertainty visualization using shape and appearance prior information
• Conclusion and future work
10Ahmed Saad ProbExplorer
Goal
• Given probabilistic segmentation results, we will allow expert users to visually examine regions of segmentation uncertainty to– Highlight suspicious regions (e.g. tumors)– Correct misclassification results without re-
running the segmentation
11Ahmed Saad ProbExplorer
ProbExplorer
Preprocessing Selecting voxels EditingProbabilistic
segmentation
Change selectionCommit an editing action
12Ahmed Saad ProbExplorer
ProbExplorer
Preprocessing Selecting voxels EditingProbabilistic
segmentation
Change selectionCommit an editing action
13Ahmed Saad ProbExplorer
ProbExplorer
Preprocessing Selecting voxels EditingProbabilistic
segmentation
Change selectionCommit an editing action
14Ahmed Saad ProbExplorer
ProbExplorer
Preprocessing Selecting voxels EditingProbabilistic
segmentation
Change selectionCommit an editing action
Before After
15Ahmed Saad ProbExplorer
Preprocessing
• A probabilistic vector field
)](),....,()([)( 21 xPxPxPxP C
Sort maxP )(xPFBG
)(xM
16Ahmed Saad ProbExplorer
Outline
• Medical image segmentation challenges• ProbExplorer framework• Case studies
– Highlight suspicious regions (e.g. tumors)– Correct misclassification results
• Uncertainty visualization using shape and appearance prior information
• Conclusion and future work
17Ahmed Saad ProbExplorer
Renal dynamic SPECT
• 4D image of size 64 x 64 x 32 with 48 time steps with an isotropic voxel size of (2 mm)3
Raw data Crisp segmentation
21Ahmed Saad ProbExplorer
Outline
• Medical image segmentation challenges• ProbExplorer framework• Case studies
– Highlight suspicious regions (e.g. tumors)– Correct misclassification results
• Uncertainty visualization using shape and appearance prior information
• Conclusion and future work
22Ahmed Saad ProbExplorer
Uncertainty-based segmentation editing
Ground truth Overestimation Underestimation
23Ahmed Saad ProbExplorer
Synthetic example
No noise no PVE
Ground truth
Observed = noise + PVE
Current segmentation
Ahmed Saad ProbExplorer
Synthetic example: push action
Push action
Source set Destination set
24
25Ahmed Saad ProbExplorer
Synthetic example: push action
is the first best guess
0.40.3
0.2Swap0.3 0.4
27Ahmed Saad ProbExplorer
Overestimated putamen
Ground truth Overestimated Putamen
Ahmed Saad ProbExplorer
Dynamic PET brain
Push actionPutamen
Background
Skull
Grey matter
Cerebellum
Source set Destination set
30
33Ahmed Saad ProbExplorer
Outline
• Medical image segmentation challenges• ProbExplorer framework• Case studies
– Highlight suspicious regions (e.g. tumors)– Correct misclassification results
• Uncertainty visualization using shape and appearance prior information
• Conclusion and future work
Ahmed Saad ProbExplorer
• Maximum-a-posteriori principle
Bayesian perspective
Likelihood PriorPosterior
34
Ahmed Saad ProbExplorer
Framework
Atlas construction Shape prior
Like
lihoo
d
Appearance prior
Like
lihoo
d
Images
Expert binarysegmentations
Probabilistic shape prior
Probabilistic appearance prior
Population representative image
New image New probabilistic segmentation
Image-to-Image registration
Alignedlikelihood
35
Ahmed Saad ProbExplorer
• is a spatial location in • is a feature vector associated with that can
be constructed from intensity, gradient, etc.• can be decomposed into:
– is the shape prior– is the appearance prior
Mathematical notations
36
Ahmed Saad ProbExplorer
Algorithm demonstration using synthetic example
Piecewise constant Blurring Noise
100 noise realizations and random translations
37
Ahmed Saad ProbExplorer
• We adopt the method used by Hamarneh and Li [JIVC 09]• Alignment of binary shapes• Shape histogram
Atlas construction:Shape prior modeling
38
Ahmed Saad ProbExplorer
• Alignment of binary shapes• Shape histogram • Distance transform DIST(X)
Atlas construction:Shape prior modeling
39
Ahmed Saad ProbExplorer
• Alignment of binary shapes• Shape histogram • Distance transform DIST(X)
• Probabilistic shape prior
Atlas construction:Shape prior modeling
40
Ahmed Saad ProbExplorer
• Multivariate Gaussian fitting
Atlas construction:Appearance prior modeling
41
Ahmed Saad ProbExplorer
• Mixture of Gaussians
• Other probabilistic segmentation techniques can be used, e.g. Random walker, Probabilistic SVM, etc.
Likelihood
42
Ahmed Saad ProbExplorer
Abnormal appearanceData
Selection
Maximum likelihood
46
Ahmed Saad ProbExplorer
Abnormal shape and appearanceData
Selection
Maximum likelihood
47
Ahmed Saad ProbExplorer
Misclassification correction
Dice: 0.32 Dice: 0.75
48
50Ahmed Saad ProbExplorer
User evaluation
• Our clinical collaborators showed how ProbExplorer can be used to achieve highly accurate segmentation from a very noisy dSPECT renal study (Humphries et al. IEEE Nuclear Science Symposium/Medical Image Conference 2009)
51Ahmed Saad ProbExplorer
Conclusion
• ProbExplorer: a framework for the analysis and visualization of probabilistic segmentation results
• We provided a number of visual data analysis widgets to reveal the different class interactions that are usually hidden by a simple crisp visualization
52Ahmed Saad ProbExplorer
Future work
• Spatial dependency between voxels during interactive editing
• Investigate the behavior of the resulting probabilistic results from different segmentation techniques
• Multi-structure atlas• Registration uncertainty visualization
53Ahmed Saad ProbExplorer
Acknowledgements
• Natural Sciences and Engineering Research Council of Canada (NSERC)
• Prof. Vesna Sossi, Prof. Anna Celler, Thomas Humphries, and Prof. Manfred Trummer