Lec12: Shape Models and Medical Image Segmentation
-
Upload
ulas-bagci -
Category
Science
-
view
10 -
download
9
Transcript of Lec12: Shape Models and Medical Image Segmentation
MEDICAL IMAGE COMPUTING (CAP 5937)
LECTURE 12: Shape Models and Medical Image Segmentation
Dr. Ulas BagciHEC 221, Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), Orlando, FL [email protected] or [email protected]
1SPRING 2017
Outline• Shape Information
– Representation– Simple processing (dilation + erosion)
• Shape Modeling– M-reps– Active Shape Models (ASM)– Oriented Active Shape Models (OASM)– Application in anatomy recognition and segmentation– Comparison of ASM and OASM
2
3MotivationMost structures of clinical interest have a characteristic shape and anatomical location relative to other structures
Brain image shows: • Ventricles • Caudate nucleus • Lentiform nucleus
What is Shape?• Shape is any connected set of points!
7
Shape (S) Points
T: boundaryP: interiorQ: exterior
Example Applications for Shape Analysis
8
NeuroScience
• Morphological taxonomy of neural cells
• Interplay between form and function
• comparisons between cells of different cortical areas
• Comparisons between cells of different species
• …
Internet/Document Analysis
• Content-based information retrieval
• Watermarking• Graphic design• WWW• Optical character
recognition• Multimedia databases• …
Visual Arts
• Video restoration• Video tracking• Special effects• Games• Computer graphics• Image synthesis• Visualizations• …
Imaging/Vision
• Pathology detection/classification (spiculated and spherical nodules)
• 3D pose estimation• Morphological
operations• Anatomy modeling• Segmentation• Registration• Volumetry
Other areas: medicine, biology, engineering, physics, agriculture, security…
Shape Models• Point distribution, Active Shape, Active Appearance
models (Cootes & Taylor)• Fourier Snakes (Szekely)• Active Contours (Blake)• Parametrically-deformable models (Staib & Duncan)• Medial Representation (m-rep)• …
9
Shape Models• Point distribution, Active Shape, Active Appearance
models (Cootes & Taylor)• Fourier Snakes (Szekely)• Active Contours (Blake)• Parametrically-deformable models (Staib & Duncan)• Medial Representation (m-rep)• …
10
Active Shape Models can be used to help interpret new images by finding the parameters which best match an
instance of the model to the image.
The shape of an organ can be characterized as being a transformed version of some template
11
template
Credit: Parmedco.ir
Deformations/transformation
What shape representation is for• Analysis from images– Extract the kidney-shaped object
– Register based on the pelvic bone shapesCredit: Pizer
What shape representation is for
13
Snapshots of subcortical shapeevolution after geodesicregression on a multi-object
complex: putamen, amygdala, and hippocampus.
Credit: Gerig
What shape representation is for
14
A graphical visualization is presented forthe left/right asymmetry of the amygdala–hippocampal complex for healthy controls (top row) and patients with schizophrenia (bottom row).
What shape representation is for
• Shape science – Shape and biology– Shape-based diagnosis
Brain structures (Gerig)
Representing Boundary Displacements
• Along figurally implied boundary normals• Coarse-to-fine• Captures along-boundary covariance
• Useful for rendering
Medial Primitives• x, (b,n) frame, r, θ (object angle)• Imply boundary segments
with tolerance
• Similarity transform equi-variant– Zoom invariance implies width-proportionality of
• tolerance of implied boundary• boundary curvature distribution• spacing along net• interrogation aperture for image
θ
b
rR(- θ)b x rR(θ)bn
Multiscale Medial Model• From larger scale medial net,
interpolate smaller scale medial net and represent medial displacements b.
Active Shape Model (T. Cootes et al., 1995)
• Representation of Shapes– Point Distribution Model (PDM)– Represent a shape instance by a judiciously chosen set of points
(features), each of which is a k-dim vector. N feature points are stacked into a long vector of length kn:
where
23
q = [p1, . . . , pn]T
pi = (xi, yi), for i = 1, ..., n
landmarks
Active Shape Model – How to?(1) Build a statistical boundary shape model that consists of a mean
shape and its allowable variations.
(2) Use the model to recognize the boundary in the given scene.
(3) Use the model to fit to the information in the scene. The resulting model is considered to be the boundary delineation.
25
(1) Mean Shape• Model is constructed via a set of landmarks or homologous points.
• Align the shapes after landmarking corresponding points– The Procrustes Algorithm is used to that the sum of instances to the
mean of each shape is minimized! Why?
• The mean shape and allowable variations are determined from a set of N training shapes
26
X
i=1,...,M
(qi � q̄)2
(1) Mean Shape – Model Variation• We now approximate any instance of the shape, including
the training instances, by projecting onto the first t eigenvectors:
• The weight vector b is identified as the characteristic of this instance of the shape:
• Varying the weights bi enables us to explore the allowable variations in the shape!
32
q = q̄ +tX
i=1
biui
b = [b1, ..., bt]T
(1) Mean Shape – Example Modes
34
1st mode
2nd mode
3rd mode
mean -2 std mean mean+2 std
Shape instances generated (talus bone of foot):
(1) Mean Shape – Example Modes
36
Shape instances generated (talus bone of foot):
3 31 1λ λ− ≤ ≤b1 3 32 2λ λ− ≤ ≤b2
(2) Placing the model – ASM Search• (0) Specify mean shape location.
• (1) Along line segments orthogonal to current shape, determine intensity profile.
• (2) Reposition landmark along line to match boundary information.
• (3) Subject landmarks to model constraints. If convergence, stop. Else go to (1).
37
38
(2) Placing the model – ASM Search),( ii YX
Need to search for local matchFor each point:
-strongest edge-correlation-statistical model of profile
Deeper in ASM Search 39
Model boundary
Model point
Interpolate at these points
,...2,1,0,1,2...
),(),(
−−=
+
insnsiYX ynxn
),( YX
xnns
ynns
ns
),( along length of steps Take yxn nns
)(xg
xdxxdg )(
))1()1((5.0)( −−+= xgxgdxxdg
Select point along profile at strongest edge
Finding the model pose & parameters• Suppose we have identified a set of points Y in the image.
Evidently, we can seek to minimize the squared distance:
Algorithm1. Initialize b=02. Generate initial model instance 3. Find T that best aligns q to Y (e.g., similarity transform)4. Invert pose parameters, to project into model frame5. Update the model parameters: 6. Repeat from step 2 until convergence
40
|Y � T (q̄ +tX
i=1
biui)|2
q = (q̄ +tX
i=1
biui)
y = T�1(Y)
b = UT (y � q̄) U = [u1|u1|...ut]
ASM Summary
43
+ Shape prior helps overcoming segmentation errors
+ Fast optimization
+ Can handle interior/exterior dynamics
Pros:
Cons:
Possible improvements:
Learn and apply specific motion priors for different actions
- Optimization gets trapped in local minima
- Re-initialization is problematic
ASM Problems
45
10 points
20 points
Talus 1 Liver 1
2D landmarking
How about 3D?
Time consuming!CorrespondenceIssue!
Alternative solutions for landmarking• Based on image registration• Based on MDL (min description length) and optimization
techniques• Based on shape characteristics of the objects
46
Oriented Active Shape Models (OASM)
48
(1) Selecting landmarks. (2) Building models. (3) Creating boundary cost function. (4) Coarse level recognition. ß(5) Fine level recognition & delineation – 2LDP. ß
training
Overview of Approach
OASM (Liu and Udupa) 49
P3
P1
P2
P4P5
P6
Rec
ogni
tion
Delineation
P1 P2 P6 P1…
- Construct a graph to setup synergy between recgn.and deln.
- Automate recognition.
(5) Fine level recognition & delineation – 2LDP
50
Delineation
P1 P2 P6 P1
Rec
ogni
tion
P1
P2
P3
P4P5
P6
- Construct a graph to setup synergy between recgn.and deln.
- Automate recognition.
OASM (Liu and Udupa)
51
Rec
ogni
tion
Delineation
P1 P2 P6 P1
P1
P2
P3
P4P5
P6
- Construct a graph to setup synergy between recgn.and deln.
- Automate recognition.
OASM (Liu and Udupa)
52OASM (Liu and Udupa)
.
.
.R
ecog
nitio
n
Delineation
P1 P2 P6 P1
P1
P2
P3
P4P5
P6
- Construct a graph to setup synergy between recgn.and deln.
- Automate recognition.
53
Rec
ogni
tion
Delineation
P1 P2 P6 P1
P1
P2
P3
P4P5
P6
- Construct a graph to setup synergy between recgn.and deln.
- Automate recognition.
OASM (Liu and Udupa)
OASM – Recognition (model placement)Coarse level recognition
• At each location in a coarse grid, evaluate the total boundary cost (from Live Wire) via 2LDP by placing the mean shape at that point.
• Select that location for which this total cost is the minimum.
54
Active Appearance Model• PLEASE READ THE FOLLOWING PAPER• T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models",
in Proc. European Conference on Computer Vision 1998 Vol. 2, pp. 484-498, Springer, 1998. (Best paper prize)
63
Summary• Shape
– Shape representation (landmark based)• ASM
– Model placement, ASM search (optimization)• OASM
– Orientedness – Cost function– Less number of landmarks– Improved results
64
Slide Credits and References• Credits to: Jayaram K. Udupa of Univ. of Penn., MIPG• Bagci’s CV Course 2015 Fall.• TF. Cootes et al. ASM and their training and applications,
1995.• S.Pizer (UNC) presentations• K.D. Toennies, Guide to Medical Image Analysis,• Shape Analysis in Medical Image Analysis, Springer.• Handbook of Medical Imaging, Vol. 2. SPIE Press.• Handbook of Biomedical Imaging, Paragios, Duncan, Ayache.
65