OPAL Workflow: Model Generation

Tricia Pang

February 10, 2009

OPAL Workflow, 10 Feb 2009 3

Motivation

ArtiSynth [1]:3D Biomechanical Modeling Toolkit

Ideally: Model derived from

single subject source High resolution model

OPAL Workflow, 10 Feb 2009 4

Motivation

Obstructed sleep apnea (OSA) disorder Caused by collapse of

soft tissue walls in airway Ideally:

Ability to run patient-specific simulations to help diagnosis

Quick and accurate method of generating modelCredit: Wikipedia

OPAL Workflow, 10 Feb 2009 5

OPAL Project

Dynamic Modeling of theOral, Pharyngeal and Laryngeal (OPAL)Complex for Biomedical Engineering Patient-specific modeling and model

simulation for study of OSA Tools for clinician use in segmenting image

and importing to ArtiSynth Come up with protocol, tools/techniques and

modifications needed for end-to-end process

OPAL Workflow, 10 Feb 2009 6

OPAL Project

3D Medical Data Biomechanical Model

OPAL Workflow, 10 Feb 2009 7

Workflow Stages

1. Imaging

2. Image processing & reconstruction

3. Reference model generation

4. Patient-specific model fitting

5. Biomechanical model

OPAL Workflow, 10 Feb 2009 8

Workflow Stages

1. Imaging

2. Image processing & reconstruction

3. Reference model generation

4. Patient-specific model fitting

5. Biomechanical model

OPAL Workflow, 10 Feb 2009 9

Stage 1: Imaging

Structures Tongue Soft palate Hard palate Epiglottis Pharyngeal wall Airway Jaw Teeth

OPAL Workflow, 10 Feb 2009 10

Data SourceMRI

Credit: Klearway, Inc.

Dental Appliancew/ Markers Cone CT of Dental Cast

Other:laser scans, planar/full CT scans, tagged MRI, ultrasound, fluoroscopy, cadaver data…

OPAL Workflow, 10 Feb 2009 11

MRI & Protocol

Normal subject vs. OSA patients Control vs. treatment (appliance)

OPAL Workflow, 10 Feb 2009 12

Workflow Stages

1. Imaging

2. Image processing & reconstruction

3. Reference model generation

4. Patient-specific model fitting

5. Biomechanical model

OPAL Workflow, 10 Feb 2009 13

Stage 2: Image processing & Reconstruction

N3 correction [2](Non-parametric non-uniform intensity normalization)

Cropping Cubic interpolation

Image registration & reconstruction (Bruno’s work)

Combining 3 data sets → high-quality data set

OPAL Workflow, 10 Feb 2009 14

Workflow Stages

1. Imaging

2. Image processing & reconstruction

3. Reference model generation

4. Patient-specific model fitting

5. Biomechanical model

OPAL Workflow, 10 Feb 2009 15

Stage 3:Reference Model Generation

Goal: High quality model Focus on bottom-up semi-automatic

segmentation approaches eg. Livewire [3]

OPAL Workflow, 10 Feb 2009 16

3D Livewire

Seed points (forming contours) drawn in 2 orthogonal slice directions, and seed points automatically generated in third slice direction

OPAL Workflow, 10 Feb 2009 17

Livewire ModelRefinement

Morphological operations

Contour smoothening(active contours [4])

3D surface reconstruction(non-parallel curve networks [5])

(Claudine & Tanaya)

OPAL Workflow, 10 Feb 2009 18

Workflow Stages

1. Imaging

2. Image processing & reconstruction

3. Reference model generation

4. Patient-specific model fitting

5. Biomechanical model

OPAL Workflow, 10 Feb 2009 19

Stage 4:Patient-Specific Model Generation

Goal: Accurate model, generated with minimal user interaction

Focus on top-down or automated approaches Morphological warping operations Deformable model crawlers

OPAL Workflow, 10 Feb 2009 20

Thin-Plate Spline Warping

Thin-plate spline (TPS) deformation [6]: interpolating surfaces over a set of landmarks based on linear and affine-free local deformation Reference

Model

Warp Result

Warp field

OPAL Workflow, 10 Feb 2009 21

TPS Warping, Phase 1

Patient MRI

Reference Model

List of corresponding points

User selects a point on both patient MRI and reference model

Hard to pinpoint landmarks on 3D model

OPAL Workflow, 10 Feb 2009 22

TPS Warping, Phase 2Reference MRI (has a pre-built 3D model)

Patient MRI

Predefined landmarks shown on reference MRI, user selects equivalent point on patient MRI

Can be improved by automated point-matching

OPAL Workflow, 10 Feb 2009 23

Chan-Vese Active Contours

Highly automated method

Combine 2D segmentation of axial slices in Matlab User-indicated start point Iterate sequentially using

previous segmentation as starting contour for Chan-Vese active contours [7]

Livewire 3D(~2 hours)

Livewire +post processing

Automated AC on axial(2 minutes)

OPAL Workflow, 10 Feb 2009 24

Deformable Organism Crawler

Automatically segment airway by growing a tubular organism, guided by image data and a priori anatomical knowledge

Developed in I-DO toolkit [8] Advantages:

Analysis and labeling capabilities Ability to incorporate shape-based

prior knowledge Modular hierarchical development framework

OPAL Workflow, 10 Feb 2009 25

Workflow Stages

1. Imaging

2. Image processing & reconstruction

3. Reference model generation

4. Patient-specific model fitting

5. Biomechanical model

OPAL Workflow, 10 Feb 2009 26

Stage 5:Biomechanical Model

Import surface mesh into ArtiSynth Work in progress Challenges:

Determining “rest” position from inverse modeling Defining interior nodes and muscle end points

OPAL Workflow, 10 Feb 2009 27

Challenges inSegmentation

Medical image data quality Bottom-up methods: Need for general

procedure and abstraction from anatomy being segmented

Top-down methods: Need good atlas model Validation with gold standard segmentation

OPAL Workflow, 10 Feb 2009 28

Future Directions in Segmentation

Deformable organism crawler Automated morphing of reference model into

patient model Additions to Livewire

Oblique slices Sub-pixel resolution Convert to graphics implementation Add smoothness by regularization

(eg. by spline, a priori model, …)

OPAL Workflow, 10 Feb 2009 29

Thank you!

Questions?

OPAL Workflow, 10 Feb 2009 30

References[1] Fels, S., Vogt, F., van den Doel, K., Lloyd, J., Stavness, I., and Vatikiotis-Bateson, E. Developing

Physically-Based, Dynamic Vocal Tract Models using ArtiSynth. Proc. Int. Seminar Speech Production (2006), 419-426.

[2] Sled, G., Zijdenbos, A. P., and Evans, A. C. Non-parametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans. in Medical Imaging17, 1 (1998), 87-97.

[3] Poon, M., Hamarneh, G., and Abugharbieh, R. Effcient interactive 3d livewire segmentation of complex objects with arbitrary topology. Comput. Med Imaging and Graphics (2009), in press.

[4] Hamarneh, G., Chodorowski, A., and Gustavsson, T. Active Contour Models: Application to Oral Lesion Detection in Color Images. IEEE International Conference on Systems, Man, and Cybernetics 4 (2000), 2458 -2463.

[5] Liu, L., Bajaj, C., Deasy, J. O., Low, D. A., and Ju, T. Surface reconstruction from non-parallel curve networks. Eurographics 27, 2 (2008), 155-163.

[6] Bookstein, F. L. Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 11, 6 (1989), 567-585.

[7] Chan, T., and Vese, L. Active contours without edges. IEEE Transactions on Image Processing 10, 2 (2001), 266-277.

[8] McIntosh, C. and Hamarneh, G. I-DO: A “Deformable Organisms” framework for ITK. Medical Image Analysis Lab, SFU. Release 0.50.