NA-MICNational Alliance for Medical Image Computing http://na-mic.org

3D Slicer and the NA-MIC Kit for IGT ResearchSteve Pieper, PhD

NAC, SPL, NA-MIC, BIRN, NCIGT, Isomics, Inc.

Founder and CEO of Isomics, Inc. a technology development company that works closely with multiple research institutions on projects including NIH grants and software development. Much of this work is done in collaboration with the Surgical Planning Laboratory at BWH.

National Alliance for Medical Image Computing http://na-mic.org

2

Acknowledgments• F. Jolesz, R. Kikinis, C. Tempany, P. Black, S. Wells, CF. Westin, M.

Halle, N. Hata, T. Kapur, A.Tannenbaum, M. Shenton, E. Grimson, P.Golland, W.Schroeder, J. Miller, W. Lorensen, A. Yarmarkovich, N. Aucoin, K. Hayes, S. Barre, W. Plesniak, D. Marcus, and many more….

V E R I TAS

National Alliance for Medical Image Computing http://na-mic.org

3

Goals

• NA-MIC Kit: Software and Methodologies for Medical Image Computing– Facilitate Research– Promote Interoperability

• Stable, Cross-Platform Run Time Environment– Full set of core features

• Flexible Module Architecture– Plug-ins added as needed

National Alliance for Medical Image Computing http://na-mic.org

4

Fully Open, Interoperable Components

• End User Application– 3D Slicer

• Image Analysis, Visualization, and GUI libraries– ITK, VTK, KWWidgets

• Large Scale Data Processing Tools – Batchmake, XNAT, BIRN GRID tools

• Software Engineering Tools– CMake, Dart, CTest, CPack

http://www.na-mic.org/Wiki/index.php/SoftwareInventory

Provided by Pieper, Kikinis

National Alliance for Medical Image Computing http://na-mic.org

5

Where Does Slicer Fit?• Shared Platform for Research and

Applications• Ensure Data Compatibility• Enable Algorithms at Run Time As

Needed• Common Developer Vocabulary• Consistent User-Centered Interface

• Leverage• NIH Investment• Best of Breed Open Source Tools• Collaborator Expertise

• Unique Environment• National Scope• Clinical and Research• Commercial and Academic

Image provided by A. Golby, F. Talos, P. Black

• Slicer is a research platform:– NOT an FDA approved medical device – NOT finished – some parts will work better than

others

National Alliance for Medical Image Computing http://na-mic.org

6

Image/Scene Management

• XML-Based MRML File Stores Scene Description

– Volumes (Images, Label Maps)

– Models

– Hierarchical Affine Transforms

– Scene Data (Cameras, Colors, Fiducials, etc)

– Undo/Redo, Scene Snapshots

• Careful Attention to Coordinate Systems

– Scalar Images

– Diffusion Images

– Time Series (fMRI)

– Visualization

Provided by S. Pieper

National Alliance for Medical Image Computing http://na-mic.org

7

More Slicer3 Featuresin Development

• Fully Scriptable in Tcl or Python– Links to scipy, numpy, pylab, matplotlib

• SlicerDaemon– All Data and API available by connecting to socket

• Execution Model– Very easy to develop plug-in modules that run either as shared

libraries or command line executables• Experiment Control

– Ties to databases and distributed computing to run analyses on large populations

– Informatics Interfaces XCEDE, XNAT, BatchMake• Much more on the drawing board…

– 3D Widgets, DICOM WG23, caBIG…

National Alliance for Medical Image Computing http://na-mic.org

Modularity Project

• Keep the base package “lean and mean”

• Modules have individual identity

– Per-module web site, svn, downloads, mailing lists, wiki…

• Allow users to assemble their own set of tools

– Customized ‘Bundles’ by task or application

• Easy to download compatible extensions

– Analogous to Firefox extensions

– Integrate extension builds into developer/nightly/release processs

• NITRC Supplement to NA-MIC helping to pay for needed infrastructure (Neuroimaging Informatics Tools and Resources Clearinghouse)

– NITRC can host neuroimaging projects (gforge implementation)

8

National Alliance for Medical Image Computing http://na-mic.org

9

Base Features vs. ModulesFeatures Base ModulesVisualization MRML: Models, Volumes, Volume Rendering, Fiducials,

Continuum Meshes, Labeled DataCan create custom MRML Nodes and behavior

Filtering None Command Line Modules

Registration Transform Display and Edit, Save/Restore Calculate Transforms, Resample Data

Segmentation Label Maps, Parcellated Surfaces Segmentation Algorithms

Quantification Label, Image, Volume Statistics; Numpy access to MRML

Applications in Python or MATLAB

Real-time Integration VTK Rendering, KWWidgets framework, Tracker Support (as Transforms)

Direct Manipulation of the MRML Scene; 2D/3D Widgets; Device Interfaces

Diffusion Imaging DWI, DTI, Fiber Bundles Tractography, Clustering, Atlases

Applications “Bundles” of Modules in Distribution: Registration, Editor, some Filters,

Customized Extensions, Domain-specific code, Optimized Interfaces…

National Alliance for Medical Image Computing http://na-mic.org

10

It’s all great, but…

A few caveats:

• It’s a big, distributed group– Pro: “Wisdom of the Crowds”– Con: “Designed by Committee”

• It’s all research– Constantly evolving– Never enough documentation– Plurality of approaches

• How well we work together will determine the impact of our work