3D Slicer - CANARIE - Advancing Canada's knowledge and ... · PDF file3D Slicer: Open-source...

Csaba Pinter

3D Slicer:Open-source Platform for Medical Image Visualization and Analysis

Laboratory for Percutaneous Surgery, School of Computing, Queen’s University, Kingston, ON, Canada

Thank you, CANARIE!

- 2 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017

+

Medical research and the“valley of death”


Nature 453, 840-842 (2008) | doi:10.1038/453840a

• “A chasm […] between biomedical researchers and the patients who need their discoveries”

• “The clinical and basic scientists don't really communicate”

• In software terms:the difficult path from Matlab scripts to clinical software

Basic research

(bench)

Clinical routine

(bedside)

Right tool for the job

Research tool

“Should it be done?”

Robust and usable

enough for clinical

evaluation, flexible,

open, portable,

community supported

Technological prototype

Clinical tool

“Can it be done?”

Innovative,

not robust,

usually single developer

Patient ready

FDA approved,

company supported,

closed source

www.senkox.comwww.unusuallocomotion.comJalopnik.com



Background for 3D Slicer

• Medical image analysis and visualization platform

• Since 1997

• $50M in funding(>2,000 years of labor)

• Professionally engineered

• 1,000+ analysis functions

• Application framework: customizable, extensible

• Completely free (BSD)• Multi-platform

Fedorov, et al. "3D Slicer as an image computing platform for the Quantitative Imaging Network." Magnetic resonance imaging 30.9 (2012): 1323-1341.

Visualization1. 2D (slice) and 3D views,

chart views

2. Configurable layout

3. Multi-modality imagefusion (foreground, background, label map)

4. Transforms, vector and tensor field visualization

5. Surface and volumerendering

6. Time sequence data

5

4

1 2, 3

6


Registration

• Manual: translation, rotation in 3D

• Automatic: rigid, deformable, with various similarity metrics, initialization methods, optimizers, masking, etc.

• Extensions: structure-based registration, Elastix, etc.


Segmentation• Manual (paint, draw, scissor, threshold, etc.)

• Semi-automatic (region-growing, fill between slices, etc.)

• Automatic (atlas-based, robust statistics, etc.)


Large user community

250 000+ downloads over the past 4 years:

500 downloads per week in 20122000 downloads per week in 2017


Community support

• Extensive documentationwww.slicer.org/wiki/Documentation/Nightly

• Tutorials: both basic and specializedwww.slicer.org/wiki/Documentation/4.6/Training

• User and developer forumhttps://discourse.slicer.org

• Weekly Slicer clinic / hangout

• Bootcamps, hackfests, project weeks


Visualization tutorial

Programming tutorial

https://www.slicer.org/wiki/Documentation/Nightly

https://www.slicer.org/wiki/Documentation/4.6/Training

https://discourse.slicer.org/

Project week

• Twice a year• Bring your own project, work

with experts• Meetings, training• Upcoming:

June 26-30, Catanzaro Lido, Italy



Slicer for translational research

3D Slicer: a cross platform system fortranslating innovative algorithms intoclinical research applications

What does a researcher need ?

• Easily deployable

• Extensible and reconfigurable rich utility libraries

• Stable base

What does a user expect ?

• Easy install and upgrade

• “Standard” clinical behavior

• Advanced functionality

• Consistent interface

Courtesy R. Kikinis

3D Slicer in clinical use

Tracking peritumoral

white matter fibers

Diagnosis of

Different Tumors

in Lung Cancer

MRI-guided

prostate

biopsy

Brain surgery

Breast cancer

surgery guidance

Radiation dose

calculations

Model-Guided Deep

Brain Simulation

Diagnosis of Osteoarthritis

Degeneration

Quantitative assessment

of COPD

Clinical

users drive

creation of

technology

Surgical

navigation


Commercial use


Interdisciplinary applications


• SlicerAstro: Visualization of hydrogen in galaxies


The NA-MIC kit


Automatic regression testing

• Automatic tests ensure detecting regression errors

• Results published on web dashboard every night

– For 3D Slicer main application

– For every extension

– For each supported platform


Slicer is extensible

The Slicer Extension Manager offers the possibility to the user to download and install additional Slicer modules


Image-guided therapy

Nephrostomy tutor: draining the urine from the kidney in a training phantom with tracking and navigation

hardware

OpenIGTLink

Registration Segmentation ...3D Slicer

Calibration, synchronization, pre-processing, simulation, record&replay

Visualization

SlicerIGT

Pivot calibration

PLUS

software

SlicerRT

DICOM-RT import

VTK, ITK, CTK, QT, DCMTK, …

... ...

ProstateNav

Device interface

LumpNav Gel dosimetry

...

...Slicelets

(custom Slicer applets)...

Tool watchdog

Model registration

Dose volume histogram

Dose comparison

Breach warningSlicer

extensions

DICOMOpenIGTLink

Quantification

hundreds of devices (imaging, position tracking, various sensors, and manipulators)

Ultrasoundscanners

Robotic devices, manipulators

Navigationsystems

Surgical microscopes, endoscopes

MRI, CT, PET scanners

Image-guided therapy in 3D Slicer


External beam plan.

Film dosimetry

Building on a platform

BrainsTools ext1.3%

Qt29.2%

VTK27.6%

ITK13.1%

Python9.9%

Numpy6.8%

DCMTK5.2%

3DSlicer core3.5%

CTK1.7%

Plus toolkit1.4%

SlicerIGT ext0.2%

LumpNav ext0.01%

SlicerIGT1.6%

LINES OF SOURCE CODE - ILLUSTRATED THROUGH LUMPNAV(NAVIGATION SOFTWARE FOR BREAST CANCER SURGERY)



PerkLab contributions to 3D Slicer

Slicer core commits in past 3 years: 726/2912 (25%)

Extension downloads (overall, as of 2017.02.)

• #1: SlicerRT: 10400

• #2: SlicerIGT: 5000

• #3: Matlab Bridge: 4200

• #7: Volume Clip: 3300

• #16: Sequences: 2200

• #27: PerkTutor: 1800

• #34: GelDosimetry: 1700

• Average extension: 1KView from SlicerRT


Example: Use custom MATLAB algorithm in 3D Slicer

• Make MATLAB algorithms easier to use

• Integrate into workflow

• Rely on the power of SlicerRT and 3D Slicer

• No programming

DICOM-RT

Non-DICOM


SlicerRT used for creating a sample prostate proton treatment plan on an abdominal phantom

Example: External Beam Planning

Example: Gel Dosimetry

25- 25 -Laboratory for Percutaneous Surgery – Copyright © Queen’s University, 2017

Example: MRI/US fusion for biopsy


Example: LumpNav (touch optimized)

27

http://www.slicerigt.org/wp/breast-cancer-surgery/


http://www.slicerigt.org/wp/breast-cancer-surgery/


Planned Canarie-funded developments

• Surface digitization using high-end and low-cost scanners

• Position tracking using low-cost 3D vision systems

• Augmented and Virtual Reality display

• Sharing of algorithms between Plus toolkit and SlicerIGT

intel.com

Rankin et al. 2017House et al. 2017


Thank you!

[email protected] http://slicer.org

mailto:[email protected]

http://slicer.org/


Appendix


• One of the 3D Slicer core developer groups

– Leading Image-Guided Therapy development group

• Interdisciplinary: computer science, electrical and mech. engineering, clinical sciences (radiology, radiation oncology, and surgery)

• Information: http://perk.cs.queensu.ca

About our lab

http://perk.cs.queensu.ca/

Without an application platform

• Each application is developed from ground up

• Completely new software is developed for each problem/procedure/device

• Significant work is needed to integrate new, advanced algorithms

• Core functionalities are already implemented

• New software modules can be developed for specific needs

• Many new, advanced algorithms are available

• Well-supported with a large user and developer community

Building on an application platform

Quick start.

Huge waste of time, money, and effort overall.

Investment at the beginning: learning.

Minimal wasted efforts.


Data import/export• DICOM: 2D/3D/4D volumes,

structure sets, dose volumes, etc. (extensible withoutSlicer core changes)

• Research data formatsfor volumes, meshes, transforms (NRRD, MetaIO, VTK, HDF, etc.)

• Common non-medical data formats (JPEG, TIFF, etc.)

• Save and complete restore of application state


Many other modules...

• Image filtering(image noise reduction, MRI bias correction, etc.)

• Surface processing

• Diffusion imaging

• Quantification, statistics

• ...


• Each optimal for

• either storage (A)

• or analysis (C)

• or visualization (B,D)

• Imposed needs

• Conversion

• Simultaneous– Visualization

– Transformation

Various representations


Typical representations:A: Contours, B: Surface, C: Image, D: Ribbons

Patient

Patient

• 1 segmentation contains N segments (structures)

– Coherence ✓

• Provides automatic conversions

– Operation ✓

• Each segment contains multiple representations

– Identity ✓

Segmentation “object”


Tumor (contour)

Tumor

Brain (contour)

Brain

• “Promoted” representation

Master representation


Patient

• Conversions use it as source

• When changed, the other representations are cleared

Tumor

image

surface

– And re-converted as needed

• When saving to disk, this representation is written

• Solves Validity ✓

3D Slicer - CANARIE - Advancing Canada's knowledge and ... · PDF file3D Slicer: Open-source...

Documents

Transcript of 3D Slicer - CANARIE - Advancing Canada's knowledge and ... · PDF file3D Slicer: Open-source...