Insight Toolkit 을 이용한 삼차원 흉부 CT 영상분석 및 폐 결절 검출 시스템

광주과학기술원 기전공학부 최욱진

2

Insight Toolkit (ITK)

• www.itk.org • 2000 년 부터 개발• Image Processing Toolkit – C++ 라이브러리 (+2 million LOC) – Java, Python, TCL 등의 언어 지원– Linux, Windows, Mac OSX, Solaris 등 다양한

운영체제에서 사용가능• Very active community: 1500+ registered

users 2014. 4. 21.

3

ITK

• Visible human 데이터를 처리하기 위해서 개발 되었음

• 영상처리 라이브러리• Segmentation• Registration• GUI 를 제공하지 않음• Visualization 기능 없음 – Visualization Toolkit (VTK)

2014. 4. 21.

4

ITK• Image IO

– PNG, JPEG, DICOM 등의 IO 처리– 확장 가능한 구조

• Numerics– Vision Numerics Library (VNL) 기반– Matrix, Vector 처리

• N-dimensional– 1D : ultrasound– 2D : X-ray– 3D : CT, MRI– 4D : 3D + time, e.g. Beating heart MRI

2014. 4. 21.

5

ITK programing model: Pipeline

2014. 4. 21.

ImageReader

Parameter

File

Filter

ObjectWriterFile

6

ITK programing model: Pipeline

2014. 4. 21.

Threshold Filter Example

7

기본 영상처리 Filter

• Denoising• Image enhancement• Edge detection• Intensity filtering• …

2014. 4. 21.

8

Image Segmentation

2014. 4. 21.

9

Image Segmentation

2014. 4. 21.

10

Segmentation 알고리즘• Region Growing

– Confidence Connected – Connected Threshold – Isolated Connected

• Watersheds

• Level Sets – Fast Marching – Shape Detection – Geodesic Active Contours – Threshold Segmentation – Canny Segmentation Level Set

2014. 4. 21.

11

Shape Detection

2014. 4. 21.

Gradient Sigmoid

RescaleSmooth

ShapeDetection

Threshold

Input Image

Binary Mask

Positive Level Set

GradientMagnitude

FeatureImage

Balanced[-0.5,0.5]

InputFeature

InputLevel Set

OutputLevel Set

12

Registration

2014. 4. 21.

13

FixedImage

Moving Image

Transform

Optimizer

Metric + Domain

Image Registration Framework

2014. 4. 21.

Parameters

14

Registration 알고리즘• Transforms

– Linear: Translation, Rigid, Affine – Deformable: B-Spline, TP-Spline, Daemon

• Metrics– Mean square– Cross-correlation – Mutual Information – …

• Optimizers – Gradient Descent – Genetic algorithms – Conjugate gradient – …

2014. 4. 21.

15

Satellite Images

2014. 4. 21.

16

ITK Application 개발C++ Glue code

2014. 4. 21.

ITK

Image Processing

GUI

MFC, QT, wxWindows,

FLTK

Visualization

OpenGL, VTK

17

ITK applications

• Slicer• Osirix• MeVisLab• XIP• …

2014. 4. 21.

18

Slicer• The Slicer Community 에서

개발하는 오픈소스 소프트웨어• Visualization 과 Image analysis• Windows, Linux, OSX 에서 동작• 주요기능

– Robust DICOM Capabilities– Interactive Segmentation – Volume Rendering– Rigid and Nonrigid Registration– SceneSnapshot Screen Capture

Functionality– 4D Image Viewer– Flexible Layouts and Slice Viewers– Extension Manager for exploring and

installing plug-ins

2014. 4. 21.

19

OsiriX• OsiriX foundation( 오픈소스 ), Pixmeo(

상용제품 )• OSX 전용 DICOM viewer• TIFF (8,16, 32 bits), JPEG, PDF, AVI, MPEG

and QuickTime• Fully compliant with the DICOM standard• DICOM communication protocol

– Any PACS or medical imaging modality

– STORE SCP - Service Class Provider, STORE SCU - Service Class User, and Query/Retrieve

• OsiriX MD– Cleared by the FDA, as a Class II Medical

Device– Certified as an European CE Class IIa, for

diagnostic imaging in medicine

2014. 4. 21.

OsiriX Foundation

20

MeVisLab• MeVis Medical Solutions AG

와 Fraunhofer MEVIS 개발• A cross-platform application

framework – medical image processing– scientific visualization

• 주요기능– Image registration– Segmentation– Quantitative morphological and

functional image analysis– IDE

• Graphical programming• Rapid user interface prototyping

2014. 4. 21.

21

XIP• Siemens 와 Washington University 개발• Open Source framework and platform

– Rapidly developing Medical Imaging applications– Multiple computing environment

• Siemens– DICOM data handling and navigation– Overlay and region of interest (ROI) objects– ITK and VTK support with automatic Inventor

wrapper generation– XIP Builder: a Visual Programming interface to

design XIP scenegraph

• Washington University– Remotely hosted grid based components and

data sources (available data sources include NCIA)

– Components for CAD algorithms – Components for quantifying changes across

time in consecutive imaging studies

2014. 4. 21.

22

폐 결절 검출 CAD 시스템

2014. 4. 21.

23

영상관리시스템

2014. 4. 21.

24

영상 뷰어

2014. 4. 21.

25

ITK 를 이용한 폐 결절 검출 시스템 개발

Java Glue code

2014. 4. 21.

ITK

Image Processing

GUI

Java Swing

Visualization

VTK

C++ Glue code

ITK

Image Processing

GUI

MFC, QT, wxWindows,

FLTK

Visualization

OpenGL, VTK

26

Java 와 C++ 비교C++• 장점

– ITK 가 C++ 로 개발되어 모든기능을 사용가능

– 실행 속도가 빠름– OS 고유기능 사용 가능

• 단점– 문법이 복잡하여 접근성이

떨어짐– 개발 속도가 느림– 멀티플랫폼 개발 어려움

Java• 장점

– 비교적 단순한 문법으로 접근성이 좋음

– 안정적임– 멀티플랫폼 개발 용이– 개발 속도가 빠름

• 단점– ITK 의 binding 지원이 완벽하지

않아서 일부 기능 사용 불가능• 거의 대부분의 기능 사용 가능

– 속도가 느림• 빠른 속도가 필요한 영상처리

부분은 ITK 를 이용하여 해결

2014. 4. 21.

27

3D LungImageitkImageSeriesReader

폐 결절 검출 Pipeline

2014. 4. 21.

Meta Data

DICOMData

3D LungMask

Lung Volume Segmentation

Nodule Candidates Detection

Nodule Candidates Label Map

False Positive Reduction

Nodules Label Map

28

LUNG VOLUME SEGMENTATION

2014. 4. 21.

29

Lung Volume Segmentation Pipeline

2014. 4. 21.

3D LungImage

ItkThreshold Filter

Parameter

3D LungMask

Remove Rim

Refine Lung Mask

Extract Lung Volume

LungLabel Map

itkLabelMapToBinaryImageFilter

itkBinaryToShapeLabelMapFilter

30

폐 영상 Threshold

2014. 4. 21.

31

폐 영역 추출

2014. 4. 21.

32

Rim 제거

2014. 4. 21.

3D to 2D2D 필터처리

2D to 3D

33

Rim 제거 : 2D 영상처리

2014. 4. 21.

34

Rim 제거

2014. 4. 21.

35

폐 영역 추출

2014. 4. 21.

36

Lung Mask 생성

2014. 4. 21.

37

Lung Mask 생성

2014. 4. 21.

38

Segmented Lung Volume

2014. 4. 21.

39

NODULE CANDIDATES DETECTION

2014. 4. 21.

40

Nodule Candidates Detection Pipeline

2014. 4. 21.

3D LungImage

ItkMaskImageFilter

3D LungMask

Nodule Candidates Label Map

Multi Thresholds Detection

ItkThresholdFilter

Rule Based Filtering

itkLabelMapToBinaryImageFilter

itkBinaryToShapeLabelMapFilter

Nodule Candidates Masks

itkOrImageFilter

41

Rule-based Filtering

• Rule-based filtering 을 통해 폐 혈관과 noise 제거

• 혈관 제거 – Volume is extremely bigger than nodule – Elongated object

• Noise 제거– Radius of ROI is smaller than 3mm– Bigger than 30mm

• Remaining ROIs are nodule candidates2014. 4. 21.

42

Rule-based FilteringRule Description

R1 Small noise

R2 Vessel

R3 Large noise

R4 Nodule

Rules for nodule candidate detection

2014. 4. 21.

43

itkBinaryToShapeLabelMapFilter

2014. 4. 21.

Label Map

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Shape Label

Object

Shape Label

Object

Binary Image

itkBinaryToShapeLabelMapFilter

결절 후보들

44

itkShapeLabelObject

2014. 4. 21.

Shape Features1. Bounding Box 2. Centroid 3. Elongation 4. Equivalent Ellipsoid Diameter 5. Equivalent Spherical Perimeter 6. Equivalent Spherical Radius 7. Feret Diameter 8. Flatness 9. Number Of Pixels 10. Number Of Pixels On Border 11. Perimeter 12. Perimeter On Border 13. Perimeter On Border Ratio 14. Physical Size 15. Principal Axes 16. Principal Moments 17. Roundness

itkShapeLabelObject

45

Multi Thresholds Detection

2014. 4. 21.

46

Rule-based Filtering

2014. 4. 21.

47

Multi Thresholds Detection (cont.)

2014. 4. 21.

48

FALSE POSITIVE REDUCTION

2014. 4. 21.

49

False Positive Reduction• 검출된 결절 후보에서 결절이 아닌 것을 제거 하고

결절을 찾는 과정– 많은 False Positive 가 포함되어 있음

• 결절 후보에서 feature( 특징 값 ) 추출• Feature 데이터를 이용하여 Classification– Rule-based Classifier– Linear Discriminant Classifier– 머신러닝 기반의 classifier

• Artificial Neural Network, Genetic programming, Support Vector Machine

2014. 4. 21.

50

Feature SelectionIndex Feature Index Feature

2-D geometric features Mean inside

Area Mean outside

Diameter Variance inside

Perimeter Skewness inside

Circularity Kurtosis inside

3-D geometric features Eigenvalues

Volume 3-D intensity based statistical features

Compactness Minimum value inside

Bounding Box Dimensions Mean inside

Principal Axis Length Mean outside

Elongation Variance inside

2-D intensity based statistical features Skewness inside

Minimum value inside Kurtosis inside

1f

2f

3f

4f

5f

6f

97 ~ ff

1210 ~ ff

13f

14f

15f

16f

17f

18f

19f

2720 ~ ff

28f

29f

30f

31f

32f

33f

Features for nodule detection2014. 4. 21.

51

itkBinaryToStatisticsLabelMapFilter

2014. 4. 21.

Label Map

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Statistics Label

Object

Binary Image

itkBinaryToStatisticsLabelMapFilter

결절 후보들

52

itkStatisticsLabelObject

2014. 4. 21.

Statistics Features1. Center Of Gravity 2. Histogram 3. Kurtosis 4. Maximum 5. Maximum Index 6. Mean 7. Median 8. Minimum 9. Minimum Index 10. Skewness 11. Standard Deviation 12. Sum 13. Variance 14. Weighted Elongation 15. Weighted Flatness 16. Weighted Principal Axes 17. Weighted Principal Moments

Shape Features1. Bounding Box 2. Centroid 3. Elongation 4. Equivalent Ellipsoid Diameter 5. Equivalent Spherical Perimeter 6. Equivalent Spherical Radius 7. Feret Diameter 8. Flatness 9. Number Of Pixels 10. Number Of Pixels On Border 11. Perimeter 12. Perimeter On Border 13. Perimeter On Border Ratio 14. Physical Size 15. Principal Axes 16. Principal Moments 17. Roundness

itkStatisticsLabelObject

itkShapeLabelObject

53

Classification ( 개발중 ) • WEKA– A collection of open

source ML algorithms• pre-processing• classifiers• clustering• association rule

– Created by researchers at the University of Waikato in New Zealand

– Java based

2014. 4. 21.

54

실험 데이터• 미국 National Cancer Institute (NCI) 의 LIDC database 사용

– The LIDC is developing a publicly available database of thoracic computed tomography (CT) scans as a medical imaging research resource to promote the development of computer-aided detection or characterization of pulmonary nodules

• The database consists of 84 CT scans (up to 2009)– 100-400 Digital Imaging and Communication (DICOM) images– An XML data file containing the physician annotations of nodules– 148 nodules– The pixel size in the database ranged from 0.5 to 0.76 mm– The reconstruction interval ranged from 1 to 3mm

2014. 4. 21.

55

실험 결과

Lung Segmenation Nodule Candidate Detection0

20

40

60

80

100

120

140

160

135 초120 초

24 초

80 초

기존 시스템 ITK 기반 시스템

2014. 4. 21.

평균 실행시간

56

실험 결과기존 시스템• 단일 쓰레드 프로그램• MATLAB, C++• 영상 크기가 크면 속도 저하

심함• 사용이 불편함

ITK 기반 시스템• 멀티 쓰레드 프로그램• Java, ITK, VTK• 안정적으로 동작• 처리속도 빠름• 사용이 편함

2014. 4. 21.

57

결론• Insight Toolkit (ITK)– 의료영상처리를 위한 라이브러리– 다양한 영상처리 알고리즘 제공– DICOM 및 다양한 영상 데이터 처리가능

• ITK 를 이용하는 Applications– Slicer– Osirix– MeVisLab– XIP– ...

2014. 4. 21.

58

결론• ITK 기반 폐결절 검출 시스템 개발– Java

• 빠른 개발• 안정성 및 확장가능성 높임

– 영상 처리 속도 개선

• 향후 계획– Weka 를 이용한 False Positive Reduction 기능 개발 (

진행중 )– Java Swing + VTK 기반의 GUI 및 Visualization 기능– 호환성 및 사용 편의성 증대

2014. 4. 21.

592014. 4. 21.

Q & A

602014. 4. 21.

감사합니다 .