The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf ·...
Transcript of The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf ·...
![Page 1: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/1.jpg)
The Impact of Deep Learning on
Radiology
Ronald M. Summers, M.D., Ph.D.
Senior Investigator
Imaging Biomarkers and CAD Laboratory
Radiology and Imaging Sciences
NIH Clinical Center
Bethesda, MD
www.cc.nih.gov/drd/summers.html
![Page 2: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/2.jpg)
Disclosure
• Patent royalties from iCAD
Disclaimer
• Opinions discussed are my alone and
do not necessarily represent those of
NIH or DHHS.
![Page 3: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/3.jpg)
![Page 4: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/4.jpg)
Overview
• Background
• Radiology imaging applications
• Data mining radiology reports and images
• Challenges and pitfalls
![Page 5: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/5.jpg)
We’ve Entered the Deep
Learning Era
• Hand-crafted features less important
• Large annotated datasets more important
• Impact: More and varied researchers can
contribute, accelerating the pace of
progress
![Page 6: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/6.jpg)
Deep Learning• Convolutional neural networks (ConvNets)
• An improvement to neural networks
• More layers permit higher levels of abstraction
• Similarities to low level vision processing in
animals
• Marked improvements in solving hard problems
like object recognition in pictures
![Page 7: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/7.jpg)
H Roth et al., SPIE MI 2015
![Page 8: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/8.jpg)
Deep Learning Improves CAD
Roth et al. IEEE TMI 2015
![Page 9: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/9.jpg)
Deep Learning Improves CAD
Roth et al. IEEE TMI 2015
![Page 10: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/10.jpg)
Lymphadenopathy CAD
Hua, Liu, Summers et al. ARRS 2012
![Page 11: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/11.jpg)
• 90 CTs with
388 mediastinal
LNs
• 86 CTs with
595 abdominal
LNs
• Sensitivities
70%/83% at 3
FP/vol. and
84%/90% at 6
FP/vol.,
respectivelyH Roth et al., MICCAI 2014
![Page 12: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/12.jpg)
• Deeper CNN model performed best
• GoogLeNet for mediastinal LNs
• Sensitivity 85% at 3 FP/vol.
HC Shin et al., IEEE TMI 2016
![Page 13: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/13.jpg)
Lymph Node CT Dataset• doi.org/10.7937/K9/TCIA.2015.AQIIDCNM
• TCIA CT Lymph Node
• 176 scans, 58 GB
• Also: annotations, candidates, masks
![Page 14: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/14.jpg)
Detection of Conglomerate Lymph
Node Clusters
A Gupta et al.
![Page 15: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/15.jpg)
Pancreas CAD
Dice 87.5%
A Farag et al. MICCAI Abd WS 2014; RSNA 2014
![Page 16: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/16.jpg)
Pancreas CAD using CNN
H Roth et al., SPIE MI 2015
![Page 17: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/17.jpg)
Pancreas CT Dataset• doi.org/10.7937/K9/TCIA.2016.tNB1kqBU
• TCIA CT Pancreas
• 82 scans, 10 GB
![Page 18: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/18.jpg)
Gao et al. IEEE ISBI 2016
Segmentation Label Propagation
![Page 19: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/19.jpg)
Gao et al. IEEE ISBI 2016
Segmentation Label Propagation
![Page 20: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/20.jpg)
Colitis CAD
Wei et al. SPIE, ISBI 2013
![Page 21: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/21.jpg)
Colitis CAD
J Liu et al. SPIE Med Imaging 2016
![Page 22: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/22.jpg)
Colitis CAD
J Liu et al. SPIE Med Imaging and ISBI 2016
• 26 CT scans of patients with colitis
• 260 images
• 85% sensitivity at 1 FP/image
![Page 23: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/23.jpg)
Spine Metastasis CAD
J Burns, J Yao et al. RSNA 2011; Radiology 2013
![Page 24: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/24.jpg)
Deep Learning Improves CAD
Roth et al. IEEE TMI 2015
![Page 25: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/25.jpg)
Vertebral Fracture CAD
Yao et al. CMIG 2014
![Page 26: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/26.jpg)
Vertebral Fracture CAD
Burns et al. Radiology 2016
• 92% sensitivity for fracture localization
• 1.6 FPs per patient
• Most common FP: nutrient foramina
(39% of all FPs)
![Page 27: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/27.jpg)
Posterior Elements Fracture CAD
Roth et al. SPIE Med Imaging 2016
• 18 trauma
patients
• 55 fractures
• Test set AUC
0.857
• 71% / 81%
sensitivities at
5 / 10 FP/
patient
![Page 28: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/28.jpg)
H Roth et al., IEEE ISBI 2015
Anatomy Classification Using
Deep Convolutional Nets
• 1,675
patients
• 4,298 images
• Test set AUC
0.998
• 5.9%
classification
error
![Page 29: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/29.jpg)
ImageNet• 14,197,122 images, 21841 synonym sets
indexed
• 1,034,908 bounding box annotations
• Annual challenge inspires fierce
competition
• ImageNet Large Scale Visual Recognition
Challenge (ILSVRC)
Image credit: http://www.image-net.org
![Page 30: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/30.jpg)
HC Shin et al. CVPR 2015
Data Mining Reports & Images
![Page 31: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/31.jpg)
Data Mining Reports & Images
• Trained on 216,000 key images (CT, MR, …)
• 169,000 CT images
• 60,000 patient scans
• Recall-at-K, K=1 (R@1 score)) was 0.56
![Page 32: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/32.jpg)
HC Shin et al. CVPR 2015 & JMLR 2016
Data Mining Reports & Images
![Page 33: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/33.jpg)
Topic: Metastases
HC Shin et al. CVPR 2015 & JMLR 2016
![Page 34: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/34.jpg)
Data Mining Reports & Images
HC Shin et al. CVPR 2015 & JMLR 2016
![Page 35: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/35.jpg)
HC Shin et al. CVPR 2016
Data Mining Reports & Images
![Page 36: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/36.jpg)
HC Shin et al. CVPR 2016
Data Mining Reports & Images
![Page 37: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/37.jpg)
HC Shin et al. CVPR 2016
Data Mining Reports & Images
![Page 38: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/38.jpg)
Challenges and Pitfalls
• Network architecture
• Convolution
• DropOut
• Memory (e.g., LSTM)
• Max pooling
• Softmax
• Number of layers
• Combining classifiers
![Page 39: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/39.jpg)
Challenges and Pitfalls
• Data
• Data augmentation
• Dataset size
• Annotation quality
• Disease (focused vs. comprehensive)
• Availability
![Page 40: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/40.jpg)
Data Augmentation
• During training, input images are sampled
at different scales and random non-rigid
deformations
• Degree of deformation is chosen such that
the resulting warped images resemble
plausible physical variations of the medical
images
• Can help avoid overfitting
![Page 41: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/41.jpg)
ConvNet training with scales and non-rigid deformations
TPS deformation fields
Data augmentation at each superpixel bounding box:
• Ns scales (zoom-out)
• Nd deformations
~800k training images from 60 patients
Roth et al. RSNA 2015
![Page 42: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/42.jpg)
Approach
• If we can create databases of the entire
radiology image & report collection of one or
more hospitals, we will have large datasets
amenable for deep learning any radiology
CAD task.
![Page 43: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/43.jpg)
Challenges and Pitfalls
• Need labels for the images
• Radiology reports
• Crowdsourcing
• Weakly-supervised learning
• Transfer learning
![Page 44: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/44.jpg)
Approach
• ImageNet approach using crowdsourcing
annotations is not feasible due to lack of
radiology expertise.
• The radiologist reports are the annotations.
• Since every radiology study has a report,
every study has already been annotated by
an expert.
![Page 45: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/45.jpg)
Challenges and Pitfalls
• Computation
• GPU acceleration allows efficient training
• Few implementations currently permit use of
GPU clusters (MxNet)
• Learning curve varies widely for publicly
available software platforms
![Page 46: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/46.jpg)
Publicly Available Code
• Caffe (AlexNet, VGGNet, GoogLeNet)
• Theano
• Torch
• TensorFlow
• CNTK (ResNet)
• MxNet
![Page 47: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/47.jpg)
![Page 48: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/48.jpg)
Conclusions• Deep learning leading to large
improvements in CAD and
segmentation
• Pace of deep learning technology
exceptionally fast
• Big data permit new advances
• Interest in deep learning and big data in
radiology image processing is soaring
![Page 49: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/49.jpg)
Acknowledgments
• Jack Yao
• Jiamin Liu
• Le Lu
• Nathan Lay
• Evrim Turkbey
• Amal Farag
• Holger Roth
• Hoo-Chang Shin
• Xiaosong Wang
• Andrew Sohn
• Nicholas Petrick
• Berkman Sahiner
• Joseph Burns
• Perry Pickhardt
• Mingchen Gao
• Daniel Mollura
• Nvidia for GPU card donations
![Page 50: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/50.jpg)
Acknowledgements
NCI
NHLBI
NIDDK
CC
FDA
Mayo Clinic
DOD
U. Wisconsin
Stanford U.
NIH Fellowship
Programs:
Fogarty
ISTP
IRTA
BESIP
CRTP
![Page 51: The Impact of Deep Learning on Radiologyon-demand.gputechconf.com/...ronald...on-radiology.pdf · • Recall-at-K, K=1 (R@1 score)) was 0.56 . HC Shin et al. CVPR 2015 & JMLR 2016](https://reader035.fdocuments.net/reader035/viewer/2022070916/5fb669ee32cc4e75d94c5d2c/html5/thumbnails/51.jpg)
To Learn More …
www.cc.nih.gov/drd/summers.html
www.cc.nih.gov/drd/info/cips.html
Lung Nodule Detection
Tissue Classification
Atlas
Spine Labeling
Virtual Bronchoscopy
Angiography (COW) Tumor Analysis