2/26/2020

1

Stepping Artificial Intelligence Into The Light of

MammographyBy: Talon Thompson

Outline

• Review the importance of mammography

• How AI performs in the market now

• Walk through how AI works in mammography

• That it won’t replace, just reduce workload

• Comprehend the importance of AI in all modalities

Importance of Mammography

• Breast cancer is the second leading cause of death from cancer in women

• With early detection the treatment outlook for the patient is considerably better

1

2

3

2/26/2020

2

Importance of Mammography

• Majority of medical and governmental organizations recommend getting screened from ages 40 - 50

• Over 42 million exams are completed in the USA and UK combined each year

Importance of Mammography

• It’s the most common type of cancer and the leading cause of cancer related deaths across the world

• Mammography is considered effective in reducing breast cancer–related mortality

Importance cont.

• Women with certain BRCA1 or BRCA2 mutations or who are untested but have first-degree relatives (mothers, sisters, or daughters) who are proved to have BRCA mutations should be tested

• Patients outlook should be to get prioritization for a complete work up instead of a single scan and come back later

4

5

6

2/26/2020

3

Interpretations

• What is the best way to image for breast tumors?

• There’s different modalities such as breast MRI, ultrasound, molecular breast imaging, and CT (research at the moment)

• Even with wide spread exams interpretation remains a challenge

• Utilized is the Breast Imaging-Reporting And Data System (BI-RADS)

• How they are:• Shaped

• Sized• Their density

all determine different qualities of the calcifications

• Could have one or a mixture of them all

Types of Calcifications And Masses

Calcifications Cont.

7

8

9

2/26/2020

4

BI-RADS Simulated Calcifications Example In Tissue

Masses

Mammography is the Current Standard For Breast Imaging

• Highest contrast available to see the very fine calcifications within the tissue

• Increased sensitivity and specificity over other modalities

10

11

12

2/26/2020

5

• Dates back to Alan Turing (early 1950’s)

• Can machines think?

• “artificial intelligence” is attributed to John McCarthy for his 1955 definition of AI as “the science and engineering of making intelligent machines”

• AI has been described as “the branch of computer science dedicated to the development of computer algorithms to accomplish tasks traditionally associated with human intelligence, such as the ability to learn and solve problems”

The Introduction of AI Into Society

AI in the market now

• Automotive Industry

• Agriculture

• Financial Services

• Marketing and advertising

• Criminal Justice

• Surveillance

AI in the Automotive Industry• Google and Tesla autonomous cars

• “Toyota’s vision of autonomous cars is not exactly driverless”• CEO of Toyota, Toyoda, said in an interview

that the automotive industry is moving towards driverless cars

• This is not exactly the direction he wants to take the company

• He explains that it’s about making them safer and more user-friendly with features that allow the “drivers” to be more productive while “behind the wheel”

13

14

15

2/26/2020

6

Surveillance: Facial Recognition

• This software has been around for awhile, but the infrastructure to store all the data is not built up yet

• From a global perspective collecting surveillance video, 2013 to 2017 the amount of information collected in one day went from 413 Petabytes (PB) to 860 PB• 1PB = 1,000 TB• 1TB = 1,000 GB• 1 PB = 1,000,000 GB

From Society to Healthcare• Radiomics

• Pictures vs Datasets

• What’s the end goal of software

• Potentials and limitations of AI

• Describe AI terminology

• 3 articles dataset’s and graph’s

Radiomics• “Field of research aiming

to find associations between qualitative and quantitative information extracted from clinical data to support evidence-based clinical decision-making”

• Meaning AI is able to process large sets of data constantly being poured into its software

Example:

• Kaiser has a lot of this data

• Mammography is one such data set

• Understanding the end goal of this data set is necessary

16

17

18

2/26/2020

7

More Than Pictures, They’re Data Sets

• Radiologists see a picture

• AI sees a grouping of data points to evaluate

• Features pulled from clinical images have different purposes

• Qualitative

• Quantitative

Clinical Image

Radiologist AI

Outcome

Analysis (pattern recognition)

Classification

Lesion No Lesion

What’s the end Goal of the Software?

• The design of an AI study and selection of technique can have a bearing on its goal of applicability to patient care• Likelihood of malignancy upgrade after lesion

procedure

• Percentage chance of malignant tumor

• Or just the management of patients by streamlining their care

• How you relate to the patient; what’s our explanation?• Proper implementation could reduce overhead and

cost

• Fewer follow up procedures and biopsies

One Such Software Goal

Outcome analysis?

• Reducing False positives• radiologists see an abnormality

on a mammogram, but no cancer is actually present

• Reducing False negatives• mammograms appear normal even

though breast cancer is present

19

20

21

2/26/2020

8

Artificial Intelligence in Breast Imaging: Potentials

• AI applications are sought after, if not for salvation then for significant aid, in bringing back the workload on physicians

• Need to accomplish this without depleting the human factor

• AI fine tunes the radiologists view of the image and helps point out areas of interest

• Shortage of radiologists • Moran, S. & Warren-Forward, H. The Australian

BreastScreen workforce: a snapshot. Radiographer 59, 26–30 (2012).

• Wing, P. & Langelier, M. H. Workforce shortages in breast imaging: impact on mammography utilization. AJR Am. J. Roentgenol. 192, 370–378 (2009).

• Rimmer, A. Radiologist shortage leaves patient care at risk, warns royal college. BMJ 359, j4683 (2017).

• Nakajima, Y., Yamada, K., Imamura, K. & Kobayashi, K. Radiologist supply and workload: international comparison. Radiat. Med. 26, 455–465 (2008).

• Greiherr, Greg. Burn Out Puts Radiologists At Risk. Itnonline.com. 20 Nov 2019.

Artificial Intelligence in Breast Imaging: Limitations

• Ribeiro et al. acknowledged there’s a lack of trust in an AI’s predictions or the model

• Physicians are still uncertain with the outcome analysis of the AI predictions

• The hospital setting has a lot of programs built into their infrastructure but communicate poorly with each other.

• Medicine benefits from human touch, the physician-patient relationship, and the medical knowledge gathered through experience

• It only knows what we train it to know based on the parameters we set it up with

Artificial Intelligence Terminology

Machine Learning (ML)

Artificial Neural Network/Neural Network (ANN/NN)

Deep Learning (DL)

Convolution Neural Network (CNN)

22

23

24

2/26/2020

9

Machine Learning (ML)

• Imagine playing a video game and the you have multiple ways of playing such as using swords, bows, and magic• You favor magic so the game learns from this

and spawns enemies that are impervious against magic causing you to change the way you play

• ML is achieved by looping through the input data to fit the output data

Artificial Neural Networks (ANN) & Neural Networks (NN)

• ANN/NN basically the same thing, but broken up specifically in the coding

• One specific attribute of NN would be turning linear function into a nonlinear function

Makes approximations to give you the best answer

Artificial Neural Network

• In an article published over 20 years ago Baker, et al. trained an ANN that predicted 23 of 23 cancers, a 100% sensitivity, in 60 patients

• Percentages from 38% up to 58% (23/40 biopsy results) and 66% (23/35 biopsy results)

• The accuracy of these interpretations for breast lesions improved from 38% to 72–80%

25

26

27

2/26/2020

10

Deep Learning (DL)

• “Deep learning is probably one of the hottest tech topics right now. Large corporations and young startups alike are all gold-rushing this fancy field. If you think big data is important, then you should care about deep learning”

• “The Economist says that data is the new oil in the 21st Century. If data is the crude oil, databases and data warehouses are the drilling rigs that digs and pumps the data on the internet, then think of deep learning as the oil refinery that finally turns crude oil into all the useful and insightful final products”

• “There could be a lot of “fossil fuels” hidden underground, and there are a lot of drills and pumps in the market, but without the right refinery tools, you ain’t gonna get anything valuable”

• DL, a type of representation learning, is important for radiology

• The images, or datasets, are the representation for the software to loop through to find what it’s trained to look for

Convolution Neural Networks (CNN)

• Takes an input image, draws a kernel (a box) around parts of the image, and assigns importance to various objects in the image until an answer is given

• Doesn’t look pixel to pixel looks at the macro level

• Facial recognition for pictures is an example

Application of Combination

Let’s traverse through three studies of combining the two

Now that we’re refreshed on the mammography info

We’ve walked through some basics of AI terminology

28

29

30

2/26/2020

11

1st Study

Introduction To An AI System for Breast Cancer Screening

Google showcased their AI in reading mammograms and how that it’s not as scary as it might seem

McKinney, et. al. International Evaluation Of An AI System For Breast Cancer Screening

Overview Of The AI System

• Goal: To evaluate the performance of AI system for breast cancer prediction

• AI trained to look at the calcification and masses

• Deep Learning models utilized

Randomization & Blinding

• The AI system only evaluated biopsy confirmed breast cancers and original decisions that were made by the physicians

• Blinding for the readers meant none of them knew about the AI system

• Blinding for the AI system meant that it didn’t have access to the test sets from the US and UK during its training period

31

32

33

2/26/2020

12

Summary of Findings

• Reduced workload of 2nd reader in UK by 88%

• The AI software showed a reduction of 5.7%/1.2% (USA/UK) in false positives and 9.4%/2.7% in false negatives

• Sensitivity exhibited in the US dataset showed that the AI system may be capable of detecting cancers earlier than normal

• Specificity advantage showed that the system it could help reduce recall rates and unnecessary biopsies

AI Prediction of Breast Cancer In US

• Article shows a better outcome analysis for the patients

• Fewer call backs for follow ups

• UK dataset compared to the nationwide screening population in age

• That can’t be said for the US dataset, it was collected from a single screening center that specializes in cancer cases

• Vast majority of images used were acquired on Hologic devices

• Future research should assess the performance of the AI system across a variety of manufacturers

• Google’s spokesperson has said the study didn’t account for the racial background of the women being screened

34

35

36

2/26/2020

13

It Won’t Replace, Just Reduce (Workload)

• UK and US radiologists were only marginally outperformed, overall, by the AI software

• Meaning there were quite a few cases that the AI missed and the radiologists caught

Radiologist Study (Discrepancies)

Discrepancies between the AI system and human readers. Two sample cancer cases are given. a, One case that all six US radiologists missed the cancer, but the AI system correctly identified the cancer. Outlined in yellow, is the malignancy that is a small, irregular mass which is associated with microcalcifications in the lower inner right breast. b, Another case that all six US radiologists caught, but the AI system missed. Left, mediolateral oblique view; right, craniocaudal view of the malignancy, which is a dense mass in the lower inner right breast.

2nd Study

Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multi-reader study

• Goal: Reduce false positive recalls

• Convolution Neural Network (CNNs) software utilized

37

38

39

2/26/2020

14

AI-assisted In Reducing False-positive Recall

• No studies have covered differing imaging devices or ethnic diversity• Study showed that breast density is slightly higher with the Asian

population and therefore harder to read • This data was incorporated into the software and was able to

produce more accurate outcomes than if the AI or radiologist read it alone

Multi-staged Testing Model

• The largest breast cancer dataset was developed among known AI algorithms to detect breast cancer• algorithm was trained with data from various institutions and multiple vendors

(GE, Hologic, and Siemens), to show it was able to compare performance in validation datasets from different countries (South Korea, USA, and UK)

Reducing Not Replacing

• Left side carcinoma originally called back by 4/14 without AI aided

• After AI aided it was called back by 13/14 radiologists

• Right side carcinoma called back by 7/14 without AI aided• All 14 radiologists recalled after the aid of AI

40

41

42

2/26/2020

15

Diagnostic Performance ROC Curve

3rd studyDetection of Breast Cancer with Mammography: Effect of an Artificial Intelligence Support System

• Goal of aiding radiologists to reduce reading time

• Convolution Neural Networks (CNNs) utilized

Effect of an Artificial Intelligence Support System

• 2 different cases where the AI aided radiologists in highlighting regions of interest to reduce reading time

• Circled are the areas that the AI saw as problem spots followed up by radiologist

• Recall rate decreased with the support of the AI with the radiologist

43

44

45

2/26/2020

16

Importance of AI in All Modalities

• AI detected specific pathological features pertaining to prostate cancer in Japan

• https://www.riken.jp/en/news_pubs/research_news/pr/2019/20191218_2/index.html

• FujiFilm showcases REiLI AI software at RSNA ’19

• http://reili.fujifilm.com/en/

• Alzheimer’s Disease using 18F-FDG in PET scanners

• https://pubs.rsna.org/doi/10.1148/radiol.2018180958

• GE working with Edison Open AI Orchestrator (One particular application is SonoCNS Ultrasound)

• https://www.gehealthcare.com/products/edison

• Varian has just been approved (as of Feb 2020) FDA 510(k) clearance for their Ethos radiation therapy unit

• https://www.varian.com/products/adaptive-therapy/ethos

Coding Sites For AI

• Treehouse

• Intro To Python

• DataCamp

• Intro to Python and Python for Data Sci Track

• Udacity

• Deep Learning and AI Nanodegree

• Coursera

• Deep Learning by Andrew Ng

• Fast.ai

ReferencesMcKinney, S.M., Sieniek, M., Godbole, V. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020). https://doi.org/10.1038/s41586-019-1799-6

Lee, Carol H. et al. Breast Cancer Screening With Imaging: Recommendations From the Society of Breast Imaging and the ACR on the Use of Mammography, Breast MRI, Breast Ultrasound, and Other Technologies for the Detection of Clinically Occult Breast Cancer. Journal of the American College of Radiology, Volume 7, Issue 1, 18 - 27

Ellen B. Mendelson. Artificial Intelligence in Breast Imaging: Potentials and Limitations. American Journal of Roentgenology 2019 212:2, 293-299

Lippert John, Gruley Bryan, Inoue Kae, and Coppola Gabrielle. Toyota’s Vision of Autonomous Cars Is Not Exactly Driverles. 19 September 2018. https://www.bloomberg.com/news/features/2018-09-19/toyota-s-vision-of-autonomous-cars-is-not-exactly-driverless

Jenkins, N. (2015), “245 million video surveillance cameras installed globally in 2014”, IHS Markit, Market Insight, 11 June, https://technology.ihs.com/532501/245-million-video-surveillance-cameras-installed-globally-in-2014.

Civardi, C. (2017), Video Surveillance and Artificial Intelligence: Can A.I. Fill the Growing Gap Between Video Surveillance Usage and Human Resources Availability?, Balzano Informatik, http://dx.doi.org/10.13140/RG.2.2.13330.66248.

Rizzo, S., Botta, F., Raimondi, S. et al. Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp 2, 36 (2018). https://doi.org/10.1186/s41747-018-0068-z

https://www.cancer.gov/types/breast/mammograms-fact-sheet

M. Kosterska-Spalska, T. Dróżdż, A. Duda-Tazbir, I. Wiek-Smuga; Kielce/PL. Microcalcifications detected on mammography classified as BIRADS 4 and 5 and their correlations with histopatologicfindigns. ECR 2010. https://posterng.netkey.at/esr/viewing/index.php?module=viewing_poster&pi=100400

OECD (2019), Artificial Intelligence in Society, OECD Publishing, Paris, https://doi.org/10.1787/eedfee77-en.

Arden Dertat. Applied Deep Learning - Part 1: Artificial Neural Networks. Aug 8, 2017. https://towardsdatascience.com/applied-deep-learning-part-1-artificial-neural-networks-d7834f67a4f6

Nahua Kang. Introducing Deep Learning and Neural Networks — Deep Learning for Rookies (1). Jun 18, 2017. https://towardsdatascience.com/introducing-deep-learning-and-neural-networks-deep-learning-for-rookies-1-bd68f9cf5883

Summit Saha. A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way. Dec 15, 2018. https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

Sickles, E. A., Wolverton, D. E. & Dee, K. E. Performance parameters for screening and diagnostic mammography: specialist and general radiologists. Radiology 224, 861–869 (2002).

Hyo-Eun Kim, Hak Hee Kim, Boo-Kyung Han, Ki Hwan Kim, Kyunghwa Han, Hyeonseob Nam, Eun Hye Lee, Eun-Kyung Kim, Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study, The Lancet Digital Health, 2020, ISSN 2589-7500, https://doi.org/10.1016/S2589-7500(20)30003-0.

Amrita Khalid. Google’s AI for mammograms doesn’t account for racial differences. January 9, 2020. https://qz.com/1781123/googles-ai-for-mammograms-doesnt-account-for-race/

46

47

48

2/26/2020

17

Thank you for your attention

Questions?

49