Dr Nick HollingsRoyal Cornwall Hospital, UK

Technological advances in reporting:

AI

American TV shows

16th

century Europe

Russian chess

championsNapoleon

(Geoffrey Hinton)

Elon Musk

1. 1517 - (Protestant) Reformation

2. 1815 – defeat at Waterloo

3. WWs I & II

4. 1991 – http, html & first web

browser

Fifth is upon us…

How / why? Man’s fascination

with technology Moore’s law (transistors x2,

£ 1/2 )

Improved object classification (2010 1:10, now better than us)

Investment: x2/yr ↑‘Productivity’…

All areas of state (and private) expenditure Defence Housing Education Healthcare

Work - blue collar and white Autonomous trucks by 2027 with loss of 17m jobs w/w (280k UK) OU

Future Humanity Inst

Journalism – Washing Post, 850 articles in 2017 Law – predicts ECHRs judicial decisions with 79% accuracy Billy Bot taking instructions for barristers , arranging fees and doing

the routine work formerly done by multiple legal clerks

15.7 trillion USD (GDP of China & India combined)

26% GDP China, 15%, US, 10% Europe(PWC )

https://www.pwc.com/gx/en/issues/analytics/assets/pwc-ai-analysis-sizing-the-prize-report.pdf

"An Account of the Principalities of Wallachia and Moldavia" inspired which author's most famous novel?”

750 servers with 8 core, 7 thread, 3.5GHz processors

16 Tb RAM process 500 Gb data/s, 1m

books

Dan Zafrir vs Project DebaterTwo debates Two debates

telemedicine space

exploration Employed

procatalepsis

Google’s AI collaboration with the Pentagon… “Avoid at ALL COSTS any mention of Google’s

involvement. I don’t know what would happen if the media [found out] we are secretly building AI weapons technologies for the Defence industry.”

Google’s chief AI scientist

except the memo did get out…

(…beat IBM, MS & Amazon)

We will only design technology that

1. is socially beneficial

2. avoids bias

3. is built & tested for safety

4. is accountable to people

5. incorporates privacy design principles

6. meets high standards of scientific excellence

7. is made available for uses that accord with these principles

We won’t design

1. technologies likely to do harm

2. weapons technologies whose primary purpose is to injure people

3. technology for surveillance that violates internationally accepted norms

4. technology that contravenes widely accepted principles of law and human rights

unchecked use of AI systems in finance, education, criminal justice, search engines and social welfare have had serious adverse outcomes

Garbage in, garbage out US jurisdictions use algorithms to determine whether

a felon will repeat offend

Incorrect assumptions, based on racially biased data

“comparatively easy to make computers exhibit adult-level intelligence or play checkers but almost impossible to give them the skills of a one-year-old when it comes to perception and mobility“

Geoff Hinton, early DL pioneerUniversity Toronto (CNNs 1987):

“…as a radiologist you are like Wile E. Coyote. You’re already over the edge of the cliff, but you haven’t yet looked down. There’s no ground underneath. It’s just completely obvious that in five years deep learning is going to do better than radiologists.”

Professor AI, University New South Wales ‘Geoffrey is extremely bright but he is

also famous for saying provocative things to get peoples’ attention’

‘Computers good at narrow tasks but cannot approach the human brain, yet. We have 50, 100 maybe 200 years before they match us so we shouldn’t worry.’

Berkley robot - 25 minutes to fold a towel!

AIMachine Learning

Deep Learning

Tasks done by computer,

originally done by humans

Giving learning algorithms access to large volumes of training data 2 or 3 hidden layers

modest computing (cf. DL)

Good at analysing images, booking hotels, telephone banking

Driving cars brittle, not good in fog, trees blocking STOP signs etc

but it learns quickly and now well established

supervised - labelled images unsupervised - no labels, algorithm identifies

patterns in data (Google News) reinforced – reward for success

Small child can identify

Aeroplane

Sky

River

How..?

Hundreds of millions of images

beautifully designed

millions of years development)

Algorithm tells computer that this combination of features is a cat

…huge number of combinations, for just for one object!

More complex algorithm?

No

Expose it to millions of training examples, just like a human

Amazon Mechanical Turk 50,000 users sorted &

labelled <1 billion images

input layer - individual pixels analysed at

middle layers detect eyes, ears, nose characteristic to ‘face’

output layer makes a guess of human rather than e.g. chimp

Once it has ‘learned’ to categorise face it can recognise another even if it hasn’t seen that particular person before

Black box uses existing data to make decisions about other data, up to 150 hidden layers

Performs automatic feature extraction, weighting and re-weighting connections until output is finally correct

Performance improves through reinforcement

Google’s Inception NN predicted Husky not Wolf - all the images

of husky had snow at the bottom.

Black box is opaque, difficult to debug if results are wrong. No one knows what actually goes on inside…

Explainable AI

• Highly intuitive. Computers should not be able beat humans…

• 10170 permutations

Trained on multiple strong amateur games to understand human play

Played itself 1000s of times, learning from its mistakes by reinforcement

2016: 4-1 victory over the 18x world champion

Played several highly inventive winning moves, overturning hundreds of years of received wisdom

Silver D et al, Nature 2016;529:484-9

Ophthalmology – equivalent to or better than Ophthalmologists at detecting diabetic retinopathy. Ting, JAMA 2017; 318(22):2211-2223. doi:10.1001/jama.2017.18152

Dermatology – 95% vs 87% (58 Ds) sensitivity for melanoma and fewer false positives. Haenssle, Annals Oncology 2018 https://doi.org/10.1093/annonc/mdy166

Cardiology – Ultromics (Paul Leeson, £300m p.a. savings in echos)

Neurology – predicting autism and dementia from brain MRI and amyloid PET

Psychiatry - suicide risk and response to Lithium

all this because of its ability to learn from large datasets and recognise patterns

2016 – robot able to sew up a pig intestine better than humans given same task

2017- robot dentist implants two teeth into a woman’s mouth with no human intervention

Loh E, BMJ Leader 2018;2:59-63

2053 – all surgeryOUFHI

Wearable technology ADA – S London, Germany, USA. Increases patient

health literacy & levels of self care

Decision support & acute stroke Ambulance triage app, London Babylon

1. automated image segmentation2. lesion detection, measurement, labelling3. comparison with historical images4. Workflow and image optimisation

Long history

Scott J, Palmer L. Neural network analysis of VQ lung scans. Radiology 1993;186(3): 661-4

Trained on 100 perfusion scans

Tested on 28 live scans – outperformed experienced observer for diagnosis of PE (p=0.39)

Breast (75%) Lung (25%) Cardiovascular (19%) Chest x-ray (19%) Liver (19%) Pulmonary hypertension (13%) Bone (13%)Others – thyroid nodules, cerebral microbleeds etc. etc.

https://www.reactiondata.com/wp-content/uploads/2018/01/Machine-Learning-Reaction-Data.pdf

Little doubt that radiologists perform better with AI than without

lung nodules 200 scans from a screening trial, nodules <6mm1:

▪ 3 radiologists - 57, 68, 46% sensitive▪ Double, 75% ▪ with AI - 96%

Four systems tested on 50 cancers where nodules were visible a year earlier than were detected by radiologists – 70% detected by CAD2

1 – Fraioli F. J Thorac Imaging 2007; 22(3):241-62 – Liang M. Radiology 2016 https://doi.org/10.1148/radiol.2016150063

Algorithm trained on 413k adult CXRs Reports categorised by NLP system into:

critical – PTX

urgent - consolidation

non urgent - HH

normal Assessed 60k imagesAnnarumma et al Radiology 2019. https://doi.org/10.1148/radiol.2018180921

Accurate to within 1 yr in 90%, 2yr 98% , in < 2 seconds (MGH)

Using attention maps computer found to be looking at same features as humans

Lee, H., Tajmir, S., Lee, J. et al. J Digit Imaging (2017) 30: 427.

Extracts quantitative features from imaging data sets

Converts them into mineable data Contextualises with other bioinformatics to

improve diagnosis and prognosis (radiogenomics) Prostate, lung cancer, glioblastoma multiforme

Shifts the radiology paradigm; no longer ‘just’ visual analysis

Gathering radiomic, genomic, proteomic, clinical, immunohistochemical etc. data together into predictive & prognostic models

Efficiencies through Precision scheduling

Identifying patients likely to miss appointments

Vetting and individually customised protocols

Background Image analysis▪ Pushing patients with OP appointments tomorrow to top of pile

▪ Red flagging bleeds, PTXs, obstruction etc

▪ Intelligent comparison of index lesions (lung/liver mets etc) with relevant priors

Reporting

▪ Automatic (structured) report assembly

▪ Semantic error detection

Automatic preparation of MDT lists into PACS folders from e.g. Somerset cancer reg.

Depts should start thinking about how PACS will display AI information:

DICOM SR & presentation state

declaration information – sensitivities and specificities of algorithm & regulatory information

Curtis Langlotz, Radiology 2019: https://doi.org/10.1148/radiol.2019190613

IBM

Buys Merge Healthcare (and its million images) October 2015, $1.5bn

Truven (and its 200m medical records) $2.6bn Feb ’16

Google buys DeepMind January 2014, $600m GE, Microsoft, nVIDIA-Baidu collaboration Over 70 other start-ups… and growing

1. No common report vocabulary2. no annotation/mark up standard3. no measurement uniformity4. more public data sets (‘top 10 conditions’)

5. ongoing disconnect btw DICOM & HL7

RSNA 2018: >70 vendors in ML pavilion “… blowing smoke as they promoted their products in the most creative

ways possible” 1

real-world questions unanswered:▪ how AI software will integrate with PACS▪ how it will be paid for

▪ US: unlikely to gain traction in marketplace if not through reimbursement or other savings▪ US: no specific procedure reimbursement codes exist for AI as yet▪ Evidence matching investment (millions) to income generation very limited

▪ whether radiology AI can be used clinically (only seven have FDA clearance) ?2

1Mike Cannavo - https://www.auntminnie.com/index.aspx?sec=sup&sub=aic&pag=dis&ItemID=1248522Kurt Schoppe - https://doi.org/10.1016/j.jacr.2018.10.0032Greg Slabodkin - https://www.healthdatamanagement.com/news/upmc-cuts-hospital-readmission-rates-with-ml-algorithm

Undone by changes in scanners Data input: manually by oncologists at Memorial Sloan Kettering

Cancer Centre Dr. Leif Jensen, Rigshospitalet Copenhagen Oncology Dept: ‘view of

international literature very limited, too much stress on American studies, too little on international, European and rest of world studies.’ https://www.statnews.com/2017/09/05/watson-ibm-cancer/

Prof Alan Karthikesalingam, Senior Clinical Scientist, GDM: ‘not ready yet’

Prof Steve Halligan, UCLH: ‘Google know a lot about computers but **** all about radiology research

1990-2010

PP s/w

• Lung nodule detection

• CT colonoscopy, CCT – Heartflow FFR

2017

• Arterys

• Quantx CADx gains FDA approval

2025

• AI deep market penetrance

2030

• Why aren’t you using AI?

Many unanswered Qs: What happens to radiologist/physician skills as we become

increasingly reliant on AI? What do we do when AI misses a critical finding? Who do you sue, even if the algorithm is known to be

better than the average clinician? What are the governance arrangements?

▪ Products licensed by the FDA are currently tightly controlled, both in their scope and refresh/update schedules

▪ unsupervised ML devices will test the limits of this governance approach as they learn new ways to detect and answer a problem

What will patients think?

Parallels with driverless cars and pilotless aeroplanes

Has my scan been checked by the machine?

Will they consent to ML and radiomic mining of their PID?

Mobile phones, deteriorating speech patterns, Alzheimer’s

Parkinson’s and arryhthmias via steering wheels(The Apple Heart Study at Stanford. Am Heart J, 2019: https://doi.org/10.1016/j.ahj.2018.09.002 )

Overall, little doubt that AI will transform healthcare: diagnosis, treatment and cost

Can computers be taught morality?

Run away car scenario & the Ethical Switch

Asimov’s 3 laws - sufficient?1. A robot may not injure a human being or, through inaction, allow a human being to

come to harm.

2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws

Potential for unintended consequences that generate a higher payoff

Adversarial learning Open AI, UN expert group

on ‘LAWS’, GDPR ‘right to explanation of algodecisions’

Worryingly naïve…

https://publications.parliament.uk/pa/cm201617/cmselect/cmsctech/145/145.pdf

‘Light touch’ approach to governance…

‘Terminator and the Rise of the Machines is utter nonsense. At best such discussions are decades away’

- Chris Bishop, Dir. Research Microsoft UK

‘AI is a fundamental risk to human civilisation’ – Elon Musk

‘AI will rapidly advance until it vastly outstrips human capabilities’ – Stephen Hawking

AI will surpass human brain power by 2023 and all human brains by 2045 (Singularity) - Ray Kurzweil, Google AI

‘Standards bodies, professional societies, government agencies and private industry must work together to accomplish these goals [provision of publically available, validated training data sets] in service of patients, who are sure to benefit from the innovative imaging technologies that will result.’

Langlotz. Radiology 2019, https://doi.org/10.1148/radiol.2019190613

‘…we little realise just how much AI has already permeated our day to day existence’

‘Fully expect a robot to be holding my hand in the nursing home, not a human being.’

Nick Hollings 2019, CCS