Sustainability & Cochrane ReviewsHow technology can help

Chris MavergamesDirector of Web DevelopmentThe Cochrane Collaboration

Structure of this talk•Sustainability of Review production

(internal sustainability)▫Technologies to assist Review production▫PICOtron and Overviews of Reviews

•Sustainability in the health information marketplace (external sustainability)▫“Nimble” content and thinking outside

Review “container”▫Linked data and Star Trek

•Summary

Information Technology Strategy

“Information and Communication Technology (ICT) is at the heart of The Cochrane Collaboration ... To a very large extent, the success of The Cochrane Collaboration has been based on its investment in ICT.”

- From the Collaboration’s information technology strategy paper (ISSF)

Our data and systems are great, but...

Can we make the machines work a bit

harder?

Some critical points

•For internal processes▫How to automate w/o creating more work▫Important that methodology remains sound

•For external consumers▫Retain context of findings and results▫Create better understanding, not confusion

Sustainability of Review production

(and updating)

Producing Reviews - challenges•We have this incredibly complex,

methodologically rigorous process and aim to be global and comprehensive with a mostly-volunteer workforce

•We’ve just cleared 5,000 reviews and it's taken nearly 20 years, and these need to be maintained and updated

• Estimates say we need min. 10,000 Reviews to be comprehensive (estimate 10 years old!)

Cochrane Reviews are fantastic but...

•…creating them is a long and laborious process

Cochrane Reviews are fantastic but...

•…creating them is a long and laborious process

Help!

Review production assistance• Reference managers

▫ EndNote, Zotero, Connotea, Mendeley and others

• Screening/appraising references▫ ScreenToGo App (email: c.huckvale@imperial.ac.uk for public beta)

• Automated abstract screening/appraisal▫ Semi-automated appraisal Support Vector Machines (SVMs)

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2824679/

• Software covering multiple steps in systematic review production▫ EPPI-Reviewer (http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=1913)▫ Distiller SR (http://systematic-review.net/)▫ EROS (Early Review Organizing Software http://rs.iecs.org.ar/)▫ Automated data extraction from Cochrane Reviews (reading the XML)

• GRADEpro• CRS

Slide courtesy: Rachel Marshall

What if authors could...?

• Pull in Risk of Bias assessment information already done on trials or studies

• See studies already included or excluded in other Reviews

• Other links between studies and Reviews via the CRS – more later in Star Trek section

• Use datasets like Drugbank and Diseasome for auto-completion or help filling out fields in RevMan for standardisation

Could we use technology to assist with…?

•Using MeSH mappings to show coverage of Reviews

•Using study links to show where gaps exist

•Priority-setting

•Creating derivative “views” or products from Cochrane data

PICOtron

PICOtron and Cochrane Clinical Answers

•Project to semi-automate populating Cochrane Clinical Answers, a derivative product of The Cochrane Library

•Python script written by Iain Marshall that extracts data from Cochrane Reviews

•Script automatically and randomly reconfigures Review titles into one of 30 question formats

PICOtron•Data is pulled from various areas of the Review

XML

•Depending on whether the result is significant and favored intervention, a sentence (narrative result per PICO) explaining these results is automatically generated

• Then, an author checks these and combines this information across PICOs into a “clinical answer“

PICOtron output

Overviews of Reviews & Network Meta-analysis

Help in preparing of Overviews of Reviews and Network Meta-analysis driven by linked data

Image from Lorne Becker

paroxetine

sertraline

citalopram

escitalopram

fluoxetine

fluvoxamine

milnacipran

venlafaxine

reboxetine

bupropion

mirtazapine

duloxetine

12 new generation

antidepressants:

Which ones are the

most efficacious?

Example of a trial level synthesis

Thanks to Georgia Salanti. Source: http://cmimg.cochrane.org/workshops-19th-cochrane-colloquium-october-2011

paroxetine

sertraline

citalopram

fluoxetine

fluvoxamine

milnacipran

venlafaxine

reboxetine

bupropion

mirtazapine

duloxetine

escitalopram

Network of Randomized Trials

Thanks to Georgia Salanti. Source: http://cmimg.cochrane.org/workshops-19th-cochrane-colloquium-october-2011

paroxetine

sertraline

citalopram

fluoxetine

fluvoxamine

milnacipran

venlafaxine

reboxetine

bupropion

mirtazapine

duloxetine

escitalopram

Network of Randomized Trials

No trials comparing

reboxetine to bupropion

available in Cochrane Reviews

?

Thanks to Georgia Salanti. Source: http://cmimg.cochrane.org/workshops-19th-cochrane-colloquium-october-2011

Other ideas?

Perhaps we can gain insight into how to improve Review production processes from how our users actually use our output?

segue ...

Sustainability in the health information

marketplace

“The problem is not information overload but filter failure.”

– Clay Shirky

•Cochrane provides a vital curation, filtering and evaluation/quality-assessment role and we need to make this clear

Again, Cochrane Reviews are fantastic BUT...

•There are problems that limit their use by some people▫Difficult to wade through all of the text▫Difficult to understand the figures,

terminology, and other bits of the Review▫Hard to compare interventions without

reading multiple Reviews▫Moving from studies in CENTRAL to Reviews

that included that study difficult▫Can be difficult to find the Review you seek

Searching The Cochrane Library

•Search for “Prozac” – no reviews•Search for “fluoxetine” – 27 reviews

Ideally, we’d restructure our content for different users

•Beginning to do this now:▫Summaries.Cochrane.org for consumers▫Cochrane Clinical Answers for clinicians

•BUT▫Takes a lot of work to reformulate reviews &

authors, CRGs, etc are busy

Wouldn’t it be nice if we could automate or semi-automate some of these processes?

Thinking outside Review container

Thinking outside Review container

Making our content “nimble”

Structured and linked data can help make our content “nimble”

Nimble content can:

• Travel Freely• Retain Context Meaning• Create New Products

- R. Lovinger, Razorfish

Linked data

What is linked data?

Semantic Web

is made up of:

Linked Data & Web of Data

Which all together comprise

Web 3.0

Current web = Web of documents

Docs are linked not data in docs

Machines aren‘t good at reading web pages

•Data on the web is meant for human consumption

•Machines need the data to be structured

•Once structured, information can be more easily shared within datasets and across web pages

It‘s about...

• Taking the complex relationships, interactions and dependencies in our data and modeling them in semantic web language and concepts for machine processing...

• So that we can do things like:▫Gain insights into our data▫Help with priority setting▫Repackage it for different users▫Later, perhaps machine can infer new

knowledge from our data and/or when our data is combined with other datasets

Cochrane Register of Studies

CRS and CENTRAL

• Lack of unique study IDs a real problem

• CRS solves this by providing a unique ID for all studies that can be referenced

• Better linking of data about trials and to Reviews▫ Example: Using forest plots to generate related

studies lists for CENTRAL

• Possibilities with linking to external sources such as PubMed

Insert witty Star Trek reference here!

Findings ontology

Image: Lorne Becker

Marshall-o-gram

Marshall-o-gram

Cochrane Review ontology

We can…•Ask questions that use data from several different

reviews

• Improve search

•Make it easier for people to find Cochrane Reviews

• Link data from studies and Reviews better

• Enhance the experience of our users by including data from other datasets

A question using multiple reviews

I’ve done a search for trials on a particular intervention for dementia.

I want to know which of the trials have been included in a Cochrane Review and a

summary of the risks of bias for the entire set of trials.

Links to the relevant Review for those trials that were included

Links to the relevant Review for those trials that were included

This study was one of 38 studies included in the Cochrane Review, <Title Here>.

Click here to see the full review

These figures summarize Risks of Bias from the trials included in the reviews in your search

RoB Summary for Cochrane Reviews on dementia

Make search work better

Enhancing the User Experience

You Say “Paracetamol”I Say “Acetaminophen”•Or, one could say any of these: Abenol (CA), Acephen, Anadin Paracetamol

(UK), Apo-Acetaminophen (CA), Aspirin Free Anacin, Atasol (CA), Calpol (UK), Cetaphen, Children's Tylenol Soft Chews, Disprol (UK), Exdol (CA), Feverall, Galpamol (UK), Genapap, Genebs, Infant's Pain Reliever, Mandanol (UK), Nortemp, Pain Eze, Panadol (UK), Robigesic (CA), Silapap, Tycolene, Tylenol 8 Hour, Tylenol, Tylenol Arthritis, Uni-Ace, Valorin

Cochrane leading in Web 3.0?

EbHC Semantic Platform

CDSR CRS/CENTRAL

DARE

HTAs

CMR

EbHC Semantic Platform

CDSR CRS/CENTRAL

DARE

HTAs

CMR

DrugBank

UMLS

Diseasome

SymptomOntology

* BBCHealth

Ontology

* Not yet created

Cochrane and EbHC ontology?

Will Cochrane have a bubble here someday?

Summary

• Technology IS at the heart of what we do• For both internal and external applications, we

can leverage these tools to further our mission

•Requires that we think differently about the “container“ of the Review

•Our data needs to become “nimble“ to meet future user needs

•We should proceed slowly, incrementally - What are the “quick wins“?

•Cochrane has the chance to lead in Web 3.0

Thank you