Collaborative Ontology Development

Natasha NoyStanford University

Monday, July 15, 13

The ontology development that we grew up with

Courtesy of Mark Musen

Monday, July 15, 13

Lots of databases and sources

The data is in different silos

Need to integrate them

Considerable benefit if you can integrate the data

Ontologies are essential to science

Monday, July 15, 13

Many ontologies today are largeand there are lots of them

• Gene ontology: 28K classes• Foundational Model of Anatomy: >80K classes• NCI Thesaurus: 80K classes• SNOMED CT: >300K classes

Monday, July 15, 13

There are lots of ontologies and more to come

BioPortal has more than 350 ontologiesonly in the field of

biomedicine

Users uploaded more than 230 ontologies to

WebProtégé in the first two months after its release

Monday, July 15, 13

To provide canonical representation of scientific knowledge

To annotate experimental data to enable interpretation, comparison, and discovery across databases

To facilitate knowledge-based applications for decision support, natural language-processing, data integration

and other applications

Scientists have adopted ontologies

Monday, July 15, 13

Ontology development has changed, too

or to any number ofusers anywhere

in the world

from a loneknowledge engineer

to a few distributed

users

Monday, July 15, 13

Courtesy of Mark Musen

Monday, July 15, 13

Collaborative Ontology Development

• Collaborative• Several users contribute to a single developing

ontology• There are mechanisms to carry out discussions and

to reach consensus

• Ontologies• From simple taxonomies• To expressive OWL ontologies

Monday, July 15, 13

Ontologies That Are Being Developed Collaboratively

Monday, July 15, 13

Gene Ontology (GO)

• Developed by the Gene Ontology Consortium• Goal: create a single terminological resource

for annotating genes and gene function from different model organisms:• drosophilla, mouse, e.coli, homo sapiens, ...

• GO: 38,000 classes

Monday, July 15, 13

Monday, July 15, 13

Key Resource: GO Annotations

Manually curated over the past 10 yearsPublicly available

345,000 annotations for homo sapiens

TP53

Gene productGO:0007569

cell aging

GO Term

PubMed article

ManualGO

Annotation

Monday, July 15, 13

Monday, July 15, 13

The Gene Ontology

Terminology for consistent description of gene products

Issue Tracker

Curators of biomedical

databases

GO Curators 3 full-time curators have access to edit GO

Anyone in the community can submit an issue or request

Monday, July 15, 13

Monday, July 15, 13

The NCI Thesaurus

A reference ontology for cancer biology, translational science, and clinical oncology

~20 full-time editors making changes

Changes are not immediately visible

A “lead editor” who approves the changes, and assigns new tasks

Monday, July 15, 13

International Classification of Diseases (ICD)

Have you looked at your medical insurance bill lately?

Monday, July 15, 13

International Classification of Diseases

Monday, July 15, 13

ICD – Why should you care?

Certificate of death

Policy making

Medical bills

Monday, July 15, 13

Developing ICD-10: Revision process in the 20th century

8 Annual Revision Conferences (1982 - 89)

17 – 58 Countries participated

1- 5 person delegations

Mainly Health Statisticians

Manual curation

List exchange

Index was done later

"Decibel” Method of discussion

Output: Paper Copy

Work in English only

Limited testing in the field

Monday, July 15, 13

ICD-11: the 21st century

• ICD-11 is being developed as an OWL ontology• Being developed collaboratively, in an open

editing process• Links to other ontologies, such as SNOMED CT• 33,000 classes

Monday, July 15, 13

Over 250 domain experts from around the world

Organized in groups, which edit different parts of the ontologyT. Tudorache, S. Falconer, C. Nyulas, N. F. Noy and M. A. MusenWill Semantic Web Technologies Work for the Development of ICD-11?International Semantic Web Conference (ISWC 2010), In-Use Track, Shanghai, China

Monday, July 15, 13

ICD-11 development process

• Each night a snapshot of the commonly edited ontology is published in a public platform to encourage feedback from the larger community http://apps.who.int/classifications/icd11/browse/f/en

• Editorial workflow• Centrally overseen by WHO• Peer-reviewed process for the content and structure• Experts may add change proposals• WebProtégé used as the collaborative ontology

development platform

Monday, July 15, 13

Modeling ICD-11: Different views

Monday, July 15, 13

Linearization

Foundation:ICD categories with

Definitions, synonymsClinical descriptionsDiagnostic criteriaCausal mechanismFunctional impact

Primary care

Morbidity

Mortality

Monday, July 15, 13

Multi-Linguality

Monday, July 15, 13

Links to Other Terminologies

Search in BioPortal

Monday, July 15, 13

All properties are reified

Multi-lingualityExternal references

MetadataEvidence

Monday, July 15, 13

related to

linguisticEntity : LinguisticEntity

LanguageTerm

id : xsd:stringlinearizationSpecification* :

LinearizationSpecificationdefinition : DefinitionTermsynonym* : LanguageTermbodyPart* : BodyPartTerm ...

ICDCategory

source : xsd:string label : LinguisticEntity ...

ReferenceTerm

label : xsd:stringlanguage : xsd:string

LinguisticEntity linearizationView : LinearizationValueSetlinearizationParent : ICDCategoryType ...

LinearizationSpecification

id : xsd:string

Term

DomainConceptsubclass of

Courtesy of Tania Tudorache

Monday, July 15, 13

Monday, July 15, 13

Ontology Development as a Collaborative Process

• Ontology development is an inherently collaborative process

• It is also inherently modular, so “stepping on someone else’s toes” is not a big issue

• Users expect Web 2.0-style interaction: • feeds, emails• watched entities• Web interface• social-networking features

Monday, July 15, 13

Dimensions of Collaborative Workflows

• Ontology size• from 100s to 10,000s of concepts

• Size of the community• Contributors (in some form): from 2-3

to dozens

• Editors: from 1-2 to 20

• Control mechanisms• Variety of roles

• Gatekeepers, etc.

• Client-server editing

• Discussion tools• mailing lists, message boards

• face-to-face meetings, telecons

• Synchronization and editing mechanisms• CVS, SVN

Monday, July 15, 13

WebProtégé

Monday, July 15, 13

“Google docs” for ontologies

Monday, July 15, 13

Collaboration Features

• Simultaneous editing• Change tracking• Threaded discussions for ontology entities and changes

(notes, discussions, proposals, reviews)• Watching ontology entities and branches and notifications• Upload and sharing of ontologies• Download any revision of the ontology• Access policies• User interface customization for domain experts• Change analysis and statistics

Monday, July 15, 13

Monday, July 15, 13

Notes and discussions

Monday, July 15, 13

Monday, July 15, 13

Change tracking

Monday, July 15, 13

Watching entities and branches

Monday, July 15, 13

Download any snapshot in time

Monday, July 15, 13

Research Challenges

• Human-Computer Interaction:• How do we enable domain experts to contribute effectively?

• What are the minimal sets of constructs necessary?

• Change analysis:• Are there patterns in how users edit ontologies?

• Can we use these patterns to guide user interfaces?

• Community dynamics:• What are the dynamics in groups that develop ontologies

collaboratively?

• Are there explicit or implicit roles?

• Do roles change over time?

Monday, July 15, 13