CACAOCommunity

Assessment of

Community

Annotation with

Ontologies

i.e. Undergrads doing functional annotation using Gene Ontology

Basic problem for Community

Annotation

• People are enthusiastic in theory but too busy in

practice

• Our approach: annotation as a teaching activity

provides incentives to participate (for some)

Student curation focused on IEA

validation

• Without much experience with GO, students can still

– look for experiments to support IEA or ISS annotations

– deprecate incorrect annotations

CACAO 0.1

• 20 students volunteered

• 16 students completed it ~ 2.5 weeks later

• Mini GO camp

– meeting 1: Survey/GO overview

– meeting 2: annotate 2 papers as group

• Paper 1: everyone

• Paper 2: divided into teams

• Each team got a list of E. coli genes; annotation on EcoliWiki

• meeting 3: Survey/Evaluate annotations/announce winners

– Peer review by competition = Teams steal points by correcting

annotations

Results

• Got many more annotations

than we expected

• Quality was not bad

• Survey provided insight into

student strategies

– Conversion of non-GO

functional annotations

– new annotations

Lessons/Refinements

• Students loved the competition

• Need to tweak the training

– More on finding appropriate papers

• Need multiple rounds

– Students did not use the mentors enough

• More time for challenges

• Better wiki tools for mentors/judges to track student

annotation

Scoreboard extension

Scoreboard extension

CACAO 1.0

• Recruiting participants for Fall 2010– ASM, ASMCUE, AgBase, …

– Not just E. coli!!

• We will offer– Content related to GO/genomics/function

– Support for assessment

• Rubrics, surveys etc.

– Plans for possible publications

• Incentives– Credit for innovative teaching on their campuses

– NSF broader impacts

– inter-institution teams for recruiting