Documentation of Scientific Results and the Handling and Keeping of Scientific Data

Thea M Drachen, Marine biologist, PhD Research Librarian, University of Southern Denmark

Course lecturer

Thea Drachen Ph.D. in marine biology 2008 (University of Technology, Sydney) Information specialist at the Research Support Service Section at Copenhagen University Library and Information Service Research information scientist at Global Information and Analysis, Novo Nordisk Reseach librarian at University of Southern Denmark E-mail: thmd@bib.sdu.dk

Morning schedule

09:30 Documentation of Scientific Results and the Handling and Keeping of Scientific Data: Welcome and introduction Part 1: Expectations and the fitness of scientific results Buzzing Break Part 2: Rules and regulations regarding research data Buzzing Break Part 3: Research data management (handling + keeping) Suggestion for further homework 10:30 Coffee break

Why this lecture?

Recent examples of scientists behaving badly…

http://retractionwatch.com/2014/03/18/researcher-intimidated-trainees-into-faking-heart-test-results-ori/, accessed 13 June 2014

http

://r

etra

ctio

nwat

ch.c

om/2

014/

04/2

5/fo

rmer

-mou

nt-s

inai

-po

stdo

c-fa

ked-

gene

-the

rapy

-dat

a-or

i/#

mor

e-20

115,

acc

esse

d 13

Ju

ne 2

014

Method not fit for purpose 1998 Andrew Wakefield and Lancet paper MMR vaccine causes autism widespread scare of vaccination – with severe direct and indirect effects 2010 retracted - found to be bad scientific practice (no control group and causality unclear)

Exerpts from American Academy of Pediatrics http://www2.aap.org/immunization/families/autismwakefield.html accessed 6 March 2014

damage done notion that MMR vaccine causes autism still rampant

May 8, 2013

Aim of the course and learning goals:

You must be able to explain and apply the norms of Responsible Conduct of Research in relation to:

• Documentation of your scientific results

• The handling and keeping of your scientific data

(aka. research data management…)

Take home messages and intended learning outcome:

• Students should know the requirements for documentation of research as described in Guidelines for Good Scientific Practice

The PhD School of SCIENCE

• Students should be able to reflect on and describe own scientific practice in relation to requirements described above.

• Students should be able to recognize and discuss the underlying principles for creating trustworthy scientific results.

(cf. The Danish Committees on Scientific Dishonesty, for instance Guidelines for research protocols, data documentation and data storage).

In short…

“Research without documentation is just anecdotes”

(Lars Holm Rasmussen, lecturer on this course)

Part 1:

Expectations and the Fitness of Scientific

Results

http://speakyssb.blogspot.com/2009_11_01_archive.html

http://speakyssb.blogspot.com/2009_11_01_archive.html

You must properly document your scientific results! - for without documentation of your research there can be no perceived quality of that research!

What are you expected to do?

Three elements from the Singapore Statement on Research Integrity and data 1. Integrity: Researchers should take responsibility for the

trustworthiness of their research. 4. Research Records: Researchers should keep clear,

accurate records of all research in ways that will allow verification and replication of their work by others.

5. Research Findings: Researchers should share data and

findings openly and promptly, as soon as they have had an opportunity to establish priority and ownership claims.

Great Expectations…

Expectation: The Vancouver Rules

Est. 1978 by a group of editors of general medical journals. 5th edition from 1997. Describes the do’s and don'ts of article publication in medicine. Authorship is substantial participation, where all the following conditions are met: 1. conception and design, or analysis and interpretation

of data AND

2. drafting the article or revising it critically for important intellectual content AND

3. final approval of the version to be published. There are no demands of data management, but managing your data will make it easier to document adherence to the protocol.

Expectation: Follow the principles of how to perform valid measurements 1. Analytical measurements should be made to satisfy an agreed

requirement.

2. Analytical measurements should be made using methods and equipment which have been tested to ensure they are fit for purpose.

3. Staff making analytical measurements should be both qualified and competent to undertake the task.

4. There should be regular independent assessment of the technical performance of a laboratory.

5. Analytical measurements made in one location should be consistent with those made elsewhere.

6. Organizations making analytical measurements should have well defined quality control and quality assurance procedures.

Source: http://www.nmschembio.org.uk/GenericArticle.aspx?m=108&amid=1285 (access date: 11 June 2014)

General expectations of high quality research

Meeting required expectations (i.e. rules etc.) from: • Scientific journals / publishers • Local standards / institutions • Funders’ requirements • Research archives • Ethical guidelines • Authoring (Vancouver rules) • Client standards • Society • The Law • …

Expectation of scientific truth – whatever that may be (cf.

your scientific method)…

http://ufm.dk/en/research-and-innovation/councils-and-commissions/the-danish-committees-on-scientific-dishonesty/the-danish-committees-on-scientific-dishonesty

The Danish Committees on Scientific Dishonesty

Guidelines for Good Scientific Practice • for research protocols and reports, data documentation and data

storage (Chapters 1 and 2)

• for agreements at the initiation of research projects (Chapter 3)

• relating to rights and duties concerning storage and use of research data (Chapter 4)

• on publication matters (Chapter 5)

• the Act on Processing of Personal Data and research projects (Chapter 7)

http://ufm.dk/en/research-and-innovation/councils-and-commissions/the-danish-committees-on-

scientific-dishonesty/the-danish-committees-on-scientific-dishonesty

The Danish Committees on Scientific Dishonesty

Guidelines for research protocols and reports, data documentation and data storage in basic health research • All persons participating in a project shall be able to see and

understand the original trial results, their processing and interpretation.

• The research results ought to be available in the longer term so as to be reassessed or applied for further research.

• An appropriate design and storage of research protocols and research reports, data documentation and data storage are accordingly crucial.

Guidelines for Good Scientific Practice, continued

Buzzing break!

Buzzing topic

How do you create trust in your work? You have 5 minutes

1. We will discuss these in

plenum afterwards.

You have 10 min.

Part 2:

Rules and regulations regarding research data management

Some laws, rules and regulations

Good scientific practice does require the research to be in accordance with e.g.: • The Danish Data Protection Agency (Datatilsynet)

• The Danish National Committee on Biomedical Research

Ethics

• The regional committees on research ethics (Regionerne).

• The Danish Agency on Animal Experiments (Dyreforsøgstilsynet)

• The Danish Committees on Scientific Dishonesty (UVVU)

• The extended practice committee at UCPH

• The Danish Code of Conduct forthcomming

Research Data Management at UCPH

In 2013 The extended practice committee on responsible conduct of research (Det Udvidede Praksisudvalg for god videnskabelig praksis) recommended that the university board create a policy on scientific data management. Exerpts from recommendations • Before research projects involving research data is started, it is the responsibility of the PI to ensure that there is a written clarification of the circumstances under which the research data is collected, stored and secured (i.e. a data management plan, DMP) • Research data should be stored so that the results can be recreated if necessary (integrity) • The responsible researcher is to back up her data (preservation) • Research data should be stored in a sufficient form and in a safe place, i.e. it must not be possible subsequently to manipulate or falsify data or make unauthorized access (security)

The Ministry of Higher Education and Science has a group working on a proposal for a Danish Code of Conduct for Research Integrity. As part of a broader hearing during April/May a conference was held in May 2014. The hearing has just reached its deadline. Chapter 2. Data management “Responsible conduct of research includes proper management of primary materials and data. The key purpose of data management is to guarantee credible and transparent research.”

• Researchers are responsible for storing their data and primary materials.

• Researchers are – unless otherwise regulated – responsible for deciding the extent to which primary material should be retained. When deciding this, researchers should consider the value of the primary materials for assessing the results of the research and take account of the physical and technical possibility of storage at the institution.

Recent developments: A Danish Code of Conduct

Responsibilities – excepts … i. Data and primary materials should be retained, stored and managed in a clear and accurate form that allows the result to be assessed, the procedures to be retraced and – when applicable – the research to be reproduced. ii. Data should be kept for a period of at least five years from the date of publication… iv. The data records should enable identification of persons having conducted the actual research… v. Results should be kept irrespective of whether or not they were published, and should contain a precise and traceable reference to the source. vii. Data and primary material should be retained in a way that makes them available for use by other researchers…

Recent developments: A Danish Code of Conduct

Buzzing break!

Buzzing topic

Which rules regarding data are most critical to your research project? Which of them are the most difficult to follow? You have 5 minutes

1. We will discuss these

in plenum afterwards.

have 10 min.

Creative Ignition (Flickr: Frustration) [CC-BY-2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

Part 3:

Research data management

- or the handling and

keeping of data

Perceived quality is fundamental to research, integrity and the ethos of science

Quality assurance through data management should be common sense as most modern research involves: Large amounts of data

Many scientists and complex organizations

Advanced and complex analytical methods

Economic and public investments

What documentation?

• Quality manual for your lab, project or institution. • Basic documentation (equipment etc). • Organization plan (who is responsible for what?). • Description of methods and procedures. • Overview of material (documents and inventory). • Scholarly publications (authoring). • Records of primary material - any material that form

the basis of the research (e.g. biological material, notes, interviews, texts and literature, or recordings).

• Data - detailed records of the primary materials that comprise the basis for the analysis that generates the results.

Why?

→ Documentation ensures integrity of data (QA) → Cf. rules and regulations! → Meet grant and funder requirements

Research Data Management Again: so why manage your research data? Primarily: Because you must (internal and external requirements)! • Documentation ensures integrity of data (QA) • Cf. rules and regulations! • Meet grant and funder requirements Secondarily: • To increase the visibility of your research1 • Save time, money, ressources • Preserve your data for yourself and others • Increase your research efficiency through documentation • Facilitate new discoveries 1. Dorch, Bertil. «On the Citation Advantage of linking to data» (5. Juli 2012).

http://hprints.org/hprints-00714715.

• Save time, money, resources

• Preserve your data for yourself and others

• Increase your research efficiency through documentation

• Facilitate new discoveries

• Increases the visibility of your research

Spinoffs of quality assurance through data management

Cancer clinical trials

Research data management

Piwowar, Heather A., Roger S. Day, og Douglas B. Fridsma. «Sharing Detailed Research Data Is Associated with Increased Citation Rate». PLoS ONE 2, nr. 3 (2007): e308.

Cancer clinical trials

Research data management

“We found that cancer clinical trials which share their microarray data were cited about 70% more frequently than clinical trials which do not.”

“This result held even for lower-profile publications and thus is relevant to authors of all trials.”

Piwowar, Heather A., Roger S. Day, og Douglas B. Fridsma. «Sharing Detailed Research Data Is Associated with Increased Citation Rate». PLoS ONE 2, nr. 3 (2007): e308.

Research data management

Piwowar, Heather A., Roger S. Day, og Douglas B. Fridsma. «Sharing Detailed Research Data Is Associated with Increased Citation Rate». PLoS ONE 2, nr. 3 (2007): e308.

All data necessary to understand, assess, and extend the conclusions of the manuscript must be available to any reader of Science. All computer codes involved in the creation or analysis of data must also be available to any reader of Science. After publication, all reasonable requests for data and materials must be fulfilled. Any restrictions on the availability of data, codes, or materials, including fees and original data obtained from other sources (Materials Transfer Agreements), must be disclosed to the editors upon submission. If there are any MTAs pertaining to data or materials produced in this research, or that you have agreed to in conducting the research that restrict you from providing data or materials, please describe these and send the editor of your manuscript a copy of these specific MTAs when you submit your manuscript. Fossils or other rare specimens must be deposited in a public museum or repository and available for research.

Scientific community, Science (accessed 11 June 2014)

http://www.sciencemag.org/site/feature/contribinfo/prep/gen_info.xhtml#dataavail

Data and materials

Science supports the efforts of databases that aggregate published data for the use of the scientific community. Therefore, appropriate data sets (including microarray data, protein or DNA sequences, atomic coordinates or electron microscopy maps for macromolecular structures, and climate data) must be deposited in an approved database, and an accession number or a specific access address must be included in the published paper. We encourage compliance with MIBBI guidelines (Minimum Information for Biological and Biomedical Investigations).

Scientific community, Science, continued

http://www.sciencemag.org/site/feature/contribinfo/prep/gen_info.xhtml#dataavail, accessed 12 June 2014

Details include but are not limited to: • Molecular structure data. … Approved databases are the

Worldwide Protein Data Bank, BioMag Res Bank, and Electron Microscopy Data Bank (MSD-EBI), and for synthetic molecules, the Cambridge Crystallographic Data Centre.

• DNA and protein sequences. Approved databases are GenBank or other members of the International Nucleotide Sequence Database Collaboration and SWISS-PROT.

• Microarray data. Data should be presented in MIAME-compliant standard format. Approved databases are Gene Expression Omnibus and ArrayExpress.

• Climate data. Data should be archived in the NOAA climate repository or other public databases.

• Ecological data. We recommend deposition of data in Dryad. Large data sets with no appropriate approved repository must be housed as supplemental materials at Science, or only when this is not possible, on an archived institutional Web site, provided a copy of the data is held in escrow at Science to ensure availability to readers.

Scientific community, Science, continued

http://www.sciencemag.org/site/feature/contribinfo/prep/gen_info.xhtml#dataavail, accessed 12 June 2014

Scientific community, Science – con’t

Science supports the efforts of databases that aggregate published data for the use of the scientific community. Therefore, appropriate data sets (including microarray data, protein or DNA sequences, atomic coordinates or electron microscopy maps for macromolecular structures, and climate data) must be deposited in an approved database, and an accession number or a specific access address must be included in the published paper. We encourage compliance with MIBBI guidelines (Minimum Information for Biological and Biomedical Investigations).

Details include but are not limited to: Molecular structure data. … Approved databases are the Worldwide Protein Data Bank, BioMag Res

Bank, and Electron Microscopy Data Bank (MSD-EBI), and for synthetic molecules, the Cambridge Crystallographic Data Centre.

DNA and protein sequences. Approved databases are GenBank or other members of the International Nucleotide Sequence Database Collaboration and SWISS-PROT.

Microarray data. Data should be presented in MIAME-compliant standard format. Approved databases are Gene Expression Omnibus and ArrayExpress.

Climate data. Data should be archived in the NOAA climate repository or other public databases. Ecological data. We recommend deposition of data in Dryad. Large data sets with no appropriate approved repository must be housed

as supporting online material at Science, or only when this is not possible, on an archived institutional Web site, provided a copy of the data is held in escrow at Science to ensure availability to readers.

Source: http://www.sciencemag.org/site/feature/contribinfo/prep/gen_info.xhtml#dataavail

As of January 2013, the BMJ will no longer publish any trial of drugs or devices where the authors do not commit to making the relevant anonymised patient level data available, on reasonable request

How to?

Research data management: How to do it / an approach!

Research data management: A state of mind!

Goodman et al. (2014) Ten Simple Rules for the Care and Feeding of Scientific Data. PLoS Comput Biol 10(4): e1003542. doi:10.1371/journal.pcbi.1003542

Rule 1. Love Your Data, and Help Others Love It, Too Rule 2. Share Your Data Online, with a Permanent Identifier Rule 3. Conduct Science with a Particular Level of Reuse in Mind Rule 4. Publish Workflow as Context Rule 5. Link Your Data to Your Publications as Often as Possible Rule 6. Publish Your Code (Even the Small Bits) Rule 7. State How You Want to Get Credit Rule 8. Foster and Use Data Repositories Rule 9. Reward Colleagues Who Share Their Data Properly Rule 10. Be a Booster for Data Science

Research data management: Use a checklist for your data management life cycle!

Steps in a typical data management plan: 1. Introduction and context

2. Data types, formats, standards and capture methods

3. Ethics and intellectual property

4. Access, data sharing and re-use

5. Short-term storage and data management

Cf. DCC. (2013). Checklist for a Data Management Plan. v.4.0. Edinburgh: Digital Curation Centre. Available online: http://www.dcc.ac.uk/resources/data-management-plans

http://www.dcc.ac.uk/

http://www.dcc.ac.uk/

http://www.dcc.ac.uk/

http://www.dcc.ac.uk/

http://www.dcc.ac.uk/

Research data management: Proper data management completes a life cycle

The END…

… and the beginning!

Suggested extra homework

Open Researcher and Contributor ID = ORCID • an open standard for uniquely identify scientists and science writers • based on Thomson Reuter’s ResearcherID • links an authors’ ResearcherID and Scopus ID (Elsevier) I.e. ORCID is a kind of social security number for publishing, and can be used as a unique and persistent identity, e.g. corresponding to DOI for digital objects, articles etc. Register at ORCID and complete publication list by importing from Web of Science and / or Scopus: http://orcid.org It only takes 5 minutes

List of useful references

Udvalgene Vedrørende Videnskabelig Uredelighed (2009): Vejledninger i God Videnskabelig Praksis med særlig fokus på sundhedsvidenskab, naturvidenskab og teknisk videnskab. Forsknings- og Innovationsstyrelsen, København.

The Danish Committees on Scientific Dishonesty (2009): Guidelines for Good Scientific Practice with special focus on health science, natural science and technical science. Forsknings- og Innovationsstyrelsen, København.

Daasnes, Camilla (2008) Persondataloven – regler og praksis for god databehandlingsskik, DDA Nyt 94, 4-9 http://samfund.dda.dk/ddakatalog//MogD/md94.pdf

EURACHEM / CITAC Working Group (1998) Quality Assurance for Research and Development and Non-routine Analysis. CITAC, Switzerland.

Practice Committee (2011) Practical advice regarding good scientific practice. University of Copenhagen, Denmark.

Prichard & Barwick (2008) Quality Assurance in Analytical Chemistry. John Wiley & Sons, Ltd, Chichester, England.

Further information about accessibility of data and materials

Cech, T. R. (2003), Sharing Publication-Related Data and Materials: Responsibilities of Authorship in the Life Sciences.

American Psychological Association, Responsible Conduct of

Research: Data Sharing and Data Archiving. National Science Foundation Policy on Data Sharing [PDF]. Committee on Ensuring the Utility and Integrity of Research

Data in a Digital Age and Committee on Science, Engineering, and Public Policy, Ensuring the Utility and Integrity of Research Data in a Digital Age(National Academy Press, Washington, DC, 2009).

CONSORT

Consolidated Standards of Reporting Trials http://www.consort-statement.org/ various initiatives to alleviate the problems arising from inadequate reporting of randomized controlled trials

Database enheden

http://databaseenheden.dk/ •Nemmere adgang til anvendelsen af databaser til opsamling af data •Udvikler både administrativt-, klinisk- og kvalitets- og forskningsbaserede databaser på regionalt og landsdækkende niveau

1960s to explain some of the horrors taking place during WW2 Question: “For how long will someone continue to give shocks to another person if they are told to do so, even if they thought they could be seriously hurt?” i.e. will people do morally wrong things just because an authority figure tells them to?

Stanley Milligram Experiment

75-450 V Prerecorded audio 40 subjects all 40 gave up to 300 V 25 continued to 450 V Repeated in 2009 by BBC 12 subjecsts 9 continued to 450 V

Stanley Milligram Experiment