Data Quality Control and Assurance · Quality assurance and Quality control Data contamination...

16
Data Quality Control and Assurance DataONE Community Engagement & Outreach Working Group

Transcript of Data Quality Control and Assurance · Quality assurance and Quality control Data contamination...

Page 1: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

DataQualityControlandAssuranceDataONECommunityEngagement&OutreachWorking

Group

Page 2: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

Definitions

QualityassuranceandQualitycontrolDatacontaminationTypesoferrors

QA/QCbestpractices

BeforedatacollectionDuringdatacollection/entryAfterdatacollection/entry

CCimagebycobalt123onFlickr

LessonTopics

Page 3: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

LearningObjectivesAftercompletingthislesson,theparticipantwillbeableto:

DefinedataqualitycontrolanddataqualityassurancePerformqualitycontrolandassuranceontheirdataatallstagesoftheresearchcycle

CCimageby0xFCAFonFlickr

Page 4: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

TheDataLifeCycle

DataONELifeCycle

Page 5: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

DefinitionsDataContamination:

Processorphenomenon,otherthantheoneofinterest,thataffectsthevariablevalueErroneousvalues

Page 6: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

Definitions:TypesofErrorsErrorsofCommission

IncorrectorinaccuratedataenteredExamples:malfunctioninginstrument,mistypeddataErrorsofOmissionDataormetadatanotrecordedExamples:inadequatedocumentation,humanerror,anomaliesinthefield

CCimagebyNickJWebbonFlickr

Page 7: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

DefiningQA/QCStrategiesforpreventingerrorsfromenteringadatasetActivitiestoensurequalityofdatabeforecollectionActivitiesthatinvolvemonitoringandmaintainingthequalityofdataduringthestudy

Page 8: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

QA/QCBeforeCollectionDefine&enforcestandards

FormatsCodesMeasurementunitsMetadataAssignresponsibilityfordataqualityBesureassignedpersoniseducatedinQA/QC

Page 9: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

QA/QCDuringDataEntryDoubleentry

DatakeyedinbytwoindependentpeopleCheckforagreementwithcomputerverification

RecordareadingofthedataandtranscribefromtherecordingUsetext-to-speechprogramtoreaddataback

CCimagebyweskrieselonFlickr

Page 10: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

QA/QCDuringDataEntryDesigndatastoragewell:

MinimizenumberoftimesitemsthatmustbeenteredrepeatedlyUseconsistentterminologyAtomizedata:onecellperpieceofinformationDocumentchangestodataAvoidsduplicateerrorcheckingAllowsundoifnecessary

Page 11: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

MakesuredatalineupinpropercolumnsNomissing,impossible,oranomalousvaluesPerformstatisticalsummaries

QA/QCAfterDataEntry

CCimagebycobalt123onFlickr

Page 12: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

QA/QCAfterDataEntryLookforoutliers:

OutliersareextremevaluesforavariablegiventhestatisticalmodelbeingusedThegoalisnottoeliminateoutliersbuttoidentifypotentialdatacontamination

Page 13: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

Methodstolookforoutliers

GraphicalNormalprobabilityplotsRegressionScatterplots

MapsSubtractvaluesfrommean

QA/QCAfterDataEntry

Page 14: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

SummaryDatacontaminationisdatathatresultsfromafactornotexaminedbythestudythatresultsinaltereddatavaluesDataerrortypes:commissionoromissionQualityassuranceandqualitycontrolarestrategiesfor

preventingerrorsfromenteringadatasetensuringdataqualityforentereddatamonitoring,andmaintainingdataqualitythroughouttheproject

IdentifyandenforcequalityassuranceandqualitycontrolmeasuresthroughouttheDataLifeCycle

Page 15: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

ResourcesD.Edwards,inEcologicalData:Design,ManagementandProcessing,WKMichenerandJWBrunt,Eds.(Blackwell,NewYork,2000),pp.70-91.Availableatwww.ecoinformatics.org/pubsR.B.Cook,R.J.Olson,P.Kanciruk,L.A.Hook,Bestpracticesforpreparingecologicaldatasetstoshareandarchive.Bull.Ecol.Soc.Amer.82,138-141(2001).A.D.Chapman,“PrinciplesofDataQuality:.ReportfortheGlobalBiodiversityInformationFacility”(GlobalBiodiversityInformationFacility,Copenhagen,2004).Availableathttp://www.gbif.org/communications/resources/print-and-online-resources/download-publications/bookelets/

http://www.gbif.org/communications/resources/print-and-online-resources/download-publications/bookelets/
Page 16: Data Quality Control and Assurance · Quality assurance and Quality control Data contamination Types of errors QA/QC best practices Before data collection During data collection/entry

AboutParticipateinourGitHubrepo:https://dataoneorg.github.io/dataone_lessons/

Suggestedcitation:DataONEEducationModule:DataManagement.DataONE.RetrievedNovember12,2016.Fromhttp://www.dataone.org/sites/all/documents/L01_DataManagement.pptx

Copyrightlicenseinformation:Norightsreserved;youmayenhanceandreuseforyourownpurposes.WedoaskthatyouprovideappropriatecitationandattributiontoDataONE.

https://dataoneorg.github.io/dataone_lessons/
http://www.dataone.org/sites/all/documents/L01_DataManagement.pptx

8. FIELD DATA QUALITY ASSURANCE/QUALITY CONTROL … 8_Field Data QAQC...Version: 7 March 2011 92 8. FIELD DATA QUALITY ASSURANCE/QUALITY CONTROL (QAQC I) Desert tortoise monitoring

8. FIELD DATA QUALITY ASSURANCE/QUALITY CONTROL … 8_Field Data QAQC...Version: 7 March 2011 92 8. FIELD DATA QUALITY ASSURANCE/QUALITY CONTROL (QAQC I) Desert tortoise monitoring

Quality Assurance of Research Data

Quality Assurance of Research Data

Standard Operating Procedure - # 20 Data Quality Assurance ...€¦ · Standard Operating Procedure - # 20 Data Quality Assurance and Quality Control Procedures for the Sitakhela

Standard Operating Procedure - # 20 Data Quality Assurance ...€¦ · Standard Operating Procedure - # 20 Data Quality Assurance and Quality Control Procedures for the Sitakhela

Data Quality Assurance

Data Quality Assurance

2012 Quality Assurance Data Modeling

2012 Quality Assurance Data Modeling

Data quality and assurance · addressing data quality and assurance that must be comprehensively addressed. In many cases, the Independent Administrator will propose quality assurance

Data quality and assurance · addressing data quality and assurance that must be comprehensively addressed. In many cases, the Independent Administrator will propose quality assurance

Quality-Assurance and Data-Management Plan for Water ... · Quality Assurance and Data Management ... This qual-ity-assurance and data-management plan provides guidance ... Policies

Quality-Assurance and Data-Management Plan for Water ... · Quality Assurance and Data Management ... This qual-ity-assurance and data-management plan provides guidance ... Policies

Quality-Assurance Data, Comparison to Water-Quality ... · Quality-Assurance Data, Comparison to Water-Quality Standards, and Site Considerations for Total Dissolved Gas and Water

Quality-Assurance Data, Comparison to Water-Quality ... · Quality-Assurance Data, Comparison to Water-Quality Standards, and Site Considerations for Total Dissolved Gas and Water

Data Quality Assurance Platform for Financial Institutions€¦ · Data Quality Assurance Platform for Financial Institutions Data Quality the way we do it ... Quality Strategy and

Data Quality Assurance Platform for Financial Institutions€¦ · Data Quality Assurance Platform for Financial Institutions Data Quality the way we do it ... Quality Strategy and

Quality Control Data Processing Operations Scanning data capture and quality assurance Quality in the Data Process Geoffrey Greenwell, Data Processing.

Quality Control Data Processing Operations Scanning data capture and quality assurance Quality in the Data Process Geoffrey Greenwell, Data Processing.

Quality Assurance and Quality Control in Longitudinal Studies...QUALITY ASSURANCE Overview As stated above, quality assurance consists of those activities undertaken prior to data

Quality Assurance and Quality Control in Longitudinal Studies...QUALITY ASSURANCE Overview As stated above, quality assurance consists of those activities undertaken prior to data

Data Collection and Quality Assurance

Data Collection and Quality Assurance

Laboratory Quality Assurance and Quality Control Data ... · PDF filestate of connecticut department of environmental protection laboratory quality assurance and quality control data

Laboratory Quality Assurance and Quality Control Data ... · PDF filestate of connecticut department of environmental protection laboratory quality assurance and quality control data

Datonix.it data quality assurance

Datonix.it data quality assurance

Data quality assurance methods for improved distributed ... · Data quality assurance methods for improved distributed temperature sensing (DTS) data Management ... Depth Speed of

Data quality assurance methods for improved distributed ... · Data quality assurance methods for improved distributed temperature sensing (DTS) data Management ... Depth Speed of

Quality Assurance in Data Processing for Census

Quality Assurance in Data Processing for Census

Developing Quality Assurance Project Plans using Data ... · PDF fileDeveloping Quality Assurance Project Plans using Data Quality Objectives and other planning tools Developing QAPPs

Developing Quality Assurance Project Plans using Data ... · PDF fileDeveloping Quality Assurance Project Plans using Data Quality Objectives and other planning tools Developing QAPPs

DATA QUALITY ASSURANCE - World Health Organization

DATA QUALITY ASSURANCE - World Health Organization

Chesapeake Bay Program Geospatial Data Quality Assurance

Chesapeake Bay Program Geospatial Data Quality Assurance

SAMPLE ANALYSIS DATA QUALITY ASSURANCE PROJECT …

SAMPLE ANALYSIS DATA QUALITY ASSURANCE PROJECT …