Assessing the Maturity of Climate Data Records

11
Assessing the Maturity of Climate Data Records John J Bates National Climatic Data Center, NOAA/NESDIS

description

Assessing the Maturity of Climate Data Records. John J Bates National Climatic Data Center, NOAA/NESDIS. Overview. Climate variability and change are having profound effects on society Society deserves full and open access to the data and methods used to produce climate products - PowerPoint PPT Presentation

Transcript of Assessing the Maturity of Climate Data Records

Page 1: Assessing the Maturity of Climate Data Records

Assessing the Maturity of Climate Data RecordsJohn J BatesNational Climatic Data Center, NOAA/NESDIS

Page 2: Assessing the Maturity of Climate Data Records

Overview• Climate variability and change are

having profound effects on society• Society deserves full and open

access to the data and methods used to produce climate products

• Scientists often use nomenclature and methods that are difficult for non-specialists, and sometimes even specialists, to understand

• Scientists must do a better job in standardizing their nomenclature and methods, and more clearly communicate with the public

Page 3: Assessing the Maturity of Climate Data Records

MotivationWhat is at stake?

• History shows that weather observations did not become useful for society until a lexicon was agreed toThe Beaufort scale did this for wind climatology and maritime commerce in the 19th century

• For climate services to benefit society, they must adopt a lexicon that sets expectations for openness, process and transparency that are accessible to the publicHow might we define a climate record lexicon useful to the public in the 21st century?

Page 4: Assessing the Maturity of Climate Data Records

MotivationClimate Change is a highly applied scientific field with major aspects related to regulation and societal wellbeing• Society is, understandably, demanding scientific process

for generating CDRs to be open and transparentIncreasingly complex observing systems require more process control to ensure quality, access, and preservation• For example the NASA/NOAA Suomi NPP satellite is

generating over 5 TB/day of dataSoftware Engineering is also increasingly complex and process management is required to optimize cost, schedule, productivity and quality• These benefits are greatest for larger organizations that

perform tasks over many years

Page 5: Assessing the Maturity of Climate Data Records

Climate Observation Best PracticesSteps to long-term monitoring

• Over the last 20-30 years many investigators have developed methods for seaming together observations with evolving coverage and accuracies

• From these experiences, common elements are emerging on best practices for Climate Data Records

How do we capture and make available these best practices and pass them to new generations?

60S-60N ERSST Annual Anomaly (1880-2009)

60S-60N ERSST Annual Anomaly (1880-2009)

60S-60N ERSST Annual Anomaly (1880-2009)

Page 6: Assessing the Maturity of Climate Data Records

Climate Observation Best Practices –Needs for Long-term Preservation

What common preservation information do we need and how to capture communicate this?

Page 7: Assessing the Maturity of Climate Data Records

A Maturity Matrix for Assessing the Completeness of CDRsThe Maturity Matrix assess Software Readiness, Metadata, Documentation, Product Validation, Public Access, and Utility on a Maturity on a scale of 1-6 in each area• Details can be found in Bates and Privette 2012

DOI :10.1029/2012EO440006 These attributes monitor process, documentation, and access to CDRs. • These are essential in ensuring openness and

transparencyThe Maturity Matrix attempts to ensure these attributes are available for a scientific assessment of a CDR• This is not itself a scientific assessmentThe climate community was invited to apply the Matrix and

comment

Page 8: Assessing the Maturity of Climate Data Records

Community Input on the Maturity MatrixThe Maturity Matrix has been evaluated by a GCOS Workshop, EUMETSAT and the EU FP7 project CORE-CLIMAX , the ESA CCI, and the CEOS SST Virtual Constellation –• EUMETSAT CORE-CLIMAX Workshop in January 2014

performed a self assessment of 30 CDRs (23 satellite, 6 in situ, one re-analysis)

• They updated the maturity matrix based on feedback and use and called it a system maturity matrix (in order to separate this from an assessment of the CDR science)

Page 9: Assessing the Maturity of Climate Data Records

Community Input on the Maturity MatrixCORE-CLIMAX workshop results:• Evaluation and Quality Control needs to consider both

scientific and process quality expressed as maturity• Because the Maturity Matrix always needs interpretation

it shall not be used for a beauty contest by adding up or averaging scores and ranking

• Metrics for scientific and process oriented assessments need further development

• Process maturity should periodically be assessed for ECV Climate Data Records entering climate (change) services

Page 10: Assessing the Maturity of Climate Data Records

CDR Maturity Matrix – Next Steps• The CEOS-CGMS Joint Working Group on Climate is

asking members to provide further evaluation of the CORE-CLIMAX System Maturity Matrix prior to recommending adoption (by later this year; NASA is a member of both organizations)

• The WCRP Data Advisory Council is considering guidelines for scientific assessments and such assessments should be considered separate from system or process maturity

• Software assessment tools (called static analysis tools) should be used to evaluate code to minimize complexity and remove excess code

Page 11: Assessing the Maturity of Climate Data Records

CDR Maturity Matrix – Final Thoughts

The stakes in climate change are too high to assume a standard research approach to the creation of CDRs. Society is demanding more documentation, openness, and transparency.

It is imperative that we, the climate community, responds with quantifiable metrics that inform society of both the scientific quality and process maturity of CDRs.