Adding Value to Data

A Rake’s Progress

Contents

• Anecdotes from A Rake’s Progress• The TERN/OzFlux facility• The OzFlux data path• Non-generic things about OzFlux• Generic things about OzFlux• Conclusions

TOGA-COARE

OASIS ‘94 and ‘95

The Savanna Project

The TERN/OzFlux Facility

Knowledge of ecosystem exchange of carbon, water & energy.

Surface fluxesRadiation

MeteorologySoil properties

Ecosystem dynamicsSpatial and temporal

dynamicsContinental & global

budgets

Vegetation typeLeaf area indexGross primary

productSoil moistureHyperspectral

OzFluxFlux tower network

AusCoverRemote sensing

eMASTLand surface models

Site characteristicsBiomass

Soil carbon & nutrientsLeaf-level photosynthesis

AusPlots, SupersitesIntensive field campaigns

The OzFlux Site Data Pathu’, v’, w’, T’

q’,c’WS, WD, T, RH

Fsd, Fsu, Fld, Flu

Fg, Sws, Tsoil

Rain

10 Hz

0.1 Hz

Fast (10 Hz raw)

Slow (30 minute average)

CR3000data

logger

Slow(30

minute average)

Fast (10 Hz)(optional with

ethernet modem)

Modmaxmodem

CF card

OzFlux Data Path

Towerdata

OzFluxQC

L3 data

L4 data

L5 data

L6 data

THREDDS server

OzFlux portal

OAI-PMH server

Community developed Python

scripts

WWW

netCDF files

netCDF files

netCDF files

PI institution

NeCTAR/RDSI

OzFlux Data Portal

• Publicly accessible archive of data• Ability to audit data provided by TERN funded

sites• Publication of data via RDA and TDDP– Make the data easy to find

• Repository of self-documenting data files– netCDF files conforming to CF Metadata

Conventions

Non-generic things about OzFlux

• Data is very homogeneous– Standard suite of instruments– Standard data collection and processing– High level of automation

• Existed prior to TERN with a long history of collaboration

• Lucky breaks– Standard data processing and portal developed before

TERN– Choice of data format compatible with public access to

data

Generic things about OzFlux

• Prior to TERN– Individual solutions to data curation– No license applied to data– Data difficult to discover

• Constituency– Community of rugged individualists

Messages for Others

• Clear vision of what data curation is achievable by your community– Accept outside help

• Solutions need to fit the community– Voluntary compliance is better than enforcement– Guiding is better than directing

• Staged solutions to help community accept change– Herding cats

Communication

• Email lists• ASN/OzFlux newsletter• Use of collaborative resources– Trello for scheduling workflow– GitHub for software updates– CloudStor+ for providing data to site Pis– ResearchGate project

What happens when communication fails?

• Establishment of 2 data paths:– Arose through misunderstanding and a lack of

formality in process– Compromise allowed co-existence as “research”

and “operational” systems– OzFlux will need to make sure boundaries

between the 2 systems remain clear inside and outside OzFlux

– OzFlux will need to make sure maintaining 2 systems does not adversely affect resources

Conclusions

• Scientists can be lead to water and they will drink if;– They can see there is something in it for them.– Change is managed, staged and suits the

community.