Pierre Jaccard1 MyOcean Quality Control for Ferryboxes 10.06.2013 - MyOcean Tutorial, NERSC, Bergen.

Pierre Jaccard 1

MyOcean Quality Control for Ferryboxes

10.06.2013 - MyOcean Tutorial, NERSC, Bergen

Pierre Jaccard 2

Quality Controlled Data

• How good are they?• Is a good data good enough?• Definitions


NIVA - Jerico/Ferrybox Worlshop Helsinki 23-25 April 2013

323.04.2013

MyOcean In-Situ Quality Control

• Real Time Quality Control Procedures– Based on ARGO QC procedures

• Argo, 2009: Argo quality control management, Version 2.4, Argo data management

– T and S QC manual– BioGeoChem QC manual

• Dealyed Mode Quality Control Procedures– T and S QC manual– BioGeoChem QC manual– Interaction with SDN2

• Parameters– Temperature, Salinity– Chla Fluorescence, Dissolved Oxygen


4

Infrastructure Types

• Vertical Profiles– ARGO, XBT, CTD– Gliders, AUV

• Time Series–Moorings– Ferrybox

23.04.2013

Pierre Jaccard 5

netCDF File Format

Variables

• TIME, TIME_QC• LONGITUDE, LATITUDE, POSITION_QC• DEPH, DEPH_QC• TEMP, TEMP_QC• PSAL, PSAL_QC• FLU2, FLU2_QC• DOXY, DOXY_QC


Pierre Jaccard 6

Variable AttributesExample for variable TEMP:

long_name = sea water temperature in-situ ITS-90 scale standard_name = sea_water_temperature units = degrees_Celsius valid_min = -2.0 valid_max = 40.0 comment = QC_indicator = QC_procedure = 7 sensor_depth = ancillary_variables = TEMP_QC uncertainty = 0 accuracy = 0 precision = 0 resolution = 0 cell_methods = TIME:point DEPTH:point LATITUDE:point LONGITUDE:point DM_indicator = R ferrybox_sensor = <name of sensor installation on ferrybox>


Pierre Jaccard 7

QC Variables Attributes


long_name = quality flag conventions = OceanSITES reference table 2 valid_min = numpy.byte: 0 valid_max = numpy.byte: 9 flag_values = func:MyO.Export.Formats.MyOcean.flags_values flag_meanings = no_qc_performed

good_data probably_good_data bad_data_that_are_potentially_correctable bad_data value_changed not_used nominal_value interpolated_value missing_value

Pierre Jaccard 8

Flags


0 No QC was performed Should not be used without control

1 Good data Can be used safely

2 Probably good data Can be used safely for some applications

3 Bad data, but potentially correctable Should not be used, but maight be improved later

4 Bad data Should not be used

5 Value changed

6 Below detection limit

7 In excess of quoted value

8 Interpolated value

9 Missing value

A Incomplete information


9

ARGO Specific Tests

• Platform identification (GTS)• Impossible speed test (GTS)

– Check drift speed of floats

• Digit rollover test– When number of digits is too small

• Grey list (ARGO)– Stop assimiliation and dissemination of data from sensors not

working properly

• Gross salinity or temperature drift (ARGO)– Detect significant sensor drift

• Deepest pressure test (ARGO)– Some kind of pressure range test

• Instrument range test (Gliders, AUV)– Detect if instrument is within oceanographic limits

23.04.2013


10

Vertical Profiles Tests

• Frozen profile test– Detect instruments that reproduce the

same profile (with too little variations)

• Density inversion• Bottom spike test (XBT)– Detect sudden temperature change

• Pressure increasing test

23.04.2013


11

Common Tests• Impossible date test• Impossible location test• Position on land test• Global range test• Regional range test• Spike test• Gradient test• Stuck value test• Rate of change in time test (Moorings)• Calibration status check

– Move data from good to probably good after a certain period of time

– Delayed mode

• Parameter relationship test

23.04.2013


12

Ferrybox Specific Tests

• Speed range test (Ferrybox)• Pump test (Ferrybox)• Pump history test (Ferrybox)• Instrument comparison test– Detect if 2 measurements of same

parameter are close

• Subsequent trip test– Detects and corrects sensor cleaning events

(biofouling) – Delayed mode

23.04.2013


13

Location Tests

23.04.2013

[TIME][[FROZEN]]

consecutive measurements at identical time[[VALUE]]

1950-01-01 00:00:00 < t < now[[LENGTH]]

require a minimum number of consecutive measurements

minimum = 900s

[SPEED][[RANGE]]

min = 0.5max = 15.0

[[FROZEN]]Consecutive measurements with identical speed

[[HISTORY]]Require sustained speed for a minimum periode of timeduration = 300s

[POSITION][[FROZEN]]

consecutive measurements at identical position

[[RANGE]]geographic placement on earth


14

Flow Tests

23.04.2013

[PUMP][[VALUE]]

System must be pumping[[HISTORY]]

Require flow through system for a minimal period of time

Duration = 600s

[OBSTRUCTION][[VALUE]]

Obstruction signal must be inactive[[HISTORY]]

Require inactive signal for a minimum period of time


15

Parameter Tests

23.04.2013

[PARAMETER][[GLOBAL_RANGE]][[REGIONAL_RANGE]]

Special range tests by geographical range and time range in year

[[GRADIENT]]ARGO gradient test on T&SModified ARGO Test on BGCWork in progress to support all parameters and

platforms[[SPIKE]]ARGO gradient test on T&SModified ARGO Test on BGC Work in progress to support all parameters and

platforms

| V2 - (V3 +V1)/2 | - | (V3 - V1) / 2 |

| V2 - (V3 + V1)/2 |


16

Regional Tests

23.04.2013

Pierre Jaccard1 MyOcean Quality Control for Ferryboxes 10.06.2013 - MyOcean Tutorial, NERSC, Bergen.

Documents

Transcript of Pierre Jaccard1 MyOcean Quality Control for Ferryboxes 10.06.2013 - MyOcean Tutorial, NERSC, Bergen.