Alberta Agriculture and Food (AF)
description
Transcript of Alberta Agriculture and Food (AF)
1
Alberta Agriculture and Food (AF)
Surface Meteorological Stations and Data Quality Control Procedures
2
Presentation Overview
• Existing and proposed (AF) network
• Data QA/QC• Parameter list
• Quality states
• QA/QC checks
• Data filling
• Conclusions
3
Meteorological Station Expansion
• 67 N-R-T scalable station platforms• ►all season ppt (GEONOR) ►temperature ►humidity
• ► GOES platform ► 2M wind speed
• ► Campbell Cr10x-2m loggers
• Additional sensors can be added later
• Data will be freely accessible and sensors can be added by any one with dollars, with the caveat that all data would be public domain.
• Currently 44 are installed and operational
• 23 more will be operational by May 1, 2008
4
AF Stations:(N = 113)
• Common Elements• ►All season ppt (GEONOR) ►Temperature ►Humidity• ►Wind speed 2 m ►GOES platform ► Campbell Cr10x-2m loggers
• 36 Drought Net Stations (AGDM)• Incoming short-wave solar radiation (26)• Net solar radiation (3)• Wind speed and direction at 10 m (36)• Soil moisture and temperature at 5, 20, 50, 100 cm (30)
• 10 IMCIN Stations• Incoming short-wave solar radiation (10)• Wind speed and direction at 10 m (10)
• 67 Agriculture Climate Monitoring Stations (AGCM)• Wind direction at 2m (15)• Incoming Short-wave solar radiation(15)
5
6
Existing and proposed stations in Alberta’s Near-Real-Time Network
AGDM (AF)
ACGM (AF)
IMCIN (AF)
Other (AENV AES SRD)
20 km buffer
7
A QA/QC and Data-Filling Decision Support System for
Near Real-Time Climate DataProviding computer-assisted quality assurance, quality control and data
filling
8
Parameter List (Hourly)
• Temperature
• Humidity
• Solar Radiation
• Wind Speed
• Wind Direction
• Precipitation (hourly and 6 hourly)
• Soil Moisture
• Soil Temperature
9
Quality States
• Valid• Not needed to be checked by a human
• Suspect• Needs to be checked by a human and validated or filled
• Invalid• Needs to be checked by a human and filled
• Missing • Needs to be checked by a human and filled
10
QA/QC Checks• Range
• within a reasonable range
• Step • maximum allowable change
• Persistence • minimum allowable change
• Like Sensor • similar value to similar sensors
• Spatial • similar value to neighboring stations (parameter dependent)
11
Methodology for Defining QA/QC checks
• We used the hourly period of record supplied by Environment Canada that contains >25 million records from 250 stations in and around Alberta
• An adjustable trigger point for the “suspect” occurrences was set at 0.01% (1:10,000) for each test• Arbitrary and adjustable (default or station specific)
• For 200 stations examine 50hourly values per day
12
Range Checks
• Three range checks
1. Valid
2. Suspect
3. Invalid
• If the data falls within the inner range then it will be marked Valid If it falls in between the outer range and the inner range it will be marked Suspect
• If data falls outside the outer range it will be marked as Invalid
• If the data is missing it will be marked Missing and then filled
Invalid
Suspect
Valid
Hourly Parameter of Interest
Invalid
Suspect
Valid
Hourly Parameter of Interest
13
Range Checks:Solar Radiation
-0 950
Invalid
Suspect
Valid
Hourly Solar Radiation (W m-2)
• Use wave function to define when day light occurs: Daylight = f(latitude, Julian day)
14
Range Checks:Temperature:
Monthly Max./Min. Extreme (0.01%) Air Temperature Readings
-60
-40
-20
0
20
40
60
1 2 3 4 5 6 7 8 9 10 11 12
Month
Te
mp
era
ture Max. Temp. Obs. Max. Temp. Est.
Min. Temp. Obs Min. Temp. Est.
15
Data Filling
• Temporal filling
• Spatial filling (IDW)
• Spatial-temporal filling (IDW+)
• Manual filling
In every parameter’s daily rollup you know how many records were filled so you can judge the validity of the daily value
16
Conclusions
• Relatively dense high quality and scalable network in the Agricultural area of Alberta
• We have a state of the art QA/QC process that is both flexible and data driven
• Reduces man power
• Capable of generating error logs for maintenance checks
17
18
Persistence Check
valid
valid
susp.susp. susp.
valid
Difference of Maximum and Minimum over n steps must begreater than y
Persistance Checks
19
Step
valid
valid
susp.
valid validvalid
Difference of maximum and minimum over n steps must beat most y
Step Checks
20
Other Tests
• Like Sensors• Relating wind speed 2M to wind speed 10M
• Relating occurrence of precipitation to humidity
• Nearest Neighbors
21
Temporal Filling:for most parameters
One value missing either sideSimply average of two values adjacent values
If more than X consecutive values are missing use spatial interpolation
Data Filling
• 3 for most parameters
• 6 for Soil Temperature
• 12 for Soil Moisture
Missing or Invalid
M M M M
Up to X values missing linearly interpolate missing values from valid end points
22
Spatial filling:Inverse Distance Weighting
Barnwell
Vauxhall
Enchant
Wrentham
Lethbridge
Iron Springs
42.5 km
30.3 km
35.9 km
33.9 km
40.5 km
• Adjustable parameter dependent radius
• Max 8 neighbors
• Rainfall = 70 km radius
• Other = 120 km radius
• Else use nearest station if within X radius
• Else use nearest station and mark as suspect
Data Filling
23
Spatial-Temporal Filling:Precipitation
Total ppt. at Barnwell using IDW = 16.4
Station 12-Oct 13-Oct 14-Oct 15-Oct 16-Oct 17-Oct 18-Oct Distance TotalBarnwell
Vauxhall 0.0 0.0 2.3 1.8 0.0 0.0 1.1 30.3 5.2
Iron Springs 0.0 0.0 11.8 1.0 0.2 0.0 3.0 33.9 16.0
Wrentham 0.0 0.0 15.4 0.4 1.2 2.7 1.4 35.9 21.1
Lethbridge 0.0 0.5 28.5 4.0 1.5 1.0 2.5 40.5 38.0
Enchant 0.0 0.0 6.6 0.0 0.6 0.0 1.2 42.5 8.4
0.44*16.4 0.35*16.4 0.21*16.4
Station 12-Oct 13-Oct 14-Oct 15-Oct 16-Oct 17-Oct 18-OctVauxhall 0% 0% 44% 35% 0% 0% 21%
Distribution of total ppt. by day at nearest station
Station 12-Oct 13-Oct 14-Oct 15-Oct 16-Oct 17-Oct 18-Oct TotalBarnwell 0.0 0.0 7.2 5.7 0.0 0.0 3.5 16.4
Estimated ppt at Barnwell
Data Filling