An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J....

26
CSIRO LAND and WATER An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water, Canberra Technical Report 19/02, May 2002

Transcript of An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J....

Page 1: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

C S I R O L A N D a nd WAT E R

An Automated Weather Database System

F. Zhou, D.J. Smith and S. Khan

CSIRO Land and Water, Canberra

Technical Report 19/02, May 2002

Page 2: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

An Automated Weather Database System

F. Zhou, D.J. Smith and S. Khan

CSIRO Land and Water Technical Report 19/02, May 2002

Page 3: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

Copyright © 2001 CSIRO Land and Water. To the extent permitted by law, all rights are reserved and no part of this publication covered by copyright may be reproduced or copied in any form or by any means except with the written permission of CSIRO Land and Water. Important Disclaimer To the extent permitted by law, CSIRO Land and Water (including its employees and consultants) excludes all liability to any person for any consequences, including but not limited to all losses, damages, costs, expenses and any other compensation, arising directly or indirectly from using this publication (in part or in whole) and any information or material contained in it.

Page 4: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

Table of Contents INTRODUCTION 1 WEATHER DATA COLLECTION AND REPORT NETWORK 1 WEATHER DATABASE 2

Database Structure 2 Weather Data Records 3

WEATHER DATABASE SYSTEM MANAGEMENT 4

System Installation 4 System Operation 4 Start-up and Shut-down 6 System Configuration 7 System Monitoring and Error Reporting 8 Weather Data Check and Patching 9 Weather Reports and Public Access 10

LIST OF FIGURES

Figure 1: Weather data collection and report network 1 Figure 2: Overall structure of the weather database 2 Figure 3: Hourly data records 3 Figure 4: Data flow in the weather data collection system 5 Figure 5: Working procedure of weather database management program 5 Figure 6: Download waiting 6 Figure 7: System termination 7 Figure 8: Error log table 9 Figure 9: Report re-generation 10

LIST OF TABLES

Table 1: Instruments and sensors for weather data measurement 2 Table 2: Tables in the weather database management system 3 Table 3: Data fields in the hourly weather data record 4 Table 4: Weather system table 7 Table 5: System configuration work 8

LIST OF APPENDICES

Appendix 1: Sample Initialisation File 11 Appendix 2: List of Error Messages and Actions 13 Appendix 3: Data Patching Protocol 15 Appendix 4: Sample Weather Report to Bureau of Meteorology 16 Appendix 5: Sample 7 Day Weather Report 17 Appendix 6: Sample 14 Day Weather Report 18 Appendix 7: Sample Current Hour Weather Report 19 Appendix 8: Sample Last 12 Hour Weather Report 20 Appendix 9: Sample Yesterday Weather Report 21 Appendix 10: Sample Monthly Weather Report 22

Page 5: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

1

INTRODUCTION The demand for more timely and localised weather data has resulted in the increasing use of Automatic Weather Stations (AWS). Data from these stations are commonly used to calculate daily reference evaporation and to predict disease and pest outbreaks. Data may also be transmitted to forecasting authorities if needed. As more descriptive biophysical models are developed to assist in land and water management and crop production, the need for high quality, contiguous weather data sets will increase. As major environmental issues such as climate change are monitored, weather data will continue to be invaluable. One of the first tasks that the CSIRO Griffith Laboratory undertook was to collect weather data. The weather data collection started in 1931 using manual observations. By 1970, a computerised system using punched cards had replaced the manual data collection. In October 1981, the weather collection system was upgraded to continuous automatic logging of hourly data. The weather data collection system could no longer serve its function in the new millenium and a new weather database management system was developed and commenced operation in October 1999. The automated weather database management system has the following features:

• Automatic scheduling of routine weather data collection and reporting • Continuous system monitoring and error reporting • Minimal human involvement • High reliability • Low development and operational cost

The application is based on the widely used Microsoft Access and it is independent of the data logger software. Therefore, the weather database management system has the potential to be applied to other similar data collection tasks. WEATHER DATA COLLECTION AND REPORT NETWORK The weather data collection and report network consists of four AWS located at Griffith, Hay, Finley, Tullakool, and the weather database management system based in Griffith Laboratory, CSIRO Land and Water (Figure 1). Each weather station is equipped with one logger (manufactured by Campbell Scientific Inc, USA) sensors, instruments and a modem. The weather database management system consists of Microsoft Access application software and the data logger software (PC208W) running in a Windows NT environment installed on a dedicated weather server. The automated database management system retrieves the weather data, checks for any errors, updates the database and prepares the weather reports on an hourly or daily basis. The weather server communicates with the weather stations via a modem and phone link and communicates with the other servers through the Internet. Table 1 lists the sensors and instruments used for weather data measurement.

Figure 1: Weather data collection and report network

Weather Database Management System

Hay AWS

Finley AWS

Griffith AWS

Tullakool AWS

National Weather Data Server

CSIRO L&W Web Server

Page 6: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

2

Sensor or instrument Weather data recorded

Accuracy Unit

Three-cup anemometer Wind speed ±1.5% km Tipping bucket rain gauge Rainfall ±0.25 mm mm Silicon solar cell Solar irradiance ±3.0% MJ/m2 Semiconductor temperature sensor

Dry-bulb temperature ±0.1 oC oC

Copper-constantan thermocouple Wet-bulb depression ±0.3 oC oC

Table 1: Sensors and instruments for weather data measurement

WEATHER DATABASE Database Structure Data in the weather database is organised and contained in tables, as shown in Figure 2. Data tables hold the hourly weather data collected from the weather stations and are updated hourly or daily when the system is running. System tables contain the parameters used for system management and the messages generated in the process. Table 2 lists the tables in the current weather database.

Figure 2: Overall structure of the weather database

Page 7: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

3

Table Name Type Comment

Comm Parameters System Communication parameters (for record only) Error Log System Error messages Event Log System Events of the weather database system. Currently,

only the start-up event is recorded. Event Type System Description of the events Long Term Weather Data System Average long term weather data Weather System System Weather system parameters Weather Club System People involved in the weather system

management Weather Data (Finley) Data Hourly weather data from Finley station Weather Data (Griffith) Data Hourly weather data from Griffith station Weather Data (Hay) Data Hourly weather data from Hay station Weather Data (Tullakool) Data Hourly weather data from Tullakool station

Table 2: Tables in the weather database management system Weather Data Records Figure 3 shows the hourly weather data records in a data table and Table 3 lists the data fields in the record. The Date field is set as the primary key and identifier of a unique record in the data table. If the weather data of a particular hour is not available for some reason, a blank record will be appended to the table and a warning message sent out. Therefore, there will always be a unique record for each hour. The blank records will be automatically filled later when the data become available. The most likely reason for a blank record is that a communication link cannot be established with the data logger when the database is to be updated. The blank records will be filled at the next scheduled update time if the communication link is resumed. An unfilled blank record after re-establishment of the communication link and data collection means that the data for the hour was not in the memory of the data logger and is subsequently lost. However, the data for the hour can easily be patched in by the weather administrator.

Figure 3: Hourly data records

Page 8: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

4

Field Name Comment

Date A date and time identifier (dd/mm/yyyy hh:00) of the hourly weather data record

Wind Speed (At 2m) Spot wind speed, recorded (km/hr) Wind Speed (Max) Maximum wind speed in the hour, recorded (km/hr) Rain Total rain in the hour, recorded (mm/hr) Solar Irradiance Total solar irradiance in the hour, recorded (Mj/m2/hr) Dry Bulb Temperature (Hourly) Spot dry-bulb temperature on the hour, recorded (oC) Dry Bulb Temperature (Max) Maximum dry-bulb temperature in the hour, recorded (oC) Dry Bulb Temperature (Min) Minimum dry-bulb temperature in the hour, recorded (oC) Dry Bulb Temperature (Mean) Mean dry-bulb temperature in the hour, calculated (oC) Relative Humidity (Hourly) Relative humidity, calculated (%) WB Wet-bulb temperature, calculated (oC) DP Dew point temperature, calculated (oC) Comment Add the comment if any field is patched

Table 3: Data fields in the hourly weather data record

WEATHER DATABASE SYSTEM MANAGEMENT System Installation The weather database management system consists of a compiled Microsoft Access database file (MDE), a Windows DLL file (dllWeather.dll ) and an initialisation (INI) file. It was developed in a Windows NT environment and it can run in both Windows NT and Windows 95 operating systems. The initialisation file has the same file name as the MDE file but with the INI extension name. The weather database management system requires that Microsoft Access, Microsoft Outlook and the PC208W logger software be installed on the system. Edit the INI file to include system-specific paths to these software applications. To install, copy the MDE and INI file into the working directory of the weather database management system. Copy the DLL file into the Windows system directory and make sure that the MFC42d.dll file exists in the system directory. System Operation The weather database management system consists of the database management program and the data logger program. Microsoft Access hosts the database management program and the PC208W logger software communicates with the data loggers at the weather stations. At a specified frequency, the data logger software sends the data request to the weather stations through the modem and phone line and the weather stations send back the weather data if the telecommunication is successful. Figure 4 shows the data flow between different parts of the system. At the same time, the database management program will determine the data record to be added for the hour in the case of an hourly update, extract the data, check if the data is in a reasonable range and fill each field of the record. This will happen to a set of records at the updating hour in the case of a daily update comprising 24 hourly records. At the same time, the program will try to fill any blank records from previous hours, if any. Once the database is updated, the program will prepare the weather reports. Since the database and the data logger software act on the same system clock, their operations are synchronised. Figure 5 shows the working procedure of the database management program.

Page 9: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

5

weather data data request

Windows NT server

Figure 4: Data flow in the weather data collection system

Figure 5: Working procedure of weather database management program

The system is designed to run unattended. It has the potential to automatically handle the switch between daylight saving time and Australian eastern standard time. It has a built-in diagnosis function for error conditions, which will be discussed later. If the system is shut down for a period of time and re-started, it will calculate how many record(s) need to be added into the database.

Initialisation

Start the e-mail and data logger software if they are not running

Start the loop and timer

Determine the record(s) to be added

Extract and check the data

Update the database

Prepare the reports

weather station modem modem

data logger software

weather database management system

Page 10: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

6

Start-up and Shut-down Starting the weather database management system is just like starting any other Windows applications or opening their files, for instance, by double clicking the icon of the MDE file. Once the weather database management system is started, it will detect if the e-mail program and the data logger program are already running. If these programs are not running, the weather database management system will start the programs. The e-mail and the data logger program are specified in the initialisation file. If the data logger program is started by the database management system, a dialog will pop up and display how long it will wait for the logger program to retrieve the data (Figure 6). The waiting time is specified in the initialisation file and can be adjusted. If the data logger program has been running while the database management system is shut down, the dialog can be immediately closed. A timer will run continuously once the weather database management system is started. To shut down the system properly, open the Terminate Scheduler dialog and click the Termination button (Figure 7). Close the dialog and then the database. If the Close button of the database is accidentally clicked while the timer is running, the weather database management system must be re-started. Terminate the Microsoft Access process associated with the weather database from the Windows Task Manager and then restart the database.

Figure 6: Download waiting

Page 11: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

7

Figure 7: System termination

System Configuration The behaviour of the weather data management system may be tailored by modifying the system parameters in the initialisation file (see the sample INI file in Appendix 1) and the Weather System table in the database (Table 4). The database management system must be shut down and re-started for any change to be effective. Field Name Comment

Site ID Unique ID for each site of the system Site Description If a site is an active weather station, the site name will be used for report

name generation. Longitude Longitude of a site Latitude Latitude of a site MetBureau ID Station identifier used in the weather report (METARAWS format) for

Bureau of Meteorology Station ID Station identifier used in the weather report (SYNOPTIC format) for

Bureau of Meteorology Data Table Data table name for storing the hourly weather data Data File Data file name for storing the unprocessed weather data Update Frequency Either hourly or daily. Hourly is the default setting. Update Hour The hour must be specified if weather data from a site is updated daily

Table 4: Weather system table

An active weather station, or the weather station being used, is represented by one record with both data table and data file specified in the Weather System table. The default update frequency for the data records from a weather station is hourly unless otherwise specified. The system parameters should only be changed when it is really necessary. Table 5 serves as a road map for system configuration changes.

Page 12: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

8

Task

Where Parameter

Switch the update frequency between hourly and daily

Weather system table Update Frequency. Specify either 1 or 24.

Change the update hour for the station where daily data record update is applied.

Weather system table Update Hour. Enter a value between 0 and 23.

Include (or remove) a weather station in (or from) the routine management work

Weather system table Data Table and Data File. To include, specify the data table and data file. To remove, leave these two fields blank.

Change the station identifier Weather system table MetBureau ID or Station ID Change the data file name Weather system table Data File Change the data table name Weather system table Data Table Change the e-mail addresses INI file cMailWeatherAddress or

cMailBureauAddress Turn the e-mail function on/off

INI file cMailErr or cBOMMailOn 1 on, 0 off

Change the e-mail or data logger software

INI file cMailProgramExe or cLoggerProgramExe

Start the data logger software from the weather database management system or not

INI file cRunLoggerProgram 1 on, 0 off

Change the minute when the database is updated

INI file cUpdateMinute. This should be set to the beginning of an hour as required by the Bureau of Meteorology

Change the hour when the 7 day and 14 day report are prepared

INI file c7And14DaysReportHour. This should be set to an early hour of the day.

Change the report names. INI file cLatestDailyReport or cLast12HoursReport or cCurrentHoursReport

Change the data format INI file Data format section Change the reasonable data range

INI file Data range section

Table 5: System configuration changes

System Monitoring and Error Reporting Occasionally, the weather stations may not function properly or the measured weather data may be out of range. Therefore, one of the functions of the weather system is to monitor the condition of the weather stations continuously. If the weather system detects an error condition, it will send an error report to the mailing list [email protected] for attention or action depending on the type of the error. The weather system can automatically correct some types of errors in the data received after it sends out a reminder. In addition, the weather system has a function to indicate if itself is still working and this may be activated by modifying the initialisation file. The error messages are stored in the Error Log table for reviewing and data patching (Figure 8). A comprehensive list of error messages and the corresponding actions are given in Appendix 2.

Page 13: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

9

Figure 8: Error log table Weather Data Check and Patching Before it is added into the database, each weather datum is checked against its reasonable range, which is specified in the initialisation file. If the data is out of range, an error message will be sent out and this may require a trip for a maintenance check of the instrument used for measuring the data. Therefore, the error report from rigorous data checking may serve as an early warning of an error condition before it becomes chronic. The reasonable data range may need to be adjusted in different seasons of a year to reflect seasonal variation. For instance, the upper temperature limit could be lowered in the winter season. Although the automatic check function has proved to be capable of filtering out the abnormal weather data for most of time, a quick look at the weather reports is worthwhile to pick up erroneous but ‘within range’ data. For example, a large rainfall event on a seemingly dry day (high radiation, low humidity) was actually from a sprinkler operating near a weather station! When a weather datum is out of range, the weather system will not update the relevant data field if it cannot determine an appropriate value for the field. This will leave a blank field in the data record, which needs to be patched manually. During working hours, the patching can be done within an hour after the error is reported and the daily weather reports calculated in the early morning of next day will contain the patched data. The patched data may be estimated from the data record of the closest weather station in the same hour or from the data record of a previous hour of the same station when sound data was available. In the database, the only original data set is the hourly data stored in the data tables of the weather stations. Therefore, the patching should only be applied to the blank fields in the data tables and the derived data is re-calculated from the patched data set. This will ensure data quality and consistency between different reports. A comment should be added to the comment field of the data record to indicate what kind of change has been made. The manual data patching protocol is described in Appendix 3.

Page 14: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

10

Weather Reports and Public Access Weather reports are generated hourly, daily and monthly. Weather data is sent to the Bureau of Meteorology by e-mail at the beginning of each hour for national weather reporting and aviation control. The weather report contains the data in METARAWS format and it also contains the data in SYNOPTIC format every three hours. Sample weather data reports for Bureau of Meteorology are included in Appendix 4. The daily weather reports are generated at 6:00 am for the Griffith, Finley and Hay stations and at 5:00 am for the Tullakool station. The daily reports provide weather data for the last 7 and 14 days and a comparison with the long-term average weather data (Griffith only). The current hour and last 12 hour (Griffith only) weather reports are also prepared at the beginning of each hour. The yesterday weather report contains the daily weather data for all the weather stations. The monthly report is created on the first day of each calendar month for all the stations. Samples of these weather reports are included in Appendices 5-10. Daily and hourly weather data is available on the web site (http://www.clw.csiro.au/services/weather/) for public access. The web server of CSIRO Land and Water in Perth retrieves the weather reports every hour. For the convenience of the users of weather reports, two files are created for each report. The file of DAT extension name is ready for viewing while the file of CDT extension name is purely a data file, which may be used for further analysis (for example, importing into an excel spreadsheet). The daily or hourly weather data may also be extracted from the database using the Generate Daily Report or Generate Hourly Report functions. Figure 9 shows the Daily Report dialog from which the daily data report may be created.

Figure 9: Report re-generation

Page 15: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

11

Appendix 1: Sample Initialisation File

[Weather system] ; weather server name cWeatherServer = Weather Server

; e-mail program and mailing list cMailProgram = Microsoft Outlook cMailProgramExe = C:\Program Files\Microsoft Office\Office\outlook.exe cMailWeatherAddress = [email protected] ; send error message via e-mail. 1 yes, 0 no. Same below. cMailErr = 1 ; send a message to indicate that the database has been updated cMailUpdateRep = 0 ; data logger program cLoggerProgram = PC208W cLoggerProgramExe = D:\PC208W\BIN\PC208W.EXE ; start the logger program from the weather program cRunLoggerProgram = 1 ; waiting time for data download before updating the database cDownLoadWaitingMinutes = 10 ; the minute when the database is updated every hour cUpdateMinute = 2 ; the data files are updated in the previous hour cAdvancedFileUpdateHour = 1 ; the hour when the daily reports are generated c7And14DaysReportHour = 6 ; the Julian day when the 7 day report starts to provide the comparison ; with long term weather data cFirst7DayReportDayNo = 7 ; the name of the weather reports, generated hourly cLatestDailyReport = yesterday.dat cLast12HoursReport = last12hours.dat cCurrentHoursReport = currenthour.dat ; 7 day and 14 day report name consists of the first three characters ; of the site name plus 7 or 14, ; eg gri7.dat and gri7.cdt is the 7 day report for Griffith ; monthly report name consists of the first three characters ; of the site name plus “_m” plus the number of the month, ; eg gri_m10.dat is the October report for Griffith [Met Bureau] ; the file which records the data sent to the Bureau cBOMFile = Bureau.dat ; send the weather report to the Bureau via e-mail cBOMMailOn = 1 ; mail address cMailBureauAddress = [email protected]

Page 16: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

12

[Campbell Logger] ; Data format cDefaultNumberOfComma = 19 cNoOfComma_Griffith = 20 cNoOfComma_Finley1 = 28 cNoOfComma_Finley = 19 cNoOfComma_Hay = 19 cNoOfComma_Tullakool = 19 cPosLoggerSingnature = 1 cPosYear = 2 cPosDay = 3 cPosHour = 4 cPosRain = 5 cPosChkSum = 6 cPosWindRun = 7 cPosPanelTemp = 8 cPosBatteryVol = 9 cPosIrradiance = 10 cPosAmbTemp = 11 cPosRh = 12 cPosMaxTa = 13 cPosMaxRh = 14 cPosMinTa = 15 cPosMinRh = 16 cPosSpotTa = 17 cPosSpotRh = 18 cPosDepress = 19 cPosMaxWind = 20 ; Data range cMaxRain = 25 cMinRain = 0 cMaxWindSpeed = 75 cMinWindSpeed = 0 cMaxIrradiance = 4.2 cMinIrradiance = 0 cMaxTemperature = 45 cMinTemperature = -4 cMaxRH = 100 cMinRH = 10 cMaxWBDepress = 20 cMinWBDepress = 0 ; Conversion factors

Page 17: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

13

Appendix 2: List of Error Messages and Actions

Error Message Comment Action The ini file {file name} could not be found.

The INI file is not in the directory where the weather database resides.

Copy the required INI file to the directory.

Failed to read the ini file completely. Copy a correct INI file to the directory.

Zero TimeDivider in the ini file. The parameter cTimeDivider in the INI file is zero.

Change the parameter to a positive non-zero integer.

No system data available in the weather system table.

No active weather station is specified in the Weather System table.

Check the weather system table.

The update hour {hour} in the weather system table is out of range.

Change the parameter to a value between 0 and 23.

The update hour has not been set in the weather system table for the daily update of {table name}.

If the data records of a weather station is to be updated daily, the update hour must be specified.

Specify the update hour of the weather station in the weather system table.

File {file name} is not found. The data file specified in the Data File field of Weather System table could not be located.

Check the path or the file name.

The system time is behind the date of last record in the table {table name}

The system time is earlier than the time stamp of the last record.

Check the system time of the computer.

Update hour for {table name} is earlier than the changing hours of daylight saving time.

This is to avoid creating two records of the same time stamp when the clock moves back after the daylight saving.

Specify an update hour which is later than the switching hour, eg from 4:00 am in AEST time zone.

Update hour for {table name} is smaller than the difference between standard time and daylight saving time.

Same as above.

The date of last record {date time} in the table {table name} was unexpected.

Inconsistency between the expected time stamp of last record and the actual time stamp of the last record.

Check the computer time. If it is right, check the time stamp of last record of the table.

No bureau file {date time} would be created due to the unexpected record date.

Warning message. No weather data would be sent to the Bureau of Meteorology. This message accompanies the above message.

Same as above.

File {file name} {date time} was not created in the expected hour. If similar message is received next hour, check the system.

The data file was not updated. Maybe the phone line is busy, or the weather station is not working, or the data logger program is not running.

Often, this message is due to failure to connect the weather station temporarily. Check the mentioned components of the system if the error messages continue.

Add blank record {date time} to the table {table name}.

This message accompanies the above message. The hourly weather data is not available so a blank record is added with only the date field filled.

Same as above.

Could not add the record of duplicated date {date time} in the table {table name}.

Check the computer time

Page 18: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

14

Unknown error in adding the record of the date {date} in the table {table name}.

Basically, the database could not be updated for some reasons.

Check the database.

No bureau file would be created because the bureau station data was not found in the system table.

The identifier(s) for the weather stations could not be found for creating the weather report.

Specify the identifiers for the weather stations which are included in the weather report to the Bureau.

No entry would be created in the bureau file for the station {identifier} because of the unexpected date of last record.

This is an accompanying message of “The date of last record {date time} in the table {table name} was unexpected”.

Same as the action for the message that it accompanies.

No bureau file was created for the hour {date time} because of the unexpected date of last records in all relevant tables.

Same as the above except it applies to all the weather stations.

Same as above.

System information not found in the weather system table.

Field(s) for active weather stations is blank.

Check the weather system table.

Site name unknown in the weather system table.

Specifically, the field of Site Description is blank.

Check if Site Description field has been filled for the active weather stations.

Zero count of data items. The reports are not created.

The weather data field(s) of a group of hourly records used for daily calculation is blank.

Check the data records and patch the data.

No record found for last 12 hours (back from {date time} in the table {table name}.

Missing records. This should not happen if the database is upgraded by the database management system all the time.

Check the data tables. Delete the block of records up to the last continuous record. Re-start the database.

No hourly record is found between {date time} and {date time} in the table {table name}.

Same as above. Same as above.

Less than 24 hourly records are found between {date time} and {date time} in the table {table name}.

Same as above. Same as above.

Update blank record for the record {date time} in the table {table name}.

Just a reminder.

{data field} {number} out of range for the record {date time} in the table {table name}.

The data item fails in the data range check.

If the error message is repeated for the subsequent hours, check the measuring sensor or instrument. Patch the data.

Missing {data field} for the record {date time} in the table {table name}.

The data item is not retrieved from the weather station.

Check the data logger.

Negative Irradiance set to 0 for the record {date time} in the table {table name}.

A reminder accompanying the error message for out-of-range irradiance.

Page 19: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

15

Appendix 3: Data Patching Protocol The patching protocols are the same for all weather variables, with the exception of rainfall. Often it is not a whole record (all data for the hour) that is missing, but just one parameter which is flagged in the error log. Obviously, if a sensor is down, it is imperative that the problem be rectified as soon as possible to capture real data and to minimise patching. The patching protocols are as follows: Up to three hours missing on an individual parameter or whole record:

Extrapolate between the last known and the next good reading. For eg 8.00am 10.0 oC 9.00am 10.00am 11.00am 12.00am 14.0 oC

would become 8.00am 10.0 oC 9.00am 11.0 oC 10.00am 12.0 oC 11.00am 13.0 oC 12.00am 14.0 oC Where there are more than three hours missing on an individual parameter or whole records: Copy the appropriate parameter or whole record as follows. Missing data from Griffith are patched with good data from Hay and vice versa. Missing data from Finley are patched with good data from Tullakool and vice versa. Rainfall patching protocol is as follows:

In the majority of instances the patched data will most likely be zero for each hour not recorded. This will be the case for the three remote stations (Hay, Finley and Tullakool). If unsure, have a look at RH, WB, DP and radiation. If RH is close to 100, WB and DP are similar and radiation is low, there may have been rain. Also have a look at the similar station (Hay and Griffith, Finley and Tullakool) and see if any rain was recorded there. The best we can do for the remote stations is manual rainfall for the day from nearby properties. This has not been needed to date. The Griffith rainfall can be patched more confidently from the manual rain gauge reading, by subtracting the amount already recorded by the AWS and filling in the missing hours, giving equal weight to each hour.

Page 20: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

16

Appendix 4: Sample Weather Report to Bureau of Meteorology

ZCZC METARAWS GTHX 0100 ///08/10KT //// 26.1/10.7 ////// RMK RF//.//000.0= GTHX AAXX 11014 94705 46/// ///08 10261 20107 555 44777= METARAWS FILX 0100 ///06/09KT //// 27.6/10.2 ////// RMK RF//.//000.0= FILX AAXX 11014 94876 46/// ///06 10276 20102 555 44777= METARAWS HAYX 0100 ///07/09KT //// 29.1/12.5 ////// RMK RF//.//000.0= HAYX AAXX 11014 94701 46/// ///07 10291 20125 555 44777= NNNN

Page 21: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

17

Appendix 5: Sample 7 Day Weather Report

Report generated on 11-Jan-2000 Griffith Laboratory CSIRO Land & Water Weather Summary (7 days) for Griffith NSW Date MaxT MinT Wind Sun Rain ET C C km/hr Mj/Sqm mm mm 04-Jan-2000 24.9 15.0 17.4 32.2 0.0 11.6 05-Jan-2000 24.2 10.1 14.4 32.1 0.0 9.4 06-Jan-2000 25.9 11.0 9.0 32.4 0.0 9.1 07-Jan-2000 28.6 10.8 7.2 32.5 0.0 9.3 08-Jan-2000 30.1 16.4 11.3 31.7 0.0 10.6 09-Jan-2000 30.9 17.5 9.2 27.2 0.0 9.5 10-Jan-2000 31.3 17.0 9.2 26.5 0.0 9.2 Average 28.0 14.0 Total 0.0 68.7 Abbreviations: MaxT - Maximum temperature of the day MinT - Minimum temperature of the day Wind - Average hourly wind speed of the day Sun - Total solar irradiance of the day Rain - Total rainfall of the day ET - Evaporation of the day Comparison with long term average: Average temperature is 1.9 degree(s) below normal. ET is 12 percent above normal. Rain for January (to last midnight in the 7 day period) is 0.0 mm. Average for the same period is 11.2 mm. Rain for this year is 0 mm. Average for the same period is 11 mm. Comparisons are based on average weather data (1962-1998) collected by CSIRO.

Page 22: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

18

Appendix 6: Sample 14 Day Weather Report

Report generated on 11-Jan-2000

Griffith Laboratory

CSIRO Land & Water

Weather Summary (14 days) for Griffith NSW

Date Dry bulb Rel. humidity DewPt Wind Rain Sun ET

(Deg C) (%) (Deg C) (Km) (mm) (Mj/Sqm) (mm)

Mean Max Min Mean Max Min

28-Dec-1999 19.2 24.2 15.9 78 99 56 15.0 291 0.0 21.3 5.6

29-Dec-1999 19.4 26.6 12.6 61 87 33 11.1 237 0.0 28.1 7.9

30-Dec-1999 19.2 25.0 12.6 55 86 26 8.8 272 0.0 30.3 9.0

31-Dec-1999 17.7 23.5 11.3 52 80 29 7.0 295 0.0 32.0 9.2

01-Jan-2000 19.0 26.0 11.3 49 78 28 7.1 250 0.0 27.2 8.7

02-Jan-2000 21.2 28.7 13.5 49 83 21 8.5 201 0.0 31.8 9.6

03-Jan-2000 24.4 31.6 15.5 44 77 20 9.8 211 0.0 29.1 10.2

04-Jan-2000 20.7 24.9 15.0 42 64 22 6.7 417 0.0 32.2 11.6

05-Jan-2000 17.3 24.2 10.1 54 98 26 6.9 345 0.0 32.1 9.4

06-Jan-2000 18.5 25.9 11.0 51 78 26 6.8 217 0.0 32.4 9.1

07-Jan-2000 20.4 28.6 10.8 49 77 23 8.0 172 0.0 32.5 9.3

08-Jan-2000 23.3 30.1 16.4 45 67 24 9.9 272 0.0 31.7 10.6

09-Jan-2000 23.9 30.9 17.5 48 73 20 10.9 222 0.0 27.2 9.5

10-Jan-2000 24.1 31.3 17.0 50 73 25 12.2 220 0.0 26.5 9.2

Page 23: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

19

Appendix 7: Sample Current Hour Weather Report

Report generated on 11-Jan-2000

Griffith Laboratory

CSIRO Land & Water

Latest Hourly Weather Report

Site Date Time Wind Rain Sun DBT RH DewPt MaxWind

AEST Km/h mm Mj/Sqm C % C Km/h

Griffith 11-01-2000 11:00 14.7 0.0 3.1 26.1 38.0 10.7 19.3

Finley 11-01-2000 11:00 11.3 0.0 3.3 27.6 33.8 10.2 17.2

Hay 11-01-2000 11:00 13.4 0.0 3.3 29.1 35.8 12.5 16.9

Total (since midnight)

Site Wind Rain Sun

Km mm Mj/Sqm

Griffith 118 0.0 8.3

Finley 69 0.0 9.9

Hay 99 0.0 9.8

Page 24: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

20

Appendix 8: Sample Last 12 Hour Weather Report

Report generated on 11-Jan-2000 Griffith Laboratory CSIRO Land & Water Weather Data (Griffith) for 12 Hours Date Time Wind Rain Sun DBT RH DewPt AEST Km/h mm Mj/Sqm C % C 11-01-2000 00:00 7.6 0.0 0.0 21.1 61.2 13.4 11-01-2000 01:00 7.2 0.0 0.0 19.9 66.3 13.5 11-01-2000 02:00 7.9 0.0 0.0 18.9 69.3 13.1 11-01-2000 03:00 5.4 0.0 0.0 17.3 81.4 14.1 11-01-2000 04:00 5.4 0.0 0.0 18.1 74.3 13.5 11-01-2000 05:00 8.6 0.0 0.0 17.5 78.1 13.6 11-01-2000 06:00 10.0 0.0 0.0 17.5 79.8 13.9 11-01-2000 07:00 11.4 0.0 0.3 18.8 73.5 14.0 11-01-2000 08:00 14.5 0.0 0.9 21.0 61.0 13.2 11-01-2000 09:00 17.4 0.0 1.7 22.9 52.3 12.6 11-01-2000 10:00 15.1 0.0 2.4 24.7 45.9 12.3 11-01-2000 11:00 14.7 0.0 3.1 26.1 38.0 10.7 Total 125.2 0.0 8.3 Max 26.1 81.4 14.1 Min 17.3 38.0 10.7

Page 25: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

21

Appendix 9: Sample Yesterday Weather Report

Report generated on 11-Jan-2000

Griffith Laboratory

CSIRO Land & Water

Daily Weather Report

Site Date DBT RH DewPt Wind Rain Sun ET

C % C km mm Mj/Sqm mm

Avg Max Min Avg Max Min

Griffith 10-01-2000 24.1 31.3 17.0 50 73 25 12.2 220 0.0 26.5 9.2

Finley 10-01-2000 25.2 33.5 18.1 49 82 19 12.0 216 0.0 28.6 10.0

Hay 10-01-2000 25.0 33.4 17.4 47 74 26 12.2 192 0.0 20.9 8.4

Tullakool 10-01-2000 25.9 34.9 18.1 46 87 19 11.2 217 7.4 24.0 9.6

Page 26: An Automated Weather Database System - CSIRO · An Automated Weather Database System F. Zhou, D.J. Smith and S. Khan CSIRO Land and Water ... system was upgraded to continuous automatic

22

Appendix 10: Sample Monthly Weather Report

Report generated on 01-Nov-1999

Griffith Laboratory

CSIRO Land & Water

Weather Summary (Oct 1999) for Hay NSW

Date Dry bulb Rel. humidity DewPt Wind Rain Sun ET

(Deg C) (%) (Deg C) (Km) (mm) (Mj/Sqm) (mm)

Mean Max Min Mean Max Min

01-Oct-1999 14.5 23.7 3.6 69 99 33 7.8 122 0.0 23.1 5.1

02-Oct-1999 17.2 26.9 10.7 80 97 42 13.1 174 31.4 16.9 1.6

03-Oct-1999 16.5 19.8 14.0 91 98 76 14.9 184 5.8 7.5 4.7

04-Oct-1999 17.2 24.4 10.3 68 97 31 10.1 269 0.0 24.3 6.3

05-Oct-1999 15.3 23.8 8.0 67 98 29 8.0 224 0.0 25.0 6.1

06-Oct-1999 15.4 25.1 4.4 61 94 19 5.9 118 0.0 25.4 6.5

07-Oct-1999 17.9 26.1 9.4 58 91 25 8.0 171 0.0 24.7 7.0

08-Oct-1999 20.6 29.4 10.3 51 87 23 8.5 215 0.0 24.0 9.1

09-Oct-1999 26.2 35.9 16.7 42 76 15 10.2 274 0.0 20.4 6.1

10-Oct-1999 19.6 27.1 14.3 80 98 49 15.5 231 16.6 13.4 5.1

11-Oct-1999 17.1 22.1 12.8 71 95 42 11.2 276 0.2 18.1 5.9

12-Oct-1999 18.4 26.1 12.0 66 96 30 10.7 119 0.0 25.9 4.8

13-Oct-1999 19.0 24.1 13.4 61 92 36 10.5 243 0.4 9.0 4.8

14-Oct-1999 20.2 27.4 15.3 76 94 47 15.3 107 0.2 18.7 5.8

15-Oct-1999 20.0 27.7 11.4 64 91 25 11.5 295 0.6 20.8 7.4

16-Oct-1999 16.1 25.6 6.6 58 95 22 5.8 175 0.0 27.0 7.2

17-Oct-1999 19.2 31.7 6.3 55 94 16 7.3 131 0.0 26.9 3.8

18-Oct-1999 13.5 16.1 10.6 84 95 73 10.7 241 7.8 4.4 5.5

19-Oct-1999 12.9 20.8 5.1 63 94 29 4.7 376 0.2 27.1 6.3

20-Oct-1999 13.1 21.5 4.3 65 98 30 5.3 218 0.0 26.6 7.3

21-Oct-1999 18.2 27.6 6.2 43 85 15 2.6 214 0.0 27.5 8.6

22-Oct-1999 14.6 20.7 8.1 50 88 21 3.0 294 0.0 26.7 5.9

23-Oct-1999 11.4 18.6 3.3 67 98 26 3.9 253 18.2 16.3 5.3

24-Oct-1999 14.4 20.8 9.3 70 97 36 8.3 365 14.8 20.6 5.3

25-Oct-1999 16.1 24.4 6.4 66 97 32 8.7 169 0.0 20.8 5.9

26-Oct-1999 16.3 22.8 9.1 64 95 24 8.0 235 1.0 23.5 6.9

27-Oct-1999 15.7 24.8 4.9 58 95 25 5.9 167 0.0 28.8 7.4

28-Oct-1999 17.0 25.5 8.2 58 91 22 6.8 165 0.0 29.0 7.9

29-Oct-1999 20.1 29.0 10.7 46 76 19 6.7 157 0.0 25.0 4.7

30-Oct-1999 19.5 24.0 16.8 63 87 45 12.0 168 1.0 7.3 7.3

31-Oct-1999 15.7 20.5 9.2 56 94 21 5.0 355 0.0 28.8 7.3

Monthly Summary

Number of days in the month: 31

Total 6705 98.2 663.5 188.9

Mean 17.1 24.6 9.4 64 93 32 8.6 216 21.4 6.1

Max 26.2 35.9 16.8 91 99 76 15.5 376 31.4 29.0 9.1

Min 11.4 16.1 3.3 42 76 15 2.6 107 0.0 4.4 1.6