Where does VIPER get its data from? Tom Pagano [email protected] 503-414-3010.

Where does VIPER get its data from? Tom Pagano

[email protected] 503-414-3010

What data do we need?

VIPER’s input and storage of data

Old and new databasesAWDBWYFORCDBSSNOTEL

Real-time data schedule

Local file import

What data do we need?

MetadataStation lists, lat/lon/elev

Historical time series dataReal-time time series data

Snowpack, Precipitation, etc

Forecast equations, configurations

VIPER Workbook

StationMetadata

HistoricalTime-Series

1 station POR

Real-timeData

All stations2 years

Equations/Configurations

VIPER Workbook

StationMetadata


1 station POR

Real-timeData

All stations2 years


Metadata

Data

AWDB

VIPER Workbook

StationMetadata


1 station POR

Real-timeData

All stations2 years


USGS webpage

Metadata

Data

AWDB

Old time series databases

Old databases and new databases

CDBS•“Final” data• Period of record ~1 year out of date•Daily, monthly•Snow, Precip, Temp Resv, Strm




SNOTEL•Provisional•Recent 2 years•Hourly (and daily)•All snotel elements•Archive kept (since ?)





WYFOR•Ephemeral•Recent 2 years•Monthly•Snow, Prec, Strm Resv, “Misc”•Archive kept (since ‘93)





WYFOR•Ephemeral•Recent 2 years•Monthly•Snow, Prec, Strm Resv, “Misc”•Archive kept (since ‘93)

AWDB•Provisional and final mixed•All years•All durations•All elements*

New time series database

*Planned

NOTE: AWDB load strategy being reworked so rest of slides may not apply in near future!

Example- where do historical monthly snow come from?

1960

1970

1980

1990

2000

Back estimated snowcourse

Site automated (aug 1980)

Brumley Snotel

Example- where do historical monthly snow come from?

1960

1970

1980

1990

2000

CDBS

1x load of cdbs… updated if Dana notified of changes

1x load daily data in off season into monthly slots (snotel.. Or daily cdbs?)

SNOTEL

Historical streamflow data

1960

1970

1980

1990

2000

CDBS

Wyfor “under” CDBS

Historical streamflow data

1960

1970

1980

1990

2000

1x load of old wyfor files since mid 1990’s

Then on top of that…

1x load of cdbs… updated if Dana notified of changes

The realtime data schedule and synchronization

AWDBdatabase

Wyfor SNOTEL

4x a day, complete load (1:20am, 5:20am, 11:20am, 10:20pm)


AWDBdatabase

Wyfor SNOTEL

Any DMP edits to snotel loadedevery 5 minutes to wyfor/awdb

DMPEdits


AWDBdatabase

Wyfor SNOTEL

Spreadsheet files loadedevery 10 minutes all day

to wyfor/awdb

CSV spreadsheet

files (non-snotel data)

CSVprocessor


AWDBdatabase

Wyfor SNOTEL

Direct edits of wyfor reach wyfor right awayBut reach AWDB 50 minutes after the hour

5-9am, 11am-4pm and 11pm. Worst method of entering data for viper.

Manual edits

If you have custom datasets, Viper can also access local CSV format files.

Same layout as individual station data sheets (e.g. data1, data2) Realtime data gets manually entered in “ExternalData” sheet.

OMG Viper is so slow!!!!

Viper is a network-hungry application.Within the USDA backbone, your connection should be very fast.Outside the backbone, speed varies from tolerable to disgruntling.

OMG Viper is so slow!!!!

Viper is a network-hungry application.Within the USDA backbone, your connection should be very fast.Outside the backbone, speed varies from tolerable to disgruntling.

The way viper connects to the database is the same way that users contact the new update report…. which can mean:

Where does VIPER get its data from? Tom Pagano [email protected] 503-414-3010.

Documents

Transcript of Where does VIPER get its data from? Tom Pagano [email protected] 503-414-3010.