Landscape Restoration Tool Using ArcMap Chris Kelley CE 394K Term Project, Fall 2009.
CUAHSI WaterOneFlow Web Services By Tim Whiteaker CE 394K.2 Hydrology 1 February 2007.
-
Upload
vernon-roberts -
Category
Documents
-
view
222 -
download
1
Transcript of CUAHSI WaterOneFlow Web Services By Tim Whiteaker CE 394K.2 Hydrology 1 February 2007.
CUAHSI WaterOneFlowWeb Services
By Tim Whiteaker
CE 394K.2 Hydrology
1 February 2007
Outline
• What are Web Services
• What is WaterOneFlow
• What is WaterML
• How do we use HIS Analyst to access WaterOneFlow
Programs Use Functions
• Functions are pieces of code that perform a specific task
Program
Mathlibrary
Add
Multiply
SquareRoot
Functions:
What’s the square root
of 16?
16
4
result = Math.SquareRoot(16)
Web Services Let Computers Share Functions
Program
Mathlibrary
Add
Multiply
SquareRoot
Functions:What’s the square root of 16
The answer is 4
• Local programs take advantage of remote resources
• Works just like local function
result = Service.SquareRoot(16)
ServiceClient
How Do Web Services Work
• SOAP (Simple Object Access Protocol) is a protocol for exchanging messages over a network
• WSDL (Web Services Description Language) is a language for describing what a web service can do
How Do Web Services Work
Client
Server
SOAP WSDL(This is what I
can do)
SOAP is like speaking the same languageWSDL is like a contract
CUAHSI Web Services (WaterOneFLow)
• Standard mechanism for flow of hydrologic data between hydrologic data servers (databases) and users.
• Provides data access to– USGS NWIS– EPA STORET– NCEP North American Model (NAM)– NASA MODIS– Daymet– NWS ASOS
http://water.sdsc.edu/waterOneFlow/
Water Data Web Sites
NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454
Time series of streamflow at a gaging station
Observation Stations
Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System
USGS National Water Information System NOAA Climate Reference Network
Map for the US
WaterOneFlow• Consistent Query Format • Consistent Data Format
WaterOneFlow Methods
• (Almost) the same for all networks
• Methods– GetSiteInfo – Give me info about a site– GetVariableInfo – Give me info about a
variable– GetValues – Give me a time series for a
variable at a site
GetSiteInfo
• Input– Network name and
site code– Authorization token
• Output– Site name– Site location– Variables measured at
the site• Variable code• Period of record
response = service.GetSiteInfo("NWIS:08158000", "")
where
GetVariableInfo
• Input– Network name and
variable code– Authorization token
• Output– Variable name– Variable description– Variable units
response = service.GetVariableInfo("NWIS:00060", "")
what
GetValues
• Input– Network name and
location (e.g., site code)
– Network name and variable code
– Start datetime– End datetime– Authorization token
• Output– Site info– Variable info– Time series
response = service.GetValues("NWIS:08158000", "NWIS:00060", _ "2006-12-31T01:00:00", "2006-12-31T08:00:00", "")
where what
when
The “Where” in GetValues
• Works for sites, or geographic locations• NetworkName:SiteCode
– NWIS:08158000
• GEOM:POINT(Longitude Latitude)– GEOM:POINT(-113 35)
• GEOM:BOX(WLon SLat,ELon NLat)– GEOM:BOX(-108 45,-107 46)
Locations
Variable Codes
Date Ranges
Extract – Transform – Load
GetSiteInfoGetVariableInfoGetValues
WaterOneFlowWeb Service
Client
STORET
NAMNWIS
DataRepositories
Data
DataData
EXTRACTTRANSFORMLOAD
WaterML
WaterML – The Output of WaterOneFlow
• Standardized for all data sources
• Formats– Object– XML (for programs that can’t handle objects)
• Defined in XML
XML – A Primer
• The Extensible Markup Language (XML) is a W3C-recommended general-purpose markup language that supports a wide variety of applications. – Wikipedia
• XML represents data• XML is both human and machine readable
<site> <name>Mansfield Dam</name></site>
XML Structure
<site></site>
<site></site>
<site />
OR
OR
element
start tag end tag
• Example – Let’s describe a streamflow site
element name = “site”
Elements can have Children
<site> <name>Mansfield Dam</name></site>
element
child element
element text
Elements can have Attributes
<site> <name>Mansfield Dam</name> <siteCode network=“NWIS”>08154510</siteCode></site>
attribute name attribute value
XML Nesting
<site> <name>Mansfield Dam</name> <siteCode network=“NWIS”>08154510</siteCode> <location> <latitude>30.39</latitude> <longitude>97.91</longitude> </location></site>
WaterML
• XML for describing water data
• Defined at
• WaterOneFlow returns data in WaterML format
http://water.sdsc.edu/waterOneFlow/documentation/schema/
Point Observations Information Model
Data Source
Network
{Value, Time, Qualifier}
USGS
Streamflow gages
Neuse River near Clayton, NC
Discharge, stage (Daily or instantaneous)
206 cfs, 13 August 2006
• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value
Sites
Variables
Values
http://www.cuahsi.org/his/webservices.html
WaterML Response
SiteInfoResponseType
VariablesResponseType
TimeSeriesResponseType
WaterML Key Elements
• Response Types
– SiteInfo
– Variables
– TimeSeries
• Key Elements– site– sourceInfo– seriesCatalog– variable– timeSeries
• values
– queryInfo
GetValues
GetVariableInfo
GetSiteInfo
SiteInfoResponseType
• Namespaces
• queryInfo
• site
Network
Sites
Variables
queryInfo
• Parameters sent to service
• URLs called (if external resource)
queryInfo – Another Example
userparameters
query URL
site
• siteInfo – info about the site
• seriesCatalog – catalogs of variables measured
siteInfo
• Name
• Site Code
• Location
geoLocation
• geogLocation – geographic coordinates– LatLon point– LatLon box
• localSiteXY – projected coorindates
European Petroleum Survey Groupi.e., NAD 83, etc.
seriesCatalog
name web service location
original data sourceseries (variables)
series
• variable – what is measured• valueCount – how many measurements• variableTimeInterval – when is it measured
variable
• variableCode – global identifier
• variableName
• units
Sites
Variables
Values
variableTimeInterval
• TimeIntervalType – date range
• TimeSingleType – single measurement
• TimePeriodRealTimeType – last n days
ISO Time
• International Organization for Standardization• 1998-03-01T14:30:00 =
March 1, 1998, at 2:30 in the afternoon
SiteInfo Response, RecapqueryInfo
site
name
code
location
seriesCatalog
variables
what
how many
when
VariablesResponseType
• variable – same as in series element
• Code, name, units Sites
Variables
Values
Additional variable Children
• variableDescription
• valueType (enumeration)– Field Observation– Sample– Model Simulation Result– Derived Value
• generalCategory – e.g., “water quality”
• sampleMedium – e.g., “water column”
-- Many are optional
TimeSeriesResponseType
• queryInfo
• timeSeries– sourceInfo – “where”– variable – “what”– values
Sites
Variables
Values
sourceInfo• SiteInfoType
– Same as siteInfo element– code, name, location
• DataSetInfoType– For data continuous in space– LatLonPointType– LatLonBoxType
variable
• Same as previous variable element
• code, name, units, etc.
values
• Each time series value recorded in value element
• Timestamp, plus metadata for the value, recorded in element’s attributes
ISO Time
valuequalifier
value Metadata Examples
• qualifiers• censorCode (lt, gt, nc)• qualityControlLevel (Raw, QC’d, etc.)• methodID• offset
– offsetValue– offsetUnitsAbbreviation– offsetDescription– offsetUnitsCode
TimeSeries Response, RecapqueryInfo
location
variable
values
Conclusions about WaterML
• Consistent Format
• Includes lots of information
• How do I use it?
HIS Analyst
HIS Analyst
• Desktop apps that use WaterOneFlow
• Examples for various application environments
• HydroObjects – for apps that aren’t web service enabled
• Workbook available at:
http://www.cuahsi.org/his/docs/HIS-workbook-20061130.pdf
HIS Workbook
• Ingesting NWIS Data into Excel
• Ingesting STORET Data into Excel
• Ingesting Weather and Streamflow Data into ArcGIS
• Plotting MODIS Data with Matlab
• Ingesting NWIS Data using VB.Net
• Ingesting NWIS Data Using Java
Excel Demo
Weather Downloader
Historical Weather Data
Forecasted Weather Data
Streamflow
Other
Recap
• Computers share functionality through Web Services
HIS Analyst
Client WaterOneFlowWeb Service
GetSiteInfoGetVariableInfoGetValues
WaterMLTime SeriesRepository