Environmental Monitoring: Database and Beyond

Environmental Monitoring:Database and Beyond

Chengyang ZhangComputer Science Department

University of North Texas

Overview Data Transmission and Storage

Database Design DBMS Database Schema Design

Data Presentation Traditional Web Interface Virtual Earth / Google Map

Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)

Overview

Data Transmission and Storage




System Architecture1

Database Server

Internet

Client Data Browsingand Processing

Base Station Node

Wireless Sensor Nodes

GPRS Modem

Gateway (Single Board Computer)

GPRS Link

Data loggers

1 Jerry’s slide

Current Data Flow

City of Denton FTP

DB serverUV Machine

Greenbelt Station

PullCOD Daemon

PushUV Daemon

Wireless Data Daemon

DBChecker Daemon

Pull/Push

Raw Data Format COD FTP Data

Directory Structure: YEAR/MONTH/cod_date_time 2007/9/cod_2007-9-21_8.30

File Format: Station_Name/Gate_Name, date, time, rain_guage_reading/gate_status RG1, 9/21/2007, 8:30:57 AM, 0.00 HC_N_Gate_STATUS, 9/21/2007, 8:30:57 AM, UP

UV Data Directory Structure: YEAR/MONTH/uv_date_time.txt

2007/9/uv_2007-9-15_12.30 File Format: Date, Time, SUV #1, UVA #2, Temp 1, Temp2

“15.09.2007”, “12:30”, 1.367, 0.000, 32, 0

Raw Data Format(cont.) Wireless Data

Data Transmission Rate every hour (send 4 data packets) can be adjusted

Data Packet Format

Directory Structure: YEAR/MONTH/greenbelt_date_time File Format: Date, time, value #1, value #2, ……

datetime Sensor Data

4 byte 18 * 4 bytes or More




Data Exchange EML SensorML KML

DBMS Selection Free/Open Source v.s. Commercial Database

We choose Free!

PostgreSQL v.s. MySQL We choose PostgreSQL!

Data Integrity Check Server-side Programming Spatial Cartridge Support(PostGIS)




Data Exchange EML SensorML KML

Database Schema Design

Stations

PK StationID

StationName Location Coorinate GaugeType IPAddress Description

Sensors

PK SensorID

FK1 StationID SensorName Manaufacturer SensorType Pheneomenon Input Output Unit ValueRange CollectionFrequency Longitude Latitude CollectionDevice CollectionMethod

*

UVData

PK,FK1 SensorIDPK Date

SUV Temp Rawdata

*

Gate

DateFK1 SensorID

Status

* Rain

DateFK1 SensorID

Value rawdata*Clam

PK Date

clam validation rawdata

*

Humidity


Humidity Rawdata

*

Streams


StreamData rawdata

*

Web Interface (Powered by PHP)

Virtual Earth Interface

Google Map Interface (KML)

Multi-Layer System Structure*

Telemetry Flood Gates

Water Quality

Sondes Clams Water Quality

Other 11 Stations

Doyle and FreemanNorth Lakes

Pecan Creek WWTPSouth Lakes

Hickory Creek Life Station

Wired SensorsWeather

Soil Moistuire

Greenbelt ParkField Research Station

Wireless Sensor Network

Soil Moisture(200 motes)

Local SensorsUV

Local SensorsOzone

Local Data Logger

EESAT337

In situ DataData ServerEESAT 341

Field UNT Campus

L2 FTP

L3Wireless Modem

Data Collection Gateway

EESAT 341

L1Wireless Modem

LAN LAN Local GIS Data

Repository

Web ServiceEarth Science Data

Physical Data Layer

Logical Data Layer

Classic Web PortalWeb Presentation

Layer

User Layer

Sensor ML / O&M

K-12 Education Outreach

Download Data

(Real Time)

Download Data(non Real

Time)

AnalysisSynthesisModeling

WMSWFSWCS

OGC

Retrieve

Publish

Semantic Layer

Ontology

Semantic Service

Catalog Service

Semantic Web Portal

Semantic Level Query/Analysis

Wireless Sensor Network

Soil Moisture(10 motes)

SOSSASSPS

* From CRI proposal

Data Exchange Motivation

Retrieve data from Web-enabled Sensor Data Repositories Publish our sensor data as web-enabled service

Technology Behind the Scene XML Web Service

XML Derivatives

• KML (Keyhole Markup Language )• A markup language for displaying

geographical data in an earth browser*

• Example<kml><Placemark><name>City of Denton: Doyle and Freeman</name><description>Rain Gauge Reading: 0 inch</description><Point><coordinates>-97.132,33.214</coordinates></Point></Placemark></kml>

* http://code.google.com/apis/kml/documentation/index.html

XML Derivatives (Cont.)

• EML (Ecological Metadata Language)• A markup language developed by/for the

ecological discipline*

* http://knb.ecoinformatics.org/software/eml/

eml

packageId: UNTCSE.1.1

system: knb

dataset

title: Greenbelt Wireless Soil Moisture Dataset

creator

individualName

contact

surName:Chengyang

surName: Shu

individualName

XML Derivatives (cont.)

• GML (Geography Markup Language )*

• A markup language defined by the OGC to express geographical fatures

• Example<gml:Polygon><gml:outerBoundaryIs><gml:LinearRing

><gml:coordinates>0,0 100,0 100,100 0,100

0,0</gml:coordinates> </gml:LinearRing> </gml:outerBoundaryIs> </gml:Polygon> <gml:Point>

<gml:coordinates>100,200</gml:coordinates> </gml:Point><gml:LineString> <gml:coordinates>100,200 150,300</gml:coordinates>

</gml:LineString>

* http://en.wikipedia.org/wiki/Geography_Markup_Language

XML Derivatives (cont.)

• SensorML (Sensor Model Language )*

• An OGC standard o describe processes of sensor measurements and observations

• discover sensors• obtain sensor information • access sensor observations • receive alerts

* http://vast.uah.edu/SensorML/

SensorML (cont.)

• SensorML (Sensor Model Language )*

* http://vast.uah.edu/SensorML/

O&M

• O&M (Observations & Measurements)*

* www.buyya.com/papers/SensorWebChapter.pdf

Web Services*

Web Services – A program that can be executed through HTTP and XML SOAP (Simple Object Access Protocol)

Request / Response WSDL (Web Service Description Language)

Describes how to connect and query information UDDI (Universal Description Discovery and Integration)

Service Registration And Discovery

* http://en.wikibooks.org/wiki/XML:_Managing_Data_Exchange/Web_services

Web Services(cont.) SOAP Examples

Request<m:SoilMoistureDataRequest xmlns:SOAPSDK1="http://www.unt.edu/cri/xml/wsdl/SoilMositure.wsdl"> <latitude xsi:type="xsd:decimal">33.214</latitude> <longitude xsi:type="xsd:decimal“>-97.132</longitude><startDate xsi:type="xsd:date"></startDate> <numDays xsi:type="xsd:integer">1</numDays> <format>24 Hourly</format> </m:SoilMoistureDataRequest >

Response<SoilMoistureDataResponse xmlns:SOAPSDK1=" http://www.unt.edu/cri/xml/wsdl/SoilMositure.wsdl "> <dwmlByDayOut xsi:type="xsd:string">.....</dwmlByDayOut>

</ SoilMoistureDataResponse >

Web Services(cont.) WSDL Examples

<definitions xmlns:SOAP-ENV=“http://schemas.xmlsoap.org/soap/envelope/” …><message name="SoilMoistureDataRequest"> <part name="latitude" type="xsd:decimal"/> <part name="longitude" type="xsd:decimal" /> <part name="product" type="typens:productType" /> <part name="startTime" type="xsd:dateTime" /> <part name="endTime" type="xsd:dateTime" /> <part name="weatherParameters"

type="typens:weatherParametersType" /> </message> <message name=" SoilMoistureDataResponse "> <part name="dwmlOut" type="xsd:string" />

</message> <service name=“criSoilMoisture"> <port name=“criXMLPort" binding="tns:criXMLBinding"> <soap:address location=" http://www.unt.edu/cri/xml/SOAP_server/criserver.php" />

</port> </service> </definitions>




Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement (SWE)

Sensor Web Enablement*

Decision Support Tools

- vendor neutral- extensive

- flexible- adaptable

Heterogeneous sensor network

In-Situ monitors

Bio/Chem/RadDetectorsSurveillance

Airborne Satellite

- sparse- disparate

- mobile/in-situ- extensible

Models and Simulations

- nested- national, regional, urban- adaptable- data assimilation

Sensor Web Enablement- discovery- access- tasking- alert notification

web services and encodings based on

Open Standards(OGC, ISO, OASIS,

IEEE)

* vast.uah.edu/SensorML/presentations/SWE_SensorML_Botts.ppt

SWE Services SOS (Sensor Observation Services)

Formerly SCS fetch observations Response to queries

WNS (Web Notification Services) asynchronous notification of sensor events two-way

SPS (Sensor Planning Services)

SWE Deployment Database (Spatial Database Preferred) Intermediate Program(Middleware)

that converts sensor schema and readings to SensorML/O&M javabeans etc,

XML Parser SAX etc.

SOAP Toolkit Apache soap toolkit etc.

Thank you!

Environmental Monitoring: Database and Beyond

Documents

Transcript of Environmental Monitoring: Database and Beyond