060730 Talk Outline

• Air Quality Information System Challenges (5min) – Real-time monitoring and data delivery (1 slide)– Characterization of pollutant in space/time/parameter (1 slide)– Agile (1 slide)

• Use Cases: (10min)– Canada Smoke: Realtime, Science, Regulatory, Public (7min)– Global Aerosol: Yearly average, Science, Policy (3min)

• AQ Infosystem for GEOSS: (10min)– Architecture (3slides)

• GEOSS - Regional Air Quality• Data. Services• Sharing/Harvesting Infrastructure• Intellectual Resources

– Engineering – DataFed design (3-4 slides/flash)– Technology – OGC Services, web services, web 2.0 (1slide)

• Discussion (5min)

• The data life cycle consists of the acquisition and the usage parts

Usage ActivitiesData Acquisition

Data Acquisition and Usage Activities(Select View Show, click to step through PPT)

• The acquisition part processes the sensory data by firmly linked procedures

The Federation focuses on data usage activities and presumes repositories

• The usage activities are more iterative, dynamic procedures

• The collected and cleaned data are stored in the repository

Data Repository

• The usage cycle transform data into knowledge for decision making

Decisions

The Network Effect:Less Cost, More Benefits through Data Multi-Use

ProgramPublic

Data Organization

Data

Data Program

Program

OrganizationData

Data

ProgramData

Orgs Develop Programs

Programs ask/get Data Public sets

up Orgs

Pay only once Richer content

Data Re-Use Network Effect

Data are costly resource – should be reused (recycled) for multiple applications

Data Reuse

Less Prog. Cost More Knowledge

Data reuse saves $$ to programs and allows richer knowledge creation

Less Soc. Cost More Soc. Benefit

Data reuse, like recycling takes some effort: labeling, organizing, distributing

VIEWS

AEROCOM

Federated Information System

Observations, Models, Services

CAPITA

Other Federations

Obs

ModelsObs &Services Services

Obs & Model

Models & Services

General data sharing and reuse can be accomplished through a federated approach. Data producers maintain their own workspace and resources (data, reports, comments). However, part of the resources are shared through a Federated Information System.

Model-Data Information System - Federation

ScientistScience

DAACs

• Current info systems are project/program oriented and provide end-to-end solutions

Info UsersData Providers Info System

AIRNowPublicAIRNow

ModelCompliance

Manager

Sample AQ Information Usage Landscape

• Part of the data resources of any project can be shared for re-use through DataFed

• Through the Federation, the data are homogenized into multi-dimensional cubes

• Data processing and rendering can then be performed through web services

• Each project/program can be augmented by Federation data and services

Providers

NASA DAACs

EPA R&DModel

EPA AIRNow

others

Public

Manager

Scientist

Users

other

• The info system transforms the data into info products for each user • In the first stage the heterogeneous data are prepared for uniform access

Uniform Access

Information Landscape: Info System Data Access, Processing and Products

• The second stage performs filtering, aggregation, fusion and other operations

Data Processing Web Service Chain

Custom Processing

SciFlo

DataFed

Info Products Reports, Websites

Forecasting

Compliance

Other

Sci. Reports

• The third stage prepares and delivers the needed info products

• The challenge is to design a general supportive infrastructure• Simply connecting the relevant provides and users for each info product is messy

Wrappers

Where?

What?

When?

Federate Data

Structuring

• Structuring the heterogeneous data into where-when-what ‘cubes’ simplifies the mess

Integrated Data System for Air Quality-IDAQ

ESIP AQ Cluster 050510 Draft rhusar@me.wustl.edu

• The info system infrastructure needs to facilitate the creation of info products

AQ Compliance

Nowcast/Forecast

Status & Trends

Find Data Gaps

ID New Problems

………

Info Needs

Reports

• Providers supply the ‘raw material’ (data and models) for ‘refined’ info products

EmissionSurface Satellite

Model

Single Datasets

Providers

Slice & Dice

Explore Data

Viewers

• The ‘cubed’ data can be accessed and explored by slicing-dicing tools

Programs

Integrate

Understand

• More elaborate data integration and fusion can be done by web service chaining• This infrastructure support for IDAQ can be provided by the ESIP Federation

Non-intrusive Linking & Mediation Data UsersData Providers

Decision Support System

Event Knowledge into the Minds of

EPA Analysts

Knowledge into the Minds of

State Analysts

DSS for Exceptional Event Decisions

Observations

Event Reports:Model Forecasts,

Obs. Evidence

Models

DecisionsEvent Knowledge into the Minds of

EPA Regulators

Decision Support System

Data Sharing

Std

. In

terf

ace

Data

Obs. & Models

Gen. Processing

Std

. In

terf

ace

ReportingDomain Processing

ControlReports

Loosely Coupled Data Access through Standard Protocols

Client request Capabilities

Server returns Capabilities and data ‘Profile’

Client requests data by ‘where, when, what’ query

Server returns data ‘cube’ in requested format

GetCapabilities

GetData

Capabilities, ‘Profile’

Data

Where? When? What? Which Format?

Server

Back End S

td.

Inte

rface

Client

Front EndS

td.

Inte

rface

Query GetData Standards

Where? BBOX OGC, ISO

When? Time OGC, ISO

What? Temperature CF

Format netCDF, HDF.. CF, EOS, OGC

T2T1

Domain ProcessingData Sharing

Std

. In

terf

ace

Gen. ProcessingS

td.

Inte

rface

Data

Control

Reports

Reporting

Obs. & Models Decision Support System

Web Services and Workflow for Loose Coupling

Service Broker

Service Provider

PublishFind

BindServiceUser

Web Service Interaction Service Chaining & Workflow

Domain ProcessingData Sharing

Std

. In

terf

ace

Gen. ProcessingS

td.

Inte

rface

Data

Control

Reports

Reporting

Obs. & Models Decision Support System

Web Services Triad:Publish – Find – Bind

Workflow Software:Dynamic Programming

Collaborative Reporting and Dynamic Delivery

Co Writing - Wiki

ScreenCast

Analysis Reports:

Information supplied by manyNeeds continuous program feedbackReport needs many authorsWiki technologies are for collaborative writing

Dynamic Delivery:

Much of the content is dynamicAnimated presentations are compellingMovies and screencasts are for dynamic delivery

Domain ProcessingData Sharing

Std

. In

terf

ace

Gen. ProcessingS

td.

Inte

rface

Data

Control

Reports

Reporting

Obs. & Models Decision Support System

Summary

• The current challenges for air quality information systems include delivery of air quality data in real time, characterization of air pollution through the integration of multi-sensory data and providing agile support to regulatory management. The talk describes the architecture and implementation of a standards based system for accessing and processing air quality data. The web services based architecture is illustrated through two use cases: (1) real time monitoring of a smoke event and (2) hemispheric transport of air pollutants.

“Core” Air Quality Information Network

• Consists of limited number of stable nodes• Provides core functionality• Members are eager network participants.• Well connected; value through compound services. • Network robustness arises from redundancy, practice,…

Candidate Nodes:

EPA: AIRS, AirNOW, VIEWS

NOAA: NCDC, HMS

NASA: OnEarth, INTEX Model

Other: Unidata

Federated Information System

• Data producers maintain their own workspace and resources (data, reports, comments). • However, part of the resources are shared through a Federated Information System.• Web-based integration of the shared resources can be across several dimensions:

Data sharing federations: • Open GIS Consortium (GIS data layers)• NASA SEEDS network (Satellite data)• NSF Digital Government • EPA’s National Env. Info Exch. Network.

VIEWSRPO

RPO Federated Data System

Data, Tools, Methods

SharedPrivate

RPO

Other Federations

Applications

PM Policy

Regulation

Mitigation

Unidata Portal

ESIP Portal

Portal

Data to be “dispersed” to multiple “portals”, each serve different clientele

Open architecture allows portals to reconfigure resource collections

User communities

Stages of AQ Data Flow and Value-Adding Processes

Domain ProcessingData Sharing

Std

. In

terf

ace

Gen. ProcessingS

td.

Inte

rface

Data

Control

Reports

Reporting

Obs. & Models Decision Support System

Analyzing

Filter/IntegrateAggregate/FuseCustom Analysis

Organizing

DocumentStructure/FormatBuild Interface

Characterizing

Display/BrowseCompare/Fuse Characterize

Valu

e-A

dd

ing

P

rocesses

Reporting

Inclusiveness Iterative/Agile Dynamic Report