Post on 19-Dec-2015
060730 Talk Outline
• Air Quality Information System Challenges (5min) – Real-time monitoring and data delivery (1 slide)– Characterization of pollutant in space/time/parameter (1 slide)– Agile (1 slide)
• Use Cases: (10min)– Canada Smoke: Realtime, Science, Regulatory, Public (7min)– Global Aerosol: Yearly average, Science, Policy (3min)
• AQ Infosystem for GEOSS: (10min)– Architecture (3slides)
• GEOSS - Regional Air Quality• Data. Services• Sharing/Harvesting Infrastructure• Intellectual Resources
– Engineering – DataFed design (3-4 slides/flash)– Technology – OGC Services, web services, web 2.0 (1slide)
• Discussion (5min)
• The data life cycle consists of the acquisition and the usage parts
Usage ActivitiesData Acquisition
Data Acquisition and Usage Activities(Select View Show, click to step through PPT)
• The acquisition part processes the sensory data by firmly linked procedures
The Federation focuses on data usage activities and presumes repositories
• The usage activities are more iterative, dynamic procedures
• The collected and cleaned data are stored in the repository
Data Repository
• The usage cycle transform data into knowledge for decision making
Decisions
The Network Effect:Less Cost, More Benefits through Data Multi-Use
ProgramPublic
Data Organization
Data
Data Program
Program
OrganizationData
Data
ProgramData
Orgs Develop Programs
Programs ask/get Data Public sets
up Orgs
Pay only once Richer content
Data Re-Use Network Effect
Data are costly resource – should be reused (recycled) for multiple applications
Data Reuse
Less Prog. Cost More Knowledge
Data reuse saves $$ to programs and allows richer knowledge creation
Less Soc. Cost More Soc. Benefit
Data reuse, like recycling takes some effort: labeling, organizing, distributing
VIEWS
AEROCOM
Federated Information System
Observations, Models, Services
CAPITA
Other Federations
Obs
ModelsObs &Services Services
Obs & Model
Models & Services
General data sharing and reuse can be accomplished through a federated approach. Data producers maintain their own workspace and resources (data, reports, comments). However, part of the resources are shared through a Federated Information System.
Model-Data Information System - Federation
ScientistScience
DAACs
• Current info systems are project/program oriented and provide end-to-end solutions
Info UsersData Providers Info System
AIRNowPublicAIRNow
ModelCompliance
Manager
Sample AQ Information Usage Landscape
• Part of the data resources of any project can be shared for re-use through DataFed
• Through the Federation, the data are homogenized into multi-dimensional cubes
• Data processing and rendering can then be performed through web services
• Each project/program can be augmented by Federation data and services
Providers
NASA DAACs
EPA R&DModel
EPA AIRNow
others
Public
Manager
Scientist
Users
other
• The info system transforms the data into info products for each user • In the first stage the heterogeneous data are prepared for uniform access
Uniform Access
Information Landscape: Info System Data Access, Processing and Products
• The second stage performs filtering, aggregation, fusion and other operations
Data Processing Web Service Chain
Custom Processing
SciFlo
DataFed
Info Products Reports, Websites
Forecasting
Compliance
Other
Sci. Reports
• The third stage prepares and delivers the needed info products
• The challenge is to design a general supportive infrastructure• Simply connecting the relevant provides and users for each info product is messy
Wrappers
Where?
What?
When?
Federate Data
Structuring
• Structuring the heterogeneous data into where-when-what ‘cubes’ simplifies the mess
Integrated Data System for Air Quality-IDAQ
ESIP AQ Cluster 050510 Draft rhusar@me.wustl.edu
• The info system infrastructure needs to facilitate the creation of info products
AQ Compliance
Nowcast/Forecast
Status & Trends
Find Data Gaps
ID New Problems
………
Info Needs
Reports
• Providers supply the ‘raw material’ (data and models) for ‘refined’ info products
EmissionSurface Satellite
Model
Single Datasets
Providers
Slice & Dice
Explore Data
Viewers
• The ‘cubed’ data can be accessed and explored by slicing-dicing tools
Programs
Integrate
Understand
• More elaborate data integration and fusion can be done by web service chaining• This infrastructure support for IDAQ can be provided by the ESIP Federation
Non-intrusive Linking & Mediation Data UsersData Providers
Decision Support System
Event Knowledge into the Minds of
EPA Analysts
Knowledge into the Minds of
State Analysts
DSS for Exceptional Event Decisions
Observations
Event Reports:Model Forecasts,
Obs. Evidence
Models
DecisionsEvent Knowledge into the Minds of
EPA Regulators
Decision Support System
Data Sharing
Std
. In
terf
ace
Data
Obs. & Models
Gen. Processing
Std
. In
terf
ace
ReportingDomain Processing
ControlReports
Loosely Coupled Data Access through Standard Protocols
Client request Capabilities
Server returns Capabilities and data ‘Profile’
Client requests data by ‘where, when, what’ query
Server returns data ‘cube’ in requested format
GetCapabilities
GetData
Capabilities, ‘Profile’
Data
Where? When? What? Which Format?
Server
Back End S
td.
Inte
rface
Client
Front EndS
td.
Inte
rface
Query GetData Standards
Where? BBOX OGC, ISO
When? Time OGC, ISO
What? Temperature CF
Format netCDF, HDF.. CF, EOS, OGC
T2T1
Domain ProcessingData Sharing
Std
. In
terf
ace
Gen. ProcessingS
td.
Inte
rface
Data
Control
Reports
Reporting
Obs. & Models Decision Support System
Web Services and Workflow for Loose Coupling
Service Broker
Service Provider
PublishFind
BindServiceUser
Web Service Interaction Service Chaining & Workflow
Domain ProcessingData Sharing
Std
. In
terf
ace
Gen. ProcessingS
td.
Inte
rface
Data
Control
Reports
Reporting
Obs. & Models Decision Support System
Web Services Triad:Publish – Find – Bind
Workflow Software:Dynamic Programming
Collaborative Reporting and Dynamic Delivery
Co Writing - Wiki
ScreenCast
Analysis Reports:
Information supplied by manyNeeds continuous program feedbackReport needs many authorsWiki technologies are for collaborative writing
Dynamic Delivery:
Much of the content is dynamicAnimated presentations are compellingMovies and screencasts are for dynamic delivery
Domain ProcessingData Sharing
Std
. In
terf
ace
Gen. ProcessingS
td.
Inte
rface
Data
Control
Reports
Reporting
Obs. & Models Decision Support System
Summary
• The current challenges for air quality information systems include delivery of air quality data in real time, characterization of air pollution through the integration of multi-sensory data and providing agile support to regulatory management. The talk describes the architecture and implementation of a standards based system for accessing and processing air quality data. The web services based architecture is illustrated through two use cases: (1) real time monitoring of a smoke event and (2) hemispheric transport of air pollutants.
“Core” Air Quality Information Network
• Consists of limited number of stable nodes• Provides core functionality• Members are eager network participants.• Well connected; value through compound services. • Network robustness arises from redundancy, practice,…
Candidate Nodes:
EPA: AIRS, AirNOW, VIEWS
NOAA: NCDC, HMS
NASA: OnEarth, INTEX Model
Other: Unidata
Federated Information System
• Data producers maintain their own workspace and resources (data, reports, comments). • However, part of the resources are shared through a Federated Information System.• Web-based integration of the shared resources can be across several dimensions:
Data sharing federations: • Open GIS Consortium (GIS data layers)• NASA SEEDS network (Satellite data)• NSF Digital Government • EPA’s National Env. Info Exch. Network.
VIEWSRPO
RPO Federated Data System
Data, Tools, Methods
SharedPrivate
RPO
Other Federations
Applications
PM Policy
Regulation
Mitigation
Unidata Portal
ESIP Portal
Portal
Data to be “dispersed” to multiple “portals”, each serve different clientele
Open architecture allows portals to reconfigure resource collections
User communities
Stages of AQ Data Flow and Value-Adding Processes
Domain ProcessingData Sharing
Std
. In
terf
ace
Gen. ProcessingS
td.
Inte
rface
Data
Control
Reports
Reporting
Obs. & Models Decision Support System
Analyzing
Filter/IntegrateAggregate/FuseCustom Analysis
Organizing
DocumentStructure/FormatBuild Interface
Characterizing
Display/BrowseCompare/Fuse Characterize
Valu
e-A
dd
ing
P
rocesses
Reporting
Inclusiveness Iterative/Agile Dynamic Report