Air Quality Community Experiences and Perspectives on International Interoperability Standards

IGARSS, 30 July 2010

Honolulu,HI

Presented by Erin Robinson

Erin Robinson, Stefan Falke, Rudolf Husar, David McCabe, Frank Lindsay, Chris Lynnes, Greg Leptoukh, Beate Hildenbrand, Oleg Goussev, Peter Sommer

Erin Robinson, Stefan Falke, Rudolf Husar, David McCabe, Frank Lindsay, Chris Lynnes, Greg Leptoukh, Beate Hildenbrand, Oleg Goussev, Peter Sommer

Air quality community spans efforts

• Monitor the atmosphere (surface, satellite and aerial systems),

• Model future atmospheric behavior and air quality conditions,

• Measure and estimate emissions of pollution and pollution forming gases and particles,

• Combine those information with socio-economic data for decision making analyses.

Informing the Public

Enforcing Standards

Hemispheric Transport

Real-time Service

Regulatory Analysis

Policy Assessment

Atmospheric Composition

Science & Education

Interoperability Efforts• GALEON• ESIP• GEOSS AIP• CyAir• HTAP/AMEN Network

Portals / Catalogs

NASA Programs/Projects

EPA Programs/Projects

NOAA Programs/Projects

• Air Quality Forecasting • NESDIS• NGDC • Hazard Mapping System• IDEA and IDEA-I

• WRAP • EDAC • LAITS• Giovanni• 3D-AQS• RS for BlueskyRAINS • Aura in AQ Forecasting• VIEWS/TSS • SAMITS • Sensor Web Architecture & Demo (Mandl)• DAACS

• AMI• AirNOW• AQS• OAQPS• HTAP • Remote Sensing Gateway• Environmental Science Connector

• DataFed • Unidata • Giovanni• LAITS• RSG• CIERA/NEISGEI• VIEWS • RSIG

Forest Service Programs/Projects• Bluesky

The Air Quality Web Landscape (not comprehensive – from a US perspective)

Mediators

• GEO Portals• Earth Observation Portal (GEO)• Geospatial One Stop• Environmental Science Connector (EPA)• Global Change Master Directory (GCMD)• ECHO (NASA)• LEAD (NSF)• AC Portal (CEOS/WGISS)

International• GMES• MACC• CEOS ACC

State• Aura in AQ Forecasting • RPOs• Vermont

Traditional Project Approach

DataDo something with the data (add value)

GenerateProduct Deliver Product

to Customer

Closed System

Interoperability Challenges

• Standalone-project-oriented approaches• Multiple interoperability standard implementations• Required tailored implementation of standards

“The user cannot find the data;

If he can find it, cannot access it;

If he can access it, ;

he doesn't know how good they are;

if he finds them good, he can not merge them with other data”

The Users View of IT, NAS 1989

Air Quality & Health Applications

Making Project Contributions Reusable

DataDo something with the data (add value)

GenerateProduct

Deliver Product to Customer

Provide Website Access (manual)

Provide web serviceaccess

Provide access to processing/analysis/value-addingservices

7

Project A combines multiple data sources to generate near-real time information for the public

Project A provides web service interfaces to some of its data and information

Project A

Project B analyzes surface and satellite data in support of regulatory analysis and provides web services to their analysis output and analysis tools

Project B

New Project

A new project uses services from projects A, B and C to meet its objectives

SOA Actions

UserProvider

Broker

The data reuse is possible through the service oriented architecture of GEOSS.

Publish–

ISO 19115,19119 Find – OpenSearch

BindOGC W*S

Community/Provider Catalogs

Metadata

ISO 19115/19119 subset for Geospatial Data

Data Providers

Data Access Service

Data Protocol

WMS, WCS (netDCF CF) + conventions

GEOSS CSR

Registration Specifies:• Where the records are located• Metadata standard• Harvest periodicity

Data Access Service

GEOSS Clearinghouse harvests metadata from distributed catalogs

OGC CSW Queryable

ISO 19115CSW Profile

OGC CSW Returnable

MetadataDescription

Data Binding

Air Quality Community Record

Com Client

Data User

OGC CSW Queryable

Air QualitySpecific

ISO 19115CSW Profile

OGC CSW Returnable

MetadataDescription

Data Binding

Air Quality Community Record

Com Client

Data User

OGC CSW Queryable

Air QualitySpecific

ISO 19115CSW Profile

OGC CSW Returnable

MetadataDescription

Data Binding

Air Quality Community Record

GetCapabilities

GetData

Capabilities, ‘Profile’

Data

Where? When? What? Which

Format?

Server

Back End S

td.

Inte

rface

Client

Front EndS

td.

Inte

rface

Clients can access any HTAP model ‘datasets’ for browsing, comparison, statistical…processing

YOUR client software can also access any HTAP model ‘datasets’Perform your types of analyses

The data the tools and the methods are shared to the Community through the AQ CoP So, participants of this project automatically become members of AQ CoP

HTAP Data Network

•FZ Juelich server offers 1000+ HTAP model ‘datasets’•WCS–netCDF–CF Standard Protocol•FZ Juelich & WashU Co-develop connectivity software

DataFed Giovanni

Objectives

• Provide access, tools, and contextual guidance to scientists and value-adding organizations in using remotely sensed atmospheric composition data, information, and services

• Help foster interoperability and application of atmospheric composition data, information and services worldwide

• Start with DLR and NASA, work with partners in CEOS and the broader AC community in advancing the AC Portal

CEOS Atmospheric Composition

Portal

19

GEO Task AR-09-02b

DLR

Giovanni

Datafed

Others

OG

C W

MS

OG

C W

CS

Beta version of the ACP by September 2010

OPeN

DA

P

DLR

Giovanni

Datafed

Others

DLR

Giovanni

Datafed

Others

Data AccessGateways

VisualizationTools

Processing/AnalysisTools

Initial focus on air quality but plan is to server broader community (climate, strat. O3)

GEOSS

User Contributed ContentUsers

Contextualmetadata

Put in back-up

CEOS Atmospheric Composition Portal

20

Analyze difference

GEO AQ-CoPMain interaction through the open, participatory website: http://geo-aq-cop.org

Summary and Issues

• The air quality interoperability efforts have found some success in using existing web standards to achieve a base level of interoperability

• Many challenges remain spanning technical issues in making standards work for air quality uses to organization issues in achieving agreement among air quality-specific conventions to standards

• As the community gains experience in how to implement and tailor the standards they will develop best practices

• The interoperability efforts described in this abstract have focused on OGC standards. However, there is a desire to evaluate the applicability of other web standards for air quality systems, in particular OPeNDAP.  

Acknowledgements

This work is not done alone, acknowledge all of the people that contribute to this effort:

• CEOS and Atmospheric Composition Portal• HTAP Network• CyAir• ESIP Air Quality Workgroup• GEOSS Architecture Implementation Pilots • GEO AQ Community of Practice

Backup Slides

Users

Data Providers

GEOSSCommon

Infrastructure

Air Quality Community Catalog

General search and use of GEOSS data and services

AQ-specific search and use of GEOSS data and services

Air Quality Community Infrastructure

SEDACDataFed

Giovanni

EDAC

NGCESA

Data web service metadata registrations

AIP-2 Workspace: https://sites.google.com/site/geosspilot2/air-quality-and-health-working-group 25

Next Steps:

• Continue to work with AQ community to improve the AQ-specific sections needed for sharper discovery

• Work with AQ data providers to register their services in AQ Community Catalogs

• Extend metadata with unstructured user-contributed content through DataSpaces

• Use the GEOSS AQ CoP to help coordinate activities

Air Quality Community Interoperability Efforts(not comprehensive)

• GEO AQ CoP: Working to connect various AQ networking activities

• GEOSS AIP-3: The GEOSS Architecture Implementation Pilot (AIP) develops and deploys new process and infrastructure components for the GCI and the broader GEOSS architecture

• HTAP: The HTAP network aims to facilitate seamless access to distributed HTAP model data, easy connectivity of data processing components through standard interfaces, and provide a set of basic tools for data processing, integration, and comparisons.

• CEOS AC Portal: The AC Portal is being developed to provide access, tools, and contextual guidance to scientists and value-adding organizations in using remotely sensed atmospheric composition data

• CyAir: Contributions to the planning, development, maintenance and coordination of systems to help the air quality community better utilize air quality related information

A joint NASA-NOAA-EPA initiative:

Objective - Prototype a near-real-time MODIS trajectory forecast product using MODIS direct broadcast aerosol optical depth over CONUS.

Goal - Improve accuracy of next day PM2.5 Air Quality Index (AQI) by providing pseudo-synoptic aerosol observations and trajectory forecast during large aerosol events.

• Provides forecast guidance for surface air quality (PM2.5)

Dust stormsSmoke from forest firesUrban/industrial haze

• System requirementsDirect broadcast receiving station for satellite dataTrajectory modelRegional meteorological forecast model outputAlgorithms to derive aerosol optical depth or related

parametersSurface mass aerosol monitors

• System flexibilityEasily adaptable to other regions of the globeEasy to modify to input other satellite data

Benefit - Enable improved mitigation of health effects caused by episodes of poor air quality

Infusing satellite Data into Environmental Applications (IDEA)

Al-Saadi et. al., September 2005, Bulletin of American Meteorological Societyhttp://www.star.nesdis.noaa.gov/smcd/spb/aq/ 28

Juelich ClientClent

GiovanniClient

31

Giovanni Instances

Giovanni Instances

Giovanni

AB

D

E CF

C

DE

BA C

F

Area Plot Time Series Model Output

Profile Cross-Section Correlations Column Densities

CALIOP CALIPSO

CloudSat

MODIS Terra

SeaWiFS

TRMM

HALOE UARS

TOMS EP, N7

Data InputsData Inputs

MLS Aura

OMI Aura

AMSR-E Aqua

MISR Terra

MODIS Aqua

AIRS Aqua

MERRA Models

http://disc.sci.gsfc.nasa.gov/giovanni

Slide provided by G. Leptoukh

SOA for Data Access & Processing

Mediator for 100+ AQ Datasets

Includes Tools for Analysis

Applied to EPA EE Analysis

Supported by NSF, NASA, RPOs, EPA

Design ConceptIEEE Systems Journal, 2008

32

Slide provided by R. Husarhttp://datafedwiki.wustl.edu/

DataFedClientServer

Atmospheric Composition Portal

Objectives

• Provide access, tools, and contextual guidance to scientists and value-adding organizations in using remotely sensed atmospheric composition data, information, and services

• Help foster interoperability and application of atmospheric composition data, information and services worldwide

• Start with DLR and NASA, work with partners in CEOS and the broader AC community in advancing the AC Portal

34

GEO Task AR-09-02b

DLR

Giovanni

Datafed

Others

OG

C W

MS

OG

C W

CS

Plan is to have a beta version of the ACP by September 2010

OPeN

DA

P

DLR

Giovanni

Datafed

Others

DLR

Giovanni

Datafed

Others

Data AccessGateways

VisualizationTools

Processing/AnalysisTools

Initial focus on air quality but plan is to server broader community (climate, strat. O3)

GEOSS

User Contributed ContentUsers

Contextualmetadata

Put in back-up

CEOS Atmospheric Composition Portal

35

Analyze difference

Hurdles

“The user cannot find the data;

If he can find it, cannot access it;

If he can access it, ;

he doesn't know how good they are;

if he finds them good, he can not merge them with other data”

The Users View of IT, NAS 1989

Remote Sensing Information Gateway

http://www.epa.gov/rsig

EPA CMAQConcentrations, Met, Emissions, DepositionEPA Statically Fused Air and Deposition Surfaces

O3, PM2.5NOAA GOES Biomass Burning Emissions; PM2.5, NOx, CO, …

- Subsetting-Regridding-Comparison-Export in common formats-Fast!

-Atmospheric Science Researchers-Air Quality Modelers and Analyst

NASA MODIS L2 AODNASA CALIPSO L1b TABNOAA GOES AOD (GASP)(Future data)NASA CALIPSO L2 Bext , AODNASA MOPITT L2 CONASA OMI L2 NO2NOAA/NASA GOME-2 NO2

Satellite:

EPA AIRNow; O3, PM2.5EPA AQS; O3, PM2.5EPA UVNet; Irradiance

Site Obs/Point Data:

Modeled:

RSIG System

Primary Users

-Air Quality Regulators and Managers-(RSIG is publicly-accessible.)

Secondary Users

Atmospheric Chemistry Earth Observations and

ModelsData Services

AQ Data and Image Information Products

RSIG WCS/WMS ServersAvailable for Other Systems

37Slide provided by J. Syzkman

Put in back-up

Source: G. Leptoukh, NASA Goddard

HTAP Data Network

Juelich – M. Schultz

DataFed – R. Husar

Giovanni – G. Leptoukh

• To Support Hemispheric AP Transport Program• To Make a contribution to GEOS

Other AQ CoP Ativities

AQ CoP Participation in GEO Tasks• US-09-01b: Development of Communities of Practice• US-09-01a: Identifying User Requirements for Earth Obs. AR-09-01b: GEO Architecture Implementation Pilot (AIP)•DA-09-02d: Atmospheric Model Evaluation Network•…..

Meeting & Coordination Activities, 2009-10

May 2009: Session on AQ and GEOSS, ISRSE, Stresa, Italy (Husar)Jul 2009: AQ Community Infrastructure, ESIP, Santa Barbara, CA, US (McCabe)Nov 2009: AQ Side Meeting at GEO-VI Plenary, Wash. DC, US (McCabe, Keating)May 2010: GEO Decision Support Concept Proposal (Falke)June 2010: AQ Participation in the GEOSS AIP 2,3. (Falke, Robinson) Monthly: Telecons, Wiki workspace (Falke, Robinson)

AR-09-01b: GEO Architecture Implementation Pilot (AIP)Testing and Using the GEOSS Common Infrastructure (GCI)

Data Search Facets

Dataset Parameter Instr. Platform

Space Time

Sharing is facilitated by the GEOSS Common Infrastructure

Catalog

of

AQ Data

though the

GEOSS Clearinghouse

GEO AQ-CoPMain interaction through the open, participatory website:

• CoP People• Member Projects• DataSpaces

GEO AQ CoP Active ParticipantsTechnologists:

Michael Decker, FZ Julich; Oleg Goussev (DLR); Kari Hoijarvi, WUSTL; Erin Robinson, WUSTL.. and others.

Facilitators:Stefan Falke, NGC; Dave McCabe, EPA/CAC); Frank Lindsey, NASA;

Rudy Husar, WUSTL; Martin Shulz, FZ Juelich .. and others.

Agency Executives: Gary Foley, EPA; Terry Keating, EPA; Lawrence Friedl, NASA .. and

others.

Point of Contact: Rudolf Husar (rhusar@wustl.edu)

Emerging HTAP Data Network

Outline

• Define the infrastructure that is emerging in order to allow data sharing and reuse to occur

• Identify the standards and conventions we are creating and the issues we’ve encountered

• Describe a few of the AQ projects/programs and their international scope

HTAP Data Network

• Stakeholders are autonomous organizations, scientists• Participation is largely voluntary• Information integration is key to success

HTAP & GEOSS Symbiosis

GEOSS is a good (system of systems) model for HTAP

HTAP is a suitable use case for the emerging GEOSS

HTAP Model Data Server Project: Connecting FZ Juelich server to WashU DataFed client