DataFed and FASTNET Tools for Agile Air Quality Analysis

Husar & Poirot

FASTNET Applications

Actual:• Working tools for RPOs for characterizing haze events, particularly from natural sources

• Test bed for new Web-Services data integration and analysis technologies (NSF, NASA)

• Case Study of ESIP for enabling AQ analysts for multiple agencies with modern IT

Potential:• Model – Data Integration – Retrospective and real time

• Quantification of satellite data for AQ application

• Regulatory support for PM25 compliance – weight of evidence approach

Intro - Management• Environmental management is a closed-loop process that includes sensing the environment, detecting possible hazards, identifying the causes,

evaluating the mitigation options and acting to reduce the hazard. The approach to the environmental management has been has been changing recently in several important ways.

         The command and control management style is giving way to broader participatory approach involving the key stakeholders throughout the management process.

         The environmental decision makers have become interested in monitoring and evaluating the effectiveness of past environmental control actions.

         The conectedness of air, water and land pollution is recognized by multimedia approach to the issue of permits.•  • In the recent past, the mitigation of environmental problems has been accomplished primarily by laws and regulations that were prepared and

enforced by governmental agencies. Other stakeholders, such as industry and public interest groups have participated in the management process indirectly by influencing the relevant governmental agencies. Conflict resolution among the stakeholders occurred primarily through the political mechanisms in the Congress where the laws created and through litigation with the regulatory agency, EPA. Once, the laws and regulations were in place, they constituted a command that had to be executed by those who performed the environmental control action. Thus the term command and control approach.

• Meagatrends NAAMS, Haze rule

• Major transformations in the nature of air pollution and its management have occurred over recent decades that effected the nature of AQ management. Health and environmental health studies implicate ozone and fine particulates as two of the most serious current air quality problems in North America. Both are secondary pollutants formed in the atmosphere from precursor gases. Hence, there is no direct way of linking their sources to their effects. The atmospheric lifetime of O3 and PM2.5 is several days so the winds carry them over 1,000 km before removal. The result is “long-range transport” (LRT) across state and international boundaries, i.e. in/out of the jurisdiction of AQ agencies. Finally, the old command and control air quality management style is giving way to a more participatory approach by including stakeholders, encouraging market-based resource allocation and applying ‘weight of evidence’ criteria (scientific arguments) to compliance management.

• These developments have significant implications on the information systems that support air quality management decisions. The information system must deliver relevant information to a broad range of stakeholders (federal, state, local, industry, international). The range of data needed for analysis and interpretation now is much richer including high resolution satellite data on PM concentrations, emissions, meteorology, and effects. Furthermore, the type of data, the level of aggregation, filtering, and the frequency at which sensory data are provided to the air quality management system differs greatly whether it is applied to policy, regulatory or operational decisions.

• Fortunately, outstanding opportunities exist now to fulfill most of these information requirements for PM management. Currently, abundant high-grade routine monitoring data are available from both surface as well as satellite platforms. New information technologies are now capable, in principle, to deliver the right information to the right place and time for decision-making. Last but not least, the stakeholders are increasingly committed to the cooperative spirit and recognize the needs and the benefits of resource sharing through collaboration.

• This project is a contribution to realize the opportunities in improved air quality management through a more effective use of existing monitoring data. The specific application area chosen for the proposed work is particulate air quality management. We will work with the US EPA, States, and NASA to simplify and integrate the use of ESE data in air quality management decisions making and enhance the availability and interoperability of ESE data with other air quality relevant data.

Natural Aerosols

• Why natural aerosols

• Messy but it characterized – self describing

• Info system to bring it together

Current Data ConnectivityCurrently, most of the data used in the PM management process are housed in the central AIRS database, populated by mandated data submissions by the states. AIRS has provisions for ‘pre-packaged’ reports, mostly reporting summaries pertinent to regulatory process, but it does not have facilities to perform detailed specialized analysis of the raw data. For this reason, past projects at the Center for Air Pollution Impact and Trend Analysis (CAPITA) at Washington University with the EPA provided access to the raw AIRS data, converted data into a uniform “Voyager” format, and distribute to states and other agencies. Similar data access and delivery systems were built by CAPITA for individual states, including California.AirNOW is the EPA’s database for collecting near real time air quality monitoring data from the states. It is currently centralized and focused on ozone and particulate matter (PM) but contains limited contextual information because it is not linked to other systems such as satellite data, weather or model forecasts. The addition of satellite imagery could provide valuable context for the interpretation of the surface air quality data in AirNOW.In 1999, the U.S. Environmental Protection Agency announced a major initiative to improve visibility in national parks and wilderness areas by reducing regional haze. In support of the resulting Regional Haze Rule, the EPA and five Regional Planning Organizations (RPOs) have established the Visibility Information Exchange Web System (VIEWS, 2002) to facilitate the exchange of data. VIEWS is an important resource for inclusion in a federated PM network.It should be noted that the PM air quality management process does not use dedicated, ‘hard-wired’ Decision Support Systems (DSS). The management decisions are built mostly on human decision hierarchy. Human decisions makers rely on expert judgment and on the richness and diversity of environmental data. The required data for an AQ decision support system come from many disparate sources, e.g. point and area source data, air quality monitoring by many sensor types, weather data, as well as data on effects, e.g. visibility degradation. The existing centralized federal data management system may be appropriate for enforcement. However, for SIP’s and other analyses it is too rigid, and insufficient in content. Hence, the development and implementation of an agile federated information system is an attractive architecture for AQ decision support.

1.3.2 Anticipated End State of ApplicationAt the anticipated end state of this project is that, 1) satellite data will be routinely incorporated into the PM management process, 2) an agile infrastructure will be in place for PM data sharing,

processing; and 3) PM management decision making will be more effective. The added satellite data from NASA and NOAA sensors will provide new and broader of PM problem. A Federated PM Network will allow the creation of value chains, where the goal of each participant is to add the maximum value to the overall information product. The main participants in the information value adding chain include data providers who supply the primary data to the decision system through their data servers and data transformers who add value to the primary data by processing (e.g. filtering, aggregation, fusion) or through display and browsing services. Transformers produce secondary data for others to use. Data integrators provide “one stop access” to the primary and secondary data as well as for transformation and rendering services. Finally, the end users (analysts, managers, and planners) choose the appropriate data sets, perform the appropriate transformation and rendering, and produce the relevant knowledge for the decision-making process. This is the concept behind SEEDS. The application and the use of REASoN and SEEDS concepts to AQ management is desirable and feasible within the scope of the proposed project.

Data in the federated system can be easily found through a well maintained catalog system whose data can be easily accessed from their custodians through appropriate wrapper services. Value adding processing is achieved through chainable software components, and finally, the results of the data accessing and processing can be delivered to the right place at the right time in the decision system. The federated information system (FIS) through its facilities for finding, accessing, manipulating and delivering data can result in considerable savings in time and money. Hence, the FIS allows these resources to be spent on more effective tasks, such as data analysis, interpretation, and quality control. More importantly, the variety and convenience of data accessible through federated system will allow decision support that is qualitatively different. In short, the main benefits of the federated information system lie in access and recycling of data and the resulting savings in time and money.

The proposed data connectivity system will be built for extensibility for different applications and for scalability in size and the variety of data sets. As the pool of available data and tools continues to expand, the network’s adaptive and extensible design would allow the community to add their own information resources. The tools within the network will create value added knowledge by facilitating the filtering, fusion, and aggregation of existing resources and in providing second order interoperable applications created by chaining services.

The end state will also include three specific software applications built on top of the federated information infrastructure: 1) An Aerosol Watch System; 2) Multidimensional distributed data browser, Voyager; and 3) A software module connection manager, ConMan. These applications will be the primary vehicles for passing the PM data through the value adding chain and delivering the high-grade knowledge to the decision makers.

The Aerosol Watch System has three components: the Virtual PM Monitoring Dashboard, Virtual Workgroup Website, and Air Quality Managers Console. The details of the Aerosol Watch System are described in section 2.1.

The concept of the Aerosol Watch System Consoles is further illustrated here using a July 2002 Quebec smoke event. The regional aerosol concentration during the passage of the Quebec smoke was the highest ever recorded, (well above the air quality standards) in many areas of New York and New England. Yet, there was no warning for the potential health hazard from any of the state air pollution agencies. Below is a brief anatomy of the event as it happened and as it might happen with an operational Aerosol Watch System. More on the Quebec Smoke Event can be found on the Aerosol Events Virtual Community (http://capita.wustl.edu/aerosolevents).

On the afternoon of July 5, 2002 the smoke emissions of the seasonal forest fires in northern Quebec had increased dramatically and the wind carried the thick smoke cloud southward. The yellowish smoke front reached NY/New England on July 6, hampering road and airline traffic due to visibilities well below one mile, and caused record high PM2.5 concentrations. The drifting smoke cloud was noted by a number of observers who routinely examine the daily satellite data, including Husar at CAPITA. While the strong visibility degradation was evident, the air pollution agencies in the Northeast were not aware of the magnitude of the smoke concentrations and extent. Hence, no advisories were issued. It was not until extensive email correspondence and news reports on CNN that the agencies began assessing the features of this smoke event.

Within several days following the event, the data from numerous real time monitoring systems were collected by several groups and displayed on the CAPITA Aerosol Events Website. Figure 2 shows the data from the MODIS sensor including the red fire location pixels as well as the spatial pattern of the smoke plume. Figure 2 also shows the 1:00PM SeaWiFS and TOMS data along with the surface extinction coefficient values collected by the automated ASOS surface meteorological network. A third image shows the time series of PM2.5 mass concentration measured at numerous continuous PM monitors. Animations of the smoke transport (not show here) were generated from 30 minute, 1km visible GOES 8 images. The raw data for all the monitoring data shown in the summaries were available through the internet in almost real time. However, the time and effort to collect, assemble, and superimpose these datasets required several days of effort; considerably diminishing the value of that information to AQ managers.

Figure 2. Real time PM data available from satellite and surface based sensors. Left image shows the MODIS reflectance and fire location pixels. On the right, the TOMS satellite and surface visibility data are superimposed on SeaWiFS reflectance image.

A possible scenario for the handling of such events in the future may consist of the following. Data from all of the relevant satellite and real time aerosol sensors are collected and displayed through the aerosol monitoring dashboard accessible by anyone as a web page. At any given time, a designated set of human observers are assigned the task of watching for “interesting” events. When such a event occurs, a notification message is transmitted to air quality managers and interested subscribers. Following the notification, the community of interested observers begins to assemble a “storybook” summary of the event and shares the summary with the air quality management group. Given such technical guidance, the PM manager decides on the appropriate action.

DataFed

• DataFed Vision• Aid air quality management and science by effective use of relevant data•  

• DataFed Goals• Facilitate the access and flow of atmospheric data from provider to users• Support the development of user-driven data processing value chains• Participate in specific application projects •  

• Approach: Mediation Between Users and Data Providers• DataFed assumes spontaneous, autonomously data providers data• Non-intrusively wraps datasets for access by web services• Mediator software provides homogeneous data views. e.g. geo-spatial, time...•  

• Applications• Building browsers and analysis tools for distributed monitoring data    • Serve as data gateway for user programs; web pages, GIS, science tools• DataFed is currently focused on the mediation of air quality data

Repositories - Data storage and access.Catalogs and Brokers - Elements that find and access resources on a distributed network.Operator and Models - Processes that operate on information, also methods to describe these processes.Applications – Shared components such as viewers, editors, discovery clients and others.