CMAS Conference, October 16 – 18, 2006CMAS Conference, October 16 – 18, 2006

The work presented here was performed by the New York State The work presented here was performed by the New York State Department of Environmental Conservation with partial support from the Department of Environmental Conservation with partial support from the U.S. EPA under cooperative agreement CR83228001. The views expressed U.S. EPA under cooperative agreement CR83228001. The views expressed in this paper do not necessarily reflect the views or policies of the New in this paper do not necessarily reflect the views or policies of the New York State Department of Environmental Conservation or those of the U.S. York State Department of Environmental Conservation or those of the U.S. EPA. EPA.

Exploring Approaches to Exploring Approaches to Integrate Observations and Integrate Observations and

CMAQ Simulations for Improved CMAQ Simulations for Improved Air Quality ForecastsAir Quality Forecasts

C. HogrefeC. Hogrefe1,21,2, W. Hao, W. Hao11, K. Civerolo, K. Civerolo11, J.-Y. Ku, J.-Y. Ku11, and G. , and G. SistlaSistla11

11New York State Department of Environmental ConservationNew York State Department of Environmental Conservation22Atmospheric Sciences Research Center, State University of New Atmospheric Sciences Research Center, State University of New York at Albany York at Albany

IntroductionIntroduction

Overview of NYSDEC/EPA/NOAA air Overview of NYSDEC/EPA/NOAA air quality pilot study and past model quality pilot study and past model performanceperformance

Description of potential approaches Description of potential approaches to improve air quality predictionsto improve air quality predictions

ResultsResults Summary and OutlookSummary and Outlook

• Establish partnership between NYSDEC, NOAA and Establish partnership between NYSDEC, NOAA and EPA in the area of numerical air-quality predictionEPA in the area of numerical air-quality prediction

• Complements the NOAA/EPA national air quality Complements the NOAA/EPA national air quality forecasting activitiesforecasting activities

• Project period: January 2005 – December 2007Project period: January 2005 – December 2007

• Apply and evaluate a state-of-science photochemical Apply and evaluate a state-of-science photochemical modeling system on an ongoing basis with special modeling system on an ongoing basis with special emphasis on PMemphasis on PM2.52.5 predictions over New York State predictions over New York State

• Assess the potential usefulness of grid-based Assess the potential usefulness of grid-based photochemical models to provide Ophotochemical models to provide O33 and PM and PM2.52.5 forecasts across New York Stateforecasts across New York State

• Archive daily concentrations fields for potential use Archive daily concentrations fields for potential use in air quality / health studiesin air quality / health studies

Overview of Pilot StudyOverview of Pilot Study

• Meteorology/Emissions: Based on NCEP/NWS48-hr ETA (prior to June 2006) or WRF (since June 2006) forecasts initialized at 12:00 UTC and 2002/2004/2005 emission inventory processed with PREMAQ

• Photochemical Modeling: CMAQ (version 4.5.1)

• Horizontal resolution: 12 km

• Vertical resolution: 22 layers, lowest layer ~40 m

• Simulation Periods: • July – September, 2004• January – March, 2005• June 2005 – present

Simulation SetupSimulation Setup

Fractional Bias (FB) as defined in Morris et al. (2005) for 24-hr average total PM2.5 predictions at all FRM monitors located in New York State calculated for 2004 - 2005

Model Performance over NYS: The Not-So-Good …

Observed and Predicted 24-hr Average Speciated PM2.5 at Eight STN Monitors in NYS (Upstate Rural, Upstate Urban, NYC Metro), July – September 2004

Hogrefe et al., 2006, JAM, in press

Correlation Correlation Coefficients for Time Coefficients for Time Series of Daily Series of Daily Maximum 8-hr OMaximum 8-hr O33

… and the promising

Correlation Correlation Coefficients for Time Coefficients for Time Series of Daily Series of Daily Average PMAverage PM2.52.5

Comparison of Comparison of Predicted Air Predicted Air Quality Index Quality Index Tendencies for Tendencies for CMAQ (left), CMAQ (left), Routine Expert-Routine Expert-Based Forecasts Based Forecasts (center), and (center), and Trend Trend PersistencePersistence

Hogrefe et al., 2006, JAM, in press

CMAQ forecasts often have CMAQ forecasts often have significant biases, especially for PMsignificant biases, especially for PM2.52.5

On the other hand, CMAQ simulations On the other hand, CMAQ simulations show skill in capturing temporal show skill in capturing temporal trends in air qualitytrends in air quality

--> Explore approaches to integrate --> Explore approaches to integrate observations and CMAQ predictions observations and CMAQ predictions for improved air quality forecastsfor improved air quality forecasts

Potential Approaches for Combining CMAQ Simulations and Observations for Air Quality Forecasts

)(1 11 ObsCMAQCMAQF ii

Simple Bias-Correction Prediction – “Adjustment 1”

111 ][2 iii CMAQObsCMAQCMAQF

Binned Bias-Correction Prediction – “Adjustment 2”

“Bias-Adjustments”

)(3 11 iiii CMAQCMAQObsF

Simple CMAQ-Tendency Prediction – “Adjustment 3”

CMAQ

Obsiiii CMAQCMAQObsF

)(4 11

Variability-Adjusted CMAQ-Tendency Prediction – “Adjustment 4”

sCMAQCMAQObsF iiii )(5 11

Slope-Adjusted CMAQ-Tendency Prediction – “Adjustment 5”

“Tendency-Adjustments”

Time Period and Domain of Time Period and Domain of AnalysisAnalysis

June – September, 2005June – September, 2005 Focus on monitors in New York StateFocus on monitors in New York State Note: Methods 1-2 and 4-5 rely on incorporating Note: Methods 1-2 and 4-5 rely on incorporating

observations not just for the current day but for observations not just for the current day but for an extended time period:an extended time period: In a routine forecast setting, this extended time period In a routine forecast setting, this extended time period

could be the past week, month, or seasoncould be the past week, month, or season In this study, we utilized the fixed time period from June In this study, we utilized the fixed time period from June

1 – September 30 also used for evaluating these 1 – September 30 also used for evaluating these methods, i.e. for any given day, both past and future methods, i.e. for any given day, both past and future observations were includedobservations were included

Future analysis will consider the impact of the choice of Future analysis will consider the impact of the choice of the “calibration” or “learning” period over which the the “calibration” or “learning” period over which the adjustment parameters in methods 1-2 and 4-5 are adjustment parameters in methods 1-2 and 4-5 are calculated on the performance of these approachescalculated on the performance of these approaches

Methods of ComparisonMethods of Comparison Focus on daily maximum 8-hr OFocus on daily maximum 8-hr O33 and daily and daily

average PMaverage PM2.52.5 Comparison of observed and simulated Comparison of observed and simulated

pollutant concentration distributionspollutant concentration distributions RMSE (total, systematic, and unsystematic)RMSE (total, systematic, and unsystematic) Categorical metrics as defined in Kang et al. Categorical metrics as defined in Kang et al.

(2005) for thresholds of 84 ppb (O(2005) for thresholds of 84 ppb (O33) and 40 ) and 40 ug/m3 (PMug/m3 (PM2.52.5)) False Alarm Ratio (FAR)False Alarm Ratio (FAR) Probability of Detection (POD)Probability of Detection (POD) Critical Success Index (CSI)Critical Success Index (CSI)

Illustration of Total, Illustration of Total, Systematic, and Systematic, and

Unsystematic RMSEUnsystematic RMSE

Unsystematic RMSE is determined by the distance between the datapoints and the linear regression best-fit line

Total RMSE is determined by the distance of the data points from the 1:1 lineSystematic RMSE is determined by the distance between the the linear regression best-fit line and the 1:1 line

• Methods 3-5 yield closer agreement with observed distributions than either the unadjusted CMAQ simulations or methods 1-2

Distributions of Daily Max. 8-hr O3 (left) and 24-hr Av. PM2.5 (right) from Observations, CMAQ Predictions, and Adjustment

Methods 1 – 5

Total, Systematic, and Unsystematic RMSE of Daily Max. 8-hr O3

• Methods 1 and 2 (the “bias-adjustment” methods) significantly reduce total RMSE, mostly by reducing the systematic RMSE

• Methods 3 -5 (the “tendency-adjustment” methods) generally show little improvement in terms of overall RMSE. While they strongly reduce the systematic RMSE, they increase the unsystematic RMSE

Total, Systematic, and Unsystematic RMSE of Daily Average PM2.5

•For PM2.5, the “binned-bias-adjustment” yields the largest reduction of total RMSE

•Similar to daily maximum 8-hr ozone, the “tendency” adjustment approaches reduce the systematic RMSE but tend to increase the unsystematic RMSE

False Alarm Ratio (FAR), Probability of Detection (POD), and Critical Success Index (CSI) For Daily Average PM2.5 Above a Threshold of 40

ug/m3

• Consistent with the narrower density functions shown before, the “bias-adjustment” methods reduce both FAR and POD, while the “tendency-adjustment” methods increase both FAR and POD

• As a result, the overall improvement in CSI over the original CMAQ simulations is relatively small for all methods

0

10

20

30

40

50

60

70

80

90

100

RMSE_O3 CSI_O3 RMSE_PM25 CSI_PM25

Perc

en

tag

e o

f S

tati

on

s

CMAQ

CMAQ-Adj1

CMAQ-Adj2

CMAQ-Adj3

CMAQ-Adj4

CMAQ-Adj5

Percentage of Stations at Which a Given Adjustment Method Performed Best for a Given Metric and Pollutant

• The “bias-correction” approaches 1 and especially 2 work best for The “bias-correction” approaches 1 and especially 2 work best for reducing the total RMSE at most sites for both Oreducing the total RMSE at most sites for both O33 and PM and PM2.52.5

• The “tendency-correction” approaches often work best for improving The “tendency-correction” approaches often work best for improving the CSI, especially for Othe CSI, especially for O33

Summary and OutlookSummary and Outlook Motivated by past CMAQ forecast evaluation results, we tested

five potential approaches for providing improved air quality forecasts based on both observations and CMAQ simulations.

While the “bias-correction” approaches 1 or 2 work best for reducing the total RMSE at most sites, the approaches that combine today’s observations with unadjusted or adjusted CMAQ-predicted temporal changes often work best for improving the CSI, especially for O3

Moreover, the best adjustment method to improve the CSI, which measures the quality of categorical forecasts, needs to be chosen on a pollutant-by-pollutant and station-by-station basis

Other studies explored more advanced techniques. For example, Delle Monache et al. (2006) and Kang et al. (2006) describe the application of a Kalman filter to generate improved air quality forecasts and report good success.

Additional methods might aim at including spatial correlation structures into the model adjustment algorithm rather than relying solely on temporal structures at individual monitors. Such analyses will be performed in the future.