Perspectives on Targeting

Sharanya J. Majumdar (RSMAS/U. Miami)

Session 3.1, THORPEX/DAOS WG Fourth Meeting27-28 June 2011.

Introduction

• 3rd DAOS WG meeting in Montreal (July 2010): reviewed entire procedure: case selection, objective techniques, evaluation methods.

• This talk: focus on outstanding issues raised in review article.

Outline1. Status of review article

2. Main Conclusions

3. Open questions and issues

4. Recommendations

Status of review article• “Targeted observations for improving numerical weather

prediction: An overview”• Version 1.0 completed 12/20/10• Version 2.0 completed 2/18/11 for WWRP/JSC• Version 2.2 sent to DAOS WG 3/21/11• Version 3.0 completed 5/24/11, sent to all co-authors

Goals• To revise Version 3.0 and finalize as a THORPEX report by 31

July• To truncate manuscript by ~50% and submit to BAMS by 31

August. How to communicate results?

Mid-latitude weather (NCEP GFS)• Szunyogh et al. (2000, 2002)– ~70% of all WSR cases improved– average 12-h gain in lead time

• Yucheng Song (2010, pers. comm.)– ~70% of 128 cases between 2004-7 improved– ‘Large’ improvements in 5/20 cases each year– Average forecast error reduction 10-20% within verification area

for NOAA’s ‘high priority’ cases– Winter T-PARC: 75% of the 52 cases were improved

• Do conclusions from Szunyogh et al. (2000, 2002) still hold?

Mid-latitude weather

• Szunyogh et al. (2002), Petersen et al. (2007), Majumdar et al. (2002, 2010): target areas often in cloudy and/or baroclinic regions.

• ETKF demonstrated the ability to quantitatively predict reduction in forecast error variance, particularly in zonal flows.

• Target regions traceable from Japan to North America for 4-7 day forecasts.

• Other THORPEX experiments: very small sample size of specific targeting cases.

Mid-latitude weather: Conclusion• “In conclusion, the average cumulative value of targeted data

in the mid-latitudes has been found to be positive but small.”• Possible reasons for small impact:– dropwindsonde data are

• relatively few• collected in limited area that does not cover full sensitivity• not deployed continuously in time.

– improved routine observational network– advancements in modeling and data assimilation combine to

improve analyses and thereby limit the potential for extra observations to improve the forecast

– targeting strategies are limited– operational data assimilation schemes not yet fully-flow-dependent

Tropical cyclones• Main metric to date: TC track. Targeted observations have

proven to be useful statistically.• Benefit to society more straightforward to define than for mid-

latitude weather.• An effective strategy: observe uniformly around TC• Extra observations in target areas have been shown to be

useful on a case-by-case basis.• Quantitative benefit differs from model to model

• 3d-Var versus 4d-Var• Treatment of routinely-available satellite versus aircraft observations

• Improvements to large-scale pattern downstream?

• See T-PARC talk for further details

General conclusions

• Spatial / temporal range of aircraft is a limiting constraint.• Cumulative adjoint-based observation impact of a small

number of targeted aircraft observations to forecast accuracy over broad verification regions is smaller than that of any individual observing system.

• Results are heavily dependent on the flow regime, the numerical model, and the treatment of both routine and targeted observations in the data assimilation scheme.

• Current assimilation methodologies are limited in their use of fully flow-dependent spatial structure functions to control the spreading of the influence of observations.

Targeting routinely available data

• ECMWF trifecta:– Kelly et al. (2007): impact over North America from

observations over the Pacific exceeds that over Europe from Atlantic observations

– Buizza et al. (2007): observations taken in SV target areas were on average more valuable than those taken in randomly selected areas (winter: forecast errors reduced by 4% over Pac / 2% over Atl).

– Cardinali et al. (2007): removal of observations in SV-sensitive areas degraded skill more so than observations selected randomly.

Targeting routinely available data

• Gelaro et al. (2010)– Global impacts of the major observation types on

24 h forecast errors are similar in each system. – Largest forecast error reductions due to

assimilation of satellite radiances, geostationary satellite winds, rawinsondes and commercial aircraft (similar to Langland and Baker 2004)

– Only a small majority (50-54%) of the total number of observations assimilated improved the forecast

Targeting routinely available data

• Bauer et al. (2011)– Influence of adding satellite data at 0.625°

resolution in ECMWF SV-sensitive areas. – 2-day forecasts of 500 hPa Z most improved when

increased data were assimilated in flow-dependent SV areas, as opposed to in randomly distributed areas or climatological SV areas.

– Forecast impacts larger in SH than NH, dependent on season.

Medium-range: conclusions• Some studies: targeted observations made negligible

differences to forecasts downstream• Others: positive impacts in mid-latitudes and tropics.• Results are not mature enough to make an authoritative

statement. • Given that model error is expected to amplify at longer lead

times for synoptic-scale processes, the value of targeted observations in the medium- and long-ranges is expected to diminish, particularly if the observations are confined to a local region.

• Broader-scale, regime-based sampling (e.g. adaptive use of satellite data) is expected to be one promising approach to address targeting for long-range forecasts.

Questions: Case selection

• Can low-impact cases be predicted in advance?– Low-impact weather– Expected low impact of extra obs on forecast

• Is ensemble spread / ETKF signal variance / any other measure an accurate predictor of potentially useful cases for targeting?

• Do we have a scientific idea of low predictability regimes in which targeted observations would likely yield significant benefit?

Questions: verification 1

• Large, fixed verification regions – Increase level of automation– Increase breadth of observations available for verification– Verification statistics are diluted in large areas that are not

directly affected by weather event

• Small, mobile verification regions– Centered on weather event that is targeted (or maximum

in spread or signal variance)– Relatively few observations available for verification

Questions: verification 2

• Many studies evaluate average improvements, in a verification region and/or over many cases.

• Better to focus on individual systems: select only high-impact, low predictability cases for regional targeting?

• Is a field campaign judged a success if (say) two high-impact forecasts are substantially improved while the improvement is minimal in others?

• How many forecasts per year are truly busted?• Local or global verification metrics?

Questions: value of targeting 1

• What types of forecast improvements are sufficient to justify a mobile observing network based on a targeting strategy?– Depends on the agency, e.g. NOAA uses aircraft

recon since the impact on NCEP models is deemed satisfactory, even if there is little impact in ECMWF.

– Expectation: the average improvement due to targeted data should be small.

Questions: value of targeting 2

• We can quantify improvement to forecast skill, but can we translate this into forecast value?

• Can we evaluate cost-effectiveness and benefits to society?

Questions: recent progress

• Significant progress since Langland (2005)?– For tropical cyclones, clearly YES– For mid-latitudes, not clear– Medium range: some progress– Advances in satellite data targeting– Adjoint-based observation impact widely used– New field campaigns: Concordiasi etc.

Recommendations

• Would like to make ~5 concrete recommendations in paper.

Potential Recommendations• Quantify benefit of adapting the network of existing regular

satellite and in-situ observations in a targeted sense:- On-request rapid-scan wind data- Targeted satellite channel selection and data-thinning- Increase observations from commercial aircraft- Request radiosondes at non-standard time- Explore the utility of observations in cloudy areas

• Annual field campaigns require continuous evaluation: to justify their benefit and to determine how to derive further benefit as observations, models and DA evolve.– Make assessments using the most up-to-date assimilation systems

and models. • Investigate regional targeting of flow regimes with lower

predictability on a continuous basis for periods of days to weeks. This may be more effective than occasional, limited-area sampling if a global verification norm is used.

Potential Recommendations• Investigate forecast dropouts.• Continue to develop innovative targeting techniques that

distinguish between observations that significantly reduce forecast error variance and those that do not.

• Evaluate whether data assimilation schemes that better account for flow dependence can identify gaps in data sensitive regions that current data assimilation schemes fail to fill. If so, does this increase the impact of targeted observations?

• Scientific studies on targeting methods in operational models; impact of targeted observations; dynamics of error growth; uncertainty prediction.