The attributes of forecast systems: a general framework for the evaluation and calibration of weather forecasts

doi:10.1017/CBO9780511617652.023

Chapter 22 - The attributes of forecast systems: a general framework for the evaluation and calibration of weather forecasts

Published online by Cambridge University Press: 03 December 2009

Zoltan Toth ,

Olivier Talagrand and

Yuejian Zhu

Edited by

Tim Palmer and

Renate Hagedorn

Show author details

Zoltan Toth: Affiliation:
National Centers for Environmental Prediction, Washington, DC
Olivier Talagrand: Affiliation:
Laboratoire de Météorologie Dynamique, Paris
Yuejian Zhu: Affiliation:
National Centers for Environmental Prediction, Washington, DC
Tim Palmer: Affiliation:
European Centre for Medium-Range Weather Forecasts
Renate Hagedorn: Affiliation:
European Centre for Medium-Range Weather Forecasts

Book contents

Get access

Summary

Reliability and resolution are the two main attributes of forecast systems. These attributes statistically relate the performance of a forecast system to verifying data in an abstract sense. Forecast attributes have been separately defined in the literature for systems that generate forecasts of particular formats or types. In this chapter, statistical reliability and resolution are defined in a general sense, irrespective of the type or format of a forecast. Statistical reliability is concerned only with the form of forecasts, whereas statistical resolution is concerned only with the predictive capability of a forecast system, related to the time evolution of the system that is being forecast.

The two main attributes are independent characteristics of a forecast system and can be quantitatively assessed by a host of different verification measures. The general definition of forecast attributes allows a systematic discussion of the relationship between the verification and calibration of forecasts. Calibration as defined here is an adjustment of the form of the forecasts, to match the distribution of verifying observations that follow the issuance of forecasts of a particular form.

Resolution, as the inherent predictive value of forecast systems, is the attribute most sought after by developers of forecast systems. Reliability, however, is equally important in real world applications. That calls for the generation of a long enough record of hindcasts to allow for a good calibration of forecasts, or, preferably, for improvements in forecast systems that directly lead to better reliability.

Type: Chapter
Information: Predictability of Weather and Climate , pp. 584 - 595

DOI: https://doi.org/10.1017/CBO9780511617652.023 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Atger, F. (1999). The skill of ensemble prediction systems. Mon. Weather Rev., 127, 1941–532.0.CO;2>CrossRef Google Scholar

Atger, F. (2003). Spatial and interannual variability of the reliability of ensemble-based probabilistic forecasts: consequences for calibration. Mon. Weather Rev., 131, 1509–232.0.CO;2>CrossRef Google Scholar

Candille, G. (2003). Validation of probabilistic meteorological forecast (in French). Doctoral Dissertation, Universite Pierre-et-Marie-Curie, Paris, FranceGoogle Scholar

Candille, G. and Talagrand, O. (2005). Evaluation of probabilistic prediction systems for a scalar variable. Quart. J. Roy. Meteor. Soc., 131, 2131–50CrossRef Google Scholar

Hamill, T. (1997). Reliability diagrams for multicategory probabilistic forecasts. Weather Forecast., 12, 736–412.0.CO;2>CrossRef Google Scholar

Joliffe, I. T. and Stephenson, D. B. (eds.). (2003). Forecast Verification: A Practitioner's Guidein Atmospheric Science. WileyGoogle Scholar

Legg, T. P. and Mylne, K. (2005). Early warnings of severe weather from ensemble forecast information. Weather Forecast., 19, 891–9062.0.CO;2>CrossRef Google Scholar

Murphy, A. and Daan, H. (1985). Forecast evaluation. In Probability, Statistics, and Decision Making in the Atmospheric Sciences, ed. Murphy, A. H. and Katz, R. W., pp. 379–437. Westview PressGoogle Scholar

Stanski, H. R., Wilson, L. J. and , W. R. Burrows (1989). Survey of Common Verification Methods in Meteorology. World Weather Watch Technical Report 8. World Meteorological OrganizationGoogle Scholar

Toth, Z., Talagrand, O., G. Candille and Y. Zhu (2003). Probability and ensemble forecasts. In Forecast verification: A Practitioner's Guide in Atmospheric Science, ed. Jolliffe, I. T. and Stephenson, D. B., pp. 137–63. WileyGoogle Scholar

Toth, Z., Zhu, Y. and Marchok, T. (2001). On the ability of ensembles to distinguish between forecasts with small and large uncertainty. Weather Forecast., 16, 436–772.0.CO;2>CrossRef Google Scholar

Wilks, D. S. (1995). Statistical Methods in the Atmospheric Sciences. Academic PressGoogle Scholar

Zhu, Y. and Toth, Z. (2001). Extreme weather events and their probabilistic prediction by the NCEP Ensemble Forecast System. In Preprints of the AMS Symposium on Precipitation Extremes: Prediction, Impact, and Responses, 14–19 January 2001, Albuquerque, NM, pp. 82–5. American Meteorological Society

Zhu, Y., Toth, Z., Wobus, R., Richardson, D. and Mylne, K. (2002). The economic value of ensemble based weather forecasts. Bull. Am. Meteorol. Soc., 83, 73–832.3.CO;2>CrossRef Google Scholar