Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-25T08:12:41.914Z Has data issue: false hasContentIssue false
  • English
  • Français

Validation of Default Probabilities

Published online by Cambridge University Press:  07 June 2012

Andreas Blöchlinger
Andreas Blöchlinger*
Affiliation:
[email protected], Zürcher Kantonalbank, Josefstrasse 222, CH-8010 Zurich, Switzerland
Get access Rights & Permissions [Opens in a new window]

Abstract

Well-performing default predictions show good discrimination and calibration. Discrimination is the ability to separate defaulters from nondefaulters. Calibration is the ability to make unbiased forecasts. I derive novel discrimination and calibration statistics to verify forecasts expressed in terms of probability under dependent observations. The test statistics’ asymptotic distributions can be derived in analytic form. Not accounting for cross correlation can result in the rejection of actually well-performing predictions, as shown in an empirical application. I demonstrate that forecasting errors must be serially uncorrelated. As a consequence, my multiperiod tests are statistically consistent.

Type
Research Articles
Information
Journal of Financial and Quantitative Analysis , Volume 47 , Issue 5 , October 2012 , pp. 1089 - 1123
Copyright
Copyright © Michael G. Foster School of Business, University of Washington 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alchian, A. A. “Uncertainty, Evolution, and Economic Theory.” Journal of Political Economy, 58 (1950), 211–221.Google Scholar
Altman, E. I. “Financial Ratios, Discriminant Analysis, and the Prediction of Corporate Bankruptcy.” Journal of Finance, 23 (1968), 589–609.Google Scholar
Andrews, D. W. K. “Chi-Square Diagnostic Tests for Econometric Models: Theory.” Econometrica, 56 (1988), 1419–1453.Google Scholar
Andrews, D. W. K. “A Conditional Kolmogorov Test.” Econometrica, 65 (1997), 1097–1128.CrossRefGoogle Scholar
Andrews, D. W. K. “Cross-Section Regression with Common Shocks.” Econometrica, 73 (2005), 1551–1573.Google Scholar
Balthazar, L. “PD Estimates for Basel II.” Risk Magazine, 17 (2004), 84–85.Google Scholar
Bamber, D. “The Area Above the Ordinal Dominance Graph and the Area Below the Receiver Operating Characteristic Graph.” Journal of Mathematical Psychology, 12 (1975), 387–415.CrossRefGoogle Scholar
Basel Committee on Banking Supervision. “The Internal Ratings-Based Approach.” Consultative Document, Bank for International Settlements (2001).Google Scholar
Basel Committee on Banking Supervision. “Studies on the Validation of Internal Rating Systems.” Working Paper No. 14, Bank for International Settlements (2005).Google Scholar
Bertolini, G.; Damico, R.; Nardi, D.; Tinazzi, A.; and Apolone, G.. “One Model, Several Results: The Paradox of the Hosmer-Lemeshow Goodness-of-Fit Test for the Logistic Regression Model.” Journal of Epidemiology and Biostatistics, 5 (2000), 251–253.Google Scholar
Bharath, S. T., and Shumway, T.. “Forecasting Default with the Merton Distance to Default Model.” Review of Financial Studies, 21 (2008), 1339–1369.Google Scholar
Blöchlinger, A. “Arbitrage-Free Credit Pricing Using Default Probabilities and Risk Sensitivities.” Journal of Banking and Finance, 35 (2011), 268–281.Google Scholar
Blöchlinger, A., and Leippold, M.. “Economic Benefit of Powerful Credit Scoring.” Journal of Banking and Finance, 30 (2006), 851–873.Google Scholar
Blöchlinger, A., and Leippold, M.. “A New Goodness-of-Fit Test for Event Forecasting and Its Application to Credit Defaults.” Management Science, 57 (2011), 487–505.Google Scholar
Blochwitz, S.; Hohl, S.; Tasche, D.; and Wehn, C. S.. “Validating Default Probabilities on Short Time Series.” Working Paper, Deutsche Bundesbank (2004).Google Scholar
Brier, G. W. “Verification of Forecasts Expressed in Terms of Probability.” Monthly Weather Review, 78 (1950), 1–3.2.0.CO;2>CrossRefGoogle Scholar
Chen, R.-R., and Sopranzetti, B. J.. “The Valuation of Default-Triggered Credit Derivatives.” Journal of Financial and Quantitative Analysis, 38 (2003), 359–382.CrossRefGoogle Scholar
Cooke, W. E. “Forecasts and Verifications in Western Australia.” Monthly Weather Review, 34 (1906), 23–24.Google Scholar
Crouhy, M.; Jarrow, R. A.; and Turnbull, S. M.. “The Subprime Credit Crisis of 2007.” Journal of Derivatives, 16 (2008), 81–110.CrossRefGoogle Scholar
Dawid, A. P. “The Well-Calibrated Bayesian.” Journal of the American Statistical Association, 77 (1982), 605–610.Google Scholar
Dawid, A. P. “Calibration-Based Empirical Probability.” Annals of Statistics, 13 (1985), 1251–1274.Google Scholar
DeLong, E. R.; DeLong, D. M.; and Clarke-Pearson, D. L.. “Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach.” Biometrics, 44 (1988), 837–845.Google Scholar
Department of the Treasury, Federal Reserve System, Federal Deposit Insurance Corporation. “Internal Ratings-Based Systems for Corporate Credit and Operational Risk Advanced Measurement Approaches for Regulatory Capital.” Draft Supervisory Guidance with Request for Comment, Federal Register, 68, No. 149 (Aug. 4, 2003), 45949–45988.Google Scholar
Friedman, M. Essays in Positive Economics. Chicago, IL: University of Chicago Press (1953).Google Scholar
Gini, C. “Measurement of Inequality of Incomes.” Economic Journal, 31 (1921), 124–126.Google Scholar
Gordy, M. B. “A Comparative Anatomy of Credit Risk Models.” Journal of Banking and Finance, 24 (2000), 119––149.CrossRefGoogle Scholar
Hall, P., and Heyde, C. C.. Martingale Limit Theory and Its Application. New York, NY: Academic Press (1980).Google Scholar
Harrell, F. E. Regression Modeling Strategies. New York, NY: Springer-Verlag (2001).Google Scholar
Hosmer, D. W., and Lemeshow, S.. Applied Logistic Regression. New York, NY: John Wiley and Sons (1989).Google Scholar
Karr, A. F. Probability. New York, NY: Springer-Verlag (1993).CrossRefGoogle Scholar
Kolmogorov, A. N. “Sulla Determinazione Empirica Di Una Legge Di Distribuzione.” Giornale dell’Istituto Italiano degli Attuari, 4 (1933), 83–91.Google Scholar
Li, D. X. “On Default Correlation: A Copula Function Approach.” Journal of Fixed Income, 9 (2000), 43––54.Google Scholar
Lorenz, M. O. “Methods of Measuring the Concentration of Wealth.” Publications of the American Statistical Association, 9 (1905), 209–219.Google Scholar
Merton, R. C. “On the Pricing of Corporate Debt: The Risk Structure of Interest Rates.” Journal of Finance, 29 (1974), 449–470.Google Scholar
Pearson, K. “On the Criterion That a Given System of Deviations from the Probable in the Case of a Correlated System of Variables Is Such That It Can Be Reasonably Supposed to Have Arisen from Random Sampling.” London, Edinburgh and Dublin Philosophical Magazine and Journal of Science, 50 (1900), 157–175.Google Scholar
Pollard, D. “General Chi-Square Goodness-of-Fit Tests with Data-Dependent Cells.” Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 50 (1979), 317–331.Google Scholar
Sandroni, A. “Do Markets Favor Agents Able to Make Accurate Predictions?” Econometrica, 68 (2000), 1303–1341.Google Scholar
Spiegelhalter, D. J. “Probabilistic Prediction in Patient Management and Clinical Trials.” Statistics in Medicine, 5 (1986), 421–433.Google Scholar
Stein, R. M. “Are the Probabilities Right?” Working Paper, Moody’s KMV (2003).Google Scholar
Swets, J. A. “Measuring the Accuracy of Diagnostic Systems.” Science, 240 (1988), 1285–1293.Google Scholar
Treacy, W. F., and Carey, M.. “Credit Risk Rating Systems at Large U.S. Banks.” Journal of Banking and Finance, 24 (2000), 167–201.Google Scholar
Wiginton, J. C. “A Note on the Comparison of Logit and Discriminant Models of Consumer Credit Behavior.” Journal of Financial and Quantitative Analysis, 15 (1980), 757–770.Google Scholar
Wilde, T. “CreditRisk+ A Credit Risk Management Framework.” Technical Document, Credit Suisse (1997).Google Scholar
Zhou, C. “An Analysis of Default Correlations and Multiple Defaults.” Review of Financial Studies, 14 (2001), 555–576.Google Scholar