13 - Confidence intervals and prediction intervals for feedforward neural networks

Summary

Artificial neural networks have been used as predictive systems for a variety of medical domains, but none of the systems encountered by Baxt (1995) and Dybowski & Gant (1995) in their reviews of the literature provided any measure of confidence in the predictions made by those systems. In a medical setting, measures of confidence are of paramount importance (Holst et al. 1998), and we introduce the reader to a number of methods that have been proposed for estimating the uncertainty associated with a value predicted by a feedforward neural network.

The chapter opens with an introduction to regression and its implementation within the maximum likelihood framework. This is followed by a general introduction to classical confidence intervals and prediction intervals. We set the scene by first considering confidence and prediction intervals based on univariate samples, and then we progress to regarding these intervals in the context of linear regression and logistic regression. Since a feedforward neural network is a type of regression model, the concepts of confidence and prediction intervals are applicable to these networks, and we look at several techniques for doing this via maximum likelihood estimation. An alternative to the maximum likelihood framework is Bayesian statistics, and we examine the notions of Bayesian confidence and prediction intervals as applied to feedforward networks. This includes a critique on Bayesian confidence intervals and classification.