4 - Hypothesis testing

Summary

From now on, we consider a variety of specific statistical problems, beginning in this chapter with a re-examination of the theory of hypothesis testing. The concepts and terminology of decision theory will always be present in the background, but inevitably, each method that we consider has developed its own techniques.

In Section 4.1 we introduce the key ideas in the Neyman–Pearson framework for hypothesis testing. The fundamental notion is that of seeking a test which maximises power, the probability under repeated sampling of correctly rejecting an incorrect hypothesis, subject to some pre-specified fixed size, the probability of incorrectly rejecting a true hypothesis. When the hypotheses under test are simple, so that they completely specify the distribution of X, the Neyman–Pearson Theorem (Section 4.2) gives a simple characterisation of the optimal test. We shall see in Section 4.3 that this result may be extended to certain composite (non-simple) hypotheses, when the family of distributions under consideration possesses the property of monotone likelihood ratio. Other, more elaborate, hypothesis testing problems require the introduction of further structure, and are considered in Chapter 7. The current chapter finishes (Section 4.4) with a description of the Bayesian approach to hypothesis testing based on Bayes factors, which may conflict sharply with the Neyman–Pearson frequentist approach.