8 - Hypothesis testing

Summary

Formulating a hypothesis

Up until now we have discussed how to define a measurement in terms of a central value, uncertainties, and units, as well as how to extend these concepts to encompass confidence levels (both one- and two-sided). A related aspect of performing a measurement is to test a theory or, as it is usually phrased, a hypothesis. For example, consider the case where a theorist writes a paper proposing the existence of some effect that can be tested via some physical process. It is then down to an experimentalist to develop a method that can be used to test the validity of that theory. In this example the default hypothesis (usually referred to as the null hypothesis and often denoted by H₀) would be that the theory is valid, and the experimenter would then embark on a measurement that could be used to test the null hypothesis.

Having defined a null hypothesis, by default the complement of that hypothesis exists as an alternative hypothesis often denoted by H₁. Given data we can test the null hypothesis to see if the data are in agreement or in disagreement with it. In order to qualify what is meant by data being in agreement with the null hypothesis, we need to quantify what we consider to be agreement (at some C L).