On the Difference in Means

Summary

ABSTRACT. Given two sets of data that are repeated measurements of the same physical quantity, one “control” and one “trial,” there are three problems of interest to the experimenter: (1) determine if something changed, (2) if something changed, what? and (3) estimate the magnitude of the change. These three problems are addressed using probability theory as extended logic. In the first section, the probability that the data sets differ is computed independent of what changed, i.e., independent of whether or not the means or standard deviations changed. In the second section, two probability distributions are computed: first, the probability that the means changed is computed independent of whether or not the standard deviations changed. Then second, the probability that the standard deviations changed is computed independent of whether or not the means changed. In the third section, the problem of estimating the magnitude of the changes is addressed. Here the probability density functions for both the difference in means and the ratio of standard deviations is computed. The probability for the ratio of standard deviations is computed independent of whether or not the means are the same, just as the probability for the difference in means is computed independent of whether or not the standard deviations are the same. This last calculation generalizes the solution of both the two-sample problem (different means and same but unknown standard deviations) and the Behrens-Fisher problem (different means and different unknown standard deviations).