14 - Backward look

Summary

Randomization

Although randomization is generally regarded as essential, there is less agreement on how to carry it out and on how to verify whether any proposed randomization procedure achieves its objectives. Here I discuss some possibilities.

Random sampling

Many statistics textbooks give the impression that every collection of observations is a random sample. This is rarely the case for designed experiments. Experimental resources are too valuable for us to choose a random subset and ignore the rest. We use the experimental units which are to hand in the laboratory or the field, or we use all suitable volunteers in a clinical trial. In a small trial it is more important that the experimental units be reasonably similar than that they be random, and in a large trial it may be more important that they be representative. If the experimental units are to be working farms, then stratified random sampling can give a representative sample, but in practice we can still use only those farms whose farmer is willing to participate in the experiment.

Example 14.1 (Small trial on volunteers) In Question 2.1, the psychologist needs to choose as his experimental units eleven of the students listed in Table 2.7. For such a small trial, he should choose people of the same sex and approximately the same age. A random sample of eleven people would almost certainly be more variable, giving him less power to detect any treatment difference.

The chief situation in which random sampling plays a role in experimentation is when observational units are much smaller than experimental units.