7 - All that glitters is not gold: the problem of error

Summary

Box 7.1 Bigger isn't always better!

In the run-up to the 1936 presidential election in America, the Literary Digest conducted a poll of more than two million voters and confidently predicted that the Republican candidate, Alf Landon, would win. On the day it was the Democrat candidate, Franklin D. Roosevelt, who won a landslide victory. The Digest had correctly predicted the winner of the previous five elections, so what went wrong in 1936?

The Digest sent polling papers to households listed in telephone directories and car registration records. In 1936, however, telephone and car ownership was more common among more affluent households and these were the people who were also more likely to vote Republican. The generally less affluent Democrat voters were thus under-represented in the sample of voters polled. In contrast, a young George Gallup conducted a much smaller poll of a few thousand representative voters and correctly predicted the Roosevelt win. As a result of this fiasco the Digest folded but Gallup polls are still conducted today.

We saw in Chapter 6 that larger studies are less likely to get the wrong results due to chance (or random sampling error) than smaller studies; however, the example in Box 7.1 shows that a large sample size is not sufficient to ensure we get the right results. The enormous presidential poll conducted by the Literary Digest didn't get the right answer because it included the ‘wrong’ people, i.e. they were not representative of everybody in the voting population.