Chapter 14 - Problem questions

Summary

Summary

Of the investigations described in this book, 19 can be said to involve questions that are too difficult for the respondents. Four of these questions are impossible for anyone to answer properly. Another 7 questions involve the use of percentages, or statistical techniques that many ordinary respondents cannot be expected to know. Five more questions require other relevant knowledge that most ordinary respondents lack, or in one case cannot be expected to think of. The remaining 3 of the 19 questions either require more time than is permitted, or involve complex mental arithmetic. There are also 9 additional questions with misleading contexts.

Questions too difficult for the respondents

Table 14.1 lists 19 examples where biased performance appears to be designed into an investigation by the use of problem questions. The questions are too difficult for the respondents, who are usually ordinary students. Some questions are virtually impossible for anyone to answer properly. The table is divided into 6 parts, according to the source of the difficulty.

Impossible tasks

Part 1 of Table 14.1 lists 4 questions that are impossible to answer satisfactorily. Questions 1.1, 1.2 and 1.3 (Fischhoff and Slovic, 1980, Experiments 1,3,5 and 6; see Chapter 3, Impossible perceptual tasks) are problems because they can only be answered correctly by responding with a probability of .5 all the time. Paid volunteers with unspecified backgrounds are given 2-choice tasks that are impossible to perform better than chance.