Traditional statistical testing, sometimes called null hypothesis significance testing (NHST), and the use of p-values has come under strong criticism. This chapter looks at the social issues of achieving significance in statistics that has led to the problems of NHST. It also discusses, though, how the framework of severe testing provides a way to understand NHST as a means of uniting experiments, statistics and evidence for research ideas. When viewed this way, NHST can still be an important approach to data analysis in HCI.

Bayesian statistics are often presented as a better, modern alternative to the Frequentist approaches centred around NHST and the resulting obsession with statistical significance. This chapter outlines the basic ideas of Frequentist and Bayesian statistics. It raises critiques of the Frequentist approach but also points out constraints on the Bayesian approach that are often omitted or overlooked. In particular, the chapter discusses how both Bayesian and Frequentist approaches rely on a move from statistical hypotheses to substantive hypotheses that cannot be justified by consideration of the statistics alone. Instead, both approaches can lead to sound knowledge through a care for data analysis, tied to the experiments that generate the data.