4 - Data collection and qualitative analysis

Summary

Designing an experiment is more than creating stimuli and tasks and deciding on the participant experience. Before conducting the experiment, the exact form of data to be collected needs to be decided, and importantly, it needs to be confirmed as sufficient for answering the research question.

This chapter focusses primarily on data collection. It describes the different types of data that can be collected for different purposes and the means of collecting it.

We make the traditional distinction between quantitative data (represented by numbers; e.g., the number of errors, a preference ranking) and qualitative data (not represented by numbers; e.g., a verbal description of problems encountered in performing the task, a video showing interaction with an interface).

In practise, there are two distinct decisions to be made about data:

• What data to collect (a decision made in advance of the experiment), and

• How to analyse the data (a decision made after the experiment has been run).

These two decisions are inextricably linked because the potential means of analysis will influence the decision on what data to collect. Any discussion about data collection therefore necessarily entails discussion on how it will be analysed.