5 - Comparing Study Designs and Down-Sampling Strategies in Corpus Analysis

Summary

This chapter throws into relief the importance of the link between corpus hits and their sources (i.e. texts or speakers) by comparing study designs uninformed and informed by such metadata. The authors’ argument draws attention to the consequences of uneven distributions of observations across the basic text units of a corpus. In the case study (a distributional analysis of the use of actually in the BNCs of 1994 and 2014), it is demonstrated that disproportionate word counts contributed by individual sources (in this case, speakers) will distort estimates for relevant subsections of corpus data (in this case, demographic groups defined by age and gender) if the analysis assigns the same weight to every observation. The proposed solution is to factor in the text (or speaker) level, but this hinges on the availability of the relevant metadata. Moreover, insights into the hierarchical structure of corpus data are shown to benefit the design stage of a study. Thus, if manual post-processing steps preclude an exhaustive analysis of corpus hits, insights into the organization of data points can direct down-sampling strategies to generate a statistically efficient subset of tokens.