Appendix 3 - Estimating text size in the newspaper archives and the World Wide Web

Summary

To estimate the amount of text analyzed in searches of digital newspaper archives or the Web, it is possible to extrapolate from frequencies in large corpora whose size is known. After some experimentation it was decided to extrapolate not from the frequencies of individual lexical items but from selected medium-frequency collocations which could with good reason be assumed to

• not vary across regional varieties of English,

• be diachronically stable (i.e. not involved in ongoing processes of diachronic change), and

• be relatively independent of register, topic, and textual genre.

Table A3.1 gives the frequencies for ten such diagnostic collocations in the BNC (c. 100 million words), the publicly accessible portions of the Bank of English corpus (> 200 million), and in seven annual compact discs of The Guardian on CD-ROM, which – since 1994 – has also included the Observer.

This table shows several things. First, as expected, the frequencies for the Cobuild Corpus are consistently higher than for the BNC, which proves that the ten collocations are indeed fairly good diagnostics. Second, the amount of text available on each compact disc has grown steadily over the years, and not only because from 1994 the discs have included the Observer. Third, and somewhat unfortunately, however, trends for individual collocations vary considerably. Thus, estimates arrived at on the basis of deep breath diverge from those arrived at on the basis of heavy rain or coming year.