4 - Annotating a corpus

Summary

For a corpus to be fully useful to potential users, it needs to be annotated. There are three types of annotation, or “markup,” that can be inserted in a corpus: “structural” markup, “part-of-speech” markup, and “grammatical” markup.

Structural markup provides descriptive information about the texts. For instance, general information about a text can be included in a “file header,” which is placed at the start of a text, and can contain such information as a complete bibliographic citation for a written text, or ethnographic information about the participants (e.g. their age and gender) in a spoken dialogue. Within the actual spoken and written texts themselves, additional structural markup can be included to indicate, for instance, paragraph boundaries in written texts or overlapping segments of speech in spoken texts. Part-of-speech markup is inserted by a software program called a “tagger” that automatically assigns a part-of-speech designation (e.g. noun, verb) to every word in a corpus. Grammatical markup is inserted by a software program called a “parser” that assigns labels to grammatical structures beyond the level of the word (e.g. phrases, clauses).

This chapter focuses on the process of annotating texts with these three kinds of markup. The first section discusses why it is necessary for corpora to be annotated with structural markup and then provides an overview of the various systems of structural markup that have evolved over the years as well as the various tools that have been developed for inserting markup in corpora.