Introduction

The ambiguity problem takes center stage in models of sentence processing because, to a first approximation, linguistic input is presented sequentially to the processing system in both spoken language and in reading. Thus, even unambiguous sequences of speech or text are briefly ambiguous. A rich empirical database now demonstrates that the processing system makes partial commitments that are closely time-locked to the input as it unfolds over time (for recent reviews see Tanenhaus & Trueswell, 1995, Pickering, Clifton, & Crocker, this volume). Because the syntactic information available to the system at any point in the sentence often underdetermines its structure, other constraints that are correlated with the syntactic alternatives could, in principle, provide information that is useful for syntactic ambiguity resolution.

For more than a decade, researchers in the sentence processing community have been relying on data from increasingly fine-grained measures of processing difficulty, especially eye-movement data, in an attempt to adjudicate among competing models of how correlated constraints are used in ambiguity resolution. But despite a rapidly growing experimental literature, an increasingly rich empirical base, and a plethora of explicit parsing models, the basic architectural issues remain unresolved. We argue that this state of affairs is likely to continue until researchers in the sentence processing community begin to develop explicit quantitative models that (a) identify and quantify the relevant sources of constraint and (b) specify mechanisms that link information integration to processing time.

Introduction

A central topic of debate in the sentence processing community has been whether or not the Human Sentence-Processing Mechanism (HSPM) is composed of a number of articulated modules (Frazier, 1987) or a single homogenous processing unit (Trueswell & Tanenhaus, 1994). This debate is still current; new modular (Frazier & Clifton, 1996; Crocker, 1996; Stevenson, 1994) and interactive (MacDonald, Pearlmutter, & Seidenberg, 1994, Tanenhaus, Spivey-Knowlton & Hanna, this volume) models are being proposed that can explain a larger portion of the data than their predecessors. The modularity question has been inextricably intertwined with our theories of human parsing strategies, since such issues of architecture will determine what kinds of representations and knowledge the parser can bring to bear in making decisions. Rational motivation for a modular parsing subsystem also derives from the fact that current theories of syntax typically posit a restricted and specialised representational framework. A further argument for modularity concerns computational complexity: A module which need only consider restricted knowledge in making decisions can function more rapidly that one which does not.

In this chapter we develop the latter, computational arguments further by considering the role of probabilistic mechanisms within a modular language comprehension system. While the use of frequency information has standardly been associated with nonmodular, interactive architectures (Spivey-Knowlton & Eberhard, 1996), we argue that the use of frequency-based heuristics is in fact a more natural ally of modular systems.

Introduction

As is evident from the other chapters in this book, ambiguity resolution is a major issue in the study of the human sentence processing mechanism. Many of the proposed models of ambiguity resolution involve the combination of multiple soft constraints, including both local and contextual constraints. Finding the best solution to multiple soft constraints is exactly the kind of problem that connectionist networks are good at solving, and several models have used them in one way or another. The difficulty has been that standard connectionist networks do not have sufficient representational power to capture some central properties of natural language (Fodor and Pylyshyn, 1988; Fodor and McLaughlin, 1990; Hadley, 1994). In particular, standard connectionist networks cannot represent constituency. Thus, they cannot capture generalizations over constituents, and in learning they cannot generalize what they have learned from one constituent to another. Since regularities across constituents are fundamental and pervasive in all natural languages, any computational model that predicts no such pattern of regularities cannot be adequate as a complete model of sentence processing. To address this inadequacy without losing their advantages, the representational power of connectionist networks needs to be extended.

This chapter discusses exactly such an extension to connectionist networks. Temporal synchrony variable binding (Shastri and Ajjanagadde, 1993) gives connectionist networks the ability to represent constituency, and thus to capture and learn generalizations over constituents.

The field of sentence parsing is currently blessed with intriguing theories of human sentence processing and with theorists who defend their theories with enthusiasm and dedication. This is far better than it was at the beginning of serious research into sentence parsing. The theory of derivational complexity was the best thing going, but it was a constantly moving target, whipping from one position to another as linguistic theory changed, and it always seemed possible to shoot it down wherever it went (see Gough & Diehl, 1978).

Things didn't get much better in the early 1970s, with the development of detective models of human sentence parsing (Fodor, Bever, & Garrett, 1914). Parsing theory was reduced to a list of “clues” that the parser might search for, and from which it might, in an associationistic fashion, project what kind of sentence it might be processing. Experimental research was dull, little more than the demonstration that this or that surface feature of sentences had something to do with comprehension (see Clark & Clark, 1977).

A turn for the better came in the mid-1970's with Kimball's principles (Kimball, 1973). Here were some explicit, economical statements of parsing preferences and decision rules that might have some broad applicability – something better than “look for a noun phrase after a transitive verb.” Frazier (1979) brought this approach to a new maturity with her proposal that a pair of strategies could account for a wide range of the phenomena that had previously been accounted for in only an ad hoc fashion.

This chapter is on syntactic and semantic processes during on-line language comprehension. We present event-related brain potential (ERP) data from a series of experiments using both written and spoken input, focussing mainly on sentence processing. The data are discussed in terms of the constraints that they impose on the architecture of the human language system, in particular with respect to the separation of syntactic and semantic knowledge bases.

The issue of the separation of different linguistic knowledge sources is an underlying theme in much psycholinguistic research and is especially relevant for a number of contrasting models on parsing. Although linguistic theory has to a large extent postulated distinct representational systems for structure and meaning, psycholinguistic models of the parsing process differ in their assumptions about the separability of syntactic and semantic knowledge bases and of the processing operations that tap into these bases.

One class of models – referred to as garden-path models in the literature – posits a separate syntactic knowledge base. This linguistic knowledge is used for the computation of a separate intermediate level of representation for the syntactic structure of a sentence. Garden-path models claim that the construction of an intermediate syntactic level is a necessary and obligatory step during sentence processing, if not for all structural assignments, then at least for a significant subset of them (e.g., Clifton & Ferreira, 1989; Frazier, 1987; Frazier & Clifton, 1996; Friederici & Mecklinger, 1996; Rayner, Garrod, & Perfetti, 1992).

This chapter discusses a range of findings from our eye-tracking research in relation to current theories of parsing. First, we discuss findings that implicate extremely rapid semantic processing and findings that suggest that the degree of semantic processing affects the ease of recovery from misanalysis. We then provide an analysis of theories of initial parsing decisions that emphasises two dimensions: whether they can draw upon all available information or not; and whether they attempt to select or foreground the analysis that is most likely to be correct at the point of ambiguity. We next discuss a range of evidence from our laboratory that provides support for theories that do not attempt to select the most likely analysis. The evidence may support an account where at least some parsing decisions are driven by syntactic information alone. Alternatively, it may support an account in which the parser draws upon all available information, but where its goal is to select the most informative analysis, not the most likely one.

Immediate Effects of Plausibility

One of the clearest conclusions of research into language comprehension is that a great deal of processing occurs incrementally. Thus, lexical access, syntactic analysis, and semantic interpretation are not delayed, but begin as soon as the relevant word is heard or read (e.g., Marslen-Wilson, 1973; Just & Carpenter, 1980). The strongest of these findings is that the processor does not delay in providing a semantic interpretation for a sentence fragment as a whole: such semantic processing of course requires that lexical and syntactic processing have already occurred.

Background

One of the most intriguing questions about the use of reference in language has to do with the multiplicity of possible forms that can be used to successfully identify a referent in a given situation. For example, a certain dog can be referred to by the definite descriptions the dog or the animal, by the proper name Fido, or by the pronoun he. Clearly, referential forms are not used randomly, and furthermore, their distribution exhibits some consistent patterns within and also across languages (Ariel, 1990; Givón, 1976). The question is then, what constraints underlie the distribution of referential forms and more specifically, what psychological mechanisms are involved in the production and resolution of referential expressions? The study of this question spans several disciplines which emphasize different aspects of the relevant issues.

Research in cognitive psychology on conceptual representation and categorization focuses on the question of what makes a certain expression more likely to be used for naming a certain object or event in the world (e.g., Rips, 1989; Rosch, 1978; Rosch & Mervis, 1975). The emphasis in this research is on how people's mental representation of world knowledge underlies naming and categorization. In contrast, much research in linguistics and psycholinguistics focuses on the study of anaphoric expressions that are linked to referents by being coreferential with another linguistic expression, the antecedent. As in the case of referring expressions in general, there is more than one anaphoric form that can be used to successfully identify the antecedent.

Introduction

An important goal in psycholinguistics is uncovering the architecture of human sentence comprehension. Most of the important issues in the field of sentence processing are questions about some aspect of the underlying architecture – for example, the relation of grammar and parser (Chomsky, 1965,1980; Miller, 1962; Stabler, 1991), the modularity of syntactic processing (Ferreira & Clifton, 1986; Fodor, 1983; Forster, 1979; Frazier & Clifton, 1996; Just & Carpenter, 1992; Mitchell et al., 1992; Rayner et al., 1992), the number of interpretations pursued in parallel (Clark & Clark, 1977; Gibson, 1991; Gorrell, 1987; Kurtzman, 1985; MacKay, 1966), or the relation of linguistic processing to central cognition (Forster, 1979). The first goal of this chapter is to define precisely what it means to have a functionally complete sentence-processing architecture. The view of architecture developed here draws on general work on architecture in cognitive science (Anderson, 1983; Newell, 1973, 1980a, 1990; Newell & Simon, 1972; Pylyshyn, 1984), rather than focusing exclusively on the issue of modularity that has dominated discussions in sentence processing. This architectural analysis defines a set of functional constraints on theories, sharpens the set of questions that need to be answered, and reveals that some common theoretical approaches, including modularity, are incomplete in significant ways. The claim is that theories of human sentence processing should take the form of complete computational architectures.

Introduction

In psycholinguistic research, models of sentence processing prevail which explain human parsing preferences with a set of universal principles derived either from general features of the architecture of the human language processor (e.g., Frazier & Fodor, 1978; Konieczny, 1996), or universal features of grammar (e.g., Pritchett, 1992; Gorrell, 1995), or combinations of both (Gibson, 1991). These accounts usually see the human language processor as a modular subsystem (or a set of subsystems, e.g., Frazier, 1990a) of the cognitive machinery (Fodor, 1983). As Pickering, Crocker, & Clifton (this volume) point out, psycholinguists try to uncover the basic architectures and mechanisms of this module, or more pointedly, the natural laws of the human language processor. These laws, i.e., the basic principles of language processing, should be valid across individuals as well as across languages. The simplest interpretation of this approach to sentence processing implies that similar constructions should be parsed in similar ways irrespective of interindividual or interlingual differences.

This approach has been challenged by a series of experiments on relativeclause attachment in sentences like (1), whose results were first published by Cuetos and Mitchell (1988). Cuetos and Mitchell provided evidence for a marked preference to attach the relative clause to the first of the two potential hosts in Spanish, whereas in English, relative clauses were preferentially attached low.

Introduction

In psycholinguistics, the investigation of grammatical aspects of interpretation has lagged behind the study of parsing. Ambiguity in the interpretation of a phrase has not generally been studied except in the case of lexical ambiguity. The present chapter will explore the preferences perceivers exhibit when a phrase might be interpreted in more than one way, focusing on the interpretation of Determiner Phrases (DPs). This will allow us to address the issue of whether the referential interpretation of a DP and its thematic interpretation must go hand in hand, occurring simultaneously, or whether these interpretation processes are initially (potentially) distinct. What is meant by “early” interpretation of a phrase (see the introductory chapter of this volume) is completely unclear until this issue of interpretive separability, or non-separability, has been resolved.

Two recent developments in linguistic theory open up a promising avenue to pursue in the investigation of semantic interpretation. The first development, critical to the arguments presented here, is Diesing's (1992) Mapping Hypothesis, which establishes a constraint between syntax and interpretation. DPs outside VP (at L(ogical) F(orm), in English) are presupposed or quantificational; the VP is identified as the “nuclear scope,” where asserted and thus focal information typically appears. Hence, only DPs within VP get an existential interpretation – essentially an interpretation where a new entity (one not presupposed in context or by shared background assumptions of the speaker and hearer) is introduced into discourse.

The architectures and mechanisms underlying language processing form one important part of the general architectures and mechanisms underlying cognition. In this book, the contributors largely focus on the question of what architectures and mechanisms underlie sentence comprehension, but it is our belief that their contributions illuminate the general nature of human language processing in the context of cognition as a whole. Because of the scope of these contributions, this introduction primarily concentrates on sentence comprehension. However, our perspective is to try to use evidence from sentence comprehension to understand the overall nature of language processing.

Let us first try to explain what we mean by architectures and mechanisms, and why these are important. We assume that the human language processor is a particular kind of computational device, and as such needs an organisation that we call its architecture. For instance, the processor might have two components, one dealing with word recognition, and one dealing with putting different words together into sentences. These processes might be completely distinct, in that the second processor could employ the output of the first processor, but have no impact on its internal workings. If so, we would know something about the architecture of the language processor. Architecture, therefore, refers to the static arrangement of components.

In contrast, the mechanisms are the dynamic workings of the language processor.