Recent advances in processing negation

Abstract

Negation is a complex linguistic phenomenon present in all human languages. It can be seen as an operator that transforms an expression into another expression whose meaning is in some way opposed to the original expression. In this article, we survey previous work on negation with an emphasis on computational approaches. We start defining negation and two important concepts: scope and focus of negation. Then, we survey work in natural language processing that considers negation primarily as a means to improve the results in some task. We also provide information about corpora containing negation annotations in English and other languages, which usually include a combination of annotations of negation cues, scopes, foci, and negated events. We continue the survey with a description of automated approaches to process negation, ranging from early rule-based systems to systems built with traditional machine learning and neural networks. Finally, we conclude with some reflections on current progress and future directions.

1. Introduction

Negation is a complex linguistic phenomenon present in all human languages, allowing for the uniquely human capacities of denial, contradiction, misrepresentation, lying, and irony (Horn 1989). Broadly speaking, negation “relates an expression e to another expression with a meaning that is in some way opposed to the meaning of e” (Horn and Wansing 2017). The key challenge to understanding negation is thus to figure out the meaning that is in some way opposed to e—a semantic and highly ambiguous undertaking that comes naturally to humans in everyday communication. As described by Lawler (2010), cognitively, negation “involves some comparison between a real situation lacking some particular element and an imaginal situation that does not lack it.”

From a linguistic perspective, it is generally understood that negation has scope and focus. Scope is “the part of the meaning that is negated” and focus “the part of the scope that is most prominently or explicitly negated” (Huddleston and Pullum 2002). Consider statement (1). The falsity of any of the propositions in (1a–1d) makes statement (1) true, thus the scope of the negation includes all the elements indicated in the right column in (1a–1d):

1. 1. This camera doesn’t take good pictures in low light

1a. Something takes something in some condition. [verb] takes

1b. This camera takes something in some condition. [agent of taking, This camera]

1c. Something takes good pictures in some condition. [theme of taking, good pictures]

1d. Something takes something in low light. [manner of knowing, in low light]

Determining the focus consists in pinpointing which parts of the scope are intended to be interpreted as true and false given the original statement. Without further context, one may consider condition (1c) or condition (1d) to be interpreted as false, and conditions (1a, 1b, and 1d) or conditions (1a–1c) to be interpreted as true, respectively. The first option yields the affirmative interpretation This camera takes bad pictures in low light, and the second option yields This camera takes good pictures when there is enough light.

Propositions where a negation is present are generally understood to carry positive meaning, or in other words, to suggest an affirmative alternative. For example, John didn’t leave the house implicates that John stayed inside the house. Hasson and Glucksberg (2006) show that comprehending negation involves considering the representation of affirmative alternatives. While not fully understood, there is evidence that negation involves reduced access to the affirmative mental representation (Djokic et al. 2019). Orenes, Beltrán, and Santamaría (2014) provide evidence that humans switch to the affirmative alternative in binary scenarios (e.g., from not red to green when processing The figure could be red or green. The figure is not red). In such multi-scenarios, however, humans keep the negated representation unless the affirmative interpretation is obvious from context (e.g., humans keep not red when processing The figure is red, green, yellow or blue. The figure is not red.).

Previous work in computational linguistics has pointed out that processing negation is beneficial for some applications. In particular, sentiment analysis has a long tradition of incorporating components to process negation (Wiegand et al. 2010), ranging from handcrafted rules to sophisticated neural network architectures. For example, like generally carries positive sentiment, but not when modified by a negation cue (e.g., don’t like). Wilson, Wiebe, and Hoffmann (2005) introduce the idea of contextual polarity, and note that negation may intensify rather than change polarity (e.g., not good vs. not only good but amazing). Jia, Yu, and Meng (2009) present a set of heuristic rules to determine sentiment when negation is present, and Councill, McDonald, and Velikovich (2010) show that information about the scope of negation is beneficial to predict sentiment. More recently, Socher et al. (2013) present a recursive neural network for sentiment analysis that captures sentiment polarity changes when negation is present. Reitan et al. (2015) show that a state-of-the-art Twitter sentiment analysis benefits from incorporating information about the scope of negation.

Outside sentiment analysis, researchers have pointed out that negation poses unsolved challenges for, among others, machine translation and natural language inference. Fancellu and Webber (2015) present a manual error analysis translating negation from Chinese to English, and Bentivogli et al. (2016) point out that neural machine translation struggles as much as statistical machine translation when it comes to translating negation. The task of natural language inference—determining whether a text entails, contradicts, or is neutral with respect to a hypothesis—and in particular existing benchmarks (e.g., SNLI (Bowman et al. 2015), MNLI (Williams, Nangia, and Bowman 2018)), do not properly account for inference in the presence of negation. A couple of insights from the literature support this claim. First, Gururangan et al. (2018) point out that negation cues (nobody, never, no, nothing, etc.) are a strong indicator of contradictions in MNLI. Second, creating adversarial examples by adding straightforward negations make existing systems underperform. For example, Naik et al. (2018) concatenate the tautology “and false is not true” at the end of hypotheses and conclude that models are not robust to negation. Similarly, Wallace et al. (2019) introduce universal triggers, that is, sequences of tokens that fool models when concatenated to any input. They show that concatenating negation cues such as nobody, never, nothing, and none to hypotheses decreases accuracy to almost zero when the gold label is entailment or neutral.

Processing negation is crucial when extracting information from clinical and biomedical texts, since negation is very frequent and missing negated observations can lead to misleading conclusions that have impact in health care and biomedical research. For example, the medical relevance of the sentence “shows no symptoms of COVID-19” is quite different from the sentence “shows symptoms of COVID-19”. Negation detection systems, such as the NegEx algorithm (Chapman et al. 2001) and its new version (Mehrabi et al. 2015), have shown good performance in clinical text. However, as Wu et al. (2014) indicate, negation is not solved because “current solutions are easily optimizable but not necessarily generalizable. Negation detection is still a challenge when considered from a practical, multi-corpus perspective, that is, one in which an algorithm is deployed in many clinical institutions and on many sources of text.” The authors conclude that in order to improve the performance in negation detection, more manually annotated data are needed in corpora that reflect the distributions of negation in different domains and genres. Cohen et al. (2017) find out that there are differences in the distribution of negation between clinical reports and biomedical articles. For example, there is more affixal negation in the clinical texts, whereas affixal level is more frequent in journal articles.

This special issue is devoted to advances in processing negation from a computational perspective. Section 2 provides information about existing corpora where negation is annotated and shared tasks. Section 3 presents previous work on processing negation, mainly on scope and focus identification.

2. Annotated corpora and shared tasks

That the interest in processing negation has grown in the last years is reflected in the number of corpora that have been published. Fancellu et al. (2017b) group corpora that contain some representation of negation into two types. The first-type corpora represent negation in a logical form, using quantifiers, predicates, and relations. One of them is the Groningen Meaning Bank (Basile et al. 2012), ^a a collection of semantically annotated English texts with formal meaning representations. It is composed of newswire texts from Voice of America, country descriptions from the CIA Factbook, a collection of texts from the open ANC, and Aesop’s Fables. Another corpus is the DeepBank corpus (Flickinger, Zhang, and Kordoni 2012), ^b which contains rich syntactic and semantic annotations for the 25 Wall Street Journal sections in the Penn Treebank (Taylor, Marcus, and Santorini 2003). The second-type corpora use string-level annotations, where negation information (cues, scope, negated event, and focus) are annotated on spans of text (e.g., BioScope (Vincze et al. 2008), ConanDoyle-neg (Morante and Daelemans 2012)). Jiménez-Zafra et al. (2020) recently surveyed corpora annotated with negation focusing on corpora that contain string-level negation. They describe the main features of the corpora based on the following criteria: the language, year of publication, domain, the availability, size, types of negation taken into account (syntactic and/or lexical and/or morphological), negation elements annotated (cue and/or scope and/or negated event and/or focus), and the way in which each corpus was tokenized, the annotation guidelines, and annotation scheme used. Information is provided about existing corpora for the following languages: English, Spanish, Swedish, Dutch, Japanese, Chinese, German, and Italian.

Corpora annotated with negation information usually include at least negation cues and either scope or focus. Some corpora also include event-level annotations indicating whether an event is negated. We note, however, that negation is ubiquitous in text. Jiménez-Zafra et al. (2020) report that the number of sentences with negation in the English corpora is between 9.37% and 32.16%, while in Spanish corpora it is between 10.67% and 34.22%, depending on the domain. Thus, virtually all corpora include examples of negation. For example, parallel corpora for machine translation, product reviews annotated with sentiment, and corpora annotated with coreference chains include sentences with negation even though the negations themselves are not explicitly annotated.

Early work on annotating negation targeted the biomedical domain, for example, explicit negation of biological relationships (Pyysalo et al. 2007), negated events (Kim, Ohta, and Tsujii 2008), and negation cues and their scope (Vincze et al. 2008). Negation cues and scope have also been annotated in product reviews (Councill et al. 2010; Konstantinova et al. 2012), short stories (Morante and Daelemans 2012), and dialogs within intelligent tutoring systems (Banjade and Rus 2016). Liu, Fancellu, and Webber (2018) present a parallel corpus of English and Chinese annotated with negation. For Spanish, Sandoval and Salazar (2013) present annotations of negation cues and scope in news articles. Cruz et al. (2017) present similar annotations in the medical domain and include syntactic, morphological, and lexical negation. The latest and largest corpus (9455 sentences) is the SFU corpus by Jiménez-Zafra et al. (2018b), who annotate product reviews for a variety of product categories with, among others, negation cues and scopes. Negation markers and either events or scopes have also been annotated in Swedish (Dalianis and Velupillai 2010), Dutch (Afzal et al. 2014), Chinese (Zou, Zhou, and Zhu 2016), German (Cotik et al. 2016a), and Italian (Altuna, Minard, and Speranza 2017).

Regarding focus of negation, existing corpora work primarily with English texts. PB-FOC (Blanco and Moldovan 2011; Morante and Blanco 2012) was the first corpus to annotate the focus of negation and targets the negations marked with argm-neg semantic role in PropBank (Palmer, Gildea, and Kingsbury 2005). All negations annotated in PB-FOC are verbal negations, that is, the negation cues (e.g., not, never) are grammatically associated with a verb. The authors choose as focus the semantic role of the verbal negation that is most prominently negated. This decision simplifies the annotation task since usually a few semantic roles span many tokens. Anand and Martell (2012) reannotate PB-FOC and argue that (a) PB-FOC overannotates foci, and (b) alternatives arising from scalar implicatures and neg-raising predicates should be separated from those arising from focus detection. Other efforts working with English texts also refine the annotations in PB-FOC. Blanco and Moldovan (2012) define fine-grained focus as the tokens that are intended to be negated, in contrast to the coarse-grained foci annotated in PB-FOC, which are always all tokens belonging to a semantic role. They reannotate the foci in PB-FOC whose syntactic constituents are a PP, ADVP, or SBAR and discover that 68% of them can be refined. Blanco and Sarabi (2016) present an alternative annotation schema. Instead of choosing as focus the semantic role that is most prominently negated, they consider all roles and rank the likelihood of the underlying positive interpretations with a score ranging from 0 to 5. Sarabi and Blanco (2016) move away from semantic roles and work with syntactic dependencies. By selecting subtrees in dependency trees, they target foci of verbal negations—and the underlying positive interpretations—in a continuum of granularity. Finally, Sarabi and Blanco (2017) follow a similar approach manipulating syntactic dependencies, but target negation cues that modify nouns or adjectives instead of verbs.

In addition to PB-FOC and the refinements aforementioned, there are two corpora with focus annotations. Banjade and Rus (2016) work with dialogs retrieved from interactions between students and an intelligent tutoring system. Their corpus, DT-neg, contains annotations of negation cues as well as their scope and focus. The authors note that the scope and focus are often in a previous dialog turn. Kolhatkar et al. (2019) work with a collection of news articles and user-submitted comments online. Their corpus, the SFU Opinion and Comments Corpus (SOCC), contains annotations of negation cues as well as their scope and focus, among other annotations. To our knowledge, these two corpora are the only ones that work with user-generated content in English. Finally, to our knowledge, there is only one corpus annotating the focus of negation in languages other than English (Matsuyoshi, Otsuki, and Fukumoto 2014). This corpus includes news articles and user reviews written in Japanese and targets negation cues and their foci.

Several shared tasks have addressed negation processing in English: in the BioNLP09 Shared Task 3 (Kim et al. 2009) and in the the i2b2 NLP Challenge (Uzuner et al. 2011), systems had to detect negated events. The ^*SEM 2012 Shared Task (Morante and Blanco 2012) was devoted to processing the scope and focus of negation, and in the ShARe/CLEF eHealth Evaluation Lab 2014 Task 2 (Mowery et al. 2014) participants had to detect whether a disorder was negated. For Spanish, there have been two editions of the NEGES task (Jiménez-Zafra et al. 2018a; 2019), where systems had to detect negation cues and evaluate the impact of negation in sentiment analysis.

3. Processing negation

The first attempts to process negation in English were mostly rule-based and focused on finding negated terms in clinical texts domain (Chapman et al. 2001; Mutalik, Deshpande, and Nadkarni 2001; Goldin and Chapman 2003; Auerbuch et al. 2004; Elkin et al. 2005; Boytcheva et al. 2005; Goryachev et al. 2006; Sanchez-Graillet and Poesio 2007; Huang and Lowe 2007; Rokach, Romano, and Maimon 2008). Some rule-based systems were developed based on lists of negations and stop words (Mitchell et al. 2004; Mykowiecka, Marciniak, and Kupść 2009; Harkema et al. 2009; Uzuner, Zhang, and Sibanda 2009; Sohn, Wu, and Chute 2012). With the surge of opinion mining, negation was studied as a marker of polarity change (Das and Chen 2001; 2005b; Polanyi and Zaenen 2006; Taboada et al. 2011; Jiménez-Zafra et al. 2017). Only with the release of the BioScope corpus (Vincze et al. 2008) did the work on negation receive a boost. But even so, despite the existence of several publications that focus on negation, it is difficult to find a negation processor for languages other than English. For English, some systems are available for processing clinical documents: NegEx (Chapman et al. 2001), ConText (Harkema et al. 2009), Deepen (Mehrabi et al. 2015), NegBio Peng et al. (2018), and NegTool (Enger, Velldal, and Øvrelid 2017).

3.1 Scope of negation identification

Compared to focus identification, scope identification has received substantially more attention, due to the existence of several corpora annotated with scopes. Inspired by the annotation of scopes in the BioScope corpus, the task of detecting negation scopes was introduced as a machine learning sequence labeling task (Morante, Liekens, and Daelemans 2008; Morante and Daelemans 2009). Subsequently, three main types of approaches have been applied to processing negation:

• • Rule-based systems, in an attempt to improve the NegEx algorithm, such as ConText (Harkema et al. 2009), DEEPEN (Mehrabi et al. 2015), and NegMiner (Elazhary 2017). Packard et al. (2014) present an approach that works over explicit and formal representations of propositional semantics, deriving the scope from the structure of logical-form meaning representations.

• • Machine learning techniques (Agarwal and Yu 2010; Li et al. 2010; Cotik et al. 2016b). Read et al. (2012) proposed an SVM-based ranking of syntactic constituents to identify the scope. Velldal et al. (2012) explore a combination of deep and shallow approaches to the problem of resolving the scope of speculation and negation. Reitan et al. (2015) apply a negation scope detection system to Twitter data.

• • Deep learning approaches (Qian et al. 2016; Ren, Fei, and Peng 2018; Lazib et al. 2018). Fancellu, Lopez, and Webber (2016) present neural networks for this task, and Fancellu et al. (2017a) present an error analysis showing that scope is much easier to identify when delimited by punctuation.

3.2 Focus of negation identification

Automated systems to predict the focus of negation as annotated in PB-FOC are built using traditional machine learning algorithms as well as sophisticated neural networks. Blanco and Moldovan (2011) build a decision tree with a feature set emphasizing semantic role information. Rosenberg and Bergler (2012) present a rule-based approach implementing several heuristics. Zou, Zhou, and Zhu (2014) introduce a graph model for focus identification that takes into account inter-sentential information in addition to intra-sentential information. Doing so allows them to consider context from a lexical and topical perspective. Sarabi and Blanco (2019) present a simpler approach to include contextual information: they train an SVM with features derived from an off-the-shelf rhetorical structure theory (Mann and Thompson 1987) discourse parser (Surdeanu, Hicks, and Valenzuela-Escárcega 2015). To date, the best results published are presented by Shen et al. (2019), who obtain 70.51 accuracy. They present a BiLSTM with a CRF layer and attention mechanisms defined at the word and topic level.

There are less efforts experimenting with the other corpora annotating focus of negation. Most use an SVM trained with lexical, syntactic, and semantic features (Blanco and Moldovan 2012; Blanco and Sarabi 2016; Sarabi and Blanco 2016; Sarabi and Blanco 2017). Matsuyoshi et al. (2014) present 16 heuristic rules to detect the focus in Japanese. Finally, van Son et al. (2018) present a replication study of the corpus by Blanco and Moldovan (2012). They present results modeling the scoring class as regression and classification tasks and suggest that three labels (true, false, and uncertain) may be a better choice than a score ranging from 0 to 5.