TOWARDS A “TEXT AS DATA” APPROACH IN THE HISTORY OF ECONOMICS: AN APPLICATION TO ADAM SMITH’S CLASSICS

Abstract

Quantitative techniques have received increasing attention in the history and methodology of economics. Nonetheless, a “text as data” approach has mostly been overlooked and its applicability to the history of economics remains to be examined. To understand what we gain from such quantitative techniques in relation to existing historical analyses, we apply some “text as data” techniques to Adam Smith’s The Theory of Moral Sentiments and The Wealth of Nations. We explore the books’ topics, styles, and sentiments. We show how word frequency analysis can be used to examine the differences between the books, shed light on conceptual discussions, and reveal an important stylistic aspect, specifically Smith’s use of personal pronouns. Style analysis shows the similarities and differences in terms of lexical richness and readability between the two books. We also show the limitations of a third technique, sentiment analysis, when applied to historical economic texts.

I. INTRODUCTION

The fields of the history and methodology of economics have recently experienced a “quantitative turn.” The quantitative turn designates researchers’ increasing use of tools and methods such as bibliometrics, prosopography, network analysis, topic modeling, and text mining (Claveau and Gingras 2016; Wright 2016; Claveau and Herfeld 2018; Geiger and Kufenko 2018; Svorenčík 2018; Baccini 2020; Edwards 2020). ¹ Methods for text analysis especially have gained popularity in other fields such as sociology (Evans and Aceves 2016), political science (Grimmer and Stewart 2013), finance (Gupta et al. 2020), and the broader field of the digital humanities.

We adopt in this article a “text as data” perspective, which entails a shift in how researchers consider text. In a recent contribution, Kenneth Benoit explained that “the essence of treating text as data is that it is always transformed into more structured, summary and quantitative data to make it amenable to the familiar tools of data analysis” (Benoit 2020, p. 463).

Economists have notably embraced the text as data approach. An example is the recent article “Text as Data” by Matthew Gentzkow, Bryan Kelly, and Matt Taddy (2019), published in the Journal of Economic Literature. The authors “provide an overview of methods for analyzing text and a survey of current applications in economics and related social sciences” (Gentzkow, Kelly, and Taddy 2019, p. 537). They note, “The rise of text analysis is part of a broader trend toward greater use of machine learning and related statistical methods in economics” (p. 570). In other words, the text as data approach is becoming an influential methodological approach in economics and will also likely become an important new element in the toolbox of historians and methodologists of economics. According to Benoit (2020), the reasons for the growing popularity of the text as data approach are the increase of computational power, the huge volume of texts available in digital format, and the developments of quantitative measures and tools specifically dedicated to text analysis. That historians and methodologists of economics have rarely used text mining can, at first sight, appear as curious; in fact, their research consists in extracting and creating knowledge from what economists produce, which is—at a very fundamental level—text (see also Komine and Shimodaira 2018).

One reason for this low interest or use arguably lies in the lack of systematic training of historians and methodologists of economics on the quantitative and computational tools needed for the quantitative analysis of texts. This is especially true when considering that some software applications require coding skills and can have, therefore, a significant cost of entry. We believe this will likely change in the next few years and that, as in political science (Grimmer and Stewart 2013), knowing how to automatically access and extract information from (digital) texts will be a valuable competency for researchers in the field.

Another reason that can explain why scholars in the history and methodology of economics have not yet embraced the text as data approach is the lack of research examining its applicability to historical economic documents.

We contribute to filling this gap by asking: What kind of questions could historians of economics investigate with text mining? Are those techniques safely applicable in our field, given the specificities of the texts examined? How do quantitative results compare with those obtained using the traditional approach? ² To answer, we adopt a case-study approach, applying text as data methods to Adam Smith’s books The Theory of Moral Sentiments (1759, TMS) and The Wealth of Nations (1776, WN). We examine the two books through the lenses of their topics, styles (lexical richness and readability), and sentiments. We chose Smith’s books because they are classics in the field, and because there are countless scholarly contributions about the relationship between them (e.g., Montes 2003, 2004). This enables us to compare the results of the quantitative and qualitative approaches more easily.

Our article can be read from three different perspectives. First, it is a showcase of the potential of the text as data approach in the history of economics. Second, readers can use our discussions of the strengths and limitations of different techniques to orient their own methodological choices. Third, the adoption of a quantitative approach sheds new or complementary light on topics discussed in Smith scholarship.

The studies by James Gherity (1993), Vivienne Brown (1994), Daniel Klein and Michael Clark (2011), Jeffrey Binder and Collin Jennings (2016), Hye-Joon Yoon (2017), and Johan Graafland and Thomas Wells (2021) are of special interest for us. Gherity (1993) claimed that a text published in The Scots Magazine in 1763 was authored by Smith. To support his claim, he complemented the qualitative analysis with the use of quantitative measures of style (such as indicators of sentence lengths and collocations). In her 1994 book Adam Smith’s Discourse: Canonicity, Commerce and Conscience, Brown “attempts a reading of Smith’s works that engages with the complexity of the texts, and with their stylistic, figurative and rhetorical forms” (Brown 1994, p. 3). We will examine Brown’s arguments later in the text. Klein and Clark (2011) study “the place of language denoting musicality or synchrony in the works of Adam Smith” (Klein and Clark 2011, p. 413). They show that while such language is present in TMS, it basically disappears in WN. They contend that “Smith invokes many modes of common experience in developing the idea of coordinated sentiment. Synchrony, however, is certainly central” (Klein and Clark 2011, p. 419). Yoon (2017) has also offered in his book The Rhetoric of Tenses in Adam Smith’s The Wealth of Nations an “investigation of the grammatical features” (Yoon 2017, p. 1) of WN. Both Brown and Yoon adopted, however, a qualitative “text as text” approach. Binder and Jennings (2016) have used topic modeling techniques to study the similarities between the main text of the 1784 edition of WN and its index. Graafland and Wells (2021) have used a semantic network data-mining approach to examine the role of virtues in WN. They contribute to Smith scholarship by “applying new digital research and visualization techniques.… These techniques allow us to analyze what Smith meant by examining empirically (quantitatively) what he said” (Graafland and Wells 2021, p. 32). They “show with more quantitative precision than traditional scholarship that the invisible hand reading dramatically misrepresents both the nuance and the sum of Smith’s analysis” (Graafland and Wells 2021, p. 31). The authors acknowledge that, although they focused exclusively on WN for their purpose, “one of the possibilities of these new quantitative methods is to allow a comparative analysis between an author’s texts” (Graafland and Wells 2021, p. 34). We adopt such a comparative stance in this article.

Considering that the text as data approach represents a novelty for many scholars in the fields of the history and methodology of economics, we provide, in an online appendix, further details about our methodological choices and some complementary graphical representations.

II. DATA COLLECTION AND PREPROCESSING

We downloaded WN from the Project Gutenberg website (1784 or 1786 edition) and TMS (1790 edition) from the University of Oxford Text Archive. ³ To conduct the text analysis, we used R 4.1.3 (R Core Team 2022). We also used several R packages, some specifically dedicated to text analysis: the quanteda family of packages, version 3.2.1 (Benoit et al. 2018), and sentimentr version 2.9.0 (Rinker 2021).

We first cleaned the two documents by removing the headers and footers added to the original text by the hosting sites (for instance, the information related to the Gutenberg Project). We then converted the two documents into a corpus object, a specific R data type used to conduct textual analysis. To transform the text into data amenable to statistical study, a few further actions need to be considered: case handling, tokenization, stemming, and removal or not of stop words.

Case Handling

The behavior of several R functions we use can be affected by the upper or lower case of the text. As an example, some functions require capital letters to detect features in the text (such as names of people and places), or to split the text into sentences by looking for the period-space-capital-letter pattern. To guarantee the consistency of our results, we converted all our text to lower case before the analysis and used the original text with capital letters only when specific functions required it.

Tokenization

Tokenization is “the process of splitting a text into tokens. This is crucial for computational text analysis because full texts are too specific to perform any meaningful computations with” (Welbers, Van Atteveldt, and Benoit 2017, p. 250). Tokens are separated by spaces and punctuation and, therefore, “most often tokens are words, because these are the most common semantically meaningful components of texts” (Welbers, Van Atteveldt, and Benoit 2017, p. 250).

We decided to split hyphenated words. We did so because Smith makes frequent use of hyphens (in words such as “self-command” but also in constructs such as “praise-worthiness,” “fellow-creatures,” “fellow-feeling,” or in some instances where modern English would use a space, such as “dining-rooms”) and such use can affect some of our results, as we shall see.

Stemming

Stemming is the process of transforming a derived word into its root (i.e., transforming “thinking” or “thinker” into “think”). This step is often taken to reduce the dimensionality of data (and thus to reduce the computational resources needed for calculation) by aggregating similar words. We decided not to stem the tokens because we consider that all inflections of words must be investigated when studying an author’s style, especially regarding language diversity and readability.

Stop Words

We kept a flexible approach to the inclusion or removal of so-called stop words: “In topic modeling you will typically want to remove or stop-out high frequency words such as the, of, and, a, an, et cetera because these words carry little weight in terms of thematic or topical value” (Jockers and Thalken 2020, p. 220). We decided to keep or remove the stop words on a case-by-case basis, depending on the specific goals of each analysis we carried out. We will see in our topics analysis that keeping stop words can also bring interesting results.

After taking these preliminary steps, we start our analysis by producing some summary statistics of the two books. Table 1 shows the number of tokens, the number of types (unique tokens), the number of sentences, and the number of pages. Note that the numbers in Table 1 are calculated after cleaning the texts, i.e., by removing numbers, punctuation, and symbols, and splitting hyphenated words. The number of pages is included for reference and refers to the printed versions of the books we cite, not to the digital versions we used for our analyses. Overall, these data show only how WN is a much longer book than TMS, with over twice as many sentences and tokens. The difference in the number of types is comparatively smaller and we will see how this data is used to measure the lexical richness of the books.

Table 1. Number of Tokens, Types, Sentences, and Pages for Each Book

III. SMITH’S TOPICS

The text as data approach offers the powerful possibility of quickly scanning through long documents, or series of documents, and providing synthetic indicators that facilitate the identification of the topics covered in the text. In this section, we assume no knowledge of the books and provide examples of the methods that can be used for topics detection. We use different and complementary word frequency analysis methods and apply them at different scales of study. We then evaluate the results in the context of existing knowledge about Smith’s books.

Detection of the Most Relevant Topics

We apply three methods to reveal the main topics (and the relationships among them) of the books: weighted term frequency, keyness, and co-occurrences. These methods bring a complementary picture of the topics covered in the two books. Using the three methods together will also show how basic methodological choices (such as excluding or not the stop words) can affect the results.

The most intuitive way to find the most relevant words would be to determine the words with the highest frequencies in the two books (see Shimodaira and Fukuda 2014; Matsuyama 2016; Komine and Shimodaira 2018; and Komine 2019, who all focus on the most frequent nouns in the texts they analyze). ⁴ However, the most frequent words are usually language components that carry little context-specific meaning (such as the stop words discussed above), and this procedure treats books on their own, not as part of a broader corpus (made of the two books in our case). A more effective approach consists in determining the term frequency-inverse document frequency (TF-IDF) score for each token. Simplifying, the TF-IDF statistic measures the frequency of a token in a book (the TF component), correcting for the number of documents in the corpus that contain the token (the IDF component; see Silge and Robinson 2017).

Another, slightly different, method is that of keyness (e.g., Bondi and Scott 2010). Keyness is a measure of how characteristic a word is to one specific document, meaning that the word is especially frequent in that document (indicated as target), when compared with all other documents in the same corpus (indicated as reference). One approach (the one we use) to keyness is to calculate a chi-square statistic on the frequencies of a word in the target and reference document. We present in Table 2 the list of most characteristic words, detected using TF-IDF or keyness (with and without stop words).

Table 2. Most Characteristic Words for Each Book, According to Different Indicators

Assuming no prior knowledge of the two books, Table 2 shows that TMS is mostly characterized by words that belong to the sphere of emotions (such as “joy,” “sorrow,” “emotions,” “love,” and “anger”) and morality (such as “virtue” and “conduct,” or also “judgments” and “approbation”). “Sympathy” is the only word that appears among the most characteristic ones using both TF-IDF and keyness, which would prompt the analyst to investigate Smith’s use of this concept. The words that characterize WN are instead mostly economic notions (such as “price,” “money,” or “tax”). “Silver” is the only word that is detected among the most characteristic ones by both the methods we use.

Overall, from a methodological standpoint, both TF-IDF and keyness are successful in producing a concise picture of what the two books are about. We can also see how the two methods are complementary and how using them together can provide a more reliable and complete list of topics. In fact, the lists of most characteristic words that TF-IDF and keyness produce are different. For instance, words such as “joy,” “sorrow,” and “emotions” are among the most characteristic in TMS only when using TF-IDF. On the contrary, personal pronouns (which we further discuss below), “sentiments,” and “virtue” (and others) are among the most characteristic words only when using keyness. Similarly, “price,” “money,” and “labour” are detected among the most characteristic words in WN only when using keyness. We suggest using both methods in combination, especially when analyzing larger corpora of documents, when the specificities of the two methods (such as the need to choose a target and a reference for keyness, or the different impact of the IDF component for TF-IDF) would have a more pronounced effect.

An additional important finding is that, according to keyness, some stop words are characteristic to TMS (as opposed to WN): “we,” “our,” “us,” “he,” “his,” and “him.” This is uncommon, considering that stop words are normally frequent in most texts. In fact, these unusually frequent stop words are all personal pronouns and highlight an important stylistic difference between the two books. We will look deeper into this result later, as part of our analysis of Smith’s style. As we noted above, however, stop words are typically removed in natural language processing analysis, and thus an important feature of the books would go unnoticed by following that standard methodological practice.

Going beyond single words’ frequencies, further insights are gained by considering how words combine with one another. In fact, a word may have low frequency but be systematically associated with another one (i.e., appearing in the same sentence), for instance because they are part of a specific construct or expression, making the association relevant. We explore this possibility by calculating co-occurrences. We first remove the stop words and keep only the tokens that appear more than fifteen times in TMS and twenty times in WN (the different thresholds account for the different lengths of the books). We then calculate the phi (φ) coefficient for each pair of tokens present in the text. This coefficient is a correlation measure that scores how often two tokens appear in the same sentence versus how often they appear in different sentences. The coefficient varies between one (the two tokens always appear in the same sentence) and minus one (the two tokens never appear in the same sentence). We finally display the fifty most strongly correlated pairs using, as in Hiroyuki Shimodaira and Shinji Fukuda (2014), Naoki Matsuyama (2016), or Julia Silge and David Robinson (2017), a network representation (Figure 1 and Figure 2).⁵ We opted for a network representation instead of a standard table format because it is more suitable to displaying possible groups of words and how such words are interconnected.

Figure 1. A Network Graph of the Fifty Strongest Co-occurrences in TMS

Figure 2. A Network Graph of the Fifty Strongest Co-occurrences in WN

Some co-occurrences highlight relevant aspects related to the contents of the books. An important result for TMS is the detection of “impartial spectator” (top right corner of Figure 1). The two separate words do not occur frequently enough to appear among the most characteristic words we reported in Table 2. However, taken together, the two words form an important co-occurrence and thus a central expression in TMS. This is a well-known result but a crucial one if we assume no knowledge of the book. For TMS, we also find “self” and “command” (that appear as “self‑command” in the book—see bottom right corner of Figure 1). For WN, we find that “silver” (a word present also in Table 2) is often associated with “gold” (bottom center of Figure 2). Also, as seen on the bottom left part of Figure 2, Smith often discusses “Spain” and “Portugal” in the same sentence, or “East” “Indies” and “West” “Indies,” but in separate sentences (there is no line connecting “east” and “west”). We also see that “forts” and “garrisons,” two military terms, tend to appear together (center part of Figure 2), and that the “army” is usually associated with “standing” (forming the expression “standing army,” i.e., professional army) in Smith’s discussion (center right of Figure 2). In addition, the words “answering,” “occasional,” and “demands” (bottom left of Figure 2) are interconnected because they are part of the expression “answering occasional demands” that appears multiple times in WN.

Other co-occurrences are instead indicative of stylistic choices. We can see how Smith, in TMS, tends to include in the same sentence words that have opposite meaning: “right” and “wrong,” “pain” and “pleasure,” “approve” and “disapprove,” “beauty” and “deformity,” “merit” and “demerit” (Figure 1; this last co-occurrence even appears in the titles of several sections of Part II of the book). Arguably, this appears as a precise stylistic or rhetorical choice, possibly made to stress the contrast among the different points of view presented in the discussion.

As well, we can also use co-occurrences to identify recurring language patterns (possibly indicative of an author’s writing habits). An interesting one is the pair “former” and “latter,” which Smith uses very frequently in both TMS (bottom left of Figure 1) and WN (top left of Figure 2). Other examples in TMS are “one” and “another,” “greater” and “part,” and “go” and “along” (Figure 1).

We found co-occurrences to be an effective method in spotting relevant associations between topics and, especially, some of the author’s characteristic expressions. Notably, even the most attentive reader would struggle in keeping track of how frequently different words appear together and would probably miss at least some of the expressions we detected, especially those that are not very frequent.

Smith on War

We illustrate in this section an example of a specific conceptual discussion by examining how Smith discusses the notion of war (e.g., Minowitz 1989; Coulomb 1998; Aspromourgos 2007; Hill 2009; Paganelli and Schumacher 2019). The first observation is that the terms “war” and its plural appear more often in WN (61.7 total occurrences every 100,000 words) than in TMS (22.9 total occurrences every 100,000 words), suggesting that Smith is more interested in discussing war within the context of an economic discussion rather than in the context of moral philosophy.

To get more insights about the semantic context in which Smith writes about war, we apply keyness (as seen before) to each book separately, where the target is now the group of sentences that contain the terms “war” or “wars,” while the reference is the group of sentences that do not contain said terms. This approach allows us to detect if there are some words that are characteristic to those sentences in which Smith mentions war, in contrast to those in which he does not. We discuss our results below and provide their graphical representations separately (see online appendix, Figure 2 and Figure 3).

Figure 3. MATTR Values throughout TMS and WN

Results for TMS show that, when mentioning war, Smith tends to use other related terms such as “enemies” and “faction.” Characteristic words are also “hazards,” “hardships,” and “tumult,” an indication of the struggles that Smith associates with war. It is noteworthy that words associated with war in TMS are not specifically “moral.” One caveat to these conclusions is that there are only a few sentences in TMS that mention war (we showed earlier that they were indeed more frequent in WN). In sentences that do not mention war, the list of characteristic words is not particularly meaningful.

Concerning WN, we see that “funding,” “contracted,” and “debt” are frequently mentioned in association with war. Hence, the economic costs associated with war are especially relevant in Smith’s discussion (see also, for instance, Paganelli and Schumacher 2019). In contrast, terms such as “price,” “labour,” and “market” are more characteristic to the passages where Smith does not examine war. However, chi-square values for these terms are very low, indicating that such terms are similarly frequent in sentences mentioning war or not.

An interesting observation concerns “peace.” The word appears to be characteristically associated with “war” in WN, but not in TMS. This seems to suggest a different use of “peace” in the two books. We can confirm this intuition by looking at the co-occurrences for “peace” in TMS (see online appendix, Table 1, for the co-occurrences of the word “peace” calculated for the whole book), and we find how it is more strongly associated with words such as “citizens,” “society,” or “order.” The quantitative results thus suggest that “peace” in WN is used as an antonym of “war.” This is not the case in TMS, where “peace” is typically (that is, not exclusively) used as a synonym of social harmony.

To conclude this section on topic analysis, the simple methods we presented are useful for researchers approaching a new, potentially large, body of (historical) literature. They enable a quick and, as we saw, effective summary of the topics treated in the documents examined and how they relate to each other. They can also be used to provide deeper insights into how an author framed a specific conceptual discussion.

IV. SMITH’S STYLE

That style matters for the interpretation of historical documents and especially for the study of Smith’s texts has been shown in Brown’s 1994 Adam Smith’s Discourse. Brown argues that “the stylistic forms of Smith’s texts are not homogeneous, and … this is indicative of their different ethical structures” (Brown 1994, p. 3). As Brown also puts it: “an exploration of the literary style of TMS is in some measure constitutive of understanding the moral argument itself” (1994, p. 26). In analyzing Smith’s style, Gherity (1993) distinguished between “soft” and “hard” elements of style. The “soft” elements are about the structure of an argument, the writing flow, the choice of a specific terminology reflecting an author’s writing habit or background knowledge, et cetera (Gherity 1993, pp. 250–257); while the “hard” elements are those that “can be measured and subjected to statistical analysis” (Gherity 1993, p. 257).

Adopting the text as data approach, we focus on the “hard” elements of style of TMS and WN. To do so, we measure the lexical richness and the readability of the two books. Further developing the results shown in the topics detection section, and complementing Brown’s (1994) work, we also compare the role of personal pronouns in the two books. We conduct our analysis on each section of the books. ⁶ To facilitate the comparison between the two books, which are of different lengths and structures, we use a percentage index to show the position of a section in a book.

Lexical Richness

Many indicators exist for the study of lexical richness (or lexical diversity). We focus on three of the most widely used, calculating them for each section of the books and comparing the results. We initially considered two indicators for measuring lexical richness: the type-token ratio (TTR) and the hapax richness. The TTR divides the number of unique or different words (types) by the total number of words (tokens) (Jockers and Thalken 2020). The approach of the hapax richness is similar to that of TTR, but with the focus shifted on words appearing only once. A word appearing only once is called a hapax legomenon and the hapax richness is the ratio of the number of hapax legomena to the total number of words of the text considered (Jockers and Thalken 2020). The idea for both indicators is the same: a text with a larger share of types or hapax legomena on the total number of tokens is lexically richer.

As we show in the appendix (online, section II), these two indicators are nevertheless not suitable for the study of texts of different lengths, as their values are related to the length of the text—a known result that we also find in our case. The effect would be that longer texts would typically be scored as lexically poorer. We therefore opted for another indicator that, by construction, is neutral to text length: the moving-average type token ratio (MATTR) (Covington and McFall 2010).

MATTR overcomes TTR’s limitations by calculating TTRs on a fixed length moving window of consecutive words, and finally using the mean of these TTRs as the final score. The researcher should decide on the length of the window (we use a window of 500 words as suggested by Covington and McFall 2010, p. 97).

Figure 3 shows MATTR scores for each section of the two books, applying a smoothing algorithm to better visualize trends. According to MATTR, TMS is lexically richer than WN.

How can we explain that WN is, according to MATTR values, generally lexically poorer than TMS? One hypothesis is that Smith adapted his style to the different audiences of the books, with philosophers being used to comparatively lexically richer texts. ⁷ It could also be that moral philosophy offers more opportunities for lexically richer discussions than economics. Jack Weinstein (n.d., in the Internet Encyclopedia of Philosophy) also argued, “As he grew older, Smith’s writing style became more efficient and less flowery.” If by “flowery” Weinstein means lexically rich, then our results support this conjecture. However, research has shown that lexical richness generally evolves during a writer’s lifetime (Smith and Kelly 2002 show that the language gets “richer” for two of the three authors they study). Our results show that Smith’s language did not become richer with time.

Readability

After a study of Smith’s Lectures on Rhetoric and Belles Lettres (see Lothian 1963), Brown (1994) has noted that Smith’s “writing appears easy and elegant, not obscure, but in spite of its own precept it was at times deeply metaphorical and intricately textured” (Brown 1994, p. 20). Are Smith’s books indeed easy to read? Is there a difference between TMS and WN? To answer these questions, we calculate readability measures for both books.

Two popular indicators to measure readability are the Flesch (1948) reading ease score and the Flesch-Kincaid (Kincaid et al. 1975) grade level score. Despite different formulations, both measures are based on the idea that a text with longer sentences (number of words per sentence) and longer words (number of syllables per word) are comparatively more difficult to read than a text with shorter sentences and words. Readability scores are thus about the structure of sentences and words, not their meaning. We provide details about the differences between the two indicators in the appendix (online, section III), while we focus below on the results of the Flesch-Kincaid grade level score. This indicator expresses readability as the grade level (based on the US school system) required to understand the text. The interpretation is thus straightforward: the higher the Flesch-Kincaid score, the more complicated the corresponding text is. We calculate the Flesch-Kincaid score for each section of Smith’s books and present our results in Figure 4.

Figure 4. Flesch-Kincaid Grade Level Scores throughout TMS and WN

We can see that both books generally require a level of education between grades fourteen and twenty to be read, where grade fourteen roughly corresponds to the second year of college education. Hence, according to this indicator, Smith’s books can be considered as difficult to read. ⁸ This contradicts, albeit from a purely quantitative perspective, Brown’s (1994) characterization of Smith’s style (see above). By a small margin, WN requires a lower grade of education than TMS. Once again, the intended audiences of the two books may have played a role in determining Smith’s stylistic choices, since we can hypothesize that eighteenth-century philosophers had a higher level of education than merchants of the same period.

Breaking down the Flesch-Kincaid score into its components sheds light on our results and offers additional insights into the readability differences between the two books. As we said, the Flesch-Kincaid score considers that a text is more difficult to read when it is characterized by long sentences and long words. The boxplots in Figure 5 show the distributions of the lengths of sentences and words in the books.

Figure 5. Boxplots of the Distribution of the Lengths of Sentences and Words in TMS and WN

Concerning the length of sentences, there is not much difference between TMS and WN, with sentences being mostly between thirty and forty words long. Concerning the length of words, we can see that TMS contains generally longer words: around 50% of sections in TMS have an average word length greater than 1.58 syllables, while only around 25% of sections in WN use similarly long words. It is reasonable to assume that the choice of words is, at least to a certain extent, driven by the topics examined, and that the differences in the lengths of the words between the two books is less of a stylistic choice than the length of sentences. Since this last indicator is practically identical in the two books, we can conclude that this aspect of Smith’s style did not change over time.

To conclude, the study of Smith’s two books through the lenses of readability and lexical richness indicators shows that TMS is marginally more complicated and lexically richer than WN. Noticeably, however, the length of sentences (which is arguably the more stylistic component of readability) is almost identical in the two books.

Smith’s Voices

We noted earlier, using keyness, that, compared with WN, TMS is characterized by the frequent use of certain stop words, all personal pronouns: “we,” “our,” “he,” and “his” (“us” is also a pronoun characteristic to TMS but is not in the list of stop words). This result is even more striking because these pronouns are more characteristic to TMS than moral words. Smith’s different use of personal pronouns in the two books thus appears to be worthy of investigation. We therefore examine in Figure 6 the relative frequencies of all personal pronouns in Smith’s two books. In line with the keyness results, we see that Smith used “I,” “you,” “he,” “him,” “we,” and “us” very frequently in TMS but not in WN. The overall picture is that the personal pronouns stand out more in TMS than in WN.

Figure 6. Relative Frequencies (%) of Personal Pronouns in TMS and WN

Showing again that the quantitative and qualitative approaches are complementary, we can frame our discussion of the above results by using some of the arguments presented in Brown’s 1994 Adam Smith’s Discourse. Brown has argued that the styles in TMS and WN are different and that these different styles are representative of their different “moral plane” (Brown 1994, p. 5). More specifically, Brown used Mikhail Bakhtin’s distinction between a dialogic and a monologic text:

Bakhtin argued that what he called novelistic discourse is characterised by a ‘dialogic’ style incorporating a range of ‘voices’ representing different cultural, political and axiological views, and within this category he included ethical discourses such as Stoic and other self-interrogative discourses. (Brown 1994, p. 4)

…

Bakhtin counterposed the dialogic form of novelistic discourse to its other: the monologic form of discourse, where a single unitary voice of authority or tradition is at work controlling the text (texts such as scientific discourse or what Bakhtin termed the epic form). (Brown 1994, p. 26)

Brown argued that TMS would be dialogic while WN would be monologic: “The openness and uncertainty of the text of TMS … is rendered stylistically in its multivocal dialogism. By comparison, WN stands as a largely single-voiced monologic text, with its expressed certainties and intellectual order” (Brown 1994, p. 37). Our analysis based on keyness and frequency of personal pronouns tends to confirm this result: the different “voices” (pronouns) are more frequent in (and more characteristic to) TMS than in WN, showing how TMS would be relatively more “multivocal” than WN.

Nevertheless, as Brown warned, a monologic text does not mean that there are no other voices but that the other voices are used to support the arguments of the main voice: “WN is not undifferentiated stylistically, but a crucial contrast with TMS is that these different kinds of authorial interventions do not challenge the formal meaning of the didactic voice” (Brown 1994, p. 38). Our results also confirm that WN is not “undifferentiated stylistically” in terms of voices. Indeed, even though relatively less frequent, all personal pronouns are also found in WN. This study of personal pronouns thus also highlights a limitation of the quantitative method. As we said, the distinction between the dialogic and monologic styles are not only about the presence of voices but about how they interact and give room to each other. Brown’s (1994) above argument can thus be fully appreciated only by careful reading of the books.

V. SENTIMENT ANALYSIS

We now apply sentiment analysis to Smith’s two books. Bing Liu (2020) defines sentiment analysis as “a field of study that aims to extract opinions and sentiments from natural language text using computational methods” (Liu 2020, p. xi). Sentiment analysis is also of particular interest for historians and methodologists of economics because of developments in the discipline with the “emerging field” of “sentometrics, which is a portmanteau of sentiment and econometrics” (Algaba et al. 2020, p. 513; see also the sentometrics R package by Ardia et al. 2021).

Sentiment analysis works by splitting the text into single words and then looking for the sentiment of each word in a lexicon, which is a dictionary in which words are associated with a sentiment score. We used a version of the method (implemented in the R package sentimentr) that also considers valence shifters (see details in the online appendix).

Note that sentimentr’s default lexicon (called Jockers and Rinker’s polarity table, see Rinker 2021) is based on US English, while Smith uses British English. To improve the reliability of our results, we expanded sentimentr’s default lexicon by including the British spelling of words. We used the variant conversion information built in the Spell Checker Oriented Word Lists (SCOWL; see Atkinson 2019), keeping for those the same scores as the corresponding American words.

Figure 7 gives two graphical representations of the sentiment scores for all sentences in the two books. The y-axis in the graph on the left indicates the direction (positive- negative) and the strength of the sentiment measured in each sentence. Positive sentiment would reflect feelings such as happiness, approval, delight, satisfaction, thankfulness, et cetera; negative sentiment would reflect sadness, agitation, agony, danger, failure, pain, et cetera. Values close to zero are essentially neutral. The black lines are obtained by smoothing the sentiment values. The cloud of points around the smoothing lines shows the strong variability of the sentiment values, not immediately appreciable by looking only at the smoothed lines. ⁹ Some patterns are visible, most notably the inversion of the two books, in terms of which one has the most positive sentiment (WN in the first half, TMS in the second half). Such patterns are, however, not very pronounced, and smoothed values are always within a narrow interval: positive but close to zero.

Figure 7. Left: Sentiment Scores for All Sentences throughout TMS and WN; Right: Boxplots of the Distribution of Sentiment Values in TMS and WN

The boxplots on the right side of Figure 7 complement the picture. We can see how most of the sentiment scores are moderately positive but with substantial variability. Most notably, many sentences have sentiment scores so positive or so negative that they fall outside the whiskers of the boxplot.

The large variability in the sentiment scores is an indication that, from the point of view of sentiment analysis, Smith used a varied and emotionally charged vocabulary. This can be illustrated by considering specific sentences whose sentiment score is either particularly positive or particularly negative. We can take this sentence from TMS:

But though these three passions, the desire of rendering ourselves the proper objects of honour and esteem; or of becoming what is honourable and estimable; the desire of acquiring honour and esteem by really deserving those sentiments; and the frivolous desire of praise at any rate, are widely different; though the two former are always approved of, while the latter never fails to be despised; there is, however, a certain remote affinity among them, which, exaggerated by the humorous and diverting eloquence of this lively author [Mandeville], has enabled him to impose upon his readers. (Smith [1759] 2002, pp. 365–366)

The sentence is filled with positive words (passions, desire, proper, esteem, acquiring, deserving, praise, approved, affinity, humorous, eloquence, lively, enabled). Some negative words are also present, but far less numerous (frivolous, fails, despised, exaggerated, impose).

An example of a negative sentence is when Smith writes in WN that “a simple augmentation is an injustice of open violence; whereas an adulteration is an injustice of treacherous fraud” (Smith [1776] 1977, p. 1260). The sentence contains several extremely negative words (injustice, violence, adulteration, injustice, treacherous, fraud) and not a single positive one.

While similar examples can be found across the two books, it would be difficult to draw any strong conclusion on the books’ general sentiments from the quantitative results. Indeed, explaining the pronounced variability of the sentiment scores would require further research, but we can make some hypotheses.

First, sentiment analysis is not usually applied to long historical texts. The method is usually applied to determine the sentiments of short and contemporary texts (tweets, online reviews) or of documents in which emotions play a relevant role (stories, novels). Smith’s books are, as we have seen, difficult to read, with long and complex sentences, and this could have an impact on the capability of the method to score each sentence.

Second, the method could fail because of the limitations of the lexicon used. As we noted, we extended the default lexicon by including the British spelling (used by Smith), but the two books are about specific subjects (moral philosophy versus commerce) and the jargon used in them is often outside the scope of the lexicon, which has been developed for other purposes. The fact that the lexicons currently in use have not been developed for the analysis of historical economic texts carries another consequence: the sentiment score associated with some words could be a poor fit in the specific context of Smith’s writings. In WN, Smith frequently uses words like “gold,” “wealth,” and “money,” and such words are all associated with a positive sentiment. We argue, however, that similar words would better be associated with a neutral sentiment, when they are used in the context of a text on commerce. The same could be said for words with a negative sentiment score, like “tax,” “payment,” or “labour.”

To conclude, despite our best efforts to adapt an existing and popular (in other fields) methodology to the context of historical economic texts, the results do not seem to be readily interpretable. Consequently, our recommendation would be to not use (or to use cautiously) sentiment analysis, in its present state, for the study of historical economic documents.

VI. CONCLUSION

We have seen that a text as data approach can shed light on some important aspects of a corpus of historical economic documents, albeit with some caveats and limitations.

The most effective methods we applied were those used for topics detection. Assuming no prior knowledge of the books, the quantitative analysis allowed us to correctly identify the key issues discussed in each book and the differences between the two books. The corpus we analyzed was small but, given the quality of our results, we expect the methods we use to be safely applicable to larger corpora of historical economic texts. We also showed how quantitative tools can be used not only to provide summary information on the overall topics of the books but also to shed light on a specific conceptual discussion. We indeed identified the different contexts in which Smith mentioned “war” and “peace” in the two books. A limitation of these methods is that they partially rely on a few methodological choices concerning, for instance, the minimum frequency for a word to be detected as relevant, or the inclusion or removal of stop words. Another similar issue is that the results of topic analysis using keyness depend on the choice of the target and reference documents. While this was not a problem in our case (since we have only two documents), applications to larger corpora would require special attention on this point.

Concerning Smith’s style, our study combines measures of readability and lexical richness, together with insights coming from word frequency analysis. The analysis returned useful insights into the components of Smith’s style. Measures of readability and lexical richness could be of interest for historians since they can be fruitfully used to inquire about the spread of ideas or texts (the hypothesis could be that documents that are easier to read spread more easily). These are, however, synthetic indicators that can be better appreciated in context, for instance by comparing the results with other documents of the same era or the same author. Important stylistic insights came also from using keyness and word frequency analysis, which allowed us to notice the importance of pronouns in TMS and, especially through the use of co‑occurrences, to spot some expressions that are characteristic to Smith’s writing. These stylistic elements are of interest in themselves and could also prove useful in other contexts. An example is their use in authorship attribution studies, i.e., solving a dispute over the authorship of a specific document (see, for instance, Gherity 1993; Martindale and McKenzie 1995) or determining the author when unknown (for a review of possible applications of authorship attribution, and methods used in that field, see Koppel, Schler, and Argamon 2009; and Stamatatos 2009).

In addition, even though it is a popular methodology in data science and machine learning applications, we recommend scholars be cautious in applying sentiment analysis to lengthy historical economic documents. We could not interpret the results of the sentiment analysis because of the extreme variability of the sentiment scores calculated (we suggested some explanations for this variability).

The limitations of the text as data approach that we highlighted are a reminder that the quantitative and qualitative approaches are not mutually exclusive. As Justin Grimmer and Brandon Stewart (2013, p. 268) put it: “the complexity of language implies that automated content analysis methods will never replace careful and close reading of texts … the methods … are best thought as amplifying and augmenting careful reading and thoughtful analysis” (see also Claveau and Herfeld 2018, p. 602). In this sense, despite its promises, the text as data approach will not provide a definitive answer to Das Adam Smith Problem.

As for future research, an immediate next step to this article would be to extend the analysis to Smith’s other contributions (such as his 1763 Lectures on Jurisprudence), other editions of his books (see Paganelli 2011), or texts by other authors for style and topics comparisons. More broadly, many more text as data methods exist or are to be developed in this active field of research; and their possible applications are vast. Such applications could include authorship attribution or the study of economists’ speeches, economics and financial news, institutional reports, or tweets concerning economists or economics.