Data at the doorstep

To acquire data about a society or to put a number on a social issue is a complex endeavor. According to Joseph Marie de Gérando (1772–1842), a philosopher among the French administrators of the poor, many obstacles stand in the way of proper formalization. For example, he identified the high rank of a philanthropist as a distorting factor in data collection. To counter this, Gérando suggests a home visit, in which a moment of intimacy is deliberately established at the doorstep:

But the visit we make to the poor man, in his own dwelling, fills up this artificial gulf much better. There, it is with his situation alone that we are both occupied; there he finds us much better disposed to listen to him; there he sees in the conversation we hold with him, not mere condescension on our part, but the testimony of sincere interest. Let nothing in our manners or expressions betray repugnance and disgust on our part, at the sight of the rags of misery. … How much will these good people be affected! (Gérando 1832, 93).

Only this scripted encounter at the doorstep, Gérando argued, can overcome the lurking epistemological problem of social distance. The contributions to this topical issue, in turn, argue that the doorstep plays an important part in the acquisition of serialized and formalized information. In times of analogue data collection, these encounters form the basis for the sciences and the administration of the social. While more and more thorough research is being done on the history of data, this particular sequence within a data journey, the very situation when data about the social were collected, has received less attention (Aronova et al. 2017; Chadarevian and Porter 2018; Gitelman 2019; Leonelli and Tempini 2020).

As an outstanding French proponent of a new empirical “science of man,” Gérando was among the first to explicitly reflect on the methods of making “true and objective observations” of something as intangible as social behavior and moral posture (Herrnstadt 2023). In his Visiteur du pauvre (Visitor of the Poor, first published 1820), he outlined a meticulous plan on how the government and its scientific institutions could obtain reliable data regarding poverty. Subsequently, it became crucial to cross the doorsteps of houses and even dwellings in order to obtain valuable information. The alleged unwillingness of the poor to reveal their true situation to almsgivers was considered a contributing factor that drove early social investigators to invent methods that included crossing the lines of privacy—although the statistical phenomenon of the dark figure (Dunkelziffer) testifies to the fact that access to information remained partial.

The deployment of a whole array of strategies, designed to counteract the potential asymmetries of data collection caused by the popular distrust and protective lies, was not limited to “philanthropists.” Whoever was sent to extract information—the surveyors, colonial data collectors, enumerators, psychologists, experimenting teachers, journalists, statisticians, nurses, missionary nuns, and social workers—had to establish a situation of direct contact. Focusing on the labor of data collection thus allows us to investigate the stratagems surrounding data. Surveying, enumerating, and interviewing depend on some situation of closeness, the goal of which was to certify and to guarantee the validity of the data obtained.

First steps in quantifying and serializing the social

Gérando’s text about the observation of the poor is but one example of how information was acquired and tailored according to the unfolding biopolitical techniques of governance relying on quantified knowledge of the social (Miller and Rose 2008). We argue that for a history of personal data or the quantification of the social, it is important to look at the plethora of scripted and mediated encounters underlying the “avalanche of printed numbers” (Hacking 1990, 27–35). Agents of the state, of academic or private data collections, all had to ask: How can we gain access to information hitherto concealed, how can we interpret and process this serialized information, and who is the right person to provoke, collect, and even exact these kinds of telling numbers?

In the early nineteenth century, a new alliance between governmental interests and scientific strategies began to take hold. Social investigators discussed the necessity of direct or even participant observation and provided reflections on prerequisites and techniques for obtaining valid personal data. Claiming to be unafraid of the filthy air, dirty surroundings or even compromising circumstances, they entered prisons (Howard 1777; Rebmann 1811), approached Parisian prostitutes (Parent-Duchâtelet 1836), and encountered a “milieu” that became constructed as the “working poor” (Kay-Shuttleworth [1832] 2018, Villermé 1840; Engels 1845) and the “dangerous classes” (Frégier 1840).

A German travel account from 1840 described the activities of the newly founded statistical societies in Birmingham, Ulster, London, and Manchester (Fallati 1840). To facilitate commensuration, members would actively improve upon the data architectures by coaxing their town’s insurance company to adjust their columns or the local hospital to open their ledgers. They would sift through colonial blue books, and compare material published by governmental enquêtes. In contrast to Gérando’s approach, however, the home visit was often left to priests or distributors of religious pamphlets, who were equipped with specific questionnaires (Fallati 1840, 34). Even these agents complained that the personal circumstances, like bedrooms, or the amount owed to a monthly savings club, was shielded from them (ibid. 36).

While enquêtes, which were organized by politically appointed commissions, could take on the form of travels or the form of a court hearing (Stieda 1832; Fallati 1846; Embden et al. 1877), the census and private initiatives had to be more complex. The statistical bureaus of the nation-state no longer relied on estimations from tax returns, parish books or the numbers given by trusted informants (Löwenfeld-Russ 1926; Yeo 2002; Didier 2011). Instead, trained enumerators were sent from door to door and counting techniques were standardized and optimized. Finally, from the late 1860s census onwards, experiments with direct self-inscription began, using specific forms or “counting-cards.” Meanwhile the Verein für Socialpolitik, a powerful association of social scientists and political economists, organized the first noteworthy social surveys conducted by private initiatives in the German context. Here, the questions were addressed to the landed gentry, for fear that the peasants, whose living-situation was the object of inquiry, would not answer correctly and truthfully enough, because they suspected ulterior motives (Gorges 1995, 324). Later, members of the landmark study on the unemployed of Marienthal entered the homes of their research objects under the false pretense of distributing winter clothing, or organized gymnastics classes to gain access to the living situation of younger girls (Jahoda [1933] 2021, 28).

Approaches to gaining access to complex social situations have developed worldwide and have been used not only to advance government interests but also to highlight social problems and especially poverty in a tradition that is referred to as “private” or “counter” statistics. In addition to such famous maps of poverty as Booth’s maps of the London poor (1889–1903), the Pittsburgh Survey (1907–1908), and Hull House’s Maps and Papers (1890s), the investigations at the Atlanta Sociological Laboratory stand out in their revelation of how problems of poverty masquerade as problems of race and reflect on the “color line” in data portraits (Battle-Baptiste and Rusert 2018; Du Bois 1899; Du Bois and Burghardt 1903; Greenwald and Anderson 1996; Yancy 1978; Wright 2002, 2016). With a comparable opposition towards the spread of racial categories in statistics, Anténor Firmin used anthropometric data to argue against the hierarchizing claims made with them by the majority of physical anthropologists (Firmin 1885).

In these famous early social surveys, the various asymmetries of observation stand out and must be carefully considered in historical analysis. During collection, data emerged from hierarchized and heavily scripted dialogues, which were conducted with hidden agendas and specific interests, and this to an extent that makes it is worth questioning how and when the perceived neutrality and objectivity of data began to be the prevailing perception. The contributions of this topical issue testify to the history of perceived trespassing, distrust and concealment surrounding data collection, which often renegotiates privacy and leaves the counted areas in a modified state of affairs. Data collection crossed the doorstep in both a literal and metaphorical sense. Observers, interviewers and data collectors literally traversed the threshold of people’s homes in order to motivate them to give away sensitive information regarding their living conditions, income, opinions, property, and even diseases. Sometimes, these inroads into the private lives of citizens could only be made via public institutions. Psychologists would use schools for observations. The classroom became an important space for data collection early on in Germany, when anthropometric data on schoolchildren were amassed under the scrutiny and racialized discrimination of Rudolf Virchow (Virchow 1885; Zimmerman 1999). In colonial contexts, administrations (bureaucratic or missionary) usually doubled as sources for data. However, court meetings, medical stations or village assemblies were sometimes also instrumentalized for this purpose. In all cases, however, governmental and scientific knowledge successfully pervaded the threshold of the home, making transparent for knowledge acquisition a space that was increasingly distinguished from the public and guarded as private sphere (Habermas 1991; Baker 1992).

Building on research from the history of data, quantification and statistics

The analysis of practices and sites of data collection builds on emerging research fields such as the history of data or the social history of quantification, global and colonial histories of statistics and governmental techniques. It suggests a new perspective for the history of human and social sciences, based on serialized sources and the results of survey methods.

There is excellent research on the history of national and international statistical bureaus, the development of statistical methods in several empirical disciplines and the emergence of a distinct statistical way of reasoning (Hacking 1990). Yet, we know comparatively little about concrete techniques of data gathering on social issues, which were applied on site in the long nineteenth century. New studies have shown that there are excellent reasons to investigate the whole “data journey” or the life cycle of empirical information making—from collecting to labelling, processing, synchronization, storage and revival of the figures (Leonelli and Tempini 2020). Data centers and databases in natural history have received due attention (te Heesen 2007; Sepkoski and Tamborini 2018; Kaplan and Lemov 2019). The history of data merges seamlessly with the strong research tradition within the history of statistics. Earlier works on this topic emphasized the element of probability and the mathematical foundations of planning (Krüger 1987; Daston 1988). Historians of statistics have identified the use of large numbers by bureaucracies and state actors as a key phenomenon from the early nineteenth century onwards (Porter 1986; Desrosières 1998; Igo 2007; Behrisch 2016 a, b). Investigating the less fortunate and conducting such asymmetric surveys served two purposes: weakening the resistance of the elites against this kind of surveillance and increasing the visibility of moral flaws (Yeo 2002).

By this period, quantitative reasoning had already become an essential element in promoting innovation in the field of population and health (Rusnock 2002; Buton 2009; Porter 2018; Hüntelmann and Falk 2021), and in the registration of migration and minorities (Hunt 2002; Göderle 2016; Steidl 2021; Kröger 2022), which gradually moved to serology and DNA classification (TallBear 2013; Mukharji 2020; Lipphardt et al. 2022). Statistical data collections dovetailed with the wider “quantifying spirit,” defined as a passion to order and systematize as well as to measure and calculate, from the late eighteenth century onwards (Frängsmyr et al. 1990). Dan Bouk describes the crucial role of the health insurance sector for the early history of personal data—as companies and friendly societies strove to valuate lives and analyze the relation between individual data and large numbers (Bouk 2018, 2022). The countryside was subject to data collection as well, due to the highly influential prices of grain, which were carefully managed and monitored. Accounting practices were actively promoted in farms, and during the Gilded Age reporters from the rural communities themselves produced highly sensitive economic indicators about the estimated yield, before the state’s enumerators took over and turned agrarian statistics into a veritable industry (D’Onofrio 2016; Didier 2011, 2020).

Early on, historians underlined the paramount importance of surveys and other data practices as “technologies of government” (Sabapathy 2020; Rose 1991; Miller and Rose 2008, 21; Mogilner 2013). The decisive role played by bureaucratic personnel in this type of empirical knowledge collection is currently a matter of renewed interest (Felten and von Oertzen 2020; Chapoutot 2020), extending to colonial statistics and administration (Krzywicki 1934; Kuczynski 1948–1953; Malègue 2018; Renard 2021) and even post-colonial settings (Serra 2018).

Scholars have emphasized the coevolution of the nation state and the addressability of its citizens through census and land surveying. Despite an ongoing interest in historical practices and a growing field of scholarly work on the making of a social knowledge (Camic, Gross and Lamont 2011), the practice of data collection has largely been left out. Moreover, there has been hardly any reflection on the social hierarchies surrounding the formalization of data, despite the fact that the history of quantification is strongly influenced by sociology (Espeland and Sauder 2007; Espeland and Stevens 2008, Diaz-Bone and Dider 2016; Bruno et al. 2016). There is, however, literature discussing the formats of data from number to word or sentence (e.g. Ellwein 1997; Wetzell 2000; Tantner 2007; Török et al. 2015; te Heesen 2022) and differences over formalization in the two traditions of statistics: the number loving tradition of political arithmetic and the Göttingen School of Statistics, which was more skeptical of tables (Lazarsfeld 1961; Bödeker 2001). Various social sciences have incorporated experimental elements into their methods or sought to constitute their objects through “observation” (Maas and Morgan 2012).

Data collection did of course reach far beyond academic methodologies. The materiality of information and bureaucratic writing and filing techniques are key factors, and excellent research, analyzing paper tools, media and electrical tabulating machines (as the census was one of the very early fields of digital data collection), has emerged (Campbell-Kelly 1996; Becker and Clark 2004; Jardine 2017; Blair 2021). Due to their focus on data processing and technology, these contributions pay less attention to social relations and assumed roles of questioning and instead focus more on data processing. The paper tools explored in this topical issue range from household diaries to formalized questionnaires, counting slips and counting-cards, to more experimental test settings and questioning-techniques.

The history of social surveying and anthropological questionnaires is by now well represented in research (Bulmer 1991; Greenhalgh 2014; Topalov 2015; Herrnstadt and Renard 2021; Midena and Yeo 2022; Corbould et al. 2023). Many studies, however, focus mainly on French and English-speaking countries and the history of social sciences and anthropology. Applied social sciences and bureaucratic knowledge-making are far less analyzed, as is the German context. Particularly, the impact of the questioning itself, the performativity of the census, the effects of home-visits with piles of papers, the struggle for truthful data and mass mobilization of survey personnel have not received enough attention.

Towards a historical epistemology of data collection

The strategies presented by Gérando, quoted in the opening passage of this introduction, point to the many hurdles that need to be overcome in situations like the unveiling of the requested data, as well as to a desire for information to be unadulterated and untainted. Before today’s technological advances allowed for a seemingly ubiquitous and hidden data capture, the threshold of the home (or a school) was a privileged site of knowledge production. Over the course of the nineteenth century, birth rates, legal customs, mental health metrics, family structures, property, poverty, household budgets, even morality and sanity were listed, processed, and recalculated. Who collected this sometimes sensitive information about persons and who decided upon the questions and categories employed in a survey, enquête, or census? How important was trust and distrust in the practice of data acquisition? And what role did the constellation “At the Doorstep” play in stabilizing the captured information as comprehensive, accurate, reliable, or objective?

The contributions of this issue accentuate two main outcomes: Firstly, we found data collections to be discussed as something surprisingly intimate and shrouded in distrust, fraught with deceit and achieved by cunning. Secondly, data collections proved to be formative in the sense that not only the aggregated numbers or the data doubles of a person had an impact (Bouk 2017), but that the very moment of data collection made a great difference or was intentionally designed to change and improve the participants in almost experimental situations. Zimmerman has shown this to be true in Virchow’s comprehensive study on school children (1999), and it is in this very encounter of enumerated and enumerators in which the social dynamics of statistical categories unfold (Hacking 1986; Surdu 2016).

(1) On objectivity, intimacy and (dis)trust

Data about the social were gathered in a variety of comparable situations. They were collected by various state agencies, in social and charitable work, in early social science or in criminology, indigenous law, psychological observation, and economic training for housewives. Dialogues at the doorstep formed the first stage of the acquisition of some form of epistemic understanding, from which the actors could later generate data, information, and other types of processed inscriptions. In order to obtain data in the first place and to believe in their validity and objectivity, scientific and/or government actors created situations of intimacy and trust, anticipating distrust and concealment of vital data. Data about people and populations are neither givens, nor simple measurements (Bonß 1982; Rosenberg 2018; Fischer et al. 2020). They emerge from processes and social situations that are constituted by several elements, all of which are epistemologically relevant.

As the interview gets more refined, the persona of the interviewer more professionalized, the role of the interviewee as well as the socio-economic framework and symbolic power of these techniques and objects changes. In the issue’s first article, Anke te Heesen analyzes the “interview” as a specific kind of interaction that developed around 1900, with a focus on the writer and critic Hermann Bahr and his Antisemitism – An International Interview (1894). What kind of information can be established in a field shielded by privacy depends on the structure of the social encounter, the media of collection, the formats of inscription and the general categories of the survey, the scientific programs, the social movements, the assumptions and imaginaries attached to it. In order to analyze this complex interplay in such “basic statistical situations,” we consider the various elements at play, comparable to the comprehensive scrutiny of all parts of an experimental system or an epistemic configuration in the history of mathematics (Rheinberger 1997; Epple 2004).

Further key elements, apart from the dimension of language, are of course the writing techniques and recording systems in use. Christine von Oertzen has, in various contributions, described the innovative paper technologies deployed by the Prussian census. In this topical issue she reveals the hopes and goals invested in the technique of self-inscription, which was implemented from the late 1860s onwards. The notion that the citizens would be required to write down the information for themselves was designed to ensure the “truthfulness” and therefore objectivity of the data obtained. As von Oertzen shows, this attempt to cut out the enumerator, as a potential source of distortion and error, was not entirely successful, since the citizens’ responses were often too wordy, ambiguous or incoherent to produce useable data.

The self-inscription on counting cards relied on self-responsible citizens, who were aware of the importance of statistics. This was not always the case and a “public interest” in precise data collection clashed even more notably their interests when sensitive and intimate issues were being surveyed, such as the number of the mentally ill living outside of the asylum walls. As Sophie Ledebur shows, “insanity counts” were suspected to be significantly underrepresenting the “real” number of the mentally ill from the early nineteenth century onwards. Linking this enumeration to the reformed census was done not just in the hope for better figures, but to gain direct access to the homes of the families concerned. To facilitate admittance Ernst Engel, the head of the Prussian statistical office, advocated the policy that the census had to be about general knowledge, not state surveillance and direct political consequences (such as draft or taxation). Yet, the census results indicating mental illness prove to be a notable exception. With the counting cards in their hands, local physicians were able to conduct detailed follow-up investigations to verify and act upon the survey’s results. Nevertheless, the enumerators maintained that the achieved results were but a mere fraction of the real amount of those “lunatics” living beyond the walls of the asylums, an uncertainty that triggered an ever-expanding statistical enterprise and provoked various methods to pin down these elusive entities.

In a survey situation, data collectors become the diplomats and envoys of the state, the social program or surveying agency. Often, they are trained a certain way in order to achieve standardized results. The enumerated in turn fear and suspect the intentions of the institutions initiating the count. Although the collectors belong to the often nameless mass of scientific amanuensis, it is possible to think of the helping hands of empiricism as a stable constant in serialized knowledge making. Their mode of knowledge making allows little room for creativity, but due to their schooling and work it is possible to think of them as developing a unique “scientific persona” for constant enumerating (Daston and Sibum 2003). Much rested on their capacity to act in a formalized manner, since the quality and commensurability of data depended on it.

The contributions to this volume investigate, from various perspectives, how these standardized relations were established between the “collectors” and “bearers” of information, and how on the side of the interested authorities in charge of the survey, strategies were put into place to overcome mistrust by a well-designed situation, an intimate atmosphere, or the use of “detective skills” to unveil hidden motives. Interviewing techniques do of course actively create professional and social roles that are crucial for a successful interview rich in information and personal insights (te Heesen, this issue).

Moreover, with professionalization, ideals of contact and trust-based data collection were created. In the eyes of the collectors, these forms of emotional devotion could appear in the guise of a personal free choice. For example, social workers (Wohlfahrtspflegerinnen), very often displayed a high amount of emotional engagement in their work that, not least, served as informational technique to gather insights about working class children, their milieu, and parental homes (Stieve 1925; Sachße 2003). The same dynamics were observed by Laurens Schlicht (this issue) in the classroom, which became a site of surveillance and data generation. His contribution analyzes how psychologists and teachers used observational and experimental techniques to study elementary-school children in Weimar Germany.

The contributions of this volume show how crucial intimacy is for the configuration of information and that, through the course of time, data collection became more and more formalized and accordant to gender norms: It entered training regulations, the professional self-image of information collectors, and was tied to gender stereotypes (Stapleford 2012). It fell to sociologists, social workers, teachers, statistical collectors, officers of the youth welfare office, police officers, missionaries and colonial officials etc., to create and maintain informational configurations that reacted to the question of trust, mistrust, and the right amount of intimacy.

(2) On the formative and classificatory power of encounters

The chosen focus on the moment of data generation brings to light the immediate impact of a census, a social survey or an enquête. It is through surveying and enumeration that citizens come into direct contact with the state and are thus subjected to an almost pastoral power of being counted in and being “seen,” as well as being allowed to weigh in. A history of data collection at the doorstep clearly reveals a specific kind of influence that statistical categories and survey work unfolds in the field right at the moment of questioning.

Most applied sciences in this volume—from early social work to the social surveying initiatives, from the home economics movement to collections on common law—combined data collection with an effort to directly sculpt society and elicit or shape social change. Power has been assigned to data about the population in various forms. Measurement mainly enhanced the legibility of societies and augmented the addressability of individuals (Scott 2020; Kula 2014). It has been shown that the classification of data often transforms the societies counted.

In one way or the other, surveying served to legitimate and install ideals of socio-political order, an effect of what one can call “power” with regard to the works of Michel Foucault (1984). If power is understood here as the ability to enforce notions of normativity in a given social constellation, then informational techniques—together with their categorizing power and their demonstration of the legitimacy of the state/interest group’s access to the private sphere—certainly belong to the ensemble of transformative governmental power techniques.

Testifying to the wide participation of women in the acquisition of numbers on social problems, Gabrielle Soudan, David Philippy and Harro Maas investigate two female reform scientists, Florence Kelley and the “chemist and propagator of the Home Economics Movement,” Ellen Richards. The clear aim of these endeavors around 1900 was to combine surveys on the living conditions of the working class with a reform program for bettering these conditions. These women did not only observe the working poor in the tradition of household budgets, they also strove to achieve a direct change in the attitudes and vital needs of the families. To intensify the contact, model kitchens were set up, in which social reform could be conducted as if under laboratory conditions. In these special reform settings, the impact was expected to emerge from the encounter, and not solely from the analysis of information acquired in the process.

Social surveying may thus have had a palpable effect, long before data were analyzed, aggregated and prepared for publication, and long before they were included in decision making processes or forged into political arguments. Studies of the colonial census show, for instance, how the British in India made use of the caste system as a counting category in their census. Anthropologists provided a scientific classification system and local interest groups filed for the recognition of new castes (Cohn 1987; Appadurai 1994). The German colonial surveys discussed in Anna Echterhölter’s contribution to this issue were designed, she argues, to interfere with commonly held assumptions on land tenure and customary law. A measuring campaign on the colonized Pacific island of Pohnpei afforded land titles to the Pacific islanders for the first time. Remarkably, the production of the surveying data lay with the Pohnpeians themselves. With the demarcation and registration of their plots of land, they committed to a new form of land tenure that diverged sharply from their own tradition, one based on private property and exclusively male lines of inheritance.

Data collections are informational techniques that have a formative influence, especially through the categorizing of people. The contributions in this issue cover various geographical areas and types of data practices without aiming for completeness. In this way, the specific impact of new classification that emerged in colonial situations (Echterhölter), in the psychiatric welfare system (Ledebur), in drastically changing political constellations (Schlicht), in established political situations (Soudan, Philippy, and Maas; von Oertzen), and as part of new techniques of information retrieval (te Heesen) can be considered in a comparative perspective.