In this chapter we discuss a few cases of scientific misconduct that turned out easy to spot, given some basic knowledge of statistics. We learn that it is always important to begin with a close look at the data that you are supposed to analyze. What is the source of the data, how were they collected, and who collected them and for what purpose? Next, we discuss various specific cases where the misconduct was obvious. We see that it is not difficult to create tables with fake regression outcomes, and that it is also not difficult to generate artificial data that match with those tables. Sometimes results are too good to be true. Patterns in outcomes can be unbelievable. We also see that it is not difficult to make the data fit better to a model. These are of course all unethical approaches and should not be replicated, but it is good to know that these things can happen and how.

Chapter 7 takes a closer look at some of the Sea Around Us fish-landings data that we assessed for Benford agreement in Chapter 5. We chose these data because of the mixed agreement findings among them: while the full dataset and several sets of subgroups indicated that the data exhibited Benford validity, when we analyzed West African countries individually, a number of them were found to have unacceptable Benford agreement and therefore problematic Benford validity. We present ways in which researchers can assess the impact of unacceptable Benford agreement on their analyses.

Chapter 3 describes and illustrates the Benford probability distribution. A brief summary of the origin and evolution of the Benford distribution is drawn and the development and assessment of various measures of goodness of fit between an empirical distribution and the Benford distribution are described and illustrated. These masures are Pearson’s chi-squared, Wilks’ likelihood-ratio, Hardy and Ramanujan’s partition theory, Fisher’s exact test, Kuiper’s measure, Tam Cho and Gaines’ d measure, Cohen’s w measure, and Nigrini’s MAD measure.

This chapter gives an overview of the remainder of the book. We first provide commonsense and social science examples of reliability and validity, two necessary conditions that data must posses to have trustworthy conclusions based upon it. We next introduce Benford’s law and offer a brief overview of other social science studies that have employed it to check the accuracy of their data. We then turn to an overview of our Benford agreement analysis procedure and introduce the concept of Benford validity. The chapter concludes with a plan for the remainder of the book.

Here we develop a discussion of the concept of validity in the social sciences. We first highlight the history of validity and how it has been conceptualized and measured over time. Next we discuss a type of social science data that is often overlooked in the validity measurement and assessment literature: data that are based on self-reporting. Despite the widespread use of self-reported data in various social science disciplines such as economics, political science, and sociology, there are still few reported attempts to check data accuracy. By way of giving examples, we overview self-reported data in four areas: (1) US prison population data, (2) COVID-19 case data, (3) toxic releases, and (4) fish landings. We then discuss the need for a tool and for an established workflow for assessing the accuracy and validity of quantitative self-reported data in the social sciences. We suggest that applying Benford’s law to these types of data can provide a measure of validity assessment for data that would otherwise not be assessed for accuracy; then we briefly introduce the concept of Benford validity. We conclude the chapter with a short review of existing studies that have applied Benford’s law to social science data in some manner.

Chapter 8 concludes that Benford agreement analyses are a proper process for assessing the validity of self-reported data when these data meet certain characteristics. The Benford agreement analysis workflow developed in previous chapters is summarized. Recommendations as to when researchers should use Benford agreement analyses to assess their data for Benford validity are discussed. The chapter concludes with some thoughts on the utility of Benford validity analyses in the social sciences.