Building causal knowledge in behavior genetics without racial/ethnic diversity will result in weak causal knowledge

Abstract

Behavior genetics often emphasizes methods over the underlying quality of the psychological information to which the methods are applied. A core aspect of this quality is the demographic diversity of the samples. Building causal genetic models based only on European-ancestry samples compromises their generalizability. Behavior genetics researchers must spend additional time and resources diversifying their samples before emphasizing causation.

Madole & Harden (M&H) propose that within-family genetic effects are analogous to randomized controlled trials (RCTs), and therefore can help identify potential genetic causes of psychological phenomena. In doing so, the authors repeat the frequent trope of RCTs as the “gold standard” (target article, sect. 1.1, para. 2) for establishing causal claims. Although it is clear that RCTs are optimally set up for such a task, using the language of “gold standard” has been criticized for obscuring the many limitations of the design (see Deaton & Cartwright [2018] for a thorough treatment of the subject and Jones & Podolsky [2015] for a historical discussion). The uncritical use of RCTs as an analogue for genetic effects opens their arguments to many criticisms, ones that are already prominent in work on behavior genetics. Among others, these include insufficient attention to conceptualization (Nguyen, Syed, & McGue, 2021) and measurement (Pelt, Schwabe, & Bartels, 2022) of the target phenotype and lack of sample diversity when seeking to establish generalizable claims (Holden, Haughbrook, & Hart, 2022). By not incorporating these major concerns in their paper, M&H unfortunately compound, rather than resolve, these issues in their otherwise productive set of arguments.

The more general criticism levied toward both RCTs and behavior genetics is the over-emphasis on method over substance. M&H continue in this tradition, devoting careful attention to establishing and calibrating causal claims. This is clearly a useful and much-needed approach to thinking about causation, especially in psychology, but we must not overlook the quality of the substantive information about which we seek to make causal claims. Here, I highlight just one aspect of such quality, the racial/ethnic diversity of samples included in behavior genetic studies, and why such a consideration must be central to any effort to build generalizable causal knowledge.

It is a fact of the design that RCTs sacrifice external validity for the sake of internal validity, being high in efficacy, showing promising results in trials, but low in effectiveness, or lack of results when translated to real-life conditions (Flay et al., 2005). RCTs have been further criticized by researchers in multicultural psychology and culturally adapted treatments for their lack of inclusion of racial/ethnic minorities and thus limited generalizability (Bernal & Scharrón-del-Río, 2001; Castro, Barrera, & Holleran Steiker, 2010; Whaley & Davis, 2007). Consistent with the mainstream view in psychology, the arguments made by M&H all assume a universal, replaceable person; it does not matter where the information comes from, so long as the proper modeling is applied (see also Yarkoni, 2022). But, of course, strong design features cannot overcome selection issues that can lead to misspecified models (Bradley et al., 2021).

The research on lithium as a treatment for manic symptoms of bipolar disorder, highlighted by the authors, illustrates their lack of attention to sample diversity. The second-generation studies cited by M&H – those that seek to identify specific causal mechanisms beyond an average treatment effect – relied entirely on data from White men (Mertens et al., 2015; Santos et al., 2021; Stern et al., 2018). It may well be that the identified mechanisms are generalizable beyond this very narrow group, but it seems prudent to investigate this question prior to broadly claiming generalizable causal knowledge. Moreover, the authors frame this kind of second-generation investigation as addressing the problem of lack of generalizability beyond the first-generation average treatment effect, but it does so only with respect to individual differences, and not demographic/population heterogeneity.

An argument could be made to justify sample homogeneity in first-generation studies, when putative causal factors are initially identified, and explore generalizability as part of the second-generation process. This is related to the issue of “portability” of findings from genetic studies (or “effectiveness” with RCTs), which refers to the fact that average effects identified through first-generation behavior genetics studies may not generalize to new contexts or populations. Indeed, the impressive Lee et al. (2018) study of educational attainment of 1.1 million individuals included only participants with European ancestry. When the researchers attempted to “port” the polygenic scores derived from the European-ancestry group to a sample of Black Americans, the 10.6% R ² attenuated by 85%, a result that they indicated was “typical of what has been reported in other studies” (p. 1115). M&H fail to mention the degree of this problem, let alone what the implications are for building generalizable causal knowledge. For example, the omnigenic model highlights the need for diversity in discovery samples to identify and separate both core and peripheral variants (Mathieson, 2021; see also Wojcik et al., 2019). Thus, diversity is central to first-generation studies.

Moreover, in discussions of portability, left unsaid is from whom to whom the results are being ported, which is nearly always from White/European samples to other ancestry or racial groups. Rarely do we seek to, for example, generalize results from African samples to other groups (Adetula, Forscher, Basnight-Brown, Azouaghe, & IJzerman, 2022). This dynamic sets up a standard in which the White/European results serve as the basis for the first-generation causal knowledge, and any deviations from it are problems to be solved or, more often, swept under the rug. Such a perspective is consistent with the deficit model that has for decades dominated psychological research on diversity (Cauce, Coronado, & Watson, 1998), and unreasonably constrains the context of discovery.

The issues mentioned here are reminiscent of the aphorism “garbage in, garbage out” in the context of meta-analyses. That is, no degree of sophisticated analyses will save your substantive conclusions if the studies included therein are weak or uninformative. To be fair, M&H clearly know this, but the issue is treated more in passing rather than as a central concern in their quest to build stronger causal knowledge, a quest which I greatly support. They use a catch phrase in similar structure to “garbage in, garbage out” when discussing causal reasoning, “no causes in, no causes out,” which, fittingly, pertains to the reasoning and not the quality of the information within it. Combining these two, I might say, “no diversity in, no causes at all.”