On the validity of the CNI model of moral decision-making: Reply to Baron and Goodwin (2020)

Abstract

The CNI model of moral decision-making is a formal model that quantifies (1) sensitivity to consequences, (2) sensitivity to moral norms, and (3) general preference for inaction versus action in responses to moral dilemmas. Based on a critique of the CNI model’s conceptual assumptions, properties of the moral dilemmas for research using the CNI model, and the robustness of findings obtained with the CNI model against changes in model specifications, Baron and Goodwin (2020) dismissed the CNI model as a valid approach to study moral dilemma judgments. Here, we respond to their critique, showing that Baron and Goodwin’s dismissal of the CNI model is based on: (1) misunderstandings of key aspects of the model; (2) a conceptually problematic conflation of behavioral effects and explanatory mental constructs; (3) arguments that are inconsistent with empirical evidence; and (4) reanalyses that supposedly show inconsistent findings resulting from changes in model specifications, although the reported reanalyses did not actually use the CNI model and proper analyses with the CNI model yield consistent findings across model specifications. Although Baron and Goodwin’s critique reveals a need for greater precision in the description of the three model parameters and for greater attention to properties of individual dilemmas, the available evidence indicates that the CNI model is a valid, robust, and empirically sound approach to gaining deeper insights into the determinants of moral dilemma judgments, overcoming major limitations of the traditional approach that pits moral norms against consequences for the greater good (e.g., trolley dilemma).

1 Introduction

A central question in moral psychology is how people make decisions in moral dilemmas that involve a conflict between moral norms and the greater good. Research on this question has predominantly relied on hypothetical scenarios such as the trolley problem, in which a runaway trolley is approaching a group of five individuals who would be killed if the trolley continues on its path. In one variant known as the switch dilemma, participants are asked if it would be acceptable to pull a switch to redirect the trolley to another track where it would kill only one person instead of five (Foot, 1967). In another variant known as the footbridge dilemma, participants are asked if it would be acceptable to push a person from a bridge to stop the trolley (Thomson, 1971). Adopting terminology from moral philosophy, participants are said to have made a characteristically utilitarian judgment if they judge the described actions as acceptable. Conversely, participants are said to have made a characteristically deontological judgment if they judge the described actions as unacceptable (Conway, Goldstein-Greenwood, Polacek & Greene, 2018). From a utilitarian view, the described actions would be acceptable because they maximize the well-being of a larger number of people. In contrast, from a deontological view, the described actions would be unacceptable because they are in conflict with the moral norm that one should not kill other people. Over the past two decades, a substantial amount of research has investigated contextual conditions that make people more or less likely to prefer utilitarian over deontological judgments, individual differences in the preference for utilitarian over deontological judgments, and the mental processes underlying utilitarian and deontological judgments (Bartels, Bauman, Cushman, Pizarro & McGraw, 2015).

Research using the trolley problem (and similar sacrificial dilemmas) has been criticized for relying on unrealistic, sometimes humorous scenarios that have little resemblance with the kinds of moral dilemmas people are facing in real-world contexts (Bauman, McGraw, Bartels & Warren, 2014; Körner, Joffe & Deutsch, 2019). In addition to addressing this concern by using realistic dilemmas inspired by real-world cases, our own research aimed to resolve a more fundamental limitation of the traditional dilemma approach: the confounding of multiple factors in the measurement of moral judgments. To resolve these confounds, we have developed a mathematical model called the CNI model of moral decision-making, which quantifies three determinants of moral dilemma judgments: (1) sensitivity to consequences, (2) sensitivity to moral norms, and (3) general preference for inaction versus action (Gawronski, Armstrong, Conway, Friesdorf & Hütter, 2017). Based on a critique of this work, Baron and Goodwin (2020) dismissed the CNI model as a valid approach to studying moral dilemma judgments. Here, we respond to their criticism, clarifying which of their concerns are justified and which ones are not. Although Baron and Goodwin’s critique reveals a need for greater precision in the description of the three model parameters and for greater attention to properties of individual dilemmas, we argue that Baron and Goodwin’s dismissal of the CNI model is based on: (1) misunderstandings of key aspects of the model; (2) a conceptually problematic conflation of behavioral effects and explanatory mental constructs; (3) arguments that are inconsistent with empirical evidence; and (4) reanalyses that supposedly show inconsistent findings resulting from changes in model specifications, although the reported reanalyses did not actually use the CNI model and proper analyses with the CNI model yield consistent findings across model specifications. We conclude that, counter to Baron and Goodwin’s conclusion, the CNI model is a valid, robust, and empirically sound approach to gaining deeper insights into the determinants of moral dilemma judgments, overcoming major limitations of the traditional approach.

2 The CNI Model

Because Baron and Goodwin’s (2020) critique is partly based on misunderstandings of key aspects of the CNI model, we deem it important to provide some background information about the model for our rebuttal of their critique. The CNI model is based on the notion that utilitarian judgments are characterized by the feature of being influenced by the consequences of a given action for the greater good, whereas deontological judgments are characterized by the feature of being influenced by the consistency of a given action with moral norms (Gawronski & Beer, 2017). Thus, to classify moral judgments as utilitarian, one would need to demonstrate that they vary as a function of relevant consequences for the greater good. Conversely, to classify moral judgments as deontological, one would need to demonstrate that they vary as a function of relevant moral norms. Whereas the former requires experimental manipulations of consequences, the latter requires experimental manipulations of moral norms. Both are largely absent in research using the traditional dilemma approach, which focuses almost exclusively on (1) cases where the benefits of a focal action for the greater good outweigh its costs (without considering cases where the benefits of the focal action are smaller than its costs) and (2) actions that are prohibited by proscriptive norms (without considering actions that are prescribed by prescriptive norms).

These limitations pose a significant challenge to the interpretation of moral dilemma judgments. For example, if a participant finds it acceptable to redirect the trolley in the switch dilemma regardless of whether it would save five lives or only one, questions could be raised about whether it is justified to categorize the participant’s responses as utilitarian, because they are unaffected by the consequences for the greater good. Similarly, if a participant is unwilling to perform a focal action regardless of whether this action is prohibited by a proscriptive norm (e.g., killing a person) or prescribed by a prescriptive norm (e.g., saving a person’s life), questions could be raised about the deontological nature of these judgments, because they may reflect general action aversion rather than effects of moral norms.

A different way of describing these issues is that the traditional approach includes two confounds in the measurement of moral dilemma judgments (Gawronski, Conway, Armstrong, Friesdorf & Hütter, 2016). First, because accepting one option implies rejecting the other, it is impossible to determine whether differences in moral dilemma judgments are driven by differences in the tendency to make a “utilitarian” judgment, differences in the tendency to make a “deontological” judgment, or differences in both (Conway & Gawronski, 2013). Second, because research using the traditional dilemma approach has focused almost exclusively on cases involving proscriptive norms, “utilitarian” judgments are confounded with general preference for action and “deontological” judgments are confounded with general preference for inaction (Crone & Laham, 2017).

To resolve these interpretational ambiguities, we proposed an alternative approach in which responses are compared across four types of dilemmas that vary in terms of whether (1) the consequences of the focal action for the greater good are either greater or smaller than the costs and (2) the focal action is either proscribed by a proscriptive norm or prescribed by a prescriptive norm. Expanding on this approach, we developed a multinomial model that quantifies the extent to which participants’ responses across the four types of dilemmas reflect a response pattern that is sensitive to consequences (see first row in Figure 1), a response pattern that is sensitive to moral norms (see second row in Figure 1), and a general preference for inaction versus action (see third and fourth row in Figure 1). Sensitivity to consequences is captured by the model’s C parameter, with higher scores reflecting a greater impact of consequences on moral judgments. Sensitivity to moral norms is captured by model’s N parameter, with higher scores reflecting a greater impact of moral norms on moral judgments. General preference for inaction versus action is captured by the model’s I parameter, with higher scores reflecting a general preference for inaction and lower scores reflecting a general preference for action.

Figure 1: CNI model of moral decision-making predicting action versus inaction responses in moral dilemmas with proscriptive and prescriptive norms and consequences involving benefits of action that are either greater or smaller than the costs of action. Reproduced from Gawronski, Armstrong, Conway, Friesdorf, and Hütter (2017). Reprinted with permission from the American Psychological Association.

Based on the processing tree depicted in Figure 1, the CNI model provides four mathematical equations that include the three model parameters as unknowns and the observed probabilities of action (vs. inaction) responses on the four kinds of dilemmas as known numerical values (see Appendix A).¹ Numerical values for the three parameters are estimated via maximum likelihood statistics, aiming to minimize the difference between the empirically observed probabilities of action (vs. inaction) responses on the four types of dilemmas and the probabilities of action (vs. inaction) responses predicted by the model equations using the identified parameter estimates. The estimated scores for each parameter reflect a probability that can vary between 0 and 1. For the C parameter, scores significantly greater than zero indicate that responses were affected by the manipulation of consequences in a manner such that participants showed a response pattern that maximizes the greater good. For the N parameter, scores significantly greater than zero indicate that responses were affected by the manipulation of moral norms such that participants showed a response pattern that is congruent with both proscriptive and prescriptive norms. Finally, for the I parameter, scores significantly greater than 0.5 indicate a general preference for inaction and scores significantly lower than 0.5 indicate a general preference for action. The adequacy of the model in describing the data can be evaluated by means of goodness-of-fit statistics, such that poor model fit would be reflected in a statistically significant deviation between the empirically observed probabilities and the probabilities predicted by the model (for more details, see Gawronski et al., 2017).

3 Deontological Responding

A central point of Baron and Goodwin’s (2020) critique is that bias against action, as captured by the I parameter, is an explanation of deontological responding rather than an alternative process. Although we would argue that bias against action is not a mental process but a pattern of responding, we fully agree that a general preference for inaction can be interpreted as a particular instance of deontological responding. In fact, we already acknowledged such an interpretation in the original publication on the CNI model when we discussed the (in)consistency of our findings with Greene’s (2008) dual-process theory of moral dilemma judgments (Gawronski et al., 2017, p. 365):

“A potential way to reconcile [our findings] with the dual-process model is to interpret general preference for inaction as an instance of deontological responding. In line with this idea, the doctrine of doing and allowing (DDA) states that actively causing harm is morally worse than merely allowing harm, which is consistent with the finding that harm caused by action is perceived as worse than equivalent harm caused by inaction […]. Conceptually, the DDA can be regarded as a deontological principle in the sense that the moral status of a behavioral option depends on its consistency with a general rule.”

Baron and Goodwin’s explication of this idea is much more detailed compared to our analysis, and we agree with almost every aspect of their arguments about a deontological interpretation of generalized inaction. That being said, we would like to note that a conceptualization of generalized inaction as an instance of deontological responding remains incomplete if it does not acknowledge adherence to both proscriptive and prescriptive norms as a distinct pattern of deontological responding over and above generalized inaction. Although both response patterns can be interpreted as instances of deontological responding in a philosophical sense, they are conceptually distinct in the sense that they involve different behavioral signatures. As we stated in the original publication on the CNI model (Gawronski et al., 2017, p. 365):

“Although the two ways of responding may be deemed deontological in a philosophical sense, they should not be conflated in a psychological theory about the mechanisms underlying moral dilemma judgment. After all, sensitivity to moral norms and general preference for inaction are functionally distinct in terms of their psychological antecedents and their behavioral outcomes. Their distinct outcomes are reflected in the fact that the two ways of responding lead to different judgments in moral dilemmas involving a prescriptive norm. Their distinct antecedents are reflected in the current finding that a given factor can simultaneously strengthen one way of ‘deontological’ responding while weakening the other way of ‘deontological’ responding.”

Thus, although we agree with Baron and Goodwin’s (2020) argument that bias against action can be interpreted as an instance of deontological responding, their dominant focus on generalized inaction ignores that adherence to proscriptive and prescriptive norms represents a distinct pattern of deontological responding that does not involve generalized inaction. That being said, there is a nuance to Baron and Goodwin’s argument that suggests potentially misleading aspects in the descriptions we have chosen for the response patterns captured by the N and the I parameter.

By describing the response pattern captured by the N parameter as sensitivity to moral norms and the response pattern captured by the I parameter as general preference for inaction versus action, our descriptions suggest that only the former, but not the latter, response pattern would be congruent with moral norms. However, as correctly noted by Baron and Goodwin (2020), a pattern of generalized inaction is congruent with the broad deontological norm first, do no harm, suggesting that congruence with moral norms is not a distinguishing feature of the two parameters. Baron and Goodwin (2020) are also correct in their observation that the response pattern captured by the I parameter may reflect either a domain-specific aversion against causing harm or a domain-independent response bias (see Hennig & Hütter, 2020). Based on these considerations, we acknowledge that some aspects of our parameter descriptions are not ideal and potentially misleading, which includes some of the descriptions in the processing tree depicted in Figure 1 (e.g., the description of 1 − N as moral norms do not drive response). For the sake of consistency with prior CNI model terminology, we will continue to describe the response pattern captured by the N parameter as sensitivity to moral norms, yet with the qualification that this description is meant to refer specifically to the effect of proscriptive versus prescriptive norms on moral judgments (rather than a pattern of generalized inaction that is congruent with the broad norm first, do no harm). Moreover, the description general preference for inaction versus action should be interpreted in a purely behavioral manner that is agnostic about whether generalized inaction on the I parameter reflects a domain-specific aversion against causing harm or a domain-independent response bias. These qualifications are important for the interpretation of the two parameters, because they imply that the I parameter may capture a norm-congruent response pattern that is distinct from the norm-congruent response pattern captured by the N parameter. Yet, regardless of these qualifications, we would still argue that both response patterns are essential for understanding responses to moral dilemmas and their underlying mental processes. We will return to this issue in the section entitled “Confounds in the Traditional Approach.”

4 Moral Norms

A bias against action is relatively easy to identify, because it involves a general preference for inaction regardless of the specific situation (see third row in Figure 1). In contrast, differential responses to dilemmas with proscriptive and prescriptive norms are more difficult to identify, because any such endeavor requires construct-valid operationalizations of the two kinds of moral norms (see second row in Figure 1). A second major point of Baron and Goodwin’s (2020) critique is that our battery of moral dilemmas for research using the CNI model does not meet this criterion.

To address this concern, we deem it helpful to first clarify how we identified the focal actions and their corresponding moral norms in the construction of CNI model dilemmas. In a first step, we identified pairs of morally relevant actions and inactions that have the same outcome (e.g., killing Person A and letting Person A die both result in the loss of Person A’s life). Whereas the identified action within each action-inaction pair was conceptually linked to a proscriptive norm (i.e., killing someone is morally prohibited), the opposite of the identified inaction was conceptually linked to a prescriptive norm (i.e., saving someone’s live is morally prescribed). In a second step, we generated hypothetical consequences of the two actions that involve costs for the well-being of others that are either greater or smaller than the benefits of each action. In a third step, we created plausible scenarios of high real-world relevance, which were designed to be as similar as possible in the four variants of each basic dilemma. Thus, based on the rationale underlying the development of CNI model dilemmas, the relevant moral norms and their corresponding actions are identified in the first step independent of their consequences and independent of secondary aspects of the particular scenarios. In all cases, the relevant proscriptive norms pertain to directly causing harm regardless of the situation (e.g., killing someone), whereas the relevant prescriptive norms pertain to directly preventing harm regardless of the situation (e.g., saving someone’s life).

Baron and Goodwin (2020) correctly point out that prescriptive norms tend to be weaker than proscriptive norms (e.g., killing someone is perceived as morally worse than letting someone die). Baron and Goodwin conclude from this asymmetry that our modeling approach is destined to fail, because it requires a switch of actions and inactions while maintaining equivalent norms, which seems virtually impossible. Although we agree that the asymmetry between proscriptive and prescriptive norms is fundamentally important, it is irrelevant for the construction of CNI model dilemmas to the extent that the proximal outcomes of a given action-inaction pair can be held constant (e.g., loss of the same life as a result of killing or letting die). From the perspective of the CNI model, the possibility of symmetric and asymmetric effects of the two kinds of norms is not a methodological obstacle, but an empirical phenomenon that is captured by the difference between the N and the I parameter. Whereas the response pattern defining the N parameter is congruent with both proscriptive and prescriptive norms, the response pattern defining the I parameter is congruent with only one of the two norms but not the other (with the specific congruency depending on whether scores are greater or smaller than 0.5; see Figure 1).

A related concern raised by Baron and Goodwin (2020) is that a considerable number of participants seem to disagree with our assumptions about relevant moral norms in the CNI model dilemmas. To support their argument, Baron and Goodwin present the results of two studies in which participants were asked to identify for a selected subset of CNI model dilemmas (and a set of newly created dilemmas) if there is a rule that favors a particular response and, if so, which response is favored by that rule. Their main finding is that many participants identified patterns of rules that conflict with the conceptual assumptions underlying the operationalization of moral norms in our CNI model dilemmas, which led them to question the validity of our operationalization.

A major problem with Baron and Goodwin’s (2020) argument and the presented data is that they conflate different levels of analysis. As we explained in Gawronski, Conway, Armstrong, Friesdorf and Hütter (2018), the CNI model is a descriptive model that quantifies patterns of stimulus-response relations at the behavioral level of analysis (see De Houwer, 2011; Gawronski & Bodenhausen, 2015). Applied to the operationalization of moral norms in our dilemmas, the “stimulus” involves descriptions of actions that cause harm (capturing the presence of a proscriptive norm) or descriptions of actions that prevent harm (capturing the presence of a prescriptive norm). The “response” involves participants’ judgments of the described actions (e.g., acceptable or unacceptable). The N parameter merely captures the extent to which participants’ responses differ across the two cases, which reflects their sensitivity to the two kinds of norms in responding to the moral dilemmas (see Figure 1). The CNI model does not make any assumptions about the mental processes underlying the observed response pattern. The latter question pertains to the mental level of analysis, which aims to identify the mental processes underlying observed patterns of stimulus-response relations at the behavioral level (see De Houwer, 2011; Gawronski & Bodenhausen, 2015).

Baron and Goodwin’s (2020) critique is based on the tacit background assumption that an experimental manipulation of moral norms is construct-valid only if the behavioral effect of this manipulation is driven by conscious thoughts about moral norms. Their data suggest that this is not the case for our manipulation of moral norms, because it does not map onto the moral norms identified by the participants in their studies. However, by requiring that conscious thoughts about moral norms must underlie norm-congruent judgments, Baron and Goodwin’s (2020) critique not only conflates behavioral effects with explanatory mental constructs (see De Houwer, 2011; Gawronski & Bodenhausen, 2015); it also ignores one of the most significant contributions to moral psychology in the 21st century: the idea that norm-congruent judgments may not necessarily be the product of conscious thoughts about norms (Greene, 2008; Haidt, 2001). For example, according to Greene’s (2008) dual-process theory of moral dilemma judgments, norm-congruent judgments are driven by affective processes that do not involve conscious thoughts about moral norms. As we noted above, the CNI model is a descriptive model of stimulus-response relations at the behavioral level and the model does not make any assumptions about underlying processes at the mental level (see Gawronski et al., 2018). Hence, the pattern of norm-congruent responses captured by the N parameters could be driven by conscious thoughts about moral norms, affective responses to the idea of causing harm, or something entirely different. From this perspective, involvement of conscious thoughts about moral norms is not a suitable criterion to evaluate the validity of our experimental manipulation of moral norms, because people may behave in line with moral norms without consciously identifying their responses as reflecting moral norms.

If mental processes are inadequate to determine the validity of our norm manipulation, what is a good alternative to evaluate its validity? A relatively simple criterion is that, although participants may not consciously think about moral norms in a manner that corresponds to our experimental manipulation, they should respond to the dilemmas in a manner “as if” they agree with the assumptions underlying our operationalization of moral norms (see above). In other words, participants should show evidence for rule-conforming judgments, even if they do not show any evidence for rule-following judgments. Multinomial modeling offers two pieces of evidence that are relevant for this question.

First, if participants did not respond in a manner “as if” they agreed with the assumptions underlying our operationalization of moral norms, the CNI model should be unable to provide accurate descriptions of the data, and thus show poor model fit across studies (Klauer, 2015). Counter to this possibility, the CNI model fit the data well in almost every published study that we conducted with our original dilemma battery (see Table 1). Out of 23 published studies, there are only two studies in which the response probabilities predicted by the CNI model significantly deviated from the response probabilities observed in the data. With an alpha-criterion of .05, this proportion is close to the probability of obtaining a significant difference in the absence of an actually existing difference. If we include every unpublished study from our group using the original CNI model battery, the proportion is even lower with only two out of 32 studies in which the model did not fit the data.

Table 1: Goodness-of-fit statistics for the CNI model in all studies published by the current authors that have used the original battery of 24 moral dilemmas for research using the CNI model.

Second, if participants did not respond in a manner “as if” they agreed with the assumptions underlying our operationalization of moral norms, the N parameter should not significantly differ from zero (Klauer, 2015). Counter to this possibility, the N parameter was significantly greater than zero in every single study we have conducted with the CNI model. The only exception is a subsample of participants who were preselected based on having psychopathy scores that placed them in the highest quartile of participants in a broader sample (Gawronski et al., 2017, Study 4b). We will return to this finding in the section entitled “’Perverse’ Responses.” Together, the two pieces of evidence support the assumption that participants respond in a manner “as if” they agree with the assumptions underlying our operationalization of moral norms.

5 Item Characteristics

The reported data on model fit and the significant difference of the N parameter from zero suggest that, on average, participants respond to the CNI model dilemmas in a manner that is consistent with the assumptions underlying our operationalization of moral norms. However, these results are based on aggregate responses to all dilemmas. Although aggregation of responses across multiple items has the advantage of reducing the likelihood of false negatives by controlling for measurement error, it can lead to false positives if an observed effect is driven by a subset of items with idiosyncratic features that are unrelated to the construct of interest (Judd, Westfall & Kenny, 2012). As noted by Baron and Goodwin (2020), multinomial modeling does not consider potential differences between individual dilemmas and their contribution to the obtained results. Although the problems associated with item-specific effects have received increased attention during the last years, there is no straightforward way to statistically control for item-specific effects within multinomial modeling. This limitation raises the question of whether the items of our dilemma battery are equally valid in capturing the manipulations of consequences and moral norms.

Baron and Goodwin (2020) aimed to address this question by analyzing item-specific proportions of “perverse” responses, which they define as responses that go against both moral norms and the greater good when the two suggest the same decision. However, from the perspective of the CNI model, this strategy remains uninformative about the validity of a given dilemma, because incidental features of a given scenario may shift responses toward action or inaction for all four variants without qualifying the effects of consequences and moral norms (see Schwarz, 1999). In other words, a given item may show “perverse” responses in terms of Baron and Goodwin’s criterion, but the four variants of a given dilemma may still reliably capture the manipulations of consequences and moral norms. Thus, to determine whether the items of our dilemma battery are equally valid in capturing the manipulations of consequences and moral norms, it is essential to analyze whether the two manipulations are effective in influencing responses on each basic dilemma. For the manipulation of consequences, this question boils down to the proportion of action (vs. inaction) responses on dilemmas where the benefits of action are greater than the costs compared to the proportion of action (vs. inaction) responses on dilemmas where the benefits of action are smaller than the costs. For the manipulation of moral norms, the question boils down to the proportion of action (vs. inaction) responses on dilemmas where a proscriptive norm prohibits action compared to the proportion of action (vs. inaction) responses on dilemmas where a prescriptive norm prescribes action. The two comparisons should reveal meaningful differences in the expected direction for each basic dilemma across the four variants.

Using these criteria, our pilot tests supported the validity of each basic dilemma that was included in the final battery of dilemmas for research using the CNI model (some other dilemmas were discarded because they did not meet these criteria). In response to Baron and Goodwin’s (2020) critique, we reassessed the properties of the dilemmas in the final battery using the data of the eight studies reported by Gawronski et al. (2017). Our reanalysis revealed that the manipulation of consequences was well captured by every single item across studies (see Table 2). However, the manipulation of moral norms revealed anomalies for one of the six basic dilemmas (see Table 2). Specifically, we found that the Abduction Dilemma showed patterns of action vs. inaction responses that were unaffected by the manipulation of moral norms in six of the eight studies. Moreover, in the two studies that revealed a significant effect of moral norms, the pattern of responses was opposite to the intended manipulation. Based on these findings, we deem the Abduction Dilemma as not suitable for our modeling approach. The results obtained for our other five basic dilemmas generally supported their validity.

Table 2: Proportion of “action” responses as a function of consequences (benefits of action greater vs. smaller than costs), moral norms (proscriptive vs. prescriptive), and dilemma in the studies by Gawronski, Armstrong, Conway, Friesdorf, and Hütter (2017).

Given that the CNI model showed adequate fit across studies (see Table 1) and the N parameter was significantly different from zero in every single case except for a subsample of participants high in psychopathy (see above), we deem it unlikely that the poor validity of the Abduction Dilemma affected any of our findings. Nevertheless, to gain greater confidence in the reliability of our findings, we reanalyzed the data of all eight studies reported in Gawronski et al. (2017) after excluding responses to the four variants of the Abduction Dilemma.² The original results reported by Gawronski et al. (2017) are summarized in Table 3, along with the results of our reanalysis. Overall, the results of the reanalysis were highly consistent with our original findings. Of the 24 comparisons, 23 revealed equivalent results in terms of whether the comparison reached statistical significance with an alpha criterion of p < .05. The only difference was obtained for a significant effect in the original analysis that turned marginal in the reanalysis. This comparison involves a greater sensitivity to consequences among women compared to men in Study 1b, an effect that we did not interpret in the original article, because it did not emerge in Study 1a. Together, these results suggest that the inclusion of the Abduction Dilemma did not affect the findings reported in Gawronski et al. (2017). Nevertheless, based on the results of our item analysis, we recommend that researchers using the CNI model exclude the Abduction Dilemma from the pool of items. To compensate for the smaller number of items (and the resulting loss of statistical power), researchers may include the dilemmas of a newly created battery by Körner, Deutsch and Gawronski (2020).³

Table 3: Estimated parameter scores for sensitivity to consequences (C), sensitivity to moral norms (N), and general preference for inaction versus action (I) as a function of gender (Studies 1a and 1b), cognitive load (Studies 2a and 2b), question framing (Studies 3a and 3b), and psychopathy (Studies 4a and 4b). The table presents the original results reported by Gawronski, Armstrong, Conway, Friesdorf and Hütter (2017), and results with a reduced dilemma set that does not include the abduction dilemma.

6 “Perverse” Responses

In the preceding section, we responded to Baron and Goodwin’s (2020) concern that “perverse” responses on a subset of CNI model dilemmas suggest problems with the validity of these dilemmas. As we explained, the validity of a given dilemma for our modeling approach depends, not on the presence versus absence of “perverse” responses, but on the sensitivity of the four variants of a given dilemma in capturing the manipulations of consequences and moral norms. However, the mere occurrence of “perverse” responses led Baron and Goodwin to raise an additional question: what leads to “perverse” responses to our CNI model dilemmas? In their critique, Baron and Goodwin suggested two potential answers: (1) participants disagree with our assumptions underlying the manipulations of consequences and moral norms and (2) participants pay insufficient attention.

We already responded to Baron and Goodwin’s (2020) first answer in the section entitled “Moral Norms.” Regarding their second answer, we would like to reiterate that the mere occurrence of “perverse” responses is irrelevant for our modeling approach. What is relevant instead is whether responses differ as a function of our manipulations of consequences and moral norms. From this perspective, Baron and Goodwin’s question about the causes of “perverse” responses must be rephrased as: what makes participants insensitive to the manipulations of consequences and moral norms?

We fully agree that lack of attention can be an important factor in this regard. After all, not reading the stimulus materials will reduce effects of any experimental manipulation that is embedded in the verbal materials participants are asked to read, and this caveat applies to any study using text-based manipulations regardless of whether they involve the CNI model. For this reason, we generally include instructional attention checks to identify participants who may not have read the dilemmas (see Oppenheimer, Meyvis & Davidenko, 2009). Such attention checks are particularly important for analyses regarding associations between response times and dilemma judgments (such as the ones presented by Baron and Goodwin), because inattentiveness can produce artifacts resulting from lower response times among participants who do not carefully read the dilemmas. Indeed, if inattentiveness can be ruled out via attention checks, associations between response times and the CNI model parameters may indicate meaningful relations between cognitive elaboration and moral judgments. However, without proper attention checks, any such associations may be driven by lack of attention, as suggested by Baron and Goodwin (2020). We generally agree with this concern.

Aside from inattentiveness, there are multiple other factors that can reduce participants’ sensitivity to the manipulations of consequences and moral norms. However, different from Baron and Goodwin’s claim that any such effects demonstrate problems with our modeling approach, we argue that they reflect meaningful variations in moral judgments. For example, as we noted above, the only case in which the N parameter did not significantly differ from zero involved a subsample of participants who were preselected based on having psychopathy scores that placed them in the highest quartile of participants in a broader sample (Gawronski et al., 2017, Study 4b). If one were to follow Baron and Goodwin’s arguments, this result would be meaningless, because it suggests that these participants disagree with our assumptions underlying the manipulation of moral norms. However, based on the known characteristics of individuals high in psychopathy (Hare & Neumann, 2008), a more plausible interpretation is that individuals high in psychopathy are less sensitive to moral norms than individual low in psychopathy. This conclusion is consistent with the difference between participants high versus low in psychopathy obtained by Gawronski et al. (2017, Study 4b) and successful replications of this effect in several follow-up studies (Körner et al., 2020; Luke & Gawronski, in press). In our view, these results do not call the validity of the CNI model into question. Instead, they demonstrate the validity of the model and its ability to provide more nuanced insights into central questions at the intersection of moral and clinical psychology. We will return to these findings in the section entitled “Confounds in the Traditional Approach.”

Another argument put forward by Baron and Goodwin is that the proportion of “perverse” responses is reduced if participants’ own judgments are used to identify the relevant moral norm in a given dilemma. As we explained above, we deem this approach problematic, because it conflates two distinct levels of analysis (see De Houwer, 2011; Gawronski & Bodenhausen, 2015). In fact, classification of a behavioral effect by means of underlying mental constructs would lead to explanatory circularity, because the to-be-explained effect becomes conceptually equivalent with the construct that is supposed to explain the effect (see De Houwer, Gawronski & Barnes-Holmes, 2013; Gawronski & Bodenhausen, 2015). That is, if norm-congruent behavior would qualify as an effect of norms only if it is mediated by conscious thoughts about norms, conscious thoughts about norms would not explain norm-congruent behavior, because the two would be conceptually identical. It makes little sense to explain norm-congruent behavior by conscious thoughts about norms if norm-congruent behavior is defined by conscious thoughts about norms. Such a conceptualization would also preclude the possibility that norm-congruent behavior might be mediated by other mental processes that do not involve conscious thoughts about norms, such as the affective mechanisms proposed by Greene’s (2008) dual-process theory (see also Haidt, 2001).

For these reasons, we reject Baron and Goodwin’s idea of using participants’ self-classifications as a criterion for moral norms. Instead, we would argue that the mechanisms underlying norm-congruent judgments is an empirical question. It might be conscious thoughts about moral norms, affective responses to the idea of causing harm, or something entirely different. In fact, it is possible that multiple distinct processes can produce the same pattern of norm-congruent judgments. Similar considerations apply to effects of consequences, which may be driven by conscious reasoning about consequences or an entirely different set of processes. In hindsight, we acknowledge that the description of the N parameter as an indicator of sensitivity to moral norms is not ideal, because it is ambiguous about the difference between behavioral effects and underlying mental processes. A less ambiguous description might refer to intrinsic aspects of the focal action (e.g., actions causing harm vs. actions preventing harm; see Cushman, 2013) rather than the congruency of these actions with proscriptive and prescriptive norms. Such a description would also address the abovementioned concern that congruence with moral norms is not a unique feature of the N parameter (see discussion under “Deontological Responding”). Yet, regardless of the chosen label, the significance of the N parameter in understanding moral dilemma judgments is supported by the finding that the N parameter explains a considerable amount of variance in traditional dilemma scores and accounts for a large number of experimental effects and associations with individual-difference variables. We will return to this issue in the section entitled “Confounds in the Traditional Approach.”

7 Order Effects and Presumed Inconsistencies

Multinomial modeling is not just an alternative way of analyzing data. Different from standard data analytical approaches (e.g., General Linear Model), multinomial modeling integrates data analysis with theoretical assumptions about observed response patterns (Hütter & Klauer, 2016). In the CNI model, these assumptions are reflected in the hierarchical relation of the three parameters in the processing tree (see Figure 1), which can be criticized as arbitrary, because they cannot be tested empirically. As we explained in the original article presenting the CNI model (Gawronski et al., 2017, Footnote 7), changes in the order of parameters in the processing tree do not affect the fit of the model, in that all six combinatorially possible models show the same goodness-of-fit for a given data set. It is therefore not possible to empirically distinguish between these models, rendering assumptions about the hierarchical structure of the three parameters arbitrary.

Baron and Goodwin express two concerns about this aspect of the CNI model. First, they claim that the order of C and N in the processing tree conflicts with the corrective dual-process theory (Greene, 2008), which suggests that deontological judgments are the product of automatic processes, whereas utilitarian judgments are the product of controlled processes that correct the impact of automatically generated deontological judgments. Second, they claim that models with different orders of C and N in the processing tree lead to inconsistent relations with external variables. Both claims are incorrect.

Regarding the consistency of the parameter hierarchy with the corrective dual-process theory (Greene, 2008), it is important to note that the hierarchical structure reflects conditional relations of the parameters in determining behavioral outcomes, not their temporal order in which their underlying processes occur. In technical terms, the hierarchical structure of parameters specifies conditional probabilities of input-output relations, and time is not a meaningful variable in the mathematics of conditional probabilities. In the CNI model, we chose the current order of C and N to be consistent with Conway and Gawronski’s (2013) earlier work using process dissociation, and the order of the corresponding parameters in their process dissociation model was chosen precisely because it is the one suggested by the corrective dual-process theory. According to the theory, the processes underlying deontological judgments influence outcomes only if they are not overridden by the corrective processes underlying utilitarian judgments. In other words, the corrective processes underlying utilitarian judgments will drive responses whenever these processes become active, and the processes underlying deontological judgments will drive responses only if the corrective processes underlying utilitarian judgments fail to drive judgments. In Conway and Gawronski’s (2013) process dissociation model, these assumptions are reflected in the dominant status of the U parameter (claimed to reflect utilitarian inclinations) compared to the subordinate status of the D parameter (claimed to reflect deontological inclinations) in the hierarchy of parameters. Within the CNI model, these assumptions are retained in the dominant status of the C parameter compared to the subordinate status of the N parameter. As we explained in the original article presenting the CNI model (Gawronski et al., 2017, Footnote 7), the position of the I parameter was chosen to permit an estimation of general action preferences along a bipolar continuum of inaction versus action instead of a unipolar dimensions reflecting relative differences in the general preference for inaction. This aspect requires that the I parameter is included at the lowest location in the hierarchy instead of being at one of the two superordinate locations.

Although the position of the I parameter can be justified on methodological grounds, we agree with Baron and Goodwin (2020) that the relative position of C and N in the processing tree remains arbitrary, raising the question of whether reversing their positions qualifies any of the reported results. Mathematically, reversing the positions of C and N changes the model equations, in that the score estimated for N is conditional upon 1 − C in the original model, whereas the score estimated for C is conditional upon 1 − N in the reversed model (see Appendix B). Although these changes do not affect the goodness-of-fit of the model (see above), they can influence relations with external variables. Thus, it is perfectly justified to ask the question of whether the findings obtained with the CNI model depend on the chosen position of C and N in the processing tree.

Baron and Goodwin (2020) claim that reordering C and N in the processing tree leads to inconsistent results, with some external variables (e.g., gender) showing relations to the parameters of the reversed model that are opposite to the ones obtained for the original model. However, a closer inspection of the results indicates that this claim is incorrect. Table 4 provides a summary of the results obtained with the original model (C-N-I order), along with a summary of the results obtained with a model in which the positions of C and N are reversed (N-C-I order). For 21 of the 24 comparisons, the two models produce the same outcomes in terms of whether the comparison reached statistical significance with an alpha criterion of p < .05. In two cases, a marginal difference obtained with the original model reaches statistical significance with the reversed model. These cases involve the effect of question framing on the N parameter in Study 3a and the association of psychopathy with the C parameter in Study 4a. In one case, a non-significant difference in the original model is statistically significant in the reversed model. This case involves the association between gender and the C parameter in Study 1a. Counter to Baron and Goodwin’s (2020) claim, all of the obtained relations between the three model parameters and external variables are directionally consistent across the two model variants.⁴

Table 4: Estimated parameter scores for sensitivity to consequences (C), sensitivity to moral norms (N), and general preference for inaction versus action (I) as a function of gender (Studies 1a and 1b), cognitive load (Studies 2a and 2b), question framing (Studies 3a and 3b), and psychopathy (Studies 4a and 4b). The table presents the results of Gawronski, Armstrong, Conway, Friesdorf and Hütter’s (2017) studies using the original CNI model (C-N-I order) and a modified version in which the order of N and C are reversed in the hierarchical structure of the model (N-C-I order).

How is it possible that Baron and Goodwin (2020) reached an entirely different conclusion in their own reanalysis of our data? The simple reason is that they did not use the CNI model in their reanalysis. Instead, they used two versions of Conway and Gawronski’s (2013) process dissociation model, one in which U dominates over D (as in Conway and Gawronski’ model) and one in which D dominates over U (opposite to the order in Conway and Gawronski’s model). Baron and Goodwin then used the two versions to separately analyze responses to dilemmas with either proscriptive or prescriptive norms. Using this approach to reanalyze the data from Studies 1a and 1b of Gawronski et al. (2017), Baron and Goodwin identified inconsistencies in the relation between gender and the U parameter across the four cases, with one case showing gender effects that are directionally opposite to the ones in the other three cases.

We deem these findings uninformative for the validity of the CNI model, because (1) the analyses did not use the CNI model and (2) a proper analysis with the CNI model yields consistent results across model specifications (see Table 4). Indeed, there is a simple technical explanation why Baron and Goodwin’s (2020) reanalysis using the PD model to compare results across dilemmas with proscriptive and prescriptive norms yields inconsistent results. In the PD model, general preference for inaction versus action is confounded with the two constructs of interest (see Gawronski et al., 2016; Hütter & Klauer, 2016), and these confounds are directionally opposite in cases involving proscriptive and prescriptive norms. For dilemmas involving proscriptive norms, a general preference for inaction versus action increases scores on the D parameter and decreases scores on the U parameter. For dilemmas involving prescriptive norms, a general preference for inaction versus action decreases scores on the D parameter and increases scores on the U parameter. Because the relative impact of these confounds varies as a function of the hierarchy of the two parameters in the processing tree, they can lead to inconsistent results as a function of model specifications (i.e., U-D order vs. D-U order) and type of norm (proscriptive vs. prescriptive) if a given variable (e.g., gender) is associated with general preference for inaction versus action. By controlling for general action preferences in the I parameter, the CNI model not only resolves these problems; it also leads to consistent results irrespective of the position of C and the N in the processing tree (see Table 4). Thus, although the results of Baron and Goodwin’s (2020) reanalysis echo our own concerns about conceptual problems of Conway and Gawronski’s (2013) PD model (see Gawronski et al., 2016; Hütter & Klauer, 2016), they have no implications for the validity of the CNI model.

8 Confounds in the Traditional Approach

Baron and Goodwin (2020) conclude their critique stating that the “CNI model requires use of congruent items that must yield enough ‘perverse’ responses (those that both violate norms and produce worse consequences) so that the model provides results that differ from standard analysis” (p. 434). As we explained above, their focus on ‘perverse’ responses reflects a misunderstanding of the model’s underlying idea. Whereas the C parameter quantifies the extent to which responses are influenced by consequences and the N parameter quantifies the extent to which responses are influenced by moral norms, the I parameter quantifies the extent to which responses reflect a general preference for inaction versus action. In technical terms, the C and the N parameter quantify the impact of two experimental manipulations; the I parameter is conceptually similar to an intercept in a General Linear Model, in that it captures general response tendencies independent of the two experimental manipulations. Thus, what matters within the CNI model are differences (or the lack of differences) in responses across dilemmas involving different consequences and moral norms, not absolute responses to a particular kind of dilemma. In fact, counter Baron and Goodwin’s claim that the CNI model requires “perverse” responses, substantial proportions of “perverse” responses would be detrimental to the model, because they would lead to (1) poor model fit in the description of data due to violations of model assumptions and (2) estimates for the C and the N parameters close to zero due to reduced effects of their corresponding manipulations.

Counter to Baron and Goodwin’s (2020) preference for the traditional approach of pitting moral norms against consequences for the greater good, we argue that the three factors captured by the CNI model are confounded in the traditional approach. These confounds render any finding with the traditional approach ambiguous, in that differences in the preference for “utilitarian” over “deontological” judgments may be driven by differences in the sensitivity to consequences, differences in the sensitivity to moral norms, or differences in the general preference for inaction versus action (or any combination of the three).

The conflation of the three factors in the traditional approach can be illustrated by means of multiple regression analyses, in which preference for “utilitarian” over “deontological” judgments on traditional dilemmas pitting a proscriptive norm against consequences for the greater good is predicted by the three CNI parameters. Table 5 presents the results of such multiple regression analyses for a series of four studies by Körner et al. (2020).⁵ Because responses on traditional dilemmas are used in the CNI model equations to estimate numerical values for the three parameters (see Appendix A), we ensured mathematical independence of predictors and outcomes by using CNI model parameters for dilemmas with odd item numbers to predict traditional dilemma scores for dilemmas with even item numbers. Conversely, we used CNI model parameters for dilemmas with even item numbers to predict traditional dilemma scores for dilemmas with odd item numbers. For the four studies reported by Körner et al. (2020), this approach led a total of eight multiple regressions. In Studies 1a and 1b, participants were asked to judge whether the described action is acceptable. In Studies 2a and 2b, participants were asked whether they would perform the described action.⁶

Table 5: Results of multiple regression analyses predicting traditional dilemma scores (i.e., preference for “utilitarian” over “deontological” judgments on dilemmas that pit a proscriptive norm against consequences for the greater good) by sensitivity to consequences (C), sensitivity to moral norms (N), and general preference for inaction versus action (I). For CNI parameters marked odd, the CNI parameters were calculated based on dilemmas with odd trial numbers and traditional dilemma scores were calculated based on dilemmas with even trial numbers. For CNI parameters marked even, the CNI parameters were calculated based on dilemmas with even trial numbers and traditional dilemma scores were calculated based on dilemmas with odd trial numbers. Reanalysis of data by Körner, Deutsch, and Gawronski (2020).

Table 5 shows the results of the multiple regression analyses. For the N parameter, greater sensitivity to moral norms showed a significant negative association with preference for “utilitarian” over “deontological” judgments in all eight cases. For the C parameter, greater sensitivity to consequences showed a significant positive association with preference for “utilitarian” over “deontological” judgments in seven of the eight regressions. For the I parameter, the results were somewhat less reliable, in that general preference for inaction versus action showed a significant negative association with preference for “utilitarian” over “deontological” judgments in four of the eight regressions. A potential reason for the mixed results for the I parameter is that estimates for parameters with a lower position in the hierarchy of multinomial processing trees tend to be less reliable compared to estimates for parameters with a higher position in the hierarchy.⁷ Nevertheless, the regression results indicate that the three CNI parameters capture unique variance in traditional dilemmas scores and the direction of their associations is consistent with the proposed confounds.

By disentangling the three determinants of moral dilemma judgments, the CNI model provides much more nuanced insights that cannot be gained with the traditional approach. For example, although reduced preference for “utilitarian” over “deontological” judgments under cognitive load has been interpreted as reflecting the resource-dependence of utilitarian reasoning about costs and benefits (Greene, Morelli, Lowenberg, Nystrom & Cohen, 2008), results obtained with the CNI model suggest that cognitive load influences moral dilemma judgments by increasing general preference for inaction versus action (Gawronski et al., 2017, Studies 2a and 2b). Moreover, Gawronski and Brannon (2020) found that greater preference for “utilitarian” over “deontological” judgments resulting from recalling personal memories of high (vs. low) power is driven by a reduced sensitivity to moral norms rather than increased sensitivity to consequences or reduced action aversion. Findings by Gawronski et al. (2018) further indicate that happiness increases the preference for “utilitarian” over “deontological” judgments (see Valdesolo & DeSteno, 2006) by reducing sensitivity to moral norms, suggesting that happiness influences moral judgments by damping negative affective responses to the idea of violating moral norms (see Nichols & Mallon, 2006) rather than negative affective responses to the idea of causing harm (see Greene, 2008). Finally, Gawronski et al. (2017, Studies 3a and 3b) found that personal involvement decreased sensitivity to moral norms and increased general preference for inaction versus action (see also Körner et al., 2020), indicating that the same factor can have opposite effects on the two patterns of deontological responding.

In our view, the most interesting insights are provided by studies that have used the CNI model to investigate associations between psychopathy and moral dilemma judgments (e.g., Gawronski et al., 2017, Studies 4a and 4b; Körner et al., 2020; Luke & Gawronski, in press). Numerous previous studies using the traditional dilemma approach have found a positive association between psychopathy and preference for “utilitarian” over “deontological” judgments (for a meta-analysis, see Marshall, Watts & Lilienfeld, 2018). Some researchers have interpreted this finding as evidence for a major flaw of moral dilemma research, because it is obvious that individuals high in psychopathy do not care about the greater good in a utilitarian sense (e.g., Bartels & Pizarro, 2011; Kahane, Everett, Earp, Farias, Savulescu, 2015). Research using the CNI model resolves this paradox, showing that individuals high in psychopathy differ from those low in psychopathy by showing (1) a weaker sensitivity to consequences on the C parameter, (2) a weaker sensitivity to moral norms on the N parameter, and (3) a weaker general preference for inaction versus action on the I parameter. The most interesting aspect of these findings is that individuals high in psychopathy showed a weaker sensitivity to consequences in a utilitarian sense, which stands in contrast to the positive association between psychopathy and preference for “utilitarian” over “deontological” judgments in the traditional approach. This discrepancy can be explained by the finding that psychopathy shows a large negative association with sensitivity moral norms and a moderate negative association with general preference for inaction versus action, which conceal a moderate negative association with sensitivity to consequences when the three factors are confounded in the traditional dilemma approach. Further research by Luke and Gawronski (in press) suggests that some of these associations are driven by a poor understanding of societal standards about right and wrong among individuals high in psychopathy. For other associations, the results suggest that psychopaths are aware of societal standards about right and wrong, but do not care about using these standards in their personal judgments. These complex findings speak against the possibility that associations between psychopathy and the three CNI model parameters merely reflect inattentiveness among individuals high in psychopathy, as Baron and Goodwin (2020) might argue based on their critique.

9 Conclusion

We greatly appreciate the opportunity to correct potential misconceptions about the CNI model, like the ones reflected in Baron and Goodwin’s (2020) critique. Although their critique reveals a need for greater precision in the description of the three model parameters and for greater attention to properties of individual dilemmas, we deem their dismissal of the CNI model as misguided, because it is based on: (1) misunderstandings of key aspects of the model; (2) a conceptually problematic conflation of behavioral effects and explanatory mental constructs; (3) arguments that are inconsistent with empirical evidence; and (4) reanalyses that supposedly show inconsistent findings resulting from changes in model specifications, although the reported reanalyes did not actually use the CNI model and proper analyses with the CNI model yield consistent findings across model specifications. Based on the available evidence, we conclude that the CNI model is a valid, robust, and empirically sound approach to gaining deeper insights into the determinants of moral dilemma judgments, overcoming major limitations of the traditional approach.