Vulnerability Appeals in the COVID-19 Pandemic: Insights from a National Survey Experiment

Abstract

This study explores the impact of vulnerability appeals during the COVID-19 pandemic using a nationally representative, preregistered survey experiment (N = 4,087) conducted in mid-2021. We explore whether providing citizens with information about the vulnerability of ethnic minority and disabled citizens to COVID-19 fosters empathy and increased support for behavioral restrictions. We observe minimal statistically significant or substantive effects, although the presence of subtle effects cannot be entirely ruled out. We identify some limited indications that individuals with disabilities exhibit increased support for restrictions when exposed to information about the vulnerability of disabled people to COVID-19, but these effects are inconsistent. Therefore, our findings provide limited evidence to confirm or rule out that using vulnerability appeals alone is effective for influencing public attitudes toward behavioral restrictions. The findings point toward avenues for future research, including a closer examination of heterogeneous responses to public health messaging among population subgroups.

Introduction

At the onset of the COVID-19 pandemic, governments worldwide rushed to implement new policy measures in an effort to curb the spread of the virus. These measures, ranging from stringent lockdowns and curfews to the closure of schools and businesses and mandatory use of personal protective equipment, resulted in profound disruption to the daily lives of citizens. In this critical period, policymakers faced the urgent challenge of developing mass communication strategies that would effectively promote compliance and cooperation with disruptive public health measures.

Early in the pandemic, it became clear that the health risks posed by COVID-19 varied dramatically among individuals with different health and demographic characteristics (Nasserie et al., 2021). Thus, a common rhetorical strategy deployed by governments was to encourage compliance by underscoring the vulnerability of members of particular minority groups to severe illness and death from COVID-19. The logic of this strategy was straightforward: individuals should seek to stop the spread of COVID-19 to protect not only themselves, but those most vulnerable to severe health outcomes. ¹ In the UK, two groups most at risk from COVID-19 were people with disabilities (PWDs) ² and people from Black and ethnic minority backgrounds. Members of both these groups were significantly more likely to be hospitalized as a result of contracting the virus (Public Health Wales, 2022, 2021), more likely to be admitted to intensive care (Thomas et al., 2021; Office for National Statistics, 2022; Kavanagh et al., 2022), and more likely to die as a result of COVID-19 (Perera et al., 2020; Public Health England, 2020). In Wales, where this study was conducted, political elites routinely emphasized the vulnerability of these groups in public communications. As illustrated in Fig. 1, a simple search of official parliamentary transcripts and governments speeches reveals that both these groups featured prominently in elite communications in the roughly 14 months leading up to the fielding of this study. However, while such appeals were widespread, their effectiveness at motivating compliance with COVID-19 restrictions remains uncertain.

Figure 1. Number of appeals encouraging citizens to follow restrictions to protect Black and minority ethnic, and disabled people, made during Senedd plenary speeches and First Minister televised briefings between March 1, 2020 and May 31, 2021.

Addressing this gap, we ask: to what extent does priming the vulnerability of certain minority groups to COVID-19 promote empathy and prosocial behavior? We address this question using a nationally representative survey experiment embedded in the Welsh Election Study (N = 4,087), fielded by YouGov in May 2021. Specifically, we examine whether appeals which underscore the vulnerability of disabled and minority ethnic citizens to COVID-19 promote affective concern and support for virus-curbing restrictions. Our findings suggest such appeals have, at best, a limited and small effect on attitudes when used in isolation. We find no evidence that vulnerability appeals substantially increase affective concern or support for behavioral restrictions, and we find some evidence of a backlash effect (diminished affective concern) in response to messages that cue both ethnic minorities and disabled people. By contrast, we find some suggestive evidence that when exposed to information about the vulnerability of disabled people in particular to COVID-19, disabled respondents report stronger support for some behavioral restrictions (we observe no statistically significant change in affective concern) than their nondisabled peers. Finally, we discuss potential limitations of our design and offer suggestions for future research.

The effectiveness of COVID-19 messaging

Existing research examining the effects of public health messaging throughout the pandemic has yielded mixed results. Messaging that emphasized the public health benefits of pandemic-related restrictions appears to have been effective in some contexts, with positive effects found in the US for intention to wear masks and perceived effectiveness of masks (Carey et al., 2022), and intention to travel less (Deslatte, 2020). Furthermore, Adida et al. (2023) highlighted the substantive impact of social norms in increasing support for mask-wearing among white American Evangelicals. However, the weight of experimental survey research on the effect of public health messaging has returned null findings on a range of attitudes and intentions (e.g. Case et al., 2022; Utych, 2021; Kuipers et al., 2021). Economic frames also appear largely ineffective, either producing null effects (Deslatte, 2020), unintended reductions in support for restrictions (Carreras et al., 2021), or have failed to replicate (Knapp et al., 2023). Similarly, Favero and Pedersen (2020) found that various prosocial frames had no effect on either respondents’ willingness to engage in social distancing behaviors, or their beliefs about the virus. Nevertheless, significant gaps remain in our understanding of what kinds of messages were successful in motivating compliance with COVID-19 restrictions. In particular, while messages which underscored the vulnerability of minority groups (e.g. PWDs, ethnic minorities) were commonplace throughout the pandemic, we do not yet know whether such messages were successful in motivating empathic concern or a willingness to abide by COVID-19 restrictions.

Stereotypes, group identity, and vulnerability appeals

Research in cognitive psychology suggests the effectiveness of vulnerability appeals is likely to vary based on the content of common group stereotypes (Fiske et al., 2002; Fiske, 2015). In the absence of specific information, people tend to rely on cognitively accessible stereotypes when forming judgments about the behavior and intentions of others (Allport et al., 1954; Fiske and Taylor, 1991). Thus, we suspect that when confronted with appeals that cue broad social categories – in our case, “people with disabilities” and “Black and ethnic minority people” – citizens are likely to respond in ways which reflect the content of common group stereotypes.

Existing work repeatedly finds that across diverse social and political contexts, PWDs are stereotyped as high in “warmth” and low in “competence” (Fiske et al., 2002, 2007). In other words, while PWDs are seen as friendly and agreeable, they are stigmatized as dependent and low in social status (Canton et al., 2023; Fiske et al., 2002; Nario-Redmond, 2010). Existing research has examined a range of group labels, including “disabled,” “physically disabled,” “blind,” and “mentally retarded,” ³ and find that all are similarly characterized as high in warmth and low in competence (Canton et al., 2023; Fiske et al., 2002). Importantly, these stereotypes are strongly associated with help-giving emotions, such as pity, compassion, and paternalistic concern (Goetz et al., 2010; Cuddy et al., 2007) These stereotypes are reflected in the welfare attitudes literature, in which PWD are reliably identified as among the most deserving recipients of government assistance (Coughlin, 1980; Van Oorschot, 2000, 2006; Thorp and Larner, 2024).

By contrast, stereotypes assigned to ethnic minorities are more variable. On the one hand, ethnic minorities perceived as poor or migrants are often stereotyped as low in both warmth and competence (Fiske et al., 2002; Fiske, 2015), which can elicit feelings of contempt and a reluctance to extend assistance (Cuddy et al., 2007; Petersen, 2012). Similar stereotypes have been applied to more general ethnic minority cues – such as “Blacks” and “Hispanics” – though these groups tend to be middling in both warmth and competence (Cuddy et al., 2009). Other more specific minority groups – such as British Indians or Asian Americans – are stereotyped as low in warmth but high in competence (Fiske, 2018). These stereotypes tend to elicit feelings of envy and intergroup threat (Cuddy et al., 2007). Importantly, the stereotypes assigned to ethnic minorities are consistently lower in warmth than those assigned to PWD, making them relatively less-likely to elicit help-giving emotions and behaviors (Fiske et al., 2002; Cuddy et al., 2007). Based on these findings, we expect respondents to report relatively more affective concern and support for behavioral restrictions in response to messages which underscore the vulnerability of disabled people as opposed to ethnic minorities.

Group stereotypes may also shape the perceived plausibility of vulnerability information. Unlike ethnicity, disability is defined by limitations in functioning resulting in part from physical or mental impairments, which can have downstream implications for other health outcomes (Krahn et al., 2015; Mitra et al., 2022; Nario-Redmond, 2019). Consequently, disability is often psychologically conflated with sickness and disease by external observers (Park et al., 2003), and individuals with specific impairments are often assumed to experience more global limitations in functioning (Nario-Redmond, 2010, 2019). Given the close conceptual association between disability and health, we may expect respondents to perceive disability status to be more directly or intrinsically related to health outcomes than other embodied characteristics, such as ethnicity. Thus, respondents may be relatively more likely to perceive information about the health vulnerability of disabled people to be convincing or plausible than appeals which emphasize the vulnerability of ethnic minority groups. These findings inform H1–H5 of our preregistered hypotheses (below). We predict that appeals which emphasize only the vulnerability of PWD will elicit more empathic concern and support for behavioral restrictions than appeals which emphasize the vulnerability of ethnic minorities, or appeals which feature both ethnic minorities and PWD.

Finally, we expect that affective concern and support for restrictions will be stronger on average among (1) disabled respondents and (2) respondents who have close relatives with disabilities. Intuitively, disabled respondents and their close relations may report stronger support for restrictions out of an instrumental desire to mitigate the risk of severe illness to themselves and their kin. On the other hand, recent work in social psychology suggests many PWDs self-consciously identify as members of a stigmatized minority group (Bogart, 2014; Nario-Redmond and Oleson, 2016), and report feelings of social and political solidarity with other PWDs (Bogart et al., 2018; Dirth and Branscombe, 2019; Nario-Redmond et al., 2013). Thus, we expect that information about the vulnerability of PWD to COVID-19 will inspire feelings of group-level political solidarity, resulting in greater affective concern and support for restrictions on average among PWD and their close relations (H6). ⁴

Experimental design

This preregistered experiment was embedded in a larger election study that was conducted online by YouGov between May 11 and May 27, 2021 (Wyn Jones et al., 2023). This was a time when COVID-19 cases and related deaths were at a relatively low point in Wales (see Fig. 2), but a majority of government restrictions were still in place (see Supplementary Materials for detailed timeline). A total of 4,087 respondents were recruited from YouGov’s online panel of over 1 million British adults with a sampling frame to approximate the demographic composition of the Welsh population. Additional poststratification weights are applied to all analyses so that model estimates can be interpreted as nationally representative. ⁵

Figure 2. Daily cases (people who have had at least one positive COVID-19 test result) and deaths in Wales in 2021 (Public Health Wales, 2022).

The postelection survey took approximately 15 minutes to complete, with the experiment placed at the end of the questionnaire. The full questionnaire is included in the supplementary materials. Respondents were randomly assigned to one of four experimental conditions – a control condition, a disability condition, an ethnic minority condition, and a combined condition. In each condition, respondents were provided with the same factual information about the COVID-19 pandemic including the number of UK residents who had been hospitalized or died as a result of the COVID-19 pandemic. In the three treatment conditions, this information was followed by a statement about how different groups – disabled people, Black and minority ethnic people, and disabled and Black and minority ethnic people – were particularly at risk of severe illness and death. The full wording is provided in Table 1.

Table 1. Experimental manipulation

Outcome measures

After reading the treatment text, respondents were asked the extent they agreed with six different statements on a 5-point scale (1–5, strongly disagree – strongly agree). The order of these statements was randomized. Three statements measured respondents’ attitudes toward behavioral restrictions:

• “It should be compulsory for people to wear masks in places where lots of people are gathered (e.g. supermarkets, shopping malls, public transport)”

• “It is worth temporarily sacrificing some of our personal freedoms if it means protecting those who are most vulnerable to COVID-19”

• “There should not be another lockdown if cases of COVID-19 begin to increase again”

Three further statements measured affective responses toward those most vulnerable to COVID-19:

• “I am very concerned about those most vulnerable to COVID-19”

• “I feel anger towards people who refuse to take action to protect those most vulnerable to COVID-19”

• “I am quite moved by what can happen to those most vulnerable to COVID-19”

Hypotheses

We preregistered the following hypotheses:

• H1: The disability condition will elicit greater anger and empathic concern than the control, ethnic minority, or mixed conditions.

• H2: The disability condition will elicit more prosocial action than the control, ethnic minority, or mixed conditions.

• H3: The mixed condition will elicit greater anger and empathic concern than the ethnic minority or control conditions.

• H4: The mixed condition will elicit more prosocial action than the ethnic minority or control conditions.

• H5: The ethnic minority condition will not elicit a significantly greater degree of anger, empathic concern, or prosocial action than the control condition.

• H6: Disabled people and proximate relations of disabled people will exhibit greater empathic concern, anger, and prosocial action in response to the disability condition than nondisabled respondents, and respondents without disabled proximate relations.

Empirical strategy

The analysis plan for this experiment was preregistered at AsPredicted: aspredicted.org/blind. We do not deviate from the preregistration plan. To test hypotheses 1–5, we estimate the average treatment effect (ATE) of exposure to the manipulations outlined above using OLS:

(1) $${Y_i} = \alpha + {\beta _1}{\rm{Trea}}{{\rm{t}}_i} + \varepsilon $$

Where ${Y_i}$ is respondent $i$ ’s attitude toward one of the six outcome variables listed above, and $Treat$ is a categorical variable for treatment group. To test Hypothesis 6, we compute the conditional average treatment effect (CATE) – that is, the effect of exposure to the treatment interacted with disability status, or relational proximity to PWD (in separate models). Power calculations for the estimation of the CATE are provided in Tables 7 and 8 of the Supplementary Materials. CATEs are computed using OLS:

(2) $${Y_i} = \alpha + {\beta _1}{\rm{Trea}}{{\rm{t}}_i} + {\beta _2}{\rm{Disability}} + {\beta _3}\left( {{\rm{Treat*Disability}}} \right) + \varepsilon $$

In addition to calculating the ATE and CATE for each individual outcome variable, we also create two indices; the first comprised of the three prosocial behavior items, and the second comprised of the three affective items. ⁶ We run the same analyses outlined above using these as outcome measures. This strategy was not included in the study’s preregistration. We conduct significance tests using p < 0.01 thresholds in addition to p < 0.05 and report both confidence intervals for treatment effect estimates in all figures. To assess the precision of any null results we observe for main effects, we report equivalence bounds using a two one-sided tests approach (Lakens et al., 2018). All models include robust standard errors and sampling weights.

Results

Results of the models testing our first five hypotheses are presented in Fig. 3. All outcome variables are standardized between 0 and 1 for ease of interpretation. Our first two hypotheses predicted that treatment effects would be greatest in the disability treatment across affective and prosocial behavior outcomes. Contrary to expectations, we do not observe any effects significant at the 95% level in any of our outcome measures including both indices. For example, the disability treatment yielded an ATE of $\beta = - 0.010$ (p = 0.485) on the compulsory masks outcome, and an effect of $\beta = - 0.011$ (p = 0.427) on the anger outcome. Of course, failure to reject the null is not the same as confirming the null. Here we estimate equivalence bounds using two one-sided t-tests. ⁷ In only one of the outcome variables – Sacrifice Freedoms – do the 90% confidence intervals fall fully within the estimated equivalence bounds meaning that we can reject the alternative hypothesis. For all other outcome variables, 90% confidence intervals fall outside of the equivalence bounds, and we therefore fail to reject the alternative hypothesis.

Figure 3. Effect of vulnerability treatments on outcome measures. Shown with 95% and 99% confidence intervals. Regression tables reported in Section E of the Supplementary Materials.

In Hypotheses 3 and 4, we predicted that the combined treatment (disability + ethnic minority) would yield greater prosocial behavior and empathic concern compared to the control and the ethnic minority treatment. We observe no support for these expectations. In fact, contrary to our expectations, we observe a statistically significant negative effect in four of our outcome variables: three at the 95% threshold (Masks compulsory $\beta = - 0.037,p\,=\,0.017$ , Anger $\beta = - 0.033,p\,=\,0.035$ , and the prosocial index $\beta = - 0.028,p=0.029$ ) and three at the 99% threshold (Concerned $\beta = - 0.039,p=0.004$ , Moved $\beta = - 0.038,p=0.007$ , and the affective index $\beta = - 0.037,p=0.003$ ). No statistically significant effects are observed for “Sacrifice freedoms” or “No more lockdowns,” but in both cases, 90% confidence intervals fall beyond equivalence bounds ([−0.0239, 0.0239] and [−0.09, 0.09], respectively) meaning that we cannot confidently rule out the presence of a small effect. Relative to the ethnic minority condition, a statistically significant difference is only observed in the “Moved” outcome, but this is a substantively small difference. In all but one outcome – Anger – equivalence test fails to reject the alternative hypothesis (see Table 12 of the Supplementary Materials).

Finally, in Hypothesis 5, we predicted that the ethnic minority treatment would not yield statistically significant results from the control condition in any outcome variables. While this is confirmed in four outcome variables, again contrary to our expectations we observe negative treatment effects at the 95% threshold in two outcomes: Concerned ( $\beta = - 0.029,p=0.026$ and Anger $\beta = - 0.031,{\rm{\;}}p=0.040$ ). Here, all equivalence tests fail to reject the alternative hypothesis, meaning we cannot confidently rule out a substantively small difference between these conditions.

Interactions (H6)

In Hypothesis 6, we predicted that we would observe treatment heterogeneity in the disability condition among respondents who are either themselves disabled, or who have close kin with disabilities. In our sample 29.15% (N = 1,176) respondents said they had a disability and 17.87% (N = 721) of respondents said they were close to someone with a disability ⁸ . Fig. 4 displays the results from our test of Hypothesis 6a, interacting a respondent’s disability status with the disability treatment. Here, we predicted that PWD would be particularly sensitive to the disability treatment, and as a result we would observe larger effects within this group of respondents. Evidence supporting this hypothesis is mixed across outcome variables. For two of our measures of prosocial behavior – agreeing masks should be compulsory in public places and willingness to sacrifice personal freedoms to protect others – we observe positive effects significant at the 99% threshold ( $\beta =0.10,\;p\,=\,0.004$ and $\beta \,=\,0.089,\;p\,=\,0.010$ respectively).

Figure 4. Treatment interacted with Disability. Shown with 95% and 99% confidence intervals. Regression tables reported in Section E of the Supplementary Materials.

Nondisabled respondents did not respond in a similar manner to the disability treatment: we observe no substantive or statistically significant effects. In other words, there is some evidence that disabled respondents appear to be more sensitive to the disability treatment than their nondisabled peers, though the absence of any statistically significant effect in any of our affective measures does highlight the inconsistency of this finding. Likewise, our test of Hypothesis 6b – where we interacted for treatment condition with an indicator for relational proximity to PWD – produced no statistically significant effects. Results are not reported in the main body of the text for conciseness, but complete results are available in Table 8 of the Supplementary Materials. In addition to these interactions, we preregistered an interaction between respondent ideology and the treatments, though we did not preregister any related hypotheses. Again, we observed no substantive or statistically significant effects. ⁹

Discussion

Throughout the COVID-19 pandemic, governments aimed to motivate compliance with behavioral restrictions using messages which emphasized the vulnerability of particular minority groups to severe illness and death. Using a large, preregistered survey experiment embedded in the 2021 Welsh Election Study, this paper has sought to evaluate the effectiveness of these kinds of appeals. We found very little support for the majority of our preregistered hypotheses. Contrary to expectations, none of our manipulations produced substantial increases in either affective concern or support for behavioral restrictions, and attitudes did not significantly vary in response to different group cues. Importantly, the results of our equivalence tests suggest we cannot rule out the presence of a substantively small effect. However, given the negative direction many of the results presented in Fig. 3, in most cases these results still run contrary to our theoretical expectations.

We also found some limited, but nonetheless significant, evidence of a backlash effect, in that both affective concern and support for restrictions were significantly diminished in response to the combined treatment. While the exact reasons for this response are unclear, it is possible that some respondents interpreted the treatment as trying to equate the experience of social groups with different characteristics and different levels of vulnerability to COVID-19. This may have reduced the plausibility of the treatment and led some respondents to react in a contrary or hostile manner. Finally, the results of our interaction models provide some evidence that appeals which targeted PWD led to increased support for behavioral restrictions among disabled respondents. More work needs to be done to clarify the psychological mechanisms underlying these results. Given that we observe no significant change in affective concern among PWD in response to the disability condition, we think it is probably more likely that this result is driven by a rational desire among PWD to mitigate their own risk of severe adverse health outcomes, rather than a broader sense of subjective identification with PWD. ¹⁰ On the other hand, given the inconsistency of these results, we caution against drawing strong conclusions about the effects of vulnerability appeals on minority group members.

These results also have some important limitations. First, we cannot rule out the possibility that our results are impacted to some extent by inattentive responding, as our design did not include a manipulation check and the treatment itself was relatively brief. Second, our results may have been impacted to some extent by the timing of our study. At the time of fielding, the pandemic was in its second year, daily new cases and COVID-related deaths were at a low ebb, and the Welsh Senedd had recently voted to relax a wide array of behavioral restrictions. ¹¹ Plausibly, respondents may have found our treatments more affectively compelling if the virus were perceived as posing a more urgent existential threat. On the other hand, by 2021 there was a much greater volume of factual information about the vulnerability of social minorities to COVID-19, and this information was widely disseminated by political elites. We may not have observed substantively different responses to our treatments if the study were fielded earlier in the pandemic, when the threat posed by COVID-19 to social minorities was less clear.

It is also important to consider how our results may have varied in response to a stronger or more detailed treatment. For example, our treatments did not include statistical information describing the specific rates of severe illness and death experienced by various minority groups. We omitted these data because the groups featured in our treatments had very different rates of severe illness and death from COVID-19 by May 2021, and including accurate information of this kind would have made it difficult to draw accurate inferences about the role of group cues in particular. Nevertheless, in the absence of this kind of evidence, respondents may have been more likely to dismiss the treatments as hyperbolic or politicized, or feel they lacked the information necessary to respond in a manner commensurate to the scale of the problem. Similarly, our treatments did not include behavioral recommendations, and we did not frame our treatments as coming from a politician or government office. Both of these characteristics may plausibly have increased the strength or persuasiveness of our treatments. Finally, it is also the case that these vulnerability appeals were rarely used in isolation, and instead were frequently communicated alongside other information. Our study does not allow us to measure the potential additive effect of these appeals when accompanied by additional information. Future research should seek to address these possibilities by (1) varying the type and volume of supporting evidence provided in vulnerability appeals, (2) including clear behavioral recommendations, and (3) varying the institutional or political source of vulnerability appeals.

Understanding how messaging shapes compliance with public health restrictions is important, both for our retrospective understanding of political behavior during moments of social upheaval and threat, and to help inform effective health policy interventions during future public health emergencies. These challenges are particularly urgent in demographically diverse societies, where messages targeted at specific groups risk alienating out-groups and producing unintended behavioral consequences. Our results indicate that while vulnerability appeals were unsuccessful in promoting affective concern or prosocial action in the full sample, there is some suggestive evidence that the effect such appeals may vary to some extent among respondents with different health characteristics. Future work should seek to clarify how vulnerability appeals might be more effectively crafted and targeted to promote positive behavioral change in the mass public, and among those most at risk from public health emergencies.