Rumors and Health Care Reform: Experiments in Political Misinformation

Abstract

This article explores belief in political rumors surrounding the health care reforms enacted by Congress in 2010. Refuting rumors with statements from unlikely sources can, under certain circumstances, increase the willingness of citizens to reject rumors regardless of their own political predilections. Such source credibility effects, while well known in the political persuasion literature, have not been applied to the study of rumor. Though source credibility appears to be an effective tool for debunking political rumors, risks remain. Drawing upon research from psychology on ‘fluency’ – the ease of information recall – this article argues that rumors acquire power through familiarity. Attempting to quash rumors through direct refutation may facilitate their diffusion by increasing fluency. The empirical results find that merely repeating a rumor increases its power.

In the summer of 2009, stories swirled alleging that President Obama’s health care reforms would include procedures to withhold care from certain citizens. The elderly, these rumors suggested, would have to meet with government officials to discuss ‘end of life’ options like euthanasia. On 7 August 2009 former Alaska governor and US vice presidential candidate Sarah Palin wrote on her Facebook page, ‘The America I know and love is not one in which my parents or my baby with Down Syndrome will have to stand in front of Obama’s “death panel” so his bureaucrats can decide, based on a subjective judgment of their “level of productivity in society”, whether they are worthy of health care. Such a system is downright evil.’ ¹ Within a week, 86 per cent of the American public was aware of Palin’s accusation. ² Though these objectively false rumors were widely discredited, the controversy surrounding death panels would not die – even long after the 2010 Affordable Care Act (ACA) was passed by the US Congress and signed into law by Obama. ³

Rumors and innuendo have long influenced the conduct of politics in destructive ways. Though not always false, rumors are regularly used in contemporary politics as a tool to spread falsehoods and misinformation. This is true not only in the United States, but in other countries throughout the world. ⁴ Rumors in political discourse are common and easy to identify, yet it remains difficult to find ways to undo the false information they spread. As the motivating example above illustrates, political rumors are often durable and highly resistant to correction, sometimes with dangerous consequences.

In this article, I engage the broad challenge of finding ways to correct rumors. I study how ordinary citizens responded to actual rumors surrounding the health care reforms enacted by the US Congress in 2010. There are, it appears, some effective strategies for countering rumors. Refuting a rumor with statements from an unlikely source – a person who makes proclamations that run contrary to their personal and political interests – can increase citizens’ willingness to reject rumors, regardless of their own political predilections. While such source credibility results are well known in the political persuasion literature, these insights have not yet been brought to bear on the study of rumor. In the present context, pairing death panel rumors with statements from Republicans flatly debunking the rumor can lead citizens – Republicans and Democrats alike – to reject the veracity of the rumor.

Though source credibility appears to be an effective tool for debunking political rumors, risks remain. Drawing upon research from psychology on ‘fluency’ – a state of mind that characterizes the ease of information processing – I argue that rumors acquire their power through familiarity. Attempting to quash rumors through direct refutation may instead facilitate their diffusion by increasing their fluency. Empirically, I find that merely repeating a rumor increases its strength. In fact, simply asking subjects to repeat the rumor to themselves – without any indication that it is true – increases its power. Evidence from a panel of subjects demonstrates that these effects persist even weeks after subjects read the initial story. This pattern holds even when rumors are repeated in the context of debunking that misinformation with a strong correction.

This article proceeds as follows: in the next section, I describe the nature of political rumors with a focus on the health care rumor. I then discuss how we can limit their power by bolstering the credibility of the source debunking the rumor. In today’s politically polarized world, non-partisan sources sometimes lack credibility; far more credible are partisan political actors who make statements that run counter to their apparent interests. I next discuss how the repetition of rumors can augment their ‘fluency’, thereby increasing the likelihood that individuals will accept misinformation as truth. To test these theoretical claims, I present the results of two experiments in which I vary the presentation of information concerning the 2010 ACA. Together, these experiments demonstrate both the power of partisan corrections to counter false information and the dominance of fluency in facilitating the acceptance of rumors.

Rumors in the Political Realm

While the conceptual murkiness surrounding the term ‘rumor’ has led to longstanding definitional debates among social scientists, a consensus regarding its meaning has emerged. ⁵ Sunstein defines the term to refer to ‘claims of fact – about people, groups, events and institutions – that have not been shown to be true, but that move from one person to another and hence have credibility not because direct evidence is known to support them, but because other people seem to believe them’. ⁶

From these points of view, rumors are a particular form of misinformation – an acceptance of information that is factually unsubstantiated – characterized by two features. First, rumors are statements that lack specific standards of evidence. ⁷ Political rumors, then, are not ‘warranted beliefs’. ⁸ Secondly, rumors are more than fringe beliefs. They acquire their power through widespread social transmission.

While rumors have been a topic of social science research for over 100 years, by some readings, they have become a more insidious political force in recent times. ⁹ In the World War II era, Allport and Postman noted that the main medium of rumor transmission was word of mouth. ¹⁰ The internet has expanded the scope of rumor dissemination. Today, anyone can publish on the web, instantly acquiring a degree of credibility and more easily reaching a larger audience. As a result, rumors may materialize suddenly and never disappear. ¹¹ This is true not just in the United States, but in other countries as well. ¹² Rumors are therefore a powerful tool in the spread of unsubstantiated information, and even falsehoods (as in the case of death panels).

The persistence of these rumors is troubling for the prospects of democracy. As Kuklinski et al. note, while much of the handwringing in the field of political behavior concerns the implications of citizens’ ignorance of basic political facts, a more serious problem for the possibility of establishing effective mass participation in a democratic system is the prevalence of misinformation. ¹³ A democracy in which falsehoods are rampant can lead to dysfunction. Rumors are an insidious form of misinformation – one that is particularly damaging for the functioning of democracy – but they are misinformation nonetheless. The key, from this perspective, is making sure that the public definitively rejects false information. Seen in this light, political rumors provide a new and interesting venue for the study of political misinformation. ¹⁴ By focusing on patently false rumors that can be debunked with uncontested facts, we can learn about the more general processes of political misperception.

Health Reform and ‘Death Panels’

While recent history bears out the power and reach of political rumors from across the ideological spectrum, in this article I consider one specific political rumor. As noted above, in the summer of 2009 rumors circulated that Obama’s proposed health care reform plan would allow government officials to decide whether individual citizens should receive health care based on a calculation of their level of productivity in society. One element of these rumors was the suggestion that elderly people would have to consult ‘death panels’ to discuss end-of-life options like euthanasia. These rumors started with statements made by former New York Lieutenant Governor Betsy McCaughey, but quickly spread to the conservative media. ¹⁵ A number of prominent Republican politicians added to the chorus, including Sarah Palin and Senator Charles Grassley, the ranking Republican member of the US Senate Finance Committee.

These rumors are patently false and have been publicly discredited in many fora. Yet they have taken seed among the American public. A poll conducted by the Pew Center in August 2009 found that 30 per cent of the public thought the ‘death panel’ rumor was true, with another 20 per cent unsure of the veracity of the statement. ¹⁶ Following various attempts to debunk the rumor, and even after the passage of the ACA in March 2010, the death panel rumors continued to persist. A July 2010 YouGov poll found that 33 per cent of the public thought the rumor was true, with another 22 per cent unsure. In a separate question, the same poll also found that 26 per cent of the public thought that changes to the health care system would require elderly patients to meet with government officials to discuss end-of-life options, including euthanasia. Another 28 per cent were unsure. These responses are consistent with other polling on health care rumors. ¹⁷

The power of this rumor extends beyond shaping the beliefs of the mass public; it also has serious policy consequences. In early 2011, the Obama administration announced that it would revise Medicare fee policies to remove provisions that provided funding for end-of-life counseling. The vast reach of the death panel rumors hung heavily over this decision. Thus the spread of death panel rumors may have, at least indirectly, contributed to the end of an important Medicare service.

How do People Respond to Rumors?

Correcting rumors is no easy task, but it may be even more difficult to correct rumors in the minds of certain types of citizens. Recent research has shown that some individuals are more predisposed than others to accept conspiracy theories and related rumors even when the theories are self-contradictory. ¹⁸

But even given these inter-individual differences in the tendency to accept fanciful beliefs, the specific presentation of information – and misinformation – should help shape many citizens’ ideas about political rumors. Obama attempted one such strategy by directly contradicting the health care rumors at town hall meetings in New Hampshire in August 2009. But, as the polls discussed above demonstrate, these strategies did not shift the tides of public opinion. This is not surprising. Consistent with Obama’s failures, political science research on misinformation paints a grim picture. Kuklinski et al. had some success in correcting false beliefs about welfare spending in the United States, but they also presented evidence that such learning is short-lived. ¹⁹ More recently, Nyhan and Reifler conducted experiments in which they corrected false statements about, among other topics, the Iraq war and Obama’s religion. ²⁰ They find that confronting citizens with the truth can sometimes backfire and reinforce existing misperceptions. For instance, conservatives who received information that Iraq did not possess weapons of mass destruction were more likely to believe Iraq had those weapons than were respondents who did not receive the correct information. ²¹

These results are both disturbing and counterintuitive; they suggest that, with regard to political issues, attempts to correct the misperceptions of ordinary citizens may exacerbate the stickiness of mistaken beliefs. Moreover, these findings contradict much of the advice in psychology on how best to address rumors. In a comprehensive review of rumor-quelling literatures in both psychology and business, spanning twenty-two different studies, DiFonzo and Bordia find that the most frequent recommendation on how to reduce belief in a rumor is the use of rebuttal, the same strategy the Obama administration attempted in 2009. ²²

Political Rumors and the Power of Partisanship

To correct misperceptions and misinformation about political controversies, a new approach seems necessary. The introduction of partisan concerns changes how people deal with new information. Surveys show that Democrats and Republicans (and liberals and conservatives) approach the same rumor in very different ways. More generally, research has found that people are more likely to accept rumors that are consistent with their pre-existing attitudes. ²³ A compelling explanation of this pattern is citizens’ tendency to engage in motivated reasoning: ²⁴ ordinary citizens are goal-directed information processers, and they perceive new information in light of their pre-existing views. ²⁵ Put another way, some citizens may be more likely to believe particular rumors because they are motivated to cling to beliefs that are compatible with their partisan or political outlook.

Because partisans tend to evaluate new information with respect to their existing views, ²⁶ encountering a rumor may have different effects for citizens of different political stripes. Still, while partisanship may color the processing of new information, it does not always control it, and even the strongest partisans can change their beliefs in response to new information. In this case, turning the power of partisanship on its head could be the key to developing effective corrective measures. Politicians who attempt to debunk rumors often appeal to non-partisan authorities as neutral referees of the truth. But in a time when people’s partisanship colors how they perceive new information, these ‘neutral’ non-partisan figures may speak with less credibility than is often presumed. More convincing are partisan politicians who speak against their own apparent political interests. Such unlikely statements may make rumor rebuttals more credible.

This notion of source credibility effects is novel in the study of rumors, but is a familiar result in persuasion research. ²⁷ Moreover, it is consistent with the formal model proposed by Calvert. ²⁸ Calvert concludes that ‘biased’ advisors – those with strong priors toward a particular policy – are especially informative sources of information for decision makers. If such an advisor rejects a policy that the decision maker presumed he would support, this unexpected advice could be ‘enough to reverse prior preferences’. ²⁹ In the present context, a Republican who debunks a rumor about a Democratic politician or policy could change the beliefs of both Democrats and Republicans (as well as Independents).

The Fluency of Information

If turning partisanship on its head seems a plausible corrective measure for rumors, what concerns remain? One is that we have few explanations for the seemingly counter-intuitive results of Nyhan and Reifler – why do some correctives actually increase the power of rumors? ³⁰ While there is much we do not know about the dynamics of rumors, we can draw on basic mechanisms studied by social psychologists outside of the context of rumors to explain these results. After all, psychologists have long known that retractions often fail to eliminate the influence of misinformation through a ‘continued influence effect’. ³¹ The key is to consider what Schwarz et al. call the ‘processing fluency’ of information. ³²

Fluency is ‘the subjective experience of ease or difficulty associated with completing a mental task’. ³³ This line of research grows out of Tversky and Kahneman’s availability heuristic. ³⁴ Cognitive tasks, as Alter and Oppenheimer note, can be described as ranging from effortless to highly effortful. ³⁵ These tasks, in turn, produce an internal state of awareness whereby people are aware of how hard they need to think to complete a task, and that awareness influences their everyday judgements. ³⁶

What is important here is that a number of psychologists have found that the difficulty with which information is processed affects individuals’ assessment of its accuracy. Specifically, they find that people use their feelings regarding how easy it is to recall or process new information as a signal of the veracity of that information. ³⁷ This is a powerful effect that goes beyond merely identifying or comprehending frequently presented information, a phenomenon that was recognized beginning in the 1980s. ³⁸ The psychological state of high fluency instead shapes the perceived authenticity of information. For instance, information that has been presented frequently will be more familiar to citizens and, as a result, is more likely to be accepted as the truth, regardless of the actual content. ³⁹ Repetition is but one path to fluency, but these findings may explain the Nyhan and Reifler results. Any attempt to explicitly discredit falsehoods may make things worse because directly countering a rumor involves repeating it. This mere mention of the rumor can increase its fluency, thereby heightening its perceived accuracy. ⁴⁰

Schwarz et al. demonstrate the importance of these processes using an informational flyer created by the Center for Disease Control (CDC) to educate patients about the flu vaccine. ⁴¹ This flyer confronts ‘myths’ about the vaccination – erroneous beliefs about the vaccine – with the proper ‘facts’. Schwarz et al. conducted an experiment in which some subjects were shown the CDC flyer. Half of these subjects were immediately given a test that repeated the ‘facts and myths’ information. These participants were asked to mark which statements were true. The other half of the subjects was given the test after a thirty-minute delay. ⁴²

Immediately after reading the flyer, participants were able to recall the information from the flyer almost perfectly – they misclassified ‘myths’ as ‘facts’ at very low rates (and vice versa). However, after the thirty-minute delay – once the memory of the substantive details of the information presented in the poster faded, but the ‘gist’ of those statements remained – the subjects’ judgements began to show a systematic error pattern. Consistent with a psychological state of increased fluency, these respondents were significantly more likely to misidentify the ‘myths’ as ‘truth’ than vice versa, and were less likely to say they would get a shot than the participants who were asked their opinions immediately. In essence, the attempt to debunk myths about the flu shot instead had the effect of further spreading mistruths by subconsciously increasing the respondents’ recognition of those myths. ⁴³ We can draw upon the same dynamics uncovered by Schwarz to understand the power of rumors.

Summary and Synthesis

Together these seemingly disparate bodies of work explain the enduring power of political rumors and provide some clues about potential correctives. Political rumors are powerful because partisans are motivated to believe falsehoods about politicians and policies of the other party. Moreover, the strategy most often employed to counter rumors – directly confronting the rumor by repeating it, then debunking it – may serve to augment the processing fluency of the rumor for individuals, thereby decreasing the likelihood that citizens will reject that rumor out of hand. To discredit these rumors and correct misinformation, it is necessary to adopt a new strategy – in particular by varying the partisanship of the authoritative source. It may be that Republicans who debunk Democratic rumors (and Democrats who debunk Republican rumors) will have more credibility than non-partisan actors. These politicized voices may have the power to overwhelm the fluency effect and correct misinformed beliefs. In my experiments, I try to break the vicious cycle of rumors with different presentational strategies.

Experiments

In 2010, I conducted two studies to test the effects of different corrections on the beliefs regarding rumors concerning the 2010 ACA. These studies varied the presentation of the rumor-related information across three dimensions: (1) the pairing of rumor and correction, (2) the partisanship of the source of the correction of the rumor and (3) the degree to which the respondent was induced to rehearse the rumor. These experiments also explored the effect of time on the power of the treatments, recording responses not only immediately following the treatment, but also many days later. In all the experiments I was concerned with patterns of rumor rejection – the rate at which subjects would definitively reject false information. The psychological literature often takes the opposite approach, focusing on patterns of rumor acceptance. ⁴⁴ I focus on rumor rejection because of the ambiguous nature of the ‘not sure’ response. From a political science perspective, any position short of outright rejecting a rumor – even an expression of uncertainty about its veracity through a ‘don’t know’ or ‘not sure’ response – enhances the credibility of rumors and is normatively undesirable. The importance of the ‘not sure’ responses can be seen in media reports of polls asking rumor questions. Such responses are often reported to be as damning as acceptance of the rumors. ⁴⁵

The experiments described in this article are especially difficult tests of the power of rumors because they exploit real-world (rather than constructed) rumors. Both of these experiments were conducted in the immediate wake of the passage of the controversial 2010 ACA and used quotes and facts drawn from the debate surrounding the act. In doing so, I traded off the ability to fully manipulate the information presented to respondents in order to maintain a realistic set of treatments. In addition, given that these experiments were conducted during an ongoing controversy, the effect sizes of my treatments will be smaller than if the information presented in the stories was completely novel to the subjects. Thus, the experimental effects that I find from exposure to my treatments are, by design, more muted than they would have been if I had been able to control the full scope of the information presented.

Study 1

In May 2010, two months after the passage of the ACA, I ran a between-subjects design experiment on an internet panel to test the efficacy of various corrective strategies. ⁴⁶ The primary purpose of this experiment was to examine the impact of the partisan identity of the provider of the rumor correction. I then used a delayed follow-up questionnaire to assess some of Schwarz’s expectations regarding fluency. ⁴⁷

I randomized the presentation of different news stories that I constructed about health care reform to subjects in four different experimental conditions (along with a control condition, in which the subjects received no news story, but were merely asked if they believed the rumor). The stories all dealt with the 2010 ACA, but presented different details about the debate surrounding that plan (the full text of the treatments is presented in Appendix A).

In the first ‘rumor’ condition, subjects were presented with the rumor in the form of quotes by opponents of the act, who warned of the possibility of death panels. In the second ‘rumor and non-partisan correction’ condition, respondents were presented both the rumor information from the first condition and a non-partisan correction of that rumor, in the form of a description of the end-of-life provisions of the ACA and supporting quotes from the American Medical Association (AMA) and the American Association of Retired Persons (AARP). The third and fourth conditions introduced explicitly partisan information in addition to the non-partisan correction. In the third ‘rumor and Republican correction’ condition, the text from the ‘rumor and non-partisan correction’ condition text was reproduced, and a quote debunking the rumor from Senator John Isakson (a Republican who helped draft the end-of-life provisions) was added to the end of the story. In the fourth ‘rumor and Democratic correction’ condition, the ‘rumor and non-partisan correction’ was also reproduced and a separate quote containing the correction was added to the end of the story. This quote was instead from Representative Earl Blumenauer (a Democrat who helped draft the provision). ⁴⁸ The specific text of the quotes from Isakson and Blumenauer differed (because they were drawn from actual quotes) but the spirit of the corrections was the same. ⁴⁹ More generally, it should be noted that while the treatments differ in a number of ways, these differences are intentionally drawn to mirror the political rhetoric surrounding the discussion of rumors concerning health care reform in the media.

Respondents were randomly assigned to one of five conditions (the four experimental treatments and the control). They were then asked a series of questions concerning the veracity of the death panel rumors and whether they support the health care plan. In the tables that follow, I present the results of the analysis of the euthanasia rumor item, which most directly taps the controversy surrounding the ACA provisions. It asks, ‘Do you think the changes to the health care system have that have been enacted by Congress and the Obama administration require elderly patients to meet with government officials to discuss “end of life” options including euthanasia’? ⁵⁰ The results are very similar for the other rumor question, which asks about death panels. To minimize the number of tables in the body of the article, I do not include the analysis of this question below, but present the full results in Appendix B. ⁵¹

In addition, I asked two questions that were designed to measure, in a general sense, how closely the respondents followed the instructions on the survey. Specifically, I designed items that serve as what Oppenheimer, Meyvis and Davidenko term an instructional manipulation check (IMC). ⁵² IMCs (or ‘screeners’) indirectly capture whether respondents are satisficing ⁵³ – simply hurrying through the questionnaire as fast as possible (the full text of these questions is presented in Appendix C). Approximately two-thirds of respondents passed each of the two screener questions, and 55 per cent of the sample passed both of the questions. In the analyses that follow, I present the results for the full sample, but focus on the ‘attentive sample’ – those respondents who passed both screener questions. I focus on this subsample because the news story treatments contain subtle differences in presentation that are likely to have their strongest effect on respondents who pay closest attention to the survey. ⁵⁴ Berinsky, Margolis and Sances show that the passage of screener questions leads to an increased power of textual treatments because people who pass screeners read the text more closely. ⁵⁵ That said, because Berinsky, Margolis and Sances also show that people who pass screener questions are different from those who fail them, it is certainly possible that there is a heterogenous treatment effect between the attentive and inattentive subgroups. That is, even if the inattentive subgroup read the material more closely, the effect of the treatment might not be the same for the two groups. The estimates presented here for the attentive sample should therefore be best thought of as the average treatment effect for this sample.

A second reason I focus on this subsample is that approximately one week after the initial survey I conducted a second wave of the experiment. I recontacted only those respondents who passed both of the screener questions. Of those contacted, about 60 per cent agreed to participate in the second wave. Ultimately, I reinterviewed 699 subjects – 41 per cent of the initial full sample. This reinterview sample was essentially identical to the initial attentive sample on measures of gender, education, party identification and political information. During the second wave, I again asked the rumor questions and support for the health care plan.

In Table 1, I present the answers to the euthanasia rumors question, broken down by experimental condition. I present two sets of significance tests. The first is a χ² test of overall significance, which tests if there are any significant differences among any of the conditions. The second set is a series of paired comparisons testing for significant differences between conditions. Because there are five conditions, there are a total of ten comparisons. I present full significance results for all the pairwise comparisons in a note at the bottom of the table.

Table 1 Effect of Treatments on Euthanasia Rumor Belief, May 2010

N=1,596; χ²(8)=19.24; Pr=0.01.

Note: the top three rows present the percentage of respondents that rejects, accepts or is unsure about each rumor. This is presented for each condition (represented by the column). The significance tests are calculated across all response categories.

^aControl is not statistically significantly different from Rumor Only (p=0.306), or from Rumor+Democratic Correction (p=0.142).

^bControl is marginally statistically significantly different from Rumor+Non-partisan Correction (p=0.093).

^cControl is statistically significantly different from Rumor+Republican Correction (p=0.029).

^dRumor Only is statistically significantly different from Rumor+Non-partisan Correction (p=0.010), from Rumor+Republican Correction (p=0.002) and from Rumor+Democratic Correction (p=0.039).

^eRumor+Non-partisan Correction is not statistically significantly different from Rumor+Republican Correction (p=0.895), or from Rumor+Democratic Correction (p=0.810).

^fRumor+Republican Correction is not statistically significantly different from Rumor+Democratic Correction (p=0.599).

N=876; χ²(8)=23.95; Pr=0.002.

Note: the top three rows present the percentage of respondents that rejects, accepts or is unsure about each rumor. This is presented for each condition (represented by the column). The significance tests are calculated across all response categories.

^aControl is not statistically significantly different from Rumor Only (p=0.145), or from Rumor+Non-partisan Correction (p=0.581) or from Rumor+Democratic Correction (p=0.216).

^bControl is statistically significantly different from Rumor+Republican Correction (p=0.004).

^cRumor Only is not statistically significantly different from Rumor+Non-partisan Correction (p=0.147).

^dRumor Only is statistically significantly different from Rumor+Republican Correction (p=0.000), and from Rumor+Democratic Correction (p=0.029).

^eRumor+Non-partisan Correction is marginally statistically distinguishable from Rumor+Republican Correction (p=0.068).

^fRumor+Non-partisan Correction is not statistically significantly different from Rumor+Democratic Correction (p=0.836).

^gRumor+Republican Correction is not statistically significantly different from Rumor+Democratic Correction (p=0.147).

Several conclusions are apparent. First, the differences among the conditions are highly significant in a statistical sense, indicating that, taken together, the different presentations of the information altered rumor rejection levels. Moving to the pairwise comparisons between the different conditions, I find that presenting the rumor by itself decreases rumor rejection rates by 5 per cent relative to the control condition, though this difference is not statistically significant. The correction conditions all seem to be effective, and increase the rates of rumor rejection by ten points over the rumor-only condition, largely by reducing the levels of ‘not sure’ responses.

The pairwise differences between each of the three correction conditions and the ‘rumor only’ condition are all statistically significant at the 0.05 level. However, the differences among the three correction conditions are not statistically significant. ⁵⁶ In addition, the ‘rumor and Republican correction’ condition – which should be the strongest correction – increases rumor rejection rates by 8 per cent relative to the control condition (58 vs. 50 per cent), a difference that is statistically significant at the 0.05 level. Thus, the corrections seem to work by increasing rumor rejection rates by reducing the level of uncertainty surrounding the rumor.

The magnitude of the differences among the conditions is magnified in the attentive sample. As the bottom panel of Table 1 demonstrates, the baseline rumor rejection rates are somewhat higher in the attentive sample, but the differences between conditions apparent in the full sample emerge more sharply in the attentive sample. Among those respondents who paid close attention to question wordings, the Republican correction – information from an unexpected source – is the most effective treatment in increasing rates of rumor rejection. The size of the differences between the ‘rumor and Republican correction’ condition and the other conditions is larger in both a substantive and a statistical sense than in the full sample. The difference in results between the full and attentive samples makes sense, because the key difference between the Democratic and Republican correction conditions is the provider of the quote at the end of the article. People who merely skim through the treatment and do not read the stimulus are likely to miss this critical difference. This result holds when we break up the sample into partisan groups. Though the relatively small size of the subgroup samples precludes the detection of statistically significant differences, there are strong patterns in the substantive size of the effects (and sometimes these differences reach conventional levels of significance). Most importantly, the Republican correction is the most powerful treatment for both Republican and Democratic identifiers (see Appendix B for full results). Thus, a simple elite cueing story alone cannot explain these results; after all, Democratic identifiers are more influenced by the Republican correction than by the Democratic correction. Rather, it is the informational content of the identity of the politician making the correction that matters.

These effects extend from rejection of the rumor to support for health care in the attentive sample, as shown in Table 2. ⁵⁷ While the euthanasia rumor question presented in Table 1 measures belief in the rumor, the policy item in Table 2 measures whether the respondent supports the policies enacted by Obama’s health care reforms. Though the effects are modest in size, both statistically and substantively, presenting the rumor by itself decreases support for the plan relative to the control condition (51 per cent compared to 42 per cent). This nine-point difference barely misses significance at the 0.10 level in the attentive sample. Presenting the rumor in combination with a non-partisan correction or the Republican correction causes support to rebound, albeit modestly. However, neither of these correction conditions is statistically distinct from the ‘rumor only’ condition. But interestingly, introducing the Democratic correction creates a backlash, reducing support for the plan to its lowest level across any condition. ⁵⁸

Table 2 Effect of Treatments on Health Care Policy Opinion, May 2010 (Attentive Sample)

N=876; χ²(4)=9.00; Pr=0.06.

Note: the top two rows present the percentages of respondents that supports or opposes each condition (represented by the columns).

^aControl is not statistically significantly different from Rumor Only (p=0.102), from Rumor+Non-partisan Correction (p=0.337), from Rumor+Republican Correction (p=0.600), or from Rumor+Democratic Correction (p=0.216).

^bRumor Only is not statistically significantly different from Rumor+Non-partisan Correction (p=0.483), or from Rumor+Democratic Correction (p=0.312).

^cRumor Only is statistically significantly different from Rumor+Republican Correction (p=0.000).

^dRumor+Non-partisan Correction is not statistically significantly different from Rumor+Republican Correction (p=0.662).

^eRumor+Non-partisan Correction is marginally statistically distinguishable from Rumor+Democratic Correction (p=0.077).

^fRumor+Republican Correction is statistically distinguishable from Rumor+Democratic Correction (p=0.027).

As Table 3 shows, there is some evidence of a subsiding effect over time. The effects of the different experimental conditions are clear, but these effects were all produced on items asked immediately post-treatment. Based on Schwarz’s work, we would expect that fluency-based backfire effects would occur after a delay. ⁵⁹ Specifically, respondents who were presented both the rumor and the correction should be less likely to reject the rumor outright than people who were given no information.

Table 3 Effect of Treatments on Euthanasia Rumor Belief Over Time for Respondents Interviewed in Both Waves, May 2010

N=696; χ²(8)=23.2; Pr=0.03.

Note: full pairwise tests of statistical significance are presented in Appendix C (C8).

N=696; χ²(8)=12.2; Pr=0.14.

Note: full pairwise tests of statistical significance are presented in Appendix C (C9).

Here, I compare the responses on the euthanasia rumor question across the two waves for the subset of respondents that was interviewed in both waves. As expected, the respondents in the ‘rumor only’ condition held steady in their beliefs. However, the effectiveness of the corrections faded across the board during the week between the waves of the survey, largely because the rates of ‘not sure’ responses increased for respondents in those conditions. As the χ² test of overall significance demonstrates, the differences among the experimental conditions were no longer significant. Not surprisingly, given this result, the statistically significant differences found in the pairwise comparison of conditions from the first wave disappear. ⁶⁰ The Republican correction remained the most effective treatment, but the rates of rumor rejection decreased by about 10 per cent relative to the first wave. Furthermore, the rates of rumor rejection for both the ‘rumor and non-partisan correction’ and ‘rumor and Democratic correction’ conditions were now lower than the control condition (though this difference is not statistically significant). Thus, over time, providing people with correct information in conjunction with the rumor may be no better than no information at all. These effects extended beyond rumor rejection to support for health care. Respondents who were exposed to the rumor, either alone or in combination with the non-partisan correction, remained less supportive than people who received no information (see Appendix B for these results).

Study 2

Study 1 demonstrated that corrections from an unlikely source – Republicans debunking health care rumors – are the most effective way to counter rumors. But the danger in debunking rumors is that by reminding citizens of the rumor, the effectiveness of correcting that rumor could fade over time. In order to build on the first study and more directly test the fluency hypothesis, I fielded a second experiment as a module in the October/November 2010 Cooperative Congressional Election Study (CCES), a panel study fielded by YouGov. ⁶¹

The subject of these experiments was again rumors concerning Obama’s health plan. As in Study 1, I randomized the presentation of treatments that consisted of (constructed) news stories about health care reform. In this second study, I used a new design. The stories in these treatments were modeled on the stories used in the May 2010 experiment and are presented in Appendix E. I had three different experimental conditions. The first ‘rumor’ condition was identical to the rumor condition in the May study. The second ‘correction only’ condition was a new treatment that did not mention the rumors concerning the death panel, but only described the actual provisions in the 2010 ACA. ⁶² I chose to hold the partisan content of the rumor correction at its most effective level, using the Republican correction in the form of the Isakson quote from the May study. The third ‘rumor and correction’ was identical to the ‘rumor and Republican correction’ condition from the May study and presented both the rumor information from the first condition and the correction from the second condition. Unlike Study 1, I did not employ the screener IMC items. ⁶³

There was no control group in this experiment. Based on the results of Study 1, we have a good sense of the power of the treatments relative to the baseline. ⁶⁴ I therefore compare the effects of the different conditions to each other. This is an appropriate analytic strategy because the purpose of the experiment is to explore the effects of differences in the presentation of information, not the differences between the treatments and ‘no information’.

In addition to the corrective strategy treatment, I designed a two-condition ‘rehearsal treatment’ to directly test the expectations regarding fluency from the Schwarz work. ⁶⁵ Respondents who received the rumor information – either alone or in combination with the correction – were asked one of two types of recall questions after the story. The stated purpose of the task was to test what they could recall from the story, but the true purpose was to see whether rehearsing the rumors would increase their power. Half of the respondents were assigned to an ‘irrelevant recall’ condition, in which they were asked a single recall question – a multiple choice question asking what office was held by Betsy McCaughey (who was quoted in the story) – while the other half was placed in a ‘long recall’ condition and received two additional questions that asked them to identify the speaker of a particular quote that repeated the content of the rumor. For instance, respondents were asked who said, ‘You have every right to fear […] [You] should not have a government-run plan to decide when to pull the plug on Grandma’ (exact text of these questions is presented in Appendix F).

I was not interested in the answers they gave to these questions per se; the simple task of answering the recall questions ensured that they would again read the incorrect information. This ‘rehearsal’ of the rumor immediately following the presentation of the study should increase the fluency with which people process the information contained in the rumor. ⁶⁶ Based on the psychological work on fluency, I expected this increased level of fluency to lower rumor rejection levels, regardless of whether the rumor was paired with the correction.

CCES respondents were interviewed first in October, and then again in November. I asked the rumor belief questions in both waves. I also asked about support for health care reform in the first wave. ⁶⁷

In Table 4, I present the effects of the rumor information treatment on rumor rejection levels (setting aside the recall treatments for the moment). The correction conditions – whether in combination with the rumor or alone – led respondents to reject the euthanasia rumor at higher rates than in the rumor-only condition, though only the rumor and correction condition is statistically distinct from the rumor-only condition at the 0.05 level. Thus, at least in the moment, the pairing of the rumor with the correction was a more effective strategy than simply presenting the correct information on its own (though the differences between these two conditions are not statistically significant).

Table 4 Effect of Treatments on Euthanasia Rumor Belief, October 2010

N=1,000; χ²(4)=12.23; Pr=0.02.

Note: full pairwise tests of statistical significance are presented in Appendix G (G7).

But Schwarz’s work suggests that the effectiveness of corrections fades over time. ⁶⁸ Table 5 presents the distribution of responses for those individuals who completed both waves of the study (837 of the initial 1,000 subjects). Subjects who completed only the first wave of the survey are excluded from the analysis. This table demonstrates that on a substantive level, by the second wave, the gap between the ‘rumor only’ and ‘rumor and correction’ conditions had diminished (see Appendix G for a more formal difference-in-difference test). As in Study 1, the passage of time reduced the effectiveness of the correction relative to the rumor on the euthanasia question. Neither of the correction conditions is statistically distinct from the rumor-only condition in a pairwise comparison test. It should be noted that this effect is not merely the result of the fading effectiveness of information over time; one month later, the distribution of responses to the euthanasia question for respondents in the ‘rumor only’ condition is unchanged. This differential pattern of information decay across conditions contrasts with the typical pattern of treatment decay found in similar studies. ⁶⁹ Again, the effect of the treatments extended from rejection of rumors to general support for health care (see Appendix G for details).

Table 5 Effect of Treatments on Euthanasia Rumor Belief Over Time, October–November 2010

N=837; χ²(4)=14.22; Pr=0.01.

Note: full pairwise tests of statistical significance are presented in Appendix G (G8).

N=834; χ²(4)=5.03 Pr=0.28.

Note: full pairwise tests of statistical significance are presented in Appendix G (G9).

To this point, Study 2 largely replicates the findings from Study 1. I next sought to see if the rehearsal treatment had the intended effect. Findings about the effect of rumor rehearsal on rumor acceptance are given in Table 6. The table presents four columns of data. The first two columns pertain to the results among respondents assigned to the ‘rumor only’ story treatment. The first column presents the distribution of responses to the euthanasia question among subjects who were assigned to the ‘irrelevant recall’ version of the rehearsal treatment. These respondents were only presented the McCaughey office identification question. The second column presents the distribution of the rumor question among those who were assigned to the ‘long recall’ version of the rehearsal treatment. In addition to the McCaughey question, these respondents were given the two death panel quote identification questions.

Table 6 Effect of Rumor Rehearsal on Euthanasia Rumor Belief, October 2010

Rumor Only: N=350; χ²(2)=5.49; Rumor+Correction: Pr=0.06; N=342; χ²(2)=0.67; Pr=0.72.

My expectation was that respondents in this condition would be less likely to reject the rumor because they have rehearsed its content, thus increasing its fluency. The next two columns of the table repeat this presentation for respondents in the ‘rumor and correction’ story condition. Since respondents in the ‘correction only’ condition all received a single recall question unrelated to the rumor, the data from those respondents are omitted from the table.

Table 6 demonstrates that, as expected, increasing fluency by simply rehearsing the rumor by answering the recall questions increased the power of the treatment across both questions. The increased rehearsal of the rumor – in the absence of any information about its veracity – was sufficient to decrease rumor rejection rates, even when the rumor was initially presented in combination with a powerful correction. Admittedly, this effect is not extremely large, but it was larger in the long-recall condition than the irrelevant recall condition for both respondents in the rumor-only and the rumor and correction condition (though the differences are statistically significant at the 0.10 level only in the ‘rumor only’ condition). ⁷⁰ Further, in line with the ‘fluency’ hypothesis, Table 6 shows increases in rumor belief as well as uncertainty. Respondents are not simply becoming more uncertain, but are actually more ready to believe the rumor after repeated exposure. ⁷¹

Because the rumor questions are asked in short succession to the treatment, it could be argued that these results are as much evidence of priming as they are of fluency. ⁷² Specifically, the recall question might serve to ‘prime’ the consideration of the veracity of the rumor.

Table 7 provides suggestive evidence that this rehearsal effect persists over time, and is not merely the influence of ephemeral priming. This table presents the Wave 1 and Wave 2 results for respondents interviewed in both waves of the CCES. The differences on the euthanasia question, which are substantively large (though, admittedly, not statistically significant), remain nearly identical a month later. ⁷³ This table also provides some evidence that, over time, the power of the correction diminishes. Though not statistically significant, the rumor rejection rates drop for those respondents in the rumor and correction condition, especially for those in the long-recall condition. The pattern of responses for those in the rumor only condition, however, remains stable. This is suggestive evidence that, in line with the expectations from Schwarz’s work, merely increasing the fluency of a rumor increases its effectiveness. ⁷⁴ This result is important because the long-recall condition more accurately represents the volume with which rumors are repeated and magnified in today’s media environment.

Table 7 Effect of Rumor Rehearsal on Euthanasia Rumor Belief Over Time, October–November 2010

Rumor Only: N=289; χ²(2)=4.82; Pr=0.09; Rumor+Correction: N=285; χ²(2)=0.52; Pr=0.77.

Rumor Only: N=289; χ²(2)=4.07; Rumor+Correction: Pr=0.13; N=285; χ²(2)=2.45; Pr=0.29.

Discussion and Conclusion

By understanding how and why people come to reject false beliefs, we can learn how to loosen their hold on destructive pieces of misinformation. In two different studies using two separate samples (one in which I measured general attentiveness to the survey, and one in which I did not), I come to a similar conclusion. Though partisanship colors how citizens process information about public policy, my studies show that under the right circumstances – with the right arguments made by the right people – corrections can increase rumor rejection rates among the mass public, regardless of their partisan predilections. In particular, corrections acquire credibility when politicians make statements that run counter to their personal and political interests.

These experiments present especially difficult tests of the power of corrections. In order to most directly address the question of how best to dislodge rumors, I used real-world rumors. However, as a result, my treatments competed against the rhetoric surrounding health reform in the political world. As I discuss below, media coverage of death panels increased rumor fluency by often repeating the rumor without an effective correction. That sustained media environment may, therefore, make it difficult for a single experimental correction to significantly increase rumor rejection rates. In addition, when constructing my treatments I limited myself to actual statements made by politicians in the heat of the health care debate. Altogether, these factors curb the power of any experimental effects; my results emerge despite – and not because of – the political environment at the time.

As a result, though the substantive size of the effects in this article is admittedly modest, there are important lessons to take from these results. To correct rumors, we must account for the power of partisanship – and try to neutralize it. As noted above, politicians who attempt to counter rumors often appeal to non-partisan authorities. For instance, to fight the death panel rumors, experts from the AMA and the AARP were called in to ‘speak the truth’. But, ironically, in a politically polarized time, those non-partisan ‘authoritative’ voices often lack authority because they are easily drowned out by more politicized ones. In the modern political environment, ‘independent’ sources that are credible to both Democrats and Republicans are hard to find. Consider, for example, the AMA and the AARP. A September 2009 NPR/KFF/Harvard telephone survey assessed the level of confidence that the public had in various groups to ‘recommend the right thing for the country when it comes to health care’. ⁷⁵ Democrats and Republicans indeed held similar views of the AMA. Among Democrats, 23 per cent of respondents said they had a great deal of confidence in the AMA and 45 per cent had a fair amount of confidence – not a ringing endorsement, but a reasonable level of trust. Republicans expressed similar levels of trust – 20 per cent had a great deal of confidence and 45 per cent had a fair amount of confidence. However, there were large partisan gaps in the assessment of the AARP. Among Democrats, 29 per cent had a great deal of confidence and 40 per cent had a fair amount of confidence. But among Republicans, only 16 per cent had a great deal of confidence and 27 per cent had a fair amount of confidence – overall, a 26 percentage-point gap across the two categories.

That said, under the right circumstances, partisanship can be harnessed as a force for truth. In particular, politicized voices can help debunk false statements circulating in society. When I paired the death panel story with a quote debunking the rumor from a Republican who helped draft the end-of-life provisions, respondents – Republicans and Democrats alike – were far more likely to reject the euthanasia rumor. In the real world, these types of corrections from unexpected partisan sources exist, but they are admittedly rare. I performed a content analysis of all evening news stories broadcast from January 2009 to December 2012 that mentioned ‘death panels’ on ABC, CBS, NBC, CNN and FOX. While the rumor was presented without a correction only 7 per cent of the time, corrections from unlikely sources were rare as well; only 10 per cent of the stories paired the rumor with a correction from a Republican source. ⁷⁶ Put simply, the media created an environment that fostered the continued spread of death panel rumors. However, this does not have to be the case. There are always partisans – like Senator Isakson on health care – who are interested in disseminating the truth. Perhaps amplifying those voices could be effective. After all, though I find that the power of corrections fade and rumors regain their strength over time, both studies demonstrated that the Republican correction remained strong, even after weeks in the midst of a heated real-world debate about the future of health care reform.

It may be hard to find such silver linings to other troubling results regarding the effects of fluency. Fluency is a powerful force – one that may provide a mechanism for understanding how rumors take hold and persist. Simply asking subjects to repeat the rumor to themselves – without any indication that it is true – increases their willingness to believe the existence of death panels, even weeks after they read the initial story. In the real world, rumors are repeated and cemented in the echo chamber of the internet. This is true not simply in the United States, but also in other countries. ⁷⁷ Thus, the effects found here may play out on a larger scale in a variety of societies. But even given these findings, there may be hope. Perhaps the power of fluency could be harnessed to increase the effectiveness of corrections. Just as rumors can multiply their power through repetition, perhaps corrections can as well. Future work should explore such possibilities. ⁷⁸ Until we know how to correct false information, rumors and innuendo will remain powerful forces in politics around the world.