Reactions to China-linked Fake News: Experimental Evidence from Taiwan

Abstract

China is accused of conducting disinformation campaigns on Taiwan's social media. Existing studies on foreign interventions in democratic societies predict that such disinformation campaigns should lead to increasing partisan polarization within Taiwan. We argue that a backlash effect, making Taiwan's citizens more united against China, is equally plausible. We conduct a survey experiment exposing participants to a real-life rumour and rebuttal to test these competing hypotheses. We find, at best, mixed evidence for polarization. Although neither rumour nor rebuttal mention China, there is consistent evidence of backlash against China. Most notably, participants across the political spectrum are more inclined to support Taiwanese independence after viewing the rumour rebuttal. These findings indicate that citizens may put aside partisanship when confronted with false news that is plausibly linked to an external actor. We conclude by discussing the broader applicability of our theory and implications for cross-Strait relations.

摘要

中国被指控在台湾社交媒体上进行虚假宣传。现有关于境外势力干预民主社会的研究表明，来自中国的虚假宣传应当导致台湾内部党派的加剧分化。在本文中，我们提出境外势力的虚假宣传同样可能导致舆论反弹，即在面对来自中国的虚假宣传时，台湾公民反而更加团结一致地反对中国。为检验上述两种不同的理论预期，我们进行了一项调查实验。我们让实验参与者阅读现实中的谣言以及对谣言的驳斥，之后通过问卷测量他们的政治态度。我们仅发现了部分支持两极分化假设的证据。虽然在谣言及其驳斥中都未提及中国，但我们的证据一致表明中国在台湾的宣传导致了舆论反弹。最值得注意的是，当看到对谣言的驳斥后，即使是持不同政见的实验参与者也都会更支持台湾独立。调查结果表明，在面对与境外势力有关联的虚假新闻时，公民有可能抛开自身党派偏见。我们在文末讨论了本理论的普适性以及本文发现的舆论反弹效应对两岸关系的影响。

Some states post false news stories (“fake news”) or rumours on social media to manipulate the views and behaviour of citizens in other countries.¹ The strategy has received widespread media coverage and the attention of policymakers across the globe. A primary concern is that democratic citizens, living in societies with unfettered social media access, may be particularly vulnerable to the disinformation campaigns of foreign actors.

Such concerns are supported by studies that find a strong polarization effect of foreign interference.² Individuals will welcome foreign interference when it helps their preferred party or candidate and will oppose foreign interference when it hurts their preferred party or candidate. Thus, when foreign actors use disinformation to promote a preferred political actor, this can result in deepening partisan divides and fragmented public opinion, ultimately threatening to undermine the democratic process.

There are few places where such concerns are more relevant than in Taiwan (Republic of China or ROC). Since Taiwan democratized, it has held free and fair elections, with regular power transitions between its two dominant political parties: the Kuomintang (Chinese Nationalist Party or KMT) and the Democratic Progressive Party (DPP). Competitive elections, high voter turnout, a history of bitter inter-party relations and notable policy differences have created a distinct partisan divide. Throughout, the People's Republic of China (China or PRC hereafter) has sought to influence Taiwan's domestic politics. China has unambiguous intentions to bring Taiwan under Beijing's authority and has engaged in overt and covert efforts to promote re-unification. In recent years, Taiwan has faced an onslaught of fake news across the many social media platforms used by Taiwan's highly connected population.³ While it is often difficult to conclusively establish the PRC's involvement, the political narrative within Taiwan has centred around China as a primary culprit.⁴

Given these conditions, Taiwan's democracy appears vulnerable to external influence and a deepening partisan divide. We argue that such conclusions may be premature. Findings on partisan polarization may not apply to Taiwan in the same manner as found in previous cases. Taiwan faces a precarious international setting with constant threats to its political autonomy. Under such circumstances, concerns about national identity and political autonomy may be more salient than political partisanship. Instead of reacting on the basis of partisanship, individuals who are exposed to false news plausibly linked to a foreign actor will feel that their nation is under threat and will react negatively towards the suspected perpetrator, regardless of their political persuasion. Rather than undermining a politician or political party, the false news story unifies a targeted population through a collective experience of feeling under attack. Hence, instead of partisan polarization, a backlash effect results: the false news story will negatively impact views of the foreign actor, even among individuals more sympathetic to the foreign actor's policy goals.

We conducted a survey experiment in Taiwan to test the polarization and backlash hypotheses. We exposed participants to a real rumour defaming the incumbent DPP administration and also showed a subset of participants a rebuttal of the rumour. We then assessed how the treatments affected participants’ views of the mainland and Taiwanese independence. We find consistent evidence for backlash. The rumour and rebuttal worsened participants’ attitudes towards the mainland, and the rebuttal strongly increased preferences for Taiwanese independence. These effects were consistent across the political spectrum. Notably, neither the rumour nor the rebuttal explicitly mentioned China, revealing that Taiwan's citizens often associate rumours spread by domestic political actors with Chinese interference. We also assessed the rumour and rebuttal's effects on participants’ views of Taiwan's dominant political parties, but, again, we find at best mixed evidence for polarization.

Using additional data gathered in the experiment, we present further evidence against polarization, showing that KMT supporters were particularly willing to alter their assessment of the effects of fake news. Our results indicate that many KMT supporters put “nation” before “party” with respect to fake news.

While we do not claim polarization is irrelevant, our findings demonstrate that polarization is not the inevitable result of fake news. Citizens appear willing to put aside partisan differences when they feel foreign interference threatens to undermine national autonomy or valued political institutions. From a policy perspective, rebuttals are a powerful tool for Taiwan's government and civil society organizations. Furthermore, our findings indicate that the Chinese government, to the extent that it is truly involved in the dissemination of fake news within Taiwan, may wish to re-evaluate its approach. Rather than undermining the incumbent DPP administration, false news stories appear to push Taiwan's public towards support for independence.

Disinformation in Taiwan

Background

Throughout its democratic era, Taiwanese politics has been dominated by two parties, the KMT and the DPP. Despite historical animosity towards the Chinese Communist Party (CCP), the KMT currently has a closer relationship with Beijing. This commonality is rooted in a shared opposition to Taiwanese independence. The KMT is the largest party in the Pan-Blue coalition, which also includes the New Party and the People First Party. Broadly speaking, the Pan-Blue emphasizes economic growth, notably through deeper economic connections to China. Voters who identify as Chinese and Taiwanese, as opposed to solely Taiwanese, tend to gravitate towards the Pan-Blue.

The DPP has its roots in the Dangwai Movement of unaffiliated candidates running for political office during the 1970s and early 1980s. Since its official establishment in 1986, the DPP has evolved into one of Taiwan's two dominant political parties. The DPP has leanings towards Taiwanese independence, and such inclinations are vehemently opposed by Beijing. The DPP is the largest party in the Pan-Green coalition, working with the Taiwan Solidarity Union (TSU), the sometimes-reluctant New Power Party (NPP) and the Taiwan Statebuilding Party (TSP). Collectively, the Pan-Green tends to represent those who self-identify as solely Taiwanese.

China's influence in Taiwan's politics and media is a prominent concern in Taiwanese society, a concern consistently linked to calls for protecting Taiwan's democracy and autonomy.⁵ Moreover, Taiwan's media are widely viewed as exhibiting partisan bias by the Taiwanese public. The DPP and KMT differ on how best to handle the issue of false news. The DPP has been more willing to use formal regulatory measures to counter disinformation, while the KMT has emphasized freedom of information, free speech and letting individuals weigh up information for themselves. The KMT has also argued that Taipei's poor relations with Beijing are at the root of the disinformation problem and must be adequately resolved first. Immediately before the 2020 presidential election, the DPP's Tsai Ing-wen 蔡英文 made the proposal and passage of the Anti-infiltration Act (fan shentou fa 反渗透法) a central campaign issue. The Act criminalizes the political involvement of “foreign hostile forces” (jingwai didui shili 境外敌对势力) in Taiwan's media and politics and includes penalties for spreading disinformation. The KMT opposed the legislation, arguing that it would kick off a “green terror” and be used by the DPP to target political opponents. The KMT also argued that the legislation undermines democratic principles and pointed the finger at the DPP's online army as the real source of most disinformation. The Act passed unanimously after a KMT boycott of the vote.

Trends in disinformation

False news stories pervade Taiwan's social media platforms.⁶ Sometimes, false stories originate as satire or humour that is misunderstood and shared by the public; however, more often these stories are written with commercial and political purposes. False news stories usually include altered or re-purposed visual illustrations and claim to have access to information unavailable to the public.

While some false stories in Taiwan's social media are farcical, such as an altered photograph of a dragon flying past Taipei 101 or a story about a Himalayan flower that blooms once every 400 years, most stories seek to exploit pre-existing anxieties among Taiwan's population. False and misleading stories about health and food safety are among the most common. For instance, drinking black tea, hot water or alcohol, smoking cigarettes, eating garlic, coating one's nostrils with soap, and a vegetarian diet are all said to stop the spread of COVID-19.⁷ Other similarly exploitative stories provide lurid depictions of crimes and societal decay. Stories about textbook revisions, with doctored photos of book pages, claim that students will soon learn about bestiality and will be encouraged to experiment with drugs. Many stories tap into anxieties about modernization and technology, claiming that invisible aspects of modern society, such as radiation, wi-fi and cellular phone signals, microwave ovens, genetically modified foods and air pollution, are imminently fatal.

An oft-cited example of the seriousness of disinformation is the case of Su Chii-cherng 蘇啟誠, a Taiwan representative stationed in Osaka, who committed suicide when false news resulted in widespread criticism of his office. Posts on Taiwan's Professional Technology Temple (PTT), a popular online bulletin board, claimed that after a typhoon, Taiwanese tourists stranded at Kansai International Airport were forced to declare that they were Chinese to secure transportation from the airport, as the PRC government had arranged for buses while the ROC government did nothing. The story quickly spread from PTT to traditional media such as the Global Times, a Chinese Communist Party newspaper. Without verifying the story, media in Taiwan followed suit, with the ROC government promising to investigate. In truth, the Japanese government arranged for the transportation of all stranded travellers; no Taiwanese were forced to declare that they were Chinese. In a sign of how complicated tracing the source of false news stories can be, investigative reporting indicated that early posts came from Beijing-based IP addresses, but a pro-DPP online influencer was also charged with inciting criticism of Su on PTT.⁸

While some false news stories may seek to undermine trust in Taiwan's democratic institutions, many are partisan in nature, attacking a particular political party or candidate. As the KMT has closer ties to Beijing, false news stories tend to accuse the KMT of selling Taiwan to China. During his presidential run, the KMT's Han Kuo-yu 韓國瑜 regularly drew massive crowds for his rallies and parades. Photos from previous, unrelated marches were circulated online as evidence that Han's supporters were waving PRC flags, a false news story designed to play up to the criticisms of Han for being too close to Beijing. Other false news stories claimed that the photos were altered to make the crowds look larger.

Conversely, false news stories targeting the DPP tend to accuse it of betraying Taiwan to governments other than Beijing, or accuse party members of using its incumbency for their own personal gain. False news stories have claimed that President Tsai is actually Japanese. During Tsai's first term, one story claimed that the DPP planned to loan 2,000 of the Palace Museum's most prized cultural relics to the Japanese for a term of 50 years. The story originated on social media in China and was summarily denounced as false by the Palace Museum. Another fake news story about the 2020 election claimed that ballots had been printed with invisible ink that would automatically change votes to Tsai Ing-wen, no matter who a voter selected for president. Previously, when the Tsai administration sought to enact pension reform, false news stories reported that only government officials would receive full pensions. Similar stories reported that officials in the Tsai administration were giving themselves unrealistically high wage raises.

Less nuanced instances of false news from China have used simplified Chinese characters and expressions uncommon in Taiwan. China is also accused of paying Taiwanese social media personalities to produce and disseminate pro-China content. In one case, the host of the YouTube channel, Under the Foot of Yushan (Yushan jiaoxia 玉山腳下), spoke with a Taiwanese accent but was later revealed to be a China National Radio journalist. Highly critical of Tsai and the DPP, Under the Foot of Yushan is still available on YouTube, including content perpetuating the disproven rumour that Tsai falsified her doctoral degree.⁹ It is also possible that the Chinese government is enabling its netizens to independently engage in such activities. For instance, one study found that a 2016 mass messaging attack on Tsai's Facebook page originated in China and showed signs of coordination.¹⁰ Notably, Facebook is blocked in China, but the attackers were able to coordinate activity and bypass China's firewall.

Despite the pervasiveness of disinformation, the effects of these stories are unclear. We have little causal evidence of how citizens respond to these rumours or how they view a foreign intervener associated with the spread of disinformation.

Existing Literature and Hypotheses

Great powers often interfere in other countries’ democratic processes.¹¹ In recent years, the spread of disinformation on social media has become an important strategy for foreign interference.¹² Foreign actors, seeking to further their own policy goals, conduct disinformation campaigns in support of preferred candidates, to discredit less-preferred candidates or to undermine citizen trust in democratic institutions and processes. The ease of sharing information on social media allows disinformation to spread widely and quickly.¹³

Literature on foreign electoral interventions distinguishes between partisan and process interventions.¹⁴ While partisan interventions aim to support or undermine a particular political actor, process interventions aim to shape the political system writ large, for instance by undermining or strengthening democracy. Here, we focus on partisan interventions. We do this because the majority of political rumours we reviewed are most readily classified as partisan, and because it has been observed that China uses a “regime-type neutral” approach to interventions in Taiwan's democracy, where damage to democracy may result from China's interventions but is not the primary goal.¹⁵

Polarization hypothesis

One prominent argument is that disinformation campaigns on social media amplify partisan polarization in targeted populations. This polarization can manifest in attitudes towards the foreign intervener and assessments of domestic political actors. Regarding attitudes towards the foreign intervener, there is considerable evidence that individuals’ attitudes are influenced by whether the foreign intervention is aimed at helping a preferred political actor. Most closely related to our study, Michael Tomz and Jessica Weeks find that Americans are less likely to disapprove of foreign interference involving the spread of fake news when the foreign interference is intended to help their preferred candidate.¹⁶ While not specific to disinformation campaigns, Daniel Corstange and Nikolay Marinov find similar polarization in Lebanon, where individuals view foreign interveners more favourably when interventions favour their preferred candidate.¹⁷ Finally, Leonid Peisakhin and Arturas Rozenas find that Russian analogue television signals targeted at voters in the Ukraine amplified pro-Russian sentiments among those with sympathies towards Russia, but further alienated those already suspicious of Russia.¹⁸

In the context of attitudes towards domestic political actors, ample evidence indicates that fake news is interpreted through a partisan lens. Individuals are more likely to believe fake news and redistribute it when it supports their partisan stance.¹⁹ Individuals are less likely to believe rebuttals of fake news when it contradicts their partisan allegiances.²⁰ Therefore, rumours can polarize attitudes towards domestic political actors because false information disparaging a political actor will reinforce pre-existing negative views among those opposed to the actor, but will do little to sway those who have positive opinions of the political actor.

Several mechanisms may explain this polarization pattern. For instance, individuals may exhibit polarization for consequentialist reasons. As they value the benefits derived from their party or candidate holding office, they will approve of foreign intervention when it helps their preferred political actor. Another possible mechanism is perceptual. Individuals exhibit motivated reasoning whereby they seek out and overweigh information that confirms their prior beliefs, while they avoid and discount information that challenges their existing beliefs.²¹ One of the most prevalent types of motivated reasoning in the political realm is partisan motivated reasoning.²² Individuals interpret information such as news reports or political events through the lens of their partisan allegiances. As a result, the same piece of information can be perceived in widely diverging ways by individuals on opposite sides of the partisan divide. In fact, some studies have found that beliefs, once firmly entrenched, can become so resistant to change that when individuals are confronted with information refuting their beliefs, they will only become more convinced of their pre-existing beliefs.²³

Regardless of the precise mechanism at work, given the strong partisan cleavages within Taiwan, the polarization hypothesis predicts that when a foreign intervener (in this case, China) is linked to disinformation campaigns undermining a political candidate or party, such interventions should deepen partisan divides.²⁴ With respect to the foreign intervener, individuals whose preferred party stands to benefit from a disinformation campaign, either more generally or through a specific piece of false news, should view the foreign actor more favourably than individuals whose party is hurt by the false news. In terms of domestic politics, a piece of fake news disparaging a political actor should negatively influence attitudes among opponents, while having little effect among supporters. Furthermore, rebuttals should be more effective among supporters than among opponents.

Backlash hypothesis

While existing studies on foreign intervention lend support to political polarization, we argue that a second hypothesis is equally plausible. Individuals have a range of overlapping identities, and context will cause such identities to vary in salience. For instance, in some contexts, a person may view themselves in terms of an economic class, while in other situations, a person may identify with their ethnic group. Identity salience will impact who is perceived as the in-group and who is perceived as the out-group, leading individuals to adjust their beliefs and behaviour accordingly.²⁵ Most notably, scholars have found that shifts in identity salience can cause changes in political beliefs and policy preferences.²⁶

Thus, in an environment where partisanship is less salient than another aspect of a person's identity, fake news can have effects other than partisan polarization. Most notably, when a foreign actor is associated with a false news story, the story may cause national identity and political autonomy factors to become more salient. Instead of experiencing the false story as relevant to domestic political contests, individuals who see the story will perceive themselves to be part of an in-group encompassing the entire political unit targeted by the attack. The out-group will be the one suspected of orchestrating the attempted manipulation. A similar logic underlies rally-around-the-flag and diversionary war theories, where leaders rely on an in-group/out-group dynamic to distract from domestic unrest.²⁷

If political autonomy becomes the central concern, we should see citizens of all political persuasions reacting negatively towards the suspected perpetrator of false news stories. Rather than successfully undermining a politician or political party, the foreign actor has united the targeted populace through a collective experience of feeling attacked by an outsider. Under such conditions, a backlash effect occurs, as the false news story has a negative impact on even those individuals more sympathetic to the policy goals of the foreign actor. The foreign government has alienated not just its political adversaries but also potential allies within the target state. In fact, one study on foreign interference finds such a backlash effect against interventions by Western governments and international organizations in Ukraine's electoral process.²⁸ However, the foreign interference in that context was process-oriented, designed to bolster democratic institutions, unlike the partisan disinformation campaign that is of central concern to our study.

We argue that Taiwan may demonstrate such a backlash effect, as political autonomy and national identity are particularly salient in Taiwanese politics.²⁹ In powerful countries such as the United States, where foreign interference is rarely threatening to national autonomy, citizens are likely to worry less about the national repercussions of foreign interference and thus interpret interference through a partisan lens, as documented by Tomz and Weeks.³⁰ However, in more vulnerable countries such as Taiwan, loss of political autonomy is a real threat. Individuals may thus experience foreign interference as an attack on political autonomy and may put aside partisan concerns.³¹ Furthermore, in Taiwan's case, loss of political autonomy would also likely mean a loss of democracy. As many people value democracy regardless of whether or not their preferred party is in power, this may provide further incentives against partisan responses to foreign disinformation campaigns.³² We illustrate the two competing hypotheses graphically in Figure 1.

Figure 1: Hypotheses about the Effects of Foreign Fake News Interference

Experimental Evidence

Design

To differentiate between the polarization and backlash hypotheses, we conducted an online survey experiment with 561 ROC citizens serving as participants. The survey was conducted in December 2019 and exposed respondents to a real rumour previously circulated on Taiwanese social media.³³ Participants were recruited online through the Pollcracy Lab of the Election Study Center at National Chengchi University. Summary statistics are reported in the Appendix. As is common for online surveys, the sample is not perfectly representative of the general population. Participants are better educated than the general population.³⁴ The sample also skews slightly towards the Pan-Green. The polarization hypothesis expects treatment effects to vary by political affiliation. To assess treatment effect heterogeneity without sacrificing power, we report effects for the sub-samples of Pan-Green supporters and Pan-Blue supporters.³⁵

Figure 2 illustrates the experiment. After answering demographic background questions, participants were randomly assigned to one of three approximately equally sized groups. Group 1 served as the control group and immediately moved on to the outcome portion of the survey. Participants in groups 2 and 3 were both shown a rumour, with group 3 immediately shown a rebuttal of the rumour.

Figure 2: Experimental Setup

The rumour

Our choice of rumour was guided by several considerations. First, for ethical reasons, we wanted to avoid creating our own rumour. Second, the rumour needed to be representative, sharing broad characteristics with other false news. For false news with political content, we therefore needed a story with accusations of a political actor betraying Taiwan in favour of an outside actor, or of corruption. Third, there needed to be a publicly available rebuttal.

One rumour satisfying all requirements was a false news story claiming that the Tsai administration was considering a proposal to lease Taiping Island 太平島 to the United States for use as a military outpost. Taiping Island is the largest naturally formed, disputed land feature in the South China Sea, and it is administered by the ROC. This rumour presents a classic example of disinformation in Taiwan, as well as concerns about how rumours spread to traditional media and impact political outcomes. Early versions of the rumour have been traced to the July 2016 Facebook posts of ROC politician Chiu Yi 邱毅. Chiu is a former KMT legislator, most recently running on the New Party legislator-at-large list in 2020. Chiu is known for his connections to mainland China. Chiu's posts claimed Tsai planned to make Taiping Island a US military base. The story's implication is that Tsai is a traitor who values her relationship with the US above ROC sovereignty. The story plays to another common theme in Taiwan's false news: a sense of injustice at being excluded from international forums. Taiwan was excluded from the Permanent Court of Arbitration proceedings relevant to Taiping in 2016, making Tsai's purported neglect of ROC sovereignty over Taiping a particularly treacherous act.

The story was picked up by traditional media in 2018. On 10 June, the ROC Ministry of Foreign Affairs (MOFA) issued a statement that the story was completely fabricated. However, on 29 June, the Chinese Ministry of National Defence denounced the proposal to lease Taiping as “very dangerous” (shifen weixian 十分危险), vowing that the People's Liberation Army would “resolutely safeguard national sovereignty, security, and territorial integrity” (jianjue hanwei guojia zhuquan, anquan he lingtu wanzheng 坚决捍卫国家主权、安全和领土完整).³⁶ The ROC MOFA responded by re-issuing their statement refuting the story, only this time in simplified characters.

The precise wording of the rumour and rebuttal used in our experiment, using language from Chiu's original posts and the official ROC MOFA rebuttal, can be found in the Appendix. So as not to bias participants’ reactions, we followed Huang Haifeng by not labelling the news item a rumour, but instead asking participants if they had “seen this news story.”³⁷ After seeing the rumour, participants in group 2 were asked to assess its credibility on a 7-point Likert-type scale, with higher values signifying higher credibility. Subsequently, group 2 moved to the outcome portion of the survey. After seeing the rumour, participants in group 3 were shown the rebuttal discrediting the rumour. We again asked if participants had “seen the statement.” Participants were asked about the rumour's credibility before moving to the outcome portion.

We are interested in the effect of rumours and rebuttals on individuals’ attitudes towards China and domestic political actors. We collected four primary outcomes of interest. With respect to China, we asked participants about their approval of the mainland government and Taiwanese independence. For domestic political actors, we asked participants about their approval of Taiwan's two main political parties: the DPP and KMT. Approval of the mainland government, DPP and KMT were scored on 7-point Likert-type scales, with higher numbers signifying greater approval. For approval of independence, we used the TEDS (Taiwan's Election and Democratization Study) 6-point scale, ranging from 1 – “unify as soon as possible,” to 6 – “declare independence as soon as possible.”³⁸

Main Results

Figure 3 graphically summarizes the qualitative results. The first column shows that the rebuttal reduced rumour credibility, but its effectiveness was limited and a mean credibility rating above three indicates that uncertainty remained about the rumour's validity even after the rebuttal. Contrary to the polarization hypothesis, the rebuttal appears to have been more effective in reducing rumour credibility among Pan-Blue than among Pan-Green supporters.

Figure 3: Mean Credibility and Approval Ratings

Note:

Column one shows credibility of rumour across the two treatment groups. Columns two through five show approval level for each outcome of interest across experimental groups.

We see consistent effects of backlash against China. The second column shows that exposure to the rumour somewhat reduced mainland government approval, and this negative effect was reinforced by the rebuttal. Importantly, neither the rumour nor the rebuttal made direct reference to China, indicating that participants associate rumours with interference by Beijing.³⁹ Rumour and rebuttal had little effect among Pan-Green supporters, as mainland government approval was already near the bottom of the scale in the control group. The rumour decreased mainland government approval among Pan-Blue supporters, but the rebuttal had little additional effect, implying that the rebuttal was most effective among unaffiliated participants. Backlash also manifested in approval of independence. The third column shows that the rumour had no effect on independence approval, but the rebuttal strongly increased approval of independence. We see no evidence of polarization, as the rebuttal actually increased support for independence more strongly among Pan-Blue supporters than among Pan-Green supporters, albeit from a lower baseline.

Regarding domestic politics, column four shows the rumour lowered DPP approval, but the rebuttal effectively restored DPP approval. We see no evidence of polarization in DPP approval. Among Pan-Blue supporters, rumour and rebuttal increased DPP approval compared to the control group, while the rumour somewhat decreased DPP approval compared to the control group among Pan-Green supporters. Column five shows that the rumour had virtually no effect on KMT approval. The rebuttal noticeably reduced KMT approval, likely because a KMT-affiliated politician posted the rumour. We find mixed evidence for polarization in KMT approval. Consistent with polarization, the negative effect of the rebuttal appears to have been driven by Pan-Green supporters and unaffiliated participants. Pan-Blue supporters viewed the KMT more favourably in the rebuttal group than in the rumour group. However, inconsistent with polarization, the rumour reduced KMT approval among Pan-Blue supporters and slightly increased approval among Pan-Green supporters.

Having studied the effects of rumour and rebuttal qualitatively, we now quantify the effects. Table 1 shows average treatment effects using difference-in-means tests. The effect of the rebuttal on credibility did not quite reach statistical significance (row three, column one). The rumour had no significant effects on any outcome of interest (row one). The rebuttal was more powerful than the rumour. Exposure to the rebuttal significantly decreased mainland government approval compared to the control group (row two, column two), and significantly increased independence approval compared to the control and rumour groups (rows two and three, column three). The restorative effect of the rebuttal on DPP approval was large and significant (row three, column four). The rebuttal significantly decreased KMT approval compared to the control and rumour groups (rows two and three, column five).

Table 1: Average Treatment Effects

Notes:

Standard errors in parentheses. *p < 0.1; **p < 0.05; ***p < 0.01.

The effect of the rebuttal on independence approval was the strongest and most precisely estimated effect. In fact, we also re-ran the analyses adjusting significance levels for multiple hypotheses testing using the (very conservative) Bonferroni correction. While all other effect estimates lose significance, the rebuttal's effect on independence approval remains significant. Thus, we place particularly high confidence in this result. However, we emphasize that this does not imply the other effect estimates are true nulls. There is no such thing as a “correct” significance level, and the direction of all full sample effects makes sense given our theoretical priors.

Our quantitative results mirror the qualitative insight that polarization, at best, played a minor role. Contrary to polarization, the rebuttal significantly reduced rumour credibility among Pan-Blue supporters and not among Pan-Green supporters (rows six and nine, column one). There is also no evidence that Pan-Blue or Pan-Green supporters’ attitudes towards China are differentially affected by the rumour or rebuttal. The rumour and rebuttal, albeit insignificantly, reduced approval of the mainland compared to the control among Pan-Blue supporters. Approval of independence increased strongly and significantly among both Pan-Blue and Pan-Green supporters in the rebuttal group.

Regarding domestic politics, if the rumour or rebuttal had polarizing effects, these effects were not large enough to be detected at conventional levels of statistical significance. The best evidence for polarization is that the rebuttal (insignificantly) increased KMT approval compared to the rumour group among Pan-Blue supporters (row nine, column five), but decreased approval among Pan-Green supporters (row six, column four). Given that Pan-Blue supporters were slightly underrepresented, this also indicates that the negative effect of the rebuttal on KMT approval in the full sample (row three, column five) may not generalize to more representative samples. Yet many other effects go in the opposite direction as that predicted by polarization.

However, we emphasize that we do not interpret our findings as indicating that polarization is necessarily absent. Given the relatively small subgroup sample sizes and the large number of comparisons, we caution against too literal a reading of the effect estimates and p-values. Instead, we prefer to interpret our findings as evidence that polarization plays a subordinate role, if at all, in mediating individuals’ reactions to the rumour or rebuttal in this setting.

While average treatment effects in randomized experiments are, by design, unbiased, including covariates can improve the precision of estimates. As a robustness check, we therefore report the estimates from linear regressions, adjusting for participants’ age, gender, monthly income, education level, political affiliation and whether or not the participant identifies as Taiwanese in Table 2.⁴⁰ The treatment effects are relative to the control group. The regression analyses reinforce the insight that treatments produced backlash against China. Rumour and rebuttal reduced mainland government approval relative to the control group. However, the effect of the rumour rather than the rebuttal is now significant. This difference in results appears to be driven by minor joint imbalances across the treatment groups in the proportions of individuals supporting Pan-Green or Pan-Blue, and identifying exclusively as Taiwanese (See Balance Table in the Appendix). When excluding these covariates, we recover the difference-in-means results. Thus, while the treatments lower approval of the mainland, it is difficult to establish conclusively the relative contribution of rumour and rebuttal. As before, the effect of the rebuttal on independence approval is large and precisely estimated. Again, there is no significant effect of rumour or rebuttal on DPP approval and the rebuttal's effect on KMT approval is no longer significant. Above, we caution against placing too much emphasis on the rebuttal's negative effect on KMT approval, as Pan-Blue supporters were slightly underrepresented, and we find some evidence of partisan polarization regarding KMT approval.⁴¹

Table 2: Linear Regression Effect Estimates

Notes:

Standard errors in parentheses. *p < 0.1; **p < 0.05; ***p < 0.01.

Additional Evidence

Thus far, there are two main takeaways from the experiment. First, we found strong evidence for backlash against China. This finding is most apparent in the rebuttal's effect on independence approval. The effect on mainland government approval is less pronounced, which can likely be attributed to mainland government approval already being low in the control group. Importantly, effects were as strong for Pan-Blue supporters as they were for Pan-Green supporters. Second, neither rumour nor rebuttal had robust effects on participants’ approval of the DPP or KMT, and polarization, at best, played a minor role.

Malleability of opinion

Using additional data from the survey, we provide further evidence that participants were willing to put aside partisan reasoning when exposed to rumour and rebuttal. Following the main portion of the experiment, we asked participants if their opinion of a) the mainland and b) Taiwan's government would be affected if the mainland government spread fake news criticizing Taiwan's government.

Figure 4 displays the proportions of respondents who believed rumours could negatively affect their views of the two governments. The first column shows that in the full sample and among Pan-Green supporters, rumour and rebuttal had little effect on the proportion of individuals who believed their views of the mainland would be negatively affected by fake news. However, a striking effect is observable among Pan-Blue supporters. While Pan-Blue supporters in the control group were much less likely than Pan-Green supporters to believe fake news would negatively affect their opinion of the mainland, exposure to the rumour and rebuttal strongly increased Pan-Blue supporters’ beliefs that fake news could negatively affect their views.⁴² In fact, in the rebuttal group, Pan-Blue and Pan-Green supporters had virtually identical beliefs. We see a similar pattern in column two with respect to Taiwan's government. While the effect was not quite as strong, it was again Pan-Blue supporters who became more likely to believe that fake news could negatively affect their views of Taiwan's government, across both rumour and rebuttal groups.

Figure 4: Proportions of Respondents Who Believed Rumours Would Negatively Affect their Views of Mainland China or the Taiwanese Government across Experimental Groups

This evidence is consistent with national concerns outweighing partisan concerns. As discussed above, the KMT sometimes downplays the role of fake news in Taiwanese society by arguing that people can judge fake news for themselves. The above results indicate that Pan-Blue supporters followed this logic in the control group and, particularly with respect to their views of the mainland, they were less likely than Pan-Green supporters to say that their views would be affected by fake news. However, once they were exposed to the rumour, which gave participants the opportunity to self-inspect their reaction to fake news, Pan-Blue supporters put aside domestic partisan battles and became more willing to say that fake news could affect their views.

We conducted one further exercise, showing that different reasoning does indeed underlie Pan-Green and Pan-Blue supporters’ self-assessment of the effects of fake news on their views. We asked participants to justify their stated beliefs about the influence of rumours on their opinions of the mainland in an open-ended response.⁴³ Following a close reading of all responses, we manually categorized the responses into several topics.⁴⁴ Among those who answered that rumours would not influence their views of the mainland, we distinguished three primary categories: (1) individuals who said their opinion would not be influenced because their opinion of the mainland was already extremely poor (Negative Prior); (2) those who said they rely on their personal skills or experience to differentiate between factual and false news (Personal Skill); and (3) those who voiced general distrust of the media or argued both sides are equally to blame (General Distrust).

Figure 5(a) shows there are marked differences in the topic frequencies between subgroups.⁴⁵ While most Pan-Green supporters who believed their opinion would not be affected did so because of a strong negative prior about the mainland, Pan-Blue supporters, in accordance with KMT narrative, were more likely to believe their personal skill allowed them to distinguish real from fake news. They were also more likely to voice general distrust of the media environment. So as not to overburden participants, we did not elicit open-ended responses for participants’ beliefs about the effects of rumours on their views of Taiwan's government, but we believe similar patterns apply there as well.

Figure 5: Reasons Given by Respondents Why Rumours Do or Do Not Influence Their Views of the Mainland

Views of China

We conclude our empirical investigation with a substantive note regarding participants’ views of China. Many participants expressed negative views towards China in the open-ended responses. Some participants expressed outright hatred and described China as the enemy.

Among participants who said fake news could influence their views of the mainland, we identified four main topics in our manual coding of the open-ended responses: (1) concerns about national autonomy, such as claims that dissemination of fake news is part of China's “re-unification war” (tongzhan 统战) or concerns about meddling in Taiwan's domestic affairs (Autonomy); (2) concerns about the political system, such as democratic processes or the relationship between people and government (Political System); (3) concerns about Taiwan's social cohesion and worries that fake news would cause chaos (Social Cohesion); and (4) greater distrust of mainland China (Distrust Mainland). Figure 5(b) displays the topic frequencies. In line with our theoretical argument, many participants voiced concerns about political autonomy, but concerns about social cohesion were another important factor. Overall, Pan-Green and Pan-Blue participants did not differ strongly in their concerns, but Pan-Blue supporters were somewhat more likely than Pan-Green supporters to mention that rumours would increase their distrust of China. This difference may be attributable to Pan-Green supporters already deeply distrusting China.

Discussion

Our experimental results show that foreign disinformation campaigns do not inevitably increase partisan polarization. Democratic citizens are able to put aside partisan differences and unite in the face of foreign interference. Our findings indicate citizens do not blindly trust information relayed to them and are willing to change their opinion even when it goes counter to partisan positions. Of particular note is the auxiliary finding that Pan-Blue supporters became more willing to admit that rumours may negatively affect their opinion of the mainland and Taiwan's government after being exposed to the rebuttal. The research on motivated reasoning shows that such willingness to question one's own convictions is not guaranteed. Thus, freedom of information and speech may make democracies vulnerable to foreign disinformation campaigns, but democracies are not simply at the mercy of foreign governments.

Of course, in real life, most rumours are not immediately followed by a rebuttal from an authoritative source, so our results may not translate one-to-one into the real world. However, for a government determined to combat fake news and oppose foreign political influence, our findings indicate the effectiveness and importance of rebutting rumours. Our experiment shows that rebuttals, even in the form of a few simple sentences, can be effective in mitigating the impact of rumours and, if desired, raising opposition to the foreign intervener. As such, efforts within Taiwan to combat rumours through easy-to-use fact-checking tools appear to be a promising approach for countering disinformation.

Furthermore, in the minds of Taiwan's citizens, fake news appears inherently linked with China. Although neither rumour nor rebuttal directly mentioned China, respondents become more strongly opposed to China. If China is truly conducting a disinformation campaign aimed at making Taiwan's citizens more pro-mainland or open to re-unification, this strategy appears to be backfiring. Of course, China may not be aiming to win over Taiwan's citizens; it may instead be attempting to sow general distrust of the media and political elites or wish to overwhelm Taiwanese citizens by flooding Taiwanese social media with a barrage of conflicting information.⁴⁶ However, as shown above, relatively few individuals voiced general distrust of the media in their open-ended responses, indicating that, so far, fake news is not associated with mass disillusionment with Taiwan's media. Further research is needed to investigate the motivations behind China's disinformation campaigns in Taiwan.

Naturally, our study has limitations. For instance, exposure to a single rumour may not have the same effects as a constant stream of fake news. However, as even a single rumour made individuals less approving of China, it is likely that if rumours are associated with Chinese interference, a higher volume would only strengthen the effects of backlash. In fact, our results match observations made about Taiwan's 2020 election during which concerns about disinformation and Chinese interference played a prominent role.⁴⁷

We again emphasize that we do not believe partisan polarization is irrelevant in Taiwan's society. Our sample size was limited and some of our findings, such as KMT supporters becoming more approving and DPP supporters becoming less approving of the KMT upon seeing the rebuttal, indicate that polarization may play a role. We encourage future work, using our findings as a starting point, to investigate when polarization is most relevant and when backlash effects are more prominent.

One avenue for future research is to investigate whether different types of rumours produce varying effects. For instance, one rumour we considered for our experiment was a 2017 online rumour that the Tsai administration planned to ban the burning of incense and joss paper in Taiwan's temples. Over 10,000 people, representing more than 100 of Taiwan's temples, joined a July 2017 march against the restrictions, despite multiple official statements that the rumour was false. Perhaps a rumour closer to the cut of daily domestic politics, or more threatening to aspects of one's personal identity, may shift a rumour's overall effect.

Finally, we believe our theoretical arguments and experimental results extend beyond Taiwan and China. Whenever foreign disinformation campaigns present a clear threat to a country's national autonomy and political institutions, we expect citizens to be more willing to put aside partisanship and view rumours through the lens of national autonomy. Thus, we expect to find evidence of backlash in smaller and militarily weaker countries that are facing disinformation campaigns carried out by a more powerful foreign actor. These effects are likely particularly pronounced when there are strong democratic norms embraced by a large portion of the population.