When does public diplomacy work? Evidence from China's “wolf warrior” diplomats

Abstract

How does public diplomacy shape global public opinion? In this note, we theorize that positive public diplomacy that emphasizes aid and friendship works, while negative messages that criticize international rivals are ineffective. We conduct an experiment, to our knowledge the first of its kind, that randomly exposes Indian citizens to real Twitter messages from Chinese diplomats. We find that positive messages emphasizing aid and friendship improve perceptions of China, even in times of escalating violent conflict. However, messages from so-called “Wolf Warrior” diplomats that harshly criticize the United States are ineffective and can backfire in times of crisis. We argue public diplomacy can be a useful tool for global powers, but that domestic political pressures have pushed some diplomats, like China's Wolf Warriors, toward nationalist messages that do not appeal to foreign audiences.

As the United States and China compete for global influence, both countries have spent large sums on public diplomacy. Public diplomacy—defined as an attempt by a government to “communicate with and attract the public” of a foreign country (Nye, 2008, p. 95)—attempts to shape global public opinion through leader visits, educational programs, diplomatic exchanges, traditional media, and social media. Each year, the United States spends over $2 billion on public diplomacy, while China spends an estimated $8 billion.¹ This investment in molding public opinion could potentially pay dividends for China and the United States. Scholarship shows that public opinion shapes important foreign policy outcomes, such as whether politicians support using military force (Tomz et al., 2020; Chu and Recchia, 2022) and whether nations join military alliances or strike trade deals (Milner and Tingley, 2011; Cooley, 2012; Goldsmith and Horiuchi, 2012). Yet despite the importance of public opinion for international relations, recent scholarship provides conflicting expectations about when public diplomacy efforts persuade.

In this letter, we theorize that positive public diplomacy emphasizing aid and friendship builds foreign support and that negative public diplomacy critiquing geopolitical rivals is ineffective and can even backfire. Examples of positive public diplomacy include Chinese diplomats’ efforts to highlight the country's pandemic aid or infrastructure programs in developing countries. These positive messages build support for increased bilateral cooperation—as well as more general goodwill toward the donor government and nation—by causing audiences to update their beliefs about the material benefits of a country's foreign policy. Negative public diplomacy, on the other hand, focuses on critiques of geopolitical rivals, such as Chinese diplomats who use social media platforms to condemn the United States. Public diplomacy in which one great power criticizes another is less relevant to the material and political concerns of the audiences in a third country, such as India, targeted by public diplomacy. We argue negative public diplomacy is ineffective at building support, and can under some circumstances even backfire.

We arrive at these conclusions based on an experiment that randomly exposed Indian citizens to actual Twitter messages from Chinese diplomats. The preregistered experiment randomized whether recipients in India saw messages that promoted Chinese aid and friendship, messages from so-called “Wolf Warrior” diplomats who criticized the United States,² or a placebo condition. This experiment was embedded in a survey that was fielded in two waves, one shortly before and one immediately after deadly clashes between the Indian and Chinese armies along the Sino-Indian border in June 2020. India is a hard case for studying the effectiveness of Chinese public diplomacy: in a recent Pew study, Indians had the most pessimistic view of China's economic rise out of 34 surveyed countries.³

Despite a history of conflict and mistrust between China and India, we find that positive Chinese public diplomacy touting aid and friendship had robust effects—strikingly, these positive messages improve public perceptions of China even immediately after violent territorial conflict. Our evidence suggests that these changes in attitudes are largely a consequence of respondents’ updating their beliefs about the generosity of Chinese foreign aid, consistent with prior research on the reputational benefits of aid (Goldsmith et al., 2014).

However, we also show that negative Wolf Warrior diplomacy is largely ineffective and may in some circumstances backfire. So-called Wolf Warrior diplomats have gained fame for using harsh language to criticize the United States and its allies, such as a Tweet calling Canadian Prime Minister Justin Trudeau a “boy” and “running dog” of the Americans. Prior to the 2020 border clash, we show that these types of messages do not move public opinion in a positive or negative direction. In the aftermath of the border clash, however, when Indian public opinion toward China grew increasingly negative, we show that exposure to Wolf Warrior diplomacy caused attitudes toward the Chinese people to fall further. These negative messages make up only around 5 percent of Chinese diplomats’ Twitter posts (Leutert and Atkinson, 2022), but the tone of these Tweets earns them media exposure in major outlets throughout the world.⁴ Domestic political pressures and the promise of global media exposure likely push a minority of Chinese diplomats toward these messages even though they are ineffective and, in times of crisis, counterproductive.

1. Does public diplomacy persuade?

Public diplomacy aims to shape global public opinion by directly engaging with foreign audiences. Increasingly, public diplomacy efforts happen through social media platforms. The US Department of State operates some 423 Twitter accounts and the Chinese Ministry of Foreign Affairs some 270 accounts.⁵ These accounts reach millions of citizens in over 100 countries. Yet there has been little if any research on the causal impact of digital public diplomacy on global attitudes.⁶

Academic research suggests that traditional public diplomacy can influence public opinion, but not always in the ways that governments intend. Foreign visits by national leaders, for example, improve perceptions of the leader's government in the host country, primarily by generating favorable media coverage (Goldsmith et al., 2021). On the other hand, Chinese Confucius Institutes promote greater interest in China and more favorable media coverage, but this does not necessarily translate into more favorable impressions of the country's government (Brazys and Dukalskis, 2019; Green-Riley, 2020).

A related strand of research investigates the role of foreign aid in improving perceptions of the donor government—this research again finds that foreign aid can either backfire or help donor countries strengthen their reputations abroad. Cross-national evidence suggests that aid can strengthen public opinion toward donor nations (Goldsmith et al., 2014). Yet several within-country studies have found small or even negative reputational returns to foreign aid (Dietrich et al., 2018; Blair et al., 2019).

This research leaves us with unclear expectations about the effects of digital public diplomacy, which differs in important ways from traditional public diplomacy and aid. On the one hand, digital public diplomacy is low-cost relative to other forms of diplomacy like leader visits or donations. As a result, foreign audiences might regard digital public diplomacy as cheap talk, and we might expect that these messages may not move foreign public opinion. On the other hand, social media platforms offer a unique way for diplomats to directly engage with foreign audiences. Instead of having their message filtered through local media outlets, diplomats on Twitter have direct control over the messages audiences see, even if they cannot control the conversation that follows. For these reasons, we might expect that digital diplomacy may be able to shape public opinion in a positive direction.

To begin to resolve these conflicting expectations, in this note, we theorize how the type of message conveyed through online public diplomacy can persuade a skeptical audience, with special attention to Chinese public diplomacy.

Digital public diplomacy can be disaggregated into two types: positive messages of friendship and negative attacks on rivals. The first of these messaging strategies resembles traditional public diplomacy: it portrays the sending country in a friendly light, usually by emphasizing its benevolent foreign aid or expressions of friendship. The second is perhaps unique to diplomacy conducted via social media. Whereas “diplomatic” is usually a byword for politeness and attention to protocol, diplomats on social media can resort to surprisingly strong language in attacks on rival countries. For example, Chinese diplomats have used Twitter to mock and attack American officials and politicians. These types of “Wolf Warrior” posts are rare—one estimate is that they constitute 5 percent of the tens of thousands of Twitter posts by Chinese diplomats (Leutert and Atkinson, 2022)—yet earn significant media attention.

The difference in tone between these two messaging strategies suggests different mechanisms of persuasion. We hypothesize that positive messages improve perceptions of the sending country (e.g. China) by leading audiences to update their beliefs about the generosity of the sending country's foreign policy and its potential benefits for the recipient country (e.g. India). This leads to improved perceptions of the sending state and increased support for bilateral cooperation.

H1 Exposure to positive messaging improves perceptions of the sending state and strengthens support for bilateral cooperation.

By contrast, negative attacks by the sending state (e.g. China) directed toward a geopolitical rival (e.g. the United States) are less likely to improve perceptions of the sending state in the recipient country (e.g. India). These types of criticism do not reveal credible information about the sending state, and are not relevant to the material concerns of audiences in the recipient country like India (whereas attacks on India's rival and neighbor Pakistan would presumably be more likely to resonate). However, it might potentially have an impact on Indian attitudes towardthe United States.

H2 Exposure to negative messaging does not alter perceptions of the sending state or strengthen support for bilateral cooperation.

As we explain in more detail below, our experiment also afforded us the unexpected opportunity to explore how a geopolitical crisis affects reactions to public diplomacy. The current literature does not provide clear theoretical expectations for how a crisis might influence public diplomacy. We suggest that a crisis between the sending and receiving state leads to significant hostility directed toward the people of the sending country, because of the hostile anti-foreign emotions that messaging around these crises can create.⁷

H3 During a crisis, exposure to negative messaging leads to worse perceptions of the sending state and its people.

Hypotheses H1 and H2 were pre-registered, as were our method for constructing outcomes and plan for analyzing them via difference-in-means tests (described below).⁸ H3 was not preregistered, and should be considered exploratory.

2. Research design

To examine our hypotheses, we conducted an experiment in India to examine the effect of different types of public diplomacy on popular attitudes. We chose India, a regional rival of China's, as a hard case to establish a lower bound for the effectiveness of this strategy. India's relations with China have not been warm in recent decades; nevertheless, the two countries have worked to increase coordination on issues of global governance through summits of the BRICS nations, and Indian universities are host to three Confucius Institutes.⁹ Thus, India is a difficult but relevant target for Chinese public diplomacy efforts.

In total, we surveyed 4,677 residents of India. The survey was distributed by Lucid, a market research firm. We used quota sampling to ensure gender balance and a variety of ages, but did not otherwise seek to make our sample representative. Nevertheless, the sample does resemble the characteristics of the key target demographic: active internet users and especially Twitter users. Our sample was predominantly young, with 82 percent under 40 years old, and more male than female, which roughly mirrors what is known of Twitter users in India.¹⁰ Because the majority of Twitter users in India use English, and because the Chinese diplomats we study typically tweet in English to Indian audiences, we conducted the survey in English.

2.1. Public diplomacy materials

Our experimental treatments were composed of genuine tweets from Chinese diplomats. The first treatment, which emphasized a message of friendship, consisted of three tweets from Sun Weidong, China's ambassador to India, and two tweets from the Ministry of Foreign Affairs. Two of these messages advertised Chinese donations to the Indian Red Cross and World Health Organization during the ongoing coronavirus pandemic. Although this method of presentation bundles the effect of five separate tweets, this brought us closer to our goal of understanding the effect of a charm offensive, as opposed to the effect of any particular tweet. Examples are listed in Table 1, and the full treatments can be found in the online Appendix.

Table 1. Examples of public diplomacy tweets

The second treatment, which criticized the United States, drew on three tweets from the Ministry of Foreign Affairs, and two from Zhao Lijian, who in addition to being deputy director of the Information Department at the Ministry of Foreign Affairs, is the most visible of China's “Wolf Warrior” diplomats. These tweets portrayed the United States as a threat to world peace. Participants who did not view either of these treatments instead viewed a placebo control, which consisted of five tweets about entertainment-related topics.

2.2. Measuring attitudes toward China

We identified four primary outcomes of interest: perceptions of China's government, perceptions of the Chinese people, attitudes about India's policies toward China, and perceptions of China's response to the Covid-19 epidemic. For each outcome, we asked participants to express agreement on a seven-point scale with 3 to 6 questions in that category. (The full list of statements is included in the online Appendix.) We then combined questions into an index for each category, a strategy that reduces measurement error in surveys (Ansolabehere et al., 2008; Broockman et al., 2017). Each outcome comprised at least three items, as recommended by Robinson (2018).¹¹

The outcome for each category was then constructed by extracting the first principal component of the corresponding group of statements. Principal component analysis is a widely-used method of identifying a latent factor from a high-dimensional measure (Thompson, 2004; Ansolabehere et al., 2008). It is often preferred to summing responses because it downweights “noisy” items that do not follow the dominant trend within responses. This form of dimensionality reduction flattens an individual's responses, but in a way that is analytically useful, much as it can be convenient to refer to a politician as “liberal” or “conservative.”

Because the Wolf Warrior treatment condition targets the reputation of the United States, we also collected an identical set of outcomes for that country. Question ordering was randomized at three levels: by country, outcome, and individual statement, which ensured a coherent survey-taking experience while guarding against question-ordering effects.

2.2.1. The crisis as a natural experiment

Our first wave of 2319 responses was collected between May 7 and 9, 2020. Thirty-seven days after the first wave of data collection ended, on the night of June 15–16, Chinese and Indian soldiers fought their largest engagement since the 1962 border war. The battle, which saw the first deaths in the conflict since 1974, took place in the Galwan River Valley, near the triple point of territory controlled by India, China, and Pakistan. At least 20 Indian soldiers were killed.

In the wake of this incident, we collected a second wave of responses, exploiting the as-if-random nature of the timing of the event to compare effect sizes in normal and crisis circumstances. We collected 2358 responses, beginning roughly 48–60 hours after news of the clash broke, and ending 22 hours later.

3. Results

3.1. Positive public diplomacy emphasizing aid improves perceptions of China

We first consider the case of friendly messaging, which was effective in improving perceptions of China (Figure 1). Exposure to this messaging improved opinions of the Chinese government (“Government”), in concordance with our first hypothesis.¹² In addition, these respondents increased their support for Indian cooperation with China (“Policy”) and also evaluated of China's response to the coronavirus pandemic more positively (“Covid-19”). The wide range of outcomes that improved after reminders of China's generosity is striking. Question-level results are available in the Appendix.

Figure 1. Effects of positive messaging.

We add two qualifications to these results. First, when the first principal component is extracted separately for each wave, the factor loadings for three related items about India's foreign policy (“Policy”) change sign.¹³ Figure 1 shows our pre-registered analysis; re-running the analysis with PCA calculated separately for each wave changes the estimate for this item, which does not attain statistical significance. Second, the persuasive effect of public diplomacy, though broad, is modest in size—in most cases only about one-tenth of a standard deviation. Online public diplomacy is unlikely to transform a country's international image on its own, but it does appear to offer a modest reputational return in exchange for a minimal investment.

Wolf Warrior-style public diplomacy, on the other hand, was in our first round survey ineffective in shaping perceptions about China or the United States (Figure 2) consistent with hypothesis two. Across all outcomes, the effects are precisely estimated to be close to zero.

Figure 2. Effects of Wolf Warrior messaging by wave.

3.2. Impact of the crisis

Can online public diplomacy shape perceptions even during a geopolitical crisis? A reasonable expectation is that violent conflict would make positive public diplomacy ineffective, as audiences turn against a perceived rival and enemy. The unexpected outbreak of deadly border violence one month after our first wave of responses was collected afforded us a rare opportunity to compare the effects of public diplomacy in two contexts whose only practical difference was a dramatic jump in geopolitical tensions caused by a militarized dispute. Because these results are not pre-registered they should be interpreted cautiously, and the analysis is exploratory.

To our surprise, friendly messaging continued to have modest positive effects on perceptions of China (Figure 3). All four outcomes displayed effect size estimates very close to those from the pre-crisis wave.

Figure 3. Effects of positive messaging by wave.

Another feature of the interaction between the border crisis and public diplomacy was a modest anti-China backlash sparked by Wolf Warrior messaging (Figure 2). In particular, perceptions of the Chinese and American peoples shifted in response to this messaging strategy during the crisis: respondents perceived Chinese people to be less trustworthy, their culture having fewer positive aspects, and their ideas and customs less welcome in India. American people and culture, by contrast, were rated more highly, which was the opposite effect of that intended by the Wolf Warrior diplomats.

4. Conclusion

In this study, we provide experimental evidence of the effectiveness of Chinese public diplomacy via social media, and leverage a natural experiment to learn how international crises affect these efforts. We conclude that a messaging strategy that emphasizes generosity and friendship is effective both in normal times and during a crisis. Attacks on rivals, however, do not appear to be persuasive and may even backfire.

Positive public diplomacy can be effective, even in the hardest of hard cases: citizens of a rival nation during a security crisis. For this reason, China will likely continue to invest in aid and public diplomacy efforts to improve its image abroad and reduce concerns about the threat it poses. However, there are important limits to public diplomacy. In particular, domestic politics can pressure a minority of diplomats to deploy nationalist messages (Martin, 2021).¹⁴ We show these messages are likely to be at best ineffective, and may in some cases backfire.