Gaming the system: suboptimal compliance with loot box probability disclosure regulations in China

Abstract

Loot boxes provide randomized rewards in video games; their purchase is linked to disordered gambling and they are present in approximately half of UK video games. The relative novelty of loot boxes means that regulators and policymakers in various jurisdictions are still deciding how to regulate them. The People's Republic of China (PRC) is the first, and presently only, jurisdiction to legally require companies to disclose the probabilities of obtaining randomized loot box rewards – an approach that is also favored by the industry as self-regulation. This study is the first to assess paid loot box prevalence in the PRC and companies’ discretionary interpretations of probability disclosure regulations. Loot boxes were found in 91 of the 100 highest-grossing PRC iPhone games. Of games deemed suitable for children aged 12+, 90.5% contained loot boxes. Probability disclosures could not be found for 4.4% of games containing loot boxes. Disclosures were implemented through various methods both in-game and on the games’ official websites; however, consistent with the concept of ‘sludge,’ only 5.5% used the most prominent format of automatically displaying the probabilities on the in-game loot box purchase page. Loot box probability disclosures should be uniform and visually prominent to best help inform consumers.

Introduction

‘Loot boxes’ are virtual video game items providing randomized rewards of in-game and potential real-world value (Nielsen & Grabarczyk, 2019; Drummond et al., 2020b; Rockloff et al., 2020; Xiao, 2020d). Some loot boxes can be ‘opened’ for free, for example, after completing in-game tasks (Nielsen & Grabarczyk, 2019; Rockloff et al., 2020; Xiao, 2020e). However, this study focuses on paid loot boxes that require the payment of real-world money to open; hereinafter, all references to loot boxes are to paid loot boxes. Loot boxes are psychologically akin to gambling (Drummond & Sauer, 2018; Rockloff et al., 2020; Xiao, 2020a), and engagement with loot boxes is related to disordered gambling (Zendle & Cairns, 2018, 2019; Brady & Prentice, 2019; Brooks & Clark, 2019; Kristiansen & Severin, 2019; Larche et al., 2019; Li et al., 2019; Zendle, 2019a, 2019b; Zendle et al., 2019a, 2019b; Drummond et al., 2020a, 2020b; Rockloff et al., 2020; Wardle & Zendle, 2020; Close et al., 2021; Hall et al., 2021). The relative novelty of loot boxes means that many countries are still deciding how to regulate them (Moshirnia, 2018; Schwiddessen & Karius, 2018; Castillo, 2019; Hong, 2019; Liu, 2019; Xiao & Henderson, 2019; Xiao, 2020d, 2020e). For example, Belgium's gambling regulator determined that loot boxes constitute gambling under existing law (Belgische Kansspelcommissie [Belgian Gaming Commission], 2018) and threatened prosecution against game companies implementing them, effectively banning loot boxes in Belgium (Cerulli-Harms et al., 2020; Xiao, 2020d). In contrast, the UK's gambling regulator has determined that it is presently unable to regulate most loot boxes as gambling under existing law (UK Gambling Commission, 2017; Cerulli-Harms et al., 2020; Xiao, 2020d, 2020e). As yet, no research has evaluated the effects of any existing loot box regulations.

Loot boxes may not necessarily be represented as ‘boxes’ per se and may instead be represented, for example, as ‘card packs,’ ‘scratch cards,’ and ‘prize wheels.’ These other representations of randomized monetization methods are also included under the ‘loot box’ terminology. The total spending on loot boxes and gambling using in-game items has been predicted to rise from US$30 billion in 2018 to US$50 billion in 2022 (Juniper Research, 2018). The in-game items (including those obtained through loot boxes) of three popular Steam PC games have been transacted 1.45 billion times between players, with an aggregated value of over US$1 billion (Drummond et al., 2020b). Extreme cases of loot box spending have been reported in the media: one UK player ‘spent nearly £700 [≈US$910] in a month’ (Thomas & Young, 2019); another spent over ‘[US]$10,000 in just two years’ (Yin-Poole, 2018); and four children spent ‘nearly £550 [≈US$715] in three weeks’ of their father's money without permission, but still failed to obtain the rare item they were hoping for (Kleinman, 2019). A dataset of actual loot box openings by Mainland Chinese gamers confirmed that the loot boxes of one popular game generated revenue for the company of over US$500,000 a day from Mainland China alone, and that the majority of loot box revenue was generated by the top 10% of spenders: indeed, the top 1% alone generated 26.3% of all revenue (Zendle et al., 2020b).

A recent survey of the 100 highest-grossing UK iPhone games found that 59% contained loot boxes, and that 94.9% of games containing loot boxes were deemed suitable for children aged 12+ (Zendle et al., 2020a). This high prevalence of loot boxes in UK games deemed suitable for children may be one reason why 22.9% of 11–16-year olds in the UK have reported paying money for loot boxes (UK Gambling Commission, 2019). An international survey found that 46.2% of video game players purchased loot boxes (Macey & Hamari, 2019), and a recent study suggests that 62% of the most popular games in Australia contain loot boxes (Rockloff et al., 2020). However, little is known about loot box prevalence and accessibility to children outside of ‘Western’ countries. The People's Republic of China (PRC)’¹ video game market is estimated to be worth seven times more than the UK's (Newzoo, 2020). The PRC has furthermore taken a unique regulatory approach to loot boxes compared to Western countries.

Loot boxes are a new consumer product that poses a challenge to regulators, as they must determine the most effective behavioral policy approach with only limited evidence to guide them. As with many other aspects of emerging technologies used by consumers, evidence of the potentially harmful effects of loot boxes on consumers will become available only gradually, so it is difficult to establish an effective balance between maintaining consumer freedom and protecting vulnerable consumers from potential harm. Another example of a consumer product posing such a challenge to regulators is social media, for which debates over evidence of potential harm (Twenge et al., 2020), or lack thereof (Orben & Przybylski, 2019), will have a significant impact on consumers. However, complete regulatory inaction due only to the lack of an established body of evidence may be inappropriate and unjustifiable, as it fails to ensure adequate consumer protection: UK policymakers have argued that the precautionary principle of public health, which states that ‘the lack of scientific certainty cannot justify [regulatory] inactivity in cases of high risk’ (Girela, 2006), applies to loot box regulation, especially in relation to restricting children's exposure and ability to purchase (Digital, Culture, Media and Sport Committee of the House of Commons (UK), 2019).

Any activity that balances enjoyment for some and potential harms for others raises issues for regulators (Nuffield Council on Bioethics, 2007). One approach is to ban the activity, as done by Belgium in relation to loot boxes (Belgische Kansspelcommissie [Belgian Gaming Commission], 2018; Xiao, 2020d). Imposing expenditure limits is another, less restrictive approach (Drummond et al., 2019; King & Delfabbro, 2019a; 문화체육관광부 [Ministry of Culture, Sports and Tourism (South Korea)], 2019; 国家新闻出版署 [National Press and Publication Administration (PRC)], 2019; Xiao, 2020b, 2020c, 2020d). Simply monitoring the situation is an alternative, nonrestrictive approach, as the UK is currently doing for loot boxes. A middle-ground approach is to not restrict the sale or use of the product in any way but to provide relevant information and help consumers make more informed purchasing decisions by requiring companies to disclose relevant product characteristics. A disclosure-based regulatory approach is, for example, used on alcohol labels that inform drinkers about alcohol content (Blackwell et al., 2018). This approach has also been recommended as an intervention for gambling-related harms (Eggert, 2004) and for the harmful use of loot boxes (King & Delfabbro, 2018, 2019b; Moshirnia, 2018; Castillo, 2019; Liu, 2019; McCaffrey, 2019; Xiao & Henderson, 2019; Xiao, 2020d). However, the potential effectiveness of this disclosure-based public policy intervention has not been assessed in the loot box context yet (McCaffrey, 2020).

Since May 2017, the PRC has been the only country to legally require video game companies to disclose the probabilities of obtaining randomized rewards from loot boxes (文化部 [Ministry of Culture] (PRC), 2016; Xiao & Henderson, 2019). This is relevant to other countries, because leading video game companies also predominantly favor the disclosure of loot box probabilities as a self-regulatory approach (NHN Japan 株式会社 [NHN Japan Corporation] et al., 2012; 한국게임산업협회 [Korea Association of Game Industry; K-GAMES], 2018; Entertainment Software Association (ESA), 2019). Indeed, Apple (2020) has required loot box probability disclosures for all games available on the iOS platform worldwide since December 2017 but does not appear to actively enforce this requirement (Kuchera, 2017; Xiao, 2020d).

Notably, PRC loot box disclosure policies allow significant discretion to game companies as to how to implement these disclosures: probabilities can be disclosed either in-game on the loot box purchase page or on the game's official website, and there are no requirements as to the size or type of symbol or wording used to signify loot box disclosures in-game; the placement of website disclosures; or how probabilities should be numerically or graphically communicated to consumers (文化部 [Ministry of Culture] (PRC), 2016; Xiao, 2020d).

Prominent disclosures, which are designed and implemented uniformly across video games, would likely be the most helpful to consumers, similar to standardized front-of-pack food nutrition labels that are used in several countries (Dubois et al., 2020). In the language of behavioral science, such clear and coordinated action could be termed as a ‘nudge,’ which is designed to help consumers make better decisions (Thaler & Sunstein, 2008), or more specifically as an ‘educative nudge’ (Sunstein, 2015). However, companies may also take various actions that inhibit the consumers’ abilities to make informed decisions: for example, by communicating in excessive and difficult to understand language (Mills, 2020) or by hiding important information within long legal documents, such as the terms and conditions (Bar-Gill, 2012). Company actions that inhibit informed consumer choices have been contrastingly termed as ‘sludge’ in the behavioral science literature (Thaler, 2018; Sunstein, 2020) or ‘dark nudges’ in the gambling and alcohol literature (Newall, 2019; Pennay et al., 2020; Petticrew et al., 2020).

Research in other domains suggests that companies may not disclose information in an optimal manner to consumers; instead, companies disclose information in a way that is more consistent with sludge or dark nudges, rather than nudges. Tobacco companies have been accused of blocking the worldwide diffusion of prominent health warnings (Hiilamo et al., 2012). UK alcohol companies have not always met their voluntary pledges around health warnings (Petticrew et al., 2016; Alcohol Health Alliance UK, 2020). Gambling companies’ disclosures about the odds of winning are provided in an arguably suboptimal risk communication format (Newall et al., 2020). Sludge in the loot box probability disclosure context could be implemented in various ways through the use of ‘obscurant friction’, which obscures the consumers’ understanding, rather than promotes it (Mills, 2020). Even though PRC law mandates probability disclosures (which are intended to promote the consumers’ understanding and help them make more informed loot box purchasing decisions), video game companies could endeavor to minimize the positive consumer protection effects of probability disclosures by making them difficult to access and understand. For example, companies could hide the probability disclosure at an obscure location that players cannot easily access, or use a format that is difficult to understand. However, tobacco, alcohol, and gambling are all established consumer industries that operate very differently from the much younger and rapidly developing video gaming industry: video game companies may instead prefer to be more socially responsible and develop trust with their player–consumers by interpreting loot box disclosure requirements in a way that is most helpful to consumers, but it is not known whether companies would do so.

The probabilities of obtaining randomized rewards through loot boxes may change during gameplay due to the presence of ‘pity-timer’ mechanics, which increase the player's probabilities of receiving rarer and more valuable rewards as more loot boxes are purchased (Xiao & Henderson, 2019; Xiao, 2020d). The ‘pity-timer’ terminology was coined by the player community and is widely accepted as the appropriate term to refer to such mechanics, regardless of whether the mechanic in fact operates as a ‘timer.’ Two possible implementations can be identified:

1. 1. Repeat purchase pity-timers: if the player does not obtain a higher rarity item after buying and opening 9 loot boxes, the 10th loot box that the player buys and opens is guaranteed to provide a higher rarity item (as implemented in Game 19: 明日方舟 [Arknights]).

2. 2. Incentivized bulk purchase pity-timers: if the player chooses the option to bulk purchase 10 loot boxes simultaneously, rather than choosing to purchase one loot box at a time, the player is guaranteed to obtain a higher rarity item from one of the 10 loot boxes bulk purchased (as implemented in Game 93: 龙珠激斗).

Pity-timers plausibly make the loot box purchase decision harder to understand, therefore potentially reducing the educative effectiveness of any probability disclosures (Weiss-Cohen et al., 2018). Pity-timers have been secretly implemented in at least one Western game – Hearthstone: initially, the company did not inform players that the mechanic has been implemented; players discovered it themselves through analyzing large samples of their own purchase history (Xiao & Henderson, 2019). Furthermore, a prior review of relevant industry patents has revealed that video game companies may implement detrimental pity-timers that reduce the consumer's probabilities of obtaining valuable rewards once the game has determined, using data of consumers’ previous purchasing behavior, that that particular consumer is likely to expend a substantial amount of money in trying to obtain the more valuable, higher rarity rewards from loot boxes (King et al., 2019). It is not known whether any games have yet implemented such a detrimental pity-timer.

Pity-timers may be relevant to loot box engagement by disordered gamblers because of conceptual similarities between the mechanic and the ‘gambler's fallacy’ – the mistaken belief that a period of losses is likely to be followed by a win (Clotfelter & Cook, 1993; Raylu & Oei, 2004), and ‘loss chasing’ – the pressure to continue expending resources to recoup losses (Zhang & Clark, 2020). In the gambling context, the gambler's fallacy is factually erroneous, because winning probabilities do not change. However, in the video gaming context, the gambler's fallacy is effectively being reinforced when pity-timers are implemented because ‘winning’ probabilities improve, and a period of losses will in fact be guaranteed to be followed by a ‘win.’ A video game player may conceivably become convinced that the gambler's fallacy is in fact true after playing a game whose loot boxes are augmented with pity-timers, because they either did not know that pity-timers were implemented at all or did not fully understand how pity-timers operated. However, if that player then transitions into playing a game whose loot boxes are not augmented with pity-timers, or transitions into gambling, this player may potentially have a stronger erroneous belief in the gambler's fallacy, due to their exposure to pity-timers. Presently, there is a lack of empirical research on the effects of pity-timer mechanics on player cognition. Indeed, the prevalence of pity-timers in video games has never been assessed. Generally, in-depth engagement with the games being studied would be required in order to identify pity-timers, as their implementation is often undisclosed. However, because pity-timers affect loot box probabilities, PRC law effectively requires video game companies to also disclose the presence of pity-timers and their effects on loot box probabilities. A failure to make this disclosure would be a failure to comply with PRC law.

The PRC's video game market is structurally different from Western countries. Steam and Android are two popular platforms in the West (Zendle et al., 2020a). Steam is also popular in the PRC but operates in a legal gray area without official sanction (Messner, 2019), meaning that games available on the platform are not required to comply with PRC law. Android is similarly popular in the PRC, but unlike in the West where the Google Play Store is the single dominant Android app store, multiple individual hardware providers’ Android app stores operate competitively and concurrently in the PRC (AppInChina, 2020), meaning that any of these stores’ country data is representative only of a specific consumer group. The Apple App Store (iPhone) is therefore the main overlapping platform across Western and PRC markets.

The present study, therefore, set out to provide a non-Western, Far East Asian perspective on loot boxes, which can be compared with Zendle et al. (2020a)'s recent UK perspective to provide cross-cultural insights on loot box regulation. A study of the following features of the 100 highest-grossing PRC iPhone games was conducted:

1. 1. The prevalence of loot boxes and age ratings of the 100 highest-grossing mobile games available on the PRC Apple App Store.

2. 2. The proportion of games containing loot boxes for which probability disclosures could be found.

3. 3. Whether these disclosures could be found in-game, on the game's official website, or at both locations.

4. 4. Features of these disclosures and the steps required to access them.

5. 5. Whether games containing loot boxes disclosed pity-timer mechanics.

Method

The 100 highest-grossing iPhone games on the Apple App Store in the PRC were selected from App Annie (2020), an authoritative independent analytics company, as of June 2, 2020. Of the initial 100 games listed, two games were duplicates of a higher-grossing game that was already on the list, and one game was removed from the Apple App Store before data extraction. These three games were excluded from analysis and replaced with the next highest-ranked games on the highest-grossing list. Additionally, the presence of loot boxes could not be determined using the below-described methodology for two games, which were replaced by the next highest-grossing games. Therefore, the sample consists of 100 games taken from the top 105 highest-grossing games.

The games studied here included up to 433 different types of loot boxes in a single game (御龙在天手游运营团队 [Yu Long Zai Tian Mobile Game Operations Team], 2020), and disclosure formats for different loot boxes may conceivably vary even within a single game. It was therefore beyond the scope of this study to record relevant features for all loot box types across these 100 games, as some loot boxes may take hundreds of hours to become available. This study, therefore, set out to record loot box prevalence across these 100 games and to record disclosure features of one loot box found within each game.

Raw data, a full library of screenshots, and archived links to website disclosures are available via https://osf.io/h7uxz/.

Variables were coded as follows.

Presence of loot boxes

iPhone games are not required to disclose the presence of loot boxes on the Apple App Store. This variable was therefore determined by first playing the game for up to 40 min. It was infeasible to determine loot box presence solely by gameplay for every game because, for example, in Game 4 (梦幻西游), loot boxes were not available for purchase until the player's character reaches level 40, which would likely require 30–50 gameplay hours. If no loot boxes were encountered during 40 min of gameplay, researchers then based their assessments on either videos of players engaging with loot boxes on Bilibili (bilibili.com), the leading video game video sharing website in the PRC, or screenshots of loot boxes found through Baidu (baidu.com), the leading PRC search engine, which were found within 2 h of internet browsing. In total, loot box presence was determined via gameplay for 75 games and via video streaming or screenshots for 25 games.

This combined approach follows Zendle et al. (2020a); however, contrary to Zendle et al., loot box presence was first assessed through gameplay, if possible, and then through videos or screenshots. This is because the loot boxes that would be encountered during the first 40 min of gameplay would probably be the loot boxes most likely to be encountered by new players, while loot boxes documented in videos or screenshots may be of more sought-after loot boxes containing better rewards that are only encountered later in the game by more experienced players. However, it should be noted that loot boxes encountered through non-gameplay methods, for example, those identified by Zendle et al. (2020a), may still hold special significance to players (e.g., they may provide the best rewards and therefore be more popular with more experienced and invested players who may be more willing to spend money buying loot boxes).

Apple age rating

This was recorded from the game's Apple App Store page.

The location(s) of disclosure

Games were coded as having disclosed probabilities in-game if such a disclosure was identified by checking the loot box's purchase page for any interactable elements, such as a question mark button (‘(?)’) or a ‘概率公示 [probability disclosure]’ hyperlink, and then for loot boxes coded via gameplay, by an exhaustive search of other game features, such as customer service bots.

Games were coded as having disclosed probabilities on the game's official website if such a disclosure was identified through online searches on Baidu and then by a manual search of the website. The search terms used were the game's Chinese name; ‘概率’ [probabilities]; and ‘公示’ [publication].

Games were credited with disclosing at a given location if one loot box could be found disclosing in this way: for example, Game 40 (忍者必须死 3) was coded as disclosing in-game, because one of two loot boxes encountered during 40 min of gameplay disclosed probabilities, as was Game 30 (梦幻西游三维版), where one of four loot boxes observed via video in 2 h of internet browsing disclosed in-game. The ‘Discussion’ section addresses issues with potential within-game variation in loot box probability disclosures.

How to access disclosure(s)?

Both in-game and website disclosures could differ in the steps required for the player to view them. For example, in-game disclosures might be clearly visible, appearing automatically when the player accesses the loot box purchase page (see Figure 1). Alternatively, in-game disclosures might be hard to access, appearing only after chatting with an in-game customer support bot (see Figure 2). Similarly, website disclosures might be easily accessed via a link placed relatively prominently on a game's homepage (see Figure 3). Alternatively, a website disclosure might appear only in a chronological list of news posts (see Figure 4), which would gradually lose prominence over time, or even be unlinked from other pages on the website, and might only be accessible via a search engine query. Unique categories of each disclosure format were developed by L.Y.X. during data extraction. In total, six categories were found for in-game disclosures, and five categories for website disclosures. These categories and their relative proportions are displayed in Tables 4 and 5 in the ‘Results’ section.

Figure 1. Game 80's (仙境传说 RO：守护永恒的爱 [Ragnarok M: Eternal Love]) in-game probability disclosure was automatically displayed on the loot box purchase page (on the right side, annotated) without requiring any additional input from the player (9.6% of in-game disclosures). © 2003–2020 心动网络 [X.D. Network Inc.].

Figure 2. Game 104's (野蛮时代 [Brutal Age: Horde Invasion]) in-game probability disclosure can be accessed after interacting with the in-game customer support system and chatting with a bot (1.9% of in-game disclosures). This series of three screenshots shows the required steps. Notably, although the game is available in simplified Chinese, the interaction with the support bot could be done only in English. © 2020 Chengdu Nibirutech Co., Ltd, a.k.a. tap4fun.

Figure 3. Game 12's (神武 4) website probability disclosure was linked directly from the homepage (9.1% of website disclosures) through a hyperlink with the text ‘概率公示 [probability disclosure]’ at the bottom of the page (annotated). © 2020 多益网络有限公司.

Figure 4. Game 50's (天龙八部手游) website probability disclosures (annotated) were published under the ‘新闻 [news]’ or ‘公告 [notice]’ tab, which were then chronologically listed alongside other posts (78.8% of website disclosures). The disclosures for various loot boxes were presented in separate posts, making it more difficult for the player to identify the disclosure of a particular loot box. © 1998–2020 Tencent.

Was a pity-timer mechanic disclosed?

Games were coded as having disclosed the presence of a pity-timer mechanic if the loot box's purchase page, or either the in-game or website disclosure, stated that the probabilities of obtaining any particular reward would increase as more purchases were made.

Inter-rater reliability analysis

Fifteen games were selected at random to be dual-coded by L.Y.X. and Y.Y. (15% of the sample) to test the inter-rater reliability of the coding, which is summarized in Table 1. L.Y.X. and Y.Y. first coded the Presence of loot boxes and Apple age rating. There was a single disagreement for the Presence of loot boxes (93.3% agreement, Cohen's κ = 0.87). Discussions revealed this was due to a false negative, and the coding was adjusted to reflect this. Then, L.Y.X. and Y.Y. assessed the other variables based on screenshots provided by the other author to eliminate potential disagreements caused by coders finding different loot boxes. There was a single disagreement each on disclosure location(s) (93.3%; Cohen's κ = 0.91) and on how to access the in-game disclosure (93.3%; Cohen's κ = 0.93). Discussions revealed that these two points of disagreement were due to a loot box that only disclosed the probabilities of one partial aspect of its randomization in-game. The final coding was adjusted to credit this game with disclosing its probabilities in-game. The coders were in 100% agreement on the other three variables.

Table 1. Inter-rater reliability.

Results

Of the 100 highest-grossing games currently available on the PRC Apple App Store, 91 contained loot boxes (91%), which a binomial test revealed was significantly higher (p < 0.001) than the 59 games previously found in the UK Apple App Store (Zendle et al., 2020a).

Table 2 shows the Apple age ratings of games. Overall, 62.6% of games (57 of 91) containing loot boxes were deemed suitable for children aged 12+, which a binomial test revealed was significantly lower (p < 0.001) than the 94.9% (56 of 59) found in the UK (Zendle et al., 2020a). However, this difference is largely due to a significantly greater percentage of the 100 highest-grossing games in the UK being deemed suitable for play by children when compared to such games in the PRC: 95% of such UK games were deemed suitable for children aged 12+, compared to only 63% of such PRC games. Of the 63 PRC games deemed suitable for children aged 12+, 57 (90.5%) contained loot boxes, which a binomial test revealed was significantly higher (p < 0.001) than the 56 of 95 games (58.9%) found in the UK (Zendle et al., 2020a).

Table 2. Apple App Store age rating of games containing loot boxes (cumulative).

Locations of where disclosures were found are displayed in Table 3. Disclosures were found for 87 of 91 games containing loot boxes (95.6%); disclosures were not found for 4 games (4.4%). Found disclosures were most often on the official website only (38.5%); 23.1% of found disclosures were in-game only. Disclosures were found at both locations for 34.1% of games.

Table 3. Locations of found disclosures.

Note: Total sample size = 91 games.

The six unique categories of in-game disclosures are described in Table 4. Most in-game disclosures showed probabilities immediately after a small generic symbol was tapped (29 games; 55.8%). However, these generic symbols themselves varied between games, from the relatively informative ‘奖励预览 [rewards preview],’ to question or exclamation marks. Eight games’ in-game disclosures (15.4%) used similar symbols but required at least one additional step, such as following a link to the game's website. In comparison, six games (11.5%) used a similar button that in some way referenced ‘概率 [probabilities]’ and is therefore a more prominent disclosure format. Five games (9.6%) displayed probability disclosures automatically, which appears to be the most prominent disclosure format found in this study, and an example is shown in Figure 1. The two aforementioned subcategories, which overall constitute 21.2% of in-game disclosures (11 games), could be considered reasonably prominent, because they were either accessible by searching for the Chinese word ‘概率 [probabilities]’ on the in-game loot box purchase page or automatically displayed. Three games’ in-game disclosures (5.8%) used arbitrary graphic elements, which themselves varied in each game. Finally, one game (1.9%) disclosed probabilities in-game only after interacting with a customer support bot, as shown in Figure 2. Example screenshots of the other in-game disclosure categories are shown in Supplementary Figures S1–S8, and screenshots for all games containing loot boxes are available via https://osf.io/h7uxz/.

Table 4. Subcategories of found in-game disclosures.

Note: Total sample size = 52 games.

The five unique forms of website disclosures are shown in Table 5. Most website disclosures (52 games; 78.8%) were linked via a chronological list of updates, which would gradually lose prominence over time: an example is shown in Figure 4. Two games (3.0%) pinned their website disclosures to the top of a chronological list of updates, which made them more prominent, but these two games nonetheless required the player to navigate to a specific section of the website that was not the homepage and was not clearly related to probability disclosure in order to access the disclosures. In comparison, six games’ website disclosures (9.1%) were linked directly from the homepage (Figure 3), and one game (1.5%) created a unique ‘概率 [probabilities]’ tab that was displayed on the homepage. Therefore, overall, 10.6% of website disclosures (seven games) could be considered reasonably prominent, because they were always accessible by using the internet browser's find command to search for the Chinese word ‘概率 [probabilities]’ on the official website's homepage. Some website disclosures published under the news and notice tabs would also be displayed on the homepage for the first few days after they were published but would eventually be removed from the homepage when newer posts are published: such disclosures cannot be considered reasonably prominent, because their accessibility from the homepage is transient. Five games’ website disclosures (7.6%) could not be accessed from exploring the website and could only be found via search engine queries. Example screenshots of the other website disclosure categories are shown in Supplementary Figures S9 and S10, and archived links to all website disclosures are available via https://osf.io/h7uxz/.

Table 5. Subcategories of found website disclosures.

Note: Total sample size = 66 games.

Of the 91 games containing loot boxes, 60 (65.9%) disclosed the implementation of at least one pity-timer mechanic in relation to one of its loot boxes. All pity-timers disclosed, except one, were ‘beneficial’ for the player in that the probabilities for obtaining higher rarity rewards would increase. Only one game (1.1%) was identified as having disclosed the implementation of a detrimental pity-timer that would reduce the player's probabilities of obtaining higher rarity rewards: Game 75 (航海王：燃烧意志) disclosed that each loot box has a 0.5% of containing an SSR (Super Super Rare) reward (《航海王：燃烧意志》官方网站 [Official Website of One Piece: Ran Shao Yi Zhi], 2018). However, Game 75 then qualified this by stating in its probability disclosure that if the player bulk purchases 10 loot boxes (each of which contains one reward) simultaneously, the player can only obtain up to two SSR rewards. However, although highly unlikely, mathematically, it should be possible to obtain more than two SSR rewards in 10 loot boxes, if the probability of obtaining SSR rewards remained constant at 0.5% for each individual opening as disclosed. Therefore, it can be surmised that at least one game sought to reduce the player's probabilities of winning higher rarity rewards when the player bulk purchases loot boxes by implementing a detrimental pity-timer.

Discussion

Of the 100 highest-grossing iPhone games in the PRC, 91% contained loot boxes. This is significantly higher than the 59% found by a recent and comparable UK study (Zendle et al., 2020a) and the 62% found by an Australian study (Rockloff et al., 2020). The much higher concentration of games containing loot boxes in the PRC reflects that the PRC and the Western (assuming that the UK and Australia are representative) video game markets are vastly different both in terms of game content and of monetization, and that the prevalence of loot boxes in different jurisdictions and cultures may vary significantly. Further academic research into loot boxes and video gaming in general should be conscious of such national and cultural differences in order to avoid inappropriate overgeneralization (Henrich et al., 2010).

Of the 63 games deemed suitable for children aged 12+, 57 (90.5%) contained loot boxes. This was significantly higher than the 58.9% (56 of 95 games) found by Zendle et al.'s (2020a) recent and comparable UK study mentioned above. A much higher proportion of the 100 highest-grossing iPhone games in the PRC were rated 17+ (37%) than in the UK (5%) (Zendle et al., 2020a). However, the inclusion of loot boxes does not appear to have been a relevant factor that increased the age ratings of iPhone games from 12+ to 17+ in either country. Loot box prevalence in PRC games deemed suitable for children aged 12+ was 90.5%, while the overall loot box prevalence for all PRC games was 91%. Similarly, loot box prevalence in UK games deemed suitable for children aged 12+ was 58.9%, while the overall loot box prevalence for all PRC games was 59% (Zendle et al., 2020a). Therefore, the seemingly divergent approach to game rating, specifically, the tendency to give games higher age ratings in the PRC, was likely due to other reasons, such as different cultural views on what degree of certain game content (e.g., violence, gore, and sexuality) is appropriate for children (人民网 [People's Daily Online], 2019).

Previous research has suggested that young people purchasing loot boxes may be a ‘high-risk’ group for experiencing gambling problems (Wardle & Zendle, 2020), and that the relationship between loot box spending and disordered gambling observed in adolescents was stronger than that which have been observed in adult samples (Zendle et al., 2019b). Children are recognized by consumer protection measures as being more susceptible to marketing and less equipped to make informed purchasing decisions than adults (Advertising Standards Authority, 2018). Policymakers have recommended restricting children's access to loot boxes by banning loot boxes from games deemed suitable for children (Digital, Culture, Media and Sport Committee of the House of Commons (UK), 2019; Parliament of the Commonwealth of Australia House of Representatives Standing Committee on Social Policy and Legal Affairs, 2020; Select Committee on the Social and Economic Impact of the Gambling Industry of the House of Lords (UK), 2020). However, video game companies in the PRC have continued to widely implement loot boxes in games deemed suitable for children. The results of this study suggest that, despite potential for even greater harm, children are not being especially protected from potential loot box harms when compared to adults. Indeed, children are just as likely to be exposed to the opportunity to purchase loot boxes as adults.

As to the relationship between age ratings and probability disclosures, games deemed suitable for younger children did not disclose probabilities in more prominent formats. Video game companies should particularly consider implementing easy-to-understand disclosures, complemented with graphic visual aids, in games deemed suitable for younger children, and to encourage and facilitate parents’ and other adults’ access to probability disclosures to minimize potential harms. The PRC has imposed restrictions on children's gameplay time and spending on video games (Xiao, 2020c, 2020d) with the aim of shielding them from potential harms and is thus treating children differently from adults by providing additional protection; however, in relation to probability disclosures, the PRC did not require games deemed suitable for children to make more prominent disclosures than games deemed suitable only for adults. PRC law may provide additional protection by either restricting children from purchasing loot boxes (this is possible because PRC video game accounts are legally required to be age-verified using real-life identification (国家新闻出版署 [National Press and Publication Administration (PRC)], 2019; Xiao, 2020c)), or requiring even more prominent probability disclosures in games deemed suitable for children.

PRC law specified that probability disclosures made either in-game on the loot box's purchase page or on the game's official website are legally compliant (文化部 [Ministry of Culture] (PRC), 2016). Of the 91 games containing loot boxes, probability disclosures were found both in-game and on the game's official website for 31 games (34.1%), while disclosures were found only at one of either location for 56 games (61.5%). Disclosure statements could not be found for four games (4.4%). This study expended considerable effort when attempting to find probability disclosures: most players would also likely not be able to find the probability disclosures of these four games. Indeed, these four games can reasonably be suspected as having failed to disclose probabilities. The results indicate that a majority of games failed to maximize the potential consumer protection effects of the law by failing to provide consumers with two alternative channels of obtaining the relevant information.

Of the 91 games containing loot boxes, only five games (5.5%) disclosed loot box probabilities in-game by displaying them automatically on the in-game loot box purchase page without requiring any additional input from the player, which is the most visually prominent (i.e., most visibly displayed) and most easily accessible disclosure format found in this study across all disclosure formats at both possible locations. As detailed in the ‘Results’ section, a few other disclosure formats at either location should also be considered reasonably prominent, because the player was always able to find the disclosures by searching for the Chinese word ‘概率 [probabilities]’ either in-game on the loot box's purchase page (21.2% of in-game disclosures) or on official website's homepage (10.6% of website disclosures). In total, 17 (18.7%) of the 91 games containing loot boxes made reasonably prominent disclosures at either location. However, only one game (1.1%) disclosed at both locations using a reasonably prominent disclosure format.

Overall, 74 (81.3%) of the 91 games containing loot boxes failed to make reasonably prominent disclosures at either location. Accordingly, most probability disclosures were made suboptimally. Such suboptimal disclosures are sludge, because they fail to provide consumers with the probabilities in a timely, transparent, and easily accessible manner (Mills, 2020). For example, one game required the player to tap on multiple hyperlinks, visit multiple pages, and scroll extensively before making the disclosure viewable (Supplementary Figure S5). Another game's disclosure text formatting was especially difficult to read, as it did not contain line breaks where appropriate (Figure 2). A third game's disclosure was displayed using images, rather than typed text, which rendered the disclosure not searchable using the internet browser's find command (《权力的游戏 凛冬将至》官网运营团队 [Game of Thrones: Winter Is Coming Official Website Operations Team], 2019). A fourth game's probability disclosures for different types of loot boxes contained in the game were displayed on different webpages, rather than compiled together on one single webpage (Figure 4). The potential consumer protection benefits of the PRC law can be improved by requiring disclosure at both locations in specific formats deemed to be reasonably prominent, rather than to allow video game companies discretion as to which disclosure format to choose.

For in-game disclosures, this study's proposed uniform disclosure format is to always automatically display a concise (possibly abbreviated) and simple to understand disclosure on the loot box's purchase page, much like the standardized nutrition labels on food (Dubois et al., 2020). Additionally, a visually prominent, industry-standard button marked with ‘detailed probability disclosures,’ which when pressed would redirect the player to a different screen with an unabbreviated, detailed breakdown of probabilities, should also be required. For website disclosures, the proposed uniform disclosure format is to ensure that there is always a hyperlink to the probability disclosure page on every single webpage of the official website, meaning that the player will be able to access the disclosure if they use their internet browser's find command to search for ‘probabilities’ or ‘probability disclosure’ on any webpage of the website. Additionally, a visually prominent, industry-standard hyperlinked button marked with ‘probability disclosures,’ which would redirect the player to the probability disclosure webpage, should be required to be implemented in the top-right hand corner of all official websites of games containing loot boxes. On the probability disclosure webpage, a concise (possibly abbreviated) and simple to understand disclosure should be provided at the top, which is then followed by an unabbreviated, detailed breakdown of probabilities.

PRC law allows complete discretion to video game companies as to how the published disclosures should be communicated to the consumer. ‘Probabilities’ presented as a percentage (1%); a fraction (1/100); a proportion (1 in 100); a decimal (0.01); odds (99-to-1); and in other manners would all technically be compliant with the law. Nearly all probability disclosures identified by this study were communicated to players as percentages. However, a handful of individual games disclosed probabilities for certain loot box types they contained as fractions or decimals but disclosed probabilities for other loot box types contained in the same game as percentages. Research in risk communication in other domains has suggested that graphical representations of probabilities, in addition to numerical representations, may be more effective at communicating information to consumers than the formats observed in this study (Garcia-Retamero & Cokely, 2017).

From a legal perspective, the disclosure rate of 95.6% must not be perceived as the compliance rate, which is lower because some disclosures were evidently inaccurate and therefore non-compliant with the law (文化部 [Ministry of Culture] (PRC), 2016). Examples include probabilities summing to less than 100% (《新斗罗大陆》手游运营团队 [New Do Luo Da Lu Mobile Game Operations Team], 2018) and probabilities summing to more than 100% (御龙在天手游运营团队 [Yu Long Zai Tian Mobile Game Operations Team], 2020). Probabilities summing to less than 100% could only have been accurate if the loot box potentially provides no rewards, but in that case, the probability of that occurring should also have been listed. Probabilities summing to more than 100% cannot have been true if each loot box provides exactly one reward; if each loot box potentially provides multiple rewards on the list, thus causing the probabilities to overlap, this should have been disclosed (Supplementary Figures S13 and S14).

From a policy perspective, the results of this study suggest that neither the PRC's regulations nor the Apple Store's self-regulatory policies (by which these games were also governed) were sufficient to encourage all of these games to display loot box probability information prominently and uniformly, which would seem to be the most beneficial for consumers. Any stakeholders, including industry bodies (NHN Japan 株式会社 [NHN Japan Corporation] et al., 2012; 한국게임산업협회 [Korea Association of Game Industry; K-GAMES], 2018; ESA, 2019), looking to enhance consumer protection around loot boxes via probability disclosures, should also delineate requirements as to the prominent and uniform placement of these disclosures (Xiao, 2020d), as the USA has done for alcohol warnings since 1989 (Hilton, 1993).

Of games containing loot boxes, 65.9% disclosed at least one pity-timer mechanic. All, except one, increased the player's probabilities of receiving rarer and more valuable rewards when they make repeat or bulk purchases. Although pity-timer disclosures were prevalent, more games may have secretly implemented pity-timers without informing players, as is known to be done in at least one Western game – Hearthstone (Xiao & Henderson, 2019), which would have rendered the games’ probability disclosures inaccurate and non-compliant with PRC law. Many games that disclosed both probabilities and the presence of pity-timers failed to disclose exactly how much the ‘winning’ probabilities would increase with repeat or bulk purchases, potentially limiting the disclosures’ usefulness to consumers, and technically rendering the disclosures inaccurate and non-compliant with PRC law.

One game was identified as having implemented and disclosed a detrimental pity-timer mechanic that reduced the player's probabilities of obtaining higher rarity rewards. Before this study, it was not known if any such mechanic has been implemented by video game companies, despite patents for such implementations having previously been identified (King et al., 2019). PRC law has effectively forced the disclosure of this detrimental pity-timer. The prevalence of detrimental pity-timers should be studied further. This study did not record the language used to present pity-timer mechanics to players and did not assess how easily understandable pity-timer mechanics were to players. The terminology of ‘pity-timer’ is an invention of the player community and has been adopted by this study. However, this term itself may be misleading, because newer players unfamiliar with such mechanics may be misled into thinking that the spending of ‘time’ is required to receive better rewards, when in fact the spending of more money is the only contributing factor. How players interact with pity-timers should be studied further. Other designs and implementations of similar, but not identical, pity-timer mechanics, which encourage players to make additional purchases in exchange for higher probabilities or guaranteed opportunities of obtaining higher rarity items, have been identified (Ballou et al., 2020). Other novel aspects and submechanics of loot boxes should also be identified and studied further (King et al., 2019).

This study was limited to recording features of only one accessible or popular loot box in each of the 100 highest-grossing games on a single platform at a single point in time. Steam is a popular platform in the West: the most popular games on Steam have also been shown to often contain loot boxes (Zendle et al., 2020a). However, the use of Steam is not officially sanctioned in the PRC, meaning that any Steam games played by PRC residents are unlikely to comply with PRC law. An improved study of PRC loot boxes could be expanded to the Android app stores, but it would be faced with logistical problems around the demographical fragmentation of the Android stores in the PRC: for example, Huawei phones are pre-installed with only the Huawei Android app store (the Huawei AppGallery) and not installed with the Google Play Store. It is also possible that some disclosures may be accessible through other methods, or that coders failed to find certain disclosures, which may be most relevant to website disclosures because of the great number of individual webpages and wealth of information that can be put on a website. However, if a disclosure was not accessible through a reasonable expenditure of coder effort, then it would seem unlikely to be found by most players.

This study aimed only to record features of one accessible or popular loot box within each game, meaning that it was unable to assess the possibility of within-game variation in loot box probability disclosures. This was not feasible for this study, because the games studied included up to 433 different types of loot boxes in a single game (御龙在天手游运营团队 [Yu Long Zai Tian Mobile Game Operations Team], 2020), and it may take thousands of hours of gameplay to systematically encounter all of the loot boxes within any one game. Given this limitation, leeway was given to games in instances where multiple loot boxes with diverging in-game disclosure features were observed by crediting these games as having disclosed probabilities.

This study could not determine whether these disclosures were made in response to PRC law (文化部 [Ministry of Culture] (PRC), 2016) or Apple's (2020) self-regulation, because the disclosures were assessed only at a single point in time when both were simultaneously in force. Longitudinal or cross-cultural studies may be best placed to answer these questions. However, the publication timestamps of some website disclosures were after the PRC law was announced, but prior to Apple's self-regulatory requirement being imposed (e.g., 完美世界 [Perfect World], 2017; 腾讯游戏 [Tencent Games], 2017), and the language of some disclosures stated that they were being made in response to ‘relevant [PRC] law’ (e.g., 吉比特网络 [G-BITS] and 雷霆游戏 [Leiting Games], 2017; 完美世界 [Perfect World], 2017; 《新斗罗大陆》手游运营团队 [New Do Luo Da Lu Mobile Game Operations Team], 2018). This study also did not explore potential associations between game popularity and disclosure policies, or whether each game company's interpretation is uniform across all of its games.

This study also could not determine the accuracy of probability disclosures. A study on disclosure accuracy would require either access to a game's underlying code or data through collaboration with game companies (Johannes et al., 2020; King et al., 2021), or the bulk purchase of large numbers of loot boxes. However, as detailed above, some disclosures with probabilities not summing to 100% were clearly inaccurate. Although this study found that a majority of games containing loot boxes disclosed at least one pity-timer mechanic, this study did not examine whether this novel mechanic is potentially especially appealing to disordered gamblers (Zendle, 2020), and how players themselves feel about the appropriateness and fairness of pity-timers being implemented (Petrovskaya & Zendle, 2021). Finally, this study could not determine the effectiveness of these various probability disclosures in shaping player behavior, a valuable research question that would require either experimental data or collaboration with the video game industry.

Conclusion

Loot boxes are a relatively new phenomenon, and their links with behavioral addiction have only recently been subject to investigation. However, the actions of some video game companies do at least seem to draw parallels with the arguably socially irresponsible corporate actions in other, more established addictive areas. For example, the alcohol (Petticrew et al., 2016), gambling (Newall et al., 2020), and tobacco industries (Hiilamo et al., 2012) have all taken various actions that likely reduce the effectiveness of their product warnings. The actions of most video game companies in this study appear to be more consistent with ideas of ‘sludge’ (Thaler, 2018; Sunstein, 2020) or ‘dark nudges’ (Newall, 2019; Petticrew et al., 2020), which inhibit optimal consumer choice, than with the traditional conceptualization of ‘nudge’ (Thaler & Sunstein, 2008), which aims to improve consumer choice. Given these findings, loot box probability disclosure regulations should require uniform and visually prominent disclosures in order to best help inform consumers.