Fake memories: A meta-analysis on the effect of fake news on the creation of false memories and false beliefs

Abstract

Fake news can affect people in negative ways. A recent line of research has demonstrated that when people are exposed to fake news they can form false memories for the events depicted in the news stories. We conducted a meta-analysis to obtain an estimate of the average rate of false memories elicited by fake news. Thirteen articles were included in the final analysis, revealing that nearly 40% and 60% of the participants reported at least one false memory and belief (respectively) after fake news exposure, while each participant remembered or believed 22% of the total number of fake news presented. Individual differences may affect the rate of false memory formation following exposure to false memories. We therefore examined moderating effects of individual difference variables assessed in the included studies. Participants with better analytical reasoning skills and a high level of interest in the news topic were least likely to report false memories for fake news, with level of interest being also a facilitating factor in remembering true news. No effect was detected for cognitive ability and objective knowledge. Our results provide insightful and practical information in the context of world-wide misinformation dissemination and its impact on people's beliefs and memories.

Fake news is omnipresent. Although misinformation (i.e., information that is erroneous or misleading; Fox 1983; Zhou and Zhang 2007) can circulate exceedingly fast due to advances in social technologies and large-scale information cascades (Vosoughi et al. 2018), the roots of fake news go back to the days before the Printing Revolution, when word-of-mouth was the primary method of news transmission (Burkhardt 2017). As Burkhard observed (2017), rumours and misleading statements have existed for centuries, and have regularly appeared in print since the emergence of newspapers. For instance, in 1835, the New York newspaper ‘The Sun’ published six pieces regarding alleged presence of life on the lunar surface, which became known as the ‘Great Moon Hoax’ (Pennycook and Rand 2021).

Fake news has been described as ‘fabricated information that mimics news media content in form but not in organizational process or intent’ (Lazer et al. 2018, p. 1094). It might significantly impact people's beliefs (e.g., Greene et al. 2021, p. 21; Murphy et al. 2021) and even memories about certain events (e.g., vaccinations and possible therapies for COVID-19 virus) (e.g., Grady et al. 2023; Murphy et al. 2019). In this meta-analysis, our interest lies in exploring the average impact of fake news on memory, specifically its potential role in generating false memories. More importantly, we assessed the effects of individual differences such as political ideology, analytical thinking, expertise) on the creation of false memories following fake news exposure.

False memories

False memories consist of remembering events that never happened or remembering them differently from how they actually occurred (Loftus 2005; Roediger and McDermott 1995). According to Mazzoni and Kirsch (2002), two interrelated but different processes play a role in the generation of false memories: belief and recollection. The concept of belief has been described as ‘the truth value attributed to the occurrence of an event’ (Scoboria et al. 2014, p. 1243). Recollection, on the other hand, has been defined as ‘the ensemble of perceptual and emotional characteristics that people may perceive as reminiscent of the original experience when events are remembered’ (Scoboria et al. 2014, p. 1251). It has now been established that belief and recollection do not always co-occur: people often believe in the occurrence of events for which they have no memory (Scoboria et al. 2004) and occasionally have memories for events that they do not believe actually occurred (i.e., nonbelieved memories; Mazzoni et al. 2010; Otgaar et al. 2014). However, most of the time, the memory for an event is accompanied by the belief that the event really happened. This is also often the case for recollection of false experiences (Mazzoni et al. 2010).

False memories arising from exposure to fake news can be categorized as suggestion-induced false memories, as they are elicited by an external ‘influence’ implemented through the fake news story (Mangiulli et al. 2022). Several paradigms have been designed to elicit false memories in laboratory settings (e.g., misinformation paradigm, memory conformity, rumour mongering, etc). Among these paradigms, the most widespread and influential method for investigating suggestion-induced false memories is the misinformation paradigm (Loftus 2005; Loftus and Klemfuss 2024). In this paradigm, participants are exposed to an initial stimulus (for example, a video of a simulated crime). Following this encoding phase, they are presented with misinformation (e.g., stating that there were tulips in the video while in fact there were roses). The presentation of this misinformation typically results in participants later misremembering the incorrect information as having been provided during the stimulus presentation (i.e. falsely remembering having seen roses instead of tulips) (Loftus 2005; Nichols and Loftus 2019; Wylie et al. 2014).

Another commonly used method to study false memory is the false memory implantation paradigm (e.g., Ceci and Huffman 1997; Loftus and Pickrell 1995; Otgaar et al. 2008, 2010). In this paradigm, participants are told that they experienced certain events in their childhood. One of these events was not experienced by the participants but was in fact invented by the experimenters (e.g., a hot air balloon ride). Participants are then asked to report every detail they remember about the experiences, including the non-experienced event. Studies using this procedure have implemented a wide array of false events (e.g., being lost in a shopping mall, receiving a rectal enema; Loftus and Pickrell 1995; Otgaar et al. 2010) and sometimes even included doctored photographs of the false event (Wade et al. 2002). In general, this work has revealed that false suggestions can lead to a significant number of participants erroneously remembering the fabricated event (Arce et al. 2023; Scoboria et al. 2017).

Fake news and false memories

In recent years, the focus has shifted to the investigation of how fake news can foment the production of false memories and, to a minor extent, false beliefs. This work has been catalysed by a growing awareness that fake news is widely shared in daily life and could be detrimental when it concerns major events, such as the COVID-19 pandemic. As Loftus and Klemfuss noted: ‘technology is almost certainly going to exacerbate the problem of misinformation in the coming years.’ (2024, p. 4).

Research on fake news and false memory has revealed that participants can come to believe in and even remember fabricated stories when presented with fake news and pictures of world events that never happened (Calvillo et al. 2023; Frenda et al. 2013; Grady et al. 2023; Greene et al. 2021; Murphy et al. 2019). For instance, in Frenda et al.'s (2013) study, 5,269 participants were asked to declare their political orientation (‘progressive,’ ‘moderate,’ ‘conservative,’ or ‘not applicable’) before being presented with three true and one fabricated political event (e.g., Barack Obama shaking hands with the president of Iran). Each fake news story was accompanied by an altered photograph representing the event in question, whereas true stories were accompanied by an unaltered photograph. After viewing the news stories, participants were asked to provide a rating for each story by choosing one of the following alternatives: ‘I remember seeing this’, ‘I don't remember seeing it, but I remember it happening’, ‘I don't remember it’ or ‘I have a different memory of how it happened’ (the first two options were classified as recalling the event). Approximately 50% of participants mistakenly recalled the false event as having truly occurred.

Individual variables

Along with a general interest in examining the effect of fake news on false memory formation, memory scientists have also started to investigate whether certain individual variables (e.g., ideological congruency, level of interest and engagement, objective and subjective knowledge, cognitive ability, analytical reasoning, conspiracy beliefs, etc.) might moderate the fake news-false memory effect (Greene et al. 2021; Greene and Murphy 2020; Mangiulli et al. 2022; Scuotto et al. 2023). This line of research on the influence of individual variables has mainly focused on the phenomenon of false memories, with only a couple of studies also addressing their effect on false beliefs (Greene et al. 2021; Murphy et al. 2019).

Several studies have found an ideological congruency effect, showing that when news or events align with participants’ political or ideological perspectives, false memories are significantly more likely to occur than when those news items are not aligned with people's belief system (Calvillo et al. 2023; Frenda et al. 2013; Grady et al. 2023; Greene et al. 2021, 2022; Murphy et al. 2019, 2021). For instance, in the above-mentioned study by Frenda and colleagues (2013), liberals were more likely to recall the false event of George W. Bush vacationing with a baseball celebrity amid the Hurricane Katrina catastrophe than conservatives, who were more likely to erroneously recall Barack Obama shaking hands with the Iranian president. The authors proposed that the likelihood of false memory formation relied on the degree of ‘fit’ between a person's opinions about the character performing the event and their thoughts regarding the behaviour displayed during the event.

In addition, the type and quality of a person's prior exposure to a specific topic may play a significant role in determining whether that person will remember fake news stories about that subject (O'Connell and Greene 2017). Although several studies have investigated the impact of the level of interest and engagement regarding a certain topic on the ease of creating true and false memories about that specific topic, the findings are inconsistent. Some studies showed that self-reported interest increased the frequency of false memories (O'Connell and Greene 2017) or both true and false memories related to a specific topic (Calvillo et al. 2023; Greene et al. 2021; O'Connell and Greene 2017). However, the results of Greene and Murphy (2020) suggested that subjects with a higher level of engagement with the topic (COVID-19) reported more true memories but did not respond differently to fake news stories. In addition, Mangiulli et al. (2022) did not find any statistically significant relationship between interest and false memory formation.

Another individual variable that has been taken into account is the individual's knowledge on a particular topic (Greene et al. 2021; Greene and Murphy 2020; Mangiulli et al. 2022; Scuotto et al. 2021). Some research has shown that objectively-assessed knowledge was associated with fewer false memories but more true memories, suggesting that expertise provided better discrimination between true and fake news (Greene et al. 2021; Greene and Murphy 2020; Scuotto et al. 2021), while other studies did not find this (Mangiulli et al. 2022).

Moreover, a variety of cognitive variables have been implicated in susceptibility to false memories for fake news. Two primary variables are cognitive ability and analytical reasoning, assessed respectively by the Wordsum test and the Cognitive Reflection Test (CRT). Wordsum is a 10-item vocabulary subtest derived from the Wechsler Adult Intelligence Scale (WAIS) vocabulary test, which serves as a measure of cognitive ability (Thorndike and Gallup 1944; Wechsler 2008). Participants encounter target words and are tasked with selecting the most similar word from a list of five alternatives. For instance, if given the word ‘EDIBLE,’ participants might choose the closest match from options like ‘auspicious’, ‘eligible’, ‘fit to eat’, ‘sagacious’, or ‘able to speak’. The test demonstrates a robust correlation with broader assessments of general intelligence, making it a reliable tool for gauging cognitive aptitude (Meisenberg 2015). The CRT assesses analytical reasoning through verbal problems designed to elicit intuitive yet incorrect responses, contrasting with correct solutions that demand slower, more reasoned analysis (Frederick 2005). One typical item presents the following scenario: ‘A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?’ The intuitive answer, ‘10 cents’, is incorrect; the correct response is ‘5 cents’ (Frederick 2005). The CRT encompasses both numerical and non-numerical problems, such as determining the volume of dirt with dimensions of 3′ deep x 3′ wide x 3′ long, where the correct answer is ‘none’ (Frederick 2005; Thomson and Oppenheimer 2016). This test correlates with various cognitive measures, including SAT scores, delay discounting and need for cognition (Frederick 2005), as well as a range of other reasoning measures (Hoppe and Kusterer, 2011; Toplak et al. 2011), and its predictive validity persists across multiple exposures (Bialek and Pennycook 2018). Lower levels of cognitive ability (Murphy et al. 2019) and analytical reasoning (Greene et al. 2021) measured by these two instruments have been linked to an increased reporting of false memories for fake news stories. These results are consistent with studies connecting lack of critical thinking skills with the recollection of fabricated events (Bago et al. 2020; Pennycook and Rand 2019, 2020) as well as with research linking lower cognitive ability with a higher propensity to include post-event misinformation in eyewitness memory reports in both younger (Zhu et al. 2010) and older adults (Roediger and Geraci 2007).

Another element that seems to affect false memory creation as a result of exposure to fake news is conspiracy beliefs. Specifically, research by Calvillo et al. (2023) and Mangiulli and colleagues (2022) revealed that participants who were more inclined to conspiracist thinking were more likely to report false memories for fake news stories than those without such convictions. Finally, some studies have shown that states of fear, anxiety, and depression related to one's own or a loved one's contraction of COVID-19 disease can increase the number of memories for true news items and decrease recollections for fake news stories (Greene and Murphy 2020; Scuotto et al. 2021).

Theoretical frameworks

The vulnerability to forming false memories and beliefs following the presentation of fake news and the biasing effect of some of the individual variables on memory accuracy can be understood through a series of memory frameworks.

First, the nested model (Scoboria et al. 2004) stipulates that events which a person remembers or believes to have happened are usually characterised by a high level of plausibility. However, a high level of plausibility does not necessarily translate into greater recollection or belief in an event. These findings corroborate Mazzoni and Kirsch's (2002) account of the mechanisms through which people decide whether an event has happened. Specifically, the authors suggest that people take the event's plausibility into account and base their belief about its occurrence on that knowledge. For instance, people believe that they were born (high plausibility event) without remembering being born.

The source monitoring framework is another prominent model that can explain how fake news can promote the production of false memories (Johnson et al. 1993; Mitchell and Johnson 2000). According to this model, two cognitive systems function in tandem to assist people in determining the origin of their mental experiences and can help in differentiating between true and false memories. That is, people evaluate their mental experiences using either the first system, which relies on heuristic judgements to quickly determine whether memory characteristics (e.g., visual and auditory details) are indicative of a true memory, or the second system, which relies on systematic and deliberate processes to establish the plausibility of the memory contents and compare them to factual knowledge. False memories can arise because of processing by either of these systems, when people mistakenly confuse internally created mental experiences with true recollections of the past. This can occur, for instance, if individuals are exposed to false information from the media. Hence, false memories (and beliefs) for a fake news story may arise whether a news reader initially processes the story's content and believes it to be true (i.e., heuristic processing), or if they assess the story's plausibility considering their personal experiences, knowledge, and biases and consider it to be likely to be true (i.e., systematic processing; Mazzoni and Kirsch 2002). Such judgements may lead the person to use their prior knowledge and recollections of relevant news events to generate a memory of the made-up event (O'Connell and Greene 2017).

Fuzzy-trace theory (FTT; Reyna et al. 2016) offers another explanation of how false memories are formed. FTT states that memories are stored in two forms: gist and verbatim traces. Gist traces involve the underlying semantics of an event (e.g., remembering a person being hit by a car) whereas verbatim traces preserve the exact surface characteristics of an occurrence (e.g., recalling that a Ferrari 812 struck a pedestrian). According to this, false memories are caused by the retrieval of gist traces when verbatim traces are unavailable (Reyna and Lloyd 2006). For example, if a person must decide whether to support or oppose anti-abortion legislation, they will extract the key information concerning pregnancy termination they have previously come across (i.e., the gist) and use this data to decide. The level of information a person has access to and their own knowledge will both help determine which elements are crucial (Reyna 2012). As a result, someone who has encountered a lot of inaccurate information about abortion and does not have a solid background on the subject may mistakenly believe that ending a pregnancy is undoubtedly a wrong choice in every respect. Thus, upon encountering a new piece of information (e.g., an anti-abortion fake news story) that considerably overlaps with his/her own gist, an individual may falsely remember having seen the information before. In other words, we expected that a person who has been exposed to a large amount of misinformation would develop an erroneous ‘gist’, through which he or she might misinterpret fake news stories as events that actually happened, generating a false memory or belief.

The current meta-analysis

Given the variability across studies in the number of true and false memories recalled by individuals after exposure to fake news, it is informative to estimate the mean effect size of this phenomenon. However, this inconsistency is not the sole or primary reason for conducting a meta-analysis. In fact, meta-analyses serve a crucial role in explaining the variation between studies, also known as heterogeneity (Lakens 2016). By identifying and analysing factors that explain this heterogeneity, meta-analyses contribute to theory evaluation and development. Our meta-analysis aims not only to compute an effect size estimate but also to examine the variation in effect sizes across studies. This approach allows us to explore moderator variables, boundary conditions, and generalizability (Ioannidis et al. 2008). By assessing and modelling the consistency of effects, we gain valuable insights into the nuanced relationship between fake news exposure and false memories, shedding light on factors that influence this phenomenon. Moreover, understanding under which conditions such effects are magnified or diminished is crucial for comprehending the broader implications of fake news dissemination (Ioannidis et al. 2008). For instance, examining when such false memory effects are smaller can inform strategies aimed at mitigating the spread of misinformation. Such insights are invaluable for guiding policymakers and stakeholders in developing effective approaches to combat the detrimental impact of fake news on society. In essence, our meta-analysis serves as a comprehensive approach to integrate, synthesize, and analyse findings from diverse studies, aiming to address inconsistencies, identify patterns, and deepen our understanding of the intricate relationship between fake news, false memories, and individual variables.

The main goal of this meta-analysis was to examine the average percentage of true memories arising from exposure to true news and false memories and beliefs arising from fake news. More specifically, rather than focusing on beliefs in facts or common knowledge (e.g., believing in God), we centred our enquiry on understanding the impact of fake news on autobiographical memories and beliefs, which consist in recollections and convictions about personal experiences and events (e.g., recalling the attack on the Twin Towers). A secondary aim was to investigate whether and which individual characteristics make people vulnerable to remembering events that never happened following fake news exposure.

We had the following hypotheses: H1: drawing from the comprehensive scientific literature on false beliefs and false memories (Muschalla and Schönborn 2021; Scoboria et al. 2017), we hypothesized that exposure to fake news would potentially elicit these phenomena, with an estimated average rate expected to fall within a broad range, approximately between 15% and 45%. This estimate served as an initial exploration of the potential magnitude of the effect, recognizing the inherent variability in the phenomenon due to moderating factors. H2: previous research has demonstrated that various individual differences, including cognitive factors such as cognitive ability and analytical reasoning, as well as interest and knowledge in the topic, can impact the likelihood of forming false memories due to misinformation or exposure to fake news (Lee 2004; Roediger and Geraci 2007; Zhu et al. 2010). Therefore, we hypothesized that such individual variables would moderate the relationship between exposure to fake news and false memories. H3: We predicted that expertise and/or a high level of interest in a particular topic would be associated with a greater propensity to form false memories related to that topic (Baird 2003; Castel et al. 2007; Mehta et al. 2011).

Method

Literature search

This meta-analysis was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher et al. 2009). A PRISMA flowchart was used to represent the screening phase and the studies selection process (Haddaway et al. 2022) and a completed PRISMA checklist is available on Open Science Framework (OSF; https://osf.io/a62eg). The OSF database was consulted to verify that a study of a similar nature had not already been conducted or preregistered by other research groups. Afterwards, a search of the scientific literature on the topic of interest was conducted. The following databases and registers have been searched for eligible studies: EBSCO, Google Scholar, PubMed, Scopus and Web of Science. Moreover, since considerable research is being conducted on fake news, it was decided to also inspect the preprint repository PsyArxiv. The literature search started on the 17^th of January 2023 and continued until the 28^th of February 2023. A language (English) filter was applied to the search. Before conducting the final analysis, on the 7^th of April 2023 a second database search was performed to discover any potential recent articles that were missed by the first search. After full-text screening, the reference section of included articles from the literature search was checked for additional eligible studies. For each database and register, titles, abstracts, subject headings and general keywords were searched with no time restrictions. Please see Supplementary Material S1 for the search strategy and keywords used. The current meta-analysis was preregistered on the Open Science Framework (https://osf.io/) and data and additional materials can be found on https://osf.io/u9nsh/.

Inclusion and exclusion criteria

Articles were initially screened independently via Covidence (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia. Available at www.covidence.org) by the two first authors (AS and HO) based on title and abstract reading. Cohen's kappa coefficient (Landis and Koch 1977) was determined to measure the selection process's inter-rater reliability (see Results section). Afterwards, the same two investigators independently examined the full texts of the potentially eligible articles by applying the following inclusion and exclusion criteria. In the event of disagreement, the decision on the inclusion of the article was taken by a third rater (the sixth author). The selection process and the reasons for the exclusion of studies are shown in the PRISMA flow diagram (Haddaway et al. 2022).

We only considered studies published in English, published in peer-reviewed journals, pertaining to the creation of true and false beliefs and memories after exposure to true and fake news stories. Specifically, studies were included if they met the following inclusion criteria:

1. a. participants were exposed to both true news and fake news stories;

2. b. the effect of at least one individual variable (e.g., analytical reasoning, cognitive ability, ideological congruency, level of interest and engagement, use of media, perceived and objective knowledge, conspiracy ideation, fear, depression, anxiety, reasoning skills, coping mechanisms, confidence level, etc.) was assessed;

3. c. studies included as outcomes the number of participants that formed autobiographical true and false memories and/or autobiographical false beliefs following presentation of true and fake news, and the influence of the individual variables possibly examined on the number of memories and beliefs;

4. d. studies reported enough data to perform the analyses (e.g., means, effect sizes);

No specific eligibility criteria regarding population and methodologies used for the experimental exposure were used. Opinion papers, commentaries, systematic reviews, and observational studies were excluded, as well as all studies not in line with the scope of our review (e.g., misinformation effect within the eyewitness memory context; see Loftus 2005).

Data extraction and selection

An Excel spreadsheet (see OSF; https://osf.io/cekjg) was used to document the coding of the retrieved studies and the data extraction process. The following study characteristics were extracted:

1. a. General study information – reference, country of study, open access of data status, recruitment method;

2. b. Participant information – sample size justification, sample size, type of sample (i.e., students, Mechanical Turks, convenience sample, etc.), age (mean, standard deviation), gender ratio, education level, other relevant demographic information;

3. c. Intervention – number of true and fake news stories used, number of true and fake news stories actually presented to each participant, format (e.g., text only or presence of other media) and order of presentation, individual difference assessed (analytical reasoning, cognitive ability, ideological congruency, level of interest and engagement, use of media, perceived and objective knowledge, conspiracy ideation, fear, depression, anxiety, reasoning skills, coping mechanisms, confidence level);

4. d. Outcome – number and percentage of participants with false beliefs and false memories formed following exposure to fake news, number and percentage of participants with true memories formed following exposure to true news, influence of the individual variables examined on the creation of false beliefs and true/false memories (outcome of linear or logistic regressions);

5. e. Contact details of the corresponding author to request missing information. For articles including multiple studies, each study was extracted separately.

Moreover, raw information about each participant was retrieved from each study dataset. Specifically, we extracted the number of true and false beliefs and memories remembered and the score on tests used to measure individual variables (e.g., CRT,: Frederick 2005; Thomson and Oppenheimer 2016; Toplak et al. 2014; Wordsum: Thorndike and Gallup 1944; Wechsler 2008).

Study datasets and information not available in open access were requested by contacting the corresponding author(s), with a reminder being sent two weeks later if a reply had not been received. Data was noted as unavailable if a reply had not been received after two reminders.

Statistical analysis

As most of the studies’ datasets were accessible on OSF, we opted to perform a random effects meta-analysis on group-level data reassessing the outcomes of each study, rather than relying on summary statistics.¹ This decision allowed us to move beyond reliance solely on aggregate summary data from each study and instead extract relevant information from each participant included in the datasets. The random effects meta-analysis model assumes that the true effect sizes vary across studies, incorporating both within-study sampling error and between-study heterogeneity. This model assumes independence among effect sizes and estimates the average effect size while accounting for variability.

Analyses were conducted on the number of participants who created at least one false belief (for studies that also reported false beliefs) and/or one true or false memory about the true or false news presented. We used R statistical software (version 4.3.1; R Core Team, 2023), in particular the ‘metafor’ package (Viechtbauer 2015). The use of percentages and proportions as effect sizes is not unprecedented in the field of memory research. Previous studies, such as the mega-analysis conducted by Scoboria et al. on memory implantation studies (Scoboria et al. 2017), have utilized summary data to estimate the prevalence of different types of memory reports across heavily different experimental procedures. In our meta-analysis we adopted an approach similar to Scoboria's to integrate findings from studies with varying experimental designs. More specifically, while there were inherent differences between studies in terms of specific events presented, the overarching experimental and scoring procedures remain relatively consistent. Overall, the use of percentages provides a straightforward insight into the prevalence of an event (Barker et al. 2021).

Furthermore, we assessed the effects of individual variables on the production of memories by including individual differences as moderators in a multivariate meta-analysis, retrieving data from individual participants in each study. The multivariate random effects meta-analysis model expands upon the random effects model by accommodating multiple outcomes or correlated effect sizes within studies. This model considers the covariance structure among effect sizes, allowing for the estimation of overall effect sizes for each outcome. Investigation regarding the influence of individual variables was executed if the characteristic had been assessed by at least three studies. This criterion precluded the possibility of performing moderator analyses regarding the influence of individual variables on the emergence of false beliefs.

We reported 95% confidence intervals (CI), which indicate whether the effect sizes were statistically significant (95%CI does not include 0) and their variation (IntHout et al. 2016). We used forest plots to visually represent the variability in effect sizes between and within studies. We also looked at the specific impact of individual studies on the mean effect size using Baujat plots and the ‘influence’ function of the ‘metafor’ package. The ‘influence’ function of the ‘metafor’ package in R facilitates a comprehensive influence analysis for meta-analyses, generating informative diagnostic plots such as the Baujat plot (Schwarzer 2022; Viechtbauer 2010). This plot, inspired by the methodology outlined by Baujat et al. (2002), is an effective tool for identifying sources of heterogeneity in meta-analytic data. Specifically, the Baujat plot (Schwarzer 2022; Viechtbauer 2010) uses a graphical approach to explore heterogeneity, depicting each study's contribution to the aggregate Q-test statistic for heterogeneity on the horizontal axis. Simultaneously, it illustrates the influence of each study on the vertical axis, providing a visual representation of the impact of individual studies on the meta-analysis.

Furthermore, we conducted statistical heterogeneity analyses to examine the overall variation between studies. To do this, we used the ‘metafor’ package in the statistical software environment R to compute Tau², I², between and within cluster heterogeneity, and Q statistic with corresponding 95% confidence intervals. All information is displayed in the forest plots. Tau² is defined as the variance in effect sizes between studies (Borenstein et al. 2021). The I² statistic can be interpreted on a scale from 0 to 100% (Higgins 2003) and statistic benchmarks of 25%, 50%, and 75% categorise low, medium, and high heterogeneity, respectively. The I² statistic measures variation on a relative scale (Borenstein et al. 2021) and therefore identifies whether the reported variance of a meta-analysis is genuine variation between studies or a sampling error (Higgins 2003).

Results

The literature search yielded 427 records, with no duplicates. Following a thorough assessment of titles and abstracts based on our eligibility criteria, 33 full-text articles were retrieved, and 13 of these ultimately met the inclusion criteria for the meta-analysis (Figure 1). The interrater reliability for studies selection, calculated with Cohen's kappa coefficient, was 0.94 (almost perfect agreement), with disagreement in respect of only one study, which was resolved by a third person (the sixth author). All corresponding authors – who were contacted because studies’ datasets were not open access – provided the dataset of their study after a maximum of two reminders, except for one author, which led to the exclusion of the respective article from the meta-analysis. Overall, we retrieved sufficient data from 12 studies, from which 15 effect sizes were extracted.

Figure 1. PRISMA flowchart of literature search.

Characteristics of the included studies

The studies included in this meta-analysis investigated the number of participants who formed at least one false or true memory or false belief following exposure to fake or true news. Study participants were both students and members of the general population from different countries and were recruited through various methods, including: social media and mailing lists (Mangiulli et al. 2022; Murphy et al. 2019, 2021; O'Connell and Greene 2017; Scuotto et al. 2021, 2023), Prolific (crowdsourcing platform; Greene et al. 2021, 2022; King and Greene 2024), Amazon Mechanical Turk (Calvillo et al. 2023; Grady et al. 2023) and news websites (i.e., TheJournal.ie; Greene and Murphy 2020; Murphy et al. 2019). All studies, except one (Scuotto et al. 2021, n = 91), included >100 respondents, with values ranging from 161 (Mangiulli et al. 2022) to 3,746 (Greene and Murphy 2021).

While all the papers examined the number of true and false memories created after viewing fake and true news, only four (Greene et al. 2021; Murphy et al. 2019, 2021; last article contains two separate studies) investigated the number of people with false beliefs related to the materials presented. Moreover, while each one of the studies assessed the influence of one or more individual variables (e.g., ideological congruency, level of interest and/or engagement, analytical reasoning, objective knowledge, etc.) on the number of reported true and false memories, none of the studies assessed the moderator effect of individual variables on the emergence of false beliefs. Regarding the specific variables, although more than half of the studies investigated the effect of ideological congruency on the number of false and true memories recalled as a result of exposure to fake news, no statistical analysis was conducted on this data. The rationale behind this is that each study examined different facets of ideological congruency (political ideology, views on feminism and abortion, etc.) employing instruments specifically designed for the experiment; consequently, the data was hardly comparable with each other. See Table 1 for a summary of the main features of the studies. A complete dataset with the information about the specific factors investigated by each study is available on OSF.

Table 1. Main characteristics of the studies included in the meta-analysis. Abbreviations: PA, power analysis; M, mean or males; F, females; SD, standard deviation; NA, not applicable.

False memories (H1)

The random effects meta-analysis model on group level data showed that 40.49% (95%CI [31.44; 49.54], 95%PI [4.60; 76.37]) of the participants (5,805 out of 14,443) reported at least one false memory (see Figure 2 for the forest plot). A high level of heterogeneity was also revealed, as the heterogeneity values were Tau² = 0.03 (SE = 0.01, I² = 99.28%, Cochran's Q (df = 14) = 1758.25, p < 0.001). No effect size was identified as influential by visual analysis of the Baujat plot and influence output (see R script for images and further details). No signs of asymmetry were discovered either by visual inspection of the funnel plot or by the trim and fill procedure (no missing studies on the right side; see R script for visualizations). Furthermore, the regression test for funnel plot asymmetry was not statistically significant (p = 0.10; for the funnel plot, see Supplementary material S2).

Figure 2. Forest plot with the outcome of the random effects meta-analysis on false memories.

True memories

The random effects meta-analysis model on group-level data revealed that 85.94% (95%CI [78.84; 93.04], 95%PI [57.71; 114.17]) of the individuals (12,470 out of 14,443) reported at least one true memory (see Figure 3 for the forest plot). The heterogeneity values were Tau² = 0.02 (SE = 0.14), I² = 99.66%, Cochran's Q (df = 14) = 1199.80, p < 0.001, indicating a significant level of variability. By visually examining the Baujat plot and influence output, the effect size from King & Greene's study (2024) was found to be an outlier (see R script for images and additional information). Excluding this study from the meta-analysis and repeating the calculation resulted in a percentage of 88.55% (95%CI [83.24; 93.87], 95%PI [68.18; 108.93]). Trim and fill procedure showed no evidence of missing studies on right side; however, both visual inspection of the funnel plot and the regression test for funnel plot asymmetry (p = 0.03) provided some indications for publication bias (for the funnel plot, see Supplementary material S3).

Figure 3. Forest plot with the outcome of the random effects meta-analysis on true memories.

False belief

The random effects meta-analysis model on group level data showed that 60.88% (95%CI [53.06; 68.71], 95%PI [43.58; 78.19]), (4,117 out of 6,762) created at least one false belief (Figure 4). As in the previous instances, a high level of heterogeneity was revealed, with the following heterogeneity values: Tau² = 0.0062 (SE = 0.01), I² = 97.64%, Cochran's Q (df = 14) = 85.24, p < 0.001. The two effect sizes of the 2021 study by Murphy (Murphy et al. 2021) and colleagues were identified as influential by visual analysis of the Baujat plot and influence output (see R script for images and further details) and their exclusion from the meta-analysis yielded a percentage of 58.04% (95%CI [50.28; 65.81], 95%PI [42.74; 73.34]). No signs of asymmetry were found upon visual inspection of the funnel plot or throughout the trim and fill procedure (no missing studies on the right side; see R script for visualisations); in addition, the regression test for funnel plot asymmetry was not statistically significant (p = 0.09), meaning that no indication of publication bias was revealed (for the funnel plot, please see Supplementary material S4).

Figure 4. Forest plot with the outcome of the random effects meta-analysis on false beliefs.

Moderator analyses

While the analyses on moderating factors were based on a limited number of studies, it is essential to note that each of these studies encompassed a substantial participant pool. Consequently, the cumulative number of participants across these studies has yielded statistically robust and meaningful outcomes. The participant counts for each moderator analysis are as follows: cognitive ability (n = 4,774), analytical reasoning (n = 6,078), objective knowledge (n = 5,446), and level of interest (n = 5,576).

Analytical reasoning (H2)

Of the 12 articles included in the present study, five assessed the participant's performance on CRT to examine the influence of analytical reasoning on news story recall (Calvillo et al. 2020; Greene et al. 2021; Greene and Murphy 2020; King and Greene 2024; Mangiulli et al. 2022). To assess whether analytical reasoning had an impact on susceptibility to create false memories for fake news or true memories for true news, we included participants’ score at the CRT (Frederick 2005; Thomson and Oppenheimer 2016) as a moderator variable in the multivariate random effects meta-analysis model on individual-level data. CRT score was indeed a statistically significant moderator for the creation of false memories but not of true memories after being exposed to fake and true news, respectively. More specifically, individuals with higher CRT scores were less likely to form false memories for fake news (estimate = −0.03, SE = 0.003, 95%CI [−0.03; −0.02], p < 0.001), but this association was absent when considering true memories for true news (estimate = −0.002, SE = 0.001, 95%CI [−0.005; 0.002], p = 0.38).

Cognitive ability (H2)

Three of the studies included (Greene et al. 2021; Mangiulli et al. 2022; Murphy et al. 2019) examined whether cognitive ability, measured by the Wordsum test (Toplak et al. 2014), affected the propensity to form false or true memories following fake or true news presentation. Results from the multivariate random effects showed that there was no significant association between the performance at the test and the percentage of individuals who reported at least one false memory (p = 0.72) or true memory (p = 0.45).

Objective knowledge

The scores obtained by participants in four of the included studies (Greene et al. 2021; Greene and Murphy 2020; Mangiulli et al. 2022; Scuotto et al. 2021) on tests assessing their background on a particular topic were included as a moderator variable in the multivariate random effects meta-analysis model on individual-level data to investigate a potential relationship between the knowledge about the subject in question and the susceptibility to recalling false or true memories related to that topic. There was no statistically significant association (p = 0.90) between the objective knowledge and the percentage of participants who reported at least one false memory about the fake news presented. Interestingly, analysis of the participants’ true memories shows that people who were objectively knowledgeable about a particular topic were more likely to remember correct information about that topic (estimate = 0.30, SE = 0.11, 95%CI [0.08; 0.51], p = 0.01).

Level of interest (H3)

The last relevant variable that was expected to influence memory performance of the participants exposed to fake and true news stories was the level of interest in a specific topic and had been investigated by five studies (Calvillo et al. 2023; Greene et al. 2021; Greene and Murphy 2020; Mangiulli et al. 2022; O'Connell and Greene 2017). The level of interest of respondents in each study was reported or adjusted on a Likert scale from 0 to 10 and added to the multivariate random effects meta-analysis model on individual-level data as a moderator. Level of interest and engagement in a specific topic had a statistically significant (estimate = −0.03, SE = 0.01, 95%CI [−0.04; −0.02], p < 0.001) influence on participants’ predisposition to recall at least one false memory after being exposed to fake news, with higher levels of interest in a topic being associated with lower susceptibility to report false memories about that topic. A statistically significant association was also concerning true memories, where higher scores on the Likert scale were linked to a greater likelihood of recalling genuine information about a particular subject (estimate = 0.03, SE = 0.003, 95%CI [0.03; 0.04], p < 0.0001).

Exploratory analyses³

Additional exploratory analyses were conducted, including a meta-regression analysis to assess the impact of the number of fake news stories presented in each study on false memory recollection. This involved incorporating the number of fake news stories as a moderator variable in the meta-analysis model. The results revealed a statistically significant effect (estimate = 0.05, SE = 0.02, z = 2.01, 95%CI [0.001; 0.096], p = 0.04), indicating that studies with multiple fake news stories were associated with a higher likelihood of participants recalling at least one fake news story. This finding suggests that increased exposure to fake news stories may enhance the likelihood of false memory formation. Notably, the number of true news stories presented did not significantly affect participants’ recollection (p = 0.80). This lack of influence can be attributed to the already high accuracy of participants’ memory for true news (85.94%).

We also performed exploratory analyses to calculate the absolute rates of remembered and believed fake and true news, thus providing a more nuanced understanding of the recall and belief patterns associated with exposure to these events. Using the ‘dataset_individual’ and ‘dataset_individual_belief’ excel sheets on OSF (https://osf.io/94u5k, https://osf.io/z5rbh, respectively), we calculated the overall percentage of true/fake news remembered/believed by individuals¹. Our analysis revealed that the percentage of remembered fake news was 22.25%, whereas the percentage of remembered true news was 53.80%. Furthermore, the absolute rate of believed fake news was 22%. It is important to note that while the percentage of fake news remembered is smaller than ~40% of individuals who recall at least one false memory, it remains a substantial percentage. Given the abundance of fake news circulating on various platforms, remembering (or believing in) one fake news out of four or five indicates a significant impact of exposure to misinformation.

Discussion

Recently, there is growing interest in how fake news can distort memory (e.g., Loftus and Klemfuss 2024). In the current meta-analysis, our primary focus was to examine the impact of fake news on false memory formation. Furthermore, we investigated whether certain individual variables (e.g., analytical reasoning, cognitive ability, objective knowledge) either magnified or mitigated the effect of fake news on false memory formation. We will now elaborate on our key results.

Our first hypothesis centred on the percentage of participants influenced by fake news presentation that subsequently formed false memories about these fake news stories. This hypothesis was framed within the context of limited systematically conducted studies on the topic of interest. The scarcity of prior research indicated a more exploratory approach to the hypothesis formulation, aiming to provide a preliminary estimate of the expected false memory rate in response to fake news exposure. As such, our hypothesis served as a starting point for investigating the phenomenon, rather than a rigid expectation with predefined outcome criteria.

Starting from that hypothesis, our meta-analytic result showed that the main percentage of individuals who created false memories due to exposure to fake news data was 40.49%. However, subsequent exploratory analyses revealed a potential inflation of this figure, stemming from the fact that the studies included in our meta-analysis investigated the percentage of participants who developed at least one false memory following the presentation of multiple fake news events. Upon analysing the absolute rate of false memories relative to the number of presented fake news events, this percentage decreased to 22.25%. This adjusted rate aligns more closely with findings from Scoboria et al. (2017), whose mega-analysis involved single-event presentations in memory implantation studies, resulting in 30.4% of participants generating false memories. The nested model (Scoboria et al. 2004) argues that memory for an event implies belief in its occurrence, and belief implies that the event is deemed plausible. We might therefore speculate that the level of plausibility of the content of the fake news presented by the studies included in this meta-analysis was quite similar to the one of the suggestions delivered in the false memory implantation studies. Plausibility has been shown to catalyse the formation of false memories (but see also Otgaar et al. 2009). Since the fake news stories were frequently subtle variants of existing reports, such as disinformation concerning the pandemic, these fake news stories might have indeed been experienced as plausible stories. Based on this, it could be argued that individuals reported false memories following fake news exposure because of source monitoring errors.

Concerning the effect of fake news on false belief, 60.88% of the participants reported to believe in at least one fake news event, which is notably higher than the false memory acceptance rates, but equally drops to 22% when considering the overall percentage of events each participant believed out of the total number of events presented. The observed false memory/belief rates in fake news studies cannot be solely attributed to the perceived plausibility of the fake news story. Other factors, such as source monitoring errors induced by the simultaneous presentation of narratives and images, may also play a significant role. While the images used in these studies were not fabricated but rather depicted the events or key figures involved, existing literature suggests that presenting individuals with a narrative accompanied by an image of the event may contribute to the creation of false memories (Garry and Gerrie 2005). Undoubtedly, this underscores the intricate interplay between verbal and visual cues in memory encoding.

In terms of recalling true news, participants showed strong memory performance, with 85.94% of them remembering at least one true news item. This percentage decreases to 54% when considering the absolute rates of recall of true events relative to the total number of true news items presented. Overall, there was very good memory performance of truthful news by participants, consistent with what previous studies have reported (Frenda et al. 2013; Johnson et al, 2023; Wade et al. 2002). It is however necessary to address the evidence of publication bias observed in the data concerning the recall of true news. This finding is particularly noticeable from the visual analysis of the funnel plot, where a couple of studies on the left-hand side of the figure stand out from the rest of the dataset. Rather than suggesting an underrepresentation of studies with null or nonsignificant results, we posit that this observed publication bias may stem from methodological variations across studies, particularly in terms of the used procedures. In the study by King and Greene (2024), identified as an outlier, the authors acknowledged limitations regarding the unique historical context (the Covid-19 pandemic) and the unfamiliarity of the news stories (related to cancer), which could have impacted the recall of true news. We posit that the evidence of publication bias is more likely associated with procedural aspects of the studies rather than selective reporting of significant findings in the literature.

Individual variables

While we find it informative to report an average rate of false memories, true memories, and false beliefs across studies, it is paramount to acknowledge the significant variability in the rate of memories recalled due to factors such as individual differences and contextual variables. Thus, while our findings offer valuable insights into the general magnitude of memories and beliefs elicited in response to news events, it's important to note that the specific prevalence in any given study may differ. Factors such as participant characteristics (e.g., cognitive and personality traits), experimental design, and stimulus materials can all influence the observed rates of false memories. Therefore, researchers should interpret our findings within the context of this variability, recognizing that the results of individual studies and each singular instance of exposure to fake news will be shaped by these factors.

Analytical reasoning

Concerning our second hypothesis, participants’ scores on the CRT were a statistically significant moderator for the creation of false memories following exposure to fake news. High performance on the test was correlated with lower susceptibility to false memories, although the effect was rather small. This result suggests that people with effective analytical reasoning skills may possess a protective factor against the creation of false memories regarding fabricated information. Importantly, while a decrease in false memories was noted among subjects with higher CRT scores, this performance did not translate into an elevated recall of true stories.

Given the absence of statistical analysis regarding the influence of ideological congruency on memory recollection, our ability to firmly position ourselves within the ongoing debate between two competing theories explaining the phenomenon of belief in fake news is somewhat constrained. On one side of the debate stands the theory of motivated reasoning, asserting that cognitive biases predispose individuals to believe false statements that align with their ideological beliefs (Slothuus and De Vreese 2010). On the other side, Pennycook and Rand (2019) advocate for the classical reasoning account of disinformation, arguing that the perceived truthfulness of fake news is primarily attributable to failures in analytical reasoning. In their perspective, individuals may not necessarily succumb to bias, but rather might exhibit a tendency to overlook critical evaluation, whether due to laziness or cognitive limitations.

Pending further data to draw an informed conclusion, our position in this discourse gravitates towards a middle ground. We propose that these perspectives need not be mutually exclusive; instead, there may be a bidirectional influence between motivated reasoning and cognitive ability. Specifically, motivated reasoning may amplify susceptibility to bias, particularly among individuals with limited cognitive resources. Conversely, elevated cognitive capacity may serve as a protective factor against cognitive distortions and enable a more accurate analysis of the news content to which individuals are exposed.

Cognitive ability

Unlike analytical reasoning, cognitive ability did not emerge as a moderator significantly influencing the recall of fake or authentic news. Specifically, among the studies analysed, only Greene et al. (2021) demonstrated a direct decline in susceptibility to fake news among participants with elevated Wordsum scores, while Mangiulli and colleagues (2022) and Murphy and colleagues (2019) found an interesting association between low levels of cognitive ability and an increased effect of ideological congruency on fake news recall that brings us back to the potential connection between cognitive ability and motivated reasoning suggested earlier. Curiously, these findings seem at odds with lines of research that have previously highlighted cognitive ability as a pivotal factor in shaping eyewitnesses’ vulnerability to misinformation (Greene et al. 2020; Zhu et al. 2010). Another possible explanation for this discrepancy lies in the assessment tool itself. While Wordsum, commonly used to assess cognitive ability, demonstrates a high positive correlation with full-scale IQ (Huang and Hauser 1998), it is essential to acknowledge its limitations as a vocabulary-based measure. First, Wordsum primarily appraises crystallized intelligence – knowledge and skills that a person has acquired throughout their life (verbal knowledge, language proficiency, expertise in specific domains, etc.) – rather than fluid intelligence, which pertains to logical thinking, novel problem-solving, and adaptability. Consequently, test scores could inadvertently intertwine with measures of education and socioeconomic status. Additionally, the test's reliance on vocabulary proficiency might underestimate the cognitive aptitude of individuals whose English language skills do not align with their general intelligence level – such as second language speakers (even though it was not the case for the studies considered in this meta-analysis) or individuals with language impairments or learning disorders, like dyslexia.

Objective knowledge and level of interest

In contrast to prior evidence indicating that level of interest, greater subject-knowledge and/or expertise could potentially enhance false memories (Baird 2003; Castel et al. 2007; Mehta et al. 2011; O'Connell and Greene 2017), our meta-analysis provided more nuanced findings. Our findings did illustrate that objective knowledge about a specific topic contributed to increase in correct memory performance to true news. Likewise, the level of interest and engagement exhibited by participants resulted in a significant increase in the recollection of true news. Intriguingly, compared to objective knowledge, the level of interest was also found to be a protective factor against the emergence of false memories for fake news. Drawing from a source monitoring perspective (Johnson et al. 1993; Mitchell and Johnson 2000), these findings suggest that a more developed schema stemming from elevated knowledge or frequent engagement with a topic generates a network of intertwined memory traces. This network can be triggered by a novel yet related story, evoking a sense of familiarity and leading to subsequent construction of a memory of the event. While this phenomenon may indeed trigger the formation of false memories, our findings suggests that such a network enhances individuals’ capacity to differentiate between true and false memories for news items.

Limitations and future directions

Some caveats concerning the current work need to be addressed. First, a pronounced level of heterogeneity was evident across the outcomes of studies. This observation could stem from a multitude of factors, including the diverse methodologies used across studies, variations in sampling methods yielding differing reference samples, fluctuations in news design, subject and presentation, and potential influences from contextual factors. Second, and more importantly, the present meta-analysis examined only four of the numerous individual differences that could potentially impact the recollection of fake and true news. The reason for this selection lies in the fact that these individual characteristics were measured by each study through different instruments, often not standardized or created specifically for the experiment in question, rendering comparison through statistical analysis impractical or extremely complex. Third, it is noteworthy that only a limited portion of the available research has explored the effect of presenting fake news on the formation of false beliefs (four out of thirteen studies), even though the presence of false beliefs may be sufficient to influence people's behaviour (e.g., Bernstein et al. 2015). Moreover, the complete absence of studies investigating the influence of individual variables in the relationship between fake news and false beliefs precluded the possibility of conducting a moderator analysis. Finally, it is critical to note that while some studies included in this meta-analysis ensured that the fake news stories presented to participants were entirely novel (e.g., Greene et al. 2021), others did not explicitly verify this aspect. Consequently, the wording ‘false memories for fake news’ may not universally reflect memories for events that never occurred but rather memories for misinformation previously encountered. This potential ambiguity highlights the need for future research to carefully consider the novelty of fake news stories when examining memory effects in this context.

To address these limitations, it would be useful to conduct further studies wherein the individual variables of interest are measured with standardized instruments or tools embraced by the scientific community. This course of action holds the promise of facilitating later assessments of their overarching impact on the recollection and belief in both false and true news stories. Moreover, an intriguing avenue for exploration lies in investigating potential moderating influences of other variables, such as suggestibility and memory distrust, quantified using tools like the Gudjonsson Suggestibility Scale (Gudjonsson 1997).

Conclusion

This meta-analysis clearly showed that fake news can drastically contaminate memory with almost half of participants forming false memories for fake news. Although the focus of this study is primarily on the domain of psychology, we believe that our findings also hold significant implications for the field of media and communication studies. By analysing how exposure to fake news, a prevalent phenomenon in the contemporary media landscape, influences memory and belief formation, we draw a conversational bridge between cognitive psychology and the field of media studies. Furthermore, by identifying individual factors that might attenuate or exacerbate susceptibility to false memories, our research supports a possible development of interventions to aid discernment in media consumption. The implications are therefore far-reaching and have overall practical significance in the context of today's information-rich environment. In a world where misinformation can have dire consequences (Greene and Murphy 2021a), from public health decisions to political choices, our meta-analysis emphasizes the importance of empirical research in guiding strategies to counter the impact of fake news on memory and belief. By addressing the identified limitations and embracing future research lines, we can advance our understanding and fortify society's defenses against the pervasive influence of misinformation.

Notes

1. 1. We conducted additional exploratory analyses that are not the main focus of this paper. More specifically, through the use of group-level data we assessed the presence of possible data dependency related to studies conducted by the same first author and studies with multiple effect sizes (Cheung 2019) by performing a three-level meta-analysis model. That is, a standard meta-analytic model that has been extended to handle nested effect sizes, in which level-1 represents a random-effects model, while level-2 and level-3 represent heterogeneity variances (Cheung 2019). All relevant material concerning these analyses (R script, tables and figures) can be found on OSF at the following link: https://osf.io/u9nsh/files/osfstorage.

2. 2. Regarding the exclusion of one dataset after multiple reminders to the corresponding author, we provide further details on the process undertaken. Initially, an email was sent to the corresponding author requesting the dataset of their study. Following this, a reminder was sent two weeks later when no response was received. Upon the author's acknowledgment of the reminder, they expressed their intention to search for the data. Subsequently, another reminder was sent due to the lack of updates received. Finally, after the author communicated their inability to retrieve the dataset, the article was excluded from the analysis.

3. 3. We extend our sincere gratitude to the reviewers for their insightful suggestion to conduct exploratory analyses based on the number of fake news articles presented in each study and on absolute rates of remembered/believed true/fake news.