Geographic and epistemic pluralism in the sources of evidence informing international environmental science–policy platforms: lessons learnt from the IPBES values assessment

Abstract

Non-technical summary

This article examines the challenges and opportunities to integrate diverse sources of evidence in assessments produced by international platforms working at the science–policy interface. Diversity (or pluralism) of sources of literature, both in terms of their geographic origin and disciplinary focus, is essential for assessments to inform decision-making across social–ecological contexts. Using the recently completed ‘Methodological Assessment of the Diverse Values and Valuation of Nature’ of the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services as a case, we find that significant effort has been dedicated to reviewing diverse literature. We discuss three strategies to expand pluralism in future assessments.

Technical summary

Representing plural views in science–policy platforms is essential to avoid reproducing geographic and epistemic biases that permeate contemporary scientific knowledge production and synthesis. The Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services (IPBES) has strived to produce assessments that incorporate information from diverse regions and knowledge systems. We explore the geographic and epistemic pluralism of the literature included in the ‘Methodological Assessment of the Diverse Values and Valuation of Nature’ (VA), and the challenges and opportunities to achieve such knowledge pluralism. We applied a bibliometric analysis to the sources of evidence cited in the VA, and reflected on the assessment development process, in which we were directly involved. Our results highlight the success of different strategies developed by VA experts to engage with diverse sources of literature. Still, most evidence was English-language academic literature produced in Western Europe, Canada, and the United States, echoing the prominence of this literature in scientific publication in environmental disciplines. Reflecting on our experiences, we discuss strategies that could further enhance the geographic and epistemic pluralism in the information reviewed for future environmental assessments produced by IPBES and other international science–policy platforms.

Social media summary

Epistemic and geographic pluralism was partially achieved in IPBES Values Assessment, and can be further enhanced in future assessments.

1. Introduction: geographic and epistemic pluralism in international science–policy platforms

1.1 The importance of pluralism in international science–policy platforms

Achieving global sustainability requires tapping into diverse sources of knowledge (Cash et al., 2003; Tengö et al., 2014). Knowledge pluralism, understood as the integration of diverse knowledge and knowledge systems (White & Lidskog, 2023), can be epistemic and geographic. Epistemic pluralism refers to the different ways in which the world and our relations to it are conceived, and how our understandings about it are generated (Gardner, 2013). Achieving epistemic pluralism implies integrating knowledge from different theoretical perspectives and knowledge systems, such as scientific and lay knowledge. It is instrumental to understand complex socio-ecosystems: for instance, traditional ecological knowledge, held by Indigenous Peoples and Local Communities (IPLCs), can provide crucial insights on ecosystem processes and functions (Berkes et al., 2000); it also forms the basis of local resource management practices that contribute to biodiversity conservation and to the resilience of socio-ecological systems (Mavhura et al., 2013; Turnhout et al., 2012). Combining diverse sources of knowledge, including both academic and non-academic knowledge, can therefore enable the achievement of the United Nations (UN) sustainable development goals (Cash et al., 2003; Díaz-Reviriego et al., 2019). Geographic knowledge pluralism refers to the integration of knowledge that is produced in diverse socio-ecological contexts. The world is a mosaic of different ecosystems, cultures, and natural resource management practices. These diverse socio-ecological contexts imply different decision-making practices, ecological challenges, values, and interests. Combining literature focusing on these diverse socio-ecological systems is therefore necessary to understand how different sustainability or conservation strategies may fare differently in different contexts (Cash et al., 2003; Merçon et al., 2019).

International science–policy platforms (ISPPs) synthesize state-of-the-art knowledge on specific topics (e.g. climate change or biodiversity loss) and translate the insights gained into policy-actionable knowledge. Their role is to serve as an ‘honest [knowledge] broker’ who informs decision-makers about the range of options available to them and the implications of each option (Pielke, 2007). Given the existence of contrasting perspectives, values, and interests informing the scientific processes of ISPPs, epistemic and geographic pluralism are necessary to ensure that diverse perspectives reflecting a wide range of worldviews and normative positions are considered (Carrier, 2013; Díaz-Reviriego et al., 2019; van den Hove, 2007). Moreover, plural perspectives on diverse policy options can help policymakers adapt scientific insights to their particular contexts, and to consider uncertainty in the decision-making process (Pielke, 2007). Achieving pluralism, particularly in the inclusion of both academic and non-academic knowledge from relevant stakeholders such as IPLCs, can also help increase the legitimacy of ISPPs (Kohler, 2022; Timpte et al., 2018). For instance, some governments, particularly from the Global South (e.g. Bolivia, Ecuador, and Cuba), have been critical of the promotion (or hindrance) of particular policies by ISPPs claiming that they could interfere with their own national biodiversity conservation strategies (Sala & Torchio, 2019), and that they inadequately represent the challenges they experience and their perspectives on how to address them. For these reasons, one major challenge for ISPPs is to produce assessments that present diverse views from different regions, cultures, and knowledge systems. Addressing this challenge requires overcoming the biases that characterize contemporary scientific production.

1.2 Biases in scientific production and reviews

Subjectivity permeates all stages of scientific activity, including selecting an object of study, setting the scope of the research, and selecting research methods and analytical tools, as well as the research writing stage, where scientists tend to selectively report results (Cairo et al., 2020; Zvereva & Kozlov, 2021), and the publishing stage, where positive results tend to be favored over null or negative ones (Ekmekci, 2017; West & Bergstrom, 2021). The idea of ‘neutral knowledge’ is therefore an oxymoron (Borie et al., 2021). By framing, researching and assessing the issues at stake in particular ways, experts implicitly legitimize certain claims and policy options over others (Dryzek, 1997; Luke, 1995; Wesselink et al., 2013).

Ensuring epistemic and geographic pluralism in assessments of the literature (such as systematic literature reviews or assessments produced by ISPPs) is therefore essential to avoid biases and to gain insights on the diversity of views and knowledges on a given topic. Yet, both epistemic and geographic biases have been noted in the literature. Scientific publishing has been shown to be biased against researchers from the Global South (Corbera et al., 2021; Maas et al., 2021; Nagendra et al., 2018; Tuyisenge et al., 2023) as a result of multiple structural inequities, including reduced funding availability and scarce representation in editorial boards, as well as more stringent editorial and peer-review processes for researchers from the Global South (Yousefi-Nooraie et al., 2006). This may be the result of an imperialist heritage of science, which delegitimizes conceptual frameworks and ways of knowing and communicating that stray from modern scientific procedures promoted in the Global North (Mignolo, 2009). In parallel, systematic literature reviews and other syntheses of knowledge tend to focus on academic knowledge, particularly from the natural sciences. This may be a result of different publication traditions: authors from different disciplines tend to publish in different outlets (for instance, scientific articles and books) some of which are difficult to access and for which publication rules vary. Further, gray literature (for instance, policy documents or reports from non-governmental organizations) is rarely considered in systematic reviews (Cairo et al., 2020; Paez, 2017; Yoshida et al., 2022) and sources of knowledge from IPLCs (e.g. declarations, stories, songs, poems, etc.) are often overlooked or contested by western scientists and institutions (Smith, 1999; Tengö et al., 2014).

Linguistic bias is of particular concern as it may hinder both geographic and epistemic pluralism. Non-native English speakers tend to spend more time producing and communicating knowledge in English (Amano et al., 2023), which represents an important cost when engaging in ISPPs using English as the working language. Non-native English speakers may also struggle to publish in English (Ramírez-Castañeda, 2020) and thus often choose to publish in their native language. In addition, systematic reviews and scientific searches are often conducted in English only (Droz et al., 2023; Ekmekci, 2017; Nuñez & Amano, 2021), as a result of the limited resources of English-speaking research teams to translate studies written in other languages (Neimann Rasmussen & Montgomery, 2018). In parallel, search engines have been shown to favor literature written in English even in multilingual searches (Rovira et al., 2021). This biases the scientific state-of-the-art, as this limits the diversity of information that is included in the reviews. Yet, non-English science contains crucial insights, for example, regarding biodiversity and conservation across diverse regions of the world (Amano et al., 2016, 2021; Egger et al., 1997; Konno et al., 2020). Failing to better incorporate literature in languages other than English limits our understanding of complex environmental issues across different regions (Amano et al., 2021; Droz et al., 2023; Konno et al., 2020; Lynch et al., 2021). These biases have to be taken into account and remedied in the work and implementation guidelines of international ISPPs.

1.3 Efforts and challenges toward pluralism in Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services assessments

One of the main tasks of the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services (IPBES) is the production of assessment reports that synthesize the state of knowledge regarding nature and its contributions to people worldwide. A unique feature of IPBES, relative to other environmental ISPPs (such as the International Panel on Climate Change), is its commitment to achieving geographic and epistemic pluralism (Borie et al., 2021). IPBES strives to guarantee equitable regional representation in all its governing bodies, taskforces, and author teams. Decisions regarding the work of IPBES are taken by consensus in plenary where all (currently 144) member states are represented and have an equal vote. IPBES rules of procedures suggest the teams of experts producing IPBES deliverables must be balanced in terms of gender, region of origin, and disciplinary representation (Borie et al., 2021; Tengö et al., 2014). Different types of expertise are considered, and experts can be academics, but can also be civil servants, working in civil society organizations and the private sector. All experts have to inform IPBES of any potential conflicts of interests.

In addition, responding to calls to integrate diverse knowledge systems, and in particular, the knowledge of IPLCs (Fazey et al., 2020; Raymond et al., 2010; Turnhout et al., 2012), IPBES has developed strategies to incorporate knowledge from diverse disciplinary perspectives, languages, knowledge systems, and regions (Montana & Borie, 2016; Stokland et al., 2022). Examples of this are the explicit consideration of IPLCs worldviews in the IPBES conceptual framework, the institution of a taskforce on Indigenous and local knowledge (ILK) providing guidance and support for the integration of ILK across IPBES deliverables, the establishment of a formal approach for including ILK in IPBES procedures, the conformation of ILK liaison groups within assessment teams, and the delivery of ILK dialogues formally established within the assessment production process (Díaz et al., 2015; Hill et al., 2020; McElwee et al., 2020). Through these processes, ILK holders and experts as well as non-governmental conservation organizations are invited to participate in all IPBES processes (as observers and experts) to ensure epistemic pluralism (Krug et al., 2020; Stevance et al., 2020). Despite these efforts, we acknowledge that achieving the inclusion of diverse IPLC voices remains a challenge. Particularly, more powerful or vocal IPLCs may be overrepresented in IPBES spaces and assessments. Continuous and innovative efforts need to be developed to better represent and involve plural worldviews in IPBES and more generally in ISPPs.

Following IPBES protocols, the elaboration of assessments follows three phases: (i) the request and production of a scoping document delineating the scope of the assessment, (ii) the production of the assessment through expert evaluation of the state of knowledge and two open external reviews, and (iii) the approval of the summary for policymakers of the assessment and the acceptance of the chapters providing the supporting evidence (IPBES, 2018). Each of these phases is characterized by principles of inclusivity (of governments and stakeholders), transparency (through independent peer-review processes), and representation of both findings and gaps in a policy-relevant manner.

Yet, key challenges remain and have been noted in a range of studies about IPBES. First, equal representation of experts from all five UN regions is rarely achieved (Báldi & Palotás, 2021; McElwee et al., 2020; Montana & Borie, 2016; Timpte et al., 2018), which can reduce the diversity of perspectives, priorities, and experiences that are key inputs for the assessments (Obura et al., 2021), and of the languages used in the selection of literature to be assessed (Lynch et al., 2021). Epistemic bias is also of concern. Recent studies identify a disciplinary bias, arguing that the social sciences are still not sufficiently represented in IPBES assessments (Díaz-Reviriego et al., 2019; Stokland et al., 2022; Vadrot et al., 2016). In addition, the absence of guidelines on how to assess the validity of non-scientific work can explain the difficulty of integrating diverse knowledge systems, and in particular ILK, to the platform's work (Gustafsson et al., 2019).

1.4 Case study and objectives of the research

The IPBES Methodological Assessment of the Diverse Values and Valuation of Nature (hereafter ‘Values Assessment’, VA) (IPBES, 2022a) and its summary for policymakers (IPBES, 2022b) were approved by the IPBES plenary in July 2022. The VA explores to what extent, and in which ways, diverse values of nature have been, and can better be, incorporated into decision-making to promote transformative changes to more just and sustainable futures. It also sheds light on the implications of including or excluding different types of values of nature from decision-making processes. Given the mandate of the VA to address values and valuation of nature from a diverse and inclusive perspective, the production of the VA was accompanied by unprecedented efforts to balance robustness and epistemic plurality within the assessment (Balvanera et al., 2022). Here, we assess how effective IPBES efforts have been toward achieving plurality in the case of the VA, and reflect, as experts directly engaged in producing the VA, what could improve in future IPBES and other ISPP assessments. We aim to respond to the following question: To what extent is the linguistic and disciplinary diversity of experts involved in the VA reflected in a geographic and epistemic pluralism of the literature reviewed in the VA?

To address this general question, we answer these specific questions:

1. (1) How diverse was the team of authors in terms of their disciplinary background and geographical location?

2. (2) What strategies were used to identify plural literature?

3. (3) Is the literature cited in the VA plural from a geographic and epistemic standpoint?

4. (4) Which initiatives best contributed to enhance pluralism of the assessed literature?

The paper is structured as follows. We first present the mixed-methods approach involving a bibliometric analysis. Then, we present the results, and highlight various strategies that can help enhance geographic and epistemic pluralism in ISPP assessments.

2. Research methods

2.1 Positionality and methodological approach

All authors of this paper participated in the IPBES VA in different capacities: PB, MC, BP, and UP were co-chairs of the assessment, with the role of leading the assessment and coordinating the experts who produced the assessment, as well as being coordinating lead authors of Chapter 1 of the assessment. GA-R, MC-F, VC, DG-J, and LG formed part of the technical support unit, with a technical and administrative role supporting the experts. ASM-S and BL were authors of Chapter 1. THM was author of Chapter 1 and coordinating lead author of Chapter 3. DFM and AVOC were contributing authors to Chapter 1. We used our direct involvement in the production of the VA to reflect on the different initiatives engaged to achieve plural literature reviews.

We used a mixed-methods approach to address the research questions. First, we built on existing descriptions of the VA process (Balvanera et al., 2022; Pascual et al., 2023) to describe the diversity of the expert team and of the literature review strategies. We produced descriptive statistics about the composition of the expert team, including the languages they speak, their nationality, and their disciplinary backgrounds. We grouped nationalities according to the UN regional groups of member states (United Nations, n.d.). We grouped literature review strategies as structured, semi-structured, and non-structured (see Pascual et al., 2023). Second, we performed a bibliometric analysis to assess the pluralism of the literature cited in the VA; this is described in more detail below. Additionally, we collectively reflected on our experience as authors of the VA to identify successful strategies to achieve geographic and epistemic pluralism.

2.2 Bibliometric analysis

We conducted a bibliometric analysis on a random sample of documents cited in the VA (documents are defined as any piece of written text, as well as pictures, videos and other digital sources cited in the VA chapters' bibliography or reviewed as part of the VA's literature reviews). (We started our analysis in 2020. Given this, we only included in our analysis the documents cited in the Values Assessment's Second Order Draft, which was finalized in 2020. The Second Order Draft is a draft published 1 or 2 years before publication of the final version of the document, and it is when new sources stop being included in IPBES assessments. Therefore, very few additional sources would appear in the final assessment (only in the case of ‘outstanding importance’ published during that year) (IPBES, 2018).) We used Yamane's formula to calculate the sample size (i.e. number of documents analyzed), opting for a 97.5% confidence level. Yamane's formula is as follows (Yamane, 1967):

$$n = \displaystyle{N \over {1 + N{( e ) }^2}}$$

where n is the sample size, N is the population size (in our case, the 13,067 documents cited in the assessment), e is the level of precision (in our case, 0.025 for a 97.5% confidence level).

The sample size was calculated as follows:

$$n = \displaystyle{{13, 067} \over {1 + 13, 067{( {0.025} ) }^2}} = 1425$$

We randomly selected documents stratifying across chapters to ensure that at least 10% of sources from each chapter were included in the analysis (see Table 1). After deleting duplicates (i.e. documents cited in more than one chapter), our final sample contained 1682 documents, surpassing the number required to achieve a 97.5% confidence level.

Table 1. Number of documents included in the bibliometric analysis

^a Chapter 1 – The role of the values of nature and valuation for addressing the biodiversity crisis and navigating toward more just and sustainable futures; Chapter 2 – Conceptualizing the diverse values of nature and their contributions to people; Chapter 3 – The potential of valuation; Chapter 4 – Value expression in decision-making; Chapter 5 – The role of diverse values of nature in visioning and transforming toward just and sustainable futures; Chapter 6 – Policy options and capacity development to operationalize the inclusion of diverse values in decision-making.

We selected different variables to assess the geographic and epistemic knowledge pluralism of the documents cited in the assessment. To assess geographic pluralism (defined as the integration of knowledge that is produced in diverse socio-ecological contexts), we used four variables. First, we identified the region of focus of the document, understood as the socio-ecosystem analyzed in the document. Ideally, the knowledge on a given region would be produced by authors within that same region, to reflect local knowledge and diverse perspectives on the matter at hand. Therefore, the second variable is the country where the first author resides and works (grouped according to UN political regions), and the third is whether another author resides and works in a different region than that of the first author. This allowed us to identify the regions which mostly finance and produce the documents cited in the assessment, and to identify works which may reflect a collaboration across geopolitical contexts. Finally, the fourth variable is the language in which the work was published. This allowed us to reflect on how language pluralism interacts with geographic and epistemic pluralism.

To understand the extent of epistemic pluralism (defined as the integration of knowledge from different theoretical perspectives and knowledge systems) in the VA, we used two different approaches. The first one, focusing on the different knowledge systems, was assessed using two variables: (i) the sector to which the author belongs (academic, governmental, IPLCs, or other, thus highlighting the arena in which the knowledge was developed); and (ii) the type of document, whether a book, a report, a law, a piece of artwork, etc. (assuming that different types of documents reflect different ways in which knowledge is shared). The second approach was to assess the academic discipline of the document (as a proxy of the different theoretical perspectives framing those documents); yet, this variable was only available for articles published in journals indexed in Scopus (which represent 54% of all the documents analyzed). The results associated with this variable must thus be interpreted with caution. As a result, our analysis of epistemic pluralism is centered around the diversity of knowledge systems.

Table 2 presents the variables used to characterize the sampled sources of the literature. The last column shows the percentage of documents for which data were available for each variable used in the bibliometric analysis. For instance, we could determine the region of origin for 95% of documents (i.e. for the remaining 5%, we were unable to identify the country of affiliation of the first author).

Table 2. Variables included in the bibliometric analysis

^a This group is composed of the following countries, besides Western European countries: Australia, Israel, Canada, Turkey, United States of America, and New Zealand.

* The graphs presented in Section 3 present results based on the analysis of the subset of documents for which the data was available.

** The remaining 45% of documents did not focus on a particular region, for instance, they were theoretical papers.

3. Geographic and epistemic pluralism in the VA

3.1 Efforts to achieve geographic and epistemic pluralism in the VA

The assessment comprised of six chapters prepared over the course of 4 years (2018–2022) by 84 experts nominated by member governments or IPBES stakeholders, and selected by the Multidisciplinary Expert Panel of IPBES, and 11 review editors approved by the Management Committee of the Values Assessment. The expert team was balanced in terms of gender and disciplinary background: 54% of experts identified as women; 54% of experts held at least one higher education degree in the social sciences, and 54% held at least one higher education degree in biophysical sciences. Two-thirds held higher education degrees in at least two different disciplines, illustrating the multidisciplinary background of the expert team. Experts were citizens of 47 countries and spoke 51 languages. Eighteen authors identified as experts on IPLCs, and two as ILK holders. (‘Indigenous and local knowledge holders are understood to be persons situated in the collective knowledge systems of Indigenous peoples and local communities with knowledge from their own Indigenous peoples and local communities’; experts on Indigenous peoples and local communities are ‘persons who have knowledge about indigenous and local knowledge and associated issues, not necessarily from IPLCs’ (IPBES, 2023).)

The diversity of the expert team was enhanced by the participation of over 200 contributing authors (who were citizens of 49 different countries, and included 25 ILK experts and 12 ILK holders). Contributing authors were integrated into the different chapters to address specific literature gaps outside the authors' expertise, whether in terms of topics, non-scientific documents, or evidence from specific regions. Sixty-nine percent of experts were academics hired at universities, the rest were employed by public or private research institutes (18%), foundations, non-governmental organizations, or cooperation agencies (7%), independent researchers (4%), or civil servants (2%).

In the assessment, different strategies were used to identify and review the literature to be assessed: 29 literature reviews were conducted, of which five were structured, 10 were semi-structured, two were non-structured, two were invited contributions, and 10 were a combination of reviews. These have been defined as follows:

‘(1) comprehensive structured reviews using search strings and search terms that defined the review's scope, the different filtering iterations, as well as defined parameters for the selection of the documents to review […] (2) semistructured reviews relying partially on expert-based search criteria […]; (3) non-structured reviews, fully based on expert criteria […]; (4) invited contributions from external experts and stakeholders through sources such as reports, news articles and art […] and (5) combinations of the above’. (Pascual et al., 2023, p. 12)

An analysis of the diversity yielded through each of these reviews is provided in Section 3.3. Ten of the literature reviews included languages other than English, and 15 included gray literature (Pascual et al., 2023). Combining expert knowledge and unstructured approaches to identify the literature, with structured protocols to review the literature identified, allowed experts speaking languages other than English to integrate literature in their native languages to the reviews and share key findings with the rest of experts in a systematized manner.

One crucial strategy as part of the literature reviews was the use of keywords beyond those used in the IPBES conceptual framework. The IPBES conceptual framework builds on a limited set of concepts (which include both scientific terms and non-scientific terms, such as ‘Mother Earth’) to be useful as a boundary object to communicate across cultures on biodiversity and its relations with human wellbeing (Díaz et al., 2015). Complementing keyword searches based on the conceptual framework by other non-scientific and regional terms was useful in achieving geographic and epistemic pluralism. For example, a literature search conducted around the concept of ‘Good Living’ allowed to integrate concepts such as ‘Buen Vivir’ from South America but also more than 20 other related concepts such as ‘Mauri Ora’, ‘Alli Kawsay’, ‘Minobimatisiwin’, ‘Vida sabrosa’, ‘Ubuntu’, ‘Satoumi’, among others, to refer to a philosophy of living in harmony with nature from different Indigenous and local perspectives. Using these keywords enabled the identification of literature in other languages such as Spanish, Portuguese, and French, and from the social sciences, humanities, and Indigenous local knowledge, providing alternative perspectives on the diverse values of nature. The additional keywords (those that did not emerge directly from the IPBES conceptual framework) were identified through the knowledge of VA authors and of ILK experts and holders who participated in the ILK dialogues, as well as through a snowball strategy (keywords were identified by reading existing literature on diverse values of nature). This allowed the use of terms that were not originally identified into literature searches.

To enhance legitimacy, relevance, and credibility, the assessment was produced in a stepwise process consisting of five drafts, all of which were reviewed through an internal peer-review process. Two drafts – the First Order Draft and the Second Order Draft – were also reviewed externally by non-IPBES experts and stakeholders and IPBES member governments to produce the final assessment report. Throughout the external review processes, stakeholders and member governments could identify gaps in the assessment and suggest additional literature to include in the reviews. All comments received through the review had to be addressed by experts, either by incorporating the comment into the VA or by offering a justification for why they did not. This process was crucial in finding and addressing research gaps and biases. As an example, the First Order Draft received 2836 comments from 210 expert reviewers and 11 governments.

In addition, different efforts were undertaken to ensure that ILK was integrated across the VA assessment. Some of these efforts are already part of the IPBES rules of procedure for producing assessments and are led by an ILK Taskforce and Technical Support Unit. An ILK liaison group (with assessment authors) was formed to ensure that ILK was considered throughout the assessment. Three ILK Dialogues were held with the aim of exchanging experiences, opinions, and worldviews regarding ‘values of nature’ from different IPLCs. These dialogues had around 160 participants representing more than 50 countries. Another effort was the Global Call for ILK contributions, launched in six languages (Arabic, English, French, Russian, Chinese, and Spanish), which aimed to gather and incorporate evidence from ILK holders. Through this Global Call, authors received 420 contributions from 73 countries; 33% of contributors self-identified as indigenous peoples and 22% identified as members of local communities. (Other contributors did not self-identify as such. This means that they can be experts on Indigenous issues without self-identifying as being a member of an Indigenous or local community.) We recognize that participation in the ILK dialogues required the capacity to travel. Likewise, the participation in the Global Call for ILK contributions required knowledge of one of the six UN official languages. This may have hindered the participation of some IPLCs. These tools can enhance pluralism in the assessment processes but cannot ensure a complete representation of all IPLCs. Additional efforts led by the ILK liaison groups included: (1) building collaborative review efforts focused on ILK, building on experts' interdisciplinary background and multilingualism to shape review protocols with ILK-specific search terms and in languages other than English to reach ILK sources; and (2) inviting contributing authors who were ILK holders from regions or countries where knowledge gaps were identified. Furthermore, these efforts also influenced the rest of the reviews by, for example, conducting semi-structured ILK reviews or providing ILK principles to be integrated in structured review protocols.

3.2 Pluralism achieved in the inclusion of the literature

The Assessment cited 13,067 sources of evidence (thereafter ‘documents’) that were reviewed in depth, including peer-reviewed academic literature, policy documents, and other sources such as ILK, regarding the diverse values of nature and their role in supporting decision-making processes. The results of the bibliometric analysis (Figures 1 and 2) illustrate the pluralism achieved in terms of geographic and epistemic diversity of literature in the VA.

Figure 1. Geographic and linguistic diversity. Panel (a) shows the country of affiliation of the first author of the documents reviewed. Panel (b) shows the focus of the documents reviewed. Panel (c) shows the percentage of documents written by a cross-regional team (when authors come from more than one of the UN regions). Panel (d) shows the language in which documents were published.

Figure 2. Diversity of disciplines and knowledge systems. Panel (a) describes the types of sources (academic, intergovernmental, non-intergovernmental, governmental, and other incl. indigenous contributions). Panel (b) shows the percentage of reviewed documents based on their format (journal article, report, book, review, book chapter, website, and other). Panel (c) shows the percentage of journal articles reviewed by discipline.

Although the expert team came from all of the five UN regions, three-quarters of documents included in the VA were produced by a first author based in the Global North, namely Western Europe and Other States (WEOS) (Figure 1a). (Authors who work in WEOS may be nationals of countries from other regions, and/or have been trained in other regions, as a result of the global mobility of scientists (Verginer & Riccaboni, 2021). Yet, there is evidence that the research focus of researchers varies across world regions and tends to be related to local challenges and priorities (Salvia et al., 2019). Researchers' place of works therefore plays a role in the focus of their research. Hence, authors' professional affiliation is a useful, albeit partial, proxy of geographic pluralism.) Twenty-eight percent of documents were produced by multi-regional teams, which suggests that few documents with first authors based in WEOS have been written in collaboration with authors from other regions (Figure 1c). Of all the sources cited, 56% had an explicit geographic focus, and this focus was distributed across the five UN regions. Still, the WEOS region appears as the one with the greatest focus (77% of documents with a geographic focus, Figure 1b). Additionally, the predominant language of documents was English (96%), followed by Spanish (2%); the remaining sources were in Chinese, Dutch, French, German, Portuguese, Norwegian, Polish, Russian, and Ukrainian, or were published in multiple languages (Figure 1d). Most of the languages spoken by experts (for instance, Hindi, Afrikaans, Jotï, Korean, Farsi, Turkish, Obijwe, or Swahili) were not represented in the literature reviewed in the assessment.

In terms of epistemic pluralism, we noted a primacy of academic documents (90%). Indigenous contributions represented just under 1% of documents, which amounts to 118 documents included in the assessment (Figure 2a). More than three-quarters of the sources used were journal articles. Books and reports were less common. Other documents, such as laws and public policies, videos, and artwork, represent 2% (n = 280) of the documents (Figure 2b). The predominant discipline of the journal articles was environmental sciences (50%), followed by social sciences (12%) and economics, econometrics, and finance (10%) (Figure 2c). Yet, this is not enough to ascertain a disciplinary bias in the assessment. Indeed, environmental sciences journals may publish articles from the social sciences or that combine social and natural sciences. Additionally, this disciplinary analysis only focused on 54% of the total of documents included in our corpus of assessed literature. Articles published in journals not indexed in Scopus, as well as books and book chapters, were not included. Incidentally, these are media that are relatively more favored by social scientists to publish their results. Therefore, our analysis remains inconclusive as to the disciplinary pluralism achieved as part of the VA.

3.3 Diverse literature review strategies can enhance pluralism

Differences were found in the types of documents identified by different literature reviews (Figure 3). Structured and semi-structured literature reviews were found to be more effective in identifying academic sources. Unstructured reviews identified a greater diversity of documents, especially related to Africa, and Latin America and the Caribbean (both written by a first author based in those regions and focusing on those regions).

Figure 3. Pluralism achieved by type of literature review. It shows the diversity represented in the type of sources, languages, country of affiliation of the first author, and regions of focus across the analyzed literature; against six different review categories ranging from more structured reviews to expert selection of the literature.

Invited contributions, on the other hand (which include, for instance, the ILK global call for contributions), provided a great extent of non-academic documents, particularly indigenous and governmental documents. They also provided a broad range of documents in languages other than English and written by a first author outside of Western Europe. For instance, 12% of documents identified through invited contributions focused on Latin America (one of the least represented regions in terms of focus, with only 5% of documents overall).

A significant number of documents (44%) were selected based on experts' knowledge (those that appear in Figure 3 as ‘main text’). Although those are mostly academic articles, they tend to be the most regionally diverse. For instance, 39% of documents included in the assessment that were written by a first author based in Eastern Europe (one of the least represented regions in terms of first authors' affiliation with only 1% of documents overall) were identified through expert knowledge, including both the assessment authors and contributing authors.

4. Discussion

The IPBES VA was unprecedented in terms of the inclusion of the balance of genders, geographic origin, and disciplinary backgrounds among its team of experts. This is an important achievement following concerns about IPBES processes having been dominated by male, natural scientists from the WEOS region in the past (Díaz-Reviriego et al., 2019; Timpte et al., 2018). Furthermore, the VA team included a significant share of experts from the social sciences and humanities as well as experts with an interdisciplinary background, which has been highlighted as an important means to experiment with more diverse literature review strategies (Droz et al., 2023). The experts conducted sustained efforts to achieve pluralism, by implementing diverse literature search strategies, which allowed them to tap into a wide diversity of sources of evidence. As a result of these efforts, documents from all regions of the world, from academic and non-academic sources, and from a broad range of disciplines, were used in the production of the VA. Still, most of the documents reviewed as part of the assessment were academic articles written in English, most of which being written by first authors residing in WEOS, and a majority of documents were focusing on the same region. Lynch et al. (2021) suggested that having a diverse team of experts is not enough to ensure linguistic diversity when conducting literature reviews that feed assessments – our results corroborate this finding and suggest that the same applies for epistemic and geographic pluralism. Although the efforts made in the VA have borne fruit, much remains to be done to enhance geographic and epistemic pluralism in future ISPP assessments. In this section, we reflect on three key points to articulate existing challenges and possible strategies to overcome them: (1) the mitigation of existing geographic biases in the scientific literature; (2) the use of diverse strategies to integrate epistemically diverse knowledge; and (3) the need to overcome the primacy of English in IPBES activities (Figure 4).

Figure 4. Barriers and opportunities to achieve geographic and epistemic pluralism in the IPBES VA. It shows the main filters and barriers driving the selection of literature that was reflected in the VA. Solutions to address these barriers emerging from the learnings of the assessment are also provided.

4.1 Mitigation of geographic biases present in the scientific literature

Most scientific articles published globally are in English (Ammon, 2001; Drubin & Kellogg, 2012) and conducted by authors based in Western Europe and the United States (Maas et al., 2021; Pasgaard et al., 2015; Skopec et al., 2020). Scientific journals' fees policies enhance this bias, as, in the pay-to-publish system, publication depends on the capacity of authors to pay for their articles to be published, but in the pay-to-read system, accessing the literature depends on the capacity of the reader to buy access to the articles. These fees can be particularly unaffordable for institutions based in the Global South, which impedes the democratization of the production of and access to scientific knowledge (Knöchelmann, 2021). These existing biases in the scientific production process imply that achieving geographic pluralism in literature reviews requires deliberate efforts to specifically search for literature from the Global South and in languages other than English. In the VA, such efforts to reduce geographic biases present in the academic literature included the following: (1) relying on expert knowledge and non-structured reviews to identify literature from each of the UN regions, in English as well as other languages; (2) inviting contributing authors from non-WEOS regions to identify and review pertinent literature from under-represented regions; and (3) using search terms in structured and semi-structured reviews that integrated concepts in languages other than English and reflected worldviews from diverse regions. The results of the bibliometric analysis suggest that these strategies were partially successful in mitigating geographical bias. Some key challenges remain: for instance, conducting searches in multiple languages requires translation work by experts fluent in the identified languages. It also requires that these experts interpret and analyze the literature themselves, demanding time and effort to build protocols and shared understanding for analyzing the literature and that is often conducted collectively when the reviews are conducted in English.

Moving forward, we suggest that IPBES and other ISPPs address the difficulties of achieving geographic pluralism considering the above biases. In IPBES, resources could be channeled through the Data and Knowledge Taskforce or other taskforces to tap into regional sources. Analytical support will be crucial: providing references from diverse regions to experts will not be enough if the experts cannot interpret them or analyze them due to language barriers. This support should come at an early stage in the design and consolidation of literature searches to provide experts enough time to properly include this information in the assessments.

4.2 Strategies to achieve epistemic pluralism

Academic literature was the most common source of information included in the VA. Experts adopted two main strategies to identify and review non-academic literature: first, they conducted diverse literature searches focused on gray literature, and second, they invited contributing authors to provide specific contributions focusing on ILK, thus achieving an integration of 10% of non-academic sources in the VA.

Some literature searches focused exclusively on gray literature and the arts. However, these required additional time to design both the search parameters and the procedures to analyze the documents (Yoshida et al., 2022). Procedures are still lacking in IPBES to assess the validity of non-academic knowledge, which may explain the difficulty for experts to include more of it in the literature reviews (Gustafsson et al., 2019). Ambiguous rules of procedures that are difficult to interpret with regards to assessing non-academic knowledge, can lead to a ‘paralysis of judgment’ which impedes the integration of gray literature in IPBES assessments (White & Lidskog, 2023). Furthermore, identifying gray literature documents can be more challenging compared to academic documents, as the latter are systematized in academic search engines (Paez, 2017) and gray literature is not always available through databases or search engines, nor is it openly available for the experts to access. Time can also be a constraint, although reports are usually produced within 3–4 years, the number of drafts and review processes and the fact that evidence sources can only be added until the preparation of the Second Order Draft of the document, limit the time that experts have to promote efforts for the integration of literature in different languages, focused on different regions, and representing diverse epistemic views. With the recent emergence of fast-track assessments within IPBES, it is likely that this will become an even bigger challenge. Furthermore, the guidance for achieving diversity within the assessments has been focused on gender and regional representation of experts, with limited guidance for authors on how to conduct and integrate reviews that allow the representation of diverse types of evidence. Moving forward, IPBES and other ISPPs could allocate resources for technical support to conduct gray literature reviews and establish working groups focused on integrating more diverse literature (both geographically and epistemically) in the assessment process. Further guidelines can also be designed for conducting non-structured literature reviews and searches for gray literature, with standards to assess the quality of the evidence beyond academic journals and that would apply particularly to ILK sources.

Another effort conducted by experts was to invite non-academic contributing authors, often practitioners or ILK experts and holders, to identify pertinent non-academic literature that could not be identified through structured reviews. Indeed, traditional structured reviews are likely to replicate the biases inherent in global scientific production, excluding to a large extent ILK because of the diverse ways of transmitting ILK other than scientific papers (for instance, through oral transmission or the arts) (Cámara-Leret & Dennehy, 2019). In fact, invited contributions were the most successful strategy to identify ILK documents, documents in languages other than English and Spanish, and focusing on regions other than WEOS. Thus, such invited contributions to tackle specific knowledge gaps should be encouraged in future ISPP assessments. To facilitate these initiatives, ISPPs could provide support to identify contributing authors. Such a process could also be facilitated by ISPP stakeholders, who could organize as a network to collectively address potential knowledge gaps. Organizing knowledge networks to address specific research gaps has also been identified by Stokland et al. (2022) as a solution to facilitate the inclusion of social sciences and humanities scholars in IPBES.

There are very different ways of conceiving what is knowable and interactions between ‘knowers’ and what is ‘knowable’, and of generating knowledge across cultures and disciplinary developments (Gardner, 2013). In this paper we were not able to fully assess to what extent such ontological, epistemic, and methodological diversity was represented in the VA due to limitations in the indicators for which we had data available. Yet, our findings support the need for generating mechanisms that ensure a wider representation of such diversity.

4.3 Overcoming the primacy of English in IPBES activities

English plays an important role as a common language in IPBES processes. However, it inadvertently acts as an exclusionary tool for experts and stakeholders who primarily speak other languages. Although some IPBES processes take place in multiple languages (this is the case of the plenary meetings, for instance), the main working language in the assessment elaboration processes is English. This means, for instance, that the external review processes take place in English, which may explain why the majority of comments received are from WEOS. English proficiency is also a requirement for the selection of experts (e.g. Curriculum Vitae in other languages are often not taken into account), and all expert meetings are conducted in English. Using a single common language may limit discussions, and concepts and literature that cannot be effectively communicated in English may be excluded from the assessments.

In the VA, experts used their own knowledge of literature in languages other than English, and relied on invited contributions from non-native English speakers, to ensure the non-English literature was considered. However, these efforts remain limited. Droz et al. (2023) have identified successful strategies to move toward multilingual ISPPs, including (a) working with multilingual teams, (b) conducting multilingual literature reviews, and (c) launching the Global Calls for ILK contributions conducted as part of recent IPBES assessments in the six official languages of the UN. The VA implemented all these strategies with limited success.

As new assessments are produced and efforts to overcome the primacy of English in IPBES assessments are undertaken a few things should be noted: (1) reviewing literature in other languages is time-consuming and this should be considered as part of the timelines defined to produce assessments; (2) assessing literature in multiple languages is connected to the selection of experts; to address language gaps in the selection of experts, formal processes to invite contributing authors to support review processes in other languages can be established; and (3) efforts to communicate progress in the assessment and receive comments in other languages can be made at the stage of external review processes by conducting workshops in key languages spoken by assessment experts targeted to different regions and allowing to collect comments in diverse languages.

5. Conclusion

In this paper, we have analyzed the geographic and epistemic pluralism of the literature included in the IPBES VA as a case study to understand the representation of diverse knowledge in ISPPs. We find that despite having a diverse team of experts and explicitly putting in place different strategies to identify and review diverse knowledge sources, certain geographic and epistemic biases remain in the literature included in the assessment. The documents reviewed as part of the VA were mostly academic sources, from the environmental sciences, produced in Western Europe and the United States, and published in English. Consequently, the efforts made in the production of the VA fell short of achieving IPBES' commitment to geographic and epistemic pluralism. Nevertheless, these efforts do shed light on potential ways forward to improve pluralism for future assessments.

We have identified key challenges that have hindered the inclusion of more diverse knowledge. Particularly, the use of English as the main working language of IPBES and the overrepresentation of academic experts compared to non-academic experts (for instance, ILK holders or civil servants), have been major barriers to identifying and reviewing more diverse sources of knowledge. However, we also identified successful strategies to promote further representation of plural knowledge such as expanding the conceptual framings to allow diverse epistemologies to be represented, opening spaces for alternative review strategies, generating opportunities for the participation of stakeholders and experts (beyond the author team) from across regions. To enhance pluralism, science–policy platforms like IPBES can undertake diverse strategies, such as developing standards to evaluate the quality of non-academic sources and literature reviews, or generating processes that facilitate the integration of diverse evidence and knowledge sources. Ensuring more significative participation of Global South experts in ISPPs (from the scoping of the assessments, the selection of experts, the participation in external reviews, to the approval of assessments), as well as the integration of non-academic experts to the authors' team, is instrumental to further facilitate the identification of diverse literature sources.

We also identify structural inequalities in the global scientific endeavor that hinder geographic and epistemic pluralism in assessment processes; those cannot be resolved solely by more efficient literature review processes within ISPPs. Achieving knowledge pluralism will require further efforts by the global scientific community more broadly. Particularly, editors of scientific journals have a role to play in facilitating a greater representation of non-English literature written by experts from the Global South in academic journals, and to foster plural publication processes. This can encompass initiatives such as receiving articles in multiple languages (as has been done by the geography journal Fennia, see Kallio & Riding, 2021) or committing to publishing articles from diverse regions to promote pluralism. We observe that these initiatives are already under way within diverse academic journals and are gaining strength. We are confident that alongside the existing efforts of ISPPs to enhance pluralism in literature reviews, and the additional strategies identified in this paper based on the experience of the VA, future science–policy assessments can achieve even more knowledge pluralism and thus enhance their pertinence for policymaking on environmental issues worldwide.