Part I - Identifying priorities and collating the evidence

Chapter Three Scanning horizons in research, policy and practice

Bonnie C. Wintle

University of Melbourne

Mahlon C. Kennicutt II

Texas A&M University

and William J. Sutherland

University of Cambridge

3.1 Introduction

Conservationists have long had to deal with a number of prominent, recurring issues, such as habitat loss and fragmentation, pollution, invasive species and wildlife harvesting, to name a few. On top of these well-known challenges, others have emerged. Over the last half century, these have included the impact of halogenated pesticides and defoliants, acid rain from coal-fired electricity generation, ecological impacts of biofuel production and atmospheric releases of ozone-depleting chemicals. In more recent times, concerns have emerged around microplastics and exploitation of the Arctic, although some changes also bring opportunities for conservation, such as using mobile phones to collect data. New and emerging issues tend to make policy and practice more difficult. They add to an already challenging agenda, and often require a response when knowledge of the problem is limited.

Emerging from the relatively new field of ‘futures’ studies, horizon scanning is still developing as a method. By crowd sourcing information and drawing on communities of practice to sort, verify and analyse that information, horizon scanning offers an efficient way to look for early indications of poorly recognised threats and opportunities (Sutherland & Woodroof, 2009; van Rij, 2010). It aims to minimise surprises by foreseeing these threats and opportunities, enabling policy-makers and researchers to respond quickly to developing problems. Horizon scanning is an approach primarily used to retrieve, sort and organise information from different sectors that is relevant to the question at hand, in a similar process to intelligence gathering. It can also include varying degrees of analysis, interpretation and prioritisation, but deciding which issues to act on, and how to act on them, typically takes place after the horizon scanning, and is assisted by other ‘futures’ tools, such as visioning, causal layered analysis, scenario planning and backcasting (e.g. Glenn & Gordon, 2009; Inayatullah, 2013; Cook et al., 2014a). Recent frameworks have also been developed to link different futures tools, such as horizon scanning and scenario planning, together (Rowe et al., 2017).

Horizon scanning outputs come in a wide range of forms. Some broadly describe a single trend that cuts across different parts of society, such as the rise of big data, or the future of a general area of interest, such as ‘Environmental Sustainability and Competitiveness’ (Policy Horizons Canada, 2011). These outputs are usually aligned with more general foresight programmes. Other exercises look at a set of more specific potential threats, such as invasive species that may arrive in the UK and threaten biodiversity (Roy et al., 2014), and compare them in an approach similar to risk assessment. For the last 10 years, conservation scientists have run annual horizon scans to identify emerging issues with the potential to impact global conservation (e.g. Sutherland et al., 2018). A similar approach has also been used to identify important scientific questions that, if answered, would help guide conservation practice and policy (e.g. Sutherland et al., 2009).

As with any policy advisory work, there is always a risk that useful information is gathered but not followed up, as decisions are often driven by other, usually non-scientific, factors. This risk may be higher with unsolicited (grassroots scans produced by a community of practitioners, researchers or academics) rather than solicited scans (called for by policy- and decision-makers). It can be unclear where the responsibility lies for integrating outputs into policy-making, and uptake depends on the organisational culture at the time (Delaney & Osborne, 2013). Schultz (2006) pointed to a conceptual contradiction between evidence-based policy and horizon scanning, where the latter searches for issues that may not be fully supported by a definitive body of evidence. A more optimistic perspective is that horizon scanning needs to be embedded in a broader strategic foresight framework, to increase the likelihood that findings are translated into practice (e.g. van Rij, 2010; Cook et al., 2014a). As mentioned above, horizon scanning identifies emerging and novel threats and opportunities as a first step, but other foresight tools serve different purposes along the pathway to adopting appropriate policy. These other foresight tools are not explicitly covered in this chapter, but we provide an example, The Antarctic Science Scan and Roadmap Challenges Exercise, of a hybrid horizon scanning activity where an accompanying road map was also produced to outline actionable recommendations (Box 3.2).

In this chapter, we introduce both general and specific approaches to horizon scanning, outline some ways of achieving and measuring impact and explore how horizon scanning may evolve in the future.

3.2 Approaches to horizon scanning

‘Exploratory horizon scanning’ identifies novel issues by searching for the first ‘signals’ of change across a wide range of sources (such as an early scientific paper describing a potentially impactful new technology). ‘Issue-centred scanning’ monitors issues that have already been identified by searching for additional signals that confirm or deny that the issue is truly emerging (Amanatidou et al., 2012). Signals can be organised into clusters (multiple pieces of information) that can either contribute to the evidence base around pre-identified issues, or form a long list of novel issues that are potentially emerging (Figure 3.1). The long list of issues can be further analysed and prioritised into a shortlist using methods detailed below. Some horizon scanning exercises take further steps to make the output more useful for the end user, for example, by assessing the policy relevance of the issues or the feasibility of addressing them, and by identifying those that warrant ongoing monitoring (Sutherland et al., 2012).

Figure 3.1 General framework for horizon scanning, reflecting the key steps in the procedure (ovals), inputs and products (rounded rectangles), key outputs (rectangles), actors and end users (triangles), and activities and methods (floating text).

Process adapted from Amanatidou et al. (2012).

There is a range of different ways to carry out horizon scanning; we introduce the main stages and provide some specific examples in the boxed texts and Table 3.1. Because our definition of horizon scanning concentrates largely on information retrieval, sorting and, to some extent, analysis and prioritisation, we focus here on methods that facilitate these activities.

Table 3.1 Approaches to horizon scanning (some activities and examples overlap)

Approach	Examples
Manual search of an invited expert group with Delphi-style prioritisation	Global conservation (e.g. Sutherland et al., 2018), Antarctic science (e.g. Kennicutt et al., 2015), bioengineering (Wintle et al., 2017), Mediterranean conservation (Kark et al., 2016)
Manual search of a large crowd-sourced group (open call) with Delphi-style prioritisation (invited)	Future of the Illegal Wildlife Trade (Esmail et al., 2019)
Automated open-source search and manual analysis/prioritisation (usually by a community of experts)	IBIS (Grossel et al., 2017), Global Disease Detection Program (Centers for Disease Control and Prevention, www.cdc.gov/globalhealth/healthprotection/gdd/index.html), HealthMap (www.healthmap.org/en/), ProMed (www.promedmail.org/)
Advanced text analytics to identify emerging issues and research areas (e.g. sentiment analysis, machine learning)	FUSE Program (www.iarpa.gov/index.php/research-programs/fuse), Meta (https://meta.org/), X risk database (www.x-risk.net/)
Manual searches within an organisation (by employees, interns or volunteers), results tagged and catalogued in a database	US Forest Service (Hines et al., 2018), UK Department for Environment, Food and Rural Affairs (Garnett et al., 2016)
Comprehensive programme (including scanning, sentiment analysis, scenario planning; manual and automated)	Singapore’s Centre for Strategic Futures (www.csf.gov.sg/), partnered with the Risk Assessment and Horizon Scanning Programme Office
Expert opinion (voting, survey)	Global Risks Report 2019 (World Economic Forum, 2019)
Regular meeting of a cross-disciplinary horizon-scanning group to discuss emerging issues and build database	Australasian Joint Agencies Scanning Network (www.ajasn.com.au/), Human Animal Infections and Risk Surveillance group (www.gov.uk/government/collections/human-animal-infections-and-risk-surveillance-group-hairs#risk-assessments-and-process)

3.2.1 Scoping

Like any major project, horizon scans need to be scoped and clear guidelines developed to assist scanners. A comprehensive scoping exercise addresses the following questions.

• What is the guiding question that defines what you want to know?

• How broadly or narrowly defined is the field of interest?

• What are the key drivers of change and activities in the field? It is common to organise thinking around a STEEP (Social, Technological, Economic, Environmental and Political factors) framework.

• What is the spatial scope? For instance, are you seeking issues with global or more localised impact?

• How far into the future should scanners be projecting?

• Who should be involved?

• Who are the potential end users?

Many of these considerations will be constrained by the resources available and the needs of the end user, but tools such as stakeholder analysis (Reed et al., 2009), domain mapping (Lesley et al., 2002) and issues trees (Government Office for Science, 2017) can be useful. Scoping exercises may also involve some pilot scanning to get a feel for how well-defined the task is. For example, preliminary scanning in a US Forest Service project that aimed to identify emerging issues that could affect forests and forestry in the future revealed that ‘natural resources and the environment’ was too broad a topic for their exercise. Instead, it was narrowed to ‘forests’ (Hines et al., 2018).

Horizon scans that rely heavily on people rather than computers to do the scanning reflect the biases of those participants. A well-structured procedure for obtaining judgements from participants (e.g. Figure 3.2) will go a long way to mitigate psychological biases (Burgman, 2015b), but in order to capture a broad array of perspectives, involving a diverse group of people to identify and prioritise candidate issues is critical. A cognitively diverse group – comprising individuals who think differently – is thought to maximise collective wisdom and objectivity (Page, 2008). A good proxy for cognitive diversity is demographic diversity. Achieving demographic diversity can be challenging in practice. For example, there may be language barriers to overcome, and people with certain occupations (e.g. scholars) may be over-represented in horizon scans conducted by researchers. Inviting contributions from further afield, both geographically and from outside immediate peer circles, broadens the scope of issues considered. This might be achieved by putting out an open call for issues online and advertising it through relevant websites and email lists (e.g. Esmail et al., 2019), or posting a call for ideas on social media.

Figure 3.2 The Delphi-style horizon-scanning approach often used in conservation (Sutherland et al., 2011).

Figure reproduced from Wintle et al. (2017), published under the Creative Commons Attribution 4.0 Licence.

3.2.2 Gathering inputs

Inputs to a scan can either be gathered manually (by people) or with the aid of automated software, which is then (usually) analysed by people. Manual scanning typically involves a group of people monitoring current research and relevant trends (e.g. technology trends, disease trends or population trends) via desktop searches, attending conferences and consulting other people in their networks. Information can be manually scanned in news articles, social media, publications, grey literature and other output of relevant organisations (such as models and projections). This is typically the first step in a ‘Delphi-style’ method that then goes on to analyse and prioritise candidate issues in a structured approach, usually involving one or more expert workshops (see Boxes 3.1 and 3.2 for examples and further descriptions of the procedure). Scanners could be provided with guidelines by a facilitator to direct their search, including suggestions of where to look. Manual methods have the advantage of accessing content that may not exist online (e.g. grey literature or unpublished research), or content that may be difficult to locate in the absence of known keywords to direct database and online searches. The downside of manual methods is that they are labour-intensive and may be exposed to the biases of the searcher, as they are less systematic.

Box 3.1. A Delphi-style method for horizon scanning in conservation

With its foundations in the Delphi Method (Linstone & Turoff, 1975; Mukherjee et al., 2015), this structured approach (Figure 3.2) was first applied in horizon scanning for conservation by Sutherland et al. (2008). There are now several variants. The key features that make this approach ‘Delphi-style’ are iteration (issues are submitted, scored, discussed and scored again) and anonymity of submissions and scoring. Typically, about 25 conservation experts from around the world participate in the following procedure. Over the course of several months, participants independently scan material from a variety of sources (e.g. papers, reports, websites, conferences) looking for issues (threats or opportunities) that are relatively novel, but that we should start planning for. Over email, each participant anonymously submits short summaries of two to five issues they have selected as the best ‘horizon-scanning’ candidates, defined as reflecting a combination of novelty, plausibility and potential future impact on global conservation. The facilitator compiles the issue summaries and circulates them back to the group, who anonymously score each issue in terms of its suitability as a ‘horizon-scanning’ item (using the definition above). A shortlist of the top scoring issues, containing perhaps twice the total number sought, is recirculated back to participants. Each participant is assigned approximately five issues (not their own) to investigate further, gathering further evidence to support or oppose the issues’ suitability. This means each issue will be cross-examined by at least two to three people. These five issues are usually assigned to people who are not considered experts in that subject matter, in the hope that they will have fewer preconceptions about the issue and that the experts will add their knowledge anyway. The whole group then meets at a workshop and systematically discusses each of the shortlisted issues (e.g. to consider new perspectives, relevant research, and whether the issue is genuinely novel or just a repackaging of an old issue). The issues are kept anonymous to reduce biases and allow for an open discussion. After the discussion, participants individually score the issues a second time. The top-scoring 15 are redrafted by one of the other group members and published each year in Trends in Ecology & Evolution (e.g. Sutherland et al., 2018).

Box 3.2. Antarctic science scan and Roadmap Challenges project

The international Antarctic community came together to horizon scan the highest priority scientific questions that researchers should aspire to answer in the next two decades and beyond. The approach included online submission of questions from the science research community, followed by a subset of 75 representatives (by nomination and voting) attending a workshop. At the workshop, approximately 1000 submitted questions were winnowed down to the 80 most important through methodical debate, discussion, revision and elimination by voting. All information used, including the 1000 submitted questions, was made publicly available in a database at a horizon scan website (Kennicutt et al., 2014). The horizon scan was followed by the Antarctic Roadmap Challenges project that was designed to delineate the critical requirements for delivering the highest priority research identified. The project addressed the challenges of enabling technologies, facilitating access to the region, providing logistics and infrastructure and capitalising on international cooperation. The process uniquely brought together scientists, research funders and those that provide the logistics for field research in the Antarctic. Online surveys of the community were conducted to identify the highest priority technological needs, and to assess the feasibility (time to development) and cost of these requirements. Sixty experts were assembled at a workshop to consider a series of topic-specific summary papers submitted by a range of Antarctic communities, survey results and summaries from the horizon scan, as well as existing documents addressing future Antarctic science directions, technologies and logistics requirements (Kennicutt et al., 2015).

Computer-assisted scanning is increasingly used for automating the process of gathering a vast quantity of inputs, often crowd-sourced and usually from the internet (Palomino et al., 2012). Several such tools are now used in agriculture and health biosecurity to provide early detection of disease outbreaks (see Table 3.1 and Box 3.3 for examples) (Salathé et al., 2012; Kluberg et al., 2016; Grossel et al., 2017). Early online information, such as a tweet about a Tasmanian devil with a tumour on its face, or a YouTube video about a new device for targeting an invasive species, although unverified to begin with, may be critical for establishing the first in a series of signals that suggests a new or emerging threat (Grossel et al., 2017). Information on the internet can be retrieved in a number of ways. Keywords can be inserted into whole web search engines and/or particular websites can be targeted in more depth (e.g. Twitter can be searched using search terms, handles and hashtags). Research, news and current affairs can also be accessed via the RSS feeds of particular news and science sites, or by email and subscription to social media and blogs. Online data are often retrieved with the help of web scraping (accessing and storing particular web pages) and web crawling (accessing and storing links, and links of links from that page) (Hartley et al., 2013). With the recent increase in ‘fake news’, web searches require some form of quality control and vetting of sources: a process that can also be useful for exposing fake news. Large volumes of text scraped from the web, articles, patents, reports and other publications can be mined and filtered for potential relevance using automated software, such as machine learning algorithms.

Box 3.3. Online horizon scanning: intelligence-gathering for biosecurity

The International Biosecurity Intelligence System (IBIS) is a generic web-based application that focuses on animal, plant and marine health, and provides continuing surveillance of emerging pests and diseases, including environmental ones (Grossel et al., 2017). It also detects other environmental issues, such as harmful algal blooms. It is open source, in that it gathers articles from regular feeds of trusted sources (e.g. industry news, research) and publicly available online material, like news reports, blogs, published literature and Twitter feeds. Searches can be directed by broadly relevant keywords, such as ‘disease’ or ‘outbreak’ or ‘dead’, in addition to specific diseases of concern (e.g. ‘oyster herpes virus’). Articles can also be manually submitted by registered users to the application directly. A large expert community – the registered users, who are self-selected and approved by the administrator – then filter the articles, promoting those that they deem important and relevant to the home page, and demoting those that appear to be irrelevant or junk. Automated tools also assist with filtering (e.g. with machine learning and network cluster analysis), but as machine learning is still in its infancy, its use is limited to disease outbreaks from trusted sources. Items classified as junk by people are retained in a database to help the system’s artificial intelligence (AI) algorithms learn. The broader user community (anyone who signs up online) is alerted to items that have been flagged by the registered users as important, via a daily email new digest. IBIS is also ‘open-analysis’, meaning that analysis of the publicly available information is performed openly by registered users. They can create or contribute to an emerging/ongoing issues dashboard that features a window for adding content, a Delphi-based forecasting section, links to related reports, share functions, comments and a map showing the location of events of interest (e.g. an outbreak). Registered users can also conduct their own searches and use integrated analytical tools to construct intelligence reports. IBIS has been effective for guiding policies and active risk management decisions for the Australian Government since 2006. The system may produce up to five Intel briefs a week on major issues affecting biosecurity and trade, allowing the government to respond to threats much faster than before. For instance, the system picked up a report of oyster herpes virus from a UK farm, which had previously purchased used aquaculture equipment from a disease-stricken oyster farm in France. Intelligence from IBIS revealed that businesses that had been closed down by the disease had been liquidating their equipment and selling to other countries. In response to this, the Australian Government changed its biosecurity policy to decontaminate all used aquaculture equipment on arrival (Burgman, 2015a).

Automated scanning is fast, systematic and comprehensive in its scope, but often relies on people – sometimes experts – to screen, review, and perhaps investigate all reports before on-posting or incorporating them (Lyon, 2010). For tools that scan across a wide range of topics, and those that use ongoing surveillance, this can be onerous and time-consuming. There are three other notable challenges to relying on online content for horizon scanning. First, material needs to already be posted on the web, and there may be a delay before an event, such as an invasive species incursion, is reported online. The second is that useful content is not always publicly available, as it can lie behind pay walls, be stored on intranets (e.g. grey literature), or secured because it is commercially, politically or personally sensitive. The third challenge is that most methods for obtaining online content rely on using the right keywords, which requires some idea of what you are looking for.

3.2.3 Sorting, cataloguing and clustering

Tagging and cataloguing content derived from both manual and automated scans (e.g. by relevance, credibility, source type, sectoral origin) (e.g. Garnett et al., 2016; Hines et al., 2018) occurs concurrently with input gathering by scanners. Content can be further reorganised and vetted at a later stage. During this process, new search terms to direct further scanning can be generated, or existing search terms refined. Content can be organised according to a framework that also considers the level of response required and the strength of the evidence, which can help prioritise risks and other identified issues at a later stage (Garnett et al., 2016). Clustering methods, such as network analysis (Könnölä et al., 2012; Saritas & Miles, 2012), are useful for capturing cross-cutting issues that affect a number of topics of interest.

3.2.4 Analysing and prioritising

At this stage, a long list of issues will have been compiled, with some more suitable to the project aims than others. This can be an opportune time to reiterate objectives. Do you seek issues that most have not heard of? Do you intend to identify broad, developing topics or very specific developments (for example, the ‘increase in hydropower’ versus ‘fragmentation effects of hydropower in the Andean Amazon’)? Are you interested in issues likely to arise soon or events that have a smaller probability of playing out in the long-term future? Does the output need to be useful to policy-makers? Many exercises, especially those with follow-up plans, aim to prioritise a select number of ‘most suitable’ issues, and the precise manner in which such prioritisation decisions are made makes a real difference to the quality of the output (Sutherland & Burgman, 2015). Our experience with exercises that aim to identify novel issues is that participants gravitate towards well-known although important issues. Avoiding this requires strong chairing and a group that accepts the objective. To help overcome the problem, each participant can be asked whether they have heard of each issue, so that well-known topics can be excluded from the shortlist.

Within a manual Delphi-style approach (described in Boxes 3.1 and 3.2), issues are prioritised through an iterative scoring or voting process, usually facilitated online or in a workshop with a group of experts. The goal is to reduce a pool of potential horizon scanning items or ideas to a smaller subset. The number of items, or issues, covered in the final list can vary, but tends to reflect around 10–30% of the initial items put forward (e.g. Kennicutt et al., 2014; Parker et al., 2014; Kark et al., 2016; Wintle et al., 2017; Sutherland et al., 2018). As a point of comparison, the horizon scans described in Box 3.1 describe 15 issues annually, while the Antarctic hybrid horizon scan identified 80, shorter, priority scientific questions (Box 3.2). The final number may be constrained by how many the end user can realistically give their attention to (for a busy policy-maker, this may only be 15–20 half-page summaries), but is also driven by the number of (in)appropriate issues submitted. The main purpose of prioritisation is to remove issues that do not satisfy the selection criteria (novelty, plausibility, potential impact) and select those that are the most urgent or time-sensitive. Prioritisation of issues will inevitably involve trade-offs, especially where different group members have different perspectives. Because individuals’ diverging opinions can be masked in aggregated scores, analysing interrater concordance (e.g. with Kendall’s W) affords insights into the level of agreement between contributors. In a diverse group, we would expect a wide variety of viewpoints to be voiced, but a core of shared opinions is often discernible (e.g. Wintle et al., 2017).

Items identified in a computerised scan (e.g. articles returned from a keyword search) are also prioritised by groups of people with varying levels of content expertise. People may be employed to sort through material, such as in governmental horizon-scanning programmes like in Singapore, or they may volunteer to do so because they are interested in the output, such as a farmer or epidemiologist concerned with news of disease outbreaks. Initially, items are sorted according to their relevance to the scanning aims (often done in the initial tagging/sorting process). Irrelevant items are discarded or moved to low priority. A second form of prioritisation involves flagging issues or topics that are particularly noteworthy (Grossel et al., 2017). This can be because signals have grown stronger (more evidence is gathered to suggest an issue is becoming a threat or presenting an opportunity for action) (Cook et al., 2014b), or it might be because the potential consequences are so severe that the issue warrants immediate attention, even when evidence is limited or the probability is low (‘wild cards’).

3.2.5 Using the output

The previous step described prioritisation within the horizon scan to reduce a candidate set of issues. In that step, issues are ideally not judged according to importance, but rather according to less-subjective criteria, such as the likelihood of occurring or exceeding some threshold within a given timeframe. Prioritising which issues are the most important, and therefore should be acted on, is a different goal, and might be decided through follow-up, explicitly values-driven exercises involving representatives from government or relevant organisations (e.g. Sutherland et al., 2012).

Bringing together a cross-section of policy-makers in a follow-up exercise can be useful, not only to identify those issues that require further monitoring or evidence before being acted on, but also to encourage prioritisation of cross-organisational issues, knowledge sharing, and collaborative development of policy. Ideally, feasibility assessments of the options available would be included (as carried out in the extension of the recent Antarctic scan, Box 3.2).

3.2.6 Evaluating the process

Assessing the success of horizon scans in identifying emerging issues is challenging, and has rarely been attempted. However, a recent review by Sutherland et al. (2019) examined the first of the annual global conservation scans described in Box 3.1 (Sutherland et al., 2010) to consider how the issues identified in 2009 had developed. This was assessed using several approaches: a mini-review was carried out for each topic; the trajectory of the number of articles in the scientific literature and news media that mentioned each topic in the years before and after their identification was examined; and a Delphi-style scoring process was used to assess each topic’s change in importance. This showed that five of the 15 topics, including microplastic pollution, synthetic meat and environmental applications of mobile-sensing technology, appeared to have shown increased salience and effects. The development of six topics was considered moderate, three had not emerged and the effects of one topic were considered low.

As part of the same exercise, 12 global conservation organisations were questioned in 2010 about their awareness of, and current and anticipated involvement in, each of the topics identified in 2009 (Sutherland et al., 2012). This survey was repeated in 2018 (Sutherland et al., 2019). Awareness of all topics had increased, with the largest increases associated with microplastic pollution and synthetic meat; the change in organisational involvement was highest for microplastics and mobile-sensing technology. Perhaps the most surprising result was the number that had not heard of what are now mainstream issues: 77% for microplastics, 54% for synthetic meat and 31% for the use of mobile sensing technology. A decade ago the idea of collecting environmental data using phones was cutting-edge.

Thus, efforts have begun to examine the development of previously identified horizon-scan topics, but further research into the impact of horizon scans, and a consideration of issues that may have been ‘missed’ (not identified but subsequently emerged as important) is needed.

3.3 Making a difference with horizon scanning

Gauging the extent to which horizon-scanning outputs inform policy, future research directions and resource investments is not always straightforward and no-one has yet tested the effectiveness of this process. In instances where the primary decision-making organisation uses horizon scanning internally to assist with deliberations (e.g. scans to set priorities for a government agency), actions can be mapped directly against outcomes. In these cases, implementing the actions indicates impact. In other cases, scans can be driven by a community outside of government to set agreed future directions that can then be used to persuade external resource allocators. Even in cases where policy appears to reflect issues flagged in a horizon scan, it is difficult to trace direct influence, as inputs from multiple sources are often blended in final policy decisions without attribution. It also may take years for real-world impact to be realised. Nevertheless, there are ways in which uptake of horizon-scanning output can be encouraged.

As a starting point, horizon scanning outputs can be matched to the organisations they are most relevant to. For example, policy-makers and practitioners can come together in a follow-up workshop to assess the importance of previously identified horizon-scanning issues for their organisation (Sutherland et al., 2012, 2019). Or, the end user (e.g. policy-makers and practitioners) can be engaged in the horizon scan from the outset, as in a recent scan of research priorities for protected areas (Dudley et al., 2018). Similarly, horizon-scanning networks involving representatives from a range of government agencies, such as the Australasian Joint Agencies Scanning Network, or the UK Human Animal Infections and Risk Surveillance group, provide an ongoing forum for sharing information on new and emerging issues that potentially impact different departments and organisations. Regular meetings and reports are used to deliver this information to policy-makers in a timely way (Delaney & Osborne, 2013).

In-depth follow-up analyses of horizon-scanning issues may also help policy-makers decide which to target first. A formal risk analysis of likelihood and consequences might be most appropriate for horizon-scanning outputs that compare similarly well-defined issues, for example, comparing one invasive species with another (e.g. Roy et al., 2014). It may be more challenging if some of the issues in the candidate set are more coarse-grained than others (e.g. comparing ocean warming with a specific emerging fungal disease in some snakes). Nonetheless, risk-based prioritisation at least offers a framework for comparing and forecasting issues (Brookes et al., 2014) and for formally considering the strength of evidence for each (Garnett et al., 2016).

Simply making horizon-scanning outputs known and available to policy-makers can encourage uptake. For example, issues identified in the annual global conservation scans (Box 3.1; Sutherland et al., 2018) have previously helped inform the UK’s Natural Environment Research Council ‘Forward Look’ strategic planning, but when a decision-maker does not already have a use in mind, it may be unclear what to do with horizon-scanning information without more context and guidance. Detecting signals and potential issues is only the first step towards making a difference: further intelligence about drivers is then needed to make sense of that information. For example, incorporating available data and modelling on air traffic movements with disease surveillance data might have helped anticipate the emergence of West Nile Virus in the United States in 1999 (Garmendia et al., 2001; Brookes et al., 2014). It is the combination of horizon scanning, intelligence analysis (which provides context for the scanning output) and forecasting the chances of events unfolding that is particularly helpful in translating scanning outputs for policy-making. This can be embedded in a workflow, parts of which can be automated, such as compiling the context, narrative and structure into a digestible report on an important emerging issue (e.g. Box 3.3). When forecasting and open-analysis communities are already in place, this workflow can be delivered efficiently (Grossel et al., 2017).

Horizon scanning that occurs within organisations is evolving into a more effective tool than it was in its infancy. To facilitate the spread of best practice and reduce duplication, the UK has seen greater integration of horizon-scanning activities between different government departments, mainly in response to the Day Review (2013). The review recommended that horizon scans: (i) look beyond short-term agendas and parliamentary terms, (ii) focus on specific areas rather than broad topics in order to get more traction, (iii) are championed by those who use them in strategic decision-making, (iv) produce shorter outputs that are more likely to get the attention of senior decision-makers and (v) draw on inputs and existing analyses sourced from a ‘wide range of external institutions, academia, industry specialists and foreign governments’. The extent to which all these recommendations have been implemented is unclear, but they represent a clear set of guidelines to follow.

There are a range of other useful frameworks that can be used for translating scanning outputs including roadmapping the steps towards acting on different horizon-scanning issues, for example, by assessing the feasibility and estimating how long it would take to develop technologies needed to address particular research gaps (Box 3.2; Kennicutt et al., 2015). The Antarctic science scan and roadmap has since been used to set National Antarctic Program goals, judge the effectiveness and relevance of past investments, and guide investment of other national programmes (National Academies of Sciences Engineering and Medicine, 2015; www.nsf.gov/funding/pgm_summ.jsp?pims_id=505320&org=OPP&from=home).

3.4 Future directions

We have discussed some of the pros and cons of different approaches to horizon scanning. If using a manual approach, structured methods are essential for mitigating the social and psychological biases that human horizon scanners are prone to, especially when forecasting complex and uncertain futures (Hanea et al., 2017). Although historically it has been criticised for confusing opinion with systematic prediction (Sackman, 1975), an iterative Delphi-style approach offers the advantage of drawing on the collective wisdom of a group, while affording individuals the opportunity to give private, anonymous judgements and revise them in light of information and reasoning provided by others. Compared with other elicitation approaches, such as traditional meetings, the Delphi method has also been found to improve forecasts and group judgements (Rowe & Wright, 2001). Manual approaches could be further improved by making the search for issues more systematic. Semi-automated tools and AI will increasingly enable searches uninfluenced by the biases of the manual searcher. For example, the Dutch ‘Metafore’ horizon-scanning approach (De Spiegeleire et al., 2016), developed in The Hague Centre for Strategic Studies, already uses some automated approaches to systematically collect, parse, visualise and analyse a large ‘futures’ database to complement their manual scanning.

Future horizon scanning and intelligence gathering may also see more open-analysis, ‘citizen science’ tools becoming adopted. While organisations are increasingly scanning open-source material (including news and social media), analyses typically remain internal (Grossel et al., 2017). This means the analyses are generally not available to external users in an unfiltered form or in a timely way, which is particularly important for risks such as disease spread. Governments may opt for confidentiality for both security and political reasons. For instance, negative public perceptions about a suspected emerging herpes virus in oysters might affect trade, which might delay the disclosure of this information by authorities, in turn delaying risk mitigation actions (Grossel et al., 2017). Intelligence tools (e.g. Box 3.3) that draw on a community of users to openly analyse news and information on potentially emerging issues offer more timely and transparent synthesis of information, which encourages more responsive decision-making. Examples of this can be seen in citizen science, for example where citizen volunteers have helped analyse satellite-based information in the wake of natural disasters to help emergency responders to rapidly assess the damage (Yore, 2017). In conservation science, involving a broader community of people in a participatory process like open-analysis may also increase public support for science and the environment (Dickinson & Bonney, 2012). More open-source and open-analysis scanning tools in the future will also likely be complemented with better information visualisation and GIS (e.g. including maps that indicate where a relevant incident has taken place) (Dickinson et al., 2012), not only for identifying novel issues and monitoring issues that are already emerging, but also for locating and efficiently communicating this information.

Advanced text analytics, including text mining, will also provide a more comprehensive and systematic approach to future horizon scans. Indeed, some horizon-scanning centres, such as Singapore’s Risk Assessment and Horizon Scanning programme, already use sentiment analysis – a way of computationally categorising subjective opinions expressed in text (e.g. positive, negative or neutral) – to uncover themes in content retrieved by their analysts. Even more sophisticated text analytics are becoming available, for example, to explore areas of disagreement, conflict or debate in the text of scientific literature to help track developments in science and technology (Babko-Malaya et al., 2013). They can also be used to detect language expressing excitement about a new idea, and other indicators of emergence, such as the increasing use of acronyms and abbreviations indicating that the scientific community is beginning to accept a technology or idea as established (Reardon, 2014). Through automation, new computational tools have the capacity to process a massive volume of papers and patents to anticipate which developments will have the biggest impact in the future (Murdick, 2015). These advances in text analytics have recently led to the development of a particularly powerful open-source AI tool, Meta (https://meta.org/), to help biomedical scientists and funders to connect emerging research areas and potential collaborators and inform investment. Due to the complexity of emerging issues (and complex environment for machines to learn in), progress towards detecting issues effectively through AI is slow. Computers may never outperform humans at natural language understanding, but steady improvements in the technology, coupled with the speed at which text can be processed by computers – in a range of languages – will undoubtedly add value to horizon scanning in the future.

3.5 Acknowledgements

BCW was supported by the Centre for the Study of Existential Risk at the University of Cambridge and funded by the Templeton World Charity Foundation (TWCF). WJS is funded by Arcadia. Geoff Grossel provided useful insights into online scanning methods and applications.

Chapter Four Generating, collating and using evidence for conservation

John D. Altringham

University of Leeds

Anna Berthinussen

Conservation First

and Claire F. R. Wordley

University of Cambridge

4.1 Introduction

Does scientific evidence really matter in conservation? In this chapter we will argue that generating, collating and using scientific evidence is key to effective conservation, illustrated by a case study from our own work: how to get bats to safely cross roads. We tell the story of bat ‘gantries’ or bridges, and show what can go wrong in the absence of robust studies that test the effectiveness of conservation interventions. We will also discuss the importance of collating or synthesising multiple strands of evidence to identify the factors that make a conservation measure effective or ineffective, using a case study on underpasses under roads. Finally, we explore a key challenge – getting scientific evidence accepted and used routinely in conservation policy and practice.

Evidence takes a multitude of forms and can be defined in many ways, but in this chapter we will mostly use ‘evidence’ to refer to scientific tests of treatments or interventions, which compare the ‘treatment’ to a ‘control’ in some way and measure the effect quantitatively. We define evidence in this way as it is a broad description that can still address causality for interventions – did treatment X cause reaction Y? For example, it is not enough to know that some bats flew along bat gantries – we need to know, at a minimum, how many, and how many still flew low across the road. But more on that later.

4.2 Why do we need evidence-based conservation?

Modern medicine has many examples illustrating why the discipline needs a robust evidence base. However, basing medical treatments on scientific evidence was not always the norm. The use of randomised controlled trials to test medical treatments was initially considered unnecessary and unethical, and it was hotly contested. A good example comes from an early champion of evidence-based medicine, Archie Cochrane, who demonstrated that randomised controlled trials were necessary and that expert judgement alone could be flawed. In 1971, he presented preliminary results from a trial comparing home care for heart patients with care in the new Coronary Care Units (note that the findings may not be the same now). He had been criticised for risking the lives of patients allocated to the ‘home care’ group. What follows is in his own words:

The results at that stage showed a slight numerical advantage for those who had been treated at home. I rather wickedly compiled two reports: one reversing the number of deaths on the two sides of the trial. As we were going into the committee, in the anteroom, I showed some cardiologists the results. They were vociferous in their abuse: ‘Archie’, they said ‘we always thought you were unethical. You must stop this trial at once’. I let them have their say for some time, then apologized and gave them the true results, challenging them to say as vehemently, that coronary care units should be stopped immediately. There was dead silence and I felt rather sick because they were, after all, my medical colleagues.

(Maynard & Chalmers, 1997)

Results such as these – where the preferred treatment of the time did not work, or actually made things worse – are used to demonstrate why scientific studies of impacts are important when treating people. A growing body of literature suggests that impact studies are also necessary for treating the health of the biosphere, although the ‘gold standard’ of randomised controlled trials is not always possible in this discipline (Pynegar et al., 2018). As we test more and more measures to conserve species and habitats, we find that many do not work. For example, studies have shown that widely used methods to make water voles move prior to building works were ineffective, risking accidental killing of the protected mammals (Gelling et al., 2018); that reintroduction programmes of species from macaws (Volpe et al., 2017) to tamarins (Beck et al., 1991) have resulted in high or total mortality for the released animals; and that artificial bat roosts built to replace those destroyed during building works often failed to attract any bats and, even when occupied, hosted about half the number of bats that the destroyed roost had (Stone et al., 2013). These results underline the need to test conservation solutions and not to simply assume that good intentions will lead to good outcomes.

Our case studies focus on the environmental impacts of roads. Road construction has been shown to harm animals through habitat degradation, loss and fragmentation, direct mortality and barrier effects (Laurance et al., 2009; Benítez-Lόpez et al., 2010; Rytwinski & Fahrig, 2012). Figure 4.1 summarises these cumulative impacts, which are likely to act at different rates and through a long extinction debt. Unfortunately, studies on a wide range of habitats and taxa from grasslands to vertebrates show that many road mitigation options simply do not work. A growing list of papers points to not only poor design of mitigation and monitoring, but a wider context of poor target setting, weak implementation, inadequate reporting and poor or absent enforcement (e.g. Rundcrantz, 2006; Tischew et al., 2010; Beebee, 2013; Drayson & Thompson, 2013; Villarroya & Puig, 2013). We will not address all of these factors in this chapter, but they are important to consider when asking why ineffective measures have persisted for so long.

Figure 4.1 The multiple causes of bat population reduction by road construction and the delayed response (extinction debt).

Adapted from Forman et al. (2003).

4.3 Case study: bats and roads

Why do we need mitigation for bats crossing roads? In the EU, all bat species have been protected under the EUROBATS agreement since 1994, in recognition of the declining populations of many species. As a consequence, whenever populations may be adversely affected by human activity, impact assessment and mitigation are a legal obligation. Over the last 10 years evidence for significant effects of roads on bats has grown and the need for effective mitigation has become increasingly evident (e.g. Altringham, 2008; Russell et al., 2009; Lesiński et al., 2010; Berthinussen & Altringham 2012b, 2015). There are clear specifications within EUROBATS to mitigate against the impacts of roads on bats.

4.3.1 The need to test mitigation: bat gantries

The EUROBATS commitment to mitigate against the impact of roads on bats is very positive, but are the mitigation strategies being used actually working? Early studies assessed the use by bats of underpasses and overpasses primarily built for other purposes, such as to carry minor roads, footpaths or streams. If bats were seen near to these structures it was generally assumed that they were effective mitigation tools (Highways Agency, 2001, 2006; reports reviewed by O’Connor et al., 2011). Underpasses, culverts, footbridges and bridges for vehicles, all of various sizes, were widely adopted as mitigation solutions (Figure 4.2). Many were not subsequently surveyed for use by bats, or qualitative surveys were written up in often confidential reports. Many studies reported ‘use’ – small numbers of bats observed in underpasses or flying over bridges of various kinds, without reference to the number still crossing the road unsafely, or not crossing at all (see Highways Agency, 2006; O’Connor et al., 2011), and many lacked convincing definitions of use. This meant that future projects could not learn from the success or failure of previously built mitigation structures.

Figure 4.2 Two underpasses found to vary in effectiveness in guiding bats safely under roads. (a) An effective underpass on the A590, Cumbria, UK; (b) an ineffective underpass on the A66, Cumbria, UK. Boxplots show the number of bats crossing per survey using the underpass and crossing over the road above at safe and unsafe heights (above and below 5 m, traffic height). The variable success of underpasses underlines the need to understand the details of conservation interventions; in this example, the location of the underpasses impacted on how effective they were.

From Berthinussen and Altringham (2012b).

In addition to multi-use structures, some ‘bespoke’ structures were built and ‘bat gantries’ or ‘wire bridges’ (Figure 4.3) were widely adopted. Bat gantries were assumed to act as navigational aids to echolocating bats, encouraging them to continue using existing ‘commuting routes’ from roosts to feeding areas (which often follow linear features such as hedgerows) after road construction, but lifting them above the traffic. Ideally, crossing points should be built on known bat commuting routes determined by pre-construction surveys, as bats tend to be faithful to particular routes. However, many were built away from known bat commuting routes for engineering reasons, to fit in with landscape topography, to combine bat routes with minor roads or footpaths, or simply to reduce cost. It was assumed bats would find the new crossing points (Highways Agency, 2001), and in some cases new hedge planting was designed to guide them to these structures. In many guidance documents, environmental statements and mitigation plans it was implicit, or even explicit, that the bats would respond as predicted (Highways Agency, 2001; Limpens et al., 2005).

Figure 4.3 Two bat gantry designs: (a) wire mesh design on the A11, Norfolk, UK; (b) wire and ball design on the A590, Cumbria, UK. Boxplots show the results of surveys carried out to test the effectiveness of the gantries in guiding bats safely over the road. Data were recorded for the total number of bats crossing per survey, the numbers crossing at unsafe heights (below 5 m, traffic height) and the numbers using the gantry according to two definitions of ‘use’ (flying within either 2 m or 5 m of the wires above traffic height). The bat gantry story neatly demonstrates the need to test conservation interventions before rolling them out on a wide scale.

From Berthinussen and Altringham (2012b, 2015).

In 2008, JDA was asked to provide evidence to a public inquiry for the effectiveness of these strategies (Altringham, 2008). No quantitative evidence was found to suggest that any of the strategies implemented were effective in protecting bats, particularly at the population level. However, neither was there evidence to suggest that they were ineffective. This prompted us to conduct our own research to determine the effects of roads on bats and the effectiveness of mitigation (Berthinussen & Altringham, 2012a, 2012b, 2015).

In our research we emphasised the difference between qualitative assessments of the ‘use’ of a structure by a small number of bats and measures of effective protection at the population level. We also stressed that the number of bats present pre-construction is rarely assessed, meaning that post-construction bat numbers may already be a fraction of what was there before. We proposed two broad measures of effectiveness: (1) measurements of local bat activity and of the movement of bats along severed commuting routes before and after road construction, to assess landscape-scale impact and the permeability of new roads; and (2) measurements of the effectiveness of the crossing structures – the proportions of bats that use them to cross safely. Our research was limited by logistics to the second measure – do mitigation structures guide bats safely across roads?

The headline result was that ‘wire and ball’ bat gantries did not alter the behaviour of bats crossing roads – they were wholly ineffective (Berthinussen & Altringham, 2012a; Figure 4.3b). This was a disturbing finding, as over the previous decade about 15 gantries had been built in the UK and continental Europe. Although our study showed that one design of bat gantry was ineffective, it was suggested that other designs would have greater success in guiding bats to fly at safe heights above roads. Our next study found that ‘wire mesh’ gantries (Figure 4.3a, of a different design to the ‘wire and ball’ structures) were equally ineffective (Berthinussen & Altringham, 2015).

In summary, a mitigation measure widely used for over a decade was essentially untested and subsequently shown to be ineffective. This underscores the need for rigorous testing of the measures that we implement in the name of conservation. We also found (albeit based on a small sample size) that building all types of crossing away from known commuting routes, even with new planting to guide bats to them, was unsuccessful (Berthinussen & Altringham, 2012a). This is important, as it shows that the location of mitigation measures is as important as the measure itself – effective measures need to be implemented with a good understanding of the local context. Furthermore, we found evidence that some underpasses were used by a high proportion of bats, and that the one green bridge tested in the UK – a large structure planted with trees, shrubs and ground cover – was used by over 90% of bats crossing the road in that area, suggesting that effective ways to allow bats to safely cross roads do exist.

4.4 Synthesising evidence

The bat gantry case study provides some insight into why we need to rigorously test the effectiveness of measures aiming to protect the natural world. However, this is just the first step towards implementing a truly evidence-based approach to conservation. The next step is to systematically bring together all the evidence, from many studies, on particular conservation measures. This approach is also borrowed from evidence-based medicine, where it has proven to be a lifesaver.

One of the most important developments in evidence-based medicine was the Cochrane Collaboration, an organisation set up to conduct systematic reviews of the scientific evidence on topics such as how well different treatments worked. In medicine – as in conservation – natural variation in populations means that it often takes large numbers of replicates for beneficial or detrimental effects to become apparent. Modern doctors, making potentially life-changing decisions, want to have the information on every study on a particular treatment to hand, not just the results from a single trial that may not be representative. The goal should be the same in conservation: to bring together all the evidence for an intervention to assess whether it works, whether it does harm, or whether it only works in certain situations or with certain variations of the intervention.

There are many examples of the importance of collating evidence in medicine. For example, a systematic review on cot death or sudden infant death syndrome (SIDS), using the studies already available in the 1970s, could have saved the lives of an estimated 60,000 babies. Due to a lack of evidence synthesis and an overreliance on expert opinion, medical practitioners advised parents to put children to sleep on their fronts until the 1990s, when studies and reviews led to the realisation that this sleeping position increased the risk of SIDS (Gilbert et al., 2005).

In conservation, collating or synthesising the data is as critical as it is in medicine (Sutherland et al., 2004). While the most rigorous method, systematic review, is very important (see Chapter 7, the Collaboration for Environmental Evidence and Mistra EviEM), more cost-effective methods of collating the evidence may also be desirable in this underfunded discipline, where the evidence itself can be scarce and variable in quality (Sutherland & Wordley, 2018). ‘Synopses’ published by Conservation Evidence (www.conservationevidence.com) follow one such method, known as subject-wide evidence synthesis (Sutherland & Wordley, 2018). Researchers draw up lists of all the interventions that could benefit a given taxa or habitat, classified according to potential threats based on IUCN criteria (Threats Classification Scheme Version 3.2); the scientific studies for the effectiveness of each intervention are then collated and summarised. For example, we produced the Bat Synopsis (Berthinussen et al., 2013, updated 2019), which provides key messages and summaries of the relevant studies, to help conservationists see which interventions for bat conservation are likely to be the most effective, and under which circumstances. The summary of this synopsis in What Works in Conservation (Berthinussen et al., 2018) takes this a step further, by using expert scoring to categorise the interventions based on levels of effectiveness, certainty in the evidence available and potential harms.

The first Bat Synopsis (Berthinussen et al., 2013) listed 78 interventions that could be implemented to conserve bats, covering areas as diverse as logging, roost provision and wind turbine operation regimes. No evidence for effectiveness was found for 48 of the 78 interventions, many of which are used routinely in the UK and elsewhere. This does not mean they are ineffective, but simply that they had not been tested quantitatively when we checked the literature. For a further 12 interventions the evidence was too limited for assessment. This demonstrates the scarcity of experimental evidence for many possible management actions, severely limiting the ability of conservationists, ecological consultants, developers and government agencies to undertake evidence-based conservation or mitigation.

Of the 18 remaining interventions, 14 had some proven value as conservation tools for bats. These included using selective logging instead of conventional logging, turning off wind turbines at low wind speeds and minimising light pollution. An update to this synopsis was published in 2019 (Sutherland et al., 2019), expanding the list of interventions to 190 and adding new studies that were published in the intervening years. There are many interventions which have had valuable evidence added in this update, but we have not yet seen a shift to a majority of interventions being tested via multiple high-quality experiments.

4.4.1 Example of evidence synthesis: road underpasses

For many of the interventions addressed in the Bat Synopsis, our greatest contribution was to demonstrate that no evidence existed for the efficacy of these measures – hopefully spurring more research and a more critical eye towards choosing conservation measures. But for a handful, we could begin to tease out what made an intervention effective in some circumstances but not others – one of the many benefits of summarising multiple studies. One such intervention is the use of underpasses to get bats to cross roads safely.

In the 2013 Bat Synopsis we found four studies, from Germany, Ireland and the UK, which between them showed that at least nine bat species used underpasses (none purpose-built for bats), with up to 96% of the bats crossing through underpasses rather than the road above (although this varied greatly) (Berthinussen et al., 2013). By summarising the key details of each study, we can see that some species use underpasses frequently while others do not appear to use underpasses at all, and that only a few species appear to use small underpasses, such as drainage pipes of diameter less than 1.5 m. There are indications that effectiveness increases with diameter and when underpasses are placed on known bat commuting routes – conclusions supported by ongoing studies (Davies, 2019). The 2019 update of the Bat Synopsis added two further studies, which tested much larger underpasses and still found the largest structures to be the most effective, but also explored the differing responses of various functional guilds of bats. These details are critical. Further testing and refining of underpasses, followed by evidence synthesis, should help to ensure that future underpasses are as effective as they can be.

4.5 Getting the evidence used

We are trying to bring this work, demonstrating the importance of generating and using evidence on the effectiveness of interventions, to as large an audience as possible, to ensure that those responsible for commissioning, designing, approving and testing mitigation structures are aware of it. The bat gantry studies have been reported in national newspapers, radio and television. This was achieved through press releases, by approaching media contacts directly and by being approachable when contacted. The work has also been reported in several books and papers (Altringham, 2011; Abbott et al., 2015; Altringham & Kerth, 2016; Sutherland & Wordley, 2017). JDA and AB have run seven workshops for practitioners on road mitigation measures for bats and talked at over 10 conferences in the UK and abroad. CFRW has mentioned this study in around 50 talks to conservationists and government agencies and used it as an example in an opinion piece on evidence use in conservation (Sutherland & Wordley, 2017). The Bat Synopsis and What Works in Conservation, which contains a summary of the Bat Synopsis, have also been widely promoted.

This awareness resulted in tens of thousands of views of the paper and relevant parts of the Bat Synopsis, and this exposure has translated into further successes. The impact of early work (Berthinussen & Altringham, 2012a, 2012b) led to a Defra-funded project to develop better mitigation monitoring protocols (Berthinussen & Altringham, 2015) and a statutory conservation agency guidance note summarising the protocols. The approximately £1 M spent on bat gantries in the UK as of 2017 (Sutherland & Wordley, 2017) was brought up in the House of Lords by Lord John Krebs in January 2018, who used it to demonstrate why the UK government’s 25-year environment plan needed to explicitly commit to being evidence-based. Some road-building projects have taken heed of the evidence. The A40 Penblewin to Slebech Park Improvement in Wales opted to mitigate impacts on bats using underpasses of varying sizes, many built on known commuting routes, and funded more rigorous monitoring (Davies, 2019).

However, not everyone is listening. Despite widespread reporting of the ineffectiveness of bat gantries in 2012, six gantries of a ‘wire-mesh’ design were built in Norfolk, England in 2014 at a reported cost of £350,000. These were probably planned before the 2012 paper was published, but plans were not modified in light of this study. In 2015 these gantries were also shown not to work (Berthinussen & Altringham, 2015). Nevertheless, seven more gantries are under construction (as of 2018) at a cost of over £1 M on the North Norfolk Distributor Road (MacDonald, 2014). In another example, the environmental statement for the proposed and controversial extension to the M4 across the Gwent Levels in Wales (Welsh Government, 2016) draws extensively on our 2015 Defra report (Berthinussen & Altringham, 2015). However, it proposes numerous culverts for bats which, by the authors’ own admission, are almost all too small to be used by the target bat species. In addition, most will not be on known commuting routes.

Furthermore, there are still inadequate mechanisms in place to assess the effectiveness of mitigation measures. A feature of many environmental statements and mitigation plans is the absence of a monitoring plan capable of assessing mitigation success or failure. There is frequently no monitoring plan at all. This appears to be due to a reluctance or inability of government agencies to enforce effective monitoring, a reluctance on the part of many developers to pay for monitoring and a lack of understanding about how to design and conduct monitoring that is fit for purpose. As a result, developers and taxpayers spend money on unproven mitigation with no prospect of improved understanding.

4.5.1 Why is evidence ignored?

Why are proven methods rejected, often in favour of methods that have been demonstrated not to work? Why is there an apparent reluctance to seek out, use or accept evidence, or to collect it, among some decision-makers, including some government agencies and ecological consultants? Sutherland and Wordley (2017) explored a few general psychological and structural reasons for this phenomenon, and more detail on this topic is given in Chapter 9 of this book. Here we share some of our own experiences of the failure to use evidence in road mitigation.

The real or perceived higher financial cost of effective mitigation solutions is one concern. Mitigation consumes a very small part of the total cost of a road-building project, but mitigation and monitoring are obvious targets when budgets are tight or overrun. Effective mitigation may or may not be more expensive than ineffective options, but ineffective mitigation is simply a waste of resources.

A desire to simplify the planning and implementation of mitigation is another reason why some parties are reluctant to challenge or change accepted approaches. Road building is complex, making off-the-shelf, approved mitigation solutions an attractive option. Being able to implement development projects as quickly and cheaply as possible can make mitigation a tick-box exercise – complying with regulation at minimal cost may be more important than implementing effective mitigation. Mitigation solutions such as bat gantries can be designed and built relatively cheaply and, if experts say they will work, then they fulfil all legal requirements and may be assumed to require little or no ‘expensive’ monitoring. To question their effectiveness can put in jeopardy budgets, work schedules, building specifications, even the project itself. A reluctance to listen to objections is understandable, but not excusable. Consultant ecologists can be placed in a difficult position. Their scientific training, personal concern about nature and professional standards all demand unbiased assessment. However, their livelihoods depend upon contracts from developers who are frequently not obliged to commit to effective mitigation and monitoring.

Finding, evaluating and applying the evidence on mitigation strategies can also be a challenge. With the existence of freely available downloadable material (such as the Bat Synopsis) in a concise, jargon-free form, decision-makers should be more aware of what works and what does not. However, ecological consultants and statutory agency staff still need time to find, read and digest the information, and require some scientific training to evaluate the evidence. The difference between quantitative evidence and anecdote is not always understood and ‘professional judgement’ may be relied on even when it runs counter to the evidence. However, it does not have to be painful for developers, consultants or planners to improve on current practice. Adoption of good mitigation practices early in a project can avoid the problems of making corrections during the project, and investment in effective technologies may lower the costs of solutions such as large underpasses and green bridges.

4.6 How can evidence use in mitigation projects be improved?

First, there should be a key requirement that mitigation structures are tested for effectiveness, not just use, and a quantitative bar set for effectiveness (Berthinussen & Altringham, 2015). Ecologists employed to assess mitigation effectiveness must be prepared to shun options proven to be ineffective. Professional bodies must fully support ecological consultants in implementing those measures shown to be effective, and sanction members who use methods known to be ineffective. Improvements may be much more evident if the enforcing statutory agencies are willing and able to deny planning permission to development projects that have poor mitigation strategies. There should be real commitment from governments to pledge to conserve species and habitats using evidence-based measures and discarding measures proven to be ineffective. This may require additional resources to assess existing and proposed legislation against evidence syntheses.

To identify effective and ineffective solutions there is a clear need for dedicated funding for rigorous tests of interventions. Monitoring interventions often requires long-term commitment which, in turn, requires adequate long-term funding. This could come from developers and government agencies, but a greater recognition by academic funding bodies of the value of applied questions would also have a huge impact. PhD projects could be encouraged to have applied components, testing interventions. Research council funding for academics to address applied questions of conservation importance and communicate them to practitioners would have a huge impact.

Greater power for statutory agencies to enforce existing laws, check up on implementation and demand replacements for ineffective solutions would dramatically improve mitigation effectiveness. A framework with greater incentives for developers to show that their mitigation has been effective would be beneficial. These could include a requirement to make the results of mitigation monitoring for effectiveness public, penalties for failures to do so and awards for new, proven effective solutions.

While many of these goals may not be realised in the near future, we can all promote approaches to conservation that are evidence-based and effective. If enough of us do it, it might just change the world.

Chapter Five Understanding local resource users’ behaviour, perspectives and priorities to underpin conservation practice

E.J. Milner-Gulland , Harriet Ibbett

University of Oxford

Paulo Wilfred

The Open University of Tanzania

Hans Cosmas Ngoteya

Landscape and Conservation Mentors Organisation

and Peni Lestari

Wildlife Conservation Society Indonesia Program

5.1 Introduction

Most of the chapters in this book focus on how best to bring science into policy, often at the national scale and mostly with a developed-world perspective. Ensuring that national policy frameworks are conducive to conservation is vital, but it is also important to improve the effectiveness of science in supporting conservation interventions on the ground. Small-scale interventions aiming to change the behaviour of local resource users in developing countries make up a large proportion of global conservation effort and funding (Brockington & Scholfield, 2010). These types of intervention are challenging to do well, and often do not produce the desired results (Larrosa et al., 2016). Typically, there is little scientific input into either the design or evaluation of these projects, and evidence of effectiveness is limited (Roe et al., 2015). Small organisations in developing countries may not have the capacity or confidence to implement scientifically informed design and monitoring, and supporting them to collate and learn from evidence may not be a major priority for researchers or donors. Increased sharing of insights and techniques to support more robust and effective interventions could transform grassroots conservation (e.g. Woodhouse et al., 2016). In this spirit, we use case studies from four locations around the world to illustrate some of the challenging steps involved in understanding conservation issues and designing suitable interventions. These steps are often skipped or not made explicit, but are critical to success; they ensure that interventions have a strong foundation in evidence, making it more likely that their desired impacts are achieved.

First, we explore how to collect robust information on the prevalence of illegal resource use, as a first step towards understanding the extent of the problem, using a case study of bird hunting in a Cambodian grassland. Next, we consider how to bring together different sources of information to understand both resource use and local perspectives on conservation, using a case study from Tanzania. These two case studies about evidence gathering lead on to the next stage: intervention design. We start with an example, also from Tanzania, of developing a Theory of Change for a conservation intervention, in which the process by which actions lead to a desired result is identified, assumptions are made clear and the progress of the intervention towards its desired impact can be monitored. Finally, we explore the challenges of implementing one particularly prevalent intervention type – alternative livelihoods projects – using an example of a shark fishery in Indonesia. Together, these case studies provide a vivid illustration of the ways in which conservation researchers and practitioners are combining efforts to ensure that interventions are based on robust evidence and therefore more likely to succeed.

5.2 Asking questions about sensitive topics

Moderating human behaviour is critical to conservation success (Gore, 2011; Milner-Gulland, 2012). However, if we are effectively to change human behaviour, we must first ensure we understand the nature of the behaviour we want to change. Central to this is determining both the prevalence of behaviours that are detrimental to biodiversity, and the characteristics of the people engaging in these behaviours. This is essential to ensure managers efficiently allocate resources to tackle threats, and that behavioural change interventions target the right audiences with the right incentives (St John et al., 2010, 2015). However, obtaining such information can be extremely challenging, especially if the behaviour in question is illegal (Gavin et al., 2010).

A common approach to ascertaining the true extent of illegal behaviours is asking direct questions (e.g. Gandiwa, 2011; Kiffner et al., 2015). Other studies mask the sensitivity of questions about illegal behaviours by mixing them with less-sensitive questions about other livelihood activities (e.g. Martin et al., 2012; Mgawe et al., 2012; Kiffner et al., 2015). Although direct questioning may help to cast some light on the nature of natural resource exploitation, it runs the risk of bias from untruthful responses (Nuno & St John, 2015). Respondents may be scared to answer questions honestly for fear of incriminating themselves, or the possible repercussions they might face from revealing their behaviour. They may avoid answering questions altogether, terminate interviews early or underreport activities. If respondents do answer sensitive questions, social desirability bias may lead them to moderate their responses so their actions appear more socially acceptable. This is especially true of data captured in group settings, where pressure from peers may prevent others speaking freely and truthfully about their activities. It is also important to consider the ethical implications of directly asking respondents about their illegal activity; research has an ethical responsibility to ‘do no harm’, yet asking such questions can cause respondents to directly implicate themselves in illegal activities, potentially leading to severe consequences.

Indirect questioning has started to become more widely used in conservation science in response to some of these challenges. The method comes from psychology, and has been used when asking questions about sensitive issues such as drug use and racial prejudice (Imai, 2011). The technique enables interviewees to respond in such a way that the interviewer cannot directly determine whether they have participated in the activity. Instead, data provide estimates of prevalence at the population level, affording both the respondent and the researcher greater levels of protection.

One form of indirect questioning increasingly applied in conservation is the Unmatched Count Technique or Item List Technique (see Gavin et al., 2010; Nuno & St John, 2015). The technique works by devising a short ‘control’ list of three to five innocuous items that are non-sensitive but relevant to the research topic, and a treatment list which also contains the sensitive item of interest (Figure 5.1). The sampled population is randomly shown either the control or treatment list. Respondents are asked to report only the total number of items that apply to them. Because only a number is reported, the researcher has no way of knowing which specific items apply to a given respondent. The difference in the mean number of items reported by the two groups provides an estimate of the proportion of respondents engaging in the sensitive behaviour (Thomas et al., 2015).

Figure 5.1 Using the Unmatched Count technique to ask about illegal bushmeat hunting in the Ugalla Wildlife Reserve, Tanzania. Picture by Paulo Wilfred.

5.2.1 Case study: Bengal florican

Ibbett et al. (2019) used the Unmatched Count Technique to investigate prevalence of illegal behaviours and to identify the characteristics of resource users in central Cambodia. In the dry season, the seasonally inundated grasslands surrounding the Tonle Sap lake are home to the world’s largest remaining population of Bengal florican (Houbaropsis bengalensis), a critically endangered bustard species (Birdlife International, 2015). Recently, agricultural abandonment, scrub advancement and the emergence of dry-season rice – a form of intensive, irrigated rice cultivation – have dramatically reduced grassland cover. The Tonle Sap florican population is estimated to have declined by 44–66% since dry-season rice was first cultivated on the floodplain in 2004 (Packman et al., 2014). However, conservation managers lack adequate understanding of the drivers of dry-season rice expansion. There is also evidence that hunting, a historic driver of decline, may persist in local communities (Packman, 2011).

Ibbett et al. (2019) used a mixed-methods approach to investigate these issues. Because hunting is potentially a sensitive activity (hunting wildlife is illegal in protected areas), the Unmatched Count Technique was selected to identify the prevalence of bird hunting and florican egg collection. The Unmatched Count Technique was combined with direct questioning and delivered through a household questionnaire, which captured information on household demographics, livelihood activities and awareness of bird species. Due to the florican’s scarcity, Unmatched Count Technique questions concerned the hunting of larger grassland birds in general, with questions phrased as ‘How many of the following animals/types of egg have people in your household caught in the last 12 months?’ A warm-up question about different fruits consumed in the household was asked in order to introduce respondents to the technique.

A sample of 616 households across 21 villages was secured. The warm-up question identified a significant difference between control and treatment groups, suggesting the technique was working as expected. However, no significant difference was identified between control and treatment groups for egg collecting or large bird hunting, suggesting the prevalence of these activities did not significantly differ from zero. When questioned directly, just 8.6% of households reported hunting birds in the previous 12 months, the majority of which were small, abundant game birds, such as buttonquail and ducks. Those that reported hunting birds were more likely to come from households which also collected other wildlife products, such as frogs and crickets.

5.2.2 Lessons learnt

While indirect questioning techniques avoid some of the pitfalls of traditional techniques, they are not without limitations. In this case, the Unmatched Count Technique failed to detect the presence of bird hunting, unlike direct questioning. This may be explained by the generally low prevalence of this activity and the probabilistic nature of the approach, which means that confidence intervals are large. Part of the issue is that the direct question was about bird hunting in general, and showed low levels of hunting of common species, while the Unmatched Count Technique question investigated targeted hunting of large bird species. Only one or two incidences of hunting large bird species were directly reported. Similar experiences of inability to estimate prevalence have been reported by others when using the Unmatched Count Technique to investigate illegal activities (e.g. Nuno et al., 2018). Therefore, the Unmatched Count Technique is unlikely to be useful when estimating the prevalence of an extremely rare activity. Indirect questioning is also not a panacea for sensitivity; if an activity is highly sensitive, particularly if it violates social norms, respondents may still not answer truthfully when the item is in a list; this can even result in negative estimates for prevalence (e.g. Fairbrass et al., 2016).

Compared to other indirect techniques, such as the Randomised Response Technique (see Nuno & St John, 2015), the Unmatched Count Technique is often preferred because it can provide higher estimates of prevalence, is simple to understand and adaptable, and thus useful in developing countries where levels of illiteracy may be high (Gavin et al., 2010; Nuno & St John, 2015). Despite this, the concept can still be difficult for respondents to grasp. Respondents may be wary, especially if they have previously had negative encounters with researchers. Taking time to thoroughly talk through the technique, using a warm-up question and explaining each list item is essential to avoid these issues. Often, conservation researchers rely on the help of translators or local research assistants. Selecting the very best help available and providing extensive training to assistants is essential in order to prevent information from getting ‘lost in translation’. Local research assistants can also provide knowledge to ensure designs are appropriate. This is particularly helpful when working in illiterate communities, or when relying on pictorial prompts.

5.3 Triangulating different sources of evidence to build a rounded picture

Social research methods such as focus groups, interviews and household surveys are increasingly being employed to investigate illegal behaviours and profile resource users (Young et al., 2018). The current decade has seen an increase in the use of these mixed methods approaches to gain a more holistic understanding of resource use (e.g. Kahler & Gore, 2012; Harrison et al., 2015). A combination of perspectives, using both qualitative and quantitative methods, is commonly preferred.

5.3.1 Case study: Ugalla Game Reserve

Ugalla Game Reserve (hereafter Ugalla; 5000 km²) in western Tanzania is predominantly miombo woodland. Its conservation value is high, serving as habitat for a wide range of species (UGR, 2006). It is part of the Malagarasi–Muyovosi Ramsar Site, and facilitates connectivity between protected areas in western Tanzania (Kalumanga, 2015; Riggio & Caro, 2017). The main legal activity in the reserve is trophy hunting, mostly by overseas tourists. A number of different approaches are used to conserve Ugalla, including irregular anti-poaching patrols and seasonal permission for fishing and beekeeping activities (July–December). These also aim to attract local support for conservation and build a sense of ownership of the reserve among local people. However, recent studies suggest that this conservation approach is ineffective (Wilfred & MacColl, 2015; Wilfred et al., 2017). Unauthorised use of natural resources (including poaching, illegal logging and fishing) is common and local communities hold negative attitudes towards the reserve and its management. In an attempt to shed light on the prevalence of illegal behaviours and inform the management of Ugalla, multiple research methods were used to gather relevant information. Between 2013 and 2016, household surveys and focus groups were conducted in villages around Ugalla, along with a survey of signs of illegal activity undertaken across the Protected Area.

For the household surveys, 533 households were randomly sampled in 2016 in the vicinity of Ugalla. The Unmatched Count Technique was used to estimate the prevalence of illegal behaviours (logging, illegal hunting and honey-gathering). The survey also included questions on households’ perceptions of the main threats to Ugalla and its wildlife, and what communities would do differently to improve Ugalla’s management effectiveness. Six single-sex focus groups from six randomly selected villages within 20 km of the Ugalla boundary, each with 4–6 participants, were conducted to verify findings from the household survey. Free-listed threats to Ugalla were ranked in decreasing order of their importance, and each threat was then divided by the total number of threats to calculate the salience score (Papworth et al., 2013). The overall score for each threat was obtained by calculating the average salience score across the focus groups. The greater the salience, the more important the threat.

For signs of illegal activity in the reserve, 10 patrol tracks were randomly selected in 2014. Six transect starting points were placed at 3000-m intervals along each road. At each point, two 1500-m transects were walked on opposite sides and perpendicular to the road. Signs of illegal activity (e.g. tree stumps, sawpits, meat smoking racks, snares, trees felled for honey extraction, fish smoking racks, poacher camps) were noted 50 m either side of the transect line (Figure 5.2).

Figure 5.2 Paulo Wilfred and his research assistant recording an illegal meat smoking rack in Ugalla Wildlife Reserve.

The Unmatched Count Technique results suggested that poaching and illegal logging were performed by 28% (SE ± 6) and 20% (SE ± 5), respectively, of surveyed households; 18% (SE ± 6) of respondents gathered honey. The top four threats to Ugalla, as identified by respondents, were poaching (40% of respondents), logging (39%), fishing (11%) and honey gathering (8%). Of the top four threats to Ugalla free-listed and ranked by focus groups, logging had the highest salience (S = 0.5), followed by poaching (0.45). Within the reserve, 867 illegal activity signs were encountered. Signs related to logging had the highest frequency, followed by honey gathering, poaching and fishing (Figure 5.3). These results indicate that levels of illegal activity in Ugalla are high. The different methods consistently suggest that logging and poaching are the commonest illegal activities.

Figure 5.3 Signs of illegal activity encountered inside Ugalla Game Reserve in 2014. Total signs = 867.

Of the activities that survey respondents and focus group participants said they would undertake if they were the Ugalla manager, the most common recommendations were to: improve the well-being of people around Ugalla (17% of respondents, S = 0.11); ensure that local people have adequate land for their livelihood activities (16%, S = 0.35); promote local participation in conservation (16%, S = 0.13); improve law enforcement (15%, S = 0.14); raise conservation awareness (15%, S = 0.14); and improve local people’s relations with reserve managers (12%, S = 0.54).

5.3.2 Lessons learnt

Paulo Wilfred’s research in Ugalla started nearly a decade ago with the overarching objective of informing conservation management. This long-term research suggests that local communities are knowledgeable about illegal activities and keen to participate in conservation efforts. For example, during household surveys, villagers from unsampled households sometimes expressed their desire to share their views and experiences about natural resources. Accordingly, researchers can facilitate liaison between reserve managers and local people.

Although Paulo’s research exposes the situation on the ground, we are not yet able to connect these observations to a good understanding of the drivers of illegal behaviour or the governance context framing reserve management. To fulfil such an objective, more targeted research is required. Ideally, this should focus on individual activities, rather than trying to investigate all illegal activities at once. Different activities are conducted by different groups of people with different rationales and link to different governance issues.

The methods applied in Ugalla were resource-intensive. For example, Unmatched Count techniques typically require high sample sizes (see Nuno et al., 2013); more than 500 households were surveyed in this study, which was all that time and funding allowed. Surveying for illegal activity signs was also challenging, because it was difficult to estimate the time the signs had been present in the environment and different signs have different biases (e.g. rangers remove snares during their normal anti-poaching patrols, potentially leading to underestimates).

The main lessons learnt from Ugalla were as follows.

• Be interdisciplinary! Don’t be afraid to use ecological survey methods, for example incorporating a field-based survey into the research design. This can provide a great opportunity to cross-validate findings from social research.

• Conservation researchers preferring mixed methods should not be overambitious. Instead, they should be realistic, choosing techniques carefully and planning activities based on the resources available, following a robust pilot study.

• While doing household surveys and focus groups, it is critical to use experienced research assistants who are neutral in the community but familiar with the study area. A survey of illegal activity signs also requires experienced field assistants, so information is collected accurately and consistently.

• Both focus group discussions and household surveys should be kept relatively short and simple to minimise participant fatigue.

5.4 Developing a Theory of Change for an intervention

It is vitally important to be clear about why we think that our intervention is the right thing to do, and what barriers there might be to success, before we start. This understanding needs to be set out in a logical way, so that it is understandable and appealing to project staff and donors, and so that it can later be tested. There are a number of approaches which can be used, falling under a general heading of causal chain models (Qiu et al., 2018). One such approach is Theory of Change (Center for Theory of Change, 2018), which shows how a project can reach its desired impact and goals through different pathways of change. It provides indicators that can be tested, thereby supporting evaluation of a project’s success or failure. This is useful both for internal and external users, to understand what works, and to guide the allocation of project resources.

5.4.1 Case study: Vijana na Mazingira

In 2016, Hans Cosmas Ngoteya designed a retrospective Theory of Change for Vijana na Mazingira (VIMA), the local conservation project which he runs in the Katavi–Rukwa ecosystem of western Tanzania. The project targets youths aged 12–35, with the goal of reducing pressure on natural resources from poaching, deforestation and encroachment. The Theory of Change was designed to support an evaluation of the effects of the project on attitudes, awareness and conservation behaviours by youths aged 18–35 participating in VIMA’s conservation education and alternative livelihood projects (Figure 5.4).

Figure 5.4 Hans Cosmas Ngoteya (second from right) setting up a beehive with local youths, as an alternative livelihood project.

In order to achieve a project’s desired impact, it is necessary first to understand the motivations for engaging in the behaviour that the project is aiming to modify. There are a number of frameworks available from social psychology that represent the factors that interact to influence behaviours. One of the most widely used in conservation is the Theory of Planned Behaviour (St John et al., 2013). Hans used the Theory of Planned Behaviour to identify the different factors underlying the motivations of the VIMA project’s recipients (Figure 5.5). Based on Hans’ local knowledge and understanding of the project, a Theory of Planned Behaviour framework was developed for four desired project impacts, each of which represented a desired behavioural change. The Theory of Planned Behaviour was then used to identify how VIMA’s activities might tackle the different motivations underlying each behaviour.

Figure 5.5 A Theory of Planned Behaviour diagram illustrating the factors underlying the poaching behaviours of individuals targeted by the VIMA project.

A clear understanding of the motivations behind the behaviour, engendered by the Theory of Planned Behaviour exercise, can enable conservationists to map out the pathways of change the project should focus on, thereby generating a Theory of Change. The Theory of Planned Behaviour gives a representation of what underlies an individual’s behaviour, and this can be used to develop a Theory of Change for the planned intervention. In this case, the Theory of Planned Behaviour exercise highlighted that, typically, youths in Katavi–Rukwa viewed poaching as a way to feed their families and generate an income through bushmeat or ivory sales. Therefore, an intervention that developed alternative livelihood programmes could be an effective approach. This could include training youths in new income-generating activities (input), thereby providing alternative income sources (output), which will reduce their dependence on natural resources (outcome) and ultimately reduce their poaching behaviour (impact; Figure 5.6). At each step of this pathway lie assumptions; for example, that any alternative income source will replace, rather than supplement, income from hunting (Table 5.1).

Figure 5.6 Theory of Change for VIMA project showing interventions at the bottom and different pathways to reach the desired impacts. Numbers 1–10 are assumptions along the pathways of change (listed in Table 5.1).

Table 5.1 Assumptions underlying the Theory of Change

1.	Participants understand the education they are given
2.	If someone is educated about environmental issues it will improve their attitude towards conservation
3.	Knowledge about conservation issues leads to a decrease in acceptance of environmentally harmful behaviours
4.	There is dissemination of information from VIMA participants to the remainder of the community
5.	If someone’s attitude towards conservation improves, they will reduce their unsustainable resource-use behaviour
6.	If communities are against unsustainable resource use, illegal resource exploitation will decrease
7.	VIMA’s alternative livelihood programmes can be put into practice and generate income
8.	There is opportunity for the rest of the community to become involved in the alternative livelihood projects
9.	Alternative livelihoods will be used to reduce unsustainable use of natural resources
10.	Decreasing dependency on natural resources will reduce poaching, encroachment and deforestation

Baseline surveys, focused on the elements of the Theory of Planned Behaviour (attitudes, knowledge, social norms), provide a set of indicators against which change engendered by the intervention can be measured. Progress through the Theory of Change can also be monitored, using a set of more process-based indicators. For example, an input indicator might be the percentage of VIMA’s target audience engaged in the alternative income activities, an output indicator might be the income generated from the alternative livelihood, the outcome might be measured as improvements in household livelihood security and the impact might be measured using an indirect questioning technique such as the Unmatched Count Technique to quantify change in poaching prevalence.

5.4.2 Lessons learnt

The requirement for robust evaluations of the effectiveness of conservation interventions is becoming more and more apparent (Sutherland et al., 2011). Practitioners are required to ensure that their activities are based upon the best available evidence and designed for accountability and learning. However, many small NGOs (such as Hans’ organisation, Landscape and Conservation Mentors’ Organisation) may not feel that they have the capacity to design and implement evaluations that are both user-friendly and robust enough to be useful for adaptive management. Lacking a rigorous framework for articulating goals and assumptions, it is easy to drift through interventions without having either a strategic plan or a means of measuring success. This can lead to ineffective interventions and failure to capture the changes engendered in order to learn and adapt and demonstrate impact to funders. The development of a Theory of Change for the VIMA project enabled Hans to identify his assumptions, and develop methods to collect information which can be used to monitor future impact and test assumptions against a relevant baseline.

5.5 Exploring alternatives to illegal behaviour

One of the key lessons learnt in the VIMA project was the importance of having a clear understanding of the motivations behind behaviour. Unfortunately, not all conservation projects that involve communities take this approach when designing an intervention. For example, alternative livelihood projects have long been used as strategy for reducing local threats to species, habitats or resources of conservation concern. Alternative livelihood projects are designed to reduce the prevalence of behaviours that are considered environmentally damaging and unsustainable (Wright et al., 2016). However, a systematic review of alternative livelihood projects conducted by Roe et al. (2015) found insufficient evidence to understand when, where or why alternative livelihood projects work. Even though there is uncertainty regarding the effectiveness of alternative livelihood projects, they continue to be a key strategy in both terrestrial and marine conservation. However, the assumptions on which they are based are often unrealistic; for example, that the alternative livelihood projects will substitute for the undesirable behaviour, that the resource users are a homogeneous group, and that targeting interventions at individuals will scale up to population-level change in pressure on resources (Wright et al., 2016).

In marine conservation, a common response to perceived over-fishing is to provide alternative employment for existing fishers. This requires that the assumption of substitutability holds, so that fishers will willingly and happily settle into a new way of making a living (Pollnac et al., 2001; Pollnac & Poggie, 2008). Pollnac et al. (2001) added that this is based on the assumption that fishing is a hard and undesirable occupation and hence an employment of last resort, that fishers are among the poorest of the poor and that the poor care little about the type of job they have as long as they make enough to live.

5.5.1 Case study: shark fishers in Tanjung Luar

Fishing pressure is generally considered to be the main cause of the decline of shark populations globally (Stevens et al., 2000; Robbins et al., 2006; Dharmadi et al., 2015). Indonesia is the world’s largest shark producer, with annual average production of 106,000 tons in 2000–2011, contributing 13% of global shark production (Dent & Clarke, 2015). Although the exact number is unknown, it is assumed that many Indonesian fishers are heavily dependent on shark fisheries as a source of income and food. However, shark production in Indonesia has been declining in recent years (Sub Directorate of Capture Fisheries Data and Statistics, 2016), which could be leading to a decline in income and livelihood security for fishers.

From 2014 to the present, the Wildlife Conservation Society (WCS) Indonesia Programme has carried out a study of shark fishers in Tanjung Luar, a shark-fishing community in East Lombok, in order to understand whether providing alternative livelihoods could help to reduce fishing pressure on sharks. Tanjung Luar is one of the main shark landing sites in Indonesia. It is home to a targeted shark fishery comprising approximately 50 boats employing surface and bottom longlines and one of the biggest fish markets on Lombok Island, with more than 5000 fishers using it to sell their catch. Fishing is the main livelihood of Tanjung Luar’s population and there are at least 150 households heavily dependent on the shark industry, either as fishers, meat processors or traders. Shark fishers in Tanjung Luar use 4–25 gross tonnage boats, with three or four crew members, and the average fishing trip is 14 days.

Due to growing international concern regarding their conservation status, several shark and ray species have been listed on CITES Appendix II. As a CITES member, Indonesia is required to implement management measures, such as quotas, size limits and export bans to ensure that international trade in these species is not detrimental to wild populations. These measures could have negative impacts on the income and livelihood security of Tanjung Luar’s fishers, who are already vulnerable to market fluctuations, particularly in export markets (Jaiteh et al., 2017). WCS Indonesia Programme’s study aimed to: (1) collect data on biological and operational characteristics of the fishery (Figure 5.7), (2) understand shark fishers’ current socioeconomic status and aspirations, (3) explore alternative livelihood options and (4) create dialogue between fishers and the management authorities.

Figure 5.7 WCS Indonesia team members measuring guitarfish at Tanjung Luar port.

Photo provided by WCS-Indonesia.

Livelihoods options explored with the fishers included diversifying the target catch to more resilient species (e.g. squid, tuna and reef fish) and tourism, yet WCS Indonesia Programme’s surveys showed that shark fishing offered higher revenues than other fisheries. An independent fisheries assessment by Masyarakat dan Perikanan Indonesia also showed similar results (MDPI, 2017). Tanjung Luar was known for its squid fishery in the 1980s, but the number of squid fishermen has increased rapidly, increasing competition and making the addition of new fishers unsustainable (MDPI, 2017). Some fishers in Tanjung Luar who catch tuna or skipjack mentioned that their catch is also declining, and The Indian Ocean Tuna Commission classifies yellowfin tuna as overfished (IOTC, 2017). Some shark fishers have already started to fish for groupers and snappers on the side, but the value of this catch is far less than their earnings from sharks. Tourism is promising, but the industry is still under-developed. To date, identifying feasible alternatives that provide economic incentives to shift away from shark fishing has proven challenging, as there are no legal or sustainable marine alternatives that offer similar profits.

Our research showed that shark fishers wish to remain shark fishers. Fishing is the only skill they know, and most of them said that they would continue to fish as usual even if their catch declined by 50%. Our landings survey showed that some commonly caught sharks are over-exploited. When findings were shared with fishers, although not all agreed with the results, shark fishers acknowledged that it is now harder to catch sharks and the sharks that are caught are smaller, a view also shared by shark fishers in eastern Indonesia (Jaiteh et al., 2017). The Tanjung Luar fishers’ response is not surprising, as similar reactions were also reported by Pollnac and Poggie (2006), with fishermen refusing to leave their existing fishery even though their incomes were declining; it is potentially their best option in the short run if they are still making a profit.

5.5.2 Lessons learnt

Based on the results of this research, instead of deploying alternative livelihood projects for shark fishers in Tanjung Luar, WCS’s Indonesia Programme chose to:

1. (1) strengthen the existing fisher institutions, which focus on tourism development, in order to help that industry to develop, become more attractive and profitable;

2. (2) maintain close interaction with shark fishers by regular home visits and conducting informal meetings; and

3. (3) facilitate formal meetings between shark fishers and the management authorities, to foster dialogue on developing management measures that ensure the sustainability of both shark and ray populations and fishers’ livelihoods.

It is challenging to establish a direct connection between livelihood interventions and conservation. Rather than trying to find new livelihoods, sometimes it is more appropriate to focus on enhancing existing livelihood strategies which do not involve exploiting the natural resource of concern, targeting those most vulnerable to conservation-imposed resource access restrictions (Wright et al., 2016). It may also be possible to establish a clearer link between livelihood sustainability and conservation as a means of building good community relations, as we opted to do. It is important to have a clear pathway demonstrating how an intervention is expected to lead to the desired outcome, e.g. by using theory of change to design the intervention after gaining a thorough understanding of community dynamics.

5.6 Discussion: interlacing research and practice

The four case studies presented here take us from research to practice; in so doing, they illustrate how integrated the two are. By starting with a strong theoretical framework (such as the Theory of Planned Behaviour) underpinning an intervention’s Theory of Change, unwarranted assumptions can be avoided, such as those which plague alternative livelihoods projects. Engaging with resource users before embarking on interventions can reveal dead ends, as illustrated in Tanjung Luar, where plans for an alternative livelihood project needed to be replaced by a more indirect process of advocacy and engagement with different parties, while building capacity for a livelihoods shift. A clear understanding of what the actual problem is, based on evidence rather than supposition, is vital; the example from Cambodia suggested that hunting was actually not a major threat to floricans, enabling conservation practitioners to focus on other threats.

Although a range of techniques is available for collecting information to underpin management, these should not be applied lightly. As the Ugalla example showed, the ideal of using mixed methods to gain a nuanced understanding takes time and resources, as well as expertise. Approaches such as the Unmatched Count Technique can look superficially appealing and easy to administer, but there are technical challenges in developing appropriate item lists, administering the questions in a way that makes respondents comfortable, and in data analysis. Even then, as the Cambodian example shows, the results may not be as informative as might be hoped. Time invested in foundational studies is well spent, but not all small NGOs can afford extensive research. Even then, however, it is possible to develop a robust Theory of Change, as a tool for exposing assumptions and supporting ongoing monitoring and evaluation, as the VIMA example showed.

Our case studies have specific lessons, but they also tell universal stories. The role of research in facilitating positive interactions between managers and local people is an interesting observation that was seen in both Ugalla and Tanjung Luar, while both the Cambodian and Ugalla case studies highlighted the importance of good local research assistants. All four case studies emphasised how research and practice need to intertwine more often and more routinely. This will enable conservationists (whether from governments or NGOs) to think through their interventions in advance, use appropriate methods to understand existing behaviour and local perspectives on ways forward, and thereby design locally appropriate, participatory interventions that support adaptive management.

Chapter Six Mobilisation of indigenous and local knowledge as a source of useable evidence for conservation partnerships

Pernilla Malmer , Masterson Vanessa

Swedbio, Stockholm Resilience Centre

Beau Austin

Charles Darwin University

and Maria Tengö

Swedbio, Stockholm Resilience Centre

6.1 Introduction

Rapid and interlinked changes in the biosphere, including degradation of the biodiversity and ecosystems that underpin human well-being, are reported with increasing regularity. As such, there is an urgent need for conservation initiatives that are capable of countering the speed and veracity of change, while meeting the needs of human societies on a crowded planet. While significant advancements in scientific knowledge in the fields of sustainability and conservation continue to be achieved, the forecasted rate of rapid ecological and social change requires the production of innovative mechanisms for management and policy.

One way of contributing to new solutions in a timely manner is to more effectively mobilise multiple knowledges, values and governance systems that can complement Western approaches to science. Together these can extend the collective knowledge base and contribute to collaboratively designing ways forward for looking after people and the biosphere. Compared with Western-based approaches, indigenous and local knowledge systems represent alternative ways of learning from and with the environment, through close and continuous observation framed by distinct worldviews with particular strengths and limitations (like all knowledge systems). Knowledge is embodied by the actors and in their practices, tools, and technologies, as well as in the institutions that organise the production, transfer and use of knowledge (Cornell et al., 2013). There has recently been more attention focused on the urgent need for science and policy to recognise and mobilise the knowledge of indigenous people and local communities who steward substantial biodiversity across the globe (Brondizio & Le Tourneau, 2016; Mistry & Berardi, 2016). Collaborative ways for mobilising knowledge and learning across diverse knowledge systems can contribute complementary knowledge, innovations and new solutions. Involvement of multiple actors and knowledges can strengthen usefulness and legitimacy in decision-making and implementation (Sterling et al., 2017a; Gavin et al., 2018).

In this chapter, we draw attention to the potential for mobilising local and indigenous knowledge systems, institutions and actors in ways that allow meaningful use of their knowledge about landscapes and their functions as evidence for conservation. By doing this, we propose that innovative and collaborative mechanisms can be designed and implemented that will create opportunities for long-term sustainable governance and conservation of biodiversity.

We introduce the Multiple Evidence Base (MEB) approach to guide the design and implementation of conservation partnerships that enable engagements with indigenous and local knowledge as evidence as an entry point to promote sustainable governance of interrelated ecosystems and human well-being (Tengö et al., 2014, 2017). The approach was developed to guide inclusive processes for collaborations across knowledge systems, based on equity and usefulness for all actors involved. It emphasises that indigenous, local and scientific knowledge systems are complementary, equally valid and useful for informing sustainable governance of biodiversity and ecosystems. The MEB focuses on the theoretical and practical potential for collaborative knowledge-weaving processes to mobilise indigenous and local actors, institutions and practices to achieve long-term conservation and sustainability targets. We argue that collaborative approaches to conservation must be equitable and fair to be effective in the long term (Brondizio & Le Tourneau, 2016; Sterling et al., 2017a; Gavin et al., 2018).

The utility and value of the MEB approach will be discussed in light of its aim to support more informed and efficient local, national and international policy processes and governance decisions for the integrated benefits of conservation, sustainable use and human well-being. We describe the current and potential role that a MEB approach may have in enhancing the efficacy of conservation science and policy by clarifying and strengthening synergies with indigenous knowledges and practices. To achieve this, we first review the peer-reviewed and grey literature to reflect on the extent of uptake of the MEB and how it has been applied in both science and policy-practice processes. Second, to illustrate the approach and reflect on successes and practical challenges, we take a deeper look at three case studies of piloting a MEB approach. The cases demonstrate the potential for the MEB approach to be used as both a framing tool for collaborative partnerships and a practical guide to weaving multiple knowledge systems. Lastly, we discuss ways forward to nurture conservation and mobilise partnerships that build on knowledge collaborations. We find that a MEB approach has potential to support the inclusion of a wider range of evidence in conservation practice, strengthen active participation of local actors and improve conservation partnerships through the recognition and revitalisation of local knowledge systems and governance.

6.2 The need for new approaches to collaborative conservation

There is a long history of attempts to reconcile conservation objectives with local livelihoods in integrated development and conservation processes, which have often been framed as ‘win–win’ opportunities with social–ecological benefits (Adams et al., 2004). In the conservation literature, the importance of involving local people is well established, with mounting evidence that processes that meaningfully engage local people are more likely to succeed in protecting biodiversity (Waylen et al., 2010; Sterling et al., 2017a) and that failure to do so can lead to lack of trust and commitment, project failure, and in the worst case, lingering conflicts (Oldekop et al., 2015). While many indigenous peoples and local communities continue to be evicted from their ancestral lands and experience colonisation in the name of conservation, there is now a move towards recognising their connections to land and endogenous obligations to care for it as synergetic with biodiversity conservation outcomes (Knox, 2017). This provides a foundation for enabling local people and conservation organisations to be strategic allies. Furthermore, there is increasing evidence that involving local actors in monitoring enhances management responses at local spatial scales, and increases the speed of decision-making to tackle environmental challenges at operational levels of resource management (Danielsen et al., 2010; Sterling et al., 2017a).

Despite these generally acknowledged realities about the usefulness of engaging with indigenous peoples and local communities, they are often included as stakeholders in conservation, without recognition of their knowledge and expertise (Danielsen et al., 2010). In the literature much attention is given to the uniqueness and utility of indigenous and local knowledge systems, which is often holistic, providing an understanding of integrated social–ecological systems, biocultural values and belief systems (Sheil et al., 2015; Sterling et al., 2017a). However, in practice, there often exists scepticism about the contemporary existence and/or effectiveness of indigenous and local knowledge as useful evidence in conservation. Similarly, holders of indigenous and local knowledge can be sceptical of the claims generated through western scientific approaches due both to the unfamiliarity of the epistemic practices employed and recent or ongoing experiences of colonisation and disempowerment (Nadasdy, 1999; Johnson et al., 2015; Kealiikanakaoleohaililani & Giardina, 2016; Mistry & Berardi, 2016).

6.3 The multiple evidence base approach: connecting knowledge systems for the benefit of conservation and human well-being

The need to engage with diverse sources of knowledge for conservation has been recognised in high-level science–policy processes, such as the Convention on Biological Diversity, and the Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services (IPBES). From the outset, IPBES had the ambition to recognise and respect the contribution of indigenous and local knowledge to the conservation and sustainable use of biodiversity and ecosystems (Díaz et al., 2015). This was used as a window of opportunity to start an open dialogue to explore current divides between indigenous, local and scientific knowledge systems, and to elicit methods for collaborations based on equity, reciprocity and usefulness for all involved (see Tengö et al., 2014). A science–policy–practice dialogue process brought together knowledge-holders and experts from diverse knowledge systems, convened by SwedBio at Stockholm Resilience Centre in collaboration with key partners representing indigenous peoples and local communities, such as the International Indigenous Forum on Biodiversity and the African Biodiversity Network. The active engagement from these networks, representing a diversity of knowledge systems and linking practices on the ground with global policy and science, created legitimacy and recognition of outcomes from the dialogues. The starting point was the pivotal dialogue meeting prior to the establishment of IPBES in the indigenous territory of Guna Yala, Panama, where essential principles for exchange across knowledge systems were identified: trust, respect, reciprocity, equity, transparency and free prior and informed consent (Tengö & Malmer, 2012). Since then, the MEB approach has developed in parallel to the IPBES, while carefully paying attention to other interests and needs of the partners.

The MEB can be understood as a deep approach to collaborative knowledge-sharing that explicitly acknowledges that challenges are fundamentally due to different perspectives and practices concerning human–nature relationships, approaches to knowledge validation, knowledge governance and who qualifies as an ‘expert’. Also, it recognises that scientists have tended to dominate the design and implementation of collaborations across knowledge systems both historically and contemporarily (Nadasdy, 1999; Mistry & Berardi, 2016). Another key component of an MEB approach is its emphasis on the need for mobilisation and validation of knowledge within knowledge systems themselves. That is, if scientific methods that often are specific and partial are applied to local knowledge that is practical, multidimensional and holistic, there is a risk of omission, misinterpretation and rejection of critical and useable knowledge.

The MEB approach views different knowledge systems as complementary and emphasises that joint analysis assists in working both with convergence and divergence (e.g. Molnár et al., 2016a; Hohenthal et al., 2018). For example, Molnár et al. (2016a) highlight that when discussing approaches to conservation in the Hungarian steppe, local herders focus on primarily utilitarian purposes, such as how they can manage the behaviour of their grazing animals in order to promote the health and diversity of grass assemblages for production. In comparison, conservationists working in the same landscapes focus almost solely on the protection of the plants themselves, with little regard to the impact on grazing animals. If this difference is ignored, or framed as a problem, it has the potential to create tension when attempting to collaboratively design and implement conservation initiatives in the region. Conversely, these different perspectives can be worked together to provide an enriched picture of exactly what is necessary for maintaining and enhancing biodiversity and social–ecological system function in the steppe.

In order to build evidence – whether new knowledge or existing – that is legitimate and useful for all actors in such collaborations for conservation, there is a need to engage with local knowledge systems and knowledge-holders from the outset, co-defining a common problem and facilitating equitable engagement through all activities, including mobilising and assessing knowledge. This process is outlined in the three phases of the MEB approach (Figure 6.1a). Collaboratively analysing and interpreting the complementary evidence from diverse sources is a way to triangulate information, strengthen legitimacy and relevance of existing knowledge and build a base for further learning.

Figure 6.1 The Multiple Evidence Base approach in action. (a) The three phases of a MEB approach: joint problem formulation, generating an enriched picture with contribution from multiple sources of evidence and joint analysis and evaluation of knowledge (Tengö et al., 2014). (b) Actors, institutions and processes are at the core of the five tasks required for successful collaboration across diverse knowledge systems. The different colours of the lines and dots in parts (a) and (b) represent different knowledge systems, or streams of knowledge within knowledge systems (Tengö et al., 2017).

As guidance for how to implement an MEB approach, five tasks were identified as critical (Figure 6.1b; Tengö et al., 2017). First, to mobilise knowledge – to ensure that the knowledge is articulated, validated internally and free to be shared with others. Second, to translate knowledge, reciprocally, so that all actors can comprehend each others’ knowledge and where it is derived from. Third, to negotiate, to jointly address convergence and divergence between knowledge systems, and the extent to which the latter can be resolved, for example by understanding differences in underlying assumptions and values (Gagnon & Berteaux, 2009; Molnár et al., 2016a). Fourth, to synthesise. Here we emphasise synthesis based on a joint process that does not require that all knowledge is validated by one knowledge system (e.g. empirical validation by science). Lastly, to apply – and this is where we iterate the need to recognise the different needs and interests by different actors. Knowledge collaborations need to be designed in a way that is is perceived as useful and leads to constructive outcomes for all involved. The bridging of knowledge systems therefore requires the creation of settings for exchange of multiple forms of knowledge and learning across key aspects of the system (Figure 6.1b). We view the outcome as weaving – knowledge collaborations that respect the integrity of each knowledge system while working them together for practical collaboration (Johnson et al., 2016; Tengö et al., 2017). In the next section, we use literature and our own experience to evaluate and discuss implementation of the MEB approach, with a specific focus on describing the outcomes in terms of evidence applied in conservation partnerships.

6.4 Reviewing the impact of the MEB in conservation and sustainability

The literature on knowledge collaborations for conservation and sustainability is wide-ranging. To focus on collaborations across knowledge systems (indigenous, local and scientific knowledge systems) and to generate further insights into the application, challenges and usefulness of a MEB approach, we reviewed articles that cite Tengö et al. (2014) or that mention ‘Multiple Evidence Base’ in the academic literature, represented by Scopus (123 articles), and the grey literature (219 results), represented by Google Scholar (as of 2018–02-01).

The results of this review demonstrate that the MEB approach has contributed to a general move towards broader participation of knowledge-holders in multi-level ecosystem assessments (Díaz et al., 2015; Nesshöver et al., 2016), as well as citizen science, the importance of the plurality of knowledge systems in conservation (Prado & Murrieta, 2015) and knowledge application in public policy and resource management (Bruckmeier, 2016). This is part of a ‘shift that has occurred in the science–policy–society interface with a move towards greater inclusivity, and efforts to transcend traditional reductionist approaches’ (Jabbour & Flachsland, 2017, p. 196).

The MEB approach is finding traction in diverse discussions including citizen science (Buytaert et al., 2014) and community-based monitoring (Johnson et al., 2015; Lyver et al., 2017), collaborative management and decision-making (Mathevet et al., 2016), community-based conservation (Nkambule et al., 2016; Sterling et al., 2017a), measuring resilience (Quinlan et al., 2016; Sterling et al., 2017b), approaches to modelling global change processes (Verburg et al., 2016), indigenous autonomy and cultural revitalisation (Gonzales, 2015), value pluralism in ecological economics (Martín-López & Montes, 2015; Kenter, 2016; Pascual et al., 2017), biocultural values and diversity (Gavin et al., 2015; Sterling et al., 2017b) and political ecology, law and environmental justice (Gambon & Rist, 2018; Hohenthal et al., 2018).

The majority of articles reviewed (51 percent) engage with the MEB approach in a relatively superficial manner to illustrate that combining multiple knowledge systems is a sustainability challenge. The literature is awash with programmatic articles with calls to include, combine and integrate knowledges to find solutions to sustainability problems (e.g. Balvanera et al., 2017; Vasseur et al., 2017). However, still very little attention is paid to exactly how this will be done. Additionally, 20 percent of articles reviewed represent collaborative processes in practice but do not apply a MEB approach. Many articles view actors as stakeholders and talk about ‘open participation and open consultation’ (e.g. Livoreil et al., 2016) rather than addressing their role as knowledge-holders and experts and the need for equitable platforms for engagement, mobilisation and translation of indigenous and local knowledge.

The MEB approach has also received significant attention in the grey literature and science–policy–practice community. For example, it is called for as a way of ensuring equitable participation for indigenous, local and scientific knowledge in monitoring of the Convention on Biological Diversity. For example the Convention’s Aichi target 18 on traditional knowledge, innovation and practices, along with the Community-Based Monitoring and Information Systems, is a bottom-up approach developed by indigenous peoples and local communities to contribute their experiences and observations through monitoring (CBD, 2014; Farhan Ferrari et al., 2015). Further, a MEB approach has been encouraged in traditional knowledge inventories, as well as in the development of safeguards for biodiversity financial mechanisms and Reducing Emissions from Deforestation and Degradation (REDD+) under the United Nations Framework Convention on Climate Change.

To illustrate the implementation of the MEB approach in the literature, we have selected a small set of pertinent case studies. Table 6.1 presents an analysis using key features of the MEB approach – joint problem formulation, validation within knowledge system and the five tasks illustrated in Figure 6.1b.

Table 6.1 Articles applying a multiple evidence base in literature

Article citation	Issue investigated including location	Multiple evidence base	Evidence of joint problem formulation and usefulness for all (Tengö et al., 2014)	Evidence of validation within knowledge systems (Tengö et al., 2014)	Evidence of application of the five tasks for successful collaboration across diverse knowledge systems (Tengö et al., 2017). 1 = mobilise, 2 = translate, 3 = negotiate, 4 = synthesise, 5 = apply
Austin et al., 2017	MEB approach to enable enriched picture of progress of an Indigenous Land and Sea Management programme run by the Wunambal Gaambera people in the Kimberley, Australia	Informing the evidence base of the Wunambal Gaambera Healthy Country Plan using western scientific and local indigenous knowledge. Parallel integration of western and indigenous monitoring data/information to support co-production of enriched picture of country and management activities by the Uunguu Monitoring and Evaluation (M&E) Committee	Collaborative and multiple evidence-based M&E committee designed the approach to conducting evaluation of progress and assessment of key targets	Yes, each stream of knowledge was internally validated and cross-checked through collaborative self-assessment by Uunguu M&E Committee	1, 2, 3, 4, 5 Local indigenous knowledge was mobilised through the planning process and the M&E committee; translation of information from various sources via monitoring methods; further negotiation and translation occurred within M&E committee; all knowledge streams synthesised through M&E committee meetings and reporting processes; and applied through adaptive management of the Healthy Country Plan
Nguyen et al., 2017	Sustainable management of eroding mangrove-dominated muddy coasts in Vam Ray, Hon Dat district, Kien Giang Province, Vietnam	Partnership between government agencies, scientists and local communities. Methods included literature review, semi-structured interviews, participatory community meetings, participatory diagramming and thematic analysis. The introduction and analysis of different knowledge systems are undertaken in participatory community meetings, semi-structured interviews, field visits, photovoice and debriefings	All parties agreed to co-formulate the problems and use local and scientific knowledge to generate and pilot new knowledge for solving them. It was agreed to build local capacity and to utilise as many local resources as possible for developing the fence and nursery construction, to solve serious erosion problems affecting the community	Local knowledge of, e.g. Melaleuca fence construction was validated by local experts based on their experience. However, facing new challenges in the community created interest in other knowledge such as scientific knowledge. The collaboration led to new knowledge about fencing for controlling coastal erosion	1, 2, 5 Local knowledge held by individuals regarding traditional Melaleuca fences, and local contexts were systematically collected and brought together with the relevant scientific knowledge in relation to sedimentation and coastal dynamics in Kien Giang, Vietnam into ecologically based, cost-effective strategies for successfully controlling coastal erosion
Robinson et al., 2016	Water management in territories of aboriginal people connected to the Girringun Indigenous Corporation (Girringun) in northern Australia	Participatory maps created in workshops of Girringun support staff, Aboriginal rangers, some Girringun elders who are also artists and some of the authors to determine the values, knowledge and management aspirations of participants for their ‘fresh water country’. Second workshop to discuss the values that the participants had for native plants and trees and to identify risks to those values and the attributes of partnerships that support these values	Co-research approach, in which Girringun representatives worked with the researchers to select participants and design the participatory mapping workshops, advising on an appropriate focus, location and design for each workshop	Yes, the integrity of each indigenous knowledge system was maintained throughout process	1, 2, 3, 4, 5 Girringun representatives and scientists created individual maps to mobilise and translate the knowledge needed for Girringun and its associated tribal groups to assess two distinct issues of concern. Collective watershed maps were also used to negotiate knowledge and although there was some variety in the information shared by different participants, the integrity of each indigenous knowledge system was maintained throughout the process. Synthesis of themes occurred through creation of targeted research ‘products’, including a one-page summary, that could be used by the research team and the Girringun Indigenous Community to translate the results of the project in a way that was useful to the participants, Girringun and the wider natural resource management community
Smith et al., 2017	Developing conservation strategies for pollinators in the context of pollinator decline in Orissa, India	Peer-to-peer validation of trends and statements distilled from focus groups including 50 smallholder subsistence farmers, including tribal people, who have personal and procedural knowledge of crop production, and rural advisors; anecdotal network. This was in preparation for integration with scientific knowledge from other regions	The problem (a potential pollinator crisis) was defined by scientists, who recognised the dearth of information on diversity of crops and pollinators and together with farmers and rural advisors collated traditional and local knowledge on the same	Yes. Peer-to-peer validation of indigenous knowledge of trends and statements distilled from focus groups including 50 farmers and rural advisors	1, 2, 4 Traditional knowledge of crop diversity and pollinators was elicited and internally validated, providing a consensus on knowledge which was collated for integration with scientific knowledge
Molnár et al., 2016a, 2016b	Mitigation of conflicts between cattle herding and conservation management of salt-steppe and wood pastures in Hungary	An inventory of objectives and practices of herders (representing traditional knowledge) and conservationists and ethnobotanists (scientific knowledge) were collected by participatory knowledge co-production in teamwork with the co-authors. Possible resolutions to potential conflicts were suggested. Methods include: (1) participatory observation, (2) semi-structured interviews with herders and conservationists, (3) co-author herders and conservation managers completed and clarified the contents of the tables in two rounds	Herders and ethno-ecologists jointly formulated the problem, and potential solutions were suggested by all parties	Yes. Herder interviews colleagues that explain their observations and experiences and jointly validate the relevance for the issues. Less focus on conservation manager’s validation. However, importance of integrity, equity and reciprocity in their knowledge-based interactions highlighted	1, 2, 3, 4 Herders’ and conservationists’ knowledge of practices were elicited in interviews. Herders’ perspectives were mobilised and translated through film. Data were negotiated among diverse author group and synthesised for joint publication
Strangway et al., 2016	A registry monitoring an aboriginal subsistence fishery in the Cree community of Waskaganish (Waskaganish Voluntary Anadromous Cisco Catch Registry) within the Environmental Impact Assessment (EIA) Follow-up Phase after diversion of the Rupert River for hydroelectric plant at Nûtimesânân in Northern Quebec, Canada	Collaboration between Hydro-Quebec and the impacted community. Bringing together different monitoring reports of cisco including the Voluntary Catch Registry of the Crees of Waskaganish First Nation, biological monitoring and complementary studies	Unclear. State-owned utility, Hydro-Quebec, proposed a monitoring programme after authorisation of river diversion which would include studies on cisco spawning success, as cisco harvesting is key for Waskaganish First Nation community’s cultural identity and subsistence economy. Four community members hired as monitors to collect data on cisco catch	Yes. In the voluntary cisco catch registry programme, compiled data were presented to land users at the end of the fishing season for interpretation and validation. Shared observations regarding the fishing season, fishing success and any other comments, as well as how the data will be presented outside of the community, were discussed and included in the final Registry reports	1, 2, 4, 5 Fishers propose mitigation measures to increase fishing success under new flow rates, and stakeholders assess their potential, with the results of the various cisco monitoring programmes, including the Voluntary Registry, featuring prominently in the decision. Once measures are implemented, the Registry programme is used to evaluate their effectiveness by collecting catch data

Notes: Examples assessing the experience of applying a MEB approach, showing the issue investigated, the location and the multiple evidence base, in literature that either quoted Tengö et al. (2014) or referred to MEB. The review examined evidence of joint problem formulation and usefulness for all stakeholders (defined by Tengö et al., 2014), evidence of internal validation within knowledge systems (defined by Tengö et al., 2014) and evidence of application of each of the five tasks for successful collaboration across diverse knowledge systems (as defined by Tengö et al., 2017).

The cases illustrate that, in different contexts, specific phases of the MEB approach presented by Tengö et al. (2014, 2017) are more or less useful, and are operationalised in different ways. The process of co-defining the problem and questions together with all knowledge-holders appears to be a challenge not taken up in all cases, often with scientists or project proponents defining a problem, and then approaching indigenous and local knowledge-holders and local communities through consultation sessions to join and support the collaboration (e.g. Strangway et al., 2016; Lyver et al., 2017; Smith et al., 2017). However, other papers do emphasise the critical role of joint problem formulation for the success of conservation interventions (Brondizio et al., 2016; Galvin et al., 2016).

Maintaining the integrity of diverse knowledge systems throughout collaborative knowledge processes also appears to be a particular challenge in science-driven processes. Actively thinking about what validation of knowledge within knowledge systems means (rather than using science to validate local knowledge) and how it may be embedded in practice is absent from most papers. There are notable exceptions that explicitly reflect upon this challenge (e.g. Austin et al., 2017) and suggest new approaches, such as peer-to-peer validation by farmers (Smith et al., 2017; Table 6.1). Other papers do not address this explicitly, but still engage with how local knowledge systems evaluate knowledge (e.g. through interactions with internally acknowledged experts and their local institutions) (Molnár et al., 2016; Nguyen et al., 2017; see Table 6.1). Additionally, joint discussion and analysis of data across knowledge systems has sometimes been incorporated through formal consultation structures or committees (e.g. Strangway et al., 2016; Austin et al., 2017; Reed & Abernethy, 2018). The articles also illustrate the progress in development of methods to facilitate the phases and activities defined in Tengö et al. (2014, 2017) to combine and relate multiple data through e.g. participatory scenario planning, focus groups (Danielsen et al., 2014), fuzzy cognitive maps and community monitoring with digital devices (Brammer et al., 2016). The use of art (Rathwell & Armitage, 2016; Polfus et al., 2017), participatory maps (Robinson et al., 2016) or film (Molnár et al., 2016) to mobilise, translate and present knowledge on an equitable platform has facilitated joint analysis and negotiation. Articles also illustrate practical ways of maintaining equity, such as creating research agreements or protocols concerning intellectual property; free, prior and informed consent; the roles and responsibilities of each member of the project team (Robinson et al., 2016); and recognising indigenous and local knowledge-holders as authors on scientific articles (Molnár et al., 2016a; Smith et al., 2017; Table 6.1).

The citations suggest that the mobilisation and translation activities suggested by Tengö et al. (2017) have had particular resonance in the conservation and sustainability literature. There has been consistent progress towards the explicit mobilisation and translation of indigenous knowledge and worldviews (Gonzales, 2015; Vogt et al., 2016; Horstkotte et al., 2017; Timoti et al., 2017). In this way, the mobilisation of multiple knowledge systems contributes to a movement towards environmental justice and pluralism in decision-making (Hohenthal et al., 2018), as well as recognising indigenous peoples’ autonomous actions towards dealing with climate change (Gonzales, 2015).

In the next section, we use three in-depth case studies to further explore the value of a MEB approach to contribute to conservation partnership based on diverse sources of knowledge.

6.5 Exemplifying MEB cases and reflecting on lessons learned

Here we present three case studies that have explicitly implemented a MEB approach (Table 6.2). The first two set out processes to address local conservation and development issues. The third is an international dialogue meeting where the aim was to create a platform to discuss a fundamental crux in conservation globally – how to realise synergies between human rights and biodiversity conservation, and support local people and conservation organisations in becoming strategic allies.

Table 6.2 Summary of MEB tasks to guide knowledge collaborations (Tengö et al., 2017) as applied in the three case studies

MEB phases	Multiple evidence-base case examples
	4.1. Piloting the MEB: Tharaka’s river is running dry	4.2. Mobilising indigenous knowledge systems for saltwater country across the Kimberley, Australia	4.3. Justice and conservation: Global Dialogue on Human Rights and Biodiversity Conservation
1. Mobilise Develop knowledge-based products through a process of innovation and/or engaging with past knowledge and experience	The preparatory process for the ecocultural mapping, where the elders of the clans start to engage and document their experiences The process of making ecocultural maps and calendars, which mobilised and synthesised knowledge on the landscape and how it has changed over time	Project objectives and research activities identified by an intercultural collaborative Working Group (WG) to ensure focus on local priorities. At the individual community workshop level, each of the indigenous ranger groups designed the specific activities, venue and participants. Focus group discussions and knowledge-holder interviews were selected as appropriate methods for indigenous people to use their knowledge to inform the process. Ranger groups were all equally resourced to facilitate and participate in research activities	Participation occurred before the actual dialogue, through interactions over internet. Preparation of indigenous community representatives to present and mobilise knowledge about ecology as well as human rights among the participants. The contributions from indigenous communities were planned to be presented during walking workshops in the Oigek Community. However, an outbreak of Marburg virus meant the dialogue was moved to Eldoret. The stories were told by community representatives attending the workshop
2. Translate Adapt knowledge products or outcomes into forms appropriate to enable mutual comprehension in the face of differences between actors	Occurred together with mobilisation in the ecocultural mapping event, where representatives from local authorities, regional authorities and national institutions were present Also later in the process, in the documentation of customary laws that were considered along with modern law, and in the gazetting of sacred sites led by the National Museums of Kenya	All research results generated by indigenous workshop participants and knowledge-holders were collated and provided in short, simple reports to relevant indigenous communities for validation. A period of one month was provided to give feedback, make amendments, add anything that was missing or embargo content Once the reports were validated they were presented to the WG who discussed how to analyse and represent results from the perspective of both indigenous people and their non-indigenous partners in collaborative management of saltwater country	The core focus for the dialogue and concerns articulated was what indigenous knowledge, practice and belief systems mean for indigenous peoples, in relation to how it is perceived by scientists and government representatives. But also, what human rights mean if applied to biodiversity conservation decisions. Dialogue was designed to encompass the very diverse ways of expression, experiences and perspectives among the participants
3. Negotiate Interact among different knowledge systems to develop mutually respectful and useful representations of knowledge	All actors accepted evidence of the critical situation for the river brought up by the ecocultural mapping, complemented with technical data from research and government institutions provided by regional authorities Negotiation also happened in the development of action plans following the mapping process, where the community and local authorites agreed upon actions to improve the condition of the river	WG and research team had regular contact to ensure a collaborative research approach and facilitate discussions on saltwater research and monitoring at a regional scale WG provided an important conduit between indigenous communities, their staff and the research community WG held a final workshop attended by indigenous people, indigenous rangers, indigenous representative bodies, scientists and federal and state governments to raise awareness and seek final feedback on project outputs	Negotiations included how to interpret biodiversity data from different approaches for management and governance of ecosystems, along with knowledge about human rights principles and legislation, and cultural and socio-economic use of biodiversity
4. Synthesise Shape broadly accepted common knowledge bases for a particular purpose	Synthesis occurred in compiling customary laws and conventional laws together with authorities, when gazetting the sacred sites and in the development of action plans for protecting the river	Indigenous participants engaged in regular synthesis of results, from scoping, defining research questions and conducting fieldwork, to analysing results and communicating outcomes. The use of the MEB approach was a result of the WG’s capacity to consider a range of possible tools and processes and choose the ones that work best for the project Indigenous people had the opportunity to continuously monitor to ensure that project frameworks and tools fitted into the holistic, contextual and current situation in the Kimberly Saltwater Country	Agreement in place, based on evidence, that synergies are possible between conservation and human rights. In policy and practice, more efforts are needed to synthesise ‘how’ this can happen. The dialogue did not aim for a synthesis, that is for a later stage, with policy decisions leading to application. When presenting a summary of evidence, it was considered important to recognise convergence, but also identify and recognise where there were still disagreements
5. Apply Use common knowledge bases to make decisions and/or take actions and to reinforce and feed back into the knowledge systems	The process led to applications to improve river conditions at multiple levels: Revitalisation of rituals and enforcement of customary law at sacred sites Government recognition of the custodians as protecting the sites Enforcement of regulations of water extraction and riparian zone protection by regional authorities	The primary outcome: a regional network of indigenous people who have negotiated as regional knowledge brokers with their elders and knowledge-holders Short-term funding secured for the WG to support implementation, modification and compliance of the best-practice approaches developed Tools developed: – Regional saltwater monitoring framework based on indigenous knowledge identified social, cultural, economic and environmental values – Digital research protocol and application systems – Set of guidelines to describe simple processes for knowledge collaborations	The evaluation showed that knowledge for conservationists about Human Rights law and implementation representing a strand of research was considered useful. There was potential for application of insights around equal benefit of conservation and human rights in all the cases brought up in the dialogue

6.5.1 Piloting the MEB approach: Tharaka’s river is running dry

6.5.1.1 Context

Drought is a recurring challenge to the livelihoods of the people in Tharaka, Kenya. Kathita River is the main water source and of paramount importance, economically, culturally and spiritually. Fourteen sacred natural sites along the river are protected by the communities for their cultural and spiritual values. In recent years, the government’s policy guidelines and regulations for protecting the river have not been upheld and traditional ecological law has not been enforced either. This has led to excessive and often illegal abstraction along the river’s course, degradation of the riverine vegetation and destruction of the catchment area. The local people, led by clan-based custodians of the sacred sites, decided to come together to find ways of protecting the river using their indigenous and local knowledge and practices and customary laws. A non-governmental organisation in the area offered to facilitate an eco-cultural mapping process to enhance the eroded local capacity to govern the river.

A preparatory process brought together custodians of the sacred sites along the river. Local community organisations, county leaders and government institutions, including the National Museums of Kenya, were successively engaged in the process. In August 2014, community members jointly developed eco-cultural maps and calendars of the past and present, which illustrated changes in the integrity of their social–ecological system. Based on these, maps of the future envisioning different scenarios were drafted, creating a collective understanding and describing alternative pathways for the future. The maps and insights were shared and discussed with different actors beyond the local community.

A couple of years after the initial process, several of the problems identified with river governance have been addressed: strategies have been formulated for local authorities to reach out to land owners to safeguard riparian reserves. Tree seedlings are raised and distributed to land owners for planting in order to protect the riparian zone. The National Museums of Kenya have, together with the communities, gazetted the sacred sites along Kathita River, which has given them a government-recognised status. Rituals are again carried out at the sacred sites and the customary rules are enforced (Mburu, 2016).

6.5.1.2 Role of the MEB

The local non-governmental organisation convening the eco-cultural mapping in Tharaka is a member of a bridging organisation who were engaged in the initial dialogue across knowledge systems and volunteered to pilot a MEB approach. Eco-cultural mapping emerged as a culturally appropriate tool for knowledge mobilisation to enhance ecosystem governance for the society at large, beyond the community benefits expected by the clans that initiated the process. This also led to a greater understanding of the roles that different actors play in the local community and who to approach, how to formulate proposals and the utility of referring to established facts from community-based monitoring of the river.

The process contributed to unifying actors towards an enriched picture of understanding that could be shared and discussed with decision-makers outside the community. The eco-cultural mapping activity focused on how knowledge can be translated and negotiated to benefit an official process of conservation of sacred sites, and better ecosystem management of the Kathita river at large, through collaboration to protect the landscape (see Table 6.2). For this step, it was important to engage with actors with the authority to act in the customary governance system. Thus, the clans that were managing the sacred sites had a critical role in mobilising other community members.

6.5.1.3 Challenges and opportunities

The power imbalance between farmers with resources to extract and use water, and the majority of the community who did not have such resources, but were still exceptionally dependent on Kathita River as a water source, proved a challenge. Community research groups have been formed to solve specific emerging problems defined by the community.

The initiative for the eco-cultural mapping process came from the communities and the local non-governmental organisation, who contacted government and later also the Natural Museums of Kenya in order to catalyse change and ensure impact. The local actors as initiator created a solid base for trustful collaborations across knowledge systems.

6.5.2 Mobilising indigenous knowledge systems for saltwater country across the Kimberley region, Australia

6.5.2.1 Context

The Kimberley region in tropical north-western Australia is globally significant for its biodiversity, relatively intact ecosystems and its aesthetic and recreational values. Indigenous peoples comprise almost half of the region’s population and have ownership or management rights over most of the land and sea. They are caretakers of a diverse cultural landscape dating back at least 60,000 years. The Australian public places high value on the cultural and natural assets of the Kimberley. The Western Australian Government concluded in 2011 that to ensure the best possible outcomes of conservation efforts in the Kimberley, a combination of indigenous knowledge and scientific knowledge was needed.

The Kimberley Indigenous Saltwater Science Project (KISSP) was established by a group of indigenous peoples and their organisations, research institutes, corporations and government organisations to investigate ways of co-producing collaborative monitoring, management and research regionally. A working group was established in 2014 with representatives from seven indigenous groups (Balangarra, Bardi Jawi, Dambimangari, Karajarri, Nyul Nyul, Wunambal Gaambera and Yawuru peoples) and key staff from local indigenous organisations. The working group recruited a team of researchers to assist the project. In total, there were 103 indigenous participants in five Traditional Owner workshops and one Knowledge-Holder interview.

6.5.2.2 Role of the MEB

Although not intentionally applied at the commencement of the project, the MEB process was followed intuitively by the experienced practitioners involved. Midway through the project, the MEB approach was formally introduced to participants, who immediately recognised its value in describing their practice. The working group agreed to adopt the MEB as an overarching framework for the KISSP project and to design regional frameworks for collaborative knowledge production, monitoring, research and management of Kimberley Saltwater Country (Table 6.2).

6.5.2.3 Challenges and opportunities

The biggest challenge faced by the KISSP was to establish engagement with indigenous peoples in the Kimberley. Prior to the formation of the working group, the project struggled for many years to create dialogue with indigenous peoples. Finally, a workshop was held to identify collaborative pathways towards project goals. The intervention of the indigenous-led working group demonstrated the potential for MEB approaches to ensure useful outcomes through intercultural and interdisciplinary projects.

Initially, lack of investment in the capacity of indigenous peoples and their organisations to engage in the research process limited progress. This should not be understood as a lack of knowledge or capacity to care for saltwater country, but rather as a need for support to mobilise their knowledges and practices to contribute to the KISSP as a collaborative, intercultural project. There was a prior assumption that indigenous peoples and their knowledge and practice could easily fit into a regional project that comprised indigenous and scientific knowledge systems side by side. There was no insight of the need for recognition and equity, and for explicit usefulness of the research products for all involved in collaborative practices. For example, there was consistently a subconscious assumption that flows of knowledge produced throughout the project would be channelled in a unilateral direction to scientists in the regional capital in the form of ‘data’ to be analysed so as to suitably inform the policy and decision-making processes of the state. The communication of this new information back to indigenous peoples in the Kimberley was more of an afterthought and, presumably, seen more as a bureaucratic demand than a practical mechanism for improving collaborative management of Saltwater Country. This assumption ignored the practical, and fairly reasonable, requirement of local indigenous peoples that any knowledge shared or co-produced through collaborative research and monitoring be made available for informing their own local decision-making and practice for looking after Saltwater Country. The indigenous-led KISSP Working Group made this point patiently and constructively and, thus, ensured that the project could produce several locally useful outputs and outcomes for indigenous peoples in Kimberley Saltwater Country.

6.5.3 Justice and conservation: Global Dialogue on Human Rights and Biodiversity Conservation

6.5.3.1 Context

The Global Dialogue on Human Rights and Biodiversity Conservation was an international meeting initiated to address the conflicts that have often emerged across the globe between conservation agencies and indigenous peoples with longstanding relationships with their ancestral territories, co-organised by SwedBio at Stockholm Resilience Centre, Forest Peoples Programme, Natural Justice and the Chepkitale Indigenous Peoples Development Project as the local host in Kenya. The organisers represented actors engaged from different scales and perspectives, which created confidence and legitimacy for the dialogue. The dialogue started from the conviction that local people and conservation organisations could be strategic allies. It was attended by conservation agencies, social justice and human rights advocates, biodiversity conservation and sustainable use experts, legal and human rights professionals, members of community-based organisations, government officials, UN organisations and academics. It was designed in a global policy-setting context, while also aiming to contribute to local ways forward. The venue for the dialogue, Eldoret, Kenya, is situated between two biodiversity-rich areas conserved by indigenous peoples as their ancestral lands. The Ogiek people are an indigenous hunter-gatherer community on Mt Elgon, at the border of Uganda, while the Sengwer people are traditionally living with and taking care of the Embobut Forests. Both Ogiek and Sengwer have been faced with repeated attempts of eviction over decades in the name of conservation. In 2011, through a conservation-related mediation method called the Whakatane mechanism, the Ogiek communities in Mt Elgon reached an initial agreement to live in and govern parts of their ancestral lands. However, the Sengwer have rather experienced increased tensions in later years.

6.5.3.2 Role of the MEB

Globally there is an increased recognition that human rights protection can, and should, be complementary to safeguarding biodiversity and ecosystems (Knox, 2017), but there is a need to mainstream how, through good case examples and methods in policy and practice. A MEB approach was introduced in the preparatory process before the dialogue as part of the multi-actor dialogue method. The design process started with informal discussions between conservation agencies, indigenous peoples, human rights professionals and the organisers a year before the dialogue took place. The long preparatory process helped mobilise knowledge and confidence as a base for common understanding of the overarching ecological, legal, institutional and political challenges among participating actors. Through the dialogue process, the MEB approach provided guidance to ensure equity, reciprocity and usefulness for all actors. In the evaluation, the community representatives stressed they had never before had experience of being recognised and presenting their stories as evidence on an equal footing with science and governments.

6.5.3.3 Challenges and opportunities

Establishing a collaboration among different actors at national level in Kenya representing government, indigenous peoples and conservation agencies that generally do not meet was the greatest challenge. Thanks to the global context of the meeting, the presence of international actors with diverse experiences contributed to a constructive dialogue. Interactions among indigenous peoples and scientists were successful because a common understanding of the MEB approach had been established during the preparation. Persistent barriers between indigenous peoples and governments still exist in local cases, in particular the Sengwer people, and should be resolved through policy and legal processes. However, establishing MEB processes whenever governance of ecosystems and biodiversity can be enhanced through collaborative processes across multiple knowledge systems can be useful for all involved in the meantime.

6.6 Sharing lessons from the three cases

In the first case from Tharaka River, the importance of mobilising indigenous and local knowledge as a solid base for translation and negotiation phases was very clear. This then helped people speak about their knowledge, and also catalysed the revitalisation of eroded institutions and rules that previously served to protect the river, including the recognition and protection of the sacred sites. As the problem formulation was owned by the community, this enabled articulation of the importance of the sacred sites for understanding previous river governance, and motivated local people to restore the river. Later, they contacted the Natural Museums of Kenya, to provide support in gazetting their biodiversity-rich sacred sites for formal national recognition. This illustrates the important role that values and beliefs in diverse knowledge systems can play for conservation, how they may be identified, and how knowledge and governance capacity is embedded in the belief systems.

In the case from the Kimberley, the use of the MEB and the role of indigenous and local knowledge in collaborative management, created space and enthusiasm for experimenting with new ways of combining knowledge systems for management and governance of Saltwater Country. The KISSP demonstrated that working with multiple knowledge systems and disciplines in the context of unequal power relations requires design, support and monitoring of mechanisms that can maintain constant dialogue (e.g. the KISSP working group). Thinking of the collaboration as ‘intercultural’ was useful for understanding what capacity development was required for all actors. No single party had capacity deficits, but the collective needed to build joint capacity for weaving knowledge systems in ethical and equitable ways.

In the Global Dialogue on Human Rights and Conservation, reaching synergistic solutions between conservation and human rights once again was about overcoming power imbalances. The dialogue was an opportunity for key actors with different knowledges, experiences, worldviews and power to meet in a neutral context. Diversity of experiences (positive and negative) across scales and a careful mix of actors helped to overcome these imbalances during the dialogue. Mobilisation of indigenous and local knowledge and strengthening confidence among participating community representatives, but also knowledge about human rights and other legal aspects before the dialogue, was critical for deliberations. The recognition of indigenous rights and the value of their knowledge and practices for conservation expressed by researchers contributed to trust followed by constructive proposals. Positive experiences from successful collaborations in conservation of indigenous lands contributed to exploring ways forward in cases where conflicts persist. The learning across different sectors and scales, such as ecologists learning about human rights aspects, was appreciated in the evaluations. It also became clear that the deepest conflicts may not relate to conflicting evidence from different knowledge systems regarding ecology, but to controversial policy, such as the eviction of people from conservation areas.

In all three cases, the main challenge of the collaborative process was to overcome power imbalances and build trust and confidence. The focus on recognising, mobilising and discussing evidence from diverse knowledge systems was an entry point that contributed to the development of strong collaborative partnerships. Designing a process that was considered useful for all involved was critical to securing successful and sustainable outcomes, new and useful ways to combine and apply knowledge from diverse knowledge systems, and sometimes the generation of new knowledge. In all cases, the aim of creating synergies across knowledge systems for providing evidence on sustainable governance could be realised when all holders of knowledge gained from collaborations. A MEB approach, on whatever level it is conducted, emphasises the importance of collaborative processes that value multiple knowledges and practices needed to sustain the social–ecological landscape to the double benefit of sustainable livelihoods and conservation over the long term. Further, the collaborative relationships of trust developed provide new opportunities to align multiple modes of governance of ecosystems, to ensure decisions and policy are based on all available knowledge.

6.7 Discussion

In this chapter, we review the use of one recent and important approach to combining the knowledge of indigenous peoples, local communities and scientists for sustainability and conservation partnerships. We have focused on the MEB and its potential for building more inclusive understanding of multiple sources of evidence, how it is generated and how it is transmitted among diverse conservation actors. We argue that such an approach is important for better understanding of interlinked social–ecological systems, strengthening conservation partnerships and identifying new evidence-based pathways towards sustainability. Our review and the three case studies show examples of different ways to move forward that recognise the complementarity and integrity of knowledge systems in addressing specific problems (Molnár et al., 2016; Smith et al., 2017), create conditions (and methodologies) for full and open dialogue on how to move ahead, overcome power inequalities and navigate cultural differences (Robinson et al., 2016; Reed & Abernethy, 2018). We demonstrate reciprocal synergies between indigenous and local knowledge and conservation science and rich cases of how cross-fertilisation leads to stronger partnerships and better outcomes. The three case studies also show that the MEB requires partnerships that are underpinned by recognition, respect and understanding of diverse knowledge systems, and that the process for producing and applying common knowledge to problems cannot be viewed separately from the outcomes of partnerships. That is to say, much like the concept of adaptive management in conservation, the diversity and dynamism of knowledge systems dictate that the process of collaboration be taken as seriously as the achievement of conservation outcomes themselves (Gavin et al., 2018).

More work is needed to further elaborate how to implement a MEB approach in different processes and contexts. The IPBES process has struggled with the tension between open collaboration and the demands for structure set by the scientific knowledge governance. There is yet some way to go to better acknowledge and solve epistemic challenges, such as diverse modes of validation across knowledge systems (Löfmarck & Lidskog, 2017; Obermeister, 2017). There is also a need to continue developing tools and approaches for bridging knowledge systems that are connected to local, cultural, social and ecological conditions. Our review illustrates that indigenous peoples, local communities and scientists have begun to tackle this challenge (Molnár et al., 2016; Robinson et al., 2016; Smith et al., 2017), but further dialogue is required, both horizontally across local scales and vertically through local to global institutions.

We have shown that a MEB approach has been particularly effective in dialogues where there are power imbalances among actors and historical bias concerning the validity or usability of knowledge systems other than western approaches to science (see also Klenk & Meehan, 2015). Building trust and respect is especially pertinent in the context of ongoing and historical injustices and abuse of indigenous rights, and requires the recognition of indigenous peoples as rights-holders and defenders of biodiversity, who maintain management and governance systems of vast ecosystems (Brondizio & Le Tourneau, 2016; Mistry & Berardi, 2016).

Tengo et al. (2017) suggest five tasks that can guide processes that build trust and agency (see Figure 6.1b), while at the same time building a stronger evidence base for action. We find in our review that the mobilisation task is often neglected, or that documentation of indigenous and local knowledge is not fully recognised. More research is needed, but mobilisation of knowledge and empowerment of knowledge-holders may be critical steps for successful knowledge collaborations that also contribute to strengthening collaborative governance capacity. We also find that explicit joint problem formulation and analysis across knowledge systems is absent from many processes and is clearly a challenge in regional and global assessments with rigid scientific formats (Livoreil et al., 2016; Nesshöver et al., 2016; Oubenal et al., 2017). Our case examples clearly show the importance of creating the right conditions for joint problem formulation.

It should be acknowledged that the implementation of a MEB approach is demanding, in terms of time and other resources, and requires strong commitment from all parties. However, we reiterate that there is mounting evidence of the potential positive outcomes in terms of novel indicators, more efficient responses to and implementation of findings, as well as for synergies between conservation and human well-being, including human rights (Danielsen et al., 2010; Johnson et al., 2015; Sterling et al., 2017b, 2017c). As found by the participants in our third case example on reconciling conservation and human rights, conservation initiatives can play a positive role by engaging with communities and increasing their recognition as actors and partners who hold important and useful knowledge.

Our experiences derive mainly from dialogues and collaborations with indigenous and local knowledge-holders who have deep connections, obligations to care for and a duty to fight for their rights to actively govern their ancestral territories. We are aware that in many other contexts, local knowledge-holders may be less empowered and traditional governance systems and cultural connections may be displaced and eroded. However, we believe that insights about dialogue and partnership between indigenous peoples, local communities and scientists can also be applied in western, urban and developing settings, where local knowledge and experience may be less evident but remains critical for nurturing effective stewardship of biodiversity and ecosystems. Ultimately, the MEB approach contributes to a much-needed conceptual mind shift to mobilise all knowledge that is useful for maintaining the life-supporting ecosystems in our world.

Chapter Seven Informing conservation decisions through evidence synthesis and communication

Andrew S. Pullin

Collaboration for Environmental Evidence

Samantha H. Cheng

Center for Biodiversity and Conservation

Steven J. Cooke

Canadian Centre for Evidence-Based Conservation

Neal R. Haddaway , Biljana Macura

Stockholm Environment Institute

Madeleine C. Mckinnon

Bright Impact

and Jessica J. Taylor

Canadian Centre for Evidence-Based Conservation

7.1 Introduction

The volume of evidence from scientific research and wider observation is greater than ever. Approximately 2.5 million articles are published annually (Plume & van Weijen, 2014) and this rate is increasing at around 3–3.5% per year (Ware & Mabe, 2015). Conservation is no exception to this trend and the result is a rapidly expanding body of potentially useful information for decision-makers (Li & Zhao, 2015). While the expansion of research represents an important increase in knowledge generation, much of this information is scattered in fragments over increasingly diverse sources. This, along with the sheer volume, makes it harder for decision-makers to find, access and digest all of the relevant information on a particular topic, resolve seemingly contradictory results or simply identify a lack of evidence. Evidence synthesis is the process of searching for, and summarising, a body of research on a specific topic in order to inform decisions. The extent of relevant research may range from nothing, or one or two primary studies, to many hundreds. Despite the obvious potential value of synthesising findings from multiple studies (where two studies may be all that is needed to add value through synthesis), methods of rigorous evidence synthesis have been largely neglected until recently. We argue that it is time to place evidence synthesis as a central pillar of evidence-informed decision-making in conservation and environmental management.

As an enterprise, evidence synthesis is very broad and includes many and diverse methodologies, some more rigorous than others. For example, syntheses labelled as ‘literature reviews’ often lack standardised methodology, fail to report their methods and therefore lack transparency or the potential for repeatability (O’Leary et al., 2016). Additionally, these literature reviews do not deal with the risk of bias in either the primary research (e.g. poor-quality experimental design and conclusions that may not be supported by a given study) or the synthesis process (e.g. selective use of information). Meta-analysis approaches have become popular where significant amounts of quantitative data are available, but they are often biased in the way they select and include studies in their analysis (Koricheva & Gurevitch, 2014). In response to these problems, more rigorous methodologies, such as systematic reviews, have been developed. These were first used in the health sector through the work of the Cochrane Collaboration (Higgins & Green, 2011), and have subsequently been applied to conservation and environmental management by the Collaboration for Environmental Evidence (Pullin & Knight, 2009; Collaboration for Environmental Evidence, 2018).

In this chapter we make a case for rigorous evidence synthesis: we explain why these methods are appropriate, how they can benefit wider society and how evidence can be synthesised, shared and used as a public good. Although evidence synthesis can inform a broad range of decision-making contexts, we focus here on two major aspects of conservation where evidence might be useful. First, in measuring the direct and indirect impacts of human activity on the natural world, and second, the effectiveness of conservation efforts to mitigate those impacts.

7.2 The central role of evidence synthesis in informing decisions in conservation policy and practice

Many factors can contribute to making a decision. In contexts where social and political stakes are high, as is common for conservation policy, scientific evidence will likely only inform decisions, rather than act as the primary driving force behind them. Although evidence is sometimes crucial, it may equally be ignored or overruled by other factors, such as political context, infrastructure and capacity. Ideally, evidence synthesis should play a central role in providing reliable evidence and enabling the wider society to understand or challenge decisions that might affect them. Making decisions without considering all available evidence might perpetuate biases, increase the likelihood of taking a wrong or costly action, or lead to missed opportunities to achieve faster or more cost-efficient outcomes. In a democratic society, comprehensive and rigorous evidence synthesis and open communication makes ‘sidelining’ (i.e. deliberately ignoring evidence) and/or biased (i.e. selective) use of evidence by authorities more difficult without challenge and transparent justification.

Unfortunately, evidence synthesis is itself often ‘bypassed’ completely or manipulated to get the answer required (i.e. policy-based evidence) (Dicks et al., 2014). There may be significant resistance to the use of transparent evidence synthesis in the face of vested interests, and this may partly explain why organised and independent evidence synthesis receives so little attention or funding. Rigorous scientific evidence could also be seen as a threat to those with entrenched beliefs. Beyond outright opposition, complacency or inaccessibility of evidence might inhibit adoption of synthesis findings even when good intentions towards informed decision-making exist.

Fortunately, most decision-makers in conservation want practical advice that is grounded in the best available evidence (Cook et al., 2013). Leveraging syntheses and integrating their findings into decision-making processes requires an understanding of how and when evidence is necessary, and what level of confidence is needed to inform a decision. Such considerations will determine the choice of synthesis method(s), which should reflect practical needs to guide management decisions or future research. Syntheses can be used either to generate a new theory, conceptual framework or hypothesis (e.g. applying existing theory to a different context) or to test an existing hypothesis (e.g. evaluating the effectiveness of an intervention). In the context of effectiveness of interventions, evidence syntheses are relevant to decisions at several critical stage points in the life cycle of a programme or initiative: (1) initial scoping of a new topic early on in strategic planning (e.g. informing a new strategy on land use for a philanthropic foundation (Snilstviet et al., 2016)); (2) identification or validation of specific intervention designs (e.g. understanding how gender composition affects outcomes of resource management groups (Leisher et al., 2016)); (3) benchmarking of institutional outcomes against other programmes (e.g. investments in community forest management by the Global Environment Facility (Bowler et al., 2010)); (4) evaluation of overall effectiveness of an intervention across multiple contexts or applications (e.g. effects of property regimes in different biomes (Ojanen et al., 2017)). Understanding the purpose of the syntheses for informing the different stages of decision-making will ensure selection of a suitable method, appropriate engagement of stakeholders and relevant communication of findings.

Some evidence synthesis methods, such as systematic review, have been described as following the ‘information deficit model’ (Owens, 2000); that is to say, they follow the assumption that the simple production and push delivery of evidence that fills a gap will be sufficient to achieve uptake. However, this perception misrepresents the full process behind the methodology. Systematic reviews can be socially inclusive, with extensive stakeholder contribution to formulating a question and approach, including setting the scope of the topic. This engagement attempts to ensure the findings of a review will fill a real and important synthesis gap (a knowledge need where sufficient primary research exists to allow synthesis) and respond to stakeholder demand. When engaging with stakeholders, a balance needs to be struck between involving them in the design of the review and independence from undue vested interest (Haddaway & Crowe, 2018). In the field of conservation, this balance is very much dependent on the nature of the question and the extent of vested interests (Kløcker Larsen & Nilsson, 2017). Many aspects of evidence synthesis are collective, with stakeholders having shared motivation to benefit from the findings. In other cases, evidence synthesis is conducted in contested areas, with stakeholders that hold opposing views and may be hostile to the process and its findings. In the latter case, it is important to have a process that allows consultation when appropriate but also provides independence when necessary. For example, for some key steps, such as initial formulation of the question, engagement with stakeholders is usually essential (Land et al., 2017), while other steps may need to be conducted free of such vested interests. To date, systematic reviews have engaged with a spectrum of stakeholders at different levels. Some reviews, for example those that are more academic or have specific commissioners (e.g. private goods reviews (Oliver & Dickson, 2016)), have only passively engaged stakeholders by informing or consulting them (typically only at the beginning of the review process), while others have employed more in-depth engagement, extending to co-design of review methods and scope (Land et al., 2017).

Alongside the purpose of syntheses, the level of confidence required to make a decision determines their method and scope. In some instances, where evidence of effectiveness is key, uncertainty in the evidence base hampers decision-making. In such instances one might ask ‘How much evidence is enough?’ or ‘How much uncertainty is acceptable?’ (Salafsky & Redford, 2013). The need for evidence synthesis in the conservation sector may also vary depending on aspects of spatial scale, complexity and controversy. For example, decisions regarding inexpensive and low-risk local-scale interventions (e.g. applied to improve biodiversity or habitat conditions in nature reserves) may benefit most from locally generated, rigorous evidence, or more commonly from primary research studies conducted in similar contexts. This evidence could be provided by a single, self-generated study (as in adaptive management), be internally generated by the relevant organisation, or come from collating evidence from similar case studies. In contrast, decisions regarding expensive, often large-scale, high-risk programmes (e.g. to eradicate poaching and illegal trade in wildlife), where stakeholders are likely to be global and might hold conflicting views, may benefit from an independent global-scale, multi-context evidence synthesis. This might require a rigorous analysis of what works, where and when and for whom, involving analysis of heterogeneity in outcome and identification of effect modifiers. Often within conservation, a broader set of evidence types (e.g. controlled trials, case studies, quantitative and qualitative research) is needed to fully capture the complexity of conservation contexts.

7.3 Key aspects of rigorous evidence synthesis and why they are needed

To be reliable, evidence syntheses should consider all available evidence and attempt to provide the most accurate and precise estimation of the truth. A suite of methodologies has been developed that maximises transparency and repeatability while minimising subjectivity, susceptibility to bias or influence of vested interest. The most widespread of these, systematic reviews and systematic maps, are well-documented secondary research methods that follow detailed guidance (e.g. Collaboration for Environmental Evidence, 2018) and use step-wise processes set out in an a-priori protocol to comprehensively identify and collate all available evidence (Table 7.1).

Table 7.1 Overview of systematic evidence synthesis stages and the issues they address. For an explanation of bias see Collaboration for Environmental Evidence (2018) or Bayliss and Beyer (2015)

Systematic review stage	Description	Defining features	Type of issue addressed
Review question identification and formulation (with stakeholder engagement)	Question is carefully identified and formulated with help of stakeholders	Social acceptance, relevance and legitimacy of the review process
Protocol	Protocol outlines the intended method in detail. Protocol is peer-reviewed and published on an open-access platform	Public acceptance, peer review	Review bias, question creep
Searching for relevant literature	Comprehensive searches for grey and commercially published literature from a variety of sources	Comprehensiveness, repeatability (through transparency)	Publication bias
Eligibility screening	Careful screening of all identified articles according to pre-determined inclusion criteria	Consistency	Selection bias, review bias
Critical appraisal of study validity (optional for systematic maps)	A detailed assessment of the susceptibility to bias and generalisability of each study	Account for variability in internal validity and power of individual studies	Susceptibility to bias in individual studies and in study weighting by reviewers
Data coding and extraction	Transparent coding and, in case of systematic reviews, extraction of study finding	Consistency, repeatability (through transparency), minimising subjectivity	Selection bias
Qualitative and/or qualitative data synthesis (not required for systematic maps)	Well-documented and comprehensive synthesis of qualitative and/or quantitative study findings	Comprehensiveness, repeatability (through transparency)	Selection bias, vote-counting, publication bias
Reporting and communication of review findings	Transparent reporting of the review results with extensive supplementary information	Repeatability (through transparency), avoiding overreach	Discussion bias

Systematic reviews in conservation and environmental management have most commonly aimed to answer specific cause-and-effect type questions, for example relating to the effect of a management intervention or exposure on a subject of concern. (e.g. ‘What is the impact of a specific factor x on a subject z?’). In contrast, systematic maps collate and catalogue available evidence on a relatively broad subject, describing the nature of the evidence base and highlighting evidence clusters and gaps, along with methodological patterns in primary research (Collaboration for Environmental Evidence, 2018). Systematic maps can be used as an initial step of an evidence synthesis pathway to identify subtopics suitable for a systematic review and subtopics where there is insufficient evidence to make synthesis of primary data worthwhile. In such latter cases, which are common in conservation, the map may identify individual primary studies that provide useful evidence (for an example of a systematic review question generated from a map, see www.eviem.se/en/projects/SR15-Prescribed-forest-burning/).

Systematic reviews were originally developed in response to an absence of easily accessible and rigorous synthesis of available evidence. However, recent assessments have shown that non-systematic reviews that aim to inform environmental policy and practice are still prevalent, but have low methodological reliability, suffering from lack of transparency and methodological rigour, and are consequently highly susceptible to bias (Woodcock et al., 2014, 2017; O’Leary et al., 2016). Moreover, the term ‘systematic review’ is often used by authors (and not challenged by editors or peer reviewers) when the reviews are in no way systematic. The production of substandard and ‘fake’ systematic reviews is increasing in all fields, from public health to environmental management and education (Haddaway et al., 2016; Ioannidis, 2016; Haddaway, 2017; Pussegoda et al., 2017); they are ‘fake’ in the sense that they lack necessary comprehensiveness, transparency and reliability (Haddaway, 2017). This further confuses the issue for potential readers, with only a handful of environmental journals requiring authors to follow accepted standards of conduct and reporting (see Collaboration for Environmental Evidence, 2018). A potential evidence user can use keywords like ‘systematic review’ in their search and have it return documents that claim to be such, when in fact they are not. The misuse of the term ‘systematic review’ can undermine efforts towards effective decision-making and is a key reason for establishing independent standards.

Stakeholders, including scientists, rarely have the time or training to differentiate between a ‘true’ systematic review and one that misses critical components of the method (resulting in increased risk of bias and lack of transparency) especially when published in an outlet such as a peer-reviewed journal. To enhance the uptake of more rigorous and reliable synthesis methodologies and maximise the potential of evidence to inform decisions, independent coordinating bodies have been founded in different sectors of society to provide guidelines and standards for evidence synthesis. In the field of medicine this process began in the 1990s with the establishment of the Cochrane Collaboration, which aimed to conduct systematic reviews in order to provide healthcare professionals with the best available evidence on the effectiveness of clinical interventions (Higgins & Green, 2011). The methods were transferred to the field of conservation and environmental management in the early 2000s (Pullin & Stewart, 2006) and are now under the coordination of the Collaboration for Environmental Evidence. These independent coordinating bodies provide guidelines for and training in the conduct of systematic reviews and systematic maps, as well as registering, endorsing and publishing such evidence syntheses. Syntheses registered through the coordinating bodies are scrutinised by methodology experts, guaranteeing a level of reliability and rigour (Collaboration for Environmental Evidence, 2018).

In circumstances where vested interests might potentially influence the outcome of an evidence synthesis, these independent organisations provide a framework and platform to assist the review team to achieve and demonstrate independence of the synthesis process. The framework allows for full engagement of commissioners and other stakeholders in formulation of the review question and planning of the review protocol, followed by independent peer review and publication of the protocol prior to the conduct of the review. In cases where conflict or the risk of undue influence from particular stakeholders is high, the review process should be conducted by an independent review team and the report submitted for independent peer review. Following this process, the review findings may be endorsed by the independent organisation.

7.4 New developments that address barriers to evidence synthesis and communication

There are persistent barriers to the conduct of environmental evidence syntheses and communication of their findings. First, the high resource costs required have been a major disincentive to producing high-quality syntheses, despite their critical value for effective conservation. Second, efficient and effective means of communicating results and facilitating their use for real-life decision-making scenarios are haphazardly applied. These barriers limit the ability of evidence synthesis to dynamically and adaptively respond to conservation challenges. However, new developments in big data and deep learning approaches are offering exciting opportunities to harness evidence syntheses and promote them to broader audiences.

Conducting rigorous evidence syntheses, such as systematic reviews, can carry both significant monetary and human resource costs (Dicks et al., 2014). These costs are particularly prohibitive for organisations with critical needs for evidence, but who have limited time and resources to engage in such synthesis efforts or even to glean needed information from lengthy synthesis reports (Elliott et al., 2014). Moreover, high costs make updating syntheses to create a dynamic evidence base with the most up-to-date knowledge effectively impossible using current technology (Garritty et al., 2010). Additionally, the window of opportunity for decision-making may be shorter than the time in which a credible synthesis can be completed. Thus, to be useful to conservation, evidence syntheses must be optimised to efficiently find, collate and communicate existing evidence (Boyack & Klavans, 2014).

In a policy space where decision-making timelines are short and demands for rigorous, reliable evidence are high, methods assisted by advances in computing can support rapid evidence collation as well as increase cost efficiency (Shemilt et al., 2016). Computer-assisted approaches range from tools that manage data and streamline the synthesis process to tools powered by machine learning algorithms that allow rapid screening and extraction of evidence with reduced human intervention (Kohl et al., 2018). Promising computer-assisted approaches, including automatic term recognition, document clustering, automatic document classification and document summarisation (Frantzi et al., 2000; O’Mara-Eves et al., 2015) have been trialled in medical and health topics (Ananiadou et al., 2009) and are beginning to be tested in ecological topics (Westgate et al., 2015; Grubert & Siders, 2016; Roll et al., 2018).

These developments are encouraging for increased efficiency of the synthesis processes and potentially enabling dynamic syntheses that continuously update with new evidence as it becomes available. However, there are certain caveats and limitations that must be considered prior to widespread employment of computer-assisted tools. First, unlike medicine and fields such as economics, the semantics of conservation are highly heterogeneous and non-standardised (Westgate & Lindenmayer, 2017), posing difficulties for both efficient and comprehensive searching, and reliable application of machine learning algorithms to sort and mine text for desired patterns. Second, thus far, the performance of these approaches remains largely untested empirically, particular for conservation and environmental topics. As the value of evidence synthesis methods is in their transparency and credibility, reliable data on the efficacy of different computer-assisted approaches are important for uptake and expansion. Third, many existing computer-assisted platforms are fee-based or require programming skills, limiting their utility to a broader field of users. To improve global ability to address pervasive environmental threats, we need to democratise access to the tools that can help decision-making worldwide, not solely in countries or among researchers with means.

7.5 Mainstreaming evidence synthesis for decision support

Efforts to engage in open science and collaborative practice between conservation and technology fields will require forming collaborative partnerships and fostering conversation between evidence producers, evidence users and data scientists, to build a cohesive and engaged community of practice to open channels of communication to all users (Joppa, 2015). This will allow the broader community to use existing efforts as a starting point and avoid reinventing the wheel and wasting already limited resources (Lowndes et al., 2017). Furthermore, collaborative partnerships and creative funding can foster the long-term sustainability of tools that can live on to serve users. Too often, tools and platforms are created in good faith but require maintenance and updating and lack the ongoing funding and personnel to do so. This is particularly important as tools are most useful when they can dynamically respond to user needs and emerging technologies. This is a critical stepping stone for breaking down barriers to understanding and using evidence synthesis methodologies, as without a dynamic toolbox, synthesis methods will reman aloof from the needs of a diversifying and widening audience.

Evidence synthesis conducted to Collaboration for Environmental Evidence standards generates systematic reviews and systematic maps that are theoretically accessible to all. Yet, simply because something is available does not mean that the potential user is aware of it, knows where to find it, or even how to make sense of it. This is particularly the case for those new to the concept of evidence synthesis. Indeed, many practitioners and policy-makers rely on past experience or consult colleagues, rather than make use of the full suite of evidence (Pullin et al., 2004; Young et al., 2016). These issues create a number of inherent challenges for those decision-makers seeking to be evidence-informed and also broader potential audiences, such as stakeholders and wider society.

One of the mantras of science communication is ‘know your audience’ (Wilson et al., 2016; Cooke et al., 2017) and to have impact, the findings of an evidence synthesis need to be effectively tailored and communicated to different groups of people in different ways and through different media. Communication efforts should, for example, be sensitive to the fact that different groups vary in their ‘trust’ of the science they encounter from different sources (e.g. academic journals, colleagues, social media) (Wilson et al., 2016; Cooke et al., 2017).

A study that surveyed the willingness of practitioners to use a synopsis of relevant literature on bird conservation found that participants were more likely to use the evidence to inform decisions if it was easily accessible and in a clearly summarised format (Walsh et al., 2014). Similar summaries are needed to complement evidence syntheses. These summaries may then need to be further refined and transformed into policy briefs. Policy briefs are often written through the cultural lens of a given organisation and a given issue, meaning that these are unlikely to be useful if prepared in a generic format. Sundin et al. (2018) recently proposed the use of storytelling as a tool to effectively communicate the results of evidence syntheses. This method could give meaning to the evidence and can be communicated through videos (e.g. see https://youtu.be/4uPowxn2skg), presentations or public forums (e.g. newspapers, magazines). Nevertheless, uptake of these methods in science communication is generally slow and also could still rely on poorly conducted syntheses (McKinnon et al., 2018).

There has also been a rise in various knowledge management platforms and data-visualisation tools to explore underlying data that support evidence synthesis (e.g. www.3ieimpact.org/en/evaluation/evidence-gap-maps/, or www.cedar.iph.cam.ac.uk/resources/evidence/). These platforms present data from synthesis projects using interactive features and intuitive visualisations. For example, the Evidence for Nature and People Data Portal (www.natureandpeopleevidence.org) allows users to filter data according to desired parameters – such as diving into a data set to examine a specific intervention or outcome or geographic region, and visualising resultant trends. Syntheses, and in particular systematic maps, can be multi-layered and complex, precipitating a need for an interface that is graphical and intuitive, allowing a broader audience to use it (Figure 7.1).

Figure 7.1 An example of an evidence ‘heat map’ linking conservation interventions with human well-being outcomes. The map allows the user to assess the evidence base for gaps and gluts as well as clicking on each box to further examine the relevant studies.

(after McKinnon et al., 2016)

If reported responsibly, these platforms and visualisations can play an important role in how stakeholders access evidence. A challenge for these approaches is to communicate that evidence syntheses are only estimates of the truth, which depend on the reliability of the evidence with which they were made. There is potential for evidence to be misinterpreted if the relative weight or reliability of a given element is misconstrued when visualised. Regardless of the output, it is important that authors of evidence syntheses communicate any uncertainty in the evidence and the risks associated with relying on studies that have high risk of bias.

Although it is laudable to communicate the findings of a topical evidence synthesis, additional efforts are also needed to communicate to practitioners the value of systematic reviews or maps, how they differ from other evidence synthesis methods and how they can be integrated with existing science advice and decision-making processes within different regions or institutions. Writing academic papers and delivering presentations at scientific conferences is unlikely to reach the typical practitioner, so creative approaches to outreach are needed to access and inform them.

Without use of rigorous evidence synthesis, policies and practice claiming to be ‘evidence-informed’ can be meaningless. For conservation and the environmental sector in general, the value of evidence synthesis has yet to be fully realised and we have the feeling that its time is yet to come. However, the recent methodological developments, awareness-raising and capacity development, together with new technologies for faster and more efficient conduct, suggest this time is not far away. Conservation is an interdisciplinary field and cannot remain for long in a state of relative evidence synthesis deficit in comparison with other sectors with which it seeks to be relevant. Although still marginalised, the methodology and infrastructure to build conservation’s evidence base through rigorous synthesis now exist at a global level. A commitment to evidence-informed decision-making that recognises the central role of rigorous evidence synthesis is required by key actors in the sector if these potential benefits are to be achieved.

Chapter Eight Aligning evidence for use in decisions: mechanisms to link collated evidence to the needs of policy-makers and practitioners

Lynn V. Dicks

University of Cambridge

Barbara Livoreil

Fondation pour la recherche sur la biodiversité

Rebecca K. Smith

University of Cambridge

Heidi Wittmer

Helmholtz Centre for Environmental Research – UFZ

and Juliette Young

Centre for Ecology and Hydrology and AgroSup Dijon

8.1 Introduction

We should not be surprised by the scale of the challenge when trying to link a body of scientific knowledge to the complex, shifting and seemingly unpredictable world of policy, or to the massively decentralised, globally distributed world of conservation practice (Young et al., 2014). One side of the challenge is developing a consensual understanding of the science itself. By nature, scientific knowledge is continually progressing, with theories, empirical data and new interpretations emerging all the time. Even within a single discipline, it can be hard to convey what is known at a particular point in time, and this often involves presenting different scientific viewpoints. For instance, there is substantial variation around the world in public health advice regarding alcohol consumption, with ‘safe’ limits in the UK being 50% of those in the USA (Wood et al., 2018). In conservation, the challenge is even greater, as relevant research cuts across the natural, physical and social sciences.

The other side of the challenge is working out how, and when, to offer relevant scientific knowledge to decision-makers, in order to have the greatest impact on the decisions being made. This is the focus of our chapter. We argue that it is a question of correct alignment: of selecting the right knowledge to address the needs of decision-makers, ensuring that knowledge is accessible to them, and articulating it within their decision-making processes.

First, we consider how well current efforts to synthesise evidence in conservation align with the needs of decision-makers. Then we describe three mechanisms that might be used to enhance the alignment of available knowledge with decision-making, starting at small local scales and moving to the global scale: decision support tools, active knowledge exchange and large-scale scientific assessments. For each mechanism, we provide examples and draw out general guidelines regarding the circumstances in which it is likely to be most effective.

8.2 How well do current evidence synthesis activities align with policy and practice needs?

When scientific evidence is needed for decision-making, the process of obtaining and analysing the evidence is often demand-led. An organisation faced with a difficult management or policy decision will undertake or commission a review to answer a specific question. For example, the UK Government Department of Environment, Food and Rural Affairs (Defra) commissioned a review of evidence on the status of pollinators (Vanbergen et al., 2014) before designing the National Pollinator Strategy for England (Defra, 2014). When this happens, the evidence synthesis is well-aligned with the policy and practice needs, summarising relevant material that can be found in the time available. However, it also puts immense time pressure on the evidence synthesis process, because decision-making can only happen once the evidence has been reviewed. This tends to lead to the selection of evidence synthesis methods such as rapid evidence assessments, traditional non-systematic literature reviews and expert consultations, which are not the most rigorous or unbiased approaches available (Dicks et al., 2017).

The Collaboration for Environmental Evidence (www.environmentalevidence.org) and the Conservation Evidence project (www.conservationevidence.com) aim to address the needs of conservation practitioners and policy-makers with more rigorous methods of knowledge synthesis, namely systematic reviews, systematic maps (Collaboration for Environmental Evidence, 2013; see also Chapter 7) and subject-wide evidence syntheses (Sutherland et al., 2019b; see also Chapter 4). They do so by actively involving stakeholders in the selection of topics to synthesise and the collation and subsequent evaluation of the evidence found (Dicks et al., 2016; Haddaway et al., 2017).

To evaluate the overall success of this alignment effort, we recently asked how well evidence collated by the Conservation Evidence project on the subject of sustainable food production matched the priority knowledge needs of decision-makers. Five independent exercises (Pretty et al., 2010; Dicks et al., 2013a, 2013b; Ingram et al., 2013; Jones et al., 2014), involving 240 people from across business, practice, policy-making and academia, had generated 286 priority questions faced by decision-makers. We sorted these into five categories, following the Driver–Pressure–State–Impact–Response (DPSIR) framework (Maxim et al., 2009). This conceptual framework describes interactions between society and the environment in a way that is meaningful for policy. Social and economic developments (Driving Forces, D) exert Pressures (P) on the environment and, as a consequence, the State (S) of the environment changes. This leads to Impacts (I) on ecosystems, human health and society, which may elicit a societal Response (R) that feeds back on D, S or I. We added a category for questions about underlying science that did not fit the DPSIR categories (Figure 8.1).

Figure 8.1 Categorisation of 286 priority questions identified by stakeholders as relevant to sustainable food production (Pretty et al., 2010; Dicks et al., 2013a, 2013b; Ingram et al., 2013; Jones et al., 2014) according to the Driver–Pressure–State–Impact–Response framework. Examples of questions are provided for each category. The extracted segment represents questions already answered by evidence summaries provided by the Conservation Evidence project.

Of all the priority questions, 189 (66%) were about responses (R), which are the focus of the Conservation Evidence project. Evidence had already been summarised that could help answer 35 of these questions (12% overall; Smith et al., 2015; Sutherland et al., 2019a).

In a similar vein, Cook et al. (2013a) investigated the contribution of systematic reviews to conservation decision-making, finding that 35% of the 43 reviews considered practical on-the-ground management, while most addressed interventions relevant to policy. Cook et al. (2013a) argued that the benefits for conservation could be significantly enhanced by increasing the number of systematic reviews focused on questions of direct management relevance.

These two analyses show there is some alignment between high-quality evidence synthesis methods and the needs of conservation practitioners and policy-makers, but it could be improved. Below, we provide a series of examples of mechanisms to enhance this alignment at a range of scales.

8.3 Decision support systems

Decision support systems are tools designed to assist decision-makers, for example, by visually or numerically illustrating different possible outcomes to a question, or leading users through logical decision steps (Dicks et al., 2014). Often software-based, they represent a link between relevant science and decision-making (Dicks et al., 2014; Figure 8.2). Decision support systems are useful for incorporating evidence into decisions related to a specific question that has been widely and repeatedly addressed. It is also important that the evidence can be converted into simple numerical or visual formats.

Figure 8.2 A schematic showing how scientific information could support environmental decision-making (Dicks et al., 2014). The triangle on the left shows an evidence hierarchy, in which summaries, such as those produced by the Conservation Evidence project, integrate evidence from across studies and systematic reviews, and form the basis for information flowing into decision support systems. In these circumstances, environmental decisions (shown by the ‘Decision’ diamond on the right) are based on the best-available evidence, combined with the expertise and local knowledge of the practitioner or policy-maker (described by the ‘Experience’ box). Dashed lines illustrate bypass routes currently taken to inform environmental decisions.

There are many decision support tools available covering various aspects of environmental science. For instance, Zasada et al. (2017) identified 60 research projects funded between 2002 and 2013 under the European Commission’s 6th and 7th Framework Programmes that had developed decision support tools for landscape and environmental management. Of these, only 61% still existed in 2014, and only half were updated after the projects that developed them ended, although this seems a pre-requisite for ongoing use. The uptake of decision support systems depends on a range of factors, including ease of use, performance, whether they are recommended by peers and the level of marketing (Rose et al., 2016). Uptake can be enhanced by ensuring that users are closely involved in the conception and design of the tools (Rose et al., 2018).

While decision support systems are often designed by researchers as a way of incorporating scientific knowledge into practice, most are based on one particular model, study or approach to a scientific question and represent a ‘bypass’ of the evidence hierarchy (Figure 8.2 and see Dicks et al., 2014). There are only a few examples where they represent the best-available scientific knowledge, based on rigorous synthesis of evidence.

One such decision support tool is the online biodiversity metric incorporated into the Cool Farm Tool (available at www.coolfarmtool.org), which provides scores for the likely benefits for biodiversity of a range of farm management actions. The actions that are included are selected according to a combination of expert judgement and assessments of summarised evidence conducted by the Conservation Evidence project. Each farm management action is assigned scores reflecting the benefit for overall biodiversity, and also for 11 species groups (e.g. woodland birds, beneficial invertebrates), weighted according to the evidence. Actions that are strongly supported by the evidence provided by the Conservation Evidence syntheses (Sutherland et al., 2019a) are scored more highly than those for which effectiveness is not known.

Another example is the set of greenhouse gas emission calculators used in agriculture to support mitigation by changing farm management. These tools incorporate models of greenhouse gas emissions and carbon storage according to vegetation type and farming practice (Richards et al., 2016). These calculators combine empirical models with emission factors collated by the Intergovernmental Panel on Climate Change (see ‘National and International Scientific Assessments’). Although the outputs from these tools are only as good as the data that they are based on, new information can be added to improve their performance as it becomes available. For example, Richards et al. (2016) demonstrated that two widely used software tools tend to overestimate emissions from smallholder farms in tropical environments, but suggest that this is probably due to a systematic bias in literature, with most data coming from temperate regions, rather than bias in the models themselves. As empirical data are included from a wider range of environments, more accurate disaggregated emissions factors will become available for different parts of the world. If the decision support systems are maintained and updated, this new knowledge will directly influence decision-making at farm level.

8.4 Active knowledge exchange mechanisms

Active knowledge exchange mechanisms are the most diverse alignment mechanism of the three considered in this chapter. Our concept is similar to that of ‘boundary organisations’ identified by some other authors (Guston, 2001; Cook et al., 2013b), in that they operate in both scientific and practical spheres, but retain distinct lines of accountability to both groups. They can take a variety of institutional forms, from a dedicated, self-funded or government-funded organisation to a network of people working together across organisations (see also Chapter 13).

The reputation of such a body depends on its ability to produce or broker knowledge that is salient, credible and legitimate (Cash et al., 2003; Sarkki et al., 2015) while maintaining transparency. Credibility refers to the scientific adequacy of the technical evidence and arguments. Salience is the relevance of the brokered knowledge to the needs of decision-makers. Legitimacy reflects the perceptions that the production of information has been respectful of stakeholders’ divergent values and beliefs, unbiased in its conduct and fair in its treatment of views and interests. Achieving all these values requires adequate attention to governance from the outset.

Here, we provide examples of knowledge exchange mechanisms operating at a subnational scale, related to a particular environmental issue or landscape (Wadden Sea case study); at a national or international scale but restricted to environmental science (EKLIPSE mechanism); and at a national or international scale ranging across all scientific knowledge (European Scientific Advice Mechanism, and UK Parliamentary Office of Science and Technology).

8.4.1 Management of the Wadden Sea

At a subnational scale, van Enst et al. (2016) provided a detailed case study of three contrasting knowledge exchange mechanisms that have been important in aligning scientific evidence with policy and management decisions around the Wadden Sea, a shallow estuarine sea in the Netherlands. Competing cockle-fishing, gas extraction and biodiversity conservation interests generate continuous debate over the scientific knowledge, and the strategic use or misuse of such knowledge has played a pivotal role in disputes (Floor et al., 2013). Knowledge exchange mechanisms were devised to improve the transparent use of evidence. Two of the knowledge exchange mechanisms were government-funded: the Wadden Academy, a science-led organisation that oversees monitoring and data-gathering, and the Netherlands Commission for Environmental Assessment, which produces official reports. The third, IMSA Amsterdam, is a commercial think-tank and consultancy, focused on mediating between stakeholders, science and policy. These three organisations worked together to improve the salience, credibility and legitimacy of the scientific knowledge that was available, allowing it to be influential in decision-making related to the cockle-fishery and gas-exploitation controversies. Their efforts ultimately reduced conflict and improved environmental outcomes for the Wadden Sea, for example by enabling more sustainable fishing methods to be adopted (van der Molen et al., 2015; van der Molen, 2018).

8.4.2 The EKLIPSE mechanism

Knowledge exchange mechanisms focused on one environmental issue can develop deep, long-term relationships between a core set of stakeholders and researchers. When operating across many different issues at national or international scale, relationships with experts and other stakeholders are generally short-term and must continually be re-established as the topic of interest to policy changes. One possible approach to this is provided by the EKLIPSE mechanism (Watt et al., 2018; www.eklipse-mechanism.eu), which engages relevant actors from science, policy and society to identify evidence relevant to European policy. EKLIPSE accepts requests for knowledge synthesis on specific issues from policy-makers and other societal actors. A wide network of knowledge-holders can respond to the request, often through the formation of an expert working group (Wyborn et al., 2018). To give an example, the European Commission requested scientific knowledge on how to evaluate nature-based solutions (solutions inspired and supported by nature) for their ability to enhance sustainability in cities. In response, EKLIPSE convened a pan-European expert group to conduct a rapid evidence assessment and build a framework for evaluating the costs and benefits of nature-based solutions. This was disseminated as a policy report and an open-access scientific paper (Raymond et al., 2017).

8.4.3 The European Scientific Advice Mechanism and UK Parliamentary Office of Science and Technology

At a larger scale, knowledge exchange mechanisms can provide an interface between science and policy across all scientific issues. Usually these are national or international, such as the UK Parliamentary Office for Science and Technology (POST; Norton, 1997) and the European Union Scientific Advice Mechanism (ec.europa.eu/research/sam/index.cfm). At this level, knowledge exchange mechanisms have tended to settle on one particular way of doing things that works. At the POST, for instance, a Board selects subjects for briefing notes, known as POSTnotes, from among ideas gathered from a range of sources, including parliamentarians, the public and other stakeholders (www.parliament.uk/post). POSTnotes are generally researched through a series of interviews with key experts. Almost 600 POSTnotes have been published since 1989, on subjects ranging from the psychological health of military personnel to new plant-breeding technologies. All are freely available online and held in the House of Commons library.

The European Union Scientific Advice Mechanism, on the other hand, responds to requests for advice from the ‘College of European Commissioners’ through a group of government-appointed scientific advisers. It delivers evidence review reports on specific issues, drawing on a network of expertise from more than 100 European scientific academies in over 40 countries (e.g. The Royal Society in the UK, Hungarian Academy of Sciences). For both it and POST, adherence to a clearly defined process is a way of building credibility and assuring transparency. However, it does not necessarily provide the flexibility to address the diversity of issues and problems faced by environmental policy decision-makers.

To summarise, active knowledge exchange mechanisms can have a range of scales, formats and institutional arrangements. This plurality is the best approach to linking science and policy in decision-making contexts, where different types of questions continually arise.

8.5 National and international scientific assessments

A longer-term approach to aligning evidence synthesis with conservation policy decisions involves governments or international bodies mandating large-scale, scientific assessments in broad areas of strong policy interest. Examples include the assessment reports conducted by the Intergovernmental Panel on Climate Change (IPCC; www.ipcc.ch), Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services (IPBES; www.ipbes.net) and Millennium Ecosystem Assessment (www.millenniumassessment.org; see Chapter 16 for further details of mechanism and function of the Millennium Ecosystem Assessment and the IPBES science–policy platform). These global assessments involve hundreds or even thousands of scientists around the world, including indigenous and local knowledge-holders in the case of IPBES (Sutherland et al., 2014; see also Chapter 16).

Generally, governments define the scope of the assessment and identify or nominate a set of experts to conduct it (IPCC, 2015). The nominated experts form working groups and develop report texts, which are subject to extensive, transparent review, first by other experts and then by governments. Following review, the report texts are converted into concise summary documents (usually called ‘Summary for Policy-makers’), the final text of which is agreed by governments. Each statement in the summary document must be traceable back to the full scientific report and, from there, to individual pieces of research or sources of knowledge. Through this process, science and policy influence one another in a two-way exchange of knowledge over very large temporal and spatial scales.

The IPCC, which has been active for almost three decades, has built a strong reputation for providing an overview of climate science across a range of disciplines, from geophysics to economics. There are now clear links from the scientific understanding of human-induced climate change and its impacts to policies controlling greenhouse gas emissions at national and international levels. Most recently, the Paris Climate Agreement of December 2015 is a global accord under which nations have made pledges and set emissions targets to keep global temperature rise below 2°C (Clemencon, 2016; Tobin et al., 2018). A large quantity of scientific research underlies these policy pledges, which would likely not have happened, or not have been so extensive, without the IPCC assessment process. Forty-five different global climate models are now being used together to link levels of greenhouse gas emissions to long-term global temperature rise under different emissions scenarios (Collins et al., 2013). There is also a plethora of analyses and modelling connecting economic activity to greenhouse gas emissions (e.g. Vandyck et al., 2016) and threshold temperate rises with specific impacts on environments, economies and human well-being (IPCC, 2014).

The Millennium Ecosystem Assessment (2005) was the first global evaluation of the status of ecosystems, and developed the ecosystem services framework for understanding how nature can benefit people. The ecosystem services concept originated in the academic world (Potschin & Haines-Young, 2016), but the Millennium Ecosystem Assessment formalised the thinking, providing a conceptual framework and nomenclature for ecosystem services. Since its publication, a growing number of countries have conducted their own national ecosystem assessments (Schrӧter et al., 2016) and the policy ground is being set for their results to be used in national natural-capital accounting. Both Aichi Biodiversity Target 2 from the Convention on Biological Diversity’s Strategy Plan 2011–2020 (Convention on Biological Diversity, 2010) and Action 5 of the EU Biodiversity Strategy to 2020 (European Commission, 2011) call for biodiversity values to be incorporated into national accounting.

Large-scale assessments are most effective at aligning scientific evidence with decisions when there is a broad issue of strong political interest, such as climate change or biodiversity loss. The assessments are expensive (see Table 8.1), so there must be substantial political commitment and a source of funds over the relatively long term.

Table 8.1 A summary of the costs associated with three mechanisms to align evidence synthesis with policy and practice in the environmental field, compared to the costs of individual evidence synthesis methods

Activity	When to apply	Cost (£)
Mechanisms to align evidence synthesis with the needs of policy and practice
Decision support tools	Specific question, repeatedly addressed	380,000–3.9 million per tool1
Knowledge exchange mechanisms	Many questions arising	600,000 per year2
International assessments	One big, broad issue	~3 million per year3

Individual evidence synthesis methods
Systematic review	Many studies address a single question	19,000–190,0001
Subject-wide evidence synthesis	Multiple sources of relevant evidence exist	Initial cost: 45,000–480,000 Update cost: 20% of initial cost1

¹ Dicks et al. (2014);

² Cost of the EKLIPSE mechanism;

³ www.ipcc.ch; www.ipbes.net.

Given the obvious power of national and international scientific assessments to influence policy, it is now more important than ever to incorporate into them the transparent, unbiased repeatable methods that have been developed for evidence synthesis. Currently, the rigour and reliability of large-scale scientific assessments rely on extensive peer review, rather than systematic searching or careful elicitation methods that reduce bias. Evidence synthesis methods are usually not reported (with some exceptions, such as chapter 6 of the Intergovernmental Science Policy Platform on Biodiversity and Ecosystem Services pollination report; IPBES, 2016). However, such assessments are conducted over long timescales, with the IPCC, for example, producing a global assessment report every 5–10 years. With this amount of time and money available (see Table 8.1) there is a clear opportunity to develop rigorous processes of evidence synthesis within this framework. As a first step, we urge policy-makers and institutions involved in commissioning large-scale scientific assessments to require authors to report their underlying synthesis methods.

8.6 What does it all cost?

The cost of the alignment mechanisms outlined in this chapter varies considerably, both within and among the different activities (Table 8.1). These costs should be interpreted in the context of total spending on scientific research. For example, the budget of the European Commission’s flagship scientific research programme, Horizon 2020, is approximately £8 billion per year.

The organisations that fund research and aspire to be evidence-informed already invest heavily in improving interactions between science, policy and practice. Unfortunately, they frequently fund expensive decision support systems that are not maintained or used a few years later (Zasada et al., 2017) and large-scale reviews or scientific assessments that do not follow clear protocols to reduce bias. The challenge in aligning evidence synthesis with decision-making is not to find the money, but to demand and enable improved rigour and continuity in activities that are already taking place.

No single mechanism will be best for aligning evidence with policy and practice in all contexts. Each has strengths and weaknesses, and can be applied in different circumstances and at different scales. International assessments have redirected policies and scientific endeavour on a very large scale, but would be unlikely to align specific scientific findings with conservation practice at smaller scales. At smaller scales, the potential of decision support systems to incorporate rigorously collated environmental evidence has hardly been tapped.

At every level, mechanisms to link synthesised evidence with policy and practice decisions need to be funded sufficiently to ensure salience, legitimacy, credibility and transparency. These linking mechanisms need access to methods of collating and communicating evidence that are well-developed, transparent and widely understood (Cook et al., 2017; Dicks et al., 2017) and are just as important as the research itself, if not more so.