Evaluating the effectiveness of protected area management in Indonesia

Abstract

Protected areas worldwide are strongholds for safeguarding biodiversity, natural habitats, ecosystem services and cultural values. Yet despite their importance, the effectiveness of protected area management varies greatly. Indonesia is a biodiversity hotspot, with 554 protected areas that cover 27 million ha across the archipelago. To assess and improve the management effectiveness of these protected areas, the Government of Indonesia applied an adapted version of the Management Effectiveness Tracking Tool (METT) to assess 422 of the country's protected areas, of which 170 were repeatedly assessed in 2015, 2017 and 2019. We investigated the METT score changes across these protected areas and the factors explaining the varying scores. Over the study years, METT scores significantly improved (mean increase of 44.1%). National parks had the highest mean score, which was 13.4 points higher than other protected area types. After correcting for spatial autocorrelation using a generalized least-squares model, we found that METT score increase was positively influenced by year of assessment and having a well-resourced management authority, with no influence of protected area size or mean protected area budget allocation per ha. The assessments identified five main threats to protected areas: poaching, illegal logging, human settlements, tourism and non-timber cultivation. The widespread and repeated use of METT across the protected areas of Indonesia and the increasing METT scores indicate an overall improvement in management and professionalism. Building on the foundational work in our study, future studies should assess the association between METT scores and progress made towards achieving the conservation objectives of protected areas.

Introduction

Protected areas are a mainstay strategy for managing biodiversity and ecosystems whilst preserving cultural values (UNEP-WCMC et al., 2018). The role of protected areas is becoming increasingly important given the global trends of biodiversity and forest loss (Betts et al., 2017; Schulze et al., 2018). Well-managed protected areas play an essential role in safeguarding habitats, maintaining viable populations of threatened species, preserving carbon stocks and contributing to poverty alleviation in surrounding areas (Geldmann et al., 2013; Watson et al., 2014; Barnes et al., 2016; Gray et al., 2017; Graham et al., 2021). Nevertheless, not all protected areas achieve their objectives, especially those that are under-resourced (Barber et al., 2012; Watson et al., 2014; Maxwell et al., 2020).

The vast Indonesian archipelago comprises > 17,000 islands and harbours exceptionally rich terrestrial biodiversity (Sanciangco et al., 2013; Veach et al., 2017; Gagné et al., 2020) and extensive tropical forests (Bertzky et al., 2013). This great diversity includes, for example, the Sumatran tiger Panthera tigris sumatrae, Sumatran orangutan Pongo abelii, Sumatran elephant Elephas maximus sumatrensis and Sumatran rhinoceros Dicerorhinus sumatrensis in Sumatra, the maleo Macrocephalon maleo, Sulawesi babirusa Babyrousa celebensis and anoas Bubalus spp. in Sulawesi, and tree kangaroos Dendrolagus spp., echidnas Zaglossus spp. and cassowaries Casuarius spp. in Papua (Ministry of Environment and Forestry, 2020b). To conserve this rich diversity of wildlife and natural habitats, protected areas form an integral part of a nationwide strategy of the Government of Indonesia, with 27 million ha (14% of the land area of Indonesia) having been designated as protected areas (Ministry of Environment and Forestry, 2020a).

Aichi Target 11 requires that by 2020 at least 17% of terrestrial and inland water areas be conserved through effectively and equitably managed, ecologically representative and well-connected systems of protected areas (CBD, 2010). This is also in accordance with the post-2020 Global Biodiversity Framework, Target 3, particularly on conserving and effectively managing protected areas. In recognition of this, the Government of Indonesia undertook a nationwide evaluation of the effectiveness of its network of 554 protected areas, starting in 2015. The evaluation set out to assess, monitor and improve the management quality of these protected areas, so that they achieve their objectives (Hockings et al., 2006; Ministry of Environment and Forestry, 2016). The findings were then used to guide management planning (strategic and financial) and enable more efficient resource allocation (Hockings et al., 2002). This evaluation also responds to the Convention on Biological Diversity Programme of Work of Protected Areas (CBD, 2004) on expanding and institutionalizing assessments of management effectiveness, assessing 60% of the total area of protected areas of each Party by 2015 and reporting the results to the global database on management effectiveness. The purpose of the evaluation is to improve the overall quality of protected area management by enabling adaptive management over time (Coad et al., 2015). This approach improves accountability by determining whether the money, time and human resources invested in a protected area are resulting in improved management (Dudley et al., 2004).

The Management Effectiveness Tracking Tool (METT) is commonly used to evaluate the effectiveness of protected area management (Coad et al., 2015; Stolton & Dudley, 2016). The framework is divided into three main themes with six management elements: design or planning (context and planning), adequacy or appropriateness (input and process) and accomplishment or achievement (output and outcome; Hockings et al., 2006). It was launched in the early 2000s and has been used to assess more than 5,000 protected areas in 127 countries (Stolton et al., 2021). Moreover, in 2021, IUCN launched an enhanced METT-4 to emphasize outcomes related to conservation, climate change and ecosystem services (UNEP-WCMC, 2021). The Government of Indonesia adapted the METT assessment terminologies to fit the national context, and METT was officially adopted (through regulation Peraturan Dirjen KSDAE No. P.15/KSDAE-SET/2015) as the Ministry of Environment and Forestry's tool for evaluating the management effectiveness of the entire protected area network, beginning in 2015 (Ministry of Environment and Forestry, 2017).

In this study, we used data from METT assessments conducted in 2015, 2017 and 2019 to evaluate temporal changes in management effectiveness scores, identified the factors that influenced these scores and determined the reported threats faced by protected areas. We then used our findings to formulate a set of recommendations to further improve protected area management in Indonesia.

Methods

Protected areas in Indonesia and METT evaluations

Based on Law No. 5 Year 1990, which regulates the conservation of natural resources and ecosystems in Indonesia, protected areas are categorized into five types based on their function: Strict Nature Reserve (Cagar Alam), Wildlife Reserve (Suaka Margasatwa), National Park (Taman Nasional), Natural Monument (Taman Wisata Alam) and Grand Forest Park (Taman Hutan Raya). These types can generally be mapped to the equivalent definitions of IUCN protected area categories (Supplementary Fig. 1). Based on Law No. 41 Year 1999 on Forestry, another type of protected area also exists: Habitat/Species Management Area (Taman Buru); however, we excluded this from our main analysis because only one such site had a METT score.

The METT assessment is divided into two main parts: baseline information regarding the protected area, which includes its condition and threats profile; and an evaluation form, consisting of 30 questions with answers grouped into four scores that represent a range of conditions, from not sufficient (0) to sufficient (3) (Stolton & Dudley, 2016). The Government of Indonesia adopted this tool in 2015, adding an explanation for each condition and adjusting the terminology to align with local management practices. Specifically, the following four modifications were made: (1) A guideline was provided with detailed explanations of each question on the METT assessment forms. This includes interpretation of global terminologies in the Indonesian context and standardized indicators for the scoring system. Assessors (protected area managers/representatives) are also required to provide certain documents as evidence material to support the evaluation. An example of this is provided in Supplementary Material 1. (2) During the assessment, the assessors were obliged to list improvement targets for the next evaluation period, including the evidence materials required to confirm any improvement. In the following METT assessment periods (in 2017 and 2019), scores were given based on whether these targets were achieved or not. (3) To reduce assessment bias, external parties were invited to join the assessment and were required to provide their perspectives regarding the ongoing evaluation. These parties include representatives of the local government, private sector, academia, local communities and NGOs who partner with the protected area authority. (4) The assessment results and the improvement targets were then verified by an appointed working group formed by the Ministry of Environment and Forestry and a decree was issued to legally formalize these.

Indonesia has 554 designated protected areas (Fig. 1), all of which are managed by the Ministry of Environment and Forestry, except for Grand Forest Park, which is managed by the local government (Supplementary Fig. 1). In total, 422 (76%) of these protected areas have been evaluated through a METT assessment. Specifically, 283 (51%), 398 (72%) and 344 (62%) of these protected areas were evaluated in 2015, 2017 and 2019, respectively (Supplementary Fig. 2a). This includes 170 protected areas (31%) that were assessed in all 3 years, and five protected areas that were evaluated in 2 years (2017 and 2019). Of the protected areas evaluated, 130 (23%) were assessed for their management effectiveness and threat levels (Supplementary Fig. 2a). Data on budget, protected area size and number of staff were available for 130 (23%), 199 (36%) and 367 (66%) of Indonesia's protected areas in 2015, 2017 and 2019, respectively (Supplementary Fig. 2b).

Fig. 1 Spatial distribution of conservation areas across Indonesia, including areas assessed using the Management Effectiveness Tracking Tool (METT) three times (in 2015, 2017 and 2019).

In this study, we assigned a protected area to one of three types based on its management authority: (1) National Park: a protected area managed by a single management authority under the Ministry of Environment and Forestry. (2) Grand Forest Park: a protected area managed by a single management authority that falls under the provincial or district/city government. (3) Nature Reserve/Wildlife Reserve/Natural Monument: includes several types of protected areas that are managed by a single provincial authority under the overall authority of the Ministry of Environment and Forestry.

Data collection

Site managers compiled data from the 170 protected areas with METT assessments conducted in 2015, 2017 and 2019 using a participatory approach that followed the guidelines prepared by the Ministry of Environment and Forestry. In general, evaluations were conducted through discussions between protected area staff and other external parties, although the involvement of the various parties in the evaluation process differed between sites and was dependent on the site-specific management and partnerships held. To ensure consistency in the approach used to complete the METT forms, one or more trained facilitators guided each site-based process.

To further reduce bias in the evaluation, which is essentially a self-assessment process, a working group set up by the Ministry of Environment and Forestry Directorate General of Nature Resources and Ecosystem Conservation verified each form. The working group proposed several recommendations to improve future METT assessments, particularly after the baseline assessment in 2015 (SK Dirjen KSDAE No. 357/KSDAE-SET/2015, SK Dirjen KSDAE No. 357/KSDAE/SET/KSA.1/5/2016). These recommendations primarily focused on the following aspects: strengthening organizational management capacity, particularly related to human resources; providing a standard framework as a reference for management implementation; optimizing resource allocation; and strengthening policy at all levels to improve management.

Data analyses

We conducted all analyses using R 4.1.3 (R Core Team, 2022) and the following packages: nlme, corrplot, tidyverse, vtable, ggpubr, dotwhisker, MASS, factoextra, gstat, sp, rr2 and ggfortify.

We conducted a Kruskal–Wallis test to investigate several questions regarding the protected area METT scores and the identified threats. We assessed differences in METT scores between protected areas and years, differentiated by their function (National Park, Grand Forest Park and Nature Reserve/Wildlife Reserve/Natural Monument) and management authority (using the same three protected area categories).

To explain the variation in METT scores across years and protected areas, after correcting for temporal and spatial autocorrelation (Supplementary Figs 3 & 4), the heterogeneity of data residuals and comparing Akaike information criterion (AIC) values (Supplementary Fig. 5), we used a generalized least-squares model. The spatial autocorrelation was significant in slightly improving the model (Supplementary Figs 3 & 4). Protected area function and management authority were highly correlated (χ ² test of independence = 1,050, P < 0.0001), and we used the latter as a single explanatory variable. We excluded Grand Forest Park (one protected area managed by one management authority under the provincial or district/city government) from the analysis because most of these protected areas had incomplete datasets for the other explanatory variables (i.e. budget, protected area size and threats).

We tested for correlation (Kendall's rank correlation test) between other continuous variables that were assumed to influence METT scores, including protected area size, budget per unit area and number of staff per unit area. As protected area size was highly correlated with the number of staff (> 0.75 correlation, P < 0.0001), we omitted the latter from the regression model. Budget and protected area size had different ranges and we standardized them using a scaling function in R and subsequently centralized them. The scale function divides the values by their standard deviation or root mean square, whereas centring subtracts the mean of each column from the values. This reduced the effect of different scales when comparing vectors (i.e. budget and area size) and brought the vectors closer to a normal distribution. We included assessment year (2015, 2017 and 2019) as an explanatory categorical variable.

We compared threat values and identified whether the 52 threats (Supplementary Table 1) differed between years and protected area types. We used a Kruskal–Wallis test to check for differences in each pair of threat indicators using the post-hoc pairwise Wilcox test with Benjamini–Hochberg adjustment. We further used the five main threats identified as explanatory variables in the regression model to explain differences in METT scores. The model was then tested using the following equation:

$$\eqalign{{\rm METTscore}& \sim{\rm budget}\;{\rm per}\;{\rm unit}\;{\rm area}\;{\rm of}\;{\rm protected}\;{\rm area}\;{\rm size} \cr & \quad + {\rm protected}\;{\rm area}\;{\rm size}{\mkern 1mu} + {\rm year}\;{\rm of}\;{\rm assessment}{\mkern 1mu} \cr & \quad + {\rm site}\;{\rm type}\; + {\rm threat}\; 1{\mkern 1mu} + {\rm threat}\; 2{\mkern 1mu} + {\rm threat}\; 3 \cr & \quad + {\rm threat}\; 4{\mkern 1mu} + {\rm threat}\;{\rm 5}} $$

To complement the interpretation of the previous model, we applied similar steps and the same equation but separated the analysis for National Park and Nature Reserve/Wildlife Reserve/Natural Monument (spatial autocorrelation using the ‘ratio’ correlation structure; Supplementary Fig. 6). We chose the best-fit model based on its AIC score, %R ² and residual plot (to inspect for heterogeneity).

Results

METT score trends over time

The mean METT scores of the focal protected areas significantly increased from 2015 (47.4 ± SD 11.2; n = 170) to 2017 (60.0 ± SD 11.3; n = 175) to 2019 (68.3 ± SD 8.8; n = 175; Kruskal–Wallis test, P < 0.001; Fig. 2a). This positive trend applied to all protected area types (Fig. 2a,b,c,e,f) except Grand Forest Park, which showed no significant change over time (P > 0.05; Fig. 2d). The largest mean METT score increase was for Wildlife Reserve (+ 64.7%), followed by Strict Nature Reserve (+ 48.4%), Natural Monument (+ 43.8%) and National Park (+ 17.2%).

Fig. 2 Temporal changes in METT scores for (a) all protected areas, (b) Strict Nature Reserves, (c) Wildlife Reserves, (d) Grand Forest Parks, (e) National Parks and (f) Natural Monuments in Indonesia that were assessed three times (2015, 2017, 2019). The boxplots show the median value (line inside the box), 25th and 75th percentiles (lower and upper box boundaries, respectively), 10th and 90th percentiles (lower and upper end of whiskers, respectively) and outliers (filled circles).

Variables influencing METT scores

Protected areas that were centrally managed (National Park) had significantly higher mean METT scores in 2019 (P < 0.0001; 75.6 ± SD 4.7) compared to protected areas under provincial government (Grand Forest Park; 68.0 ± SD 21.5) or Nature Reserve/Wildlife Reserve/Natural Monument (67.4 ± SD 9.24; Fig. 3).

Fig. 3 METT scores for the three management authority types of protected area in Indonesia: National Park (Type A), Grand Forest Park (Type B) and Nature Reserve/Wildlife Reserve/Natural Monument (Type C). The boxplots show the median value (line inside the box), 25th and 75th percentiles (lower and upper box boundaries, respectively), 10th and 90th percentiles (lower and upper end of whiskers, respectively) and outliers (filled circles).

For the six METT elements, the scores significantly differed between years (P < 0.0001) except for the context element, which is related to protected area legal status. This had a maximum score of 100 for all years because for a protected area to receive its legal status in the form of a Government of Indonesia decree, it would need to have already administratively fulfilled all of the evaluation requirements within the context element. METT scores for each element were similar and increased over time except for the output element (measured by goods and services produced by the protected area), which had a lower median value (Fig. 4).

Fig. 4 Scores for each of the six METT elements: (a) context, (b) planning, (c) input, (d) process, (e) output and (f) outcome for protected areas in Indonesia. The boxplots show the median value (line inside the box), 25th and 75th percentiles (lower and upper box boundaries, respectively), 10th and 90th percentiles (lower and upper end of whiskers, respectively) and outliers (filled circles).

The generalized least-squares analysis identified nine variables that explained 70% of the variation in protected area METT scores (Table 1). The year of assessment significantly influenced METT scores, with scores increasing by a mean of 44% over time. National Park had the highest METT scores compared to other protected area types. The budget per unit area, the protected area size and the five main threat variables did not significantly influence METT scores (Fig. 5). In the additional analysis in which we analysed National Park and Nature Reserve/Wildlife Reserve/Natural Monument separately, we found similar patterns, with year of assessment significantly influencing the METT score increase. Budget per unit area was not significant for both site types, with relatively large confidence intervals found in the National Park analysis (Supplementary Fig. 6).

Table 1 Summary of the best-fit model using the generalized least-squares model after correcting for spatial autocorrelation correction, which explained the trend in Indonesian protected area Management Effectiveness Tracking Tool (METT) scores (n = 223; Fig. 5). Total variance explained is %R ² = 70, Akaike information criterion = 1,576.7, with estimates of range and nugget parameters being 9.58 and 0.57, respectively.

***P < 0.001; *P > 0.05.

Fig. 5 Coefficient estimates (with 95% confidence intervals) for all variables that could influence the management effectiveness of protected areas in Indonesia (Table 1), which were tested based on the following equation: METT score ~ budget per unit area + protected area size + year of assessment + site type + threat 1 + threat 2 + threat 3 + threat 4 + threat 5.

Analysis of protected area threats

The five main threats to protected areas, as identified by their managers and staff, were poaching, illegal logging, development of settlements, tourism activities and non-timber cultivation. The five least important threats were introduction of novel genetic material, medicine farming, pathogens, oil and gas drilling and noise pollution associated with flight pathways (Supplementary Fig. 7). Each threat changed in severity from year to year but generally decreased from 2015 to 2019. Significant decreases were observed in the following threats: medicine farming, wood and pulp plantation, oil and gas drilling, illegal logging, loss of keystone species, habitat change and destruction, drought, extreme temperatures, loss of cultural links, local knowledge and/or management practices, and destruction of cultural heritage buildings, parks and sites (Kruskal–Wallis test, P < 0.05; Supplementary Fig. 7). More threats were identified in National Parks compared to other protected area types (Supplementary Fig. 8).

Discussion

To realize the aim of the Government of Indonesia of improving the effectiveness of protected area management and monitoring systems, 76% of the 554 protected areas in Indonesia have been evaluated using METT over a period of 5 years (2015–2019), of which 170 (31%) were evaluated three times. The METT process is prone to biases because it is based on a self-assessment approach (Stolton et al., 2021), and as the Government of Indonesia planned to improve protected area management through the METT process, assessors may consciously or unconsciously increase METT scores. Several steps were taken to reduce this potential bias, such as ensuring that evidence to support the evaluation is provided, and that improvement targets are formalized in a decree and then assessed in the follow-up METT cycle. In addition, external parties were invited during the METT process and required to provide their objective assessment on the condition of the protected areas. The METT assessment has also been adopted as one of the requirements for various conservation projects (e.g. Global Environment Facility projects and World Bank projects; Stolton et al., 2021), and therefore improved METT scores in certain protected areas could be related to these projects. However, this was not directly measured in our study, and the METT score improvements could be partially explained by the checks-and-balances process that has been set up to ensure that improvements are valid and measurable.

Factors influencing the management effectiveness of protected areas

Over the assessment years, METT scores increased across all protected area types. Protected areas with a single management authority, managed by the central government and with resources solely dedicated to the protected area (National Park) increased their METT scores less than other types but tended to start with a higher baseline METT score and had attained a higher METT score by 2019. When comparing the median values, we found that National Parks were better funded than Nature Reserves/Wildlife Reserves/Natural Monuments (the amount of funding available annually was c. 9 times greater in National Parks) but with fewer human resources per unit area (c. 10 times fewer; 2015–2019). However, it is also important to note that National Parks cover a much larger area than Nature Reserves/Wildlife Reserves/Natural Monuments (c. 246 times larger; Supplementary Table 2). This could indicate that although National Parks have fewer human resources per unit area, they may be more efficient in allocating their financial resources. Several studies have found that having sufficient and properly allocated human resources and financial support is important for effective management of protected areas (Wilson et al., 2006; Geldmann et al., 2015, 2019; Powlen et al., 2021). National Parks in Indonesia are often equipped with good management systems, demonstrated by their clearly demarcated boundaries, careful planning processes (annual and 10-year plans) and well-established facilities and infrastructure that improve their protection.

We found that budget did not significantly influence the management effectiveness of the protected areas. However, this does not mean that budget is not important for protected area management, but rather that budget allocation or budget efficiency may be more appropriate factors for measuring the role of funding. A global review by the World Conservation Monitoring Center (James et al., 1999) showed how the conservation costs of protected areas could vary by region, economic development level, population pressure and the degree of protected area fragmentation. We propose a more detailed study considering these factors to more clearly determine the role of budget. It has been shown previously that, on average, effective protected area management requires a budget of USD 0.9–9.0 per ha (Bruner et al., 2004). After adjusting these figures for inflation, the budgets available at the time of our study for National Parks and Nature Reserves/Wildlife Reserves/Natural Monuments in Indonesia would be insufficient (median values of USD 4.4 and 0.5 per ha, respectively). Thus, rather than being a non-significant factor, the current budgets may not suffice for METT score improvement. Moreover, in the additional analyses (Supplementary Fig. 6) we found that the confidence intervals for budgets in National Parks were large, implying significant variance in terms of how budgets influence different parks, although this was not significant overall.

Additionally, bias could arise during the documentation of budget data, with National Parks usually including staff salaries as part of the budget, whereas others (Grand Forest Parks and Nature Reserves/Wildlife Reserves/Natural Monuments) may exclude these from their total budgets. Addressing this possible disparity could help to further standardize future METT assessments.

In addition to providing key ecosystem services, National Parks, such as those in the Java–Bali bioregion (e.g. Gunung Halimun Salak National Park, Gunung Merapi National Park), typically comprise mountainous terrain and have strong management systems and park authorities that collaborate with surrounding communities. These features increase the protection of their primary forests and minimize land-cover change (Dwiyahreni et al., 2021). Several studies have shown that power dynamics between different parties, such as the government and local communities who have been granted direct access to the protected area, represent an important determinant of successful protected area management (Reed et al., 2018). Although the exact dynamics between the government as the management authority and the communities neighbouring protected areas require further investigation in the Indonesian context, National Parks tend to have stronger collaborations with surrounding communities and other civil society organizations. This may partially explain why this protected area type had the highest management effectiveness score. For example, in collaboration with various civil society organizations, several National Parks (e.g. Bukit Barisan Selatan, Gunung Leuser, Berbak-Sembilang and Kerinci Seblat), established a monitoring system through Spatial Monitoring and Reporting Tool (SMART) patrols (Risdianto et al., 2016; Efendi et al., 2019; Lubis et al., 2020; Ariyanto et al., 2021). This law enforcement monitoring enables National Parks to better identify threats and allocate budgets and human resources accordingly. Linking METT application to the use of SMART could strengthen METT as SMART is a widely used, standardized tool that can help in regular field-based data collection and provide automatization of data analysis and data sharing. SMART can also help in obtaining evidence, particularly for threat assessments (Stolton et al., 2021), reducing the bias in the self-assessment approach of METT.

We found year of assessment to influence protected area METT score. This could be because previous evaluations identified aspects in protected area management that needed to be improved and subsequently were. These management recommendations were also formally integrated within government policy (SK Dirjen KSDAE No. 357/KSDAE-SET/2015, SK Dirjen KSDAE No. 357/KSDAE/SET/KSA.1/5/2016), which lists several recommendations for protected area managers such as establishing a mechanism for handling law enforcement in each protected area, and facilities for tourism and research activities (accessibility, sanitation, safety, comfort and information) in accordance with the existing management plan.

We found that scores for all METT elements (except context) increased significantly over time. However, the range of scores for the output element was larger than the ranges for other elements, with several protected areas scoring 0 for the output element over the years. The output element is measured according to the existing regular work plan and its implementation and whether visitor facilities (if they are needed) are adequate. Some protected areas did not yet have an annual work plan in 2019, which could be because they lacked a long-term management plan or did not consider a work plan to be an important aspect of management. Most protected areas that received 0 scores were Strict Nature Reserves, which are closed to visitors, so the scoring of this indicator may not be relevant to such protected areas.

METT is considered to be limited in its measurement of outcome elements (Mascia et al., 2014), and it may not be directly relevant for the output element. This is because it measures whether the protected areas deliver economic benefits for local communities and whether indicators of the important values (e.g. biodiversity, ecological, cultural) are improved compared to when these values were first designated. Thus, improving the outcome element may be more indirectly related to the planning, input and process elements, which could explain why these elements varied in a more similar fashion to each other than the output element.

Periodic METT evaluations are important for increasing the effectiveness of protected area management, and they have become a standard approach in Indonesia. However, studies that investigate trends in periodic evaluations remain limited. Such studies could uncover new management challenges whilst also ensuring the delivery of management objectives such as increasing populations of protected species, maintaining forest cover and carbon stores, and ensuring the sustainability of ecosystem services, all of which require long-term monitoring (Geldmann et al., 2015). The Government of Indonesia enacted a policy to ensure that METT would be used as the standard evaluation method (SK.234/KSDAE-KK/2015), and protected areas should be assessed biannually to anticipate changes in management effectiveness and threats whilst measuring the impacts of interventions over time (Geldmann et al., 2015).

Another potential bias was introduced by selecting the 170 protected areas that were assessed three times (with an additional five protected areas that were evaluated twice) , as the number of assessments is probably related to management capacity (i.e. well-managed protected areas will be assessed more frequently). Nevertheless, our study contained a large sample of various protected area types, and a preliminary analysis of the protected areas that had been evaluated twice showed a similar trend of increasing METT scores.

Threats to protected areas

The five key threats to protected areas in Indonesia, as reported by their management authorities, were related to extractive activities that threaten wildlife and forest cover. This aligns with a previous global study that found poaching, illegal logging, non-timber forest product extraction and tourism to be the most significant threats (Leverington et al., 2008). A study from the Brazilian Atlantic Forest identified the expansion of human settlements as a primary threat to protected areas (Coelho Junior et al., 2020), as did a study on natural UNESCO World Heritage sites such as the three National Park clusters in Sumatra (Setyawati et al., 2021). National Parks appear to be subject to more threats than other types of protected areas, possibly because of their intrinsic value and the various benefits they provide (i.e. ecosystem services) and the well-developed tourism facilities that provide easy access. Moreover, National Parks are usually monitored more intensively and regularly, and threats are documented more accurately than in other protected area types, which could contribute to the perception of more threats.

Our study found that illegal extractive activities tend to decrease METT scores, whereas legal extractive activities, such as non-timber cultivation and tourism, tend to increase METT scores. This is probably because legal extractive activities, when identified and subsequently addressed through better management, can increase effectiveness scores. The output element was evaluated based on whether the area has sufficient tourism facilities. Tourism has a negative impact on METT scores because the associated infrastructure and visits can damage vegetation, cause changes in hydrology and soil conditions and spread pathogens and invasive species (Pickering & Hill, 2006). However, even if the threats from tourism are relatively high, such as in Bangko-Bangko Natural Monument (West Nusa Tenggara Province) and Pancar Mountain Natural Monument (West Java Province), management effectiveness can still be increased if protected area managers can anticipate these threats and develop adequate tourism facilities. It has been argued previously that the paradigm of tourism being a threat to protected area management may need to be repositioned as an opportunity to enhance protected area effectiveness if tourist-related activities are managed to minimize their negative impacts on the area through sustainable practices and investments in nature-based solutions (Weaver & Lawton, 2017; IUCN, 2022).

Most of the threats that decreased over time are related to extractive activities. However, other phenomena such as drought and extreme temperatures (which are partly attributable to natural causes but may also be exacerbated by human-induced climate change) were also decreasing threats. Further investigation using long-term data is required to determine how these threats are related to management effectiveness and to validate our findings.

METT is limited in its ability to measure the outcomes of management effectiveness, and there is a need to consider other, more objective metrics of protected area performance (e.g. deforestation rates and species population trends). Such metrics are missing from our study because of a lack of comparable data across the 170 focal protected areas. Nevertheless, several studies have been conducted to measure these aspects, particularly using quasi-counterfactual methods. Graham et al. (2021) used matching methods to compare protected areas with non-protected landscapes and found that protected areas that had completed management reporting using METT had three times less forest cover loss than similar landscapes without protection. In addition, Powlen et al. (2021) found that protected areas with greater management effectiveness are better at reducing deforestation compared to those with lower management effectiveness or to unprotected areas.

Future studies

We have identified several follow-up studies that could improve our understanding of the value of management effectiveness evaluations. These studies would build on knowledge of the importance of measuring intervention impacts that are not directly measured in the METT approach as a proxy for the ‘input’ and ‘process’ elements (Mascia et al., 2014). We recommend that the following areas should be examined by future studies: (1) factors that could influence the effectiveness of a single management unit (National Park); (2) other proxies that could be used to better understand the role of budget (e.g. budget efficiency, budget allocation) in influencing management effectiveness; (3) the role of local communities and other stakeholders in protected area management effectiveness (taking into account available human resources and funding); (4) benefits of SMART use for improving management effectiveness; (5) in-depth analysis of threats to protected areas, especially on how potential threats such as tourism and non-timber cultivation, which are legal and benefit local communities, can be adapted to improve sustainable management and/or how phenomena such as droughts and extreme weather could be explained by management effectiveness. Such studies would help to ensure that Indonesia achieves its global biodiversity and protected area targets.