Researchers and practitioners are increasingly embracing systems approaches to deal with the complexity of public service delivery and policy evaluation. However, there is little agreement on what exactly constitutes a systems approach, conceptually or methodologically. We review and critically synthesize systems literature from the fields of health, education, and infrastructure. We argue that the common theoretical core of systems approaches is the idea that multi-dimensional complementarities between a policy and other aspects of the policy context are the first-order problem of policy design and evaluation. We distinguish between macro-systems approaches, which focus on the collective coherence of a set of policies or institutions, and micro-systems approaches, which focus on how a single policy interacts with the context in which it operates. We develop a typology of micro-systems approaches and discuss their relationship to standard impact evaluation methods as well as to work in external validity, implementation science, and complexity theory.

In 2018, Nigeria began the implementation of a cash transfer programme (CCT) for poor and vulnerable people. We evaluated the impact of cash transfer on household livelihood outcomes in Nigeria. Using multistage cluster sampling methodology, beneficiaries and non-beneficiaries within the same locality were randomly selected to participate in a survey to assess the impact of cash transfer on food security and food diversity.

When gender, marital status, educational status, and age were controlled, beneficiaries were about three times more likely than non-beneficiaries to report experiencing little or no hunger. Children 0–59 months of beneficiaries were twice likely to have at least three meals a day compared to children of non-beneficiaries. Difference in differences regression analysis showed that on the average, beneficiaries of the cash transfer significantly consumed more diverse food than non-beneficiaries. Beneficiaries of the CCT experienced fewer episodes of severe hunger, have more meal frequency, and higher household dietary diversity than non-beneficiaries. This shows that the CCT programme is effective and can directly mitigate adverse effects of malnutrition with its long-term negative impact on children and thus must be expanded to more vulnerable people across all states in Nigeria.

This article evaluates the long-term impacts of the Chicago Child-Parent Centers (CPC), a comprehensive early childhood program launched in the 1960s, on physical and mental health outcomes. This study follows a cohort of 1539 participants born in 1979–1980 and surveyed most recently at age 35–37 by employing a matched study design that included all 989 children who entered CPCs at ages 3 and 4 (1983-1985) and 550 comparison children of the same age from randomly selected schools participating in the usual district early childhood programs in kindergarten. Using propensity score weighting that addresses potential issues with differential attrition and non-random treatment assignment, results reveal that CPC preschool participation is associated with significantly lower rates of adverse health outcomes such as smoking and diabetes. Further, evaluating the economic impacts of the preschool component of the program, the study finds a benefit-cost ratio in the range of 1.35–3.66 (net benefit: $3896) indicating that the health benefits of the program by themselves offset the costs of the program even without considering additional benefits arising from increased educational attainment and reduced involvement in crime reported in earlier cost-benefit analyses. The findings are robust to corrections for multiple hypothesis testing, sensitivity analysis using a range of discount rates, and Monte Carlo analysis to account for uncertainty in outcomes.

Conservation scientists are increasingly recognizing the need to evaluate the effectiveness of interventions to improve human–wildlife coexistence across different contexts. Here we assessed the long-term efficacy of the Long Shields Community Guardians programme in Zimbabwe. This community-based programme seeks to protect livestock and prevent depredation by lions Panthera leo through non-lethal means, with the ultimate aim of promoting human–lion coexistence. Using a quasi-experimental approach, we measured temporal trends in livestock depredation by lions and the prevalence of retaliatory killing of lions by farmers and wildlife managers. Farmers that were part of the Long Shields programme experienced a significant reduction in livestock loss to lions, and the annual number of lions subject to retaliatory killing by farmers dropped by 41% since the start of the programme in 2013, compared to 2008–2012, before the programme was initiated. Our findings demonstrate the Long Shields programme can be a potential model for limiting livestock depredation by lions. More broadly, our study demonstrates the effectiveness of community-based interventions to engage community members, improve livestock protection and ameliorate levels of retaliatory killing, thereby reducing human–lion conflict.

We present a meta-evaluation of the literature on the impacts of active labour market policies for unemployed people over 50, extracting 82 impacts for analysis. The meta-evaluation includes only impact evaluations that examine both a group of beneficiaries and a control group of comparable non-beneficiaries. On average, we find that active policies have a slightly negative effect (−0.8 percentage points) on the probability of unemployed people over 50 finding a job and that this negative effect disappears 24 months after policy implementation. However, this effect is very different when disaggregated by policy type. Direct job creation policies have a clear negative effect (−3.9 percentage points), and training policies have a positive average effect, either in isolation (2.4 percentage points) or when combined with search assistance or counselling (1.7 percentage points). We also find slight differences by gender, with the effect of active policies being greater for women than for men. These results have important implications, given that the European Commission and the Organisation for Economic Co-operation and Development have made active policies the cornerstone of their efforts to improve the re-employment of older people. Our results support training policies, either in isolation or in combination with search assistance and counselling. The greatest impacts are obtained after 12 months of policy implementation.

There has been a rapid expansion in the use of payments for environmental services (PES) as a key conservation finance policy. However, there is insufficient understanding of how gender can affect PES implementation and outcomes. We present results from a case study in Viet Nam, where a national PES programme has been in place for a decade. Through panel household survey data, focus groups and interviews, we examined how women have been involved in PES policies, what the impacts have been on decision-making by men and women, participation rates and use of PES income over time, and the potential conservation outcomes. Our research confirms that resource use varies between men and women, and changes in access rights can fall disproportionately on women. Participation in PES has been lower for female-headed households and for women within male-headed households, although gradually more equitable participation has evolved within households. Female-headed households reported expending more yearly effort on PES activities despite protecting less land, and also increased their conservation activities over time as they presumably became more familiar with PES. Use of income from PES also showed differences between male and female-led households, with men more likely to spend funds on non-essential goods. Within households, although men initially decided how to spend PES money, decision-making has become more equitable over time. We conclude with some recommendations on how to increase attention to gender in PES projects and future research to improve outcomes.

The Makira Natural Park (Makira) is the largest terrestrial protected area in Madagascar, supporting nearly 200,000 people living around the park, who derive a significant proportion of their animal-source foods from hunting. Makira is currently managed with three different policies: (1) traditional management with no external assistance; (2) community-based forest management (Gestion Contractualisée des Forêts; GCF) with non-governmental support and externally developed policies; and (3) strict external management policies (Zone d’Occupation Controlée; ZOC), meaning a zone of controlled residence for communities inside the core protected area. Here, we longitudinally assess the impacts of these policies on wildlife hunting and consumption by using a difference-in-differences method to evaluate policy efficacy. We found that externally developed conservation policies led to drastically reduced wildlife hunting and consumption in Makira. We also found that public education in the form of environmental awareness-raising, conducted 1 year before adoption, led to a greater concealment of hunting behaviours. Overall, wildlife hunting and consumption have decreased in the Makira region since the adoption of GCF and ZOC policies and could be effective at reducing the unsustainable wildlife hunting across the Makira, particularly when supported by economic development and the creation of alternative sources of livelihood and food.

Subsidies to promote tree plantations have been questioned because of negative impacts of the forestry industry. Quantitative evidence on the socioeconomic impacts of afforestation subsidies or of tree plantations is elusive, mainly due to data scarcity. We assess the overall impact of a tree plantation subsidy in Chile, using our original 20-year panel dataset that includes small area estimates of poverty and the subsidy assignment at the census-district scale. We show that forestry subsidies – on average – in fact, do increase poverty. More specifically, using difference in difference with matching techniques, and instrumental variables approaches, we show that there is an increase of about 2 per cent in the poverty rate of treated localities. We identify employment as a causal mechanism explaining this finding, since we found a negative effect of tree plantations on employment, and therefore, on poverty. We suggest reassessment of the distributional effects of the forest subsidy and forestry industry.

The effectiveness of many widely used conservation interventions is poorly understood because of a lack of high-quality impact evaluations. Randomized control trials (RCTs), in which experimental units are randomly allocated to treatment or control groups, offer an intuitive way to calculate the impact of an intervention by establishing a reliable counterfactual scenario. As many conservation interventions depend on changing people's behaviour, conservation impact evaluation can learn a great deal from RCTs in fields such as development economics, where RCTs have become widely used but are controversial. We build on relevant literature from other fields to discuss how RCTs, despite their potential, are just one of a number of ways to evaluate impact, are not feasible in all circumstances, and how factors such as spillover between units and behavioural effects must be considered in their design. We offer guidance and a set of criteria for deciding when RCTs may be an appropriate approach for evaluating conservation interventions, and factors to consider to ensure an RCT is of high quality. We illustrate this with examples from one of the few concluded RCTs of a large-scale conservation intervention: an incentive-based conservation programme in the Bolivian Andes. We argue that conservation should aim to avoid a rerun of the polarized debate surrounding the use of RCTs in other fields. Randomized control trials will not be feasible or appropriate in many circumstances, but if used carefully they can be useful and could become a more widely used tool for the evaluation of conservation impact.

Forest conservation incentives are a popular approach to combatting tropical deforestation. Here we consider a case where direct economic incentives for forest conservation were offered to newly titled smallholders in a buffer zone of a protected area in the northeastern Ecuadorian Amazon. We used quasi-experimental impact evaluation methods to estimate changes in forest cover for 63 smallholders enrolled in Ecuador's Socio Bosque program compared to similar households that did not enroll. Focus group interviews in 15 communities provided insight into why landowners enrolled in the program and how land use is changing. The conservation incentives program reduced average annual deforestation by 0.4–0.5% between 2011 and 2013 for those enrolled, representing as much as a 70% reduction in deforestation attributable to Socio Bosque. Focus group interviews suggested that some landowners chose to ‘invest’ in conservation because the agricultural capacity of their land was limited and economic incentives provided an alternative livelihood strategy. Interviews, however, indicated limits to increasing enrollment rates under current conditions, due to lack of trust and liquidity constraints. Overall, a hybrid public–private governance approach can lead to larger conservation outcomes than restrictions alone.

Nearly 10% of the world's total forest area is formally owned by communities and indigenous groups, yet knowledge of the effects of decentralized forest management approaches on conservation (and livelihood) impacts remains elusive. In this paper, the conservation impact of decentralized forest management on two forests in Tanzania was evaluated using a mixed method approach. Current forest condition, forest increment and forest use patterns were assessed through forest inventories, and changes in forest disturbance levels before and after the implementation of decentralized forest management were assessed on the basis of analyses of Landsat images. This biophysical evidence was then linked to changes in actual management practices, assessed through records, interviews and participatory observations, to provide a measure of the conservation impact of the policy change. Both forests in the study were found to be in good condition, and extraction was lower than overall forest increment. Divergent changes in forest disturbance levels were in evidence following the implementation of decentralized forest management. The evidence from records, interviews and participatory observations indicated that decentralized management had led to increased control of forest use and the observed divergence in forest disturbance levels appeared to be linked to differences in the way that village-level forest managers prioritized conservation objectives and forest-based livelihood strategies. The study illustrates that a mixed methods approach comprises a valid and promising way to evaluate impacts of conservation policies, even in the absence of control sites. By carefully linking policy outcomes to policy outputs, such an approach not only identifies whether such policies work as intended, but also potential mechanisms.

This article evaluates the impact of the Chile Solidario anti-poverty programme. The evaluation is based on propensity score matching and a difference-in-difference estimator along with databases of Social Assistance Committee forms. The results show a positive but small impact on employment and housing along with a slightly negative impact on self-generated income. They also suggest that gains tend to be concentrated in the first phase, during which beneficiaries work with a family support professional, and that these benefits may not be sustainable. Participant families show absolute gains in income and employment, but these may be attributed to environmental conditions rather than the programme; this raises doubts about the premise that these families were initially marginalised from the economy and social networks.