Mind the adoption gap: Findings from a field experiment designed to scale up the availability of fodder shrub seedlings in Malawi

Abstract

While dairy production has the potential to diversify smallholder agriculture and increase incomes, there are multiple constraints. One is the consistent provision of quality feed. High protein, leguminous fodder shrubs – also referred to as Fodder Tree Technology (FTT) – can help address this constraint, yet adoption levels are generally low. Implemented in Kenya and Malawi, the Shrubs for Change (S4C) project is employing several approaches to address this situation, including those informed by behavioral science. Given that approximately 500 shrubs per cow are needed to generate enough leaf matter to bolster milk production, promoting FTT at scale necessitates the production, distribution, and successful planting of large numbers of shrub seedlings. We implemented a field experiment in Malawi’s Southern Region in late 2021 to test the effectiveness of a social learning intervention intended to motivate dairy farmers to significantly scale up the production of FTT seedlings. This intervention involved meeting with dairy farmers in 39 randomly selected milk production zones to review the numbers of seedlings being produced vis-à-vis local demand, coupled with the development of action plans to address identified production gaps. While we find that this intervention increased the setting up of private nurseries by 10% (p < 0.05), it only increased overall seedling production by an average of 20 additional seedlings per dairy farmer (p > 0.1). We offer several explanations for this lower than expected and statistically insignificant result, which point to the need for iterative rounds of engagement with farmers when supporting them to take up FTT and other complex agronomic and sustainable land management innovations.

Introduction

Dairy production has the potential to diversify smallholder agriculture and bolster income (Banda et al., 2021; Chagunda et al., 2016). This is especially in contexts where land holdings are too small to make crop production alone economically viable (Harris et al., 2021). Yet, there are multiple issues that affect the realization of this potential. One is striking the right balance between high genetic potential for milk production on the one hand and adaptability to local conditions on the other (Baur et al., 2017). Another is market access, where milk can be sold consistently and at fair prices (Chindime et al., 2017). Given its effect on milk yields and quality, appropriate and consistent feeding is an area of concern in small-scale dairy production, where producers typically do not operate on an economies of scale large enough to make commercial feed economically viable (Franzel et al., 2014). Moreover, while quality grasses are accessible during much of the year, they are deficient in protein. And matters can be far worse in the dry season, when they are largely inaccessible, forcing small-scale producers to rely on suboptimal feed, such as maize stover (Chakeredza et al., 2007; Kawonga et al., 2012).

Since the Government’s first effort to commercialize the smallholder dairy sector in 1969, efforts have been undertaken in Malawi to address the above issues. This started early on with the organization of dairy farmers into Milk Bulking Groups (MGBs). MBG members deliver their milk to central collection points daily, where it is quality checked, cooled, and collected by milk processing companies (Revoredo-Giha and Renwick, 2016). According to the Malawi Milk Producers Association (MMPA), there were (at the time of writing) 121 MBGs operating in the country, representing approximately 17 000 members (MMPA, 2021). Approximately 80% of Malawi’s milk is produced in the Southern Region (Revoredo-Giha and Renwick, 2016), where the Shire Highlands Milk Producers Association (SHMPA) – the Southern Region’s MMPA affiliate – supports 23 MBGs.

Over the years, there has also been considerable effort – by SHMPA and various Non-governmental organization (NGO) led programs – to improve the genetic potential of smallholder dairy cows. This has primarily been through crossing local Zebu breeds with exotic breeds (e.g., Holstein Friesian). Pass-on programmes are also popular. Here, farmers are provided with female calves on the condition that their first offspring are passed on to another farmer. These programmes typically target women, and the cows associated with them are referred to locally as mkakaz cows. As a result of these genetic improvement interventions, many smallholder dairy cows in Malawi can potentially produce 20–30 liters of milk per day under ideal conditions.

One of these conditions is appropriate feeding, which can dramatically influence milk yields (Bateki et al., 2020). Historically, SHMPA has worked to source maize bran and brewer’s spent grains for the MBGs it supports for onward distribution to their members. World Agroforestry (also known as ICRAF), under the Agroforestry for Food Security Programme (AFSP), furthermore promoted leguminous, high-protein fodder shrubs (also referred to as fodder tree technology [FTT]) among several hundred smallholder dairy producers in various parts of Malawi as well (Hughes et al., 2019).

The appropriate use of FTT can significantly increase milk yields without commensurately increasing production costs, as compared to the use of commercial feed (Franzel et al., 2014). Moreover, the resulting increases in milk yields can, particularly if combined with complementary nutrition educational messaging, result in increased domestic milk consumption, thereby improving nutrition outcomes. Moreover, there are potential sustainable land management benefits when FTT is properly integrated within cropping fields, e.g., erosion control and soil fertility enhancement. Yet, efforts to promote the broad uptake of FTT among small-scale dairy farmers has had varying success (Kiptot et al., 2015; Toth et al., 2017).

The Shrubs for Change (S4C) project (2020 to 2022) is a joint effort by ICRAF and SHMPA to build on the work of AFSP and efforts in other countries to scale up the production and utilization of FTT. Also being implemented in Kenya and funded by Fund International Agricultural Research (FIA) of the German Federal Ministry for Economic Cooperation and Development, S4C is seeking to support 7,500 smallholder dairy farmers take up FTT in Malawi’s Southern Region. Key interventions include training, feeding demonstrations, seed provision, nursery establishment, and technical backstopping. In addition, S4C seeks to integrate insights from behavioral science to enable smallholder dairy producers to overcome the FTT adoption hurdle. For example, to avoid information overload, training is delivered in short sessions, timed to coincide with key phases of the FTT production and utilization cycle. Simple and visual ‘learning posters’ form the basis of this training, which are kept by the MBGs to serve as reminders for their members.

The initial plan in Malawi was to sell fodder seedlings to the participating dairy producers at subsidized prices, with an option for making payments via deductions from milk sales. Mental accounting theory (Thaler, 1999) and time-discounting theory (Frederick et al., 2002) suggest that this decoupling of the collection and payment for the FTT seedlings/seeds, combined with the repayment time lag, should significantly facilitate initial uptake. An attempt was made in the 2020 planting season to combine the above with the issuing of coupons for bulk purchase discounts to randomly selected producers. Given that appropriate FTT use requires the presence of about 400–500 shrubs on farm in order to produce the requisite leaf matter (Franzel et al., 2014), coupon holders were to be offered increasingly higher per shrub discounts depending on the number collected. However, this experimental setup required the production of significant numbers of shrub seedlings, plans for which were frustrated by travel restrictions associated with the COVID-19 pandemic.

SHMPA, nevertheless, supported the MBGs to set up and manage group nurseries. However, the number of seedlings produced was inadequate, and attempts were made to make up for the shortfall through the provision of seeds for direct seeding. Unfortunately, this was also frustrated in some locations due to variable rainfall patterns.

Experiences in the 2020/21 farming season led to a realization that the lack of appropriate planting material is a substantive barrier affecting FTT adoption in S4C’s catchment area in Malawi. The project team therefore decided to decentralize seedling production to a unit below the MBG, the MBG zone. As indicated in Supplementary Figure S1, each of the 20 MBGs participating in S4C comprises between two and 10 zones. Relevant training and seed support were therefore provided to dairy producers residing in each zone. FTT varieties promoted included Gliricidia sepium, Acacia angustissima, Leucaena leucocephala, Calliandra calothrysus, and Leucaena pallida. However, monitoring revealed that, again, the number of seedlings being produced in most zones was far below what was needed to enable appropriate FTT adoption for most of the dairy farmers.

This paper documents a field experiment of an intervention that was designed to overcome this newly prioritized constraint. This intervention is described in the next section, followed by our causal identification strategy and data collection exercise. The results of our field experiment are then presented. In the discussion section, we highlight what the results mean going forward for the project and the scaling of complex agronomic and sustainable land management innovations in general.

Materials and Methods

Intervention

As mentioned above, the project team recognized that most group nurseries set up within the MBG zones would fail to generate enough fodder seedlings to enable the participating dairy farmers to each establish the target of 400–500 shrubs per cow on their farms. This is certainly not unusual for group nurseries. While there are benefits, e.g., skills training and peer learning, per capita productivity is typically lower as compared with individually managed nurseries (Böhringer et al., 2003). The essence of the intervention was to facilitate a process with each MBG zonal committee to critically reflect on the numbers of seedlings being produced vis-à-vis demand among dairy farmers within their zones.

The facilitated exercise was carried out separately in each zone by a team of four trained facilitators from October 8–14, 2021. Following introductions, the exercise started by supporting the group to develop a fodder shrub vision statement for their zone in the local language, Chichewa. The English template for this vision statement is as follows:

By 2023, XX dairy producers in XX Zone, XX MBG will have 400–500 shrubs per cow established and well managed on their farms, feeding harvested leaf matter mixed with other feed to their cows daily, leading to higher milk yields and more income.

A participatory gap analysis was then facilitated with the participants. This included a review of the following:

1. a. Number of dairy producers in the zone who have enough shrubs already on their farms.

2. b. Number of dairy producers who would benefit from establishing shrubs on their farms to realise the above vision.

3. c. Number of farmers who have so far expressed strong interest in the zone in planting fodder shrubs on their farms this year.

4. d. Number of farmers in the zone that have or are gearing up to establish their own fodder nurseries and how many seedlings each is producing.

5. e. For zonal group nurseries, the number of fodder seedlings they expect to produce this year and, in turn, how many dairy producers in the zone are expected to get seedlings from this nursery and how many seedlings each is expected to receive.

6. f. Considering responses to d and e, the number of dairy producers in total who would be able to receive 400–500 seedlings per cow for this year based on current seedling production plans underway (d plus e).

7. g. The gap there is likely to be between the number of dairy farmers who have expressed interest in planting fodder shrubs this year (c minus f).

8. h. The gap there is likely to be between the number of dairy farmers who should ideally be planting fodder shrubs in order to realize the above vision (b minus f).

Of the 39 zonal groups where the intervention was implemented, 31 identified a significant gap in the number of seedlings that are likely to be produced in their zones for the 2021/22 planting season vis-à-vis demand. The gaps identified ranged from 1,000 to 100 000 seedlings. These groups were then supported to develop action plans to address their respective seedling production gaps. This included requests for ICRAF and SHMPA for more seed and potting sleeves, based on the numbers of additional seedlings required. Given its availability, additional Gliricidia sepium seed, as well as sleeves, were provided to the 31 zonal groups during the period of October 29 to November 4, 2021.

Causal identification and data analysis strategy

The above intervention was implemented in just under half of all the MBG zones that fall under the 20 MBGs participating in S4C. These zones were randomly selected from the universe of all MBG zones, stratified by MBG. The sample proportion was set at 49% to ensure that less than half of the zones associated with MBGs with odds numbers of zones would be selected. This resulted in the random selection of 42 MBG zones. These were targeted for the above intervention, which was successfully implemented in all but three zones. In these zones, group nurseries had not been established, so it was not possible to implement the intervention as planned. Two of these zones belong to one MBG. Given that randomization was stratified by MBG, we removed this MBG from the study. This was not the case for the third zone, so we left it in our data set.

In our analysis, we compare the zones in relation to the outcome measures described below, as per their original unit of assignment, i.e., intention to treat (ITT) analysis (Athey and Imbens, 2017). We do so with ordinary least squares (OLS) regression, with robust standard errors and MBG dummy variables given our stratified randomization procedure. Our main model is presented in equation 1, where the effect of the intervention, ${X_i}$ , on zonal seedling production, ${Y_i}$ , is estimated holding the effects of each MBG dummy variable, $D{n_i}$ , constant.

(1) $${Y_i} = \;{\beta _0} + \;{\beta _1}{X_i} + \;{\gamma _2}D{2_i} + \;{\gamma _3}D{3_i} + \ldots + \;{\gamma _n}D{n_i} + \;{e_i}$$

As a robustness check, we complement OLS with robust regression. As can be expected with seedling counts, considerable variability and extreme values are present in our data set, despite our winsoring these to the 99^th percentile. We therefore find this an appropriate complement to our OLS models. Robust regression assigns less weight to influential observations, thereby generating estimates that are more applicable to the bulk of the distribution (Verardi and Croux, 2009).

For all models, our unit of analysis is the MBG zone, rather than individual dairy farmers. While it is true that some dairy farmers set up their own individual nurseries, most participated in group-level nurseries, where the seedlings were to be shared prior to their planting on individual farms. Indeed, the primary aim of the intervention was to boost the production of fodder seedlings in each zone, thereby further justifying our analysis at this level. We acknowledge that this has implications for statistical power, given the implicit smaller sample size. However, the intervention was expected to scale up the production of seedlings considerably (i.e., by at least 100 additional seedlings per farmer), and our study is adequately statistically powered to detect these expected effects, particularly for the overall sample.

Given that the intervention, as explained above, was not implemented in one of the zone’s assigned for the intervention – coupled by the fact that eight other zones where the intervention was implemented did not identify seedling production gaps and, therefore, can be considered as not fully treated – we complement our ITT analysis with two-stage least squares (2SLS) regression (Imbens, 2010). This enables us to generate a type of local average treatment effect (LATE), which is an estimate of the effect of the intervention on only those zones that were fully treated. This procedure scales up the ITT effect estimates proportional to those assigned to be treated but – for reasons explained above – were not fully treated.

Data collection

The principle aim of the intervention was to motivate the zonal dairy farmer groups to produce more shrub seedlings. This was to take place by expanding existing zonal group nurseries, setting up new ones, or encouraging farmers to establish their own nurseries. Consequently, our study sought to evaluate the extent to which more seedlings were produced in the treated zones. This, therefore, required the counting of successfully germinated seedlings. Given the large numbers involved and the fact that some dairy farmers set up their own nurseries, this was not straightforward.

From December 1 to 11, 2021, a team of five enumerators and one supervisor visited both treated and untreated MBG zones to carry out the seedling germination counting exercise. To prepare for this, the enumerators participated in a one-day practical training session. They were further aided by one-by-one meter measuring frames and a mobile data capture tool, developed using the Open Data Kit (ODK) platform. The enumerators were each assigned to separate zones. There they met with the MBG’s zonal committee and captured information on the number of dairy farmers in the zone and whether any had established their own individual nurseries. They visited all the group fodder nurseries in the zones, where they took photographs and captured geocoordinates.

The enumerators began the counting exercise by first recording whether potted or bare-rooted seedlings were being produced in the nursery. If both were presented, the seedlings associated with each were counted separately. If there appeared to be less than 2,000 seedlings, the enumerators performed direct counts of those that had successfully germinated. If there appeared to be more than 2,000 germinated seedlings, the enumerators implemented the following systematic random sampling protocol. This first involved measuring and recording the length and width of each nursery, thereby enabling the computation of the area of all beds combined. Using their measuring frames, they counted successfully germinated seedlings in randomly selected one-by-one meter sections of the beds. For beds with homogenous germination, three one-by-one meter sampled counts were taken. Six sampled counts were taken for beds with heterogenous germination.

The data capture tool was programmed to automatically compute a sampling interval and a random starting point based on the combined length of the beds and the sample of counts to be taken. The enumerators were directed to a starting point on the bed (selected at random between one meter and the sampling interval). At this first sampled section, they placed their measurement frames one the bed, took a photo, and counted the number of successfully germinated seedlings in the frame (see Supplementary Figure S2 for an example). The auto generated sampling interval then directed the enumerators to proceed a given number of meters forward where they placed their measurement frame to undertake and record their next count. These one-by-one meter counts were then automatically averaged and multiplied by the area of all beds combined to generate an estimate of the number of geminated bare-rooted or potted seedlings in the nursery. These estimates were displayed in the data capture tool to the enumerators to check if they seemed plausible based on their visual observations of the beds.

The enumerators also counted successfully germinated seedlings produced dairy farmers’ individual if present in the zone. When there were six or fewer individual nurseries, all were visited and subjected to the counting exercise. In addition to counting, this also involved the taking of pictures and the capturing of geocoordinates. Where there were more than six dairy farmers who had established individual nurseries in a zone, their names were inputted into the data capture tool. Then up to eight names were automatically selected at random, and the enumerators were directed to visit and perform the counts for the first six individual nurseries, only relying on the latter two when replacement farmers were required.

A backchecking procedure was also implemented. From the second day of data collection onward, two zones from the previous day’s work were purposively selected for backchecking. All five of the enumerators were involved in backchecking at some point, but not in relation to the zones they had personally enumerated. Given that the back-checker was often the last person to be dropped off during the day’s data collection, s/he was often accompanied by the ICRAF supervisor. The backchecking tool was updated daily. The geocoordinates captured during the previous day directed the back-checker to the exact location of the zone in question’s main nursery. S/he then implemented the same procedure described above to count the number of successfully germinated seedlings. If present, they were also directed to one purposively selected individual nursery to do the same. The backchecking results were juxtaposed with those collected in the main germination counting exercise. In general, the differences were significantly less than 5%. However, one error was identified where an enumerator recorded the length of a bed when they should have recorded its width, thereby inflating the area of the entire nursery and the associated estimate. In several other cases, the differences were identified as a result of germination that had taken place in the short timeframe between the initial and back-check visits. This was directly witnessed by the ICRAF supervisor, and it is one of our study’s limitations elaborated below. The backchecking was complemented by daily reviews of the previous day’s collected data, photos, and geocodes. Several additional data entry errors were identified and corrected accordingly.

Outcome measures

From the above, we added up all the germinated seedling counts from each zone’s potted and bare-rooted nurseries. The case was different for individual nurseries. Here, we computed the average number of the counts obtained for the sample of farmers visited. We then multiplied this average figure by the total number of dairy farmers with individual nurseries present in the zone. Where the number of dairy farmers was six or less (and therefore sampling was not implemented), this is equivalent to adding up all the counts of individual nurseries together. In the 14 zones where the number was greater than six, there is inevitably some sampling error in the estimates, given the small sample size. That said, there are only four zones where over 10 farmers with individual nurseries were recorded, so the potential measurement error implications are limited.

From the above, one possible candidate for our primary outcome variable could have been the estimated number of successfully germinated seedlings in each zone. However, interpretation would be challenging; the extent to which the resulting number is enough for a given zone depends significantly on how many dairy farmers there are in that zone. For example, 2,000 seedlings divided among 40 farmers is very different than this same number divided among four farmers. We therefore considered the number of fodder seedlings potentially available per dairy farmer in the zone to be more appropriate. We also assumed that not all the germinated seedlings would necessarily mature into plantable seedlings. We were therefore advised by ICRAF agroforestry scientists to factor in a 30% attrition rate. Our primary outcome variable is therefore an estimate of the average number of plantable seedling expected to be available per dairy farmer residing in each zone, considering this attrition rate. We complement this with a measure of the percentage of farmers that could potentially have 400 seedlings available for planting, assuming the seedlings are not shared evenly. This is a useful metric to gauge how far each zone is away from a target of 400 shrubs per dairy farmer.

A key mechanism expected to lead to an increased number of shrub seedlings in the treated zones was through farmers setting up their own individual nurseries. We therefore also examine the treatment groups in relation to the number of farmers who set up individual nurseries, as well as the percentage of farmers with such nurseries. Of interest also are the numbers of germinated seedlings disaggregated by nursery type (bare rooted and potted) and those produced in individual nurseries.

Results

Covariate balance

While not without shortcomings, a common procedure in both experimental and quasi-experimental studies is to compare treatment groups in relation to several nontreated-related exogeneous variables, i.e., covariates (Bruhn and McKenzie, 2009). This is done to assess the extent to which they are balanced, thereby providing confidence that the randomization (or the alternative nonexperimental balancing) procedure ‘worked’. Our data set, admittingly, does not comprise many of such variables. Nevertheless, we compare the two groups in relation to three, i.e., number of dairy farmers in the zone, proportion of these who are female, and the altitude of the zone’s main group nursery (Table 1). We find no statistically significant differences between the two groups. However, there are 2.5 fewer dairy farmers, on average, residing in the treated zones, after controlling for MBG fixed effects. Fortunately, the nature of our primary outcome variable – number of estimated plantable seedlings available per dairy farmer – controls for this difference automatically.

Table 1. Covariate comparison of treated and untreated MBG zones

Standard errors in parentheses.

*p < 0.1; **p < 0.05; ***p < 0.01.

MBG level fixed effects used in all regressions (strata used in random assignment).

Individual nursery establishment

As explained above, it is of interest to examine the extent to which the intervention motivated individual dairy farmers to establish their own nurseries. We find some evidence that it did, albeit not to the extent expected (Table 2). Half of the zones assigned to the intervention were found to have individual nurseries, as compared to approximately one-third for the zones assigned to the control group (Figure 1). Our ITT and LATE treatment effect estimates are 18% and 26%, respectively, which are both statistically significant at the 10% level. The percentage of dairy farmers found with individual nurseries is much lower, at 16% and 5% for two treatment groups, respectively. The ITT estimate is 10% and the LATE estimate is 14%. Both estimates are statistically significant at the 5% level.

Table 2. Individual fodder nursery treatment group comparison

Standard errors in parentheses; ITT = Intention to Treat; OLS = Ordinary Least Squares Regression.

Dif. = Difference; LATE = Local Average Treatment Effect; 2SLS  = Two-Stage Least Squares Regression.

MBG level fixed effects used in all regressions (strata used in random assignment).

*p < 0.1; **p < 0.05; ***p < 0.01.

Figure 1. Percentage of MBGs with individual nurseries (with 95% confidence intervals).

Number of plantable seedlings expected and seedlings germinated

The results for our primary outcome variable – estimated number of seedlings available per dairy farmer – are in a similar (and expected) direction, as is the case for individual nursery establishment. The density plots presented in Figure 2 visually illustrate that there is a difference between the two distributions. However, the effect sizes are modest and statistically insignificant (Table 3) and save for the robust regression estimates. We find that the intervention appears to have encouraged farmers assigned to the treated zones to establish 20 seedlings on average more per farmer. For the zones that were fully treated, this estimate increases to 29 seedlings. The robust regression estimate is statistically significant at the 10% level, but lower at 15 seedlings per dairy farmer. These estimated effects represent only a small fraction of the target of 400–500 seedlings per dairy farmer per cow. We therefore remain confident that our study was adequately statistically powered, as we expected at least 100 additional seedlings to be made available per farmer in the treated zones.

Figure 2. Density plots for estimated number of plantable seedlings available per farmer.

Table 3. Main outcome variable treatment group comparison

Standard errors in parentheses; ITT = Intention to Treat; OLS = Ordinary Least Squares Regression.

LATE = Local Average Treatment Effect; 2SLS = Two-Stage Least Squares Regression.

MBG level fixed effects used in all regressions (strata used in random assignment).

*p < 0.1; **p < 0.05; ***p < 0.01.

While progress is less than desired, our analysis of our complementary indicator – % of dairy farmers for which 400 plantable seedlings could be made available – reveals that S4C has made some progress. That is, if the seedlings were pooled together and directed to a subset of the dairy farmers in the zones, 19% and 14% would be in a position, on average, to plant 400 seedlings in the zones assigned to the treated and untreated groups, respectively.

Comparing the two treatment groups in relation to other complementary indicators – germinated seedlings counted in total; germinated group nursery seedlings (potted and bare rooted); and germinated seedlings in individual nurseries – illustrate that the intervention appears to have had a consistent and positive, albeit modest and largely statistically insignificant, effect on both group and individual seedling production.

These average figures, however, mask the large variation in seedling production across S4C’s catchment area (Figure 3). One clear way our study is statistically underpowered is in its ability to test for the intervention’s possible differential effects, e.g., by district. There appears a likely difference in the intervention’s performance in Mulanje District – where our effect estimate is 56 more seedlings available per farmer – as compared to Thyolo District, where our effect estimate is in a slightly negative direction at seven fewer trees available per farmer. Another noteworthy potential differential effect is between zones with female only membership and those with mixed female and male membership. Our ITT and LATE effect estimate are, respectively, 39 and 63 additional seedlings for zones with mixed groups. For zones with women-only members, the average effect estimates are, again, slightly negative at seven and 11 fewer seedlings available per farmer. However, the results of these statistical interaction tests are statistically insignificant, given statistical power limitations.

Figure 3. Estimated number of plantable fodder seedlings available per dairy farmer by MBG zone location and treatment status.

Discussion

The complexity of challenges associated with promoting the successful adoption of agroforestry and other complex agronomic and natural resource management practices is well documented (Andersson and D’Souza, 2014; Meijer et al., 2015; Stevenson et al., 2019). There are multiple barriers that must be overcome to enable farmers to overcome the adoption hurdle. S4C explicitly recognizes this challenge with respect to promoting the successful uptake and use of FTT. It is therefore using a variety of approaches – many of which are informed by insights from behavioral science – to overcome it.

One key challenge that was foreseen but became more prominent following the first year of S4C’s implementation in Malawi was the low availability of FTT planting material. This is a common challenge in agroforestry. However, it is particularly formidable in the case of FTT, given the large number of shrubs needed on farm to generate the requisite leaf matter (400–500 per cow). While efforts were made in S4C’s second year to increase the availability of FTT seedlings by decentralizing their production to the MBG zone level, it was clear from field monitoring that this would not generate the numbers needed in many of the zones. Hence, we designed and implemented a participatory exercise to support dairy farmers at the zonal level to review the number of seedlings they were producing vis-à-vis demand. Of the 39 randomly selected zones where the intervention was implemented, dairy farmers in 31 of these identified significant production gaps and developed action plans to address these. This included making requests to ICRAF and SHMPA for additional FTT seed and planting sleeves, which were distributed between two to three weeks following the intervention’s implementation.

We did find that the intervention induced some farmers to establish individual nurseries, but not in the numbers expected or desired. We further did not find that it bolstered FTT seedling production significantly. Our estimated ITT and LATE estimates for our primary outcome variable—the estimated number of plantable FTT seedlings potentially available per dairy farmer—are modest, at 20 and 29 additional seedlings per farmer, respectively. These treatment effect estimates are also statistically insignificant, save for our robust regression coefficients that are significant at the 10% level. While there appears to be considerable variation in the effectiveness of the intervention in inducing seedling production across districts and zones with female only and mixed membership, our study was statistically underpowered to detect such differences. Nevertheless, the magnitude of these differential subgroup effect estimates is below 100 additional seedlings per farmer, and therefore would unlikely to make a significant contribution to the 400–500 seedling per cow target if found to be statistically significant.

Our study and, by extension, our conclusions face several limitations. First, and with the benefit of hindsight, we carried out the data collection exercise several weeks too early. We were under pressure to undertake the counting exercise before the seedlings were distributed and out-planted in farmers’ fields. Malawi’s 2021/2022 rainy season was forecasted to commence in late November 2021. While there were some early showers, the rainy season came late to the S4C’s catchment area, only settling in during the latter weeks of December. As noted above, seedlings were still germinating during the counting exercise, particularly among those zonal groups and individual dairy farmers who planted the distributed seed late. Consequently, our counting exercise may have underestimated the number of seedlings available for planting, particularly in the treated zones that received the additional planting material.

This underestimation may be magnified by the second limitation of our study. In cases where a) no individual nurseries were reported in the treated zones and b) additional planting material was distributed, committee members were asked why the former was the case. Most reported that the farmers were keeping the seed and waiting for the rains, given a low availability of water. While establishing nurseries at the onset of the rains is considered late, it is possible that these farmers may have planned to directly sow the seeds in their fields. As reported above, direct sowing was a strategy attempted during the 2020/21 season, which met with varying success, partly because of late planting and partly because of erratic rainfall patterns in some areas of S4C’s catchment area.

Assuming the zones assigned to receive and not receive the intervention are supported similarly during the 2021/22 rainy season, carrying out a follow-up survey to count the numbers of seedlings successfully established in representative samples of farmers’ fields would be useful. This would both help verify a) the extent of the abovementioned underestimation and b) the extent to which estimated numbers of plantable seedlings are correlated with the numbers of shrubs surviving in farmers’ fields.

The results of our field experiment reinforce the view that promoting the successful adoption of complex agronomic and sustainable land management innovations can be challenging. Indeed, many issues were observed during our germination counting exercise that testify to this. The office of one MBG, for example, had been robbed, and the package of FTT seeds being kept in storage was one of the items that were stolen. Another zonal group had successfully established a large nursery of bare-rooted seedlings, but a heavy down poor of early rain washed the seeds into the walkways between the beds. Another group assumed that the early rains were adequate for irrigating their group nursery, only for the enumerator to find that most of the germinated seedlings had dried up. Finally, many of the individual nurseries that were visited composed of too few seedlings or suffered low rates of germination, assumingly because of inconsistent watering. A key lesson – likely applicable to other similarly complex agronomic and sustainable land management innovations – is that multiple rounds of support and iterative engagement with farmers are likely needed in order to bring adoption levels to a point where the expected impacts can take place and at a desired scale. This has cost implications, but there appear to be no quick fixes or easy answers.