What works and why? Evaluation of a community nutrition programme in Kenya

Abstract

Objective

To investigate the nutritional impact of a community-based programme that focused on social cohesion and action.

Design

The change in nutritional status of children aged 12–60 months was examined over a period of 3 years in Makueni District in Eastern Province of Kenya in six communities in which an intervention programme of Participatory Learning and Action was introduced and in ten communities in which only basic preparations were made but no intervention was started.

Setting

The intervention was part of the Government of Kenya Community Based Nutrition Programme and was supported by the Government of Denmark.

Subjects

Children aged 12–60 months.

Results

Among communities without intervention there were similar levels of underweight (mean Z-score: −1·63 v. −1·50 (NS); % with Z-score < –2: 36·6 % v. 34·5 % (NS)) and stunting (mean Z-score: −2·0 v. −1·99 (NS); % with Z-score < –2: 44·3 % and 47·4 % (NS)) at baseline and after 3 years. By contrast, among communities who had received interventions, there were significant improvements after 3 years in the levels of underweight (mean Z-score: −1·66 v. −1·37 (P < 0·02); % with Z-score < –2: 42·9 % v. 31·4 % (P < 0·035)) and stunting (mean Z-score: −2·05 v. −1·59 (P < 0·05); % with Z-score < –2: 52·7 % v. 39·7 % (P < 0·02)).

Conclusions

The results indicate considerable potential for using Participatory Learning and Action as a community-based approach to effectively address child undernutrition. It is suggested that these interventions are developed, implemented and evaluated more widely as a mean of tackling childhood undernutrition and improving child survival and development.

The 2008 Lancet series on maternal and child undernutrition highlighted the threat posed by the current high levels of child undernutrition in many developing countries to reaching the Millennium Development Goals by 2015. It also provided a wake-up call to national governments and development agencies in countries with a high burden of child undernutrition by showing that adequate nutrition in early life is a prerequisite for human capital formation and economic development. Despite this strong call to action, and an increasing political momentum to tackle the problem, the global nutrition crisis is still increasing in gravity and scope as countries suffer multiple structural assaults from food, fuel and financial crises and climate change. Simultaneously, as Brinkman et al. ⁽¹⁾ suggest, energy consumption has declined between 2006 and 2010 in nearly all developing regions, with young children being among those most affected. In the wake of the Lancet series a technical consensus has been reached on a package of single-shot, targeted interventions that ‘work’⁽²⁾. The attention of politicians and health planners is now focused on how to scale up this package. The recent inter-agency working group, Scaling Up Nutrition, has emphasised thirteen specific interventions for the prevention and/or treatment of undernutrition. Several authors have pointed out that, on their own, these interventions do not address the drivers of child undernutrition, which include poverty and social injustice, and should not preclude investing in interventions that can offer sustained nutritional gains over time by addressing their social root causes^{(3, 4)}. The importance of the social causes, including social capital and cohesion, were highlighted in the report from the WHO Commission on the Social Determinants of Health, which concluded that, ‘social injustice is killing adults and children on a grand scale’⁽⁵⁾. The report recommended that health, nutrition and social sector planners should tackle the drivers and underlying causes of poor health and nutrition, which are the inequitable distribution of power and human and financial resources. The report also identified social cohesion as key to building a better living environment⁽⁵⁾. Social cohesion refers to the trust, mutual understanding, and shared values and behaviours that bind members of human networks and communities together. Social cohesion is a sub-set of the broader concept of social capital which encompasses the active connections that can lead to empowerment through linking communities and networks with different levels of power in society⁽⁶⁾. Social cohesion is increasingly also being defined in relation to horizontal and vertical bonds between groups at different levels in society. Most interestingly, literature indicates that social capital and cohesion are increasingly perceived as important for better health⁽⁷⁾. Horizontal social cohesion, such as interaction between individuals with relatively equal power, may change to diagonal and vertical cohesion through a scaling up of a participatory process including individuals and groups from different levels of society, and can catalyse a process of change at the local level as well as wider institutions and systems change^{(4, 8)}.

Given the need to find effective ways to change the social determinants of poor child nutrition, there are very few data on the effectiveness of social interventions to inform programming. Some studies have suggested that the use of participatory and ‘bottom-up’ processes, especially when they are scaled up effectively, can lead to sustainable improvements in child health and undernutrition⁽⁹⁾. However, there are methodological challenges in evaluating the impact of complex, indirect interventions on child nutritional status, and a review by Pridmore and Carr-Hill⁽¹⁰⁾ concluded that the evidence base is ambiguous. Many of the studies reviewed were methodologically flawed and in all cases the findings were highly contextual.

As there is such a dearth of information on the nutritional impact of community-based programmes which focus on social cohesion and action rather than provision of food or nutritional supplements, the opportunity was taken to analyse data from communities in Kenya who had been supported by a government-run programme, namely the Community Based Nutrition Programme (CBNP) of the Department of Social Services, over a 3-year period (1999–2002). The main intervention of the CBNP programme was the Participatory Learning and Action (PLA) process. These data were compared with similar communities who had not received this particular intervention.

Method

The study used a mixed-methods approach to assess the effect of a five-stepped social educational process, known as PLA⁽¹¹⁾, on social cohesion and capital as well as on child nutrition in communities included in the CBNP. Quantitative assessments of the nutritional status of children in the intervention v. the non-intervention communities were supplemented by qualitative inquiries of the process by which change in child nutritional status had been achieved.

The intervention

The intervention involved facilitating the participation of the communities included in the CBNP in the five-stepped social educational PLA process with the aim of building social cohesion, inter-sectoral collaboration and community action through a cyclical process of critical reflection. This in turn was expected to strengthen demand for community access to (government) services that would improve child nutritional status. The PLA intervention was implemented through the following steps (see Fig. 1). The community data collection, analysis and problem solution all had a focus on identifying ways to address this high level of malnutrition. Although Fig. 1 shows the specific community-driven interventions emerging from the PLA process, the main difference between intervention and non-intervention communities was the PLA process.

Fig. 1 Illustration of the PLA process in the intervention communities (PLA, Participatory Learning and Action; CBNP, Community Based Nutrition Programme)

Step 1: Increasing awareness of the problem and building support for the intervention

A baseline survey was carried out to determine the nature and extent of ill-health and child undernutrition in the intervention and non-intervention communities. Social marketing, using Participatory Educational Theatre⁽¹²⁾ and the Child-to-Child approach⁽¹³⁾, was used to disseminate the findings of this survey to all stakeholders, from central to local level, in order to raise awareness and promote buy-in and ownership of the findings. This process initiated and sustained a dialogue between and within the different groups of stakeholders, which led to commitment for action from formal and non-formal leaders at all levels of the government and from the intervention communities themselves.

Step 2: Establishing multi-sectoral teams at the district and community levels

Each team had representatives from relevant line ministries responsible for service delivery and from relevant non-governmental organisations (NGO) and other key stakeholders such as faith-based organisations and non-formal leaders.

Step 3: Training multi-sectoral teams in PLA facilitation technique

Each team co-opted local community members and divisional officers from relevant line ministries (e.g. health, agriculture and water services) who would be trained by experienced trainers in facilitating PLA processes in the communities.

Step 4: Intervention involving data collection, community reflection, analysis and implementation

Qualitative data were gathered and analysed by community members facilitated by the PLA facilitation teams mentioned in Step 3. PLA methods and tools using visualisation techniques were applied in order to collect different types of data. For example, spatial data were collected from maps drawn (e.g. social map, resource map and farm sketch); followed by time-related data (e.g. time line, trend line, seasonal calendar and transect walk); and finally institutional (e.g. Venn diagram) and gender analyses were done, the last to assess the relative degree of access to and control over resources held by men and women, boys and girls. Thereafter, solutions for addressing undernutrition were identified and prioritised and a solution matrix was developed and put into a visual time line together with a realistic budget. This plan was developed through a planning dialogue between the different key stakeholders (see Table 1 for an overview of problems and actions identified by the intervention community). Lastly, implementation of the community action plan was done. The plan was visually displayed in order to enhance accountability. Existing organisational structures for implementation were used to oversee the implementation and where none existed new structures were established. The total implementation phase was on average 3 years and during this phase no cash or in-kind hand-outs, such as food supplements, were given.

Table 1 The main synergistic interventions chosen by the intervention communities

Step 5: Monitoring, evaluation and re-planning

This was based on the community action plan and partly performed by the communities themselves for issues that referred to community matters using their visual plans, and partly by the PLA facilitation team for issues that referred to service provision and quality of services. The monitoring and evaluation provided input into the next phase of planning, thereby providing an opportunity for a cyclical process of critical reflections on improvements.

Sampling

Makueni District in Eastern Province had been selected by the Government of Kenya for a pilot PLA intervention on the basis of being a district with poor health and nutrition indicators^{(14, 15)}. In Makueni District there were two kinds of communities chosen: (i) communities who were about to begin CBNP activities, these communities were called intervention communities; and (ii) communities who were not intending to start a CBNP activity within the next 3 years, these were called non-intervention communities. The latter were selected on the basis that there would be minimal crossover from the intervention communities. The long-term plan was that the PLA interventions through the CBNP should cover the whole district; however, PLA could not be started in all communities at the same time. The selection of communities to be in the PLA intervention was made on an informal basis around a range of logistical issues (see Table 2 for a detailed list of names of communities). Within the six intervention communities and ten non-intervention communities, children aged 12–60 months were selected. Based on information from experienced District Social Development Officers there were no striking differences between intervention and non-intervention communities. As this programme was a government intervention and not designed as a study, there was no randomisation procedure done. However, as shown in Table 3, the anthropometric indices in children in intervention and non-intervention communities were similar at baseline, thus justifying comparison.

Table 2 List of intervention and non-intervention communities in Makueni District, Eastern Province of Kenya

Table 3 Comparison of mean Z-scores between the non-intervention and intervention communities at baseline (1998), Makueni District, Eastern Province of Kenya

CBNP, Community Based Nutrition Programme; W/A, weight-for-age; H/A, height-for-age; W/H, weight-for-height.

Evaluating the intervention

The baseline measurements were made in 1998, the intervention started in the beginning of 1999 and the follow-up measurements were made in 2002. At each measurement, all available children in the community between the ages of 12 and 60 months were measured. This resulted in 162 and 167 children being measured at baseline in the non-intervention and intervention communities, respectively. At follow-up, 300 and 299 children were measured in the non-intervention and intervention communities. The differences in number between baseline and follow-up were due to increased resources and capacity of the government. Weight was measured to the nearest 100 g. Length for children up to 24 months and height for older children were measured to the nearest 0·1 cm. The data were entered and handled using the statistical software package SPSS version 5. Differences in mean Z-scores for height-for-age (H/A), weight-for-height (W/H) and weight-for-age (W/A) were analysed using ANOVA. The differences in the proportions of children with Z-score < –2 using the National Center for Health Statistics standards were examined by χ ² analysis.

Qualitative data to illuminate the process by which the change in child nutritional status had been achieved were collected in three ways:

1. 1. Semi-structured interviews were conducted with fifty-five key informants purposively selected to include a balance of male and female informants and a range of stakeholders.

2. 2. Secondary data from government sources, bilateral and multilateral donor agencies and NGO, as well as published material and grey literature, were analysed.

3. 3. Data systematically generated and analysed during the PLA process (as described above in Step 5 of the intervention) were also used.

Results

Levels of underweight (low W/A) decreased significantly in the intervention group during the 3 years of the programme, whereas no change in underweight was found at follow-up compared with baseline in the non-intervention group (Table 4). Similarly, there was a significant reduction in levels of stunting (low H/A) in the intervention group whereas no change was noted in the non-intervention group (Table 5). Thinness (wasting; low W/H) was at similar levels at baseline and follow-up in both non-intervention and intervention groups (Table 6). Using the data in Tables 4 to 6, we tested the differences in Z-scores for significance; none achieved statistical significance.

Table 4 W/A of children aged 12–60 months at baseline (1998) and follow-up (2003) in the non-intervention and intervention communities, Makueni District, Eastern Province of Kenya

W/A, weight-for-age; CBNP, Community Based Nutrition Programme.

Table 5 H/A of children aged 12–60 months at baseline (1998) and follow-up (2003) in the non-intervention and intervention communities, Makueni District, Eastern Province of Kenya

H/A, height-for-age; CBNP, Community Based Nutrition Programme.

Table 6 W/H of children aged 12–60 months at baseline (1998) and follow-up (2003) in the non-intervention and intervention communities, Makueni District, Eastern Province of Kenya

W/H, weight-for-height; CBNP, Community Based Nutrition Programme.

During the PLA process the community, assisted by the facilitation team, identified underlying causes of undernutrition in their young children to be:

1. 1. Erosion of social cohesion between service users, providers and formal and informal leaders.

2. 2. Poor access to and control over resources.

3. 3. Poor provider and user capacity to plan and take collective action on issues relating to health, nutrition and social services.

4. 4. Poor accountability structures which impair the enforceability of agreed plans; this includes weak community leadership and organisational setup.

Solutions to these issues were suggested and action taken, such as: (i) establishing multi-sectoral teams which would facilitate a dialogue that created better cohesion between public service providers and end-users; (ii) identifying and assessing existing resources and services available with the help of the facilitators; (iii) training of local leaders in relevant technical topics and leadership; and (iv) building structures to ensure accountability through access to timely and relevant information that could be acted upon. The results of these interventions were an increased demand for public services in the communities as well as more targeted, effective and efficient services from the health service providers. Furthermore, through the increased awareness on gender differences in access to and control over resources, men started helping woman with household chores and child care resulting in a reduction of the hours of work per woman from 18 to 12 h/d.

A participatory evaluation was carried out after the first round of PLA and the results were analysed with the assistance of the same facilitation team. The evaluation process involved group reflection on the activities carried out and the outcome showed that the PLA process was widely perceived to have strengthened the social cohesion and capital of the intervention communities.

Key factors for success

The lessons learned from the present study suggest that the following factors were important:

1. 1. The level of pre-existing social cohesion (trust) in a community.

2. 2. The flexible use of culturally appropriate, participatory methods and tools that respond to local diversity and embrace complexity.

3. 3. The inter-sectoral collaboration and the sharing of knowledge and experiences at all levels of government tiers.

4. 4. The caring attitude and supportive behaviour of (local) leaders and the facilitation team.

5. 5. The building of vertical, horizontal and diagonal linkages and networks and the involvement of stakeholders from different levels of the public sector.

6. 6. Having a technical ‘neutral’ ministry leading the implementation of nutrition strategies, including having a funding structure that can support and sustain multi-sectoral actions.

It is the combination and harmonisation of all of these factors that may lead to the sustainable personal, institutional and professional changes that are needed to reduce social inequity and contribute to permanent gains in child nutritional status.

Intervention costs

The cost of the invention was calculated after the CBNP programme had effectively started over the 3-year period (1999–2002) and included the first twenty-six intervention communities (about 6000 people). The total cost was $US 158 000, as shown in Table 7. This total did not cover the capital costs of transport (thirty bicycles, six motor cycles and one four-wheel-drive vehicle) or the cost of honoraria for the local research groups who implemented the baseline and follow-up surveys.

Table 7 Estimated recurrent cost of the programme in Makueni District, Kenya (1999–2002)

The overall running expenses of the ‘core’ programme were therefore about $US 9 per person per year for 3 years (1999–2002). A further calculation of cost in a replication phase has shown that the implementation figure could be reduced to $US 0·60 per person per year. Moreover, a participatory process evaluation done in 1997 showed that a variety of forms of diffusion was occurring (both geographical and technical) within the neighbourhood of the PLA implementation areas, which could contribute to lowering the replication cost⁽¹⁶⁾.

Discussion

The present study shows the results of analysing routinely collected data from a government programme supported by an international bilateral donor. It was clear that a programme promoting social cohesion and action could be established through collective actions between communities, their leaders and government officers from relevant line ministries, thus enhancing the efficiency of different government departments by working together more effectively. One of the important changes was for example that the different sector interventions were harmonised and aligned to the community action plans. In addition, the programme increased demand for services among intervention communities and enhanced actions taken by community members themselves. While many NGO and some government ministries have established similar programmes, it is disappointing to record that very few, if any, systematic evaluations have been performed of such activities⁽¹⁰⁾.

The present study was not in any way a randomised controlled trial of an intervention. The differences between the Z-scores of intervention and non-intervention groups at baseline were tested and none achieved statistical significance. The facilitators as well as government officers were the same in both communities. The key factor separating them was the PLA process. The number of children measured in intervention and non-intervention communities and between baseline and follow-up varied. This was due to capacity and cost constraints of the government.

The findings show that over a period of 3 years, using a multi-sectoral approach: (i) built social cohesion and voice through the PLA process; and (ii) facilitated an ongoing dialogue through access to information that could increase public accountability. This resulted in reduced overall child undernutrition in the intervention group compared with the non-intervention group.

The Lancet series on undernutrition reports that education on complementary feeding in populations where sufficient food is available may increase H/A Z-score by 0·25 and provision of food hand-outs in populations with insufficient food security may increase H/A Z-score by 0·41⁽²⁾. Hence, compared with other nutritional interventions, typically known to improve child growth by an average Z-score of about 0·3⁽¹⁷⁾, the results of the present study showed a significant change in Z-score by 0·46 in H/A (Table 5). It also showed a significant decrease in the percentage of children with W/A Z-score <−2 (Table 4) and therefore adds to the effectiveness of the PLA approach.

What lessons can be learned from the study for future nutrition programme design when existing structures and systems are under pressure? And what are the key factors that contributed to the success of the PLA process and to cohesion building in the intervention communities?

Community level

From the outset, the intervention focused on what the communities could achieve by and for themselves through the PLA process, facilitated by the multi-sectoral team. The ongoing dialogue and partnership built through this process were community-driven and not dependent on hand-outs such as food or cash. The dialogue between community members, service providers and the formal and informal political leaders enabled the community to build cohesion through sharing experiences and developing the understanding and skills needed to plan and act together to increase their voice. The methods and tools used in the process were negotiated and chosen in partnership with the different stakeholders; they were popular in the community and among its leaders, which was important both for getting the process started and for sustaining it. Using child undernutrition as an entry point provided a focus for the participation and cohesion building process which the community could easily understand. Lastly, having a ‘neutral’ sector coordinating the multi-sectoral response and using nutrition as an entry point not only ensured the creation of a true multi-sectoral team with no significant sector or resource distortions, but also facilitated harmonisation in implementation.

System level

Creation of effective broad-based multi-sectoral teams demands support from top-level systems planners and politicians. This can be challenging given the organisational compartmentalised set-up of public systems in separate ministries and with separate vertical funding structures.

Since the introduction of the Sector Wide era (SWAp) in the 1990s, the introduction of the Paris Declaration and the Accra and Busan Agenda for Action, concurrently with the weak understanding of a ‘sector’, the SWAp approach and its implementation have ended up in a sector ‘narrow’ approach determined by the flow of fund structures, facilitated by the degree of simplistic implementation and often compounded by technical capacity and system constraints. Working in multi-sectoral teams has proved difficult in the past but given the multi-sectoral causation of child undernutrition and the urgent need to take action, there is no choice but to carefully examine and widen the SWAp approach and try again.

The present study showed that integrating social educational processes such as PLA as part of wider programme implementation, in this case the CBNP, may create an enabling environment for social action and change that can address the underlying causes of poor nutrition. Through facilitating and dialogue between stakeholders at different levels, the multi-sectoral facilitation teams were able to support vertical and diagonal social cohesion building that would help reduce differences in power, resource bases and status. It also facilitated community access to important services, such as health, education, agriculture and water and sanitation, not to leave out provision of social safety nets. The PLA process furthermore created political representation of the users of services in decision-making bodies from local to district levels. The ongoing dialogue created a better fit between different perceptions of community needs, demands and decision-making processes. But for such a fit to be achieved, it was crucial that the facilitation team had patience and demonstrated respect and flexibility in the knowledge building and local decision-making processes.

Policy level

At the policy level the PLA process was used by the CBNP to encourage and facilitate various government ministries to implement common plans. However, it is important to note that during the implementation phase the control of the facilitation processes moved from being owned by central government actors to being owned by local communities.

PLA may provide a useful tool to identify and address the underlying causes of child undernutrition, causes that are rarely addressed in traditional nutrition programmes including in the Lancet series on nutrition. The PLA approach stresses the importance of understanding the local context for designing socio-culturally appropriate and sustainable interventions. Unfortunately no follow-up study was later done in the non-intervention communities due to financial constraint by the Kenyan government.

Conclusion

The present study has provided an outline of the guiding principles underlying the PLA approach, principles which may be applied in various contexts. Given the effectiveness and low replication costs of the strategy of integrating a PLA approach in community-based nutrition programmes, it now needs to be tried and tested within a wider sectoral and geographical framework to provide solid evidence for its effect in combating and potentially preventing child undernutrition.