Ethiopia has a high burden of child undernutrition. The overall population prevalence of stunting increases distinctly in infants after 6 months of age. By 24 months of age, ~47 % of children became stunted, ~30 % of children were underweight and ~11 % were wasted in 2014( 1 ). A community-based participatory nutrition promotion (CPNP) programme was designed to promote appropriate feeding and hygiene behaviours among caregivers of children 6 to 23 months of age( Reference Kang, Kim and Sinamo 2 ). This age range has been recognized as the most potentially responsive intervention period to mitigate the adverse effects of undernutrition( Reference Victora, de Onis and Hallal 3 ). The CPNP programme was added to the context of government programmes of Essential Nutrition Action (ENA) and Community-based Management of Acute Malnutrition (CMAM). Its effectiveness in improving some feeding behaviours of caregivers and reducing child stunting was demonstrated in a cluster-randomized trial( Reference Kang, Kim and Sinamo 2 , Reference Kang, Suh and Debele 4 ).
Despite an increasing number of publications in impact evaluation for public health programmes especially in the realm of international development( Reference Cameron, Mishra and Brown 5 ), the process of how interventions achieve expected outcomes remains unexamined or only briefly described in most community trials( Reference Oakley, Strange and Bonell 6 ). In an attempt to fill these gaps, a few theory-driven process evaluations have been utilized to open the ‘black box’ of programming( Reference Saunders, Evans and Joshi 7 , Reference Rossi, Lipsey and Freeman 8 ).
Programme impact pathway (PIP) analysis – one of the methodological approaches to understanding the implementation process and mechanism of interventions – is ‘the pathway from an intervention input through programmatic delivery, household and individual utilization to its desired impact’( Reference Kim, Habicht and Menon 9 , 10 ). In PIP analysis, the causal mechanisms of a programme are explored by assessing various operational components of the programme and uncovering its successes and failures. This procedure helps us better understand how and why programmes could or could not achieve their expected impact.
A few PIP or process evaluation studies are available that evaluated the programme processes for large-scale complementary feeding programmes involving social and behaviour change communication strategies( Reference Robert, Gittelsohn and Creed-Kanashiro 11 – Reference Kim, Ali and Kennedy 16 ). In a PIP study for an infant and young child feeding programme in Bangladesh, the authors described the capacity of the implementation staff, the extent of fidelity to the intervention and identified programme facilitators and barriers such as family support and lack of resources( Reference Avula, Menon and Saha 14 ). A social franchise model in Vietnam demonstrated its effective management of the franchise system and showed some challenges in service utilization( Reference Nguyen, Menon and Keithly 15 ). A process evaluation in Ethiopia assessed implementation fidelity to the training messages among service providers and caregivers, and underscored the importance of addressing gaps in the quality of nutrition services given to various groups( Reference Kim, Ali and Kennedy 16 ).
Using a similar PIP approach, the current study attempted to describe mechanisms of CPNP: implementation, utilization and influence. A conceptual diagram was created for a PIP through which the CPNP programme would be expected to impact child feeding and hygiene practices among caregivers and, finally, child nutritional status (Fig. 1). As a result, the PIP analysis encompassed five specific programme components along the potential impact pathway of the CPNP programme.
-
1. Degree of implementation of the CPNP programme: How many CPNP sessions were held per kebele (smallest administrative unit in Ethiopia) and how many participants attended per session? To what extent were the CPNP sessions implemented following the intervention protocol?
-
2. Participants’ perception of the nutrition sessions: To what extent did the participants perceive their own engagement in the nutrition sessions?
-
3. Participants’ message recall: To what extent did the participants recall key messages from the nutrition sessions and food ingredients of the CPNP menu?
-
4. Utilization of child feeding and hygiene practices among participants: Were there differences in practices of child feeding and hygiene among participants compared with non-participants at the early stage of the programme?
-
5. Participants’ engagement in other ongoing nutrition programmes: Were there differences in the level of engagement in the ENA programme among participants compared with non-participants?
Fig. 1 Conceptual programme impact pathway of the community-based participatory nutrition promotion (CPNP) programme, Habro and Melka Bello districts, Ethiopia (2012–2014). (ENA, Essential Nutrition Action; CMAM, Community-based Management of Acute Malnutrition)
Methods
Study area and settings
The present study was conducted in the Habro district in the West Hararghe zone and the Melka Bello district in the East Hararghe zone, Oromia National Regional State, Ethiopia. The two districts depend on mixed crop agriculture, with khat and coffee as main cash crops. These areas have been in persistent poverty with frequent natural disasters such as drought.
Programme description
The Ethiopia Integrated Nutrition Project (EINP), which involved three nutrition programmes (i.e. CPNP, ENA and CMAM), was implemented to promote optimal child feeding and reduce child undernutrition in the two districts stated above.
The CPNP programme was designed by World Vision Ethiopia and World Vision Korea, and details of the programme have been described elsewhere( Reference Kang, Kim and Sinamo 2 ). In brief, the CPNP engaged caregivers in a 2-week group nutrition session, emphasizing utilization of locally available resources and engagement of community members. The nutrition sessions were conducted in twenty-three kebeles (the smallest administrative unit in Ethiopia). A total of twenty-three female operators, who had completed at least secondary education, were recruited from their kebeles and led group nutrition sessions at different villages within the kebele; eight supervisors supported the operators’ activities. First, the operators screened programme participants ahead of each session. Participants were mainly children 6 to 24 months of age. Underweight children (weight-for-age Z-score <−2) were prioritized for participation, but non-underweight children were not excluded from the group. Approximately seven to twelve child–mother pairs attended each session. The participants were encouraged to contribute daily food ingredients and materials such as firewood. Each session began with singing a song about the importance of child nutrition. After the song, two mothers prepared complementary foods, which were different types of thick porridge, according to recommended recipes utilizing locally available and affordable ingredients. World Vision nutrition experts designed the recipes through a local market survey and a field test. Nutrition messages were discussed regarding optimal breast-feeding, preparing nutritious complementary foods, hand washing and cleanliness of the house, and health-seeking practices. As the complementary foods were being prepared, caregivers washed their hands and the children’s hands with soap and fed their children on the spot. All enrolled children were weighed on the first and twelfth day of the session and then after 1 month. In the 2 weeks following the nutrition session, participants had follow-up visits from the operators or the supervisors every other day (five to six times). Once a CPNP session was carried out in a small village unit within a kebele, the next session was prepared in a neighbouring village within the same kebele. First, the CPNP operation training was provided to the EINP staff, who then delivered the training to the operators and supervisors. The first CPNP session was held at all twenty-three kebeles in August 2012 and continued until August 2013 for a total of 13 months.
In the EINP, the ENA programme was defined as delivering monthly nutrition counselling by community volunteers to all mothers with children under 2 years of age in the project area. The main ENA messages included adequate breast-feeding and complementary feeding practices, caring for sick children and the importance of micronutrients. At the time of formulating the EINP, it was reported that the ENA programme was poorly implemented and community members were sceptical of community volunteers’ capacity for delivering nutrition counselling. To enhance the ENA programme, the EINP provided ENA training to health extension workers (HEW) and community volunteers in that area.
In the CMAM programme, severely wasted children (mid-upper arm circumference less than 11·0 cm but without other medical complications) were referred to the nearest health post/health centre to receive Plumpy’Nut, a type of therapeutic food, until their recovery( 17 ). On occasion, families with moderately wasted children (mid-upper arm circumference of 11·0 to 11·9 cm) received a monthly ration of corn–soya blend and vegetable oil. The first distribution of corn–soya blend was put in place in August 2012.
Original effectiveness trial
The PIP study was embedded in a cluster-randomized trial evaluating the effectiveness of the CPNP programme in reducing child undernutrition( Reference Kang, Kim and Sinamo 2 , Reference Kang, Suh and Debele 4 ). Six intervention clusters implemented the CPNP programme along with the ENA and CMAM programmes (intervention area) while six control clusters implemented only the ENA and CMAM programmes (control area). The sample size (n 2064) was determined to detect an effect size of 0·2 length-for-age Z-score at the end of the 12-month follow-up between children in the two areas. The estimation was based on a power of 80 %, type 1 error fixed at 0·05 and intra-class correlation coefficient fixed at 0·054 with twelve clusters. Establishment of twelve clusters in the study area, the random selection procedure of children from a roster prepared by HEW/community volunteers and the detailed study flow from visit 1 to visit 5 are described elsewhere( Reference Kang, Kim and Sinamo 2 ). At enrolment (survey visit 1), the trial subjects comprised 1790 children 6 to 12 months of age paired with their mothers living in the study area (n 876 in intervention area, n 914 in control area). These child–mother pairs were visited every 3 months, totalling five visits (from visit 1 to visit 5) for 1 year. Visit 1, the first round of data collection, was conducted from 20 November 2012 to 19 February 2013; visit 5, the last round of data collection, was conducted from 20 November 2013 to 19 February 2014.
Study subjects and sample size
Study subjects for the five programme components were as follows (Fig. 2): (i) for component 1, 372 CPNP nutrition sessions implemented in the intervention area for 13 months; (ii) for components 2 and 3, as a subset of mothers enrolled in the original effectiveness trial from the intervention area, 197 mothers who were identified to attend the CPNP sessions before survey visit 2; and (iii) for components 4 and 5, all 1790 child–mother pairs enrolled in the effectiveness trial. The first survey visit (visit 1) was conducted 3 months after the CPNP nutrition sessions started, due to a delay in the preparation of the child roster. Thus, the mother–child pairs in the intervention area at the time of visit 1 included both CPNP participants and non-participants, who were neighbouring mothers living in the intervention area but never attended the nutrition sessions.
Fig. 2 Diagram for study flow and composition of study subjects following the programme impact pathway of the community-based participatory nutrition promotion (CPNP) programme, Habro and Melka Bello districts, Ethiopia (2012–2014)
Data collection
Data collection methods and main outcomes for the five programme components are listed in Table 1. For the degree of implementation of the CPNP programme (component 1), we reviewed the attendance records of 372 nutrition sessions and the records of observation checklists of 114 sessions (30·6 % of 372 sessions; all available records). The following information was extracted from the attendance records: number of children per session, age at enrolment, sex, attending days, weight (kg) at first and twelfth days, session starting dates and completion dates. Parental or household-level information was not available in the attendance records. The observation checklists in the study were adapted from the observation lists developed by World Vision International, which were used in similar nutrition programmes( 18 ). A total of thirteen items in the observation checklists were extracted as fidelity indicators as follows: session preparation (four items), appropriate hand washing (two items), message discussion (one item), food preparation (four items) and support from HEW/community volunteers (two items). Each item was graded by supervisors for any three days out of the twelve session days. The performance level was assessed as ‘well’ (4 points), ‘somewhat’ (2 points) or ‘poor or absent’ (0 point) at each observed day. If the three-day total score for each item (range: 0 to 12) was 6 or above, we considered the session activity to be appropriately explained and performed according to the guidelines.
Table 1 Summary of data collection methods and main outcomes by evaluation component of the CPNP programme, Habro and Melka Bello districts, Ethiopia (2012–2014)
CPNP, community-based participatory nutrition promotion; I, intervention area; C, control area; ENA, Essential Nutrition Action.
* Survey visits started 3 months after the CPNP session started and continued every 3 months, for a total five times (visit 1 to visit 5).
† Plumpy’Nut was provided to severely acute malnourished children through the Community-based Management of Acute Malnutrition programme.
A recall survey was conducted to explore the participants’ perception of the nutrition sessions (component 2) and message recall (component 3). At visit 2, mothers were asked about how actively they had participated in session activities such as child feeding, food preparation and hand washing during food preparation and before child feeding. The mothers rated their participation as (i) ‘I actively participated’, (ii) ‘I participated, but not actively’ or (iii) ‘I didn’t participate’. Perceptions regarding the sessions (which included satisfaction with the sessions, confidence in putting the CPNP lessons into practice and feeling that the sessions were useful) were assessed using 5-point Likert scales but were later categorized into binary indicators. The mothers were asked whether they had contributed materials to the sessions, which items they had contributed, and to recall the nutrition messages discussed during the sessions and the food ingredients used at the sessions. The mothers were also asked how many follow-up visits they had from the operators/supervisors after completion of the nutrition sessions.
To examine utilization of child feeding and hygiene practices at an early stage of the programme (component 4), 1790 mothers in the effectiveness trial were asked at visit 1 about their infant and young child feeding practices based on 24 h dietary recall questionnaires( 19 ). Hand-washing practices were assessed by asking whether the mothers washed their hands with soap at four critical time points: after defecation, after cleaning the child’s bottom, before preparing food and before feeding the child.
To assess mothers’ engagement in other ongoing nutrition programmes (component 5), 1790 mothers were asked at each visit whether they had ever engaged in an ENA activity by HEW/community volunteers in the past 3 months and whether their child ever received Plumpy’Nut in the past 3 months from the CMAM programme. The mothers were also asked if they ever attended CPNP nutrition sessions in the past 3 months.
Variables construction and statistical analysis
Data were entered into Microsoft® Office Excel 2007 and double-checked by data entry personnel. Data analysis was performed using the statistical software package Stata 12.0. All statistical tests were two-sided and significance was set at P<0·05.
Descriptive analyses were conducted for each of the five programme components of the PIP. For degree of implementation of the CPNP programme (component 1), we computed the average number of sessions per kebele, the average number of participants per session, the percentage of sessions held out of the expected number of sessions and the percentage of participants out of the expected number of participants. In addition to this, the percentage of the monthly programme enrolment and the percentage of cumulative enrolment were calculated to find the total number of participants. Fidelity was defined as ‘the percentage of nutrition session activities completed by operators that adhered to the CPNP guidelines’. Fidelity indicators constructed from the attendance records included percentages of the following: (i) sessions having seven to twelve participants; (ii) children 6 to 24 months of age; (iii) participants attending for 10–12 d; and (iv) children whose weight increased by more than 200 g by the 12th day of the sessions. Fidelity indicators from the observation checklists included percentages of the following: (i) clean session place; (ii) clean food utensils; (iii) water use from a safe source; (iv) available latrine; (v) hand washing before feeding; (vi) hand washing before food preparation; (vii) key message discussion; (viii) food ingredients according to recipe; (ix) at least seven food ingredients; (x) age-appropriate portion size; (xi) food contribution from participants; (xii) support from HEW on at least one day; and (xiii) support from community volunteers on at least one day.
The following variables were evaluated regarding the participants’ perception of the session (component 2): material contribution to the CPNP session, perceptions of the session (e.g. satisfaction, confidence in putting the CPNP messages into practice) and active participation in the main CPNP activities (e.g. food preparation, hand washing, child feeding). These variables were presented as percentages of those participants who accepted/participated/perceived specific elements out of the 197 mothers who were identified to attend the CPNP sessions before survey visit 2.
The participants’ message recall (component 3) was presented as a percentage of those who remembered specific CPNP messages (e.g. preparing nutritious complementary foods) as well as specific food ingredients that were used at nutrition sessions out of the above 197 participants.
Evaluating utilization of child feeding and hygiene practices at visit 1 (component 4) involved the following indicators: the percentages of currently breast-feeding, minimum dietary diversity, minimum meal frequency and hand washing with soap. Currently breast-feeding was defined as ‘the proportion of mothers who continued breast-feeding the child at the time of the visit’. Following WHO infant and young child feeding guidelines( 19 ) we defined the following: minimum dietary diversity as ‘the proportion of children 6–24 months of age who receive foods from four or more food groups’; minimum meal frequency as ‘the proportion of the breast-fed and non-breast-fed children 6 to 24 months of age who receive solid, semi-solid or soft foods the minimum number of times or more’; and hand washing with soap as ‘the proportion of mothers who washed their hands with soap at four critical time points’. We tested if there were significant differences in these feeding and hygiene indicators among CPNP participants as compared with non-participants in the intervention area or the control group. Multivariate logistic regression analyses were performed to examine the hypothesis; geographic clustering and variables that were significantly different between groups at visit 1 were controlled for.
For the participants’ engagement in other ongoing nutrition programmes (component 5), the proportion of mothers/children who were engaged in the ENA or CMAM programme was computed by summing up all events (e.g. the mother was engaged in the ENA activity in the past 3 months or the child received Plumpy’Nut in the past 3 months) from visit 1 to visit 5 divided by the total number of mothers/children assessed from visit 1 to visit 5 (visit 2 to visit 5 for receiving Plumpy’Nut). Mean percentages of those indicators were compared by two-sample tests of proportions (z test) between CPNP participants, non-participants in the intervention area and the control group.
Results
Following each programme component, we describe findings of the implementation, utilization and influence of the CPNP programme.
Degree of implementation of the community-based participatory nutrition promotion programme
Data from attendance records indicated that nutrition sessions were overall successfully performed. A total of 372 CPNP sessions were held in twenty-three kebeles during 13 months (56 weeks) of programme implementation. According to our programme design, we expected one session to be held every 3 weeks per kebele, a total of eighteen sessions during 13 months. The results revealed that each kebele had a new session every 3·5 weeks and had an average of 16·2 CPNP sessions, with a range of 13–20 sessions in the twenty-three kebeles (Table 2). On average, 8·9 child–mother pairs attended each session, with a range from 4 to 20 child–mother pairs, compared with an average of 10 child–mother pairs that were expected. The amount of actual sessions held was 89·9 % of the 414 expected sessions and 79·7 % of the 4140 expected participants attended the sessions. Out of 3299 participants who attended 372 sessions, ~60 % had attended at least one session by the sixth month of the programme and ~80 % had attended at least one session by the ninth month.
Table 2 Degree of implementation of the CPNP programme during the 13-month implementation period, Habro and Melka Bello districts, Ethiopia (2012–2014)
CPNP, community-based participatory nutrition promotion; EINP, Ethiopia Integrated Nutrition Project.
* All information was extracted from the EINP programme monitoring report (August 2012 to August 2013). Data were not available for: child age (n 17); child weight either at admission (n 16) or at 12th day of sessions (n 104); attending days (n 3); outlying weight-for-age Z-score (<−6 or >6; n 7).
† All information was taken from all available observation checklists (n 114) regularly prepared by supervisors and analysed for this evaluation purpose. Each item was graded for any three days out of twelve session days as ‘well’ (4 points), ‘somewhat’ (2 points) or ‘poor or absent’ (0 point), at each observed day. If the three-day total score for each item (range: 0 to 12) was 6 or above, the session activity was considered to be appropriately performed according to the guidelines.
We marked 90 % as the cut-off to assess the satisfactory level of each fidelity indicator (Table 2). This fairly high standard was based on our investigators’ subjective perception of the large percentage points found among many fidelity outcomes. Indicators from the attendance records showed that 91·7 % of the sessions consisted of seven to twelve participants, which is an appropriate number for a group activity in one session; 95·9 % of the children were 6 to 24 months of age, meeting the target child’s age criteria; and 90·3 % of participants attended the sessions for 10 to 12 d. As an immediate outcome after completing a nutrition session, the percentage of children whose weight increased by more than 200 g was moderate at 66·7 %.
Most indicators from the observation checklists also showed a satisfactory level of fidelity (>90 %) but a few indicators did not meet the cut-off level. Out of 114 nutrition sessions, 96 % were held in a clean space and 98 % used clean food utensils. Hand washing before feeding (99 %) and before food preparation (100 %) was practised at almost all sessions. Discussion related to key nutrition/health message was also operated at a satisfactory level (97 % of sessions). We found that 98 % of sessions that prepared complementary foods were prepared according to the recipe (i.e. washing ingredients with clean water, cutting and boiling them). However, only 73 % of sessions used at least seven food ingredients at all three observed days and 88 % of sessions offered age-appropriate portions to children. Both the availability of latrines and the use of drinking-water from a safe source were relatively low at 86 and 85 %, respectively.
It is worth noting that nutrition sessions often involved the engagement of key community health personnel even though that was not a necessary component in programme protocol; 66·7 % of nutrition sessions were supported by HEW and 57·9 % were supported by community volunteers. This occurrence shed light on the possibility of the CPNP programme being integrated into the existing community health system to enable it to be community owned.
Although the extent of follow-up household visits after CPNP sessions was not evaluated for all participants, poor level of follow-up visits by operators/supervisors was reported among the participants from the recall survey at visit 2 (n 197; subjects for components 2 and 3). Of the participants, 72·3 % were visited at least once and only 5·8 % were visited five or six times in the 2 weeks following the nutrition session.
Participants’ perception of the nutrition sessions
Overall, participants’ perception of the nutrition sessions was positive (Table 3). The participants (n 197) rated their participation level as ‘active’ in various activities (e.g. hand washing, food preparation) with a range of 62·1 to 79·2 %. Almost all (94·9 %) of the participants replied that they had contributed to the sessions, with most of them bringing at least two items to the sessions (mean 2·1 (sd 1·6)). More than 90 % of the participants replied that they were satisfied with the sessions and 80·2 % replied that they could confidently put the CPNP messages into practice.
Table 3 Participants’ perception of the sessions (self-assessment of the degree of participation and contribution, and perception of the nutrition sessions) of the CPNP programme, Habro and Melka Bello districts, Ethiopia (2012–2014)
CPNP, community-based participatory nutrition promotion.
* This analysis only included CPNP participants who attended nutrition sessions by visit 2.
† The total number of respondents is 191 (n 6 treated as missing values as the respondents did not reply to the questions).
‡ The total number of respondents is 187 (n 10 treated as missing values as the respondents did not reply to the questions).
Participants’ recall of messages and food ingredients
By and large, participants recalled session messages at a moderate level and recalled only a few types of food ingredients used in the sessions (Table 4). Out of 197 CPNP participants, most mothers (88·3 %) recalled the message of preparing nutritious complementary foods, 64·0 % recalled the message of responsive feeding, 69·0 % remembered the message of age-appropriate feeding, 64·5 % recalled use of drinking-water from safe sources, 65·0 % recalled the need for cleanliness of the house, 61·4 % recalled the need for cleanliness of food utensils and 68·0 % recalled the message of hand washing with soap; only 31·0 % recalled health-seeking practices. On average, women remembered five out of the eight messages (mean 5·0 (sd 2·0)). Of the participants, 72·6 % replied that they were preparing complementary foods at home as they had practised at the sessions. However, participants remembered only three out of nine food ingredients used in the sessions (mean 3·2 (sd 1·8)). Food ingredients such as grains (81·2 %), oil (58·3 %), sweet potatoes/potatoes/yams (42·1 %) and milk (41·8 %) were recalled relatively successfully, yet fruits (14·2 %), eggs (18·3 %) and yellow-coloured vegetables (21·8 %) were recalled less.
Table 4 Participants’ recall of session messages and food ingredients in the nutrition sessions of the CPNP programme, Habro and Melka Bello districts, Ethiopia (2012–2014)
CPNP, community-based participatory nutrition promotion.
* This analysis only included CPNP participants who attended the sessions by survey visit 2.
Participants’ utilization of child feeding and hygiene practices
The CPNP programme was found to impact the participants’ dietary diversity and hand-washing practices at an early stage of programme implementation.
At visit 1, the mean age of the children was 8·7 (sd 1·9) months and 47·5 % of them were female. In the intervention area, 141 mother–child pairs were CPNP participants who had already attended nutrition sessions at the time of visit 1. We assumed that the CPNP participants identified at visit 1 (at the enrolment) would be practising what they learned at the CPNP sessions, whereas other neighbours in the same intervention area might not be practising these behaviours. We compared the infant and young child feeding practices of the CPNP participants (n 141) with the non-participants (n 695) in the intervention area and with the control group (n 869). No differences were found across the three groups regarding the proportion of currently breast-feeding (Table 5). The percentage of those who met the minimum dietary diversity was greater among participants than non-participants in the intervention area (34·0 v. 19·9 %, P=0·04) and greater than the control group (34·0 v. 16·8 %, P=0·01). No difference in the proportion of minimum meal frequency was found between participants and non-participants. A larger portion of participants reported that they washed their hands with soap at four critical time points compared with the control group (35·7 v. 26·0 %, P=0·01), but not greater than that of non-participants (35·7 v. 43·7 %, P=0·91).
Table 5 Infant and young child feeding and hygiene practices at survey visit 1, Habro and Melka Bello districts, Ethiopia (2012–2014)Footnote *
Ref., reference category; CPNP, community-based participatory nutrition promotion; minimum dietary diversity, proportion of children 6–24 months of age who receive foods from four or more food groups in the past 24 h; minimum meal frequency, proportion of children who receive solid, semi-solid or soft foods the minimum number of times or more in the past 24 h for breast-fed and non-breast-fed children 6 to 24 months of age.
* A total of 1706 mothers were evaluated for feeding and hygiene behaviours out of 1780 mothers at visit 1: control area, n 870; non-participants, n 695; participants, n 141.
† OR, 95 % CI, P values (comparison between non-participants or participants and control area) were computed by logistic regression accounting for clustering effect, child’s sex and age, father’s education level, type of nearest health facility, household wealth index, household food insecurity and district location.
‡ OR, 95 % CI, P values (comparison between participants and non-participants) were computed by logistic regression accounting for clustering effect, child’s sex and age, father’s education level, type of nearest health facility, household food insecurity and district location. Household wealth index was excluded in these regression models because there was no difference between non-participants and participants.
§ Hand-washing practices were assessed by asking whether or not the mother washed her hands with soap and water at four critical time points: (i) after defecation; (ii) after cleaning the child’s bottom; (iii) before preparing food; and (iv) before feeding the child.
Participants’ engagement in other nutrition programmes
Overall, mothers’ engagement in the ENA programme was low over the project period with highest engagement (28·3 %) at visit 2 and lowest engagement (7·5 %) at visit 5. At visit 1, the ENA engagement level among non-participants was similar to that of the control group (10·2 v. 13·2 %). The nature of the ENA programme implementation is intermittent to accommodate the HEW/volunteers’ workload and community events such as harvesting. For this reason, a year of participation in the ENA programme was counted for comparison across the three groups. A year of ENA activity in the intervention area was higher (23·1 v. 12·4 %, P<0·0001) compared with the control area (Table 6). The CPNP participants showed a significantly greater tendency of being involved in ENA activities than the non-participants in the same intervention area (28·2 v. 18·3 %, P<0·0001) or mothers in the control area (28·2 v. 12·4 %, P<0·0001). When counting all cases of receiving Plumpy’Nut over four visits (visit 2 to visit 5), the CPNP participants showed lower receipt of Plumpy’Nut compared with the control group (4·4 v. 7·7 %, P=0·0003), but higher receipt was found compared with non-participants (4·4 v. 2·1 %, P=0·0006).
Table 6 CPNP participants’ engagement in other ongoing nutrition programmes, ENA and CMAM programmes, Habro and Melka Bello districts, Ethiopia (2012–2014)Footnote †
CPNP, community-based participatory nutrition promotion; ENA, Essential Nutrition Action; CMAM, Community-based Management of Acute Malnutrition.
* n is total number of mother–child pairs surveyed across five visits.
† n is total number of mothers engaged in the programme of interest across five visits, each visit was 3 months apart (visit 2 to visit 5 for receiving Plumpy’Nut).
‡ P values were calculated by two-sample test of proportions; comparison between non-participants or participants and control area.
§ P values were calculated by two-sample test of proportions; comparison between participants and non-participants.
║ Out of 1790 mothers enrolled in the effectiveness trial, a total of 1786 mothers (910 in control area (CA), 729 non-participants (NP), 147 CPNP participants (CP)) were assessed for ENA engagement at visit 1; 1547 mothers (805 CA, 521 NP, 221 CP) at visit 2; 1534 mothers (785 CA, 509 NP, 240 CP) at visit 3; 1509 mothers (768 CA, 450 NP, 291 CP) at visit 4; and 1414 mothers (719 CA, 424 NP, 271 CP) at visit 5.
¶ Out of 1780 children enrolled in the effectiveness trial, a total of 1547 children (804 CA, 522 NP, 221 CP) were assessed for receiving Plumpy’Nut at visit 2; 1545 children (789 CA, 514 NP, 242 CP) at visit 3; 1509 children (768 CA, 450 NP, 291 CP) at visit 4; and 1414 children (719 CA, 424 NP, 271 CP) at visit 5. There was no assessment of this outcome at visit 1.
Discussion
The present PIP analysis evaluated implementation, utilization and influence of the CPNP programme in rural Ethiopia. The CPNP programme was successfully conducted with high fidelity for most of the programme elements, while a few activities achieved only a moderate level. The participants’ positive perceptions of the nutrition sessions support the finding that the programme was well accepted in this setting. Their recall of session messages and food ingredients used during demonstrations was moderate. Dietary diversity improved among the participants at the initial stage of the programme, but other feeding and hygiene outcomes did not. More engagement in the ENA programme was found among CPNP participants. These sequential and interlinked outcomes in this PIP analysis help us understand the programme’s effectiveness on child nutritional status.
The CPNP programme was delivered in the study area, achieving 90 % of the expected number of sessions and engaging 80 % of expected participants. Thus, the degree of delivered nutrition sessions and the number of participants would result in letting community members recognize the CPNP programme distinctly, meet participants frequently and actively share the participants’ information.
High fidelity (>90 %) concerning various programme elements – such as achieving seven to twelve participants per session, participants attending for 10–12 d and other performance indicators related to session operation – indicate that the essential elements of the programme were well arranged and executed. The session activities that did not achieve the 90 % cut-off for adequate fidelity should be examined to strengthen the programme design and carefully monitored as the programme progresses in the future. Activities with low fidelity such as maintaining a minimum number of food ingredients at the nutrition sessions (73 %) or using water from a safe source (85 %) might result from a lack of available resources due to poor economic status or improper coordination skills. A high admission of non-underweight children may have led to only a moderate level (~67 %) of children who gained more than 200 g by the end of the session. Out of all admitted children, only 29·3 % were underweight and the remaining were mildly underweight (−2≤weight-for-age Z-score <−1) or above. Underweight children may show faster weight gain in the process of catch-up growth than those who are not( Reference Golden 20 ). Thus, because our group of participants contained a high proportion of non-underweight children, we might not be able to show the level of weight gain that would be present in a group comprised only of underweight children. However, considering the programme effectiveness in reducing the prevalence of underweight (6 %) and stunting (8 %)( Reference Kang, Kim and Sinamo 2 ), we suggest that such flexibility of involving non-underweight children could leverage engagement of more mothers. The broader scope allows the programme to amplify the impact. Although the weight gain indicator will be diluted, the number of children who will be impacted in a variety of ways is increased.
It is noteworthy that this programme operated entirely on the community’s resources except the payment for the operators and supervisors. There was resistance from the community at the start of the CPNP programme regarding their own material contribution to the non-governmental organization-led programme. However, the issue was resolved through several sessions of dialogue with community leaders.
The rates of recall for the various messages (~61–88 %) were largely similar to or higher than those of other studies that involved message recall for complementary feeding( Reference Robert, Gittelsohn and Creed-Kanashiro 11 , Reference Kim, Ali and Kennedy 16 ). CPNP participants were supposed to discuss one of the suggested messages every day with operators and to repeat the message the following week. Furthermore, the CPNP messages overlapped with some of the ENA messages, making it easier for mothers to remember familiar messages from a variety of sources. Conversely, the participants struggled to remember food ingredients used during cooking demonstrations. Some important food groups such as yellow-coloured vegetables, eggs and legumes were recalled at low levels. To strengthen the programme impact on utilization of diverse food ingredients, we need to investigate whether these outcomes resulted from low affordability due to the poor economic conditions at the household level or whether these outcomes resulted from poor messaging during nutrition sessions. Additionally, poor follow-up visits were reported, so identifying the reasons behind the gap and strengthening this activity would potentially increase the effectiveness of the CPNP programme.
Regarding the higher level of engagement of CPNP participants in the ENA programme, these participants might have been more motivated to further their learning through the experience of previous sessions which directly addressed their needs and provided tools applicable to their lives( Reference Williams and Williams 21 , Reference Sirard, Pfeiffer and Pate 22 ). Increased interest in health benefits and psychosocial factors such as a desire for social interaction might be also related to their ENA participation( Reference Nguyen, Kim and Nguyen 23 , Reference Hardin-Fanning and Ricks 24 ). Furthermore, according to the EINP final evaluation report( 25 ), those who had participated in the CPNP programme seemed to play greater roles in the ENA programme, a pre-existing government-led nutritional programme, through active support and leadership. Given that non-participants’ engagement was similar to that of the control area at several survey visits including visit 1, we may conclude that higher ENA involvement among participants resulted mainly from the influence of the CPNP programme. Participation in the nutrition session might enlighten mothers about the importance of nutrition education and information sharing. Thus, we suggest that enhancing the CPNP programme would strengthen the ENA activity in this context where the degree of implementation of ENA activities is far below the expected level. Ultimately, it was also suggested that the interaction between the CPNP and the ENA activity could bring a synergistic effect in improving feeding and hygiene in the community, which might again lead to better child growth outcomes.
A slightly higher proportion of CPNP participants received Plumpy’Nut compared with non-participants, likely due to worse nutritional status among the participants prior to CPNP participation. However, as no information was available regarding when the severely acute malnourished condition was manifest among the respondents (i.e. whether it was before or after the CPNP session participation), it would be difficult to define a potential programme path regarding this result.
A distinctive feature of the current process evaluation study was the utilization of various data sources, such as programme monitoring records and a household survey, which included a nested recall survey. Inter-comparison with these data resources enabled an in-depth understanding of the multifaceted aspects of intervention implementation, saving the additional cost of collecting and processing these data. Programme attendance records and observation checklists showed a potential to be used as useful data sources in the process evaluation.
The present study has a few limitations. First, we had no information regarding the level of knowledge, capacity and activity of HEW/community volunteers in the study area due to logistical limitations. Thus, these factors might confound the effect of the CPNP programme on feeding and hygiene. Second, there were no true baseline data for feeding and hygiene practices and engagement in the ENA programme. Thus, our interpretation of the propagation of proper feeding practices or high ENA engagement among CPNP participants is limited. Third, fidelity indicators used in the study were not completely fine tuned, so there remains some doubt about the degree of accuracy in the findings. Nevertheless, most indicators were straightforward enough to assess overall programme quality. In the future, updating these monitoring modules, elaborating the indicators and appropriately training staff will be necessary for the preparation of programme scale-up.
Our PIP study added to the knowledge of process evaluation studies related to community-based nutrition programmes in developing settings. The findings revealed the programme’s accomplishments and shortages, areas for future improvements, and a potential understanding of the mechanism of CPNP programme effectiveness. The results of the current study will be applied to scaling up the CPNP programme and designing similar social and behaviour change communication models by governments and non-governmental organizations in Ethiopia as well as other East African communities.
Acknowledgements
Acknowledgements: The authors are grateful to all the participating mothers and their children in Habro and Melka Bello districts. They thank all the following: the Ethiopia Integrated Nutrition Project (EINP) team, Habro and Melka Bello Area Development Program offices and World Vision Ethiopia head offices for their administration and field support; the International Ministry Division, World Vision Korea; Wholistic Interest Through Health for their research support; Seeun Park for her support in the field and reviewing the original manuscript; Dr Keith West, Dr David Paige, Dr Bridget Aidam and Jane Kim for reviewing the original manuscript; and Martin Blair and Rachel Oemcke for editing assistance. Financial support: The research was supported by World Vision Korea (project #E197684), Seoul, South Korea and Korea International Cooperation Agency (KOICA), Seongnam-si, South Korea. The opinions stated in this paper are those of the authors as individuals and do not necessarily represent the views of KOICA or World Vision Korea. The funding agencies had no role in the design of the study, data collection and analysis, or presentation of the results. Conflict of interest: Y.K. and P.C. have no conflicts of interest. S.C. was a former employee of World Vision Korea. S.Y. is an employee of World Vision Korea. Authorship: Y.K. and P.C. designed the study. S.C. (former World Vision Korea staff member) set up the EINP in Habro and Melka Bello districts. Y.K. conducted the research, analysed the data and prepared the manuscript draft. S.Y. supported data analysis. P.C., S.C. and S.Y. edited the manuscript and substantially contributed to interpretation of the results and the discussion. Y.K. had primary responsibility for final content. All authors read and approved the final manuscript. Ethics of human subject participation: This study, including its original effectiveness and process evaluation, was approved by the ethical committee at the Oromia Regional Health Bureau, Ethiopia. The original trial was registered with Current Controlled Trials, ISRCTN 89206590. Written consent was obtained from each participant who was involved in the effectiveness trial.
