Assessing the scale and spread of an experiential teaching kitchen in after-school programming among school-age children

Abstract

Objective:

To scale-out an experiential teaching kitchen in Parks and Recreation centres’ after-school programming in a large urban setting among predominantly low-income, minority children.

Design:

We evaluated the implementation of a skills-based, experiential teaching kitchen to gauge programme success. Effectiveness outcomes included pre–post measures of child-reported cooking self-efficacy, attitudes towards cooking, fruit and vegetable preference, intention to eat fruits and vegetables and willingness to try new fruits and vegetables. Process outcomes included attendance (i.e., intervention dose delivered), cost, fidelity and adaptations to the intervention.

Setting:

After-school programming in Parks and Recreation Community centres in Nashville, TN.

Participants:

Predominantly low-income minority children aged 6–14 years.

Results:

Of the twenty-five city community centres, twenty-one successfully implemented the programme, and nineteen of twenty-five implemented seven or more of the eight planned sessions. Among children with pre–post data (n 369), mean age was 8·8 (sd 1·9) years, and 53·7 % were female. All five effectiveness measures significantly improved (P < 0·001). Attendance at sessions ranged from 36·3 % of children not attending any sessions to 36·6 % of children attending at least four sessions. Across all centres, fidelity was 97·5 %. The average food cost per serving was $1·37.

Conclusions:

This type of nutritional education and skills building experiential teaching kitchen can be successfully implemented in a community setting with high fidelity, effectiveness and organisational alignment, while also expanding reach to low-income, underserved children.

Healthy nutrition during childhood lays the foundation for life-long health.⁽¹⁾ Early childhood is especially formative, as children begin making their own food choices and establishing life-long food preferences and dietary practices.^(2–4) For children from low-income, minority communities, lack of exposure to a variety of healthy foods can predispose them to both the early emergence of obesity and preferences for foods high in calories but low in nutrient value.^(5,6) Therefore, developing strategies to improve nutrition-related attitudes and behaviours may be an important component of improving nutritional quality among children and may also contribute to reducing health disparities.

Teaching kitchens deliver curricula designed to improve nutrition behaviours by teaching cooking skills. They can consist of both observational components where people learn by watching and experiential components where people learn through hands-on activities to teach specific skills.⁽⁷⁾ Previous trials of teaching kitchens among school-age children have demonstrated effectiveness in improving cooking attitudes, cooking self-efficacy, fruit and vegetable preference (FVP) and willingness to try new fruits and vegetables (WTT).^(8–11) These studies indicate that although children are largely dependent on their parents to make food decisions for them, significant improvements in attitudes and behaviours around healthy food choices can be achieved through implementation of experiential teaching kitchens among school-age children. However, issues of fidelity, adaptation in various settings and optimal dose are poorly characterised, which makes it difficult to determine if such interventions can be scaled-out to reach more diverse populations. Furthermore, the majority are school-based programmes, which have unique challenges surrounding implementation (e.g., availability of instruction time, instructor training and school-board curriculum restrictions).

The purpose of the current study was to scale-out an experiential teaching kitchen in Parks and Recreation centres’ after-school programming in a large urban setting among predominantly low-income, minority children. The programme’s success was evaluated by (1) evaluating effectiveness on behavioural outcomes in community Parks and Recreation centres, (2) assessing the ability of the organisation to adopt the programme across centres with high fidelity while tracking how adaptations impacted programme fidelity and (3) considering the capacity of the organisation to maintain programming without outside supports.

Methods

We implemented a skills-based, experiential teaching kitchen by integrating the curriculum into existing after-school programming held at community Parks and Recreation centres in Nashville, TN. These centres serve predominantly low-income families from urban and suburban Nashville, most of whom come from households with annual incomes below the federal poverty level (e.g., $26 200 for a family of four).⁽¹²⁾ We initially planned to implement the programme in all twenty-five centres, two of which had previously participated in a pilot test of the teaching kitchens. Inclusion criteria consisted of (1) all children attending the after-school programme and (2) ability to complete the survey in English. Completion of the survey before and after the programme’s 4-month implementation was required for inclusion in child-level analyses. Additionally, children younger than 6 years old were excluded from survey analyses, due to small sample size (n 3) and survey response patterns that suggested failure to understand the instructions (e.g., drawing pictures instead of circling responses). The study, including materials, protocols and consent procedures, received approval from the Vanderbilt University Institutional Review Board (IRB No. 180147). Written informed consent was obtained from parents prior to collecting individual-level outcome data for children, though children without parental informed consent could participate in the programming without contributing data. Children provided signed assent prior to completing each survey.

Teaching kitchen sessions were delivered by staff of the Parks and Recreation department, who had no previous training or certification in nutrition education prior to participating in our programme. There was a range of kitchen facilities available at each of the twenty-five participating centres: some centres had full teaching kitchens, while others had no formal kitchen facilities, in which programme sessions were taught in multi-purpose rooms. Prior to delivering the programme, each of the staff underwent a 2-h training and certification session delivered by the study team. This 2-h training included (1) scientific background and an overview of the philosophy of experiential teaching kitchens, (2) specific instruction on the curricular components, safety/hygiene training and how to facilitate an interactive session and (3) instruction on fidelity measurement and data collection approaches. Teaching kitchen sessions were designed to be 30 min long and to be delivered twice/month during the 4-month spring semester of the school year (for a total intended dose of eight sessions per centre). At each session, facilitators would implement one recipe that was selected by the Parks and Recreation department centrally to facilitate distribution of food to the participating centres city-wide. One recipe was taught per session, and sessions consisted of 8–20 students. Each session focused on one recipe and consisted of group teaching, where the facilitator provided instruction and demonstrated skills, followed by either small-group implementation, where students worked in groups of 3–5 to complete various components of the recipe, or individual production from start to finish. The teaching kitchen curriculum was designed to be predominantly experiential. The curriculum for each session consisted of several components, regardless of which recipe was chosen. These included: (1) nutrition education specific to each recipe, (2) teaching students how to read recipe instructions, (3) teaching skills necessary to prepare each recipe and (4) discussion of how students could prepare each recipe at home (Fig. 1).

Fig. 1 Summary of content, timing and evaluation of experiential teaching kitchen programme

The nutrition education curriculum was developed by our study team, which includes registered dieticians, experts in health behaviour change and pediatricians. Recipes were designed to be quick, healthy, affordable, accessible and child-friendly. Specific recipe criteria included: maximum of 30 min of combined preparation and cooking time; maximum of 10 g of sugar and at least 5 g of fibre per serving; maximum cost of $3·00 per serving; ability to purchase with available, Supplemental Nutrition Assistance Program-eligible foods; ability to be used in a basic home kitchen with limited resources; and the inclusion of hands-on directions and talking points geared towards school-age children. Each session was designed to cover multiple culinary preparation skills including (1) washing of produce, (2) reading recipes, (3) measuring ingredients, (4) combining ingredients, (5) cutting/dicing foods, (6) using the oven/hot plate and (7) hand hygiene.

Data collection

Data were collected at the individual and organisational level. Individual-level child data included dose received by each child (number of sessions a child attended from 0 to 8) and surveys administered to children at baseline (January 2019) and following the final teaching kitchen sessions (May 2019). To assist children with low-literacy, surveys were read aloud, and response options included pictures when possible. All surveys were conducted in English. Effectiveness outcomes at the child level were measured using an adapted, 24-item survey that consisted of previously validated items,^(8,9,13) although some scales were slightly adapted for use with children as young as 6 years old. Example adaptations include shortening the FVP survey by removing several items, reducing text and using pictures whenever possible (e.g., using smiling or frowning faces to express agreement or disagreement). The exact questions, response options and guided instructions are included in online Appendix 1 (Child Survey). The 24-item survey analysed in the current study consisted of five scales and included eight cooking self-efficacy (se) items, six FVP items, six cooking attitudes (AT) items, two intention to eat fruits and vegetables (ITE) items and two WTT items. To generate summary scores, the responses from each domain were summed such that higher scores indicated a more positive response for each scale.

Organisational level data included child attendance at each centre, the number of sessions delivered divided by the number of sessions intended, satisfaction (measured by asking teaching kitchen facilitators about their satisfaction with the programme), adaptations made to the programme and cost. These data were collected by electronic surveys of facilitators following each teaching kitchen and included eighteen items (online Appendix 2). Fidelity to the curriculum was directly observed by study staff on 40 % of teaching kitchens and used a structured fidelity guide (online Appendix 3). This guide included nine domains, which were scored on a 3-point scale (0 = not delivered, 1 = partially delivered, 2 = delivered as intended). Total fidelity was calculated as an average of the nine items, with a maximum possible score of 18. Adaptations to the study curriculum were evaluated using a structured categorisation process and recorded by facilitators as a part of a REDCap⁽¹⁴⁾ electronic survey following each session.⁽¹⁵⁾

Child surveys were collected on paper and then entered into a secure REDCap database. Facilitator surveys were sent electronically, with information stored in REDCap.⁽¹⁴⁾

Data analysis

Univariate statistics were used to describe outcomes at baseline and follow-up. Equality of pre and post means was tested using two-sample paired t tests with significance defined using a two-sided α of 0·05. While the pragmatic nature of the current study precluded the use of a randomised control trial design, evaluation of the follow-up outcomes made use of information about intervention adherence at the individual level by employing a dose–response analysis. This dose–response approach necessitated the inclusion of all children with complete data in the primary analyses, regardless of the number of intervention sessions attended (i.e., children who attended zero sessions were also analysed). Effectiveness was determined by the significance of the relationships between dose received (number of sessions attended from zero to eight) and each of the five outcomes at follow-up, using ordinary least-squares multivariable linear regression models adjusting for baseline outcome score, child age (years) and gender (male or female). Analysing dose as a continuous variable allowed for the interpretation of significant results in one-unit additive increments from zero to eight. To address the question of whether there were group differences between the relatively high percentage of children who attended zero sessions and those who attended one or more sessions, baseline and follow-up outcome scores were also summarised by dichotomised dose, and secondary analyses were conducted to evaluate the association between the dichotomised dose variable and each of the child outcomes, adjusting for the same set of covariates as the primary analyses.

Because of the presence of non-normally distributed residuals, a robust se methodology was employed to allow for appropriate inferences from the obtained CI. Regression coefficients with robust 95 % CI and P-values are presented. To be included in the analysis, children were required to have complete data at baseline and follow-up. We conducted attrition analyses utilising χ ² and two-sample, independent t tests comparing age, gender and baseline outcomes between children with only baseline data and those with both baseline and follow-up data. Statistical analyses were conducted using Stata version 15.1 (StataCorp).⁽¹⁶⁾

Results

Of the children enrolled in the after-school programming, 369 met criteria for analysis, with both pre-and post-surveys complete. Among those 369 children, the mean (sd) age was 8·8 (1·9) years and 53·7 % identified as girls. Baseline mean (sd) values for each of the outcomes are presented overall and stratified by participant age and gender in Table 1. Attrition analyses comparing children with only baseline data (n 327) to children with both baseline and follow-up data were unable to detect significant differences in age, gender or baseline outcomes for FVP, ITE and WTT. Mean baseline self-efficacy and cooking attitudes were higher among those with full data (19·1 v. 17·3; P = 0·007, and 19·8 v. 19·0; P = 0·04, respectively).

Table 1 Baseline values of child-reported outcomes, overall and stratified by child age and gender

At follow-up, children had significantly higher mean scores on all five outcome measures. For self-efficacy, the mean increased by 14·2 (P < 0·001); for cooking attitudes, the mean increased by 6·1 (P < 0·001); for FVP, the mean increased by 4·2 (P < 0·001); for ITE, the mean increased by 1·6 (P < 0·001); for WTT, the mean increased by 2·3 (P < 0·001). Box plots (Fig. 2) indicate the observed distribution of each scale at baseline and follow-up timepoints for all eligible children (n 369).

Fig. 2 Observed survey scale scores at baseline and follow-up. Y-axes represent the possible range for each of the scales shown. P-values are from two-sampled paired t tests on the equality of means. , baseline; , follow-up

Multivariable analysis

Results from linear regression models that included baseline outcome score, age and dose are summarised in Table 2 and indicated the following: girls had significantly higher follow-up cooking self-efficacy scores (B = 2·27; 95 % CI 1·00, 3·55; P = 0·001) and cooking attitudes (B = 2·18; 95 % CI 1·31, 3·05; P < 0·001) scores compared with boys. Younger children had significantly higher follow-up ITE (B = –0·12; 95 % CI –0·22, –0·02; P = 0·02) and WTT (–0·18; 95 % CI –0·29, –0·07; P = 0·002) scores. When treating attendance as a continuous independent variable (zero to eight sessions), increased attendance at sessions was associated with higher follow-up cooking self-efficacy (B = 0·42, 95 % CI 0·22, 0·63; P < 0·001), cooking attitudes (B = 0·30, 95 % CI 0·17, 0·43; P < 0·001) and intention to try new fruits and vegetables (B = 0·12, 95 % CI 0·05, 0·18; P = 0·001). Results were similar when treating attendance as a dichotomous independent variable (zero v. one or more), indicating that children who attended one or more sessions had higher scores on the cooking self-efficacy and ITE, controlling for baseline score, child age and child gender (online Appendix 4).

Table 2 Multivariable linear regression models for each of the five cooking outcomes. Here, we show results of five separate multivariable regression outcomes. The outcome for each is the scale score for the corresponding outcome at the follow-up timepoint. The independent variables include the baseline score for each outcome, child age, child gender and attendance at sessions (continuous variable with possible range of 0–8). This indicates that children who attended more sessions had higher scores on the cooking self-efficacy, cooking attitudes and intention to eat fruits and vegetables, controlling for baseline score, child age and child gender

Organisational outcomes

Of the targeted twenty-five Parks and Recreation centres, twenty-one successfully implemented at least one teaching kitchen during the study period. Four centres were unable to implement the programme as one centre did not have an after-school programme and the other three centres were short-staffed and unable to commit a staff member to training and implementation of the programme. Mean (sd) participant attendance was 2·7 (2·7) sessions, with 134 children (36·3 %) not attending any sessions and 135 children (36·6 %) attending at least four sessions (Table 3). Anecdotally, common reasons for children not attending sessions included that some centres decided to focus their programming on specific age groups because of limited staff availability, and sometimes parents would pick up children before the teaching kitchen sessions began.

Table 3 Distribution of the number of experiential teaching kitchens attended by children

Box plots indicate the observed distribution of each scale at baseline and follow-up timepoints for all eligible children (n 369). The line in the middle of the box represents the median. The lines at the top and bottom of the box represent the 25th and 75th percentile, respectively. The top and bottom whiskers represent upper/lower adjacent value. Top dots represent outside values.

* Represents the number of sessions attended by children. For example, 134 (36.3 %) attended zero sessions, 45 (12.2 %) attended a single session and 21 (5.7 %) had perfect attendance by being present for all eight sessions offered.

Among the 40 % of sessions directly observed, average fidelity was 97·5 %. Of the 173 post-sessions electronic surveys filled out by facilitators, forty-one reported an adaptation to the curriculum. The most common adaptations included adding elements to the recipe (n 10), substituting elements to the recipe (n 9), tailoring (n 7) and removing elements from the recipe (n 6). Other, less common, changes included a more substantial drift from the intervention (n 2), loosening structure (n 2), lengthening/extending (n 1) and repeating elements (n 1). Modifications such as substituting or removing elements were most often made due to food allergies. Adding elements, such as adding chocolate chips to the banana oatmeal cookie recipe or substituting tortilla chips for celery, were done to ‘add more variety’. Examples of tailoring include boiling cauliflower on the stove top instead of microwaving it as instructed for the mashed cauliflower recipe.

The average cost of food per child was $1·37 per serving. Based on survey data from facilitators, the majority of sessions were rated as ‘Very satisfied’ (52·0 %) or ‘Satisfied’ (42·8 %), with only 4·0 % of sessions rated as ‘Dissatisfied’ and 1·1 % of sessions being reported as ‘Very dissatisfied’. Representative comments recorded in free-text fields on the survey included: ‘This teaching kitchen was extremely easy to prepare for and the kids had so much fun. Their favorite part was peeling and slicing the onions. Who knew crying would be that much fun!’ and ‘The kids were, at first, against trying it but after they tasted it, they all loved it’. When teaching kitchen facilitators reported that the children were dissatisfied, it was usually due to the type of food included in the recipe, either that the children were not familiar with the food or that the food was unavailable for the session (i.e., pomegranates had to be substituted with apples).

Discussion

We were able to successfully implement an experiential teaching kitchen in an urban, low-income after-school programme based in local Parks and Recreation centres, scaling up from two centres that had previously participated in a pilot to 21/25 of the city’s local Parks and Recreation centres. Consistent with previous trials,^(8–11) our evaluation found clinically and statistically significant improvements in cooking self-efficacy, cooking attitudes, FVP, ITE and WTT among children enrolled in the programme. In addition, we were able to train local staff to implement the teaching kitchen curriculum with high fidelity. Centres were able to implement the programming with a low cost of $1·37 per serving, which is within the typical budget of our local Parks and Recreation department. Taken together, this suggests that Parks and Recreation centre after-school programmes that often reach low-income, minority children may be a viable option for wide-scale dissemination and implementation of healthy lifestyle programming.

There were four main challenges to implementing the teaching kitchens: (1) some of the facilities were not suitable for cooking hot meals, limiting the ability of certain centres to complete all of the recipes; (2) timely food delivery was also a challenge for some centres, which limited the freshness of certain types of produce that could be included; (3) adaptations made by local staff were often for expediency and safety (i.e., food allergies), but in some cases reduced the nutritional value of the teaching kitchen recipe and (4) it was difficult to maintain consistent staffing throughout the programme, which was a challenge to fidelity. However, we found that having master–trainers was an important component of programme sustainability, indicating the importance of hiring at least some nutrition or culinary professionals for future intervention implementation.

The majority of previous studies in adults and children who have used a community-based approach to improve cooking skills have led to improvements in self-reported outcomes.^(17–20) Our results are consistent with these findings and advance the literature by both describing the process by which such a programme can be scaled-up to multiple centres and highlighting several areas for future research in experiential teaching kitchen design. First, there were statistically significant sub-group differences on the follow-up outcomes. Results suggest that girls had higher follow-up cooking self-efficacy (2·27 points) and cooking attitudes (2·18 points) than boys, after adjusting for baseline score, age and dose. This differs from previous findings and suggests additional exploration into gender disparities in early dietary practices.⁽⁹⁾ Results also suggest that younger children had higher follow-up ITE and WTT, highlighting the importance of early intervention. This is a novel observation enabled by the current study’s broad age range (6–14 years), in contrast to previous studies with more narrow age ranges.^(17–20)

When scaling up a community-based nutritional intervention, there are at least two theoretical domains that need to be considered, including (1) implementation fidelity and (2) course corrections during implementation.⁽²¹⁾ The current study included a systematic assessment of programme fidelity, which was consistently high (97·5 %). In addition, the use of a systematic approach to tracking and assessing the impact of adaptations to the study protocol provided a nuanced understanding of the implementation outcomes. Each of the twenty-one centres that implemented the teaching kitchen had unique contextual factors that resulted in adaptations to the programme. For example, many of the teaching kitchen sessions occurred in centres without physical kitchen space but rather used cooking utensils in a multi-purpose room. This indicates that a wide variety of settings may be appropriate for this type of programming. In addition, by creating an electronic survey that facilitators used to record those adaptations, we were able to (1) identify threats to study validity and either provide support to overcoming barriers or re-train facilitators in real-time, (2) identify barriers that were common to multiple sites and provide system-wide support and tailoring of the intervention content to address those barriers and (3) provide regular progress reports to the Parks and Recreation leadership team about quality of programme implementation. This is important, as we occasionally observed facilitators making unhealthy substitutions (e.g., tortilla chips for celery), which would have had a negative outcome on the healthfulness of the programme. Because we monitored these adaptations in real-time, we were able to take corrective action quickly.

It should be noted that of the 369 children included in the analysis, 134 (36·3%) did not attend any of the teaching kitchen sessions, reflecting the pragmatic real-world setting of after-school interventions. Because the children who did not attend any sessions were included in the main analyses (i.e., simple equality of pre and post means tests as shown in Fig. 2), the observed statistically significant improvements in child outcomes could be an under-representation of the potential benefit of participating in experiential teaching kitchens. The dose analysis supports this possibility, indicating that children who attended more sessions demonstrated the greatest benefit. For example, the cooking self-efficacy attendance regression coefficient in Table 2 indicates that each additional session attended by a child was associated with a 0·42-point increase on the cooking self-efficacy scale, even after adjusting for potential differences in baseline self-efficacy, age and gender. The results from the dose analysis strengthen the conclusions about the effectiveness of these teaching kitchens and point to areas of future research in implementation science aimed at improving accessibility of interventions to maximise increased dose receipt. In addition, our data suggest that < 100 % attendance should not preclude the implementation of an experiential teaching kitchen, as meaningful improvements can occur even if children do not attend every session.

The utility of both the dose analysis and the adaptation measurement tool in the current study suggests that future implementation projects can benefit from recognising that real-time assessments of adoption and adaptation measures can be used to give timely support and feedback to maximise programme quality.

Strengths and limitations

Study strengths included implementing the programme in a non-school setting under typical field conditions, such as child absences, staff turnover and real-time adaptations. In addition, survey instruments were based on previously tested, reliable and valid scales. Also, unlike previous skills-based teaching kitchens, the current study included a broad range of child ages, which may increase the ability to generalise to other age groups not previously studied.

The current study had several limitations. First, as a programme evaluation primarily designed to address the implementation outcomes of teaching kitchens in community settings, the study was not designed to confirm the effectiveness of previously published trials. Consequently, the pre–post design did not have a control group and necessarily does not allow for a causal inference of effectiveness. In addition, dose was not randomised, and dose analysis results should be interpreted with caution and without inferring causality. This type of programme evaluation is in line with the concept of a ‘plausibility’ assessment, whereby researchers attempt to exclude alternative potential causes of the observed effects.⁽²²⁾ The large outcome improvements and positive dose–outcome relationships observed in the current study align well with the effectiveness demonstrated in previously published trials and support the hypothesis that the intervention may have resulted in meaningful changes for this predominantly low-income population. A second limitation was that several of the outcomes were based on child report, and, as such, they were subject to social-desirability bias. However, this might be expected to have also biased the baseline reports, suggesting that the absolute differences may still be meaningful. Third, detailed demographic data at the individual level were not available (e.g., ethnicity, household income, food security, participation in the Special Supplemental Nutrition Program for Women, Infants and Children (WIC), etc.). However, with the broad inclusion criteria there were few limitations on those children who could participate, leading to a sample that was generally representative of the children enrolled in local Parks and Recreation after-school programming, which in Nashville is mostly low-income, minority children. Fourth, a relatively large number of children did not have sufficient follow-up data for analysis, and there were some baseline outcome differences between children with missing and children with complete data. However, it is not clear what effect this might have had on the results, if any. Additionally, attrition is not unusual among participants enrolled in Parks and Recreation after-school programming.⁽²³⁾

Conclusions

Implementing and scaling-up this experiential teaching kitchen in Parks and Recreation centres indicates that this type of nutritional education and skills building programme can be conducted and spread in a community setting with high fidelity, organisational alignment and without diluting effectiveness, while expanding reach to low-income, minority children.