Workplace nutrition knowledge questionnaire: psychometric validation and application

Abstract

Workplace dietary intervention studies in low- and middle-income countries using psychometrically sound measures are scarce. This study aimed to validate a nutrition knowledge questionnaire (NQ) and its utility in evaluating the changes in knowledge among participants of a Nutrition Education Program (NEP) conducted at the workplace. A NQ was tested for construct validity, internal consistency and discriminant validity. It was applied in a NEP conducted at six workplaces, in order to evaluate the effect of an interactive or a lecture-based education programme on nutrition knowledge. Four knowledge domains comprising twenty-three items were extracted in the final version of the NQ. Internal consistency of each domain was significant, with Kuder–Richardson formula values>0·60. These four domains presented a good fit in the confirmatory factor analysis. In the discriminant validity test, both the Expert and Lay groups scored>0·52, but the Expert group scores were significantly higher than those of the Lay group in all domains. When the NQ was applied in the NEP, the overall questionnaire scores increased significantly because of the NEP intervention, in both groups (P<0·001). However, the increase in NQ scores was significantly higher in the interactive group than in the lecture group, in the overall score (P=0·008) and in the healthy eating domain (P=0·009). The validated NQ is a short and useful tool to assess gain in nutrition knowledge among participants of NEP at the workplace. According to the NQ, an interactive nutrition education had a higher impact on nutrition knowledge than a lecture programme.

Chronic non-communicable diseases (NCD) have become one of the main public health problems worldwide, particularly in the developing countries^{( 1 )}. In Brazil, a middle-income country, the prevalence of obesity has rapidly grown over the past decades, with an estimated 50 % of the adult population being overweight^{( 2 )}. Locally, the adult population has a high prevalence of metabolic syndrome^{( 3 )}, and 43 % of workers participating in a Worker’s Meal Program were evaluated as being overweight^{( 4 )}. Among the dietary factors that have contributed to this trend are the steady rise in the daily intake of energy, animal proteins and fats, cholesterol and SFA, and a marked decrease in the intake of starchy roots, fruits and vegetables by the population^{( 2 )}. The workplace has been recognised as an important location for NCD prevention and health promotion of the economically active population and thus could be particularly appropriate for interventions.

According to the World Health Organization and the World Economic Forum (WHO/WEF) joint report^{( 5 )}, targeting unhealthy dietary habits can effectively improve NCD-related outcomes among adults in the working environment. Workplace Nutrition Education Programs (NEP) have the potential to improve workers’ knowledge about healthy eating^{( 6 )}. Recent systematic reviews have critically examined the effectiveness of dietary and physical activity interventions in the workplace on weight control^{( 7 – 9 )} and (less frequently) on dietary outcomes^{( 6 , 10 )}. In addition, the relationship between nutrition knowledge and dietary intake has been reviewed recently^{( 11 )} and revealed the relative paucity of good quality studies on this important topic. In fact, the WHO/WEF joint report^{( 5 )} drew attention to the lack of workplace dietary intervention studies in low- and middle-income countries and highlighted the need for simple and validated measures of physical activity and diet to be used in these settings.

Reports on the development of psychometrically validated instruments to assess nutrition knowledge were developed for the adult population in general^{( 12 – 14 )}, adolescents^{( 15 )} and other specific groups such as obese adults^{( 16 )}, consumers^{( 17 )}, university students^{( 18 )} and athletes^{( 19 )}. More recently, factors associated with nutrition knowledge of low-income caretakers^{( 20 , 21 )} living in high-income countries have been studied, but none of the studies have focused on the validation of a nutrition knowledge questionnaire (NQ). These observations underscore the need for further studies addressing methodological issues, such as study design and validated measures of nutrition knowledge^{( 6 )}, as part of intervention strategies aiming to improve dietary behaviour in the workplace.

Considering the alarming trend of NCD affecting people throughout the world^{( 1 )}, and the premise that worksite programmes hold potential for reducing workers’ risk of developing them^{( 6 )}, we conducted a NEP with overweight white-collar office workers. One of the goals of the NEP was to improve participants’ knowledge on healthy eating, using a psychometrically sound measure. This study aimed to validate a questionnaire assessing knowledge in nutrition and its utility in evaluating the changes in knowledge among participants of an NEP applied at the workplace.

Methods

Subjects

For the NQ validation, dietitians and last-semester undergraduate university students majoring in nutrition were invited to comprise the experts in nutrition group (Expert, n 88). Last-semester non-health course students comprised the lay group (Lay, n 48). Construct validity was evaluated in the group of participants of the NEP (n 165) at baseline.

The nutrition questionnaire and its validation

A panel of three dietitians and a psychologist generated the first version of the NQ, on the basis of the Food Guide for the Brazilian Population^{( 22 )} and existing validated instruments^{( 12 , 13 )}. The NQ was based on the curriculum of the NEP and focused on the relationship between eating habits and NCD, the benefits of fruit, vegetable and fibre intake, food sources of different fats, sugars and salt, healthy food choices and nutrition labelling. The first draft of the questionnaire was applied in a pilot study to a group of twenty-three overweight workers of a university-related organisation, with similar educational levels to those of the study population. At the panel’s discretion, items that had poor interpretability or wording and items that had inadequate degrees of difficulty (too easy or too difficult) were excluded. Items were considered too easy when more than 90 % of participants answered the item correctly and too difficult when 90 % or more answered incorrectly. In this version, seven questions were selected or adapted from items 1, 2, 3, 4, 7, 8 and 11 of Scagliusi’s questionnaire^{( 12 )}, which had validated the Portuguese version of the National Health Interview Survey on Cancer Epidemiology, a NQ applied to the US population. From the questionnaire published by Parmenter & Wardle^{( 13 )}, we adapted their question numbers 2, 4 and 20. Other items included were about typical eating habits of Brazilians (one question) and on Brazilian legislation on nutrition labelling (five questions). The resulting NQ contained forty-one items nested in twenty questions.

The NQ was evaluated for its construct validity using confirmatory factor analysis and discriminant validity. Exploratory factor analysis was used to determine the most appropriate number of factors (nutrition domains) and their respective items. The criterion to define the number of factors was the Kaiser Method (eigen values>1). Factor loadings >0·30 were used as criteria to retain the item in each factor^{( 23 )}. Confirmatory factor analysis was used to assess the factor validity. The root mean square error of approximation (RMSEA) and the χ ² test of minimum discrepancy^{( 24 )} evaluated the factor validity. The RMSEA ranges from 0 to 1, with smaller values indicating better model fit. A value of 0·06 or less is indicative of acceptable model fit^{( 25 )}. The Kuder–Richardson formula 20 (KR-20)^{( 26 )} was used to assess the reliability of each factor, and the results were considered significant when KR-20≥0·60^{( 23 )}. Normality of distribution was verified using the Kolmogorov–Smirnov test, and the discriminant validity was assessed using a one-way ANCOVA with group (Expert and Lay) as between-subject factor and age as covariate to compare the mean NQ scores obtained from the Expert and Lay groups.

Nutrition Education Program

The NEP was an education programme that enrolled office workers from six workplaces. The NQ was applied to the NEP participants before and at the end of the education programme in order to evaluate the gain in knowledge. The medical services of the workplaces were contacted, and those who agreed to the study protocol were selected to participate in the study. Each workplace medical service invited participants using an internal email system and folders. The six participating workplaces were then randomised to one of the two education programmes: the interactive programme (three workplaces) or lecture programme (three workplaces). The cluster randomisation method was used in order to avoid the interaction (contamination effect) between the two groups. The criteria for participation in the NEP were based on BMI (≥25 kg/m²), having completed high school and those who had at least three meals per week at the workplace cafeteria. Participation in a weight-loss programme (diet or medication) or in medical treatment that affected body weight were exclusion criteria. A total of 383 workers responded to the invitation, and after a personal interview 240 were selected according to the above criteria. Initially, a total of 127 workers from three workplaces participated in the interactive programme and 113 from three other workplaces attended the lecture programme. The interactive programme consisted of six interactive classes (60 min each, twice a week, within 2 months), whereas the lecture programme offered two lectures on healthy eating (90 min long), 1 month apart. Both programmes were delivered on site, at the six workplaces that entered the study. The curricula of both programmes were based on the Food Guide^{( 22 )}, and were developed and delivered by a group of trained dietitians and nutrition students. Only those participants who answered the NQ before and after the end of the programme were entered in the statistical analysis (interactive, n 94 and lecture, n 71).

To calculate the participants’ NQ scores, the item scores were summed, and the mean was calculated for each domain. The item scores ranged from 0 to 1. A two-way repeated-measures ANCOVA with programme (interactive and lecture) as between-subject factor, time (pre- and post-test) as within-subject factor and sex and age as covariates was used to compare mean outcome results between the interactive and lecture groups. Data were analysed using the free software R^{( 27 )}, and the confirmatory factor analysis was performed by the SEM package (an R package for structural equation modelling). All tests were performed considering bilateral alternative hypotheses and a level of significance of 5 % (P<0·050).

The human ethics committee of the Health Sciences Faculty from the University of Brasilia approved the research, and all subjects signed the informed consent.

Results

The study sample comprised 301 participants. Most of the participants were female (n 218, 72 %). Among the NEP participants, the mean age was 34 (sd 12) years, and the majority had completed college degree (58 %) (Table 1).

Table 1 Socio-demographic characteristics of the participants

* The Lay group included the non-health-course students.

† In the χ² test, the interactive group value was significantly different from the lecture group value (P<0·050).

‡ Two participants in the Expert group did not inform their age.

§ In the χ² test, the Expert group value was significantly different from the Lay group value (P<0·050).

In the construct validity analysis, initially, the factor structure of the NQ was examined by exploratory factor analysis considering baseline responses of all forty-one items. The exploratory factor analysis considered all participants (n 301) described in Table 1. According to the criteria set for this analysis, five domains were considered, which comprised all of the nutrition domains used in the questionnaire construction. However, in the discriminant validity test, the domain good dietary fats with two items did not differ between Expert (n 88) and Lay groups (n 48), because over 90 % of both groups scored correctly. Therefore, the domain was disregarded because of the lack of discrimination. By performing once again the exploratory factor analysis, twenty-three of the thirty-nine remaining items presented loading values >0·30 and were retained for further analyses. These twenty-three items, nested in nine questions, were distributed in four domains. On the basis of the retained items and factor loadings, new domain names were generated, as shown in Fig. 1. The final version of the NQ is presented in the online Supplementary Material. In the validated questionnaire, questions 1 and 2 were selected from Scagliusi et al.^{( 12 )}, whereas questions 3, 5 and 6 were adapted from the Parmenter & Wardle questionnaire^{( 13 )}.

Fig. 1 Retained items and nutrition domains generated after factor analysis. NCD, non-communicable diseases.

All domains presented good internal consistency, with KR-20 values>0·60, ranging from 0·61 to 0·84 (Table 2). As there were significant differences (P<0·050) in age between Expert and Lay groups (Table 1), the discriminant validity test considered age as a control covariate. Education was not considered as a control covariate, although it was also significant. This difference occurred because of the definition of the lay group that consisted of only last-semester students of non-health-related courses. In the discriminant validity test, both the Expert and Lay groups scored>0·52, but the Expert group scores were significantly greater than the Lay group in all domains: healthy eating (P<0·001), dietary salt (P<0·001), diet and NCD (P=0·033) and dietary trans-fats (P<0·001).

Table 2 Reliability scores and discriminant validity mean scores of Expert and Lay groups of the Nutrition Knowledge Questionnaire validation study (Mean values and standard deviations)

KR-20, Kuder–Richardson formula 20; NCD, non-communicable diseases.

One-way ANCOVA with group (Expert and Lay) as between-subject factor and age as covariate.

* The Lay group included the non-health-course students.

† P-value comparing the Expert group and the Lay group.

Factor validity was examined by confirmatory factor analysis considering post-test data of the remaining twenty-three items. The four domains presented a good fit in the confirmatory factor analysis (RMSEA<0·001 and χ ²=192·042, df=218, P=0·897). Considering the same analysis within each group, both interactive (RMSEA=0·010 and χ ²=226·913, df=225, P=0·452) and lecture (RMSEA<0·001 and χ ²=212·153, df=227, P=0·752) groups presented a good fit.

As there were significant (P<0·050) differences between the interactive group and the lecture group according to sex and age (Table 1), the comparison of NQ scores applied to the NEP participants considered both sex and age as control covariates. The results are presented in Table 3. In the within-group comparison, the overall questionnaire scores increased significantly because of NEP intervention, in both groups (P<0·001). In addition, a significant increase (P<0·050) in all four dietary knowledge domains was observed in the interactive group. However, in the lecture group, only the healthy eating (P<0·001), dietary salt (P=0·002) and dietary trans-fats (P=0·002) domains showed an increase in knowledge because of the intervention.

Table 3 Mean scores of pre- and post-tests of the nutrition knowledge questionnaire* (Mean values and standard deviations)

NCD, non-communicable diseases.

* Two-way repeated-measures ANCOVA with programme (interactive and lecture) as between-subject factor, time (pre- and post-test) as within-subject factor and sex and age as covariates.

† Individual contrasts in two-way repeated-measures ANOVA were performed to establish the significance of pre- and post-test score differences.

‡ Time×programme interaction tests of two-way repeated-measures ANCOVA were performed to establish the significance of difference between interactive and lecture groups.

The increase in NQ scores because of the type of intervention (between group comparison) was significantly higher in the interactive group than in the lecture group, and the difference was observed in the overall score (P=0·008) and the healthy eating domain (P=0·009). Although other significant differences were not observed because of the intervention type, we noted that the interactive group tended to score higher (P<0·200) than the lecture group in all other nutrition domains of the study.

Discussion

In this study, an NQ was psychometrically validated, and it was useful in the assessment of knowledge gain among participants of an NEP conducted at the workplace. According to the NQ, the interactive NEP had a higher impact on nutrition knowledge than the lecture programme. It is recognised that workers’ eating patterns are influenced by various factors in the working environment^{( 5 , 6 )}, in addition to cultural and social determinants that influence those patterns. Thus, successful workplace health promotion interventions should ideally be based on multicomponent methodologies and conceptual models that include informational, behavioural and environmental policy approaches^{( 6 , 28 )}. Furthermore, being knowledgeable about healthy eating appears to affect individual attitudes towards nutrition^{( 29 )}.

Studies have focused on the development of psychometrically validated instruments to assess the nutrition knowledge of adults^{( 12 , 13 , 15 – 17 )}. In the present study, a factor analysis was used to evaluate the theoretical construct that represented the underlying process of nutrition knowledge^{( 30 )} valid for a group of overweight office workers. The results suggest that 44 % of the original items had low discriminant or construct validity, and thus they were not useful to evaluate the nutrition concepts being taught in the NEP, highlighting the importance of conducting construct validation of these instruments.

The lowest KR-20 score was seen in the factor diet and non-communicable diseases. This domain had one question with spontaneous response about diseases related to eating habits (item 1 of the final NQ). In this question, the respondent had no repertoire of possible diseases to choose from, and after confirmatory factor analysis the final version of the NQ considered acceptable only two possible chronic diseases (diabetes and hypertension) among all possible answers. Parmanter & Wardle^{( 31 )} recommend the use of multiple-choice items with one correct answer, or two response options (true/false, yes/no, agree/disagree), in order to avoid the ambiguity that can accompany open answer questions. Despite having used an item based on a previously validated instrument, our questionnaire’s item on diet and non-communicable diseases may have been compromised by such ambiguity. Pasquali et al.^{( 32 )} note that the accuracy of the answers is a criterion that must be carefully considered in the design of items. Therefore, in the future, other alternative forms of questioning, rather than open-ended ones, should be considered and tested^{( 31 )}.

The discrimination power of the NQ is based on the idea that experts in nutrition tend to score higher than lay people^{( 12 , 13 )}. The significant differences between scores of the Expert and Lay groups suggest that the NQ had satisfactory construct validity^{( 13 )}.

Overall, the NEP had a positive effect in most nutrition knowledge domains evaluated, as indicated by the increase in the within-group mean scores of both interactive and lecture programmes. The between-group comparison indicating a significantly higher overall questionnaire score for the interactive compared with the lecture NEP suggests that a more elaborate education programme with active interaction between specialist and participants of the NEP has a better impact on improving participants’ knowledge than lectures. In addition, the scores obtained by the participants were within the recommended range of item difficulty^{( 33 )}. Thus, the validated NQ was useful in assessing the changes in knowledge among participants of an NEP conducted at the workplace.

Worksite dietary intervention research has been conducted in many places with differing methods. However, the use of psychometrically sound measures to evaluate nutrition knowledge is rare^{( 34 )}. Systematic reviews on the subject have concluded that worksite programmes are associated with improvement in dietary intake but evidence is limited^{( 10 , 34 )}.

Both knowledge in nutrition and eating behaviour are multi-dimensional and complex phenomena, and it is well recognised that nutrition knowledge plays a pivotal but only a partial role in people’s eating behaviour^{( 11 , 35 )}. Spronk et al.^{( 11 )} have nicely updated the information on the relationship between nutrition knowledge and food intake and highlighted the paucity of well-designed studies on the subject. Although nutrition knowledge has been evaluated in many countries, the comparison between them is hindered by the methodological heterogeneity of the studies^{( 36 )}. Many of them did not use a validated NQ for that specific population or used only self-reported food intake information^{( 36 )}. Accordingly, the use of validated measures, such as the one reported here, is part of a necessary effort to advance the quality of instruments used to assess the relationship between nutrition knowledge and eating behaviour and the effectiveness of workplace NEP, as emphasised by recent reviews^{( 5 , 6 , 10 , 28 )}.

Last, we recognise the limitation of the short intervention period of the study. Multicomponent strategies, with longer periods of follow-up, would be necessary to observe the impacts on the NEP participants’ eating behaviour and the effects on NCD. Additional limitations include using factor analysis of short-answer questions and the lack of a NQ reproducibility test before its use in the intervention. Still, we have succeeded in obtaining a short and useful NQ tool to assess nutrition knowledge among adults in the workplace. Further studies are needed to assess the impact of nutrition knowledge, as tested by this instrument, on the long-term workplace dietary behaviour of adults at risk for NCD.

Conclusions

The validated NQ is a short and useful tool to assess gain in nutrition knowledge among NEP participants in the workplace. The NQ tested here was a valid instrument to assess the knowledge in most of the domains concerning healthy eating among workers with high school or higher education levels. Further, interactive NEP had a better impact than the lectures on increasing nutrition knowledge among participants.