The creation of supportive environments and the development of healthy public policy are central to supporting health-enhancing practices at the population level(1). The ultimate goal of healthy public policy action is to shift social norms and the physical environment so that they naturally reinforce health-enhancing practices while discouraging those practices that compromise health(Reference Wechsler, Devereaux and Davis2, Reference Lang and Rayner3). Such approaches are acknowledged as a means to address the complex factors involved in the development of chronic disease(Reference Brownson, Haire-Joshu and Luke4) and also for their value towards improving dietary patterns in the population(Reference Glanz and Mullis5).
In 2005, the Canadian coastal province of Nova Scotia (population: 921 727)(6) became a leader, both nationally and internationally, by launching the Healthy Eating Nova Scotia (HENS) strategy (Table 1). At the time, HENS was the first and only province-wide comprehensive healthy eating strategy in Canada(7) and viewed as a critical step towards addressing the high levels of chronic disease observed among Nova Scotians(7, 8). The HENS strategy is an evidence-based, population-health framework and key policy support to the provincial Chronic Disease Prevention Strategy(8), with a vision of improving the nutritional health of Nova Scotians through a call for coordinated action in four areas: breast-feeding; children and youth; fruit and vegetable consumption; and food security(7). The province's former Department of Health Promotion and Protection (NSHPP)Footnote † was recommended as the government lead agency for the strategy, although implementation was viewed as a collaborative and cross-sectoral responsibility, with oversight and visioning provided by multiple stakeholders, represented on the HENS Guiding Group (HENS GG).
Table 1 Synopsis of the Healthy Eating Nova Scotia (HENS) strategy and prioritized evaluation questions
In June 2010, we undertook a first-phase evaluation of the HENS strategy on behalf of the HENS GG and NSHPP. The goal of this evaluation was to learn what was working well in relation to the implementation of HENS, with findings forming the basis of recommendations made to the HENS GG in support of ongoing strategy implementation. The purpose of the present paper is to describe the process undertaken to evaluate the implementation of the HENS strategy and to highlight the key findings and lessons learned.
Methods
We conducted three evaluation activities in the autumn and winter of 2010/11, as follows:
1. A document review, designed to review processes used to support implementation of the HENS strategy and to provide context for the activities and resources put into place to enable implementation and evaluation. We designed the review to identify any higher-level formal policies or initiatives that were created, led by or leveraged as a result of HENS, in particular those that fell under the HENS priority action areas.
2. A survey of key HENS stakeholders, designed to elicit feedback and perspectives on the implementation of HENS from broader stakeholders interested in the healthy eating agenda in Nova Scotia.
3. A series of key informant interviews, designed to elicit perceptions and observations of HENS implementation by both past and present members of the HENS GG, as well as selected other individuals identified for having important historical insight into strategy development and implementation.
Each evaluation activity was designed to align with questions and indicators from the HENS evaluation framework that were prioritized by members of the HENS GG Research and Evaluation Subcommittee (Table 1). The study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human participants were approved by the Dalhousie University Health Science Research Ethics Board. Written informed consent was obtained from all participants.
The remainder of this section summarizes the key methods undertaken for each component of the evaluation.
Document review
The document review provided an objective way to categorize activities associated with the HENS strategy. We obtained documents for review primarily from representatives from NSHPP (n 156 documents, dated from years 2004 to 2010, including those just prior to strategy launch). The documents we reviewed were specific to the higher-level work of the HENS GG and its committees and not related to broader working group activities (e.g. policy development processes) across the priority areas. Examples of documents we reviewed included meeting minutes, email correspondence, media releases, presentations to stakeholders, environmental scans, and consultation reports. The lead author (S.M.S.) and a research assistant (G.H.) reviewed each document independently, and then discussed similarities and contradictions in relation to the prioritized evaluation questions and indicators from the HENS evaluation framework (Table 1). After consensus was reached, we organized the findings by document, indicators present within the document, and alignment with HENS strategic directions (Table 1).
Stakeholder survey
We developed an online ‘Opinio’ survey for distribution to HENS stakeholders (for more information on ‘Opinio’, visit http://objectplanet.com/opinio/). The survey was password-protected and hosted on a secure server by the Network and Systems Division of Dalhousie University. It comprised a total of fifty-eight questions including nominal, ordinal (‘Likert-scale’, ranking) and open-ended questions. We designed the survey to allow stakeholders to respond to questions based on their knowledge of the strategy, whereby stakeholders who self-identified as having ‘limited’ to ‘no’ familiarity with HENS were directed to some questions but not others, while those identifying having ‘some’, ‘good’ or ‘excellent’ familiarity with HENS were directed to additional questions to obtain greater depth into their perceptions. We focused on prioritized questions from the HENS evaluation framework (Table 1), specifically perceptions of strategy leadership and resourcing, and on identifying related policy or other locally led initiatives (Table 2).
Table 2 Summary of questions: stakeholder survey and key informant interviews
HENS, Healthy Eating Nova Scotia; NSHPP, Nova Scotia Department of Health Promotion and Protection; HENS GG, HENS Guiding Group.
*Those identifying as 1 or 2 were tagged as ‘ABC’ respondents; those identifying as 3, 4 or 5 were tagged as ‘XYZ’ respondents.
We used a combination of purposive and snowball sampling strategies to recruit stakeholders to participate. Approximately 220 individuals received a ‘request to participate’ email from the Healthy Eating Coordinator at NSHPP. These individuals represented a variety of stakeholder groups, including government, public health and community organizations. Invitees were also asked to forward the survey link to other individuals in their respective networks whom they deemed appropriate to respond, although we are unable to quantify the uptake of this request.
The survey was ‘live’ for 3 weeks in the autumn of 2010, during which time 120 people responded to the survey. Response rates for the survey questions varied according to the question being asked. Of 220 people directly invited to participate, 120 responded, a response rate of 55 % (purposive sample only). Within this overall response rate, seventy-one respondents self-identified as having ‘some’, ‘good’ or ‘excellent’ familiarity with the strategy and therefore answered an additional set of questions (32 % of purposive sample; 59 % of all respondents). Respondents were mostly from health-related backgrounds (e.g. dietetics, nursing, dental hygiene, lactation consultation, health promotion, physical activity), followed by educators, policy makers, agricultural and land-use representatives.
Key informant interviews
We conducted face-to-face or telephone interviews with past or present members of the HENS GG and other key informants (n 16, 84 % response rate) in the autumn and winter of 2010/11, with analysis occurring concurrently. Informants were mostly dietitians/nutritionists (n 12) and represented government, public health, non-government or community organizations. Remaining informants represented food and agriculture, early-childhood and public education sectors. The interview guide (Table 2) was based on prioritized questions from the HENS evaluation framework (Table 1) and both the guide and preamble were sent to all participants in advance of the interview. We followed a semi-structured interviewing format, which allowed for probing and the insertion of additional questions depending on the line of questioning and information provided by the informants. Interviews were digitally audio-taped with consent and lasted approximately 1·5 h. Interviews were transcribed verbatim and imported into NVIVO qualitative data analysis software version 8 (2008) to organize the data and facilitate coding into emerging concepts and major patterns. We organized emerging concepts and patterns into descriptive themes that were supported by informant narratives; collectively these were used to provide context for the impressions of HENS implementation across the province. We followed the principles of qualitative description in our methodology, in that it involved less interpretation and stayed ‘closer’ to the data(Reference Sandelowski9, Reference Neergaard, Olesen and Anderson10).
The research team collaborated regularly with members of the HENS GG Research and Evaluation Subcommittee throughout the evaluation. The intent of these meetings was to refine the data collection tools; discuss emerging findings and clarify points of uncertainty; provide further context or background; fill gaps in understanding; and provide an opportunity to reflect on the validity of findings. Feedback from the broader HENS GG membership was also incorporated.
Findings
The findings from the three evaluation activities were integrated to elucidate a comprehensive picture of strategy implementation, both objectively through the document review and subjectively through the stakeholder survey and the key informant interviews. Across the three evaluation activities, we identified the emergence of three common themes. These were: (i) leadership and momentum; (ii) role clarity; and (iii) achieving outcomes.
Leadership and momentum
Leadership and momentum formed the dominant theme across all three evaluation activities. Formal leadership for the strategy was supported by several groups: NSHPP; HENS GG; district health authority staff (led by public health nutritionists); and formal partnerships (e.g. working groups directly linked to HENS priority areas and identified actions). These leadership entities helped forge the momentum for strategy action to take place. Momentum for the strategy was supported primarily through the numerous direct and indirect contributions to strategy-related work and action-planning activities led by the HENS GG and working groups linked with HENS priority areas.
Both respondents and informants identified that the strategy was strengthened by having a provincial government department specifically mandated to promote population health. The leadership provided by NSHPP staff to HENS in the roles of funding, coordination, policy and resource development, and leveraging partnerships was highlighted. Similarly, informants representing community organizations discussed the uniqueness of having a respectful and collaborative working relationship with NSHPP and suggested that this contributed to the success of the strategy thus far:
What I've discovered is that [NSHPP] is one of the biggest advocates for this strategy. Can this really be? That hasn't played out in the way that other groups have played out. There is a respect that is communicated all the time. (Key informant)
However, some informants cautioned that the visible strengths of NSHPP risked other organizations not always taking full ownership in supporting the strategy or understanding their role in strategy-related action:
My biggest dream has always been – how do we ensure the sustainability of [HENS]? Because let's say if one day healthy eating was no longer a [government] priority? Then what? (Key informant)
The HENS GG was valued for reflecting the principle of shared ownership for the strategy and for continued inter-sectoral commitment:
The public will be more confident of HENS because it's not isolated inside a government department … we know that works. (Key informant)
Informants frequently noted that the Group had struggled with identity and a sense of ‘unique’ purpose over the evolution of the strategy, but they maintained it offered a ‘safe’ and ‘comfortable’ environment for dialogue and exchange, strategy visioning and communication. Respondents identified important roles for the HENS GG in networking and communication, also acknowledging these roles could be enhanced. However, other respondents acknowledged that they were unfamiliar with the work of this Group or how it provided leadership to the strategy. Informants suggested that greater diversity of membership could also be explored (e.g. cultural and sectoral), depending on the future needs of the strategy.
We found that partnerships played a key role in leadership and momentum for strategy-related activity. These partnerships ranged from formal (e.g. directly linked to HENS-related action) to informal and included academia; health associations, councils, coalitions and networks; food grower associations; provincial government departments and agencies; the school system; and early childhood and family resource centres. Both respondents and informants recognized that leadership could be enhanced through the ongoing development of partnerships, by not only strengthening relationships within the health system, but also broadening relationships outside the health and education areas of the provincial government and in under-represented groups (e.g. low-income, youth, mental health); diverse communities (e.g. African Nova Scotian); and through enhancing connections with the food industry and business community. Enhancing partnerships was one way of addressing a perceived challenge in strategy implementation – understanding the relevance of the strategy to organizational priorities and where organizations could play a role in supporting it. Enhanced partnerships would also help broaden the strategy so that it was not perceived to be a nutrition issue only:
Sometimes challenges arise when organizational priorities change and shift … when there is a lack of clarity around [HENS-related] roles, relationships, reporting and expectations. (Survey respondent)
Direct (financial) and in-kind contributions were important to build and sustain momentum for the strategy. Contributions ranged from additional funding for public health nutrition positions or initiatives (e.g. social marketing campaigns) to provision of space for holding meetings. We could not ascertain the dollar value of the many contributions from the documents reviewed; however, we identified that contributions came from a variety of sources that included provincial and federal government departments, district health authorities, and non-government and community organizations. These were echoed in comments made by both respondents and informants. NSHPP played a critical role in the disbursement or leveraging of these financial and in-kind resources, and non-government or community organizations were recognized and highly valued among respondents and informants for their mostly in-kind contributions to the strategy.
The findings also suggest that the strategy has evolved a ‘life of its own’ where strategy ownership, leadership and momentum are building outside the formalized leadership entities and their supports. The ‘life of its own’ concept encompasses those organizations, groups, actions and activities that were recognized to be HENS-related (or generally related to an increasingly favourable climate for the promotion of a healthy eating agenda) but may or may not have formally linked. This transition from formalized work to the strategy taking on a ‘life of its own’ was a welcome observation for informants who felt that it best reflected the population-health approach that was at the core of the strategy.
Role clarity
Role clarity refers to how the HENS mandate (a provincial strategy) is understood in the district health authorities and the role of public health nutrition in this context. We determined that increased human resource capacity (i.e. public health nutritionists dedicated to support HENS at the provincial, NSHPP department and district health authority levels) was a critical factor in strategy momentum and strategy successes to date. Informants remarked that a loss or dilution of these resources would negatively impact the strategy and its progress. Respondents also perceived that the addition of public health nutrition positions at the provincial and district health authority levels was important for strategy development and action. This is due largely to the mandate of public health nutritionists in Nova Scotia to support HENS-related work within the context of a population-health approach.
There is no provincial programme related to HENS and district health authorities operate with autonomy. As a result, informants described that a varied and district-specific approach was applied to HENS-related activity. This was identified as presenting challenges to how public health nutritionists ‘do the work of HENS’ (a provincial strategy) at the local level:
There needs to be that broader level of support. If the District Health Authority didn't think it was a good strategy then the nutritionists aren't even being supported to work on it … [activity] is not consistent and that's because of our system – how we're structured. (Key informant)
This perception was echoed in the survey, where comments provided by some respondents suggested divergent understandings of the role of public health nutrition in the districts and their effectiveness in these roles:
Understanding the role of public health nutritionists can bring and encourage focus on the true root causes of ill-health and poor nutrition. This would allow [them] to focus on advocacy and policy development. (Survey respondent)
All I know is that the nutritionists are not available to do nutrition work. The support they are able to ‘provide’ to the strategy is extremely broad and unable to reach down to the ground level. (Survey respondent)
Moreover, informants and respondents also perceived that district-level support for the HENS strategy affected access to funds provided to the districts by NSHPP to support HENS. Initially, HENS-designated financial resources were allocated to each district in support of the strategy but this shifted in fiscal year 2007/08 whereby the funding for HENS was incorporated into a broader chronic disease prevention funding envelope. Consequently, several informants and respondents suggested that this change negatively impacted strategy momentum in several districts because of a perceived ‘inability’ to access funds to support HENS activity.
Achieving outcomes
From the document review, we identified a number of positive outcomes related to the strategy, including partnerships (either enhanced or developed, specifically the activities of HENS-related working groups), resourcing the strategy (e.g. increased public health nutrition capacity, funding initiatives and/or activities) and policy. Several informants discussed how these outcomes had been achieved in a short period of time since the strategy launch, noting that population-level strategies take time to evolve and make an impact. This was echoed in comments made by several survey respondents:
Although there are limitations, the HENS strategy has stimulated more work and discussion around food-related policy in the last 5 years than in the 20 years prior to HENS coming into place. (Survey respondent)
Informants, and several survey respondents, regarded the HENS strategy document itself as successful merely by being the foundational piece unifying much of the formalized population-level nutrition work that occurred between 2005 and 2010. The strategy provided credibility and relevance to their work and helped leverage decisions in relation to population health.
Respondents identified that leveraging of partnerships (community-level partnerships were particularly embraced), resources (financial and human) and the policy work (provincial and local) to be representative of the success of HENS.
The active work on policy resulted in numerous visible successes – notably in the school and early childcare settings. Several informants and respondents recognized that work on breast-feeding, food security and in other settings (publically funded institutions and recreation for example) were augmented by having a HENS strategy to guide the work. Table 3 summarizes visible actions and activities (including policy) identified by respondents and informants.
Table 3 Actions and activities leveraged through the Healthy Eating Nova Scotia (HENS) strategy (2005–2010)
*Provincial policy.
Our review of the documents, combined with the insights of the respondents and informants, identified that momentum for policy development was greatest within the health and education sectors, and specifically within schools and institutions, but that work was still needed beyond these areas (Table 4). Several respondents also noted the importance of ongoing support for the policies throughout the development and implementation processes:
Policy enforcement on current strategies is as much of a priority as developing new ones. Let's place our eggs in a few baskets only and ensure we finish what we started before adding more to the strategy. (Survey respondent)
Table 4 Suggested areas for Healthy Eating Nova Scotia-related policy expansion or development
Both respondents and informants reflected on the strategy coinciding with a general cultural shift recognizing the importance of healthy eating and its value to population health. This reality created complexity for informants, resulting in them questioning whether achievements should be linked with HENS (‘does it matter?’) or discussing how HENS was able to take advantage of this shift and add value to the agenda for healthy eating as a point of advocacy.
Discussion
The HENS strategy is a pioneering strategy in Canadian population-level health policy that has built significant momentum in support of healthy eating in the province of Nova Scotia. The evaluation identified some of the key successes and challenges with broad strategy implementation that might be relevant to other initiatives of this type in Canada and elsewhere. Since its launch, the strategy has fostered significant investments in the areas of leadership, resourcing strategy-related activities and policy development. Along with the development and enhancement of inter-sectoral partnerships, these investments have resulted in numerous critical outputs at a provincial and local level, as well as help build momentum towards shared responsibility for the four priorities outlined in the strategy. Shared leadership was viewed as central to the success of the strategy, and the role of the provincial government in facilitating this was articulated across all three evaluation activities.
Having a provincial government department dedicated to population health and prevention work was instrumental in supporting the collaborative advancement of policy and other strategy-related action. Its leadership, however, needs to be recognized within further strategy implementation, requiring ongoing relationship building and role clarification so that the leadership is not perceived to be taken for granted, nor is the strategy considered ‘government heavy’. This is particularly critical given the current economic climate and fiscal restraint practices within governments, which may have repercussions for level of government engagement on a go-forward basis.
Role clarification is also critical to support broader strategy action by understanding the context of a provincial strategy at the local level. The current public health system structure contributes to how the HENS strategy is valued at the district level and how public health nutritionists are supported in their districts to do HENS-related work, including accessing funds. This in turn affects the perception of the effectiveness of these resources. Public health nutritionists have a unique set of skills that can be applied in meaningful ways towards population-level health action(Reference Fox, Chenhall and Traynor11). Enhancing district-level understanding of the HENS strategy and the role and scope of public health nutrition may bring greater cohesion and momentum to HENS actions and activities across the province.
The current evaluation has a number of strengths. First, to our knowledge, this process represents the first evaluation of its type, detailing the steps involved in evaluating the implementation of a population-level healthy eating strategy. Second, the three evaluation activities enabled both the process and the outcomes of strategy implementation to be captured. Third, the document review enabled us to identify objective evidence for policy development, resources and leadership that we subsequently validated through the stakeholder survey and interviews and through iterative engagement with members of the HENS GG.
There are some limitations to our approach including the recruitment of individuals who were already aware of the strategy for the survey and interviews. However, the document review enabled objective information on the policy work and other activities to be captured, thereby mitigating the risk of biased responses from participants. Finally, the evaluation was not able to ascertain the impact of the strategy on healthy practices at the population level, nor was it a goal for the evaluation. Future evaluations should consider exploring the contribution of this strategy (outputs and policy development) to population-level shifts in healthy eating patterns and enhanced food security.
Conclusions
The present evaluation of HENS has identified several critical factors that have enabled the strategy to build momentum for population-level action on healthy eating. Investing in inter-sectoral collaboration, partnerships and resourcing – both direct and in-kind – have been instrumental to providing leadership for the numerous outputs in support of the strategy. Clarifying roles related to public health nutrition in the context of the local application of a provincial strategy is also an important consideration for ongoing strategy implementation. These insights offer support for health policy stakeholders to guide development and implementation of other comprehensive, evidence-based, population-level healthy eating strategies.
Acknowledgements
Sources of funding: The evaluation was funded by a grant provided to S.F.L.K. from the Nova Scotia Department of Health and Wellness on behalf of the HENS GG. Conflicts of interest: S.F.L.K. is a member of the HENS GG Research and Evaluation Subcommittee, of which the Nova Scotia Department of Health and Wellness (study sponsor) is also a member. S.M.S. has worked with the Nova Scotia Department of Health and Wellness on the development of the social marketing campaign of vegetables and fruits – a provincial health promotion activity guided by the HENS strategy. Both S.M.S. and S.F.L.K. have professional connections to multiple HENS stakeholders and are themselves considered stakeholders of the strategy. S.M.S. and S.F.L.K. declined participation in the evaluation (stakeholder survey and/or key informant interview) to minimize the potential for bias. Authors’ contributions: S.M.S. was the lead researcher for this evaluation, including data collection, analysis and interpretation, and prepared the manuscript. S.F.L.K. provided intellectual support throughout the evaluation and reviewed and edited the manuscript. Acknowledgements: The authors thank the research assistants, Greg Hayward and Erin Hemmens, for their contributions; and members of the HENS GG Research and Evaluation Subcommittee for their review of the manuscript. They also thank the HENS GG and the Nova Scotia Department of Health and Wellness for supporting the evaluation.
