Comparison of methodological quality between the 2007 and 2019 Canadian dietary guidelines

Abstract

Objective:

With significant shifts in the dietary recommendations between the 2007 and 2019 Canadian dietary guidelines, such as promoting plant-based food intake, reducing highly processed food intake and advocating the practice of food skills, we compared their differences in guideline development methods.

Design:

Two reviewers used twenty-five guided criteria to appraise the methods used to develop the most recent dietary guidelines against those outlined in the 2014 WHO Handbook for Guideline Development.

Setting:

Canada.

Participants:

2007 and 2019 dietary guidelines.

Results:

We found that the 2019 guidelines were more evidence-based and met 80 % (20/25) of the WHO criteria. For example, systematic reviews and health organisation authoritative reports, but not industry reports, constituted the evidence base for the dietary recommendations. However, recommendations on food sustainability and food skill practice were driven primarily by stakeholders’ interests. By contrast, less information was recorded about the process used to develop the 2007 guidelines, resulting in 24 % (6/25) consistency with the WHO standards.

Conclusions:

Our analysis suggests that a more transparent and evidence-based approach is used to develop the 2019 Canadian dietary guidelines and that method criteria should support further incorporation of nutrition priorities (food sustainability and food skills) in future dietary guideline development.

National food-based dietary guidelines are the foundation of food and nutrition policies. They guide the public in making food choices to maintain well-being and prevent diet-related non-communicable diseases. Dietary guidelines are updated periodically to reflect the current body of evidence as well as changes in priorities for public health nutrition⁽¹⁾. In 2019, Health Canada published its updated food-based dietary guidelines⁽²⁾. Compared with the 2007 version, the significant changes include increased plant-based food intake, limiting highly processed food and sugar-sweetened beverage intakes and encouraging food skill practice, such as home cooking and eating together. With these differences seen between two versions of the guidelines, we wanted to understand whether the methods and processes involved in developing the guidelines might also differ.

Earlier studies have suggested that the methods used to develop dietary guidelines vary across countries. For instance, a global review of national food-based dietary guidelines has reported inconsistencies across the guidelines in retrieving evidence, assessing the quality of evidence and synthesising the evidence when formulating the recommendations⁽³⁾. Additionally, many guidelines did not disclose conflicts of interest and funding sources⁽³⁾, raising a question of the food industry’s influence in dietary recommendations.

On a per-country basis, previous studies have compared the contents of dietary recommendations between versions of the dietary guidelines, such as in Canada (1992 v. 2007)⁽⁴⁾ and Brazil (2006 v. 2014)⁽⁵⁾. However, it remains unclear how methods used to update dietary guidelines evolve within a country. Therefore, the objective of this case study was to compare the differences in guideline development methods between the 2007 and 2019 Canadian dietary guidelines, aiming to inform potential areas for improvement in future dietary guideline development.

Methods

Document sources

All documentation for both of the Canadian guidelines is available online. This includes the dietary guidelines for 2007⁽⁶⁾ and the dietary guideline development history⁽⁷⁾, as well as the dietary guidelines for 2019⁽²⁾ and its accompanying evidence⁽⁸⁾, consultation^(9,10) and revision processes⁽¹¹⁾. We also referred to a published perspective by Bush et al.⁽⁴⁾ to extract relevant data for the 2007 guidelines. Additionally, we contacted Health Canada for assistance when there was uncertainty during the appraisal of the guideline methods.

Assessment of guideline methods

We adopted twenty-five criteria (Table 1) outlined in the 2014 WHO Handbook for Guideline Development⁽¹²⁾ to appraise the methods used to develop each version of the Canadian dietary guidelines. We identified relevant information and rated the methods used in each guideline against the description of each criterion as Yes, No or Unclear. If ‘Yes’ was rated, the verbatim text was extracted to substantiate the rating. ‘No’ was selected when there was a direct statement of a specific method that was not employed. In cases where no clear explanations were mentioned to substantiate the respective criterion, ‘Unclear’ was rated. We then calculated the proportion of those rated ‘Yes’ as being consistent with the WHO criteria for each guideline.

Table 1 Appraisal of methodological rigour of the Canada dietary guidelines against the 2014 WHO Handbook for Guideline Development

Extraction of recommendations

Additionally, we extracted the key recommendations to understand the methods linked with the respective version of the guidelines. Recommendations are not always clearly indicated. To support consistent extractions, we used the 2014 WHO Handbook⁽¹²⁾ to define recommendation, which ‘tells the intended end-user of the guideline what he or she can or should do in specific situations to achieve the best health outcomes possible, individually or collectively’. We also adopted a set of criteria⁽¹³⁾ to identify recommendations. These include ‘consistent semantic and formatting indicators’, ‘a summary section to facilitate identification of recommendations’, using ‘decidable and executable wording’, and ‘avoiding embedding recommendation text within long paragraphs’.

Data extraction

Two reviewers (Z.D. and C.M.K.) independently extracted the data and critically appraised the methods and processes used to develop the guidelines. Disagreements were resolved by consensus with L.B.

Results

Table 1 describes how methods were used to develop the 2007 and 2019 Canadian guidelines according to the WHO criteria. Data extraction in this regard is listed in online Supplemental Tables 1 and 2. Overall, the guideline development process appears to be more transparent in the 2019 guidelines than the 2007 guidelines. We found that the 2019 guidelines met 80 % (20/25) of the criteria, while the 2007 guidelines met 24 % (6/25). However, neither guidelines clearly described disclosure or management of conflicts of interest among the stakeholders involved in the guideline development.

A notable difference between the two guidelines is in the evidence used to substantiate the recommendations. The 2007 dietary recommendations only include a few authoritative reports, including the WHO/FAO joint report, US DRI and the 2005 US Dietary Guidelines Advisory Committee Report⁽⁴⁾. In contrast, the evidence underpinning the 2019 recommendations consists of systematic reviews and a summary of evidence from various health organisation authoritative reports. This evidence supports the recommendations on the influence of dietary patterns, saturated fats, processed meat, Na and ‘free sugars’ in overweight/obesity, metabolic risk and dental decay^(2,8) . However, for both guidelines, it was unclear whether a ‘risk of bias’ assessment (‘11. Evidence quality assessment’ in Table 1) was applied to rate the individual studies.

Furthermore, the 2019 guidelines met most of the criteria in the domain of ‘Recommendations development.’ It administered two phases of online public consultation after the draft of the recommendations^(9–11) and explicitly stated that industry commissioned reports were excluded from the evidence base to substantiate the dietary recommendations⁽²⁾. Interestingly, the recommendation of food skills in the 2019 guidelines (home cooking and eating together) was formulated primarily based on ‘broad stakeholder interest’, as there was insufficient consistent evidence⁽²⁾.

Online Supplemental Table 3 compares the main dietary recommendations between the guidelines. Both guidelines target those 2 years and older and include a brief recommendation on physical activity, although the 2007 version also includes specific recommendations for a particular age group. Furthermore, the 2019 version has shifted towards increased plant-based food intakes, more specific recommendations on limiting highly processed food and sugar-sweetened beverage consumption (e.g. ‘Replace sugary drinks with water.’), and greater emphasis on food skills and the cultural aspect of food practice. Additionally, no recommendations are made for food serving sizes, dairy intake, acceptable macronutrient distribution range and vitamin D supplement intake for adults over 50 years in the 2019 guidelines.

Discussion

Both the 2007 and 2019 Canadian dietary guidelines have emphasised the importance of healthy food consumption to prevent non-communicable disease in the Canadian context. It is encouraging to see that the 2019 guideline recommendations are primarily based on systematic reviews and have met 80 % of the 2014 WHO criteria for public health guideline development. Additionally, the exclusion of industry reports as a source of evidence for the 2019 guidelines suggests a stricter approach to reduce conflicts of interest.

Several shifts in the methods used in developing the 2019 guidelines may explain the differences in the dietary recommendations. For example, the 2019 guidelines used systematic review methods to synthesise evidence and formulate the recommendations based on consistent findings, including those pertaining to dietary fibre, wholegrains, meat and meat alternatives in association with cardiometabolic risk⁽⁸⁾, as well as those pertaining to dietary unsaturated fats^(14,15) , highly processed food intake and sugar-sweetened beverages^(16,17) . By contrast, only three documents were cited in the 2007 guidelines to support the recommendations⁽⁴⁾. Second, we recognise that the body of evidence and nutrition priorities may shift during different time points. For example, dietary impact on the planet sustainability is a new addition to the 2019 guidelines, drawing on emerging evidence from dietary changes in green-house gas emissions and land and water use in recent years⁽¹⁸⁾. Also, even though the impact of improving food skills on human health was drawn on systematic reviews of observational studies⁽¹⁹⁾, they are ‘highly relevant’⁽²⁾ to public health nutrition, as stated in the 2019 guidelines.

However, the removal of the previous 2007 dairy food group in the new guidelines remains unclear, even though osteoporosis was one of the health outcomes included in the evidence review for the 2019 guidelines⁽⁸⁾. From the literature, we have seen consistent findings in the effect of dairy products on bone growth in children^(20,21) and on lowering the risk of osteoporosis and fractures in older adults⁽²²⁾. Future work is needed to understand the impact of the removal of dairy recommendations on bone health across different age groups.

From the methodological standpoint, we believe that it is appropriate to use the criteria outlined in the 2014 WHO Handbook to appraise both the 2007 and 2019 Canadian guidelines. As a newer guideline, we acknowledge that the 2019 guidelines describe the most recent methods used to develop the dietary guidelines, but think that it is important to determine whether the methods used to develop both guidelines have remained static or changed over the years before drawing conclusions about future improvements. On another note, guideline development standards, such as using systematic review methods and editorial independence, have been evolving since 2003⁽²³⁾. Because less information was recorded about the process used to develop the Canadian dietary guidelines before and during 2007⁽⁷⁾ than the 2019 guidelines, over three-quarters of the items were marked ‘Unclear’ for the criteria in the 2007 guidelines. This suggests that lower transparency of the guideline development process could be an issue for the 2007 guidelines. Consequently, we were unable to rule out whether these processes marked as ‘Unclear’ were a case of not being mentioned or of the methods not being implemented.

We note that various organisations, such as the FAO/WHO⁽²⁴⁾ and the European Food Safety Authority⁽²⁵⁾, have proposed methods to develop food-based dietary guidelines, using a scientific process to establish the diet–disease relationship and formulate the dietary recommendations. Because our study focused on evaluating the rigour of guideline development methods, we chose to use the 2014 WHO standards, an internationally recognised guideline development method for developing clinical and public health guidelines. Such standards cover compressively from the representation of guideline development group to peer review process, with emphasis on systematic review methods to search evidence (e.g. methods for assessing quality and analysis), transparency of evidence used (e.g. type of evidence and use of evidence tables), equity, feasibility and rating of the importance of outcomes⁽¹²⁾. These criteria ensure credible guideline recommendations that accurately reflect the balance of potential benefits and harms, and they are similar to those listed in The Appraisal of Guidelines for Research and Evaluation II related to guideline development methods, but not reporting⁽²⁶⁾.

However, methods in the 2014 WHO Handbook⁽¹²⁾ are not tailored specifically for formulating nutritional guidelines and recommendations. Therefore, using these standards to rate dietary guideline methods may be viewed as a limitation. Further, the 2014 WHO standards seem to lack adequate methods to evaluate evidence relevant to the shifts in nutrition priorities, such as food sustainability and food practice, including eating together and home cooking^(18,19) . Because current evidence-based methods primarily rely on clinical trial findings⁽²⁷⁾, we propose using approaches such as the logical maps to guide research questions in dietary guideline development. This may ensure evidence collection and evaluation are driven by the research questions rather than by the available methods, such as the hierarchy of evidence⁽²⁷⁾. Therefore, future methods in developing dietary guidelines could consider allowing high-quality observational studies to address emerging food and nutrition priorities, which are not always suitable for randomised controlled trials.

Generalisability of the findings from two recent Canadian dietary guidelines may be limited in the North American context as a high-income Western country. However, such limitations may primarily affect the specific types of foods recommended rather than guideline development methods. Although two reviewers independently extracted and appraised the data, we cannot rule out the possibility of overlooking certain information during the process of rating the guidelines, even after consulting with the guideline development authority.

Conclusion

In summary, this case study suggests that the 2019 Canadian dietary guidelines have adopted greater transparency and a more evidence-based approach in formulating the recommendations. Our analysis also suggests new method criteria should support further incorporation of nutrition priorities such as food sustainability and food skills in future dietary guideline development.