Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-19T02:59:11.506Z Has data issue: false hasContentIssue false

Dietary supplements, quality scores and missing data in the review of validation studies

Published online by Cambridge University Press:  02 August 2010

Beatrice A. Boucher*
Affiliation:
Population Studies and Surveillance, Cancer Care Ontario, Toronto, ON, Canada Department of Nutritional Sciences, University of Toronto, Toronto, ON, Canada Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada, email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Type
Letter to the editor
Copyright
Copyright © The Author 2010

As stated in the foreword(Reference Willett1) to articles by Henriquez-Sanchez et al. (Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso2) and others, this collection presents valuable work that identifies quality features of dietary validation studies and supports the use of FFQ in future research. The authors' substantial effort and results are appreciated; however, inconsistencies in classifying studies as including supplements, assigning quality scores and presenting complete validity data may have limited their findings. The paper by Henriquez-Sanchez et al. (Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso2) will be primarily examined, but the comments may also apply to others in the collection.

Henriquez-Sanchez et al. (Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso2) report that 23–31 % of validation studies included dietary supplements in their analyses, but there were no large differences in correlation coefficients between studies with or without dietary supplements. This finding is surprising, given reports that the inclusion of supplements improves intake assessment and validity correlations(Reference Boucher, Cotterchio and Kreiger3, Reference Messerer, Johansson and Wolk4). Although Henriquez-Sanchez et al. observed that correlations improve when supplement data from both FFQ and reference methods are included, the inconsistent use of this criterion when reviewing the studies may account for the results just mentioned. Individual studies may describe FFQ as listing supplements, but it cannot be assumed that these data were collected in the reference methods or included in the validity analyses, unless explicitly stated by the original authors. Further, it is erroneous to report a validation study as including supplements if these data were not specifically collected in both dietary methods and also used in the analysis. Four studies in the review of Henriquez-Sanchez et al. serve to illustrate this point. Flagg et al. (Reference Flagg, Coates and Calle5) describe their FFQ as including supplements, but do not specify collecting these data in recalls, and even state that they were excluded from validity correlations. Similarly, Jain et al. (Reference Jain, Howe and Rohan6) describe both the FFQ and records as including supplements, but state that these were not included in their analysis. The study that was assigned the highest quality score(Reference Sudha, Radhika and Sathya7) describes its FFQ as including supplements, but does not explicitly state that these were included in the recalls or analysis. Although none of the three studies describes supplements in their analyses, all were evaluated by Henriquez-Sanchez et al. as including supplements. In contrast, another study reports including supplements in the FFQ, recalls and validity analysis(Reference Boucher, Cotterchio and Kreiger3), yet was classified as excluding these data.

Inconsistent assignment of study quality scores by Henriquez-Sanchez et al. and by the research group evaluating minerals(Reference Serra-Majem, Pfrimer and Doreste-Alonso8) was also evident, despite having similar group members, study objectives and the same evaluation criteria. When the list of studies using recalls or records for vitamins A, C, D and E(Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso2) was compared to those for minerals(Reference Serra-Majem, Pfrimer and Doreste-Alonso8), eighteen of the same studies were differently scored by the two research groups, many by at least one point(Reference Riboli, Elmstahl and Saracci9Reference Fornes, Stringhini and Elias11). Although seemingly small, these differences were large enough to shift most studies into the next category of quality, described by Serra-Majem et al. (Reference Serra-Majem, Andersen and Henriquez-Sanchez12). A quality score to weight study findings seems appropriate(Reference Willett1), but to be fully useful, guidelines for its derivation must be clear and applied consistently and with equal rigour across studies. Shortcomings in this process might explain the inconsistent scoring between groups.

Finally, the introductory paper in this collection reports that the aim was to review all published validation studies(Reference Serra-Majem, Andersen and Henriquez-Sanchez12). However ambitious, some validation studies(Reference Subar, Thompson and Kipnis13Reference Wirfalt, Jeffery and Elmer16) seem to have been missed or excluded from the reviews(Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso2, Reference Serra-Majem, Pfrimer and Doreste-Alonso8). There is also incomplete reporting of nutrients from included studies. Relevant nutrients were sometimes not reported, or were included only in one review, when both vitamins and minerals were listed in the original studies. Examples include Osowski et al. (Reference Osowski, Beare and Specker17) (included in mineral review only, but original study also reported vitamin D) and Boucher et al. (Reference Boucher, Cotterchio and Kreiger3) (included for vitamin C, but original study also reported vitamins A, D and E; not included in mineral review, but original study reported Fe, Ca, Se and Zn). Although data in the reviews were adequate for certain evaluations, missing data from excluded studies or incomplete reporting would have strengthened the findings overall and might have meaningfully contributed to those areas where the data are limited. Validity data were reported as being particularly sparse for vitamins B6, B12, niacin and vitamin D(Reference Henriquez-Sanchez, Sanchez-Villegas and Doreste-Alonso2), as well as for Se, Zn and iodine(Reference Serra-Majem, Pfrimer and Doreste-Alonso8), in addition to the general lack of studies including supplements. Most studies just cited as missing or incompletely reported, included data for at least one of these limited nutrients – two reported supplements in their analyses(Reference Boucher, Cotterchio and Kreiger3, Reference Mares-Perlman, Klein and Klein15).

In conclusion, although examples of inconsistencies in the article by Henriquez-Sanchez et al. have been identified, the full extent of these problems and their effects on quality scores, study weighting and findings cannot be determined without a more complete review of all studies. This may be a worthwhile exercise for the authors to consider undertaking given the important impact of their work on future research, including guiding decisions about supplement data collection, analysis and reporting in validation studies.

The author has no conflict of interest. There was no funding associated with this letter.

References

1Willett, W (2009) Foreword. Br J Nutr 102, S1S2.CrossRefGoogle ScholarPubMed
2Henriquez-Sanchez, P, Sanchez-Villegas, A, Doreste-Alonso, J, et al. (2009) Dietary assessment methods for micronutrient intake: a systematic review on vitamins. Br J Nutr 102, S10S37.CrossRefGoogle ScholarPubMed
3Boucher, B, Cotterchio, M, Kreiger, N, et al. (2006) Validity and reliability of the Block98 food-frequency questionnaire in a sample of Canadian women. Public Health Nutr 9, 8493.CrossRefGoogle Scholar
4Messerer, M, Johansson, SE & Wolk, A (2004) The validity of questionnaire-based micronutrient intake estimates is increased by including dietary supplement use in Swedish men. J Nutr 134, 18001805.CrossRefGoogle ScholarPubMed
5Flagg, EW, Coates, RJ, Calle, EE, et al. (2000) Validation of the American Cancer Prevention Study II Nutrition Survey cohort food frequency questionnaire. Epidemiology 11, 462468.CrossRefGoogle ScholarPubMed
6Jain, M, Howe, GR & Rohan, T (1996) Dietary assessment in epidemiology: comparison of a food frequency and a diet history questionnaire with a 7-day food record. Am J Epidemiol 143, 953960.CrossRefGoogle Scholar
7Sudha, V, Radhika, G, Sathya, M, et al. (2006) Reproducibility and validity of an interviewer-administered semi-quantitative food frequency questionnaire to assess dietary intake of urban adults in southern India. Int J Food Sci Nutr 57, 481493.CrossRefGoogle ScholarPubMed
8Serra-Majem, L, Pfrimer, K, Doreste-Alonso, J, et al. (2009) Dietary assessment methods for intakes of iron, calcium, selenium, zinc and iodine. Br J Nutr 102, S38S55.CrossRefGoogle ScholarPubMed
9Riboli, E, Elmstahl, S, Saracci, R, et al. (1997) The Malmo Food Study: validity of two dietary assessment methods for measuring nutrient intake. Int J Epidemiol 26, S161S173.CrossRefGoogle ScholarPubMed
10Kristal, AR, Feng, Z, Coates, RJ, et al. (1997) Associations of race/ethnicity, education and dietary intervention with the validity and reliability of a food frequency questionnaire. Am J Epidemiol 146, 856869.CrossRefGoogle ScholarPubMed
11Fornes, NS, Stringhini, ML & Elias, BM (2003) Reproducibility and validity of a food-frequency questionnaire for use among low-income Brazilian workers. Public Health Nutr 6, 821827.CrossRefGoogle ScholarPubMed
12Serra-Majem, L, Andersen, LF, Henriquez-Sanchez, P, et al. (2009) Evaluating the quality of dietary intake validation studies. Br J Nutr 102, S3S9.CrossRefGoogle ScholarPubMed
13Subar, AF, Thompson, FE, Kipnis, V, et al. (2001) Comparative validation of the Block, Willett, and the National Cancer Institute food frequency questionnaires. Am J Epidemiol 154, 10891099.CrossRefGoogle Scholar
14Jain, MG, Rohan, TE, Soskolne, CL, et al. (2003) Calibration of the dietary questionnaire for the Canadian Study of Diet, Lifestyle and Health cohort. Public Health Nutr 6, 7986.CrossRefGoogle ScholarPubMed
15Mares-Perlman, JA, Klein, BEK, Klein, R, et al. (1993) A diet history questionnaire ranks nutrient intakes in middle-aged and older men and women similarly to multiple food records. J Nutr 123, 489501.CrossRefGoogle ScholarPubMed
16Wirfalt, AKE, Jeffery, RW & Elmer, PJ (1998) Comparison of food frequency questionnaires: the reduced Block and Willett questionnaires differ in ranking on nutrient intakes. Am J Epidemiol 148, 11481156.CrossRefGoogle Scholar
17Osowski, JM, Beare, T & Specker, B (2007) Validation of a food frequency questionnaire for assessment of calcium and bone-related nutrient intake in rural populations. J Am Diet Assoc 107, 13491355.CrossRefGoogle ScholarPubMed