The internet is an important source of nutrition information for the public(Reference Adamski, Truby and Klassen1), however online content is largely unregulated(Reference Zhang, Sun and Xie2). Although previous research has considered the quality of scientific and nutrition reporting in newspapers(Reference Robinson, Coutinho and Bryden3,Reference Kininmonth, Jamil and Almatrouk4) , there is a paucity of literature examining online information. Moreover, there is a lack of validated assessment tools to assess the quality of online nutrition information. This work aimed to develop and validate a tool that could assess the quality of online information and discriminate between poor, satisfactory, and high-quality nutrition articles.
Using a previously validated tool for assessing health information in UK newspapers(Reference Robinson, Coutinho and Bryden3), the assessment criteria were adapted for online content. Based on the literature assessing online information in healthcare(Reference Zhang, Sun and Xie2), a quality assessment framework was defined as follows:
Criteria: rules by which online nutrition information quality is judged.
Indicators: attributes that indicate whether the webpage content meets the criteria.
Based on initial pilot testing with 19 webpages, 10 equally weighted quality indicators of three criteria (currency (date), credibility and readability) were established as the basis of a new online quality assessment tool (OQAT). From a minimum of zero, the maximum score achievable was 10.
Validation and reliability testing of the novel OQAT was executed by independent raters applying the OQAT to nutrition information shared over a 24-hour period on the microblogging platform Twitter.
Internal consistency was measured using Cronbach's alpha and interrater reliability was assessed utilising Cohen's kappa coefficient.
Over one 24-hour period (19 April 2021), 2,894 Uniform Resource Locators (URLs) were collected from Twitter posts that contained the word ‘nutrition’. After manual screening for relevance to human nutrition, these represented: 11 news articles, 220 blogs and 5 press releases. For validation of the OQAT, news articles (n = 11) were assessed by independent raters using both the OQAT and the existing tool developed for assessing newspaper articles(Reference Robinson, Coutinho and Bryden3). Comparison of scores using Cronbach's alpha found moderate internal consistency (a = 0.444).
To assess interrater reliability, n = 38 blogs, news and press releases were randomly selected using Microsoft Excel. Both raters used the OQAT independently to score articles using the agreed criteria and instructions. Discrepancies were identified, discussed and consensus reached. Cohen's coefficient demonstrated high interrater agreement after consensus (k = 0.92, p < 0.001).
The new OQAT for nutrition information was found to have moderate internal consistency and high interrater reliability. The moderate level of agreement against the newspaper-targeted tool evidenced the OQAT's new properties. Specifically, the OQAT better assesses the unique characteristics and wider range of online content (easy access to global content, can be written by anyone at any time, often discursive yet informative). In summary, this novel OQAT provides a simple method of quality assessment for researchers, as well those generating online content and members of the public accessing online information.
