Electronic Dietary Intake Assessment (e-DIA): relative validity of a mobile phone application to measure intake of food groups

Abstract

Automation of dietary assessment can reduce limitations of established methodologies, by alleviating participant and researcher burden. Designed as a research tool, the electronic Dietary Intake Assessment (e-DIA) is a food record in mobile phone application format. The present study aimed to examine the relative validity of the e-DIA with the 24-h recall method to estimate intake of food groups. A sample of eighty university students aged 19–24 years recorded 5 d of e-DIA and 3 d of recall within this 5-d period. The three matching days of dietary data were used for analysis. Food intake data were disaggregated and apportioned to one of eight food groups. Median intakes of food groups were similar between the methods, and strong correlations were found (mean: 0·79, range: 0·69–0·88). Cross-classification by tertiles produced a high level of exact agreement (mean: 71 %, range: 65–75 %), and weighted κ values were moderate to good (range: 0·54–0·71). Although mean differences (e-DIA–recall) were small (range: –13 to 23 g), limits of agreement (LOA) were relatively large (e.g. for vegetables, mean difference: –4 g, LOA: –159 to 151 g). The Bland–Altman plots showed robust agreement, with minimum bias. This analysis supports the use of e-DIA as an alternative to the repeated 24-h recall method for ranking individuals’ food group intake.

Dietary assessment methods using mobile phone technology are becoming more widespread with the ubiquitous ownership of mobile phones and popularity of apps. A recent review found that satisfaction and preference for digital methods of collecting dietary data were higher than for conventional methods such as paper-based food records^{( 1 )}. Although these methods are promising, the usability testing and validity testing of such tools are necessary before application in dietary studies.

The electronic Dietary Intake Assessment (e-DIA) is a mobile phone app, designed to allow the digital recording of all foods and beverages consumed, either weighed or estimated. It is integrated with an Australian food composition database. Uniquely, no immediate nutrition information is provided to the user that may bias food selections while recording intake – an ideal feature for research purposes. In a recent comparison of the e-DIA against 24-h recalls, the e-DIA showed potential to measure energy, macronutrient and micronutrient intakes at a population level, but not individual level^{( 2 )}. Mean and median intakes, correlations and weighted κ showed good agreement of estimated energy and nutrient intakes between the methods, and Bland–Altman plots revealed no obvious bias. Nonetheless, in addition to energy and nutrients, it is important to assess the relative validity of reported food group intakes, given the recent shift in nutrition epidemiology from a nutrient focus towards food-based research to improve our understanding of the relationship between diet and health^{( 3 , 4 )}. A focus on nutrients tends to oversimplify this complex relationship by neglecting food synergy and food combinations^{( 3 )}. This study aimed to assess the relative validity of the e-DIA as a dietary assessment tool, using the 24-h recall as a comparison method, to measure intakes of food groups in a sample of university students.

Methods

Subjects and recruitment

Recruitment strategies and screening procedures used to obtain a sample of eighty-nine student volunteers from the university campus have been previously described^{( 2 )} and are summarised in Fig. 1. In brief, inclusion criteria specified that students should be aged between 19 and 24 years, enrolled in the Science or Engineering faculties and in their 2nd, 3rd or 4th year of study. Participation was incentivised by the option to enter a draw for a chance to win an Apple i-Pad mini following completion of the study. This study was conducted according to the guidelines laid down in the Declaration of Helsinki, and all procedures involving human subjects were approved by the university’s Human Research Ethics Committee (reference no. 2014/136). Written informed consent was obtained from all subjects.

Fig. 1 Flow chart of participant recruitment. e-DIA, Electronic Dietary Assessment.

Procedure

The study procedure and details of the e-DIA application have been previously described^{( 2 )}. In brief, participants were required to attend 30-min introductory sessions in small groups, where basic anthropometric measures were collected and training was delivered on the use of the e-DIA application. Study equipment was provided at this session, comprising digital kitchen scales (Salter 1066WHDR; Salter Australia) and metric cups and spoons, for use in combination with the e-DIA to facilitate accurate portion size estimation. A Food Model Booklet ^{( 5 )} was provided to perform a similar function during the recalls. Participants were instructed to complete 5 consecutive days of food records, including 3 weekdays and 2 weekend days using the e-DIA. Participants were sent a text message reminder before each collection day and were encouraged to maintain their usual diet. Starting days were staggered so that all days of the week were represented across the sample.

The e-DIA mobile web application runs on Android, iOS and supports most mobile Internet browsers. It can be downloaded by participants on their personal mobile phones or run from a mobile web browser. By deploying the e-DIA as a mobile web application, its compatibility and potential participation are maximised. The app features an electronic food diary, linked to the AUSNUT 2007 food and nutrient database^{( 6 )}, containing 4225 foods, beverages and supplements. An additional database was integrated that included all foods available for sale at university union food outlets. This database was developed as part of an ongoing study of the university food environment.

Users searched for and selected the closest match to foods and beverages consumed, enhanced by aspects of the application such as the ability to select meal and snack options, a drop-down list and a ‘favourites’ option (Fig. 2). If a food or close match was not available on the database, participants were requested to enter this manually into the e-DIA. Participants were advised to enter individual ingredients when entering mixed dishes. If this was not possible, participants were to select the closest possible generic match from the database or, failing this, to enter it as a new food item (custom foods). Weights or household measures of foods consumed were recorded, as either raw or cooked, as were place of purchase and consumption, and method of preparation. Although weighing of food was preferred, an option to quantify foods by household measures was included. Data were automatically uploaded to the administrator’s server at midnight and stored in a cloud-based database at the University of Sydney. The study investigators checked all entries the following day. If inconsistencies or issues in data entry were identified, participants were contacted to clarify these. Data recorded on the e-DIA were deleted at midnight every day, such that it could not be used as a prompt during the recall.

Fig. 2 Screenshot of electronic Dietary Assessment (e-DIA) application.

As a reference measure, three 24-h recalls were collected on randomly selected days (2 weekdays and 1 weekend day) during the 5-d study period to allow for unannounced recall interviews. Randomisation was achieved using the random number generator in Excel (Microsoft version 14.6.0, 2011). Recalls were conducted by phone interview with a trained researcher and lasted approximately 30 min. Participants were requested to use the Food Model Booklet, measuring cups and spoons to estimate portion sizes of meals and snacks. A standard script and recording form were utilised, modified from the US Agricultural Department Five-Step Multiple-Pass Method^{( 7 )} to reflect the Australian food supply. This method has been extensively validated^{( 8 – 10 )} and has been shown to produce energy estimates within 3 % of total energy expenditure measured with doubly labelled water. To enable appropriate validity testing of food groups, the same 3 d of data collection by both methods were used. Data from the recalls were manually entered by trained researchers into FoodWorks 7 Premium^{( 11 )}, a nutrient analysis software system linked to AUSNUT2007^{( 6 )}.

Food group coding

Food intake data collected from the e-DIA and recall methods were systematically disaggregated to enable apportioning to eight food groups comprising five core (fruit, vegetables, grains (cereals), meat and alternatives, dairy products and alternatives) and three discretionary food groups (discretionary foods, discretionary beverages (excluding alcoholic beverages) and alcoholic beverages)^{( 12 )}. The five core food groups are those recommended for consumption every day, whereas the discretionary foods are mostly energy-dense, nutrient-poor foods recommended to be consumed only sometimes and in small amounts. For more detailed food classification, refer to the online Supplementary Table S1. Foods were disaggregated using standard recipes from the AUSNUT2007 Recipe File ^{( 13 )} or, where necessary, a recipe was compiled from popular recipe web sites such as Taste.com.au. Intakes per day from each food group were summed and averaged over 3 d of recording for both e-DIA and 24-h recalls. All outliers were checked against original records and any discrepancies were assessed.

Statistical analysis

Owing to skewed intake data, non-parametric tests were used to compare differences (Wilcoxon’s signed-rank tests) and correlation coefficients (Spearman’s rank) of food group intakes between e-DIA and recalls. Data were coded into tertiles of intake to facilitate cross-classification analysis, reported as exact agreement and extreme disagreement. In addition, weighted κ statistics were computed using linear weights and interpreted using Altman’s reference values^{( 14 )}. To assess agreement, Bland–Altman plots^{( 15 )} were constructed with reference lines to indicate the mean difference between the methods (e-DIA–recall) and the 95 % limits of agreement (LOA, mean difference plus and minus 1·96 times the standard deviation). Assessments of the normality of mean differences per food group were undertaken as above, with the addition of one-sample t tests, in order to determine the necessity of applying log-transformations. The results of analyses presented in the main body of this article were conducted per consumer, defined as those who reported having consumed from the food group in either method during the study period. Data were analysed using SPSS (version 22.0, 213; IBM Corporation) and VassarStats (2015; Vassar College)^{( 16 )}, with the latter used solely to produce weighted κ statistics. P values<0·05 were considered to be statistically significant.

Results

The sample consisted of eighty students, 63 % female (n 50), with a mean age of 21·0 (sd 1·5) years. Only one participant did not consent to disclose her anthropometric data. Mean BMI was 23·0 (sd 3·8) kg/m², with thirteen participants (16 %) being overweight or obese; nine of the original eighty-nine persons recruited discontinued from the study citing time constraints as the reason.

Table 1 shows the median intakes from each food group, results of the Wilcoxon’s signed-rank test and the Spearman’s rank correlation coefficient calculated between methods. No significant differences were found in median intakes between the e-DIA and recall for any of the food groups. Spearman’s rank correlation coefficients (r) ranged from 0·69 (discretionary beverages) to 0·88 (discretionary foods and alcoholic beverages), with a mean of 0·79.

Table 1 Intake of food groups in the electronic Dietary Intake Assessment (e-DIA) and 24-h recalls, Wilcoxon’s signed-rank test (P) and Spearman’s rank correlations (r), for consumers only (n) (Medians and 25th and 75th percentiles (P25–P75))

* Wilcoxon’s signed-rank test.

† Spearman’s rank correlations were significant at P<0·001.

Table 2 shows the results of a cross-classification analysis and weighted κ values by tertile of intake from each food group, except for alcoholic beverages due to small numbers of consumers. The percentage of exact agreement, where participants were classified in the same tertile of intake, ranged from 65 % (vegetables) to 75 % (dairy products and alternatives and discretionary foods) with a mean of 71 %. Conversely, the percentage of extreme disagreement (classification into opposite tertiles) was the highest for discretionary beverages (8 %) and the lowest for dairy products and alternatives and discretionary foods (1 %), with a mean of 4 %. The strength of weighted κ values was moderate to good and ranged from 0·54 (vegetables) to 0·71 (dairy products and alternatives), with a mean of 0·63.

Table 2 Cross-classification by tertile of food group intake derived from the electronic Dietary Intake Assessment and 24-h recalls, consumers only (n) (Weighted κ statistics with their standard errors)

–, Insufficient consumers for analysis by tertiles.

* Exact agreement, per cent of cases cross-classified into the same tertile.

† Extreme disagreement, per cent of cases cross-classified into extreme tertiles.

Table 3 shows the mean difference in calculated intake from each food group between the e-DIA and recall methods, as well as the standard deviation and LOA of the difference. A relatively small mean difference between the e-DIA and recall methods was obtained, with an equal number of positive and negative values. The magnitude of the mean difference ranged from –1 g for meat and alternatives to 23 g for discretionary beverages. The Bland–Altman plots (Fig. 3(a)–(h)) showed no evidence of systematic bias, with roughly equal numbers of outliers distributed above and below the LOA. For fruit intake, one extreme outlier was found because of a very high intake of fruits reported in the recall but not in the e-DIA.

Fig. 3 Bland–Altman plot for (a) fruits, (b) vegetables, (c) grains (cereals), (d) meat and alternatives, (e) dairy products and alternatives, (f) discretionary foods, (g) discretionary beverages and (h) alcoholic beverages, for electronic Dietary Intake Assessment (e-DIA), and 24-h recalls, with 95 % limits of agreement.

Table 3 Agreement between mean intake (g) from food groups, between the electronic Dietary Intake Assessment (e-DIA) and 24-h recalls, as indicated by mean difference and 95 % limits of agreement (LOA), consumers only (n) (Mean differences and standard deviations)

Discussion

This study is the first to compare food group intakes using a mobile phone food diary app with repeat (three) 24-h recall as the reference measure with an Australian food composition database. All food groups had similar mean and median intakes for both methods, with no consistently higher or lower values for either method. Correlation coefficients were moderate to strong ranging from 0·69 to 0·88. Cross-classification into tertiles suggests that the e-DIA is able to rank intakes of food groups, and hence may be appropriate for research that aims to correlate the levels of intake of broad food groups with health outcomes. The Bland–Altman plots similarly revealed good agreement between e-DIA and 24-h recalls at the group level, without apparent bias, and with most data points located within 2 sd of the mean. However, the wide LOA suggest large variability in reported intakes at the individual level. These results support our previous findings assessing the relative validity of energy and nutrient intakes, and, collectively, suggest the potential of e-DIA as an assessment tool for dietary analysis at the population level.

As focus appropriately shifts from nutrient-centric to food-based research and advocacy, dietary assessment tools must be validated to measure intake from food categories of interest. Among the existing literature, many of the validation studies that include food groups in analysis refer to development of FFQ^{( 17 – 24 )} or diet quality indices^{( 25 – 29 )}. It is difficult to compare our results as there are no previous validation studies of food records in a mobile phone application format for food groups. Comparisons with validation studies of energy and nutrients using 24-h recalls and mobile phone applications^{( 30 )} or Personal Digital Assistant technologies^{( 31 – 33 )} reveal that correlation coefficients for nutrients (r 0·54–0·85) were similar to ours (r 0·69–0·88). A small validation study (n 26) comparing core food group intake from 7 d using a Digital Image Food Record with recalls among youth aged between 9 and 12 years found a wider range and on average lower correlations (r 0·20–0·90) compared with the present study^{( 34 )}. In the rare examples where the Bland–Altman analysis was used, the LOA were similarly described as being wide^{( 30 , 32 )}. Carter et al. ^{( 30 )} commented that wide LOA may be expected given that the reference measure is not a true reflection of absolute intake itself, largely due to the impact of recall bias. Moreover, a recent review concluded low-to-moderate validity of mobile phone food recording methods in estimating individual intakes^{( 35 )} and attributed this finding to the inherent biases related to self-reporting.

Strengths and limitations

A major strength of the present study was the ability to use e-DIA to collect dietary intake data without providing feedback to the participants as occurs with commercial apps. Such feedback might influence recording and/or may elicit unintended behaviour changes^{( 36 )}. The app is integrated with the Australian national food composition database consisting of over 4000 foods. This greatly reduced the workload of researchers for data entry and coding, although careful checking of all foods and beverages recorded each day, as was done in this study, may be useful to obtain reliable outputs. The app allows recording of food and beverages consumed in real time, and therefore does not rely on memory. Other strengths of this analysis include the use of a diverse range of statistical techniques that facilitates a balanced interpretation of results. In addition, a standardised method of disaggregation of recipes was applied across both data sets by a single researcher.

Using 24-h recalls as the reference method introduces several limitations to the study design, such as portion size estimation and reliance on memory. The multiple-pass method and portion size aids were designed to minimise the impact of these errors^{( 8 )}. As the 24-h recalls captured the same days that the participants digitally recorded their food intake into the e-DIA, there was potential for the recording process to have improved the recall of food and beverages, thereby augmenting the correlation between results from the two methods. However, all records were deleted from the app at midnight and recalls were conducted up to a maximum of 22 h after their deletion (i.e. 22.00 hours). As both methods relied on self-report, further validation of the e-DIA will be undertaken, using objective measures such as biomarkers.

The usability of the e-DIA also had some limitations, including the need for training before use, the burden of recording foods prospectively and searching through a large food database. When entering food items into the e-DIA and dependent on what they searched, participants may be presented with a long list of food options that could be challenging to navigate. Two search features were implemented in the e-DIA after preliminary testing to help alleviate this usability issue. First, the list of food options shortens as the user enters characters of the food name. Second, a ‘favourites’ or ‘history’ feature was used where food items that were entered previously are shown at the start of the list. Organising the drop-down list according to the most commonly consumed version of foods within a category has the potential to further improve the usability of the app.

As a convenience sample of university students was recruited, findings should not necessarily be extrapolated to all adult populations. University student volunteers are likely to have higher levels of motivation, education and be more technologically literate^{( 37 )}. However, markers of diet quality such as mean intake of fruits and vegetables were not dissimilar to those reported in the most recent National Nutrition and Physical Activity Survey (NNPAS)^{( 38 )} among persons between 19 and 30 years of age (mean fruit intake/d: 107 g NNPAS v. 87 g in recalls, mean vegetable intake/d: 163 g in NNPAS v. 180 g in recalls).

Conclusions

This unique study supports the potential of the e-DIA as an alternative to the repeat (three) 24-h recall as a dietary assessment tool for purposes of ranking individuals and estimating group mean intakes of food groups. Ability to evaluate dietary intake by a food-based rather than nutrient-centric approach can increase the translatability of findings into public health messages. Future research exploring the validity of the e-DIA will target study populations other than university students and use nutritional biomarkers to facilitate objective comparisons.