Assessing lifetime diet: reproducibility of a self-administered, non-quantitative FFQ

Abstract

Objective

To demonstrate test–retest reliability (reproducibility) of a new self-administered lifetime diet questionnaire, with a focus on foods relevant to cognitive health in older age.

Design

The reproducibility of dietary recall over four or five life periods was assessed by administering the questionnaire at two time points to an older cohort. The period between questionnaire administrations was 7 weeks. Polychoric correlations measured the association between recall at time 1 and time 2 and the weighted κ statistic measured the level of recall agreement for food groups across the two administrations of the questionnaire.

Setting

Adelaide, South Australia.

Subjects

Fifty-two cognitively healthy, older-age, community-dwelling adults completed the Lifetime Diet Questionnaire; mean age 81·8 (sd 4·4) years, range 70–90 years.

Results

The questionnaire showed very good reproducibility in this sample with a mean polychoric correlation coefficient of 0·81 between administration at time 1 and time 2, and an average weighted κ of 0·49 for the level of recall agreement between food groups.

Conclusions

The demonstrated reliability of this lifetime diet questionnaire makes it a useful tool to assess potential relationships between long-term dietary intake and later-age cognitive outcomes.

Lifetime diet assessment: rationale

Possible relationships between diet and cognitive status in elderly people have been the focus of considerable study, with the aim of ameliorating the burden of dementia and decline within the ageing population. Although results are inconsistent between studies, there is a growing body of epidemiological evidence suggesting that dietary factors contribute to cognitive health in old age via nutrient influence on brain metabolism^(1–5) and indirectly via their contribution to midlife vascular risk factors linked to later cognitive decline^(6–11).

However, an important caveat to the effectiveness of diet as a potential modifier of cognitive health is the interaction between dietary intake and genetically determined responses to that intake. For instance, in a number of studies, significant associations between dietary intake and cognition have been found only in either the presence or absence of the ApoE-4 allele^{(8, 12–14)}. In addition, genetic responses may also change with age, so there are times when an individual may be particularly susceptible to either damage or protection from dietary intake⁽¹⁵⁾.

The complex interactions across the lifespan between genetic and environmental factors suggest the importance of a life-course approach to cognitive ageing, yet most longitudinal studies of nutrition and cognition in the elderly focus on relationships between cognitive performance and dietary intake after the age of 65 years, when many of these interactions that determine cognitive status have already occurred⁽¹⁵⁾. Thus it follows that examining possible relationships between older-age cognition and intake of cognitively relevant foods across the lifetime is a worthwhile, yet previously neglected, endeavour when attempting to elucidate the role of diet in cognitive maintenance and decline. Cohort studies over so many decades are generally not feasible and therefore the best option for gathering data from the distant past is usually by participants’ recall⁽¹⁶⁾.

Long-term dietary recall appears to rely largely on people's generic knowledge of their diet; individual episodes of eating particular foods are quickly lost from memory but often repeated instances of eating are superimposed upon one another so that although details are lost, a general pattern or ‘script’ remains⁽¹⁷⁾. In a series of experiments designed to explore the cognitive processes underlying dietary recall, Smith et al.⁽¹⁸⁾ concluded that generic knowledge about one's habitual diet contributed significantly to reports of intake and that respondents could respond credibly about their average consumption frequencies, although precise estimates of dietary intakes for extended periods of time were unlikely to be accurate. This finding was supported by Fraser et al.⁽¹⁹⁾, who found better recall validity when non-quantitative methods of assessment were used.

Recall accuracy of long-term diet has been shown as being improved by the inclusion of memory cues to ‘locate’ participants in the appropriate period. Episodic memories, memories of life experiences peculiarly relevant to the self, may serve as prompts to generic memories encoded during a particular time⁽²⁰⁾. Therefore a number of questionnaires designed to assess dietary memory from the distant past have included autobiographical questions such as ‘Where were you living?’, ‘What job did you have?’ and ‘Were you married?’^{(16, 19, 21)}. Lifetime diet assessment would appear then to be a plausible undertaking if informed by the cognitive processes underpinning long-term dietary recall.

The aim of the present study was to assess the reproducibility of a new, non-quantitative FFQ designed to access long-term dietary memory, with the focus on cognitively relevant foods. Demonstrating reproducibility is an important first step in validating a new dietary measure. Although high correlations or levels of agreement between the two time points do not imply validity, reliability is a necessary condition for validity and low reproducibility indicates that the questionnaire has little utility⁽²²⁾. Very few studies have addressed the reproducibility of remote dietary recall. Cumming and Klineberg⁽¹⁶⁾ assessed the reproducibility of lifetime recall for a small number of foods in a sample of older people, aged between 65 and 100 years, with interviews one to three months apart; items assessed were beverages (including tea, coffee, alcohol and milk), cheese and stewed fruit. Spearman rank-order correlations ranged from 0·47 for coffee consumed at 50 years to 0·81 for tea consumed at age 20 years. Hislop et al.⁽²³⁾ administered a past diet questionnaire to women, aged between 40 and 70 years, referring to four different age periods: childhood, teens, younger adulthood and older adulthood (over 40 years). The questionnaire was administered at two time points four to six years apart. Overall, it was found that the weighted κ statistics for individual food items were consistently in the moderate range across all life periods. Interestingly, the FFQ was found to be more reliable for specific food items from the distant past than the recent past. Finally, in the study by Maruti et al.⁽²⁴⁾ that assessed the reproducibility of recalled adolescent diet from 15 to 35 years in the past, the Spearman rank correlations for food groups ranged from 0·48 for breads/cereals/grains to 0·70 for beverages. Unfortunately, results from these studies are not comparable given the disparity between their designs; however, they all suggest the reproducibility of long-term dietary memories.

The Lifetime Diet Questionnaire

For the current study, a Lifetime Diet Questionnaire was developed as a self-administered instrument to enable time- and cost-efficient sampling of large groups of participants. This questionnaire aimed to assess the intake of potentially cognitively relevant foods and beverages from childhood to older age, in older adults. The life period was divided into the following: Childhood, 5 to 18 years; Early Adulthood, 19 to 30 years; Adulthood, 31 to 45 years; Middle Age, 46 to 60 years; and Older Age, 61 to 75 years. At the beginning of each life-period section, autobiographical cue questions were included to help participants locate themselves in the appropriate period. The rationale for using these particular life periods was to capture the general (albeit not universally experienced) changing life circumstances that help delineate periods of time in people's memories.

Food groups rather than meals were used as the organising structure of the questionnaire. Although some studies have shown meal-ordered rather than food group-ordered instruments are more accurate and reliable, eating customs have changed during the 70 years covered by this questionnaire⁽²⁵⁾ so participants could find meal-based questions problematic. In addition, it has been shown that sociodemographic and lifestyle factors affect food choices throughout adulthood⁽²⁶⁾; therefore the food options given to assess lifetime diet needed to be general enough to capture these potentially different dietary patterns. The food groups used were based on the core food groups from the Dietary Guidelines for Australian Adults ⁽²⁷⁾ and included other relevant groupings such as beverages. The food items were organised under the following broad food group headings: vegetables, fruits, dairy products, cereals, takeaway food, protein-based food, seafood, sweets, snack-food, fats and oils, tea, coffee, alcohol, and multivitamin supplements. Foods were listed under these headings as either: (i) items containing a single food, such as ‘cow's milk’ or ‘eggs’; or (ii) items comprising a list of similar foods, for example ‘oranges, lemons, grapefruit’ and ‘lentils, dried peas/beans’. The food groups and their items were the same for each life period, with exceptions being food items that were unlikely to have been consumed during a particular life period; specifically, alcohol-related questions in childhood and lard for the later life periods. In total, seventy-four to seventy-nine questionnaire items were included for each life period. Within each food group, specific foods were selected that had been either: (i) explicitly associated with cognitive health in the literature, such as cruciferous vegetables⁽²⁸⁾, berries⁽²⁹⁾ and fish⁽³⁰⁾; or (ii) were considered deleterious, such as sweets and high-caloric foods^{(8, 31)}. In addition, lists of fruit and vegetables high in antioxidants were consulted^{(32, 33)} as a guide to the foods included. The consumption frequency options given for the foods were ‘daily’, ‘2 to 3 times per week’, ‘2 to 3 times per month’ and ‘rarely/never’. It is important to emphasise that the Lifetime Diet Questionnaire was not intended as an instrument to comprehensively record long-term dietary intake, but rather to differentiate between people on their general frequency of intake of foods that could potentially influence older-age cognitive status.*

Methods

Participants

Participants were recruited from the Ageing and Cognitive Change Study at the School of Psychology, University of Adelaide, South Australia⁽³⁴⁾. This was a 6-year study of cognitive ageing in older community-dwelling South Australians, screened for dementia at baseline; relevant ethical approval was gained for the current study from the University of Adelaide human ethics committee. Of the seventy-four people who agreed to participate, fifty-two completed the questionnaire at both time points; mean age 81·8 (sd 4·4) years, range 70·3–90·4 years. One person was excluded from the analyses because their second questionnaire was completed a month after all other questionnaires were returned. An acceptable level of association in a reproducibility study, as an indicator of reliability, is a correlation of 0·7⁽³⁵⁾. With fifty-one participants, the power of the study to achieve a significance level of 0·01 was >0·99⁽³⁶⁾.

Procedure

The Lifetime Diet Questionnaire, an information sheet and completion instructions were posted to participants. They were requested to complete the questionnaire for each life period in chronological order, one day at a time over five days, to minimize memories from one life period intruding into another period. In addition, given that each life period covered up to 15 years, participants were asked to recall their most representative diet and to report average consumption of seasonal foods. All participants completed the first four life periods, i.e. ‘Childhood’, ‘Early Adulthood’, ‘Adulthood’ and ‘Middle Age’. The fifth life period, ‘Older Age’, applied only to participants who were aged 80 years or over.

Five weeks after the completion date of their original questionnaire, each continuing participant was sent a repeat questionnaire with reminder information and instructions. The mean time between completion of the first and second administration of the questionnaire was approximately 7 weeks; mean 50·4 (sd 9·5) d.

Results

Missing data

Missing data in the Lifetime Diet Questionnaire were of two types: item non-response and multiple consumption frequencies reported for a food item. Analyses were performed using the SPSS for Windows statistical software package version 17·0·1 (SPSS Inc., Chicago, IL, USA). The Expectation Maximisation (EM) procedure⁽³⁷⁾ was used to estimate values for missing responses but convergence would not occur for Childhood on the first questionnaire or Adulthood on the second questionnaire. Due to the non-responses and the ‘rarely/never’ responses being matched at greater than chance rate across both administrations of the questionnaire, it was considered appropriate to recode non-responses as being equivalent to a food being eaten rarely/never and those missing data remaining (due to multiple responses to an item) were successfully estimated with the EM procedure. The χ ² statistic for each of the estimated data sets was not significant, suggesting the data were missing at random and therefore that the EM procedure would not lead to systematic bias in subsequent analyses⁽³⁸⁾.

Analyses

Polychoric correlations were used to assess the strength of the relationships between participants’ recall of their diet across the two administrations of the Lifetime Diet Questionnaire (SPSS 17·0·1 HETCOR extension). Polychoric correlations are appropriate when variables are ordinal or categorical but can be assumed to reflect an underlying continuous variable⁽³⁹⁾. Consumption frequency in this questionnaire was separated into discrete categories such as ‘daily’ or ‘2 to 3 times per week’; in reality, however, consumption of any food can be assumed to be a continual graduation from never eating it to eating it very often. Polychoric correlations capture this latent quality of consumption frequency and overcome the problem of attenuation that can occur when non-continuous dietary data are dealt with as being categorical⁽⁴⁰⁾.

The average correlation between consumption frequencies of individual food items at each time point was calculated by person and within each life period. Table 1 shows the average correlations between individuals’ recall of their total diet together with the 95 % confidence intervals and the P value as calculated by a permutation test for each life period.*

Table 1 Mean polychoric correlations between the first and second administration of the Lifetime Diet Questionnaire among cognitively healthy, older-age, community-dwelling adults, Adelaide, South Australia

***P < 0·001 (as calculated by permutation test).

The test–retest correlations for each life period are in the range considered good for a survey instrument and compare favourably with other reproducibility studies for dietary assessment questionnaires⁽²²⁾. It could be argued, however, that the high correlations between the two administrations of the questionnaire were due to the same (or very similar) diet being recalled across all life periods and at both administration time points; for example, if participants were simply recording their current diet for each period. Therefore, to examine whether the Lifetime Diet Questionnaire captured possible (and likely) change in dietary intake during the lifetime, paired-sample t tests were used to compare the mean correlations between the first and second administration of the questionnaire (Childhood, Early Adulthood and Adult periods only) with the mean correlations between each of these life periods and recall of Middle Age diet at both time points. If the same diet was being recalled for all questionnaires, then it could be expected that there would be no significant difference between these mean correlations; however, if the Lifetime Diet Questionnaire was sensitive to recalled dietary changes, then a significantly higher correlation could be expected for the same life period (at both administration time points) compared with the correlation for a particular earlier life period with the Middle Age life period.* These correlation pairs and their differences are presented in Table 2.

Table 2 Mean differences in correlations for early-life diet recall at two time points compared with early-life diet recall and middle-age diet recall at two time points in cognitively healthy, older-age, community-dwelling adults, Adelaide, South Australia

CHD, Childhood; EAD, Early Adulthood; AD, Adulthood; MAGE, Middle Age; T1, time 1 administration; T2, time 2 administration.

***P ≤ 0·001.

†One outlier was removed from each of the EAD (T1/T2) and AD (T1)/MAGE(T1) variables to improve normality; skewness and kurtosis values for all variables fell between ±2.

The mean correlations for dietary recall at time 1 and time 2 for the same life periods, for Childhood and Early Adulthood, were significantly higher than the mean correlations between these early periods and Middle Age. There were no significant differences, however, in mean correlations between the two recalls of Adult diet, and the mean correlations for the recall of Adult diet with Middle Age diet. This would suggest that a different dietary intake was being recalled for earlier life periods compared with Middle Age, indicating either a very plausible change in diet from these earlier periods to middle age or that the current diet was exerting a greater influence on memory of these more relatively recent periods.

The level of agreement for memory of individual foods (as opposed to total diet) was assessed by the weighted κ statistic (using the Analyse-it^® statistics add-in software for Microsoft Excel^® version 2·20; Analyse-it Software Ltd, Leeds, UK); the distributions for most items were too far from bivariate normal for the polychoric correlation to be calculated. Weighted κ takes into account the degree of agreement between items on an ordinal scale; items that are in perfect agreement are given a weight of 1 while different weights w_i are assigned to items that differ by i categories. Thus if there are k categories the weights are calculated by w_i = [1 − (i/k − 1)]. Therefore, on a 4-point ordinal scale, disagreement by one category is weighted by 0·67, disagreement by two categories is weighted by 0·33 and disagreement by three categories is given zero^{(42, 43)}. Values of κ thus range from 0 to 1, with 1 being the highest level of agreement. The weighted κ averages for food groups within each life period are presented in Table 3.

Table 3 Average weighted κ statistics for food items within food groups between the two administrations of the Lifetime Diet Questionnaire among cognitively healthy, older-age, community-dwelling adults, Adelaide, South Australia

N/A, not applicable.

With the exception of Early Adulthood, the average weighted κ for each life period was >0·5 which indicated a moderate to good level of agreement between items in the two administrations of the present reproducibility study⁽⁴²⁾. Weighted κ ranged from 0·27 for fats/oils in Early Adulthood to 1·00 for soya products in Childhood. There were no food groups that had consistently higher or lower levels of agreement across all life periods, although tea/coffee and soya products had the highest level of agreement and other protein foods had the lowest level of agreement across two out of the five life periods.

For the current sample, higher κ values were associated primarily with foods consumed rarely/never or, to a much lesser extent, consumed daily; 81 % of responses for items with a weighted κ of >0·6 had a mode of 1 (consumed rarely/never) and 17 % had a mode of 4 (consumed daily), leaving only 2 % of responses that had a mode of 2 to 3 times per week or 2 to 3 times per month.

Discussion

The Lifetime Diet Questionnaire demonstrated excellent reproducibility for all life periods in a group of cognitively healthy elderly people. This is in agreement with previous studies that demonstrated elderly people can recall dietary intake reliably, with correlations between assessments equal to, or greater than, those of younger cohorts⁽⁴⁴⁾. Nevertheless, it is acknowledged that reproducibility itself does not demonstrate the validity of dietary memories. The influence of current diet on past diet reporting has been well documented^(45–48) and in the case of the Lifetime Diet Questionnaire it could be argued that the questionnaire's reproducibility may be due to participants consistently reporting similar versions of their current dietary intake for all periods and across both questionnaire time points. However, dietary patterns have been shown to change over time^{(21, 26, 49)} and so it could be expected that if the questionnaire was validly assessing lifetime memory, there would be stronger associations between two recalls of the same dietary period than between that life period and a later life period. Significantly higher reproducibility correlations were demonstrated for the early-life periods (Childhood and Early Adulthood) compared with the mean correlation between these life periods and the Middle Age period at both administrations of the questionnaire. Thus the Lifetime Diet Questionnaire appeared sensitive to temporal change in individuals’ reported intake between early life and middle age. The lack of a significant difference in the mean correlation between the two recalls of adult diet and that of middle age was not surprising given that the largest changes in diet could be expected to occur in late adolescence and early adulthood as the transition is made from living with parents to the establishment of independent households and lifestyles⁽⁵⁰⁾. In addition, dietary choices have been shown to be associated with demographic factors such as income, occupation and life circumstances⁽²⁶⁾ which, for this older population, may have been relatively stable from adulthood to middle age compared with earlier periods. The demonstrated success of the Lifetime Diet Questionnaire in discriminating between dietary intakes across life periods may have been due, in part, to the non-quantitative and list-based approach used that utilised participants’ generic dietary memory to successfully recall their typical diet for a given period⁽⁵¹⁾.

It was noteworthy that despite Childhood being the most distant period, it was the period with highest reproducibility for diet recall in the present cohort. Autobiographical memory research has suggested that durable memories in childhood are formed by focused parental attention, guidance and interaction^{(52, 53)}. It is possible that meal times and food choices in childhood were particularly salient as a focus for these forms of close communication and so led to more vivid recall of diet during this period.

Unfortunately, no actual dietary records from the past were available for this elderly sample of participants with which to validate dietary memory data obtained using the Lifetime Diet Questionnaire. A number of studies have examined the validity of distant dietary memory by comparing recalled diet with dietary records from the relevant period. Generally, stronger correlations were seen in studies where the interval between the original records and the recall period was less than 10 years⁽²²⁾. However, a study by Dwyer and Coleman⁽⁵⁴⁾ showed that the memory of middle-aged people for food intake up to four decades earlier did not decline inevitably over time; rather, time-related memory loss varied from food to food. It was suggested that the pattern of consumption frequency may define memory accuracy so that foods eaten rarely, and those eaten every day, were more likely to be reported accurately. This is in accordance with the relationship found herein between levels of agreement and consumption frequencies for food groups across the two administrations of the Lifetime Diet Questionnaire; those foods with higher levels of agreement were those that were recalled as being eaten rarely/never or eaten daily. Although average weighted κ values for food groups generally fell within the moderate range, it was not possible to make a meaningful comparison between these results and those from other studies; given that the magnitude of κ is dependent on the proportion of items in each category and the number of categories used, two quite disparate values of κ can be representative of the same absolute levels of agreement^{(42, 43)}. Nevertheless, the moderate agreement between the two questionnaire administrations was consistent across all summary food groups and for all five life periods, thus suggesting the Lifetime Diet Questionnaire effectively demonstrated that long-term dietary memories are reproducible for older adults.

Conclusions

Retrospective lifetime dietary recall is a new approach to evaluating the possible long-term contribution of dietary intake to age-related cognitive decline. The Lifetime Diet Questionnaire has been proposed as a self-administered instrument to assess dietary intake from the past in cognitively healthy older people without the necessity of actual dietary records, which are usually unavailable for elderly populations. When the questionnaire was administered at two time points to a group of cognitively healthy, older, community-dwelling adults, the average reproducibility correlation coefficient was 0·81 for recall of dietary intake across five life periods and the average weighted κ for summary food groups was 0·49. Considering the length of period of recall we consider this a very good outcome. Although studies to assess the questionnaire's validity by comparing recalled intake with actual dietary records are desirable, the current results provide an encouraging first step in demonstrating the reliability of the questionnaire and its potential utility in assessing long-term dietary intake. Given the possible influence of lifetime diet on older-age cognitive functioning via its interaction in earlier life with biological and environmental factors, such an instrument is a valuable contribution to the investigation of temporal relationships between dietary intake and age-related cognitive decline.