Reliable, accurate and feasible methods of dietary and physical activity assessment are required in order to establish associations between these behaviours and health, to monitor whether these behaviours are meeting government recommendations and targets, and to determine the effectiveness of interventions and initiatives which aim to change these behaviours(Reference Taren, Dwyer and Freedman1, Reference Livingstone, Robson and Wallace2). Public health agendas in most countries include tackling obesity as a priority(3), and thus there is a particular need for appropriate tools which specifically measure components of dietary and physical activity behaviours that have an impact on energy balance. Currently, measurement of diet and physical activity in large populations relies mainly on self-report, paper-based methods. However, these methods can be time consuming for both participants and investigators, and separate instruments for the measurement of diet and physical activity are often used, adding to the burden.
Approaches to dietary and physical activity assessment that incorporate novel technologies are of high interest and a number of approaches are currently being explored and developed(4). Computer-based methods offer a promising approach for both dietary and physical activity assessment, with advantages including the ability to administer methods in a standardised manner and, if self-administered, elimination of interviewer-associated bias. Other advantages include instant data entry and storage without the need for additional coding (decreasing coding errors), which can include automatic data checks that can be implemented to ensure complete datasets are provided, and instant data analysis(Reference Kohlmeier, Mendez and McDuffie5). In addition, Internet-based programs can be delivered instantly and simultaneously to large numbers of individuals, without the cost of postage, regardless of geographic location, and can be completed at times and in locations convenient to the participant.
A number of programs (some Internet-based) have been reported in the literature that collect data, using a previous or typical day recall, on dietary intake of adults(Reference Slimani, Deharveng and Charrondière6–Reference Moshfegh, Rhodes and Baer9) and children/adolescents(Reference Baranowski, Islam and Baranowski10–Reference Vereecken, Covents and Matthys12), and on physical activity (all designed for use by children)(Reference McMurray, Harrell and Bradley13–Reference Ridley, Olds and Hill16). To our knowledge, only two computerised assessment programs that collect data on diet and physical activity simultaneously have been published in the literature: peas@tees(Reference McLure, Reilly and Crooks17) and the Synchronised Nutrition and Activity Program (SNAP™)(Reference Moore, Ells and McLure18). These programs are designed for use by children and, to our knowledge, no programs exist that simultaneously collect data on diet and physical activity that are designed for use by adults.
The present paper describes the development of the Synchronised Nutrition and Activity Program for Adults (SNAPA™) and its test–retest reliability and concurrent validity against direct dietary observation and combined heart rate and accelerometry at the group level.
Methods
Participants and sampling
A volunteer sample of participants was recruited for the test–retest reliability study through invitation posters and leaflets distributed in shops and community venues throughout the Tees Valley region of the North East of England, and articles published in a local newspaper. Participants for the validation study were recruited from workplaces, colleges and universities (staff and students) in the Tees Valley region. The study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the School of Health and Social Care Ethics Committee, Teesside University. Written informed consent was obtained from all subjects.
Measures
Description of the Synchronised Nutrition and Activity Program for Adults (SNAPA™)
The development of SNAPA™ was informed by previous Durham University and Teesside University projects (peas@tees and SNAP™) that successfully developed and evaluated a prototype online assessment tool and an updated version(Reference McLure, Reilly and Crooks17, Reference Moore, Ells and McLure18), which records dietary intake and physical activity behaviours in children. The present study aimed to develop and evaluate a similar prototype for use in an adult population.
SNAPA™ is an Internet-based program written using PHP, MySQL and AJAX (incorporating JavaScript and XML technologies) to allow secure data transfer and storage. SNAPA™ asks users to recall foods and drinks consumed, and activities carried out, the previous day. Asking users to recall ‘actual’ rather than ‘usual’ behaviours has some cognitive benefits that may lead to more accurate recall. There is some evidence to support this using physical activity recall examples(Reference Sallis and Saelens19). When recalling previous events, autobiographical memory is used, which consists of episodic memory (memories arisen from specific events, objects or people at a specific time and/or place) and semantic memory (general knowledge and facts)(Reference Tulving, Tulving and Donaldson20). Recalling actual events forces the cognitive distinction of specific episodes and the use of episodic memory rather than semantic memory(Reference Durante and Ainsworth21). The use of semantic memory increases the chance of ‘intrusion’ memories (memories of events that the individual believes occurred, but in reality did not)(Reference Bower, Black and Turner22). Recalling recent behaviours (previous day) compared with longer recalls (past week, month, year) reduces the reliance on memory and results in more accurate recalls(Reference Smith, Jobe and Mingay23).
SNAPA™ was developed to collect data on dietary and physical activity behaviours simultaneously. The simultaneous measurement of diet and physical activity in one instrument not only provides a more streamlined approach, reducing burden for both subjects and investigators, but may also provide added cognitive benefits that aid recall. Dietary and physical activity events are inexplicably interconnected by temporal and causal links; accessing information about one can enable information to be accessed about the other(Reference Bradburn, Rips and Shevell24). Dietary and physical activity events generate effective memory markers (or retrieval cues) for each other. These memory markers strengthen the ability to retrieve specific information from episodic memory(Reference Smith25).
SNAPA™ follows a relatively open structure, i.e. not based on a typical 09.00–17.00 hours working day, to allow flexibility for a variety of living patterns (e.g. part-time workers, shift workers, unemployed, retired, homemakers and students). The program is segmented into four time periods: morning, afternoon, evening and after midnight (if required). For each time period, users are asked to report foods and drinks consumed as part of eating occasions or meals (breakfast, lunch, dinner, tea, supper or snack), a strategy shown to enhance recall of dietary behaviours(Reference Smith25). Users are also asked to report the time and location of each eating occasion in order to add context and generate additional memory markers. The food and drink items recalled are selected from a predefined list of commonly consumed foods (n 102) and drinks (n 18) developed from findings from the National Diet and Nutrition Survey(Reference Henderson, Gregory and Swan26) and pilot research on dietary habits of local adults where participants were asked to list commonly consumed food, drinks and food combinations (e.g. fish and chips; Teesside University, FC Hillier, unpublished results). All food and drink options in SNAPA™ were reported by at least 25 % of respondents from the National Diet and Nutrition Survey(Reference Henderson, Gregory and Swan26). Of a total of 120 food or drink options reported in the National Diet and Nutrition Survey(Reference Henderson, Gregory and Swan26), twenty-five are not included in the SNAPA™ food and drink item lists. There was no scientific justification to using a cut-off point of 25 % or more respondents; however, it was believed to be sufficient to ensure that the most popular foods and drinks consumed by British adults were included in the SNAPA™ options. If a food or drink option is not available from the list, users are instructed to choose the most similar option available. Additional questions are asked, if appropriate, about the type of food or drink item selected e.g. ‘diet’ version, reduced fat, reduced sugar, and whether it was cooked in fat (Fig. 1). Each food and drink item (using additional information if applicable) is assigned nutrient values using an underlying database based on UK food composition tables(27), along with an assigned portion size based on average intakes of adults in the UK(Reference Wrieden and Barton28). Users are asked to report fruit and vegetables (FV) consumed as numbers of portions using portion size guidance based on that provided by the National Health Service ‘5aDay’ initiative and the Food Standards Agency(29, 30). For composite food and drink items containing fruit and/or vegetables, FV portions are assigned using standard recipe data(27).
Fig. 1 Example of the dietary intake section of the Synchronised Nutrition and Activity Program for Adults (SNAPA™).
In a similar process to that which was developed to record diet, participants were asked to report their activities, from a list of common physical and sedentary activities, for each time period. This list was developed using the findings from the General Household Survey(Reference Fox and Rickards31) and pilot research on activities of local adults (Teesside University, FC Hillier, unpublished results). Activities are reported in terms of duration (time the activity started and time the activity finished) and intensity, using graphics illustrating ‘low’, ‘medium’ and ‘high’ intensities (Fig. 2). Each activity reported is assigned a metabolic equivalent (MET) value derived from the Compendium of Physical Activities(Reference Ainsworth, Haskell and Whitt32). A MET is the estimated resting energy expenditure, with activities defined in multiples of resting metabolism. The intensity thresholds adopted were 3 MET for moderate physical activity and 6 MET for vigorous physical activity. Users are asked to enter activities that were carried out for at least 10 min. This instruction was designed to (1) reduce user burden, so that time was not spent reporting activities for every minute of the day, and (2) report physical activity bouts of 10 min or more, in line with the Department of Health recommendations(33).
Fig. 2 Example of the physical activity section of the Synchronised Nutrition and Activity Program for Adults (SNAPA™).
To ensure completeness of the datasets, at the end of each time period, and at the end of the recall, a review of all items reported was given, with the option of adding new items using a prompt of commonly forgotten foods (biscuits, cakes, sweets, chocolate, other confectioneries, crisps, peanuts, other snacks, sauces and dressings), drinks (coffee, tea, soft drinks and milk) and activities (walking (the dog/to the shop/to the bus stop/to work), shopping, looking after children, house work and dancing (at a nightclub/party)). To add context to the data collected and aid interpretation, users were asked four questions, based on those asked in the recall method used in the Low Income Diet and Nutrition Survey(Reference Nelson, Erens and Bates34), to determine whether data reported were representative of their usual dietary and physical activity behaviours: ‘Would you say yesterday was an average day?’; ‘Do you think that what you had to eat yesterday is what you would eat in an average day?’; ‘Do you think what you had to drink yesterday is what you would drink in an average day?’; and ‘Do you think the amount of time, difficulty and types of physical activities you did yesterday are what you would do in an average day?’.
SNAPA™ was developed in the first instance to assess FV portions, percentage of food energy from fat (% fat) and minutes of moderate-to-vigorous physical activity (MVPA). The rationale for this is based on current national public health physical activity and healthy eating recommendations for England, which focus on increasing FV consumption, decreasing fat consumption and increasing MVPA(33, 35–38). Our long-term planned programme of work includes, if this initial work proves feasible and successful, the development of SNAPA™ to assess additional nutrients, food groups, types of physical activity (e.g. sedentary behaviour), and dietary and physical activity patterns.
Combined heart rate and accelerometry
An objective measurement of MVPA was derived using combined heart rate and accelerometry (Actiheart®; CamNtech, Cambridge, UK) with branched equation modelling(Reference Brage, Brage and Franks39), as a reference method for comparison against the physical activity data reported using SNAPA™. The method provides accurate measures of energy expenditure in adults(Reference Assah, Ekelund and Brage40–Reference Thompson, Batterham and Bock43). Combining physiological measurement with motion sensor measurement results in more accurate measures of physical activity when compared with using motion sensors alone(Reference Avons, Garthwaite and Davies44–Reference Strath, Bassett and Thompson48).
The Actiheart® monitor is small (main component is 32 mm in diameter and 6 mm in depth, with a wire of approximately 100 mm in length running to a smaller unit (5 × 11 × 22 mm)), lightweight (10 g), fully waterproof monitor and has a battery life of 21 d. The monitor is attached to two electrocardiogram electrodes, placed either at the upper (at the level of the third intercostal space) or lower (just below the apex of the sternum) chest position. In the present study, the position of the monitor was determined by the participant's preference, and in most cases, the lower chest position was used. A previous study(Reference Brage, Brage and Ekelund49) showed that although cleaner heart rate data might be obtained in the lower position, there is no substantial difference in energy expenditure or count variables produced by monitors positioned at each site.
In order to calibrate the data collected by the Actiheart® monitor to the individual, each participant was asked to carry out a simple step test. The step test is a submaximal test, which starts at a rate of 15 steps/min (an intensity of approximately 4 MET; moderate intensity) and builds to a rate of 33 steps/min (approximately 8 MET; vigorous intensity) over 8 min. Pre-participation screening guidelines were used to exclude individuals for whom carrying out exercise at this intensity may be contraindicated and require medical supervision(50).
Participants were instructed to wear the monitor continuously for 9 d and only to remove it when changing electrodes. Data for the first and last days were discarded as they did not cover a full 24 h period; therefore, data collected represented physical activity levels for seven full days. Data were collected in epochs of 1 min.
Dietary observation
Direct dietary observation was used as a reference method for comparison against the dietary data reported using SNAPA™. Direct dietary observation took place during lunchtime periods in the participants' workplace, college or university. Following a protocol based on one previously used with children(Reference Domel, Baranowski and Leonard51), trained observers recorded food and drink items brought to the eating area, along with estimated portion sizes for each food/drink item, and recorded how much of each food/drink item was consumed as ‘all’, ‘most’ (approximately three-quarters of portion), ‘half’, ‘some’ (approximately one-quarter of portion) or ‘none’. Food/drink items recorded were coded, and the portion size of the food/drink item consumed was converted into grams. Data were entered into the dietary intake analysis software WISP version 3.0 (Tinuviel Software, Anglesey, UK) for nutritional analysis.
Test–retest reliability study protocol
Participants attended a study day held either at Teesside University or at a community venue in Middlesbrough. Participants completed SNAPA™, recalling previous-day diet and activities, in a quiet IT suite with a researcher on hand if any assistance was required. Participants completed a distraction task (a general knowledge quiz game) before completing SNAPA™ for a second time. The duration between the completions of SNAPA™ was approximately 1–2 h.
Concurrent validity study protocol
On the first day of the study (day 1), participants were fitted with an Actiheart® monitor. Between days 3 and 9, participants were asked to complete SNAPA™ on each of the five working days (Monday to Friday). A personalised email containing the web link used to access SNAPA™ was sent to each participant on the morning of the first day on which they were required to complete the program. An email containing a reminder to complete the program, along with the web link, was sent on the morning of each subsequent day on which they were required to complete the program. On four of the working days (between days 2 and 8), their dietary intake was observed during the lunchtime break. After seven full days of wearing the monitors (day 9), participants were asked to meet with the researcher to complete a step test and return the Actiheart® monitor. All participants who completed the study received an incentive of high-street shopping vouchers to the value of £30.
Data analysis
The short-term test–retest reliability of SNAPA™ was evaluated for each outcome variable (% fat, number portions of FV and total minutes of MVPA). A systematic shift in the mean (bias) from test to retest was assessed using the mean difference and its uncertainty (90 % CI). The random error component was assessed using the typical (standard) error of measurement and the appropriate form of the intraclass correlation coefficient (ICC): ICC (3, 1) for the reliability of a single measure and ICC (3, 2) for the reliability of the mean of two measures. Uncertainty in the typical errors and ICC is expressed using 90 % CI.
For the analysis of food item agreement between foods and drinks reported using SNAPA™ and those observed eaten during the direct dietary observation, a method adapted from two previous observation studies in children(Reference Baranowski, Islam and Baranowski10, Reference Warren, Henry and Livingstone52) was applied. Food items were classified as ‘observed eaten’ if any of the food was eaten, i.e. would only not be classified as observed if the food item was coded as ‘none’ eaten by the observer. Food items reported in SNAPA™ under the meal classification ‘lunch’ or ‘dinner’ (a term used locally for the midday meal) and/or reported at the time that the dietary observation took place were included in the analysis. A food item was classified as a ‘match’ if the food item which was observed as eaten was reported in SNAPA™ (or an appropriate alternative if an exact match was not available in SNAPA™, e.g. ‘malt loaf’ was reported as ‘bread’); an ‘omission’ if the food item which was observed as eaten was not reported in SNAPA™; or a ‘phantom’ if a food item was reported in SNAPA™, but not observed as eaten. Match and phantom rates for each day for the total sample were calculated using the following formulas:
For the concurrent validity analysis, mean daily values of % fat, FV portions and minutes of MVPA reported using SNAPA™ by each participant were used to calculate mean daily values for the sample as a whole. Minutes of MVPA per d as estimated using combined heart rate and accelerometry were calculated using all minutes of MVPA measured (MVPAALL) and using only minutes carried out in bouts of 10 min or more (MVPA10+), i.e. each minute in the bout must have reached moderate intensity and the bout must have lasted for at least 10 min.
The accuracy of SNAPA™ for the dietary data, in terms of % fat and the number of portions of FV, and physical activity data was initially assessed by the mean difference between the methods (SNAPA™ − reference method). Linear regression analysis was carried out to determine the correlation coefficient, and Passing–Bablok (type II) regression to evaluate bias(Reference Passing and Bablok53) using the Analyse IT® software (Analyse IT® Software Limited, Leeds, UK). Fixed bias is indicated by substantial departure from a zero intercept, with a slope substantially different from one revealing a proportional bias.
Results
Test–retest reliability
A total of forty-four participants (age 41·4 (sd 17·3) years; 64 % female; BMI 27·9 (sd 4·9) kg/m2) completed the test–retest reliability study. The means for the repeat administrations of SNAPA™ were 57·6 (sd 60·4) and 56·4 (sd 58·9) min of MVPA, 26·3 (sd 9·4) and 26·2 (sd 9·4) % fat, and 3·5 (sd 3·1) and 3·8 (sd 3·1) portions of FV. There was no substantial systematic shift in the mean from test to retest for any of the outcome variables. For % fat, the 90 % CI for the mean difference between tests 1 and 2 was − 1·9 to 2·2 % (P = 0·93). The typical error was ± 5 % fat (90 % CI 4·2, 6·3). The ICC (3, 1) was 0·72 (90 % CI 0·55, 0·83) and the ICC (3, 2) was 0·84 (90 % CI 0·71, 0·91). For portions of FV, the 90 % CI for the mean test–retest difference was − 1·1 to 0·30 (P = 0·38). The typical error was ± 1·7 portions (90 % CI 1·4, 2·1 portions). The ICC (3, 1) was 0·70 (90 % CI 0·52, 0·82) and the ICC (3, 2) was 0·82 (90 % CI 0·69, 0·90). For minutes of MVPA, the 90 % CI for the mean difference between test occasions was − 15·2 to 15·7 min (P = 0·89). The typical error (standard error of measurement) was ± 36 min (90 % CI 31, 46 min). The single measure ICC (3,1) was 0·62 (90 % CI 0·42, 0·76) and the average measure ICC (3, 2) was 0·76 (90 % CI 0·59, 0·87).
Concurrent validity
Sample characteristics
A total of seventy-seven participants (age 34·4 (sd 11·1) years; 61 % female; BMI 25·1 (sd 4·5) kg/m2) were recruited for the concurrent validity study. The majority of participants had jobs that were mainly office-based (n 50), four participants were PhD students and seventeen were further education students on a sports-based course, and the remaining participants were manual workers (industry and garden centre, n 6).
Compliance with the methods
The compliance rates for each assessment method are displayed in Table 1. Of the participants, six did not complete SNAPA™. However, two of these participants could not complete the program because SNAPA™ was not compatible with an updated version of the Internet Explorer web browser (version 8; Microsoft Corporation, Redmond, WA, USA). It is unknown why the other four participants did not complete the program; however, these participants did not use computers as part of their job roles, although they did have access to the Internet at work and home, and may have had less opportunity to access SNAPA™. For the remaining participants, compliance with SNAPA™ was good, with the highest percentage of participants completing SNAPA™ on the 5 d requested. Additionally, eleven participants (14 %) completed SNAPA™ on more days than requested.
Table 1 Participants completing number of days' data by each assessment method
(Number of participants and percentages)
SNAPA™, Synchronised Nutrition and Activity Program for Adults.
* CamNtech, Cambridge, UK.
Compliance with the Actiheart® monitor was good, with 91 % providing some data and 84 % providing data for all 7 d requested. Of the incomplete datasets, two were lost: one set because of a fault with the monitor and one set because of a researcher error when downloading the data. Moreover, two participants removed their Actiheart® monitors for 1 day due to a special occasion. The remaining incomplete sets of data from the Actiheart® monitors (n 10) were a result of the participants having an allergic skin reaction to the electrodes. During the method comparison analysis between SNAPA™ and Actiheart®, data for one participant (male) was removed as preliminary analysis(Reference Hopkins, Marshall and Batterham54) revealed it as an extreme outlier.
The lowest compliance was for the direct dietary observation method. Dietary observation data were collected for just over 70 % of all the participants, with only 17 % being observed on all of the 4 d requested. Of those who were observed, the highest percentage of participants was observed at three lunchtimes (29 %).
Paired data for the comparison between SNAPA™ and dietary observation were collected for forty-six participants for a mean of 2·7 d; and for sixty-three participants (mean of 4·1 d) for the comparison between SNAPA™ and combined heart rate and accelerometry.
Dietary behaviours
SNAPA™ appeared to show good agreement with direct dietary observation at a food item agreement level, with high match rates and low phantom rates (Table 2). During ninety-seven eating occasions, 467 food items were observed as eaten. A total of 102 of the food items observed as eaten were not recalled in SNAPA™ (forgotten foods). The most commonly forgotten food item was fruit (24 % of 102 forgotten foods), followed by yogurt (14 %), vegetables (13 %), bread (7 %), chocolate, biscuits and desserts (7 %), crisps (5 %), rice and potatoes (5 %), soup (4 %), meat and fish (4 %), extras/sides (e.g. coleslaw, gravy; 4 %), tea and coffee (3 %), soft drinks (including sugar-sweetened carbonated beverages; 3 %), complete meals (e.g. chilli con carne, cottage pie; 3 %), sandwiches (2 %) and cheese (1 %). A total of seventeen ‘phantom’ foods were recalled in SNAPA™ but not observed as eaten, and of these, the most common phantom foods were chocolate, sweets and biscuits (35 % of seventeen phantom foods), followed by tea and coffee (29 %). Sandwiches, crisps, fruit, vegetables, milk and chicken/turkey were each reported as a phantom food once (6 % each). Of the 467 food items observed during the lunchtime observation sessions, there were thirteen occasions when the food item eaten was not available on the SNAPA™ food option list. In all cases, an appropriate alternative was selected and the selection was considered a ‘match’ for this analysis.
Table 2 Food item agreement between Synchronised Nutrition and Activity Program for Adults (SNAPA™) and direct dietary observation (n 46)
The mean between-method differences for the dietary variables are displayed in Table 3. SNAPA™ overestimated intakes in terms of % fat, compared with direct observation. There were no substantial differences between the methods in terms of FV intake. Linear regression analysis revealed correlations of 0·42 (90 % CI 0·19, 0·60), with a standard error of the estimate (SEE) of 10·0 percentage points (90 % CI 8·5, 12·0), and 0·56 (90 % CI 0·36, 0·71), with a SEE of 0·65 portions (90 % CI 0·56, 0·79), between SNAPA™ and direct dietary observation estimations of % fat and FV intake, respectively. Passing–Bablok (type II) regression analysis revealed no substantial biases for % fat or the number of portions of fruits and vegetables (where there was almost perfect agreement; Table 4).
Table 3 Agreement between Synchronised Nutrition and Activity Program for Adults (SNAPA™) and direct dietary observation at a nutritional level (n 46), and combined heart rate and accelerometry (Actiheart®; CamNtech, Cambridge, UK) (n 63)
(Mean values, standard deviations and 90 % confidence intervals)
% Fat, percentage of food energy from fat; FV, fruit and vegetables; MVPA, moderate-to-vigorous physical activity; MVPAALL, all activities at three or more metabolic equivalents (MET); MVPA10+, activity at three or more MET in bouts of 10 min or more.
Table 4 Passing–Bablok regression variables, Synchronised Nutrition and Activity Program for Adults (SNAPA™) v. direct dietary observation at a nutritional level (n 46), and SNAPA™ v. combined heart rate and accelerometry (Actiheart®; CamNtech, Cambridge, UK) (n 63)
% Fat, percentage of food energy from fat; FV, fruit and vegetables; MVPA, moderate-to-vigorous physical activity; MVPAALL, all activities at three or more metabolic equivalents (MET); MVPA10+, activity at three or more MET in bouts of 10 min or more.
Physical activity behaviours
SNAPA™ under-reported minutes of MVPA per d, when compared with all minutes of MVPA measured by the Actiheart® monitor, but over-reported when compared with minutes of MVPA measured by the Actiheart® monitor that were carried out in bouts of 10 min or more (Table 3). Linear regression correlations between minutes of MVPA estimated using SNAPA™ and minutes of MVPA measured by the Actiheart® monitor were 0·44 (90 % CI 0·25, 0·59), with a SEE of 50 min (90 % CI 44, 59), when including all minutes measured by the Actiheart® monitor, and 0·27 (90 % CI 0·06, 0·49), with a SEE of 31 min (90 % CI 27, 36), when only minutes carried out in bouts of ≥ 10 min measured by the Actiheart® monitor were included.
Passing–Bablok regression analysis revealed a fixed bias, but no substantial proportional bias, when all minutes of MVPA measured by Actiheart® were included in the analysis. However, when only minutes carried out in bouts of 10 min or more measured by the Actiheart® monitor were included, a substantial proportional bias, but no substantial fixed bias, was revealed (Table 4).
Discussion
The absence of systematic shifts in the mean for all three primary outcome variables in the repeated completions of SNAPA™ in the test–retest reliability study indicates that there is no short-term learning or fatigue effect associated with repeat administrations of SNAPA™. The ICC suggest acceptable relative reliability with moderate to large coefficients for the average of two measures. However, some caution is required as this is an estimate of reliability in the very short term; reliability would be expected to be attenuated over a longer period of time such as that associated with interventions or monitoring.
The results from the present study indicate that SNAPA™ shows good agreement with direct dietary observation at a food item level with high match rates and low phantom rates. Surprisingly, the most commonly forgotten foods tended to be the ‘healthy’ (or perceived ‘healthy’) foods such as fruit, vegetables and yogurt, with bread, rice and potatoes also forgotten by a small proportion of participants, and the most common phantom foods were chocolate, sweets and biscuits. These findings are in contrast to the common perception that individuals tend to over-report ‘healthy’ and under-report ‘unhealthy’ foods.
Results from the food item agreement analysis are difficult to compare with other studies, as, to our knowledge, no other studies have performed dietary observation of adults in free-living settings, although a number have been carried out in a research centre setting(Reference Conway, Ingwersen and Moshfegh55–Reference Beasley, Riley and Jean-Mary57), or reported food item agreement in adults. In lunchtime observation studies in children, match rates between 46 and 70 %(Reference Baranowski, Islam and Baranowski10, Reference Warren, Henry and Livingstone52, Reference Baxter, Royer and Hardin58, Reference Baxter, Thompson and Litaker59), and phantom (or intrusion) rates between 24 and 54 %(Reference Baranowski, Islam and Baranowski10, Reference Baxter, Royer and Hardin58, Reference Baxter, Thompson and Litaker59) have been reported. Results from the present study show higher match rates and lower phantom rates; however, protocols and analysis were not identical between the studies and it may be expected that adults are able to recall behaviour more accurately than children.
The effect sizes for the association between SNAPA™ and dietary observation for % fat and FV intake were moderate to large(Reference Cohen60), with correlations ranging from 0·39 to 0·56, similar to those found in other studies(Reference Vereecken, Covents and Maes61–Reference Bingham, Gill and Welch63), although lower than those reported in a study evaluating a personal digital assistant (PDA)-based dietary assessment program (DietMatePro) also using lunchtime observation as a reference method(Reference Beasley, Riley and Jean-Mary57). Although % fat intake was overestimated, no substantial constant or proportional biases were identified.
In terms of physical activity, validity correlations between SNAPA™ and the reference method ranged from 0·27 to 0·44, and are comparable with other studies evaluating self-report physical activity assessment tools(Reference McMurray, Harrell and Bradley13–Reference Ridley, Olds and Hill16, Reference Albanes, Conway and Taylor64–Reference Welk, Dzewaltowski and Hill67). However, SNAPA™ underestimated minutes of MVPA per d compared with all minutes of MVPA derived from combined heart rate and accelerometry, and overestimated minutes of MVPA compared with combined heart rate and accelerometry when the 10 min bout criterion was applied. It may have been expected that there would be a higher agreement between the methods when comparing SNAPA™ with the combined heart rate and accelerometry data when the 10 min bout criterion was applied as, when completing SNAPA™, participants were instructed to complete activities carried out for at least 10 min. Although the mean difference between SNAPA™ and combined heart rate and accelerometry was lower when the 10 min bout criterion was applied compared with all minutes of MVPA measured, a proportional bias was identified. Physical activity may be more difficult to recall accurately, compared with dietary behaviours, as an estimation of duration and intensity of the activity is required in addition to a description of the activity. A potential source of error may be that activities can be clumped together with start and end points not actually corresponding with the activity being recalled. For example, a person cycling to work may define the start of that activity as the time they leave the house to the time they start work, which may be 30 min, although the actual time spent cycling was 25 min. Accelerometers and pedometers provide objective measurements of physical activity and are now affordable for most research projects. However, self-reported information on physical activity data is important for public health research as it adds context to the objective data and can capture activities that may not be captured well by objective monitors such as swimming and cycling. Combining SNAPA™ and an objective measurement of physical activity may be a promising approach and worth investigating in the future.
The biggest weakness identified through the food item agreement analysis, echoed by feedback from the participants, is the limited food item and activity options. As well as causing frustration for users, this limitation could also have an impact on the accuracy of the data collected. Expansion of the option lists, as well as grouping and searching strategies, will be explored and incorporated into future developmental work of SNAPA™. Another limitation of the tool is that it is currently designed to collect only data on key dietary and physical activity behaviours that are in line with current national public health recommendations for England. The assessment of other nutrients, food groups, dietary patterns and sedentary activities would also be useful. Our long-term planned programme of work includes the development of SNAPA™ to assess these additional behaviours.
There are limitations of the reference methods used in the present study, particularly with relation to measuring exact diet as this is extremely challenging because a true ‘gold standard’ method does not exist. The present study aimed to overcome the errors involved in comparing a self-report method against another self-report, albeit one is generally considered to be a ‘standard’ or ‘reference’ method (e.g. food records or 24 h multiple-pass recall diet interviews(Reference Thompson, Subar, Couston and Boushey68, Reference Willett69)). Although direct dietary observation is also a subjective method, it is carried out prospectively by trained observers, eliminating memory- and subject-associated biases, so it is considered a suitable reference method for the validation of self-reported methods(Reference Mertz70). It can be carried out on a relatively limited budget and does not require specialised equipment or facilities. However, direct dietary observation does have limitations, although these are minimal in a school setting since children are accustomed to eating lunch at a specified time during a school day (lunch break), in a dedicated location (dinner hall), and in the presence of adults (teachers and canteen staff). The large number of staff present during school lunchtimes limits the perceived novelty of new and additional personnel being present, and also allows for the use of blinding techniques (i.e. each child will be aware that they will be observed on some days but not all days, but they will not know which days they will be observed).
Finding an equivalent scenario in adult populations is challenging, but workplaces and further education institutions, where staff (and students) have scheduled lunch breaks and dedicated eating areas, offer a reasonable opportunity for direct dietary observation, although the set-up is more dynamic and less structured compared with school settings and not all of staff (and students) choose or are able to eat their lunch in dedicated eating areas. In addition, where participants consume pre-made meals brought from home it is difficult to identify (and quantify portion sizes for) hidden items such as spreads and fillings in sandwiches, sugar added to drinks and any fat added during cooking/preparation; children who eat packed lunches at school are sometimes excluded from direct observation studies because of this difficulty in determining the food content(Reference Baranowski, Islam and Baranowski10).
A major limitation of using dietary observation is that data are only collected for one eating occasion in the day and are not representative of a whole day's intake. While efforts were made to keep the eating occasion as normal as possible, participants were asked to eat in locations where researchers could observe a group of people; therefore, in many cases, participants were eating their lunch in different locations to where they would usually and, in some cases, with different people. In addition, it was often the case that the observations were carried out in small groups; therefore, participants were aware that they were being observed which may have caused some embarrassment and discomfort. All of these factors will have added emphasis to eating event, which may have enhanced recall by making the event more memorable. Usual eating behaviours may also have been affected as participants knew they were being observed.
The main reason for non-compliance with the Actiheart® monitor was adverse skin reactions to the electrodes. Anecdotal evidence provided by some of the participants suggested that changing the electrodes more regularly reduced itchiness and adverse reactions. A selection of electrodes, approved for use with the monitor by the manufacturers, were trialled with participants who reported adverse reactions to identify if any particular make or type was more or less acceptable.
In the present study, two participants could not complete SNAPA™ because they were using the latest version of Internet Explorer (version 8) with which SNAPA™ was incompatible (which also affected its use in other web browsers on the same personal computer, e.g. Mozilla Firefox). This problem highlighted the importance of retaining technical support when using Internet-based assessment methods to ensure compatibility with new technological developments. In addition, four other participants, with no reported IT issues, did not complete SNAPA™ during the study. Through anecdotal feedback received at the end of the study, it was clear that lack of computer skills and/or opportunities to use the Internet was a barrier to completing SNAPA™.
The present study relied on a volunteer sample which may not have been fully representative of the general population for which SNAPA™ is targeted. It was obvious that the study attracted some highly motivated individuals who were particularly interested in physical activity and sport and/or diet, including further education students completing a sport-based course. It was, therefore, expected that, overall, the study sample would report higher average levels of physical activity compared with the national (UK) average. However, in addition to the combined heart rate and accelerometry monitoring, participants also wore accelerometer monitors (Actigraph™ GT3X (uniaxial mode); Actigraph™, Pensacola, FL, USA) for 7 d during the study (FC Hillier and AM Batterham, unpublished results) and the accelerometry outcomes in this sample were comparable with the recent Health Survey for England data(Reference Craig, Mindell and Hirani71). Some participants did comment that the vouchers were the main incentive for them participating, so this approach may have been effective in attracting less motivated individuals. A range of individuals with different occupations were recruited for the concurrent validity study including those in academic, management, administration and manual roles. Participants, on average, were classified as overweight. Previous research has demonstrated that under-reporting is associated with increased BMI and body fat(Reference Briefel, Sempos and McDowell72–Reference Pryer, Vrijheid and Nichols76). However, larger sample sizes than recruited in the present study are required to explore agreement between SNAPA™ and the reference methods in subgroups by weight status and education. The majority of participants in the study were white British; therefore, further research is required before SNAPA™ could be used in more ethnically diverse populations. Despite the computer/Internet access issues experienced by a very small proportion of the participants, SNAPA™ appeared to be well received with few usability issues; however, the majority of participants were regular users of computers and the Internet (at home and at work).
Conclusion
SNAPA™ is a promising tool for measuring specific energy balance-related behaviours at a group level in adult populations, though at this stage, it is less accurate for physical activity behaviours than for the selected dietary behaviours. Further development and evaluation work is now required to address outstanding usability issues, and the generalisability and transferability of the tool across different population groups. The present development work will also include the (ongoing) updating of the program so that it is fit for purpose in terms of technology. We are also planning additional work to assess to what extent the simultaneous measurement of diet and physical activity provides added cognitive benefits that aid recall of each behaviour.
Acknowledgements
This study was funded through development funding provided by Durham University. The authors wish to thank all the participants of the present study, along with the workplace representatives and Scott Lloyd from NHS Stockton who assisted with the recruitment of workplaces. We would also like to thank David Cumbor for the programming aspects of the development of SNAPA™. Thanks goes to Liane Azevedo, Alisha Crayton, Wayne Douthwaite, Nicola Heslehurst, Sally McLure, Catherine Nixon, Claire O'Malley, Claire Pedley, Sarah Smith and Frances Thirlway for their assistance during the data collection phases. F. C. H., A. M. B., H. J. M. and C. D. S. contributed to the study design. S. C. developed the software. F. C. H. and A. M. B. performed the data analysis. F. C. H. drafted the manuscript. All authors contributed to the revised successive drafts of the manuscript and approved the final content. The authors declare that they have no conflicts of interest.
