The challenge of control groups in dietary research
Diet can impart favourable effects on health and disease risk, and can be used in the management of disease. Rigorous research design and methodology is essential in informing the precise influence of diet in each of these realms. The gold standard method for investigating the effectiveness of a therapeutic intervention (for example, drug, nutrient, food, dietary advice) is the randomised, double-blind, placebo-controlled trial. The design and conduct of drug trials is closely regulated by national and international bodies such as the Medicines and Healthcare products Regulatory Agency, the Food and Drug Administration and the European Medicines Agency. In contrast, guidelines on conducting clinical trials of dietary interventions (i.e. food or nutrient intervention, or dietary advice) do not exist.
Use of placebo controls is relatively straightforward in drug and nutrient trials as products (e.g. capsules, liquids or powders) can be developed that mimic the drug or nutrient without containing the active ingredient. However, placebo design presents a major obstacle in food or dietary advice trials, and this has contributed to a paucity of placebo-controlled trials investigating the effect of dietary interventions in healthcare. This review evaluates the types of controls used in dietary trials and presents the advantages and disadvantages of each using examples from the literature. Other relevant issues such as blinding, adherence and biases will also be discussed. An example of the development of a novel placebo (sham) diet for use in an irritable bowel syndrome (IBS) trial is provided, that has until now not been detailed and will prove beneficial for future placebo-controlled dietary advice intervention trials. A glossary of terms is provided in Table 1.
Table 1. Glossary of terms relevant in dietary intervention trials
*Some definitions adapted from( Reference Elsenbruch and Enck 51 ).
Controls, placebo and blinding in dietary research
Benchmarking the physiological and clinical effects of an intervention group against a control group is essential for providing unambiguous evidence that the intervention is superior to not having the intervention. The effects of a drug, nutrient, food or dietary advice can be explained by its pharmacological, toxicological and/or nutritional properties. In addition, the effects can also occur due to the interaction between the individual, the prescriber (or the researcher) and the drug, nutrient, food or dietary advice creating the placebo response( Reference Harris and Johns 1 ). In addition to these, food interventions or dietary advice can exert placebo effects that are influenced by previous exposure, expectation and response to particular foods, personal and cultural beliefs regarding food and diet, sensory satisfaction, taste preferences and the support and reassurance of the dietitian or nutritionist providing the advice. The response to food intervention or dietary advice is therefore the sum of its impact on nutritional physiology/biochemistry and the complex factors impacting the placebo response( Reference Yao, Gibson and Shepherd 2 ), further highlighting the importance of placebo control in trials of these interventions. Bearing this in mind, there are a number of possibilities when considering the use of controls in dietary intervention studies.
Uncontrolled trials
Uncontrolled trials of food or dietary advice evaluate the effect of an intervention without a control group, and conclusions are based on the paired changes that occur within the intervention group only. Although uncontrolled trials fall outside the recommendations by The International Conference on Harmonization guidelines( 3 ), it has been estimated that one-third of all clinical trials are uncontrolled( Reference Saccà 4 ). This approach is subject to limitations based upon the lack of opportunity to compare against a group not receiving the intervention. Therefore, it is impossible to exclude that any changes occurring over the duration of the intervention would not have occurred had the intervention not taken place, although inter-subject variation is controlled for when undertaking paired comparisons.
Despite these limitations, uncontrolled trials are generally easy and cheap to conduct and are appropriate for the evaluation of novel, untested, dietary interventions. They are therefore useful for exploratory studies that inform the design of larger controlled studies. Uncontrolled trials may be appropriate in patient groups in whom there are ethical risks of not providing an intervention, such as those at nutritional risk, e.g. oncology( Reference Fine, Segal-Isaacson and Feinman 5 ), paediatrics( Reference Chumpitazi, Hollister and Oezguen 6 ) or in diseases with rapid or fatal progression( Reference Saccà 4 ). Uncontrolled trials may also be appropriate in extremely rare conditions where a sufficient sample size for both an intervention group and a control group is impossible. Therefore, although uncontrolled trials are a source of only very weak clinical evidence( Reference Saccà 4 ), they may be appropriate in some isolated cases. Finally, although the placebo effect is impossible to measure in uncontrolled trials, and may be particularly strong for subjective endpoints such as self-reported symptoms, it could be argued that uncontrolled trials suitably represent the effects of dietary intervention achievable in real-life, as the placebo effect is commonly applied as part of many therapeutic interventions in nutrition and dietetic practice( Reference Fässler, Meissner and Schneider 7 ).
Controlled trials
There are four common types of controls utilised in intervention trials of nutrient, food or dietary advice. The following section will describe these approaches and address the advantages and disadvantages of each.
No treatment, wait list, external and historical controls
The first type of control is the no treatment control, in which participants do not receive the intervention, nor do they receive a placebo or comparative intervention. Despite having no intervention or placebo, it is important that participants in the no treatment control group are evaluated using the same outcome measures at the same timepoints as those receiving the intervention to lead to a comparable Hawthorne effect between groups (the effect of measurement on response to measurement; Table 1). Although this approach could be considered superior to the uncontrolled trial, one key issue is that participants are unblinded, i.e. they have knowledge of their treatment assignment. This can result in significant expectation bias in the intervention group (i.e. the expectation of benefit could lead to more favourable outcome in those receiving treatment), which also exists in uncontrolled trials. However, there is a risk of uneven expectation bias between the no treatment control group (i.e. the expectation of lack of benefit could lead to less favourable outcome) and the intervention group. This may be particularly important in trials of treatments with subjective outcomes (e.g. quality of life, symptom reporting).
A special type of no treatment control that is commonly used in dietary intervention studies is a wait-list control (i.e. patients waiting for a routine appointment) who present a convenient no treatment control population( Reference Harvie, Chisholm and Schultz 8 – Reference Sainsbury, Mullan and Sharpe 10 ). The advantage of this is the ethical benefit of patients obtaining treatment who are seeking care. However, the disadvantage is that these patients are not randomised to this group, leading to a risk of allocation bias. Furthermore, at least according to behavioural research, the use of wait-list controls can overestimate treatment effect, as they change less than expected for individuals who are concerned about their behaviour( Reference Cunningham, Kypri and McCambridge 11 ). However, other evidence suggests the expectation of future intervention in wait-list controls could also lead to unwanted improvement in endpoints, essentially leading to an underestimation of effect in the treatment group. For example, wait-list controls in energy restriction studies have lost weight( Reference Austel, Ranke and Wagner 9 ), in coeliac adherence studies they have reported improvements in quality of life( Reference Sainsbury, Mullan and Sharpe 10 ) and in IBS they have reported symptom improvements( Reference Staudacher, Lomer and Anderson 12 ). Despite this, no treatment controlled trials, including those utilising wait-list controls, are appropriate for trials with objective outcomes that might be less likely to respond to biases (e.g. the effect of a dietary intervention on blood cholesterol) and in trials where blinding is difficult( 3 ).
External or historical control groups utilise participants external to the trial. For example, in studies using hospitalised patients, historical data are collected for the external group from medical records. Of course, this can potentially be limited by the level of detail that can be acquired from previously documented records. Externally or historically-controlled trials are generally also hazardous as it can never be guaranteed that the controls and the treatment group are truly sampled from the same population. Interestingly, untreated historical-control groups are reported to have worse outcomes than concurrent control groups, probably reflecting a selection bias( 3 ). Overall, this approach is generally not recommended other than in situations where no other control group is available( 3 ).
Active comparator groups
A third type of control is an active comparator group. In most instances where a dietary intervention is compared with another active intervention, the comparator group (for it is no longer an inactive control group) receives a standard treatment. For example, in a food intervention study investigating the effect of prunes on constipation, the treatment group were compared with an active comparator group in which another food was consumed, i.e. psyllium( Reference Attaluri, Donahoe and Valestin 13 ). In dietary advice studies, an active control might receive dietary advice that is known to have some established efficacy and is used as current best practice. For example, standard low fat dietary advice has been compared with Mediterranean dietary advice in a large multicentre trial investigating the effect of diet on cardiovascular risk (PREDIMED)( Reference Estruch, Ros and Salas-Salvadó 14 ). In Crohn's disease, the use of whole-protein enteral nutrition has been used as an active comparator when evaluating the effect of elemental enteral nutrition on achieving remission( Reference Verma, Brown and Kirkwood 15 ), and standard advice to reduce fibrous foods in active Crohn's disease was used as an active comparator with a novel low microparticle diet( Reference Lomer, Harvey and Evans 16 ). Standard nutritional counselling has also been compared with enteral nutrition for post-surgical patients with gastrointestinal cancer( Reference Gavazzi, Colatruglio and Valoriani 17 ). In IBS, dietary advice considered the best practice at the time has been used as an active comparator when evaluating the effect of a diet low in fermentable carbohydrates (low FODMAP diet)( Reference Staudacher, Whelan and Irving 18 , Reference Bohn, Storsrud and Liljebo 19 ).
Standard treatment might also consist of standard physician care, for example when evaluating the effect of dietary intervention on weight and CVD risk factors( Reference Green, Anderson and Cook 20 ). While representing real life clinical practice, standard physician care may be limited by differing follow-up frequency between groups resulting in an uneven Hawthorne effect. For example, in the study of dietitian-led team care incorporating Dietary Approaches to Stop Hypertension advice v. standard physician care on cardiovascular risk, the active comparator group were asked to see their physician for follow-up care with no other follow-up throughout the 6-month duration of the trial( Reference Green, Anderson and Cook 20 ).
Trials with active comparators are used to establish the effect of a new dietary intervention as equivalent or superior to current practice (dietary or otherwise) and might be considered more ethically acceptable as all participants receive active treatment at the outset. This is particularly relevant in trials of patients with serious morbidity( Reference Temple and Ellenberg 21 ). Interestingly, physicians are more likely to recommend participant involvement in, and are more likely to prescribe drugs tested in, trials with active comparators than placebo-controlled trials( Reference Halpern, Ubel and Berlin 22 ), and patients prefer involvement in active comparator trials over placebo-controlled trials when evaluating drug efficacy( Reference Welton, Vickers and Cooper 23 ); whether this is also true for dietary trials is unknown.
One problem with an active comparator trial is the difficulty of applying homogenous advice across all the participants in the comparator group, particularly those that utilise standard care. For example, advice to implement a high-fibre diet in the active comparator group will likely vary from patient to patient according to habitual fibre intake and dietary preference. This is also commonly the case when patients in an active comparator group receive standard medical care. Another issue that has arisen is when final evaluation reveals the composition of the intervention diet is not sufficiently different from the active comparator diet; a proposed point of weakness of the PREDIMED trial( Reference Appel and Van Horn 24 ). Poor adherence of participants within the active comparator group can also be a challenge.
Blinding the active comparator diet can be difficult, which leads to a risk of uneven expectancy distribution and reduces internal validity of the trial. This may be particularly so where the active comparator is ‘standard advice’ that has been commonplace in clinical practice for some time (e.g. low-fat dietary advice for CVD). Previous exposure to ‘standard advice’ should be considered as an exclusion criterion in these situations to help minimise unblinding.
Placebo controls
The fourth and final example of a control is the placebo control. This is a dummy or inert treatment that appears as identical as possible to the intervention of interest. For example, in a food intervention study investigating the bone protective effect of dried plums, individuals were compared with a placebo control group, which was allocated a different food with no bone protective properties, i.e. apple( Reference Hooshmand, Brisco and Arjmandi 25 ). The placebo-controlled trial is considered the most robust of clinical trials. Randomisation and double blinding enable minimisation of subject bias and observer bias( 26 ). Where disease risk factors or disease endpoints are of interest, placebo controls also specifically help to account for natural progression of disease that would occur had the intervention not been prescribed( 26 ). This type of control is generally easily accomplished in drug trials as well as in nutrient or nutraceutical supplementation studies. For example trials evaluating prebiotics( Reference Silk, Davis and Vulevic 27 , Reference Clarke, Green-Johnson and Brooks 28 ) or specific nutrients( Reference Yusuf, Dagenais and Pogue 29 , Reference Sepehrmanesh, Kolahdooz and Abedi 30 ) can incorporate a placebo control in the form of a capsule or sachet produced to replicate the intervention in appearance and taste.
Conducting placebo-controlled trials in food interventions or dietary advice interventions is, however, significantly more challenging. For example, there is a multitude of studies that investigate the effect of whole-diet alterations (i.e. multiple contemporaneous alterations to the diet) on disease endpoints such as Mediterranean diet for improving cardiovascular health, the Atkins diet and Nordic diet for modulating weight, or the low FODMAP diet for managing symptoms of IBS. However, placebo-controlled trials of whole diets are extremely rare largely because of the difficulties firstly of using a placebo control that does not significantly alter the outcome of interest and secondly of maintaining blinding.
There are two methods by which a successful placebo control can be applied in studies of whole-diet alteration trials. Firstly, feeding studies can be undertaken that administer all food and fluid to participants in the trial. The placebo control in feeding studies can be created bespoke for the purposes of the trial. It is developed to be inert in nature, and is nutritionally matched in all aspects except for the active component being investigated( Reference Barrett, Gearry and Muir 31 ). There is, therefore, a lower risk of controls experiencing improvements in the outcome of interest (e.g. plasma cholesterol or IBS symptoms) compared with active comparator trials. Furthermore, placebo controls in feeding studies can be created to be almost indistinguishable to the intervention. For example, in a placebo-controlled crossover feeding study that evaluated gluten-free, casein-free diets in autism, most parents of children could not distinguish the placebo diet from the experimental diet( Reference Elder, Shankar and Shuster 32 ). In this feeding study, all meals and snacks were prepared and provided to patients for 12 weeks, and diets were individually adapted based on food preferences. With extreme effort both the patient and the investigator can be blinded to both diets. However, feeding studies are burdensome for the researcher in terms of time and economic costs and are therefore often short-term (e.g. <1 week). These factors, in addition to the artificial nature of total food provision, means that feeding studies have limited external validity as in routine clinical practice patients are not fed a therapeutic diet in a controlled environment.
Secondly, it is possible to conduct placebo controlled studies of whole-diet alterations using dietary advice. Dietary advice studies have the advantage over feeding studies of being representative of what is achievable in real life settings. Typical difficulties encountered in everyday practice, such as non-adherence( Reference Herrera, Moncada and Defey 33 , Reference Glanz 34 ) and the potential for information to be misconstrued on transmission from practitioner to the patient, are replicated in these types of trials. As well as generally being less burdensome in terms of cost and time these types of trials could be argued to have greater clinical validity than feeding studies.
A placebo control can be incorporated into dietary advice studies by using a re-supplementation control, where the same dietary advice is given to participants in both the intervention and control groups, followed by re-supplementation of the excluded food component to the placebo group. One study has taken advantage of this study design in order to investigate the impact of the low FODMAP diet on symptoms and immune function( Reference Hustoft, Hausken and Ystad 35 ). Following a low FODMAP diet run-in period for all patients, the placebo group received fructan supplementation in order to increase FODMAP intake back to habitual levels, while the treatment group received placebo sachets (and thus were on a low FODMAP diet). A similar design was applied in a study investigating the effect of gluten supplementation on gastrointestinal symptoms and fatigue in participants with self-reported gluten intolerance. After a 2-week run-in period of a gluten free diet, the placebo group received gluten in order to normalise gluten intake, whereas the treatment group received placebo (and thus were on a gluten free diet)( Reference Biesiekierski, Peters and Newnham 36 ). These types of re-supplementation studies present a novel way of incorporating a placebo control in the evaluation of a dietary advice intervention. Re-supplementation studies are only possible if the dietary components of interest are available in supplemental form, and it assumes the components exert the same biological effects when supplemented compared with when consumed in the diet.
Alternatively, dietary advice trials can be placebo-controlled with the application of sham dietary advice. In this case, dietary advice is provided that is formulated to modify food intake without altering intake of nutrients or the specific food component being investigated. There is a paucity of research studies utilising sham diets, probably because of the difficulties of formulating and administering such a diet.
There are at least seven sham-controlled dietary advice randomised controlled trials investigating the effect of whole-diet interventions reported in the literature (Table 2)( Reference Mitchell, Hewitt and Jayakody 37 – Reference Staudacher, Lomer and Louis 44 ). Most evaluate the effect of an exclusion diet in gastrointestinal conditions, are of considerable size and are up to 16 weeks in duration, a length of time which broadly reflects clinical practice. The rationale for the choice of foods included in the sham diet in these studies is based on self-reported tolerance( Reference Carroccio, Mansueto and Morfino 39 ), the patient's usual diet( Reference Dalvit-McPhillips 40 ), is relatively arbitrary( Reference Mitchell, Hewitt and Jayakody 37 , Reference Atkinson, Sheldon and Shaath 38 , Reference Bentz, Hausmann and Piberger 41 , Reference Gunasakera, Rajendran and Mendal 42 ) or excludes another dietary component( Reference Lomer, Grainger and Ede 43 ). For example, one study in patients with Crohn's disease reduced microparticle intake (inorganic calcium, food additives titanium dioxide and silicates) and compared it with a group that were provided sham dietary advice that included avoidance of the food additives sulphates and sulphur dioxide( Reference Lomer, Grainger and Ede 43 ).
Table 2. Sham diet-controlled dietary advice randomised controlled trials (RCT)
Overall, very little information is provided on the design of the sham diet, and nutrient intake is not routinely measured to confirm its equivalence to the treatment. This is imperative in dietary studies where multiple dietary factors have potential to impact on endpoints (e.g. carbohydrate, protein and fat in CVD)( Reference Mok, Haldar and Lee 45 , Reference Thorning, Raziani and Bendsen 46 ). Although collinearity is almost inevitable in dietary studies (e.g. altering intake of carbohydrate will lead to a change in the intake of other nutrients), confirmation that there is a clear difference in intake of the dietary component of interest between the sham diet and intervention diet is vital. There is a recommendation that the number of foods removed in a sham exclusion diet be comparable with the intervention diet( Reference Yao, Gibson and Shepherd 2 ); however, detailed guidance for development and implementation of sham diets is scarce.
Design and development of a sham diet for use in a placebo-controlled low FODMAP dietary advice trial
Here, the design and development of the first ever sham diet for use in a low FODMAP dietary advice randomised controlled trial is reported, in order to illustrate how the challenges described can be overcome, and to provide practical recommendation for sham diet development in other settings. The low FODMAP diet is an exclusion diet which has demonstrated effectiveness in reducing symptoms such as abdominal pain and bloating in IBS( Reference Marsh, Eslick and Eslick 47 , Reference Staudacher, Irving and Lomer 48 ). It requires restriction of a number of short-chain carbohydrates that are ubiquitous throughout the human diet, and a majority of evidence of its effectiveness is based on dietitian-led dietary advice provided to participants.
A number of criteria for the sham diet were developed in order to ensure its integrity as a placebo control for the low FODMAP diet. These criteria were developed as an approach to interpreting fundamental principles in the use of placebos (their similar presentation as the intervention to facilitate blinding, physiologically inert with regards to the outcome of interest), but specifically tailored to dietary intervention studies (Table 3). These criteria in specific relation to the trial of the low FODMAP diet are: (1) to be a convincing exclusion diet in order to encourage blinding that it is actually a placebo; (2) to contain a similar number of specialist new products as the low FODMAP diet; (3) to restrict an equivalent number of foods compared with the low FODMAP diet; (4) to take the same amount of time for shopping and involve the same level of adaptation when preparing meals as the low FODMAP diet; (5) to take the same amount of time and comprehension to teach as the low FODMAP diet; (6) to be feasible to follow; (7) to modify dietary carbohydrate sources (for ethical purposes patients were informed that the unnamed active intervention diet involved altering carbohydrate intake); (8) to alter dietary intake but maintain FODMAP intake; and (9) to not alter fibre intake, which may impact on symptoms( Reference Moayyedi, Quigley and Lacy 49 ). These criteria have been modified for application across all types of dietary advice trials and although these generic criteria for design of a sham diet have not been validated in trials, they provide practical approaches to facilitate blinding and limit the physiological impact of the sham diet (Table 3).
Table 3. Important criteria for the development of a sham diet that may improve blinding and maintain the placebo nature of the diet
The sham diet was designed following a systematic selection of foods to be included (suitable foods) and excluded (unsuitable foods). Suitable and unsuitable food lists for the low FODMAP diet were used as a starting point for creation of suitable and unsuitable food lists for the sham diet, in order to create some restriction (criterion 3), while neither increasing nor decreasing fructan (criterion 8) or fibre intake (criterion 9). Considering that many exclusion diets alter grain intake, some grains were restricted to give the impression that the sham diet was a true exclusion diet (criterion 1), to increase the burden of teaching (criterion 5) and following the sham diet (criterion 4), to focus the sham diet on carbohydrate intake (as does the low FODMAP diet), which was referred to in the patient information sheet (criterion 7), and to necessitate the inclusion of new food products in the diet (criterion 2). Some regularly-consumed high FODMAP foods were allocated to the suitable list in order to maintain FODMAP intake during the sham diet (criterion 8). For example, approximately half of the fruit and vegetables considered suitable on the low FODMAP diet were assigned to the unsuitable list on the sham diet and vice versa (criterion 3), while dairy products were allocated to the suitable list, to ensure lactose intake was maintained on the sham diet (criterion 8). Next, the habitual diet of individuals with IBS was examined from a previous study( Reference Staudacher, Lomer and Anderson 12 ) and the top 10 % of foods contributing to energy and carbohydrate intake were allocated as being suitable on the sham diet in order to promote feasibility (criterion 6) and maintenance of nutrient intake (criteria 8, 9). Finally, the number of unsuitable foods on the sham diet was confirmed as being approximately equivalent to that of the low FODMAP diet (criterion 3).
Implementing and evaluating a sham diet
Dietary counselling in sham-controlled trials should be equivalent in duration for all participants, and ideally counselling should be provided to all participants by the same researcher. Access to written dietary resources has been associated with greater likelihood of response to lifestyle interventions( Reference Swinburn, Walter and Arroll 50 ). Therefore if this type of information is to be provided, both intervention and sham diet groups should receive a similar level of written support, i.e. the general format and length of the resources should be identical (criterion 5).
The evaluation of a sham diet should include assessment of its achievement of the criteria described in Table 3, and this can be performed in a variety of ways. One approach is to undertake a pilot study whereby participants are advised to follow the sham diet and undertake a dietary assessment at baseline (habitual diet) and during the sham diet (criteria 8, 9). An acceptability questionnaire can evaluate feasibility and other important outcomes (criteria 4, 6), as well as assessment of blinding (criterion 1). The sham diet can also be evaluated as part of the final randomised controlled trial, and this can be undertaken both during the trial (i.e. an a priori interim analysis) and at the end of the trial (i.e. final analysis). If an interim analysis of a sham diet is undertaken, then it should be performed late enough so that sufficient numbers can be included in the analysis, but early enough in the case that the sham diet requires alteration. If changes to the sham diet are required this may require contact with the body providing ethical approval, and alterations should be carefully recorded and reported in the subsequent publication. In regards to the final analysis, evaluation of changes in dietary intake between baseline and the sham diet and between sham and the intervention diet should be reported in any publication to confirm the placebo nature of the sham diet. Interim and final analyses must be conducted by an investigator who is blinded to the dietary allocation, in order to prevent researcher bias during dietary coding. Clearly, dietary assessment should use gold standard methods where possible.
Conclusions
High-quality placebo-controlled evidence for food or dietary interventions is vital for verifying their role in optimising health or for the management of disease. This is especially important where the benefits of dietary intervention are coupled with potential safety implications such as compromising nutrient intake. The challenges with conducting placebo-controlled research in dietary trials are acknowledged. Sham diets are one approach of implementing placebo controls in dietary advice trials. Any new sham diet should be rigorously designed, implemented and tested as described. Feasibility, preservation of blinding and maintaining intake of the dietary component being investigated in the treatment group are major priorities when designing a sham diet, which we propose can be addressed with careful consideration of the recommendations outlined.
Financial Support
H. S. was funded by the National Institute for Health Research.
Conflicts of interest
K. W. and M. C. L. are co-inventors of a mobile application to assist patients in following the low FODMAP diet.
Authorship
H. S. and K. W. conceived the theme of the manuscript; H. S. and K. W. wrote the manuscript; K. W., M. L. and P. I. edited the manuscript. All authors approved the final manuscript prior to submission.
