Pitfalls in the use of randomised controlled trials for fish oil studies with cardiac patients

Abstract

Randomised controlled trials (RCT) examining the effects of fish oil supplementation on cardiac outcomes have yielded varying results over time. Although RCT are placed at the top of the evidence hierarchy, this methodology arose in the framework of pharmaceutical development. RCT with pharmaceuticals differ in important ways from RCT involving fish oil interventions. In particular, in pharmaceutical RCT, the test agent is present only in the intervention group and not in the control group, whereas in fish oil RCT, n-3 fats are present in the diet and in the tissues of both groups. Also, early phase studies with pharmaceuticals determine pharmacokinetics and pharmacodynamics to design the dose of the RCT intervention so that it is in a predicted linear dose–response range. None of this happens in fish oil RCT, and there is evidence that both baseline n-3 intake and tissue levels may be sufficiently high in the dose–response range that it is not possible to demonstrate a clinical effect with a RCT. When these issues are considered, it is possible that the changing pattern of fish consumption and fish oil use over time, especially in cardiac patients, can explain the disparity where benefit was observed in the early fish oil trials but not in the more recent trials.

Randomised controlled trials of n-3 fats in cardiac patients

The first published randomised controlled trials (RCT) in this area began recruitment in 1983 and reported a beneficial effect of fish or fish oil consumption on all-cause mortality and IHD mortality^{( 1 )}. Similarly, the next published RCT, which began recruitment in 1993, reported a beneficial effect of fish oil consumption on all-cause mortality, coronary death and sudden death^{( 2 )}. These findings are in agreement with the results of prospective cohort studies that showed that increased fish consumption is associated with decreased CHD mortality^{( 3 , 4 )}. However, RCT that recruited participants having CHD risk factors or existing CHD during the years 2002–2007 have reported no effect of fish oil supplementation^{( 5 – 8 )}. This review examines the temporal changes during these years that could have affected the suitability of RCT to be the most appropriate methodology for examining the effects of fish oil supplementation on cardiac outcomes. It also contrasts the nature of RCT with fish oil in cardiac patients with that of RCT in drug development, which is the origin of modern RCT methodology.

Randomised controlled trials have been placed at the top of the evidence hierarchy

The first publication of a blinded RCT in medicine is the Medical Research Council study on streptomycin in tuberculosis patients in 1947–1948. By 1970, the Food and Drug Administration required that evidence for drug efficacy be based on controlled clinical trials with by-group intention-to-treat analyses^{( 9 )}. The acceleration of drug use in therapy with approximately 1200 drugs being approved by the Food and Drug Administration in the period 1950–2008^{( 10 )} has undoubtedly reinforced the primacy of RCT for providing the proof of efficacy of an experimental treatment. This position has not been shaken by the generally accepted arguments that clinical drug trials have poor external validity^{( 11 – 13 )}, that the use of composite endpoints, which is very common, can provide statistical significance that is driven by the most prevalent but often least important component^{( 12 , 14 )}, and that in industry-funded trials the results and conclusions overwhelmingly favour the sponsor's product^{( 15 )}.

Differences between trials with pharmaceuticals and those with fish oil

RCT that examine the effects of n-3 fats from fish and/or fish oil on cardiovascular outcomes will share many of the limitations mentioned above. However, pharmaceutical RCT have at least one characteristic not shared by n-3 RCT. In drug trials, the test agent is administered to those in the experimental group, but not to those in the control group. In fish oil trials, the test agents, which are EPA and DHA, are present in both experimental and control groups.

In a RCT with a new drug, one group will receive the drug, usually at a dose determined from prior pharmacokinetic and pharmacodynamic studies. The control group will not receive the study drug and the blood concentrations of the study drug in the control group will be zero. In addition, it is standard practice to prohibit concomitant use of drugs with the same or similar action to the test drug. Thus, RCT with pharmaceuticals are designed to maximise the differences between the treatment arms and thereby optimise the chance of detecting an effect if it exists. This is in contrast to all RCT that have examined the effects of dietary fish and/or fish oil consumption on cardiac outcomes because the non-study intake of n-3 fats cannot be prevented.

Non-study intake of n-3 fats from fish and fish oil in randomised controlled trials

In any RCT that examines the effects of fish oil supplementation on cardiac outcomes, it would be advantageous to include participants with (1) a history of low n-3 intake at baseline and (2) zero-to-low non-study n-3 intake during the study.

A sample of seven of the larger RCT that examined the effects of fish or fish oil on cardiac outcomes were selected to span the period from the first large study that commenced in 1983 to a study that finished in 2012^{( 1 , 2 , 5 – 8 , 16 )}. The average sample sizes of the seven RCT selected for this discussion were 3587 and 3581 for the fish oil and control groups, respectively. A recent meta-analysis^{( 17 )} included studies in which six of the seven RCT cited herein were reviewed as well as studies with, for example, group sizes ranging between 18 and 60^{( 18 – 20 )}. Therefore, our sample is not an exhaustive list such as might be used to examine clinical outcomes in a meta-analysis. It is used to demonstrate only the points on trial design that are the topic of this discussion. Of these seven RCT, only three paid attention to baseline exclusions relevant to n-3 intake. These were exclusion of those already taking fish oil supplements and unwilling to discontinue their use^{( 5 , 6 )} and exclusion of those intending to eat the intervention diet^{( 1 )}. The other four RCT had no exclusions relevant to baseline or continuing non-study n-3 intake. It is not possible to prevent n-3 intake in the control group during the study. Participants may be instructed not to consume non-study n-3-containing oils, but the intake of EPA+DHA from fish is poorly controlled. None of the reports of the seven RCT indicated that the control group was instructed not to eat fish, to limit fish intake or even not to use n-3 supplements. In one study, 76 % of the participants had consumed at least one fish meal per week, with 33 % consuming at least two fish meals per week^{( 8 )} (Table 1). In only one study, it was stated that the use of non-study n-3 supplements was ‘discouraged’, although it was also stated that ‘We made no study-specific dietary recommendations pertaining to consumption of fish or other marine products’^{( 5 )} (Table 1). Non-study n-3 intake could decrease the difference in n-3 status between the intervention and control groups. One of the studies, known as Diet and Reinfarction Trial-2 (DART-2) in which the participants were recommended to eat two meals of oily fish per week, reported that fish intake at the end of the follow-up period was 47·7 g/d in the fish advice group and 39·4 g/d in the no-fish advice group^{( 21 )}. These intakes were statistically significantly different, but such a small difference is unlikely to be functionally significant.

Table 1 Advice or comments relevant to non-study n-3 intake in randomised controlled trials with fish or fish oil

DART, Diet and Reinfarction Trial; GISSI-P, GISSI-Prevenzione trial; MI, myocardial infarction; ORIGIN, Outcome Reduction with Initial Glargine Intervention.

The RCT under discussion recruited over a period spanning from 1983 to 2007 (Table 1). The start of recruitment falls into two periods, namely 1983–1993^{( 1 , 2 , 16 )} and 2002–2004^{( 5 – 8 )}. The use of non-study fish oil supplements may not have been a substantive problem in the early trials, but it is more likely that non-study fish oil or increased fish intake may have occurred in the later trials as community and food industry awareness of n-3 fats increased and the availability of fish oil supplements increased. The advice given by the American Heart Association in 1961 made no distinction among different types of polyunsaturates^{( 22 )}. In the Nurses' Health Study and the Health Professionals Follow-up Study, which surveyed participants' supplement intake from 1980 to 1986, respectively, no questions on fish oil intake were asked until 1988. In 1990, 1·6 and 3·3 % of the female and male participants, respectively, were taking a fish oil supplement, and by 2006, these values were 18·1 and 22·2 %^{( 23 )}.

By 2000, the American Heart Association recommended that ≥ 2 fish meals per week be consumed, and by 2002, the American Heart Association recommendation included fish and/or fish oil supplementation for those with CHD^{( 24 , 25 )}. It is difficult to determine when fish oil came into community use to an extent that cardiac patients similar to those participating in these trials were likely to use it. However, it is presumed that the production of refined fish oil for human consumption followed demand. The change in uses of fish oil in the last 50 years with mainly hardened oils being used for margarines and negligible production of refined oils for human n-3 consumption in 1960 suggests that before 1990 the use of fish oil supplements was negligible and certainly not at levels observed today^{( 26 )} (Table 2).

Table 2 Change in the uses of fish oil*

* Adapted from Shepherd & Jackson^{( 26 )}.

Importation of fish oil into the USA accelerated after 2000^{( 27 )} along with the changing advice of the American Heart Association in 2000^{( 24 )}. Thus, during the period of recruitment and follow-up in the recent trials, ranging from 2002 to 2012^{( 5 – 7 )}, fish oil supplements had become available, which makes it more likely that they were being used by cardiac patients such as those participating in the trials (Fig. 1).

Fig. 1 Quantity of n-3 oils imported into the USA (adapted from Bimbo^{( 27 )}).

Thus, during the period of commencement of these RCT, starting in 1983 and extending to 2007, it may have become increasingly difficult to recruit subjects with low baseline n-3 intake from marine sources and to maintain low n-3 intake from marine sources in the control group for the trial duration unless specific instructions were given and reinforced during the trial.

Non-study intake of n-3 fats from non-marine sources in randomised controlled trials

The n-3 fatty acids, EPA and DHA, are present not only in fish and fish oil, but also in red meat, eggs and dairy products. Although EPA and DHA are present in much higher quantities in fish than in red meat and poultry, almost half of the average long-chain n-3 PUFA intake for adult Australians originates from food other than fish, and this reflects the 6-fold higher intake of red meat and poultry compared with that of fish and seafood^{( 28 )}. In addition, the long-chain n-3 PUFA can be synthesised from dietary α-linolenic acid (ALA), which is present in vegetable oils such as rapeseed and soyabean and in smaller amounts in leafy green vegetables, although conversion is limited. Dietary ALA is capable of modestly increasing tissue EPA levels in short-term studies, although most studies have not reported increases in DHA levels as a proportion of total fatty acids^{( 29 )}. However, radiotracer studies have shown that dietary ALA can be a precursor of DHA in men and to a greater extent in women^{( 30 )}, and this is supported by the presence of plasma DHA in vegans at levels only slightly lower than those present in omnivores^{( 31 , 32 )}. Thus, even if trial subjects do not take fish oil supplements or eat fish, they will have EPA and DHA in their tissues, which originate from dietary ALA, EPA and DHA in omnivores and dietary ALA in vegans.

Tissue EPA and DHA

Erythrocyte EPA+DHA levels are closely correlated with human myocardial EPA+DHA levels^{( 33 )}. Whole-blood or erythrocyte EPA+DHA levels are also correlated with the risk of sudden cardiac death and primary cardiac arrest^{( 34 , 35 )}. A correlation between CHD mortality and erythrocyte EPA+DHA levels, the ‘omega-3 index’, has been observed in cohort, case–control and intervention studies^{( 36 )}. Thus, the biochemical and physiological effects of dietary EPA and DHA are most probably due to their levels in the target tissue, which, for prevention of arrhythmias, is the myocardium.

There is an implicit expectation that the randomised allocation will create two distinct groups with regard to both n-3 intake and subsequent tissue levels of EPA+DHA. This is probably not the case in part due to non-study n-3 intake as discussed above and also due to non-linear dose–responses and to variability from unexplained non-dietary sources. Data from the Cardiovascular Health Study cohort showed a non-linear relationship between intake and plasma phospholipid EPA+DHA levels, with the steepest dose–response being observed at intakes up to approximately 400 mg/d and smaller increases in plasma EPA+DHA levels occurring thereafter^{( 37 )}. Modelling of data from the Framingham Offspring cohort indicated that the dietary intake of EPA+DHA and fish oil supplementation accounted for only 25 and 15 %, respectively, of the variation in erythrocyte EPA+DHA levels^{( 38 )}. This is supported by intervention studies that have shown up to 10-fold variation in plasma phospholipid or erythrocyte EPA levels in healthy volunteers given purified EPA and substantial variation in erythrocyte EPA+DHA levels only partially explained by the dose of fish oil^{( 39 – 41 )}.

Thus, a RCT with a fish oil intervention may not have distinct groups in relation to tissue EPA+DHA content, and this could be due to a combination of non-study n-3 intake and variability in tissue levels that is related to factors other than n-3 intake, both of which have been highlighted by others^{( 42 – 44 )}.

This outcome is illustrated by examining erythrocyte EPA+DHA levels in subjects who participated in a RCT that examined the effects of fish oil on post-operative atrial fibrillation after cardiac surgery^{( 45 )}. Effort was made to recruit subjects with a low erythrocyte EPA+DHA content at baseline. Potential subjects were excluded if they had consumed ≥ 1 fish meal per week or they had taken fish oil supplements. Most of those excluded from the RCT agreed to be followed in an observation group. The RCT intervention was oral fish oil containing 4·5 g/d of EPA+DHA for 3 weeks before the scheduled date of surgery. The control group consumed Sunola oil, a monounsaturated sunflower oil containing no measurable EPA or DHA^{( 45 )}. The fish oil group had significantly increased mean erythrocyte EPA+DHA levels at the time of surgery than at the baseline and compared with the control group at the time of surgery (Table 3).

Table 3 Erythrocyte EPA+DHA levels in a randomised controlled trial of fish oil intervention for post-operative atrial fibrillation after cardiac surgery^{( 45 )} (Mean values, standard deviations and number of participants)

^a,b,cMean values with unlike superscript letters were significantly different (P< 0·05; linear mixed-effects model with post hoc Tukey's test).

* Consumed monounsaturated oil for 3 weeks before surgery.

† Consumed fish oil concentrate supplying 4·5 g/d of EPA+DHA for 3 weeks before surgery.

‡ Excluded from the trial due to the consumption of one or more than one fish meal per week or fish oil supplementation – agreed to be followed.

However, the scatter plots demonstrated the substantial overlap in EPA+DHA values between the control and fish oil groups after the intervention, i.e. at the time of surgery (Fig. 2). In fact, the upper 50 % of the control subjects and the lower 50 % of the fish oil subjects could be in the alternate group at the time of surgery. Also, subjects in the observation group, who were those who did not meet the inclusion criterion of consuming ≤ 1 fish meal per week or not ingesting fish oil, had mean and individual values indistinguishable from the fish oil group after the intervention (Table 3 and Fig. 2).

Fig. 2 Erythrocyte EPA+DHA levels in a randomised controlled trial of fish oil intervention for post-operative atrial fibrillation after cardiac surgery. The details of the trial and the mean fatty acid values have been published in Farquharson et al. ^{( 45 )}. Control oil group, monounsaturated oil for 3 weeks before surgery (n 71); fish oil group, fish oil concentrate supplying 4·5 g/d of EPA+DHA for 3 weeks before surgery (n 66); observation group, excluded from the trial due to the consumption of one or more than one fish meal per week or fish oil supplementation – agreed to be followed (n 44).

Thus, unless modern n-3 studies with cardiac patients exclude potential subjects on the basis of their background fish or fish oil intake, it is unlikely that any dietary n-3 intervention will create a distinct group with regard to tissue EPA+DHA levels. This will be exacerbated if there is no rigorous exclusion of those consuming marine n-3 fats from the control group during the study.

Problems arising from the ‘test agent’, EPA+DHA, being present in both the control and intervention groups

The overlap in EPA+DHA levels between the treatment and control arms has important implications for the final statistical comparisons. For binary outcomes, i.e. those with only two possible results such as a fatal heart attack occurring or not occurring, it is beneficial to maximise the mean difference in EPA+DHA levels between the treatment groups. For an adverse binary outcome, one can suppose that higher levels of EPA+DHA following intervention are associated with lower probabilities of experiencing the adverse event. For a fixed event rate in the control arm, a decreased distance between the mean EPA+DHA levels of the control and intervention groups will be associated with a smaller difference in event rates due to treatment and a subsequent decrease in the power to detect differences between the groups for a given sample size. In the case of continuous outcomes, both the mean difference between the groups and the variability within the groups with regard to EPA+DHA levels are important. When assuming a linear association between the levels of EPA+DHA following intervention and the mean of some continuous outcome, an increase in the within-group variability of EPA+DHA levels will result in an increase in the within-group variability of the continuous outcome, which in turn will reduce the power to detect a given effect size. Consequently, for continuous outcomes, one should try to maximise the between-group difference in mean EPA+DHA levels while also minimising the within-group variability.

There is also relevance for study design. If baseline n-3 intakes and/or n-3 tissue levels are at or near the extreme of the linear range for influencing cardiac outcomes, the clinical effect of an intervention providing more n-3 fats is greatly weakened. Mozaffarian & Rimm^{( 46 )} pooled data from twenty cohort studies and randomised trials to generate a spline plot of the relative risk of CHD death against EPA+DHA intake. This plot demonstrated a dose–response from very low intake up to 250 mg/d of EPA+DHA and then little further reduction with higher intakes^{( 46 )} (Fig. 3).

Fig. 3 Relative risk of CHD death with the intake of the n-3 fats, EPA and DHA (adapted from Mozaffarian & Rimm^{( 46 )}).

Similar relationships were observed in a meta-analysis of eleven prospective cohort studies that examined the dose–response between the intake of fish and the risk of CHD death^{( 47 )} and also in a study with a single cohort that specifically examined sudden cardiac death^{( 48 )}. The meta-analysis incorporated studies that commenced from 1958 to 1989 with the average baseline year being 1972. A plot of the weighted relative risks for all categories of fish intake in each of the cohort studies demonstrated a significant protective effect of low fish intake compared with no intake, but then diminishing protection with increased intake^{( 47 )}. Similarly, the study that examined sudden cardiac death showed that dietary fish intake was associated with a reduction in the risk of sudden cardiac death, but there was a plateau after the consumption of more than one fish meal per week^{( 48 )}.

Thus, the relative change in the risk of CHD mortality with n-3 consumption will be greatest when the comparator has consumed little or no n-3.

Mozaffarian & Rimm^{( 46 )} have integrated these and other observations to predict the relationships between EPA+DHA intake and various clinical or biomarker cardiovascular outcomes (Fig. 4).

Fig. 4 Potential dose–responses for clinical events with the intake of the n-3 fats, EPA and DHA (adapted from Mozaffarian & Rimm^{( 46 )}). BP, blood pressure. A colour version of this figure can be found online at http://www.journals.cambridge.org/bjn

Endpoints

Besides the issue with background intakes that may be near or above the plateau for effect, the composite endpoint is an additional problem. The cohort studies, case–control studies and RCT that do find an effect of n-3 fat intake on coronary mortality generally find no effect on non-fatal myocardial infarction^{( 3 )}. This can be explained if the main cardiac effect of n-3 fats at modest doses is the prevention of fatal arrhythmia. Therefore, inclusion of outcomes for which there is little or no evidence of an effect of n-3 fats, such as non-fatal cardiovascular events, percutaneous interventions and coronary artery bypass grafting (CABG), in a composite primary endpoint greatly increases the chance of a null finding even if fish oil intake is associated with one of the components of the composite primary outcome measure, and this point has been made by others^{( 43 )}. During the Alpha Omega Study, it was realised that the event rate of the original primary endpoint of fatal CHD would be too low and therefore major cardiovascular events were considered to be the primary endpoint, which was a composite of fatal and non-fatal myocardial infarction, sudden death, mortality from heart failure, non-fatal stroke and fatal stroke, percutaneous intervention and CABG^{( 49 )}. This is expected to greatly decrease the chance of detecting an effect on the primary outcome measure, which is always the main aim of a RCT.

This is not an issue unique to n-3 studies. The problems arising from composite endpoints in drug trials have been outlined, and these centre on the interpretation of results. The trial may show a benefit of a drug on a composite endpoint where the benefit is driven only by a single element of the composite endpoint, but the announcements that accompany the results can imply, by association, that there is a benefit for all elements of the outcome measure^{( 50 )}.

Conclusions

RCT are accepted methods for examining drugs, and the reasons are obvious. They can show causality and they are not impaired by the use of the drug at baseline or in the control group during the trial. These issues are easily controlled, and they contrast greatly with RCT that examine the effects of fish oil supplementation in cardiac patients. The ability of a RCT to detect changes in CHD mortality will be greatest if the baseline intake of EPA and DHA is very low and non-study EPA+DHA intake is low. If baseline or non-study intake is near or above the threshold for effect, then the RCT is unlikely to show any further effect of an n-3 intervention.

It can be argued that none of the issues mentioned above are weaknesses in the conduct of RCT in subjects with cardiac disease because RCT produce answers to the question they are designed to address. For example, the Alpha Omega Study found no effect on a composite outcome that included fatal+non-fatal CVD+percutaneous intervention+CABG of daily consumption of margarine containing 400 mg EPA+DHA in subjects already consuming EPA+DHA at median intakes of 130 (interquartile range 60–210) mg/d^{( 6 )}. Thus, this amount of dietary EPA+DHA added to a median background intake of 130 mg/d of EPA+DHA had no effect discernible by this RCT. It was adequately powered and that is the answer. However, an approach that took into account (1) tissue EPA+DHA levels at baseline and where this placed participants on the expected dose–response curve, (2) the effect of study and non-study n-3 intake on the final tissue EPA+DHA levels in each group, and (3) the use of existing published information to choose an outcome measure that is more likely to be sensitive to the intervention perhaps could have yielded a positive result. Phase III clinical trials with pharmaceuticals invariably consider pharmacokinetics and pharmacodynamics in the planning of the RCT^{( 51 )}, but n-3 trials in CVD patients do not.

This discussion has focused on the shortcomings of individual RCT of fish or fish oil interventions. However, these shortcomings of the test agent being in each group along with the lack of control over baseline n-3 status and no allowance for the intervention being in a range where the outcome is responsive to dose inevitably affect the conclusions of meta-analyses.

Analysis of fish oil RCT mirrors that of drug RCT. This is a by-group comparison of the intention-to-treat sample and it is the standard for publication. This is understandable for drug trials, but the lack of distinct groups in fish oil trials limits the conclusions on efficacy that can be drawn from such an analysis. Given this limitation, it is appropriate to acknowledge the value of post hoc regression analyses that examine the relationships between tissue levels of n-3 fats and outcomes. These could accompany the initial publication of the by-group comparisons and assist in drawing the conclusions. It follows that meta-regression analyses may be more informative than standard meta-analyses that compared group outcomes.

These limitations point out the relative usefulness of cohort and case–control studies and ‘real-life’ clinic studies for assessing n-3 benefits at the community level and the patient level, respectively. An example from the latter category is the retrospective matched cohort analysis of data from the General Practice Research Database in the UK that showed that prescription of licensed n-3 fats at a daily dose of 1 g to patients within 90 d of a first myocardial infarction reduced all-cause mortality^{( 52 )}. The intention of such a study is the same as that of Phase IV drug trials; that is, to examine how the drug or, in this case, fish oil performs in real-life practice with the vagaries of compliance.

Recommendations for randomised controlled trials with fish oil

The trial design should aim to maximise the appropriate differences in n-3 tissue levels between the control and the intervention group(s). The following exclusion criteria should be used:

1. (1) Exclude those who habitually consume more than one fish meal per week.

2. (2) Exclude those taking fish oil supplements and are unwilling to discontinue their use.

3. (3) Exclude those who are not certain that they will not use fish oil or similar n-3 supplements for the duration of the trial.

4. (4) Exclude those whose tissue n-3 levels are above a certain threshold (realistically this would be erythrocyte n-3 levels).

These exclusions will greatly reduce the recruitment of cardiac patients due to the recommendations that they generally receive to increase n-3 intake.

To address the variability in the relationship between the intake and tissue levels of n-3 fatty acids, a treat-to-target approach could be used. This would employ dosage adjustment to reach a certain target erythrocyte EPA+DHA level in the intervention group, rather than relying on a fixed dose of fish oil. This would be similar to the approaches followed in statin trials with target LDL levels or diabetes trials with target glycated Hb levels and anti-hypertensive trials with target blood pressures.

The primary endpoint should not include outcomes for which there is little or no evidence for an effect of dietary n-3 fats. This would exclude non-fatal myocardial infarction, hospital admission for cardiovascular causes, percutaneous intervention, and coronary artery bypass surgery, all of which have been used in composite primary endpoints in fish oil trials with cardiac patients^{( 6 , 8 )}.

Clearly, these criteria would increase the difficulty of conducting RCT involving fish oil supplementation in at-risk cardiac patients. However, to continue further RCT without addressing especially the issue of non-discrete n-3 groups and whether baseline n-3 intake already places individuals high on the dose–response curve is an inefficient and flawed approach for studying the potential benefits of dietary n-3 fats for cardiac mortality.