Vitamin E and the risk of pneumonia: using the I² statistic to quantify heterogeneity within a controlled trial

Abstract

Analyses in nutritional epidemiology usually assume a uniform effect of a nutrient. Previously, four subgroups of the Alpha-Tocopherol, Beta-Carotene Cancer Prevention (ATBC) Study of Finnish male smokers aged 50–69 years were identified in which vitamin E supplementation either significantly increased or decreased the risk of pneumonia. The purpose of this present study was to quantify the level of true heterogeneity in the effect of vitamin E on pneumonia incidence using the I² statistic. The I² value estimates the percentage of total variation across studies that is explained by true differences in the treatment effect rather than by chance, with a range from 0 to 100 %. The I² statistic for the effect of vitamin E supplementation on pneumonia risk for five subgroups of the ATBC population was 89 % (95 % CI 78, 95 %), indicating that essentially all heterogeneity was true variation in vitamin E effect instead of chance variation. The I² statistic for heterogeneity in vitamin E effects on pneumonia risk was 92 % (95 % CI 80, 97 %) for three other ATBC subgroups defined by smoking level and leisure-time exercise level. Vitamin E decreased pneumonia risk by 69 % among participants who had the least exposure to smoking and exercised during leisure time (7·6 % of the ATBC participants), and vitamin E increased pneumonia risk by 68 % among those who had the highest exposure to smoking and did not exercise (22 % of the ATBC participants). These findings refute there being a uniform effect of vitamin E supplementation on the risk of pneumonia.

The effect of vitamin E supplementation on mortality has been studied in numerous randomised trials, the results of which have been pooled in several meta-analyses^{( 1 – 3 )}. Usually meta-analyses calculate a single estimate of effect, such as a 4 % increase in mortality by vitamin E^{( 1 )}. The calculation of a single estimate is based on the assumption that there is a uniform size of effect that is informative for all the included trials, and also applies to populations not included in the analysed trials.

Biology is complex, and it is possible that the effect of vitamin E on health outcomes depends on various characteristics of people and on their lifestyles. Therefore, a single universal estimate of vitamin E effect might be substantially misleading for some population groups. We found in our previous analyses of the Alpha-Tocopherol, Beta-Carotene Cancer Prevention (ATBC) Study on Finnish male smokers that the effects of vitamin E supplementation were modified as follows: the risk of common cold by age, smoking and residential neighbourhood^{( 4 )}, the risk of tuberculosis by vitamin C intake^{( 5 )} and mortality by age and vitamin C intake^{( 6 )}. These findings challenge the notion that the health effects of vitamin E are uniform over the entire ATBC Study population. However, a quantitative estimation of the true within-trial heterogeneity in vitamin E effects has not been carried out previously.

The I ² statistic was developed for the quantification of true heterogeneity between multiple controlled trials included in a meta-analysis^{( 7 , 8 )}. The I ² value estimates the percentage of total variation across different studies, which is explained by true variation in the treatment effect rather than by chance variation. The range of the I ² scale is from 0 to 100 %, and a value greater than about 75 % indicates a high level of true treatment heterogeneity^{( 8 )}. To our knowledge, the I ² statistic has not been used previously to quantify the level of true heterogeneity between the subgroups of a single randomised trial.

Vitamin E is an antioxidant and it influences the immune system^{( 9 , 10 )}. Therefore, it might influence infections of the lungs exposed to O₂ and airborne oxidants. In our previous analyses of the ATBC Study data, the effect of vitamin E on pneumonia incidence differed from the null effect for several subgroups, which were identified by different types of reasoning: by the level of smoking, physical activity, weight and dietary vitamin C intake^{( 11 – 15 )}. The goal of this study was to quantify the level of true heterogeneity in the effect of vitamin E on pneumonia risk over the identified ATBC Study subgroups by using the I ² statistic.

Methods

Participants

The rationale, design and methods of the ATBC Study, to examine the effects of vitamin E (dl-α-tocopheryl acetate, AT, 50 mg/d) and β-carotene (BC, 20 mg/d) on the incidence of lung cancer and other cancers and the primary findings, have been described in detail^{( 16 , 17 )}. The ATBC Study is registered at ClinicalTrials.gov under the identifier NCT00342992. In brief, males aged 50–69 years who smoked ≥5 cigarettes/d at entry (n 29 133) were randomised into one of four intervention arms – placebo, AT, BC or AT+BC – according to a 2×2 factorial design. Supplementation with vitamin E in the form of dl-α-tocopheryl-acetate increased the mean serum levels of α-tocopherol by 50 % compared with baseline^{( 17 )}. The intervention continued for 5–8 years until April 1993. The trial was approved by the review boards of the participating institutions, and all participants gave their written informed consent. Compliance with supplementation was high: 90 % of the subjects took >90 % of their prescribed capsules during their active participation in the trial^{( 17 )}.

Baseline characteristics

Before randomisation, the participants completed questionnaires on medical and smoking histories and general background characteristics^{( 11 , 12 , 16 , 17 )}. The baseline questionnaire enquired about the intensity of leisure-time physical activity in terms of the following three alternatives: (1) light: reading, watching TV, listening to the radio or going to movies; (2) moderate: walking, fishing, hunting or gardening quite regularly; and (3) heavy: actual physical exercise such as jogging, skiing, swimming, gymnastics and court and field sports quite regularly. In the current analysis, ‘exercise during leisure time’ combines positive responses to alternatives (2) (n 15 191) and (3) (n 1744).

Outcome and follow-up time

The outcome of this study, the first hospital-treated case of pneumonia after randomisation, was ascertained from the national Hospital Discharge Register using the volunteer’s unique personal identification number, given to all Finnish residents, for linkage^{( 11 )}. Follow-up time began from the day of randomisation and continued until the date of the first hospital discharge for pneumonia, death or the end of the trial, whichever came first. There was a total of 167 968 person-years of observation (median follow-up 5·8 years).

Statistical methods

The effect of vitamin E supplementation on pneumonia incidence was estimated by Cox’s proportional hazards models. The trial participants to whom vitamin E alone or in combination with BC were administered (AT and AT+BC) were compared with the no-vitamin E supplement groups (placebo and BC). The exceptions were subgroup 3 in Fig. 1 and 2 and subgroup A in Fig. 3, for which the comparison was restricted to no-BC participants because of the significant interaction between AT and BC^{( 15 )}. We calculated the risk ratio (RR) and the 95 % CI of the RR using the PROC PHREG program of the SAS package of programs (release 9.4; SAS Institute Inc.). Forest plots were constructed using the metagen and forest programs of the R program package; the I ² statistic with its 95 % CI and the Cochran Q test-based χ ² values for heterogeneity were calculated^{( 18 )}. To test the statistical significance of interaction between vitamin E supplementation and the set of subgroups, vitamin E and the subgroups were first added to the Cox’s model. The statistical significance of the interaction was thereafter calculated from the change in −2×log (likelihood) when the vitamin E subgroup interaction terms were added to the model.

Fig. 1 Proportion of participants and the effect of vitamin E on the incidence of pneumonia in the Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study, 1985–1993. The left-hand side shows the proportion of participants in six subgroups. The right-hand side shows the effect of vitamin E supplementation on the risk of pneumonia for the same subgroups. Group 3 shows the estimate of vitamin E effect based on the no-β-carotene participants, because vitamin E and β-carotene had a significant interaction in that subgroup^{( 15 )}. Groups 1 and 2 had 60 and 289 participants, respectively, overlapping with group 3. In Fig. 1 and 2, the overlapping participants are included in groups 1 and 2, so that these two subgroups are consistent with the study of Hemilä & Kaprio^{( 14 )}. RR, risk ratio.

Fig. 2 A forest plot of six subgroups on vitamin E and the incidence of pneumonia in the Alpha-Tocopherol, Beta-Carotene Cancer Prevention (ATBC) Study, 1985–1993. The subgroups of Fig. 1 are shown in the same order in this forest plot. The percentage shown after group identification indicates the proportion of ATBC Study participants falling in that subgroup. On the right-hand side, the vertical line indicates the no-vitamin E level. The horizontal lines indicate the 95 % CI for the vitamin E effect, and the squares at the centre of the horizontal lines indicate the point estimates of the effects in those particular groups. The sizes of the squares indicate the relative weights of the groups. The Cochran Q test χ ²=37·6 (5 df) corresponds to P=10⁻⁶. The two ‘rest of the participants’ groups 4 and 5 are redundant, and when they are combined to a single ‘rest of the participants’ group (4+5) the I ² increases to 89 % (95 % CI 78, 95 %) with χ ²=37·5 (4 df) corresponding to P=10⁻⁷ (see the online Supplementary Fig. S1). RR, risk ratio; TE, treatment effect on the logarithmic scale; seTE, standard error of TE.

Fig. 3 A forest plot of three subgroups on vitamin E and the incidence of pneumonia in the Alpha-Tocopherol, Beta-Carotene Cancer Prevention (ATBC) Study, 1985–1993. Group A in this forest plot includes participants who started smoking at ≤20 years of age and smoked ≥20 cigarettes/d at study entry and did not carry out leisure-time exercise (23·0 % of the ATBC participants). Group C includes males who started smoking at ≥21 years of age and smoked 5–19 cigarettes/d at study entry and carried out leisure-time exercise (7·6 %). Group B includes all the other participants (69·4 %). The estimate of effect shown for subgroup 3 is based on the no-β-carotene participants only, as vitamin E and β-carotene had a significant interaction in that subgroup; see Hemilä & Kaprio^{( 15 )} for the origin of these three subgroups. In the forest plot on the right-hand side, the vertical line indicates the placebo level. The Cochran Q heterogeneity test χ ²=25·7 (2 df) corresponds to P=10⁻⁵. When the analysis was restricted to the no-β-carotene participants (n 14 573), then I ²=88 % (95 % CI 65, 96 %; P=0·0003) (see the online Supplementary Fig. S2). RR, risk ratio; TE, treatment effect on the logarithmic scale; seTE, standard error of TE.

Results

The ATBC Study included males aged 50–69 years who smoked ≥5 cigarettes/d at entry. Further characteristics of the participants have been described previously^{( 11 – 17 )}. There were 898 pneumonia cases during the follow-up period corresponding to an average rate of 5·3 pneumonia cases per 1000 person-years. Among all 29 133 ATBC participants, the pneumonia cases were identically distributed between the vitamin E and no-vitamin E groups, 449 v. 449, corresponding to the average effect of vitamin E supplementation of RR 1·00 (95 % CI 0·88, 1·14).

To quantify the level of heterogeneity in vitamin E effect, the ATBC participants were divided into six subgroups on the basis of previous findings (Fig. 1). The primary cut-off point for the subgroups was the age at which the participant initiated smoking (≤20 v. ≥21 years), which significantly modified the effect of vitamin E in the first series of subgroup analyses^{( 11 )}. The second-level subgroups 1 and 2 were formed by the subject’s body weight and dietary vitamin C intake^{( 14 )}, and subgroups 3 and 6 were formed by the level of cigarette smoking at baseline and the level of exercise at leisure time at baseline^{( 15 )}. The participants who did not fall into these second-level subgroups were classified as ‘the rest’, and they comprised subgroups 4 and 5. A forest plot of the six subgroups is shown in Fig. 2. The number of pneumonia cases in the six subgroups is shown in the online Supplementary Table S1.

Essentially all heterogeneity over the six subgroups was true variation in the vitamin E effect rather than chance variation: I ²=87 % (95 % CI 73, 93 %) (Fig. 2).

In subgroup 6, vitamin E supplementation decreased the risk of pneumonia by 69 % (95 % CI 44, 87 %; n 2216). This group included people who started smoking at a later age (≥21 years), smoked just 5–19 cigarettes/d at study entry and carried out leisure-time exercise^{( 15 )}. This subgroup in which vitamin E was beneficial covered 7·6 % of the ATBC participants.

The three groups – 1, 2 and 3 – for which vitamin E increased pneumonia risk by 209 % (95 % CI 45, 560 %; n 468), 134 % (95 % CI 7, 408 %; n 1328) and by 68 % (95 % CI 18, 140 %; n 3022), respectively, included males who started smoking at a younger age (≤20 years). In addition, these participants had low body weight and vitamin C intakes above the median (group 1), high body weight (group 2), smoked ≥20 cigarettes/d at study entry and did not carry out leisure-time exercise (group 3)^{( 14 , 15 )}. In all, these three subgroups in which vitamin E was harmful covered 28 % of the ATBC participants.

Vitamin E supplementation did not influence pneumonia risk among the rest of the participants (groups 4 and 5). These two subgroups covered 66 % of the ATBC study participants. In Fig. 1 and 2, these two groups are shown separately to illustrate the background of the subgroup division. However, maintaining the two ‘rest of the participants’ groups separately is redundant, as both of them are consistent with no effect. When these two groups were combined, the heterogeneity over the remaining five subgroups increased to I ²=89 % (95 % CI 78, 95 %) (online Supplementary Fig. S1). When the five subgroups were allowed independent vitamin E effects in the Cox’s regression model, the statistical model was improved by χ ²=42·3 (4 df) corresponding to P=10⁻⁸.

When small subgroups are formed, the balance of the baseline variables might be compromised. The uppermost subgroup 1 was small with only 468 participants – that is, only 1·6 % of all ATBC Study participants (Fig. 1 and 2). Nevertheless, the baseline differences in relevant variables between the vitamin E and no-vitamin E participants in this subgroup were close to zero with narrow CI. Furthermore, inclusion of baseline variables in the Cox’s model did not substantially change the estimate of vitamin E effect (online Supplementary Table S2). Thus, the difference in pneumonia occurrence between the vitamin E and the no-vitamin E participants in subgroup 1 cannot be explained by an imbalance in relevant baseline variables. The other groups, 2, 3 and 6, in which vitamin E significantly affected pneumonia risk are much larger, and a baseline imbalance is of even less concern.

A simplified analysis with only three subgroups was also carried out (Fig. 3). This division was based on the age at initiating smoking, the level of cigarette smoking at baseline and the level of leisure-time exercise at baseline^{( 15 )}. Group A had the highest smoking levels without leisure-time exercise. Group C had the lowest levels of smoking with active leisure-time exercise. Thus, the characteristics of group C are the opposite of group A. The effects of vitamin E also point to the opposite directions in these two subgroups. Group B includes participants who did not belong to group A or C. The I ² statistic for heterogeneity in this set of three subgroups was 92 % (95 % CI 81, 97 %), indicating that essentially all the heterogeneity in this subgroup division was a true variation of the vitamin E effect and not chance fluctuation. When the three subgroups were allowed independent vitamin E effects in the Cox’s regression model, the statistical model improved by χ ²=28·7 (2 df) corresponding to P=10⁻⁶.

Discussion

The number of pneumonia cases in the ATBC Study was evenly distributed between the vitamin E and the no-vitamin E participants, indicating no overall average effect with great accuracy. Nevertheless, within the ATBC Study population, there was a high level of true heterogeneity for the effect of vitamin E on pneumonia risk as shown in the present study. Not only the I ² point estimates but also the entire 95 % CI ranges of the I ² were above the 75 % level, which has been judged as the threshold for high level of true heterogeneity^{( 8 )}. This indicates that the overall average zero effect is not applicable for all ATBC participants. It follows, therefore, that there cannot be a uniform vitamin E supplementation effect on pneumonia risk over the Western male population, as Finnish males of the ATBC Study form a subgroup of Western males.

All the variables used to define the subgroups of Fig. 1 have a biological rationale: smoking has an influence on vitamin E metabolism^{( 19 )}, vitamins C and E interact^{( 19 , 20 )} and sporadic physical activity causes oxidative stress^{( 21 )} against which antioxidant vitamin E may protect. Finally, the dose–effect relationship is a basic concept in pharmacology. Consequently, the effects of a fixed vitamin E dose may depend on body weight as the dose per body weight varies^{( 14 )}.

When the modification of vitamin E effect is complex and defined by half a dozen or more variables, there is no unambiguous way to form subgroups that are distinguished by different sizes of the vitamin E effect. Pragmatic cut-off limits are used in Fig. 1–3; yet, it is unreasonable from the biological perspective to assume exact cut-off points. Nevertheless, the main issue in this study is not the specific locations of the cut-off points, but the finding of the very high level of true heterogeneity in the vitamin E effect over the 29 133 ATBC participants.

The level of true heterogeneity of vitamin E effect depends on the combination of the sizes of the vitamin E effects for the subgroups and the sizes of the subgroups themselves. Thus, the estimate of I ²=92 % in Fig. 3 is not a characteristic of vitamin E but it is generated by the combination of the specific subgroup sizes and the effects of vitamin E within the particular subgroups of the ATBC Study cohort.

The high level of true heterogeneity in the effect of vitamin E on pneumonia has important implications. First, it provides a strong argument against the opinion that subgroup analyses of randomised trials should be strongly discouraged because they can lead to false-positive findings due to the multiple comparison problem^{( 22 – 25 )}. Altman stated that biological plausibility is a weak criterion when deciding whether a subgroup finding is likely to be real, as in his view ‘doctors seem able to find a biologically plausible explanation for any finding’^{( 22 )}. Although there is much room for speculation at the molecular level of biology because the number of genes and proteins is huge, the number of variables relevant at the population level of biology is much more limited. Few variables are as important at the population level as smoking, which modified the effect of vitamin E (Fig. 1–3).

Many trials are small and they do not have the statistical power to analyse subgroup differences. For example, one study on vitamin E and respiratory infections included 652 participants who were followed-up for 788 person-years^{( 26 )}, and another study included 617 participants followed-up for 540 person-years^{( 27 )}. In contrast, the ATBC Study included 29 133 participants followed-up for 168 000 person-years. Consequently, the ATBC Study, when analysed as subgroups, may be considered to be a large series of small studies covering a wide range of population groups with different characteristics. A large, randomised trial has consistent treatment and outcome definitions. Therefore, a subgroup analysis of a large trial is much more informative than a comparison of a series of small trials with slightly varying interventions and outcome definitions, even when the total number of participants in the latter might be the same. Although the multiple comparison problem is a relevant concern in subgroup analysis of small studies, it is not a reasonable explanation for the narrow CI of the I ² statistic found in the present subgroup analysis (Fig. 2 and 3).

Biology is complex and it is unlikely that the belief in a uniform treatment effect is usually justified. The groups of people in whom a treatment is either most or least effective can be found only by comparing the effects on different groups of people. Feinstein wanted to ‘rescue the scientific importance of valid pathophysiologic subgroups from being forgotten or destroyed by excessive vehemence in suggestions that all subgroups are evil’^{( 28 )} and Lagakos commented that ‘avoiding any presentation of subgroup analysis because of their history of being over-interpreted is a steep price to pay for a problem that can be remedied by more responsible analysis and reporting’^{( 29 )}. Given the long-term commitment of study participants and the resources invested, it might even be considered as an ethical duty of the researchers to analyse large trials extensively rather than simply calculating a single overall average effect. Nevertheless, it is also important to carry out subgroup analysis with caution and not over-interpret the findings.

The second implication of the high level of true heterogeneity within the ATBC Study cohort concerns the pooling of diverse randomised trials in meta-analyses. Calculation of a pooled estimate of effect is based on the assumption that there is a uniform effect that is informative. However, small studies have wide CI and may not reveal heterogeneity even if the biological effect does differ between the studied populations. On the other hand, large studies may include people who vary substantially in their characteristics and in the effects of treatments; yet, the overall average effect may camouflage substantial variations between subpopulations as shown in Fig. 1–3. Therefore, the pooled estimates of meta-analyses can be spuriously precise and may suffer from ecological fallacy, which means that study-level analysis can lead to different conclusions than corresponding individual-level analysis^{( 30 , 31 )}. Analyses of the ATBC Study also found evidence that the effect of vitamin E on mortality was heterogeneous^{( 6 , 32 )}. Therefore, the averages calculated in meta-analyses, such as the 4 % increase in mortality for vitamin E supplementation^{( 1 )}, may not be valid for many population groups.

The third implication of the heterogeneity in vitamin E effects is that cohort studies on nutrition and health may often be misleading. In cohort studies, confounders are adjusted to allow the calculation of a single estimate of effect over the study population. For example, in their cohort study with male US health professionals between 40 and 75 years of age, Merchant et al.^{( 33 )} reported no association between daily vitamin E intake and community-acquired pneumonia. However, when several variables modify the effect of vitamin E on pneumonia risk (Fig. 1–3), it is evident that the effects of vitamin E should be investigated separately in subpopulations defined by those modifier variables, instead of calculating a single average effect adjusting for those variables as if they were confounders. Large trials such as the ATBC Study can give accurate effect estimates for subgroups as shown by the current study. However, similar subgroup analyses in cohort studies are much more challenging or impossible because of the close associations between dietary variables with each other and with numerous other lifestyle factors^{( 34 )}.

Finally, vitamin E supplementation has been proposed for improving the immune system^{( 35 )}. However, in the ATBC Study, 28 % of males had an increased risk of pneumonia because of vitamin E administration (Fig. 1). In addition, the combination of vitamin E supplementation and a high level of dietary vitamin C intake increased the risk of tuberculosis by 72 % (95 % CI 4, 185 %)^{( 5 )}, and vitamin E increased the risk of common cold in a subpopulation of the participants^{( 4 )}. Thus, even though subgroup 6 of Fig. 1 indicates that some people may benefit from vitamin E by gaining protection against infection, there is evidence of harm in some other people. Given the current limited understanding about who might benefit, vitamin E should not be suggested for the general population for improving the immune system.

Although the 69 % reduction in the risk of pneumonia is a substantial effect in subgroup 6 (Fig. 1), given the pneumonia rate of about six cases/1000 person-years, approximately 250 people would need vitamin E supplementation for 1 year to prevent one episode of pneumonia in males in that subgroup. Community-acquired pneumonia in middle-aged people is usually cured quite rapidly by antibiotics and rarely leads to long-term or permanent sequelae; thus, the practical significance of vitamin E is not clear even in this subgroup. Furthermore, the ATBC Study participants were mostly born in the 1920s and 1930s and lived through the WWII years. Therefore, the estimate of effect calculated for the 7·6 % subgroup of the ATBC Study cohort should not be generalised to current middle-aged males in Western countries.

In conclusion, the I ² statistic may be a useful measure when analysing within-trial heterogeneity in large, randomised trials. The numerical estimates of vitamin E effect in the analysed subgroups of the present study are much less essential than the high level of true heterogeneity over the entire ATBC Study cohort. When an effect is heterogeneous, great caution should be exercised in the extrapolation of the effect estimates to other contexts. The high level of true heterogeneity found in the current study indicates that the uniform effect estimates calculated in meta-analyses and cohort studies on vitamin E may often be misleading. There seems to be a need for further research on vitamin E for non-smoking, middle-aged and older males who exercise in their leisure time.