Quantifying the indirect effects of key child survival interventions for pneumonia, diarrhoea, and measles

Summary

To date many studies have measured the effect of key child survival interventions on the main cause of mortality while anecdotally reporting effects on all-cause mortality. We conducted a systematic literature review and abstracted cause-specific and all-cause mortality data from included studies. We then estimated the effect of the intervention on the disease of primary interest and calculated the additional deaths prevented (i.e. the indirect effect). We calculated that insecticide-treated nets have been shown to result in a 12% reduction [95% confidence interval (CI) 0·0–23] among non-malaria deaths. We found pneumonia case management to reduce non-pneumonia mortality by 20% (95% CI 8–22). For measles vaccine, seven of the 10 studies reporting an effect on all-cause mortality demonstrated an additional benefit of vaccine on all-cause mortality. These interventions may have benefits on causes of death beyond the specific cause of death they are targeted to prevent and this should be considered when evaluating the effects of implementation of interventions.

INTRODUCTION

Although child mortality rates are continuing to decline globally, there are still more than 8 million children who die each year before reaching their fifth birthday [1]. Infectious diseases, including diarrhoea, pneumonia, and malaria remain the leading causes of death despite simple and effective interventions for each of the main causes of child mortality [2]. The Child Health Epidemiology Reference Group (CHERG) recently reviewed the scientific evidence for all key child survival interventions and estimated the effect size for each intervention on cause-specific mortality [3].

Child survival interventions are often thought of as acting on one infection and one cause of death (COD), e.g. insecticide-treated nets (ITNs) prevent malaria infections and thus malaria deaths. However, for many years it has been noted that selected child survival interventions appear to benefit more than the targeted disease. Extensive literature reviews and effect-size estimates for interventions such as ITNs and malaria case management, case management of pneumonia, oral rehydration solution (ORS) for diarrhoea, and measles vaccine have been published [4–8]. Analyses have also attempted to predict the impact of malaria intervention scale-up on all-cause mortality, under various assumptions of the burden of indirect malaria mortality [9]. We sought to go beyond these published reviews and analyses to quantify the benefit of selected interventions on: (1) disease-specific mortality for the disease targeted by the intervention (i.e. the direct effect), and (2) disease-specific mortality for diseases not targeted by the intervention or all-cause mortality (i.e. the indirect effect). Our estimates are based on data from published intervention studies.

METHODS

Identification and selection of studies

We conducted a systematic search in PubMed and Cochrane Library databases for studies published to the end of 2009 (Supplementary online Appendix). We searched without restrictions on year or study design including randomized controlled studies, observational studies, community studies, case-control, pre- vs. post-intervention comparisons, and natural experiments according to the CHERG standards for systematic reviews [3].

For malaria interventions we used the following key words and MeSH terms in various combinations: malaria, chemoprophylaxis, treatment, ITN, and child mortality. For pneumonia case management we used the following key words and MeSH terms in various combinations: pneumonia, case management, and child mortality. For oral rehydration solution we used the following key words and MeSH terms in various combinations: diarrhoea, dysentery, fluid therapy, oral rehydration solution, and oral rehydration therapy and child mortality. We included only studies where ORS was compared to intravenous fluids or no treatment, excluding studies that used different ORS formulations in the comparison arm. Because ORS is relatively widespread and contamination of the comparison arm is a concern in ORS studies, we excluded studies that did not adequately describe the coverage levels achieved in the intervention and comparison arms, and studies where there was little difference in coverage between intervention and comparison arms. For measles vaccine we used the following key words and MeSH terms in various combinations: measles, measles vaccine or vaccination, and child mortality. We included only studies evaluating the standard medium-titre measles vaccine as the main intervention with either concurrent controls or historical comparisons. We screened all titles and abstracts and the full articles of papers meeting all inclusion and exclusion criteria based on abstract review. Studies were also identified by hand searching the reference lists of retrieved articles. Studies that met the above criteria and that reported the effect of the intervention on all-cause mortality or on causes other than those the intervention was intended to impact were double-abstracted into a database. Studies conducted in special populations, adults, or high-income settings, and thus not generalizable to children in low- and middle-income settings, were excluded. We also excluded cohort and quasi-experimental studies including pre-post study designs that did not control for confounding.

We verified that each study was represented only once in the database: in cases where our searches found multiple publications for a single study, data from all the publications were abstracted into a single row to avoid double counting. Variables abstracted included the study setting, design, population, definition of the intervention, co-interventions, mortality rates, effect sizes for all-cause and cause-specific mortality, and confounders adjusted for in the analysis. For the pneumonia case-management studies, we were able to obtain and abstract all-cause and pneumonia-specific mortality data from the unpublished data tables for a previously published meta-analysis [10], to supplement the information available from the published reports of the studies (Tables 1, 3, 5, 7). In all other cases, we relied on published data.

Analytical methods

We used the same analytical techniques for each of the interventions. As this approach contains numerous steps, we provide a detailed example for a malaria study in Fig. 1; this method was used for all interventions, with the modifications described below. For studies reporting only the effect of the intervention on all-cause mortality, we estimated the all-cause deaths averted (per 1000 child-years or live births, as appropriate) by the intervention by multiplying the baseline or comparison mortality rate by the study all-cause mortality relative rate reduction or hazard ratio. We then assumed that the intervention prevented deaths attributable to the targeted COD first (i.e. ITNs would prevent all malaria deaths before preventing deaths from other causes). Where possible, we abstracted the proportion of deaths by cause reported in the paper for the control/baseline group. If no COD data were reported in the study, if COD data were not disaggregated by intervention and comparison group, or if COD definitions or verbal autopsy methods were not clearly described, we used the country-, age-, and year-specific COD profiles from the Lives Saved Tool (LiST) to estimate the proportion of deaths from the targeted cause. The LiST tool uses peer-reviewed COD estimates that have undergone extensive review by CHERG and country consultation [2, 11–13]. In the case of measles vaccine, where the vaccine is very effective and coverage levels are generally high, we modelled the country-specific COD profile in LiST, setting the measles vaccine coverage to 0%, to simulate a counterfactual population for which measles vaccine was not available. We calculated the anticipated number of deaths in the control/baseline group for the targeted COD by multiplying the all-cause mortality rate in the control/baseline group by the reported or modelled proportion for that COD (Tables 2, 4, 6, 8). We then compared the all-cause deaths averted by the intervention with the estimated number of deaths from the targeted COD in the control/baseline group and calculated the excess deaths averted. We attributed all excess deaths averted to the indirect effects of the intervention. If there were no excess deaths averted, i.e. the all-cause deaths averted were less than the deaths expected from the targeted cause we assumed that the intervention had no indirect effect on mortality in that particular study; all deaths averted were from the targeted cause.

Fig. 1. Method of calculating cause-specific and all-cause deaths averted: the example of malaria.

For studies that reported intervention effect sizes for cause-specific mortality, we calculated the all-cause and cause-specific number of deaths assumed to have been averted by the intervention. The total number of all-cause deaths averted was calculated as described above. For the effect of the intervention on cause-specific mortality we multiplied the proportion of deaths in the control/baseline group from the targeted cause (as reported or from LiST) by the control/baseline mortality rate to calculate the total number of cause-specific deaths that the intervention theoretically could prevent. We then estimated the cause-specific deaths averted by the intervention by multiplying the cause-specific mortality relative rate reduction by the number of cause-specific deaths in the baseline/control group (Tables 2, 4, 6, 8). If the study-reported cause-specific mortality relative rate was >1, we assumed that the intervention averted no deaths from that cause, not that the intervention caused additional deaths.

If the difference between the total and cause-specific deaths averted was positive, these ‘excess deaths averted’ were attributed to the indirect effects of the intervention on other causes of mortality and a relative mortality rate for causes other than those targeted by the intervention was calculated or abstracted from the paper, where available. When at least three studies reported an effect on all-cause mortality, we performed fixed- and random-effects meta-analyses in Stata statistical software, version 11 [14]. If there was statistical evidence of heterogeneity or if the study settings or designs varied substantially, we reported the DerSimonian–Laird pooled relative risk and 95% confidence ntervals (CIs); in other cases we reported the Mantel–Haenszel pooled relative risk and 95% CIs. In the case of measles vaccine, where the study designs varied greatly and the estimates of the proportion of mortality due to measles were modelled resulting in a wide range of effect sizes, we did not compute a summary measure.

RESULTS

Malaria

We identified 1242 studies for screening in the initial review. After title, abstract, and review of the full papers, a total of seven ITN studies from five countries met our inclusion and exclusion criteria [15–21] (Table 1). Five studies reported the effect of ITNs on all-cause mortality [17–21] while two studies reported cause-specific reductions in malaria and non-malaria mortality [15, 16]. In Table 2 we estimate the malaria and non-malaria deaths averted in ITN studies that reported reductions in all-cause and cause-specific mortality.

Table 1. Insecticide treated nets: characteristics of included studies

ACT, Artemisinin-based combination therapy; DSS, demographic surveillance site; ITN, insecticide-treated net; RCT, randomized controlled trial; VA, verbal autopsy.

Table 2. Indirect effects of ITNs estimated from studies reporting all-cause mortality

ITN, Insecticide-treated net; MR, mortality rate; RCT, randomized control trial.

^a Study did not report percent of deaths attributable to malaria. Estimate abstracted from LiST with year of estimate given.

^b Study reported malaria-specific relative mortality reduction of 0·79 was used to calculate the malaria deaths averted.

^c Study reported malaria-specific relative mortality reduction of 0·86 was used to calculate the malaria deaths averted.

Of seven studies reporting effects of ITNs on all-cause or cause-specific mortality, four observed a reduction in all-cause deaths greater than what would be expected from reduction in malaria deaths alone ranging from 1·24 to 4·03 deaths averted/1000 child-years (Table 2). We included all seven studies in the random-effects meta-analysis and found a pooled non-malaria mortality relative rate of 0·878 (955 CI 0·773–0·998) of borderline statistical significance (P = 0·046), equivalent to a reduction of 12% in non-malaria mortality in children aged 1–59 months (Fig. 2). Among the studies that recorded reductions in other causes of death, Binka et al. [15] reported a non-significant increase in deaths from acute respiratory infection (ARI) in the ITN arm but a non-significant decrease in deaths from diarrhoea [15]. The effect of ITNs on diarrhoea may have averted 1·78 deaths/1000 child-years (data not shown).

Fig. 2. Forest plot of ITN studies contributing to the random-effects meta-analysis.

We identified one study assessing the effects of malaria prophylaxis and of case management with chloroquine and one assessing the effect of case management alone on all-cause mortality [22, 23]; we did not find any studies reporting the effect of case management on causes of death other than malaria (Tables 3 and 4). In the study by Menon et al., we calculated that 14 non-malaria deaths/1000 child-years were averted by chloroquine chemoprophylaxis, which is equivalent to a 36% reduction in non-malaria deaths, relative to the control arm [23]. In the study by Kidane & Morrow [22], the effect on all-cause mortality was less than what would be expected had all malaria deaths been averted; thus we assumed no indirect effect on non-malaria deaths.

Table 3. Malaria case management: characteristics of included studies

NR, Not reported; RCT, randomized controlled trial; VA, verbal autopsy.

Table 4. Indirect effects of malaria case management estimated from studies reporting all-cause mortality

ITN, Insecticide-treated net; MR, mortality rate; NA, not applicable.

Pneumonia

We identified 3376 studies for screening in the initial review. After title, abstract, and review of the full papers, a total of six eligible studies of pneumonia case management were included (Table 5) [24–29]. These studies were also included in a previously published meta-analysis [10]; no new studies were identified for the analysis presented here. Five of six studies reported the effect of pneumonia case management on non-pneumonia causes of death while one reported a reduction in all-cause mortality. All six studies reported fewer non-pneumonia deaths, relative to the control arm (Table 6). Of the five studies reporting non-pneumonia reductions in mortality, three identified specific causes other than pneumonia for at least some deaths, while two reported COD only as pneumonia or non-pneumonia. Reductions were noted in diarrhoea mortality (0·77–1·47 deaths/1000 child-years, n = 3 studies) [24, 26, 28], measles mortality (1·10 deaths/1000 child-years, n = 1 study) [26], and malaria mortality (2·33 deaths/1000 child-years, n = 1 study) [28] (data not shown). A random-effects meta-analysis of the six studies found a pooled non-pneumonia mortality relative rate of 0·794 (95% CI 0·784–0·922), or a 20% reduction in non-pneumonia mortality (Fig. 3).

Fig. 3. Forest plot of pneumonia case management studies contributing to the random-effects meta-analysis.

Table 5. Pneumonia case management: characteristics of included studies

CHWs, Community health workers; DSS, demographic surveillance site; EPI, expanded programme on immunizations; NR, not reported; VA, verbal autopsy; VHWs, village health workers.

Table 6. Indirect effects of pneumonia case management estimated from studies reporting all-cause mortality

ITN, Insecticide-treated net; MR, mortality rate; cRCT, cluster randomized controlled trial.

^a Study reported pneumonia-specific relative mortality reduction of 0·81 was used to calculate the pneumonia deaths averted.

^b Study reported pneumonia-specific relative mortality reduction of 0·51 was used to calculate the pneumonia deaths averted.

^c Study reported pneumonia-specific relative mortality reduction of 0·46 was used to calculate the pneumonia deaths averted.

^d Study reported pneumonia-specific relative mortality reduction of 0·28 was used to calculate the pneumonia deaths averted.

^e Study reported pneumonia-specific relative mortality reduction of 1·07 was used to calculate the pneumonia deaths averted.

Diarrhoea

We identified 254 studies for screening in the initial review. After title, abstract, and review of the full papers, there were no papers that met our inclusion and exclusion criteria. Although several quasi-experimental and pre/post studies did assess the effect of ORS on all-cause or diarrhoea mortality, none of these studies controlled for confounding and many did not report coverage levels, or had similar coverage of ORS in both arms [30–39]. No RCTs or cluster RCTs were found that met our inclusion criteria.

Measles

We found 583 studies for screening in the initial review. After title, abstract, and review of the full papers, a total of 11 papers were included reporting on 10 unique studies assessing the effect of measles vaccine on all-cause mortality [40–50] and three papers reporting on one study assessing the effect of the vaccine on non-measles causes of death [51–53] (Table 7). Of the 10 studies reporting benefits of measles vaccine on all-cause mortality, seven found all-cause mortality reductions consistent with a benefit of the vaccine on non-measles deaths, taking into account the modelled proportion of measles deaths in the study setting in the absence of measles vaccine (Table 8). Similarly, the study reporting the effects of measles vaccine on other causes of death reported a reduction in non-measles mortality relative to the control arm [51–53]. The greatest reduction was observed in diarrhoea deaths, a 46% decline which would have resulted in a reduction of 5·13 deaths/1000 child-years among measles-vaccinated children in the study [51–53].

Table 7. Measles vaccine: characteristics of included studies

ARI, Acute respiratory infection; CCM, community case management; DSS, demographic surveillance site; IPTi, intermittent preventive treatment of malaria in infants; MCH, maternal and child health; NA, not applicable; PHC, primary health care.

Table 8. Indirect effects of measles vaccine estimated from studies reporting all-cause mortality

ITN, Insecticide-treated net; MR, mortality rate; NA, not applicable; NR, not reported.

^a The formula simplifies to 1 – (1 – B)/D.

^b Study reported estimate of % measles deaths.

^c Study reported measles-specific relative mortality reduction of 0·05 was used to calculate the measles deaths averted.

DISCUSSION

We sought to quantify the effects of key child survival interventions on child mortality above and beyond the observed effect on the targeted COD. While the idea that child survival interventions might impact more than one COD is not new, to our knowledge this is the first systematic approach to estimate the magnitude of these indirect effects.

Among malaria interventions we demonstrated that there does seem to be a small additional effect of ITNs on non-malaria causes of death in some settings. It is unclear which non-malaria causes of death were impacted, given that only one study reported disaggregated non-malaria causes of death [15]. Although most studies reported statistically significant reductions in all-cause mortality, these effects were accounted for by the high rates of malaria mortality in many of these study sites. We observed that in 3/7 studies, the mortality reduction could be completely explained by possible reductions in malaria mortality in the intervention setting. Because malaria is difficult to define in the field, where slide-confirmed malaria is still not routine and other causes of fever such as pneumonia may be confused with malaria, the relatively small magnitude of the indirect effects observed may in part be explained by misclassification in assigning COD. A study of malaria chemoprophylaxis and case management in a setting with moderate malaria mortality, however, did provide evidence suggesting a relatively large indirect effect from chloroquine. This result, however, is based on estimates of the proportion of malaria deaths in that setting, and could therefore be skewed by the difficulty in measuring malaria mortality; a second study of malaria case management, with a higher estimate of the proportion of malaria deaths, showed no indirect effect. There have been several studies of indoor residual spraying alone or in combination with other malaria control measures that have recorded beneficial effects of malaria control on more than just malaria mortality [54–56]. Unfortunately, many of these studies have been observational with no adjustment for confounding and thus were excluded from our review. However, it is worth noting that the consistently observed benefit is suggestive of a benefit on mortality that extends beyond malaria-specific mortality, but studies with the ability to appropriately control for confounding are needed before conclusions can be drawn.

The effect of pneumonia case management was consistent in studies reporting cause-specific and all-cause reductions in mortality. Although one study observed no effect of case management on pneumonia mortality, all studies found an effect on one or multiple non-pneumonia causes of death as well as on the overall non-pneumonia mortality rate. Pneumonia case management involves widespread promotion of antibiotics that can influence the faecal flora and prove to be protective against certain diarrhoea pathogens and thus diarrhoea mortality in general. We did not include the pneumococcal conjugate or Haemophilus influenzae type b vaccines in this analysis because attributing aetiology-specific mortality at the study or country level has not yet been done. However, it has been reported that the benefit of vaccine may well go beyond the reduction in pneumonia-mortality that would be expected if the vaccine only impacted one specific pneumonia aetiology [57]. This supports the effect observed in pneumonia case management and also suggests that pneumonia may be a contributing COD even in deaths directly attributable to other causes.

Most studies of measles vaccine suggested a moderate to large effect of the vaccine on non-measles mortality, but only one study provided cause-specific mortality data explaining which other causes of death the vaccine may have acted upon. Measles leads to a decrease in immune function, increasing the susceptibility of the child to secondary infections for weeks or possibly months following illness [58]; vaccination may decrease the rates of these secondary infections. Our analysis of the indirect effects of measles vaccine, however, was hampered by the fact that many studies reported only the relative all-cause mortality risk in children who had received measles vaccine vs. those who had not, adjusted for possible confounders. Our estimates of the relative risk of non-measles mortality in vaccine recipients relative to non-recipients thus depend in large part on modelled estimates of the proportion of measles deaths expected in the absence of measles vaccine, which may explain the wide range of indirect effects we found, ranging from no effect to an 81% reduction in non-measles mortality. In a few cases, the overall indirect mortality relative rate was >1. For both studies, the all-cause mortality reduction was small, and the proportion of deaths due to measles was large relative to the all-cause effect size. Thus for both studies the measles vaccine only averted some of the measles deaths; because of the formula we used to calculate the indirect relative mortality rate reduction, this appears as an effect size >1.

We were not able to estimate the indirect effects of ORS because we were unable to find studies that met our inclusion and exclusion criteria. Because it would be unethical to actively deny rehydration treatment to children with diarrhoea, no RCTs exist with a comparator that would allow us to estimate the indirect effect of ORS on mortality. Natural experiments and pre/post studies in the 1970s and 1980s took advantage of the introduction of ORS to attempt to measure its effect on mortality; however, these studies did not control for confounding and in some cases experienced contamination of the comparison area or poor coverage in the ORS arm. Given these limitations, the data could not be used to attempt to assess the indirect effects of ORS.

Most studies included in this review did not calculate mortality rates or deaths averted for the targeted cause or other causes of deaths. To perform this analysis we used all available data from the included studies, provided methods were adequately described, and in some cases country-level COD profiles from the LiST tool to estimate the presumed deaths averted from the targeted cause [2, 11, 12] (Table 9). LiST, including country COD profiles and the effectiveness values for interventions, has been used in a number of previous analyses and proven to be an excellent tool for modelling reductions in child mortality [59, 60]. However, there are important limitations in using published country-level COD estimates in lieu of study-specific estimates. LiST estimates are country-level, published peer-reviewed estimates, but are not site-specific which means that there could be great variability in these estimates compared to the actual proportions that would have been observed in the study site. In using this method, for studies reporting only all-cause mortality reductions, we assumed that the intervention would target the index disease first and prevent 100% of the possible deaths from that cause before preventing deaths from other causes. Although this assumption may be flawed, it is the most conservative approach to estimating possible indirect effects and allows for a consistent approach across diseases and interventions. Although all quasi-experimental studies included in this analysis did control for confounding, the full effect of the coverage of other child survival interventions in the intervention areas was not fully described in these papers and thus cannot be fully accounted for in this analysis.

Table 9. Availability of data on the proportion of deaths from the targeted cause

^a Includes one study where a measured estimate from the same country, but a different study, was used.

^b Although several of these studies included estimates of the proportion of measles deaths, most did not provide a definition of a measles death, and those that did used widely varying definitions. Therefore in the interest of consistency modelled estimates from LiST were used.

In this review we included all types of study designs to capture as many studies as possible; all studies were included in previously published primary reviews of their main effects on cause-specific mortality [4–64–6, 10], except in the case of ITNs where we included several additional studies. However, we applied our own inclusion and exclusion criteria to ensure all studies met a minimum quality standard specific for this analysis. We were limited in our interpretation to the published studies that often included rigorous studies implemented under conditions not typical of real life. In some cases, despite attempts made by the researchers to mimic real life, the presence of a study benefits the control group in that mortality is decreased without an intervention. This may bias our results towards the null.

This analysis was designed to bring together published literature in an effort to quantify the indirect effects, if any, of selected interventions on causes of death other than those the interventions were intended to prevent. While in some cases we observed effect sizes larger than what might be expected if an intervention were only operating on its targeted disease, additional studies are needed to determine if these larger effect sizes (consistent with indirect effects of the interventions) are consistent across populations with different COD profiles. In addition, as new studies are designed and performed to assess mortality reductions, possible indirect effects should be considered and causes of death recorded for the major childhood diseases despite power and sample size restrictions. Until COD data are reported in addition to all-cause mortality reduction, we will not be able to move beyond projected indirect effects using modelling as we did in this analysis. With additional data to better describe averted COD, programmes such as LiST would be better able to capture the true mortality reduction. In addition, future versions of LiST may be able to incorporate uncertainty bounds on estimated mortality reductions.

At the present time, while we know some interventions do avert more deaths than the direct cause, the indirect causes are often ignored during the scale up of interventions and not captured as part of ongoing programme evaluations. It should also be noted that as the impact of interventions may be greater than what has been demonstrated in tightly controlled trials, the reverse would also be true. That is, eliminating or scaling back selected child survival interventions could result in a higher than expected increase in child mortality. Both of these effects, i.e. the indirect effects possible if choosing to initiate or scale up an intervention as well as the potential indirect effects of limiting or discontinuing an intervention should be considered during the programme planning process. Finally, as multiple interventions are rolled out simultaneously in many countries to increase child survival, research needs to be done to quantify the combined effect of intervention scale up. If indirect effects can be observed when considering isolated interventions designed to target one disease, it is possible these effect sizes may change with several interventions delivered at once.