Optimal design of studies of influenza transmission in households. II: Comparison between cohort and case-ascertained studies

Summary

Both case-ascertained household studies, in which households are recruited after an ‘index case’ is identified, and household cohort studies, where a household is enrolled before the start of the epidemic, may be used to test and estimate the protective effect of interventions used to prevent influenza transmission. A simulation approach parameterized with empirical data from household studies was used to evaluate and compare the statistical power of four study designs: a cohort study with routine virological testing of household contacts of infected index case, a cohort study where only household contacts with acute respiratory illness (ARI) are sampled for virological testing, a case-ascertained study with routine virological testing of household contacts, and a case-ascertained study where only household contacts with ARI are sampled for virological testing. We found that a case-ascertained study with ARI-triggered testing would be the most powerful design while a cohort design only testing household contacts with ARI was the least powerful. Sensitivity analysis demonstrated that these conclusions varied by model parameters including the serial interval and the risk of influenza virus infection from outside the household.

INTRODUCTION

Influenza virus is associated with substantial morbidity and mortality worldwide [1]. Households are an important confined setting for influenza transmission [2–6], and it has been estimated that around a third of all influenza transmission occurs in households [3–5]. Recent household studies have investigated the effectiveness of antiviral treatment and prophylaxis [7–11], hand hygiene [12–15], face masks [13–17], and transmissibility of seasonal influenza [18] and 2009 pandemic influenza A(H1N1) [19–27].

There are two main types of design for household studies that are useful in investigating the efficacy of interventions to prevent household transmission. The first type is a cohort study in which a cohort of initially uninfected households can be recruited and then followed up through periods of influenza activity [6, 28]. This design will be resource intensive if the expected number of households in which an infection occurs is relatively small. Alternatively, households can be enrolled in a study once an influenza infection is identified in one member (an ‘index’ case), and subsequently followed up to observe secondary infections. This design is termed a case-ascertained design [29], and efficient planning of case-ascertained studies was the focus of our first paper in this series [30]. Compared to cohort studies the case-ascertained design may suffer from selection bias, because index cases who present to clinics with symptoms may have more severe illness than influenza virus infections on average, resulting in the introduction of potential selection biases in the assessment of transmission dynamics [25]. Additionally, difficulty in interpreting data from case-ascertained studies may occur when treatment efficacy decreases with time since symptom onset [13, 14] due to the possibility of infectious contacts being made before the intervention was applied. In both cohort and case-ascertained studies, influenza virus transmission is typically measured via the secondary attack proportion (SAP), defined as the proportion of household contacts that are infected with influenza virus from the index case [19], and sometimes approximated by the proportion of household contacts of an index case that subsequently become infected with influenza (regardless of the presence or absence of direct transmission from the index case) [31].

While some household studies rely entirely on self-reports of symptoms and signs that are associated with acute respiratory illnesses (ARIs) [16–21], generally home visits can be arranged to collect specimens and allow virological confirmation of influenza virus infections [13, 15, 23–26]. Furthermore, specimens for virological confirmation can be collected for all household members at some optimal time after symptom onset in the index case [13, 30] or specimens can be collected only when household contacts report ARI [8, 10, 30]. This raises an important question: which combination of study design and paradigm for collection of specimens would be the most powerful option to evaluate and compare the protective effect of interventions? While smaller sample sizes are required in case-ascertained designs to observe an equivalent number of secondary infections compared to a cohort study [29, 30], this question has not been previously studied systematically in the literature.

Identifying and selecting an appropriate study design is part of good clinical practice in all clinical research studies. Poorly designed studies may waste precious resources, have inappropriate statistical power and put participants at unnecessary risk and inconvenience [32]. In the present study, we evaluated which study designs make most cost-effective use of resources for maximizing statistical power, in the context of a potential trial of a non-pharmaceutical intervention (NPI).

METHODS

As a basic scenario, we consider planning a study to assess the effectiveness of a NPI study in which we aim to demonstrate that the NPI reduces household transmission compared to a control or minimal intervention. We consider four possible designs for this study:

1. (1) A household cohort study with confirmatory diagnosis of index cases with reverse transcriptase–polymerase chain reaction (RT–PCR) followed by routine collection of specimens of all household members and testing with RT–PCR regardless of reported illness.

2. (2) A household cohort study with confirmatory diagnosis of index cases with RT–PCR followed by collection of specimens and testing with RT–PCR of household members upon report of ARI.

3. (3) A case-ascertained study with routine virological testing of all household members with RT–PCR regardless of reported illness.

4. (4) A case-ascertained study with ARI-triggered collection and testing of specimens of household members.

Here we define ARI as presence of two of the following signs or symptoms: fever (⩾37·8°C), cough, headache, sore throat, or myalgia [13, 25].

Cohort studies

In these scenarios, we assume that households are recruited in advance of an influenza epidemic. Households are randomly assigned in equal numbers to either an intervention or control group. Household members are instructed to implement the intervention if any household member develops ARI. The aim of the intervention is to reduce household transmission rather than to reduce the risk of infection from the general community. Participating households are encouraged to contact the study team if any member of the household developed symptoms of ARI. Bi-weekly phone calls can also be made to monitor for ARI. As soon as a household reports ARI, a home visit is made to collect respiratory specimens for virological testing. If an index case tested positive for influenza virus infection, either one additional home visit would be made to collect specimens for testing household contacts or a home visit would be made after another household member reported ARI.

Case-ascertained studies

In this scenario, we assume that relatively inexpensive point-of-care rapid tests for influenza are used to identify influenza virus infection in individuals with ARI presenting to a study clinic [33]. Once a case is identified, the index case and his/her household are recruited for inclusion in the study. For those index cases with a positive rapid test result, an initial home visit is conducted as soon as possible to collect respiratory specimens from all household contacts for laboratory testing to determine whether there were any co-primary cases. If a co-primary case was identified, that household would not be enrolled in the study [13, 34]. We assume that a home visit confirms that the remaining household members are negative for influenza and also that the household can be enrolled. It is assumed that the intervention takes place in the enrolled household between 12 and 60 h after symptom onset of the index case. Either an additional home visit 6 days after symptom onset in the index case is made [30] or a home visit is conducted immediately after a report of onset of ARI in a household contact to collect respiratory specimens for virological confirmation of potential secondary infections.

Sources of data

Some basic parameters are required for the simulation characterizing influenza transmission in households (Table 1). We assume that in addition to the index case there are three additional household members. We also assume that within-household tertiary transmission is relatively rare and has limited effect on the estimation of the SAP. Estimation of the SAP using the generalized estimating equation (GEE) approach described below accounts for clustering within households but does not explicitly model chains of transmission [35]. While our model adopts a single value for an overall mean SAP, we allow the actual SAP to vary stochastically in households. We also assume there is some small correlation between household SAP and illness severity. The index cases with greater illness severity are more likely to be enrolled in case-ascertained studies as their symptoms must be severe enough to require medical attention.

Table 1. Baseline epidemiological parameters

ARI, Acute respiratory illness; CA, case-ascertained; NPI, non-pharmaceutical intervention; PCR, polymerase chain reaction; SAP, secondary attack proportion.

Costs must also be considered, since an optimal design must involve a trade-off between the power and the cost required for a certain sample size with the number of follow-up visits. We specified the recruitment, enrolment and testing costs per household for case-ascertained and cohort studies based on previous household studies conducted in Hong Kong.

Statistical analysis

For both cohort and case-ascertained study design variants, we estimated the power of each potential variant to detect a NPI effect in terms of a reduction in the estimated SAP in the intervention vs. the control group, expressed as an odds ratio. A logistic regression model with GEE [35, 36] accounting for within-household correlation was fitted to the simulated data. The statistical power was estimated as the number of simulated datasets in which the intervention effect was identified at a significance level of P⩽0·05.

Due to the nonlinearities in the transmission dynamics of influenza, we used a simulation approach to compare alternative study design variants [37]. For each study design variant we used a Monte Carlo approach to simulate a set of 2500 datasets. The power of each variant was evaluated by statistical analysis of the set of 2500 simulated datasets and compared across design variants. We chose 2500 iterations to ensure the Monte Carlo error was ⩽0·01 [38]. Further technical details are provided in the online Supplementary Appendix.

Sensitivity analyses

To examine the impact of variations in key model parameters on the optimal study design, we performed a set of sensitivity analysis varying several model parameters. The effect of changes in the community probability of infection (CPI) was examined, because it influences the rate at which a susceptible individual acquires infection from the community during the influenza epidemic [39]. Another sensitivity analysis examined the optimal study variants with shorter [3, 19, 40] or longer [41] serial intervals than assumed in the baseline scenario. If the serial interval was shorter, this could allow more infectious contacts before the intervention is implemented. Finally, we examined the sensitivity of enrolment cost per household on the optimality of case-ascertained and cohort studies.

RESULTS

The results of our analysis using baseline parameter values are shown in Figure 1. The number of households that can be recruited per arm given a fixed fieldwork budget is given in Figure 2. The case-ascertained design with ARI-triggered collection of samples had the highest power. This was followed by a cohort study with ARI collection of samples and a case-ascertained study with routine collection of virological specimens regardless of reported symptoms. The least powerful design was found to be a cohort study with routine collection of specimens once an index case was identified regardless of reported symptoms in the household contacts. Budgets of US$1.7 million (1152 households per arm), US$1.9 million (1659 households per arm), US$2.1 million (1104 households per arm), and US$2.3 million (1557 households per arm) were required to achieve a study of 80% power for a case-ascertained study with ARI trigger, a cohort study with ARI trigger, a case-ascertained study with routine testing, and a cohort study with routine testing, respectively.

Fig. 1. Statistical power of competing study designs as a function of study budget. ARI, Acute respiratory illness; CA, case-ascertained; PCR, polymerase chain reaction.

Fig. 2. Number of households per arm that can be enrolled for competing study designs as a function of study budget. ARI, Acute respiratory illness; CA, case-ascertained; PCR, polymerase chain reaction.

Sensitivity analyses

The total budget needed to achieve 80% statistical power from sensitivity analyses varying key parameter is shown in Figure 3. Sensitivity analyses showed that either case-ascertained or cohort designs could be the most powerful design depending on parameter values. The power of cohort designs was sensitive to the CPI (Fig. 3, Appendix Fig. S1). A higher CPI causes cohort studies to have higher power. The power of case-ascertained designs was highly sensitive to the serial interval (Fig. 3, Appendix Fig. S2). As was intuitively expected, shorter serial intervals led the intervention to be less effective. The power of both case-ascertained and cohort studies were sensitive to enrolment costs, leading either case-ascertained or cohort studies to be most powerful depending on parameter values (Fig. 3, Appendix Figs S3, S4).

Fig. 3. Total budget necessary to achieve 80% statistical power. Diamonds represent estimates from sensitivity analyses while squares represent estimates from the main analysis. (a) Sensitivity analysis showing the effect of changes to the community probability of infection (CPI). Lower diamonds represent results with a low CPI of 0·10 and upper diamonds represent results with a high CPI of 0·30. Changes in CPI do not affect the results for case-ascertainted studies. (b) Sensitivity analysis showing the effect of changes to the serial interval of influenza. Lower diamonds represent the model results with a shorter serial interval (mean 2·6) and upper diamonds represent the model results with a longer serial interval (mean 3·6). (c) Sensitivity analysis showing the effect of changes to the enrolment cost of households. Lower diamonds represent the model results reducing enrolment costs by 33% and upper diamonds represent increasing enrolment costs by 33%. ARI, Acute respiratory illness; CA, case-ascertained; PCR, polymerase chain reaction.

DISCUSSION

Careful consideration is required when planning household transmission studies of influenza. Our results illustrate that case-ascertained designs are the most resource efficient design for testing NPI efficacy. However, this conclusion was sensitive to the costs of enrolment, cost of laboratory methods, and the risk of influenza in the community. Since all cohort studies are very sensitive to CPI, and because this probability markedly varies throughout the course of influenza seasons and cannot be accurately predicted in advance, case-ascertained designs may be considered to be less vulnerable to epidemic dynamics than cohort designs. As we found in the first paper in this series, using ARI to trigger home visits can often be a cost-effective strategy in both case-ascertained and cohort designs. However, it should be noted that use of ARI trigger may miss some asymptomatic and subclinical infections [42] and also that conducting routine home visits may encourage intervention adherence for control purposes.

Several practical considerations may help in the choice between a case-ascertained and cohort design. Testing NPIs using a case-ascertained design may be practically considered as the best-case scenario, because households have just been recruited into the study and taught how to use an intervention. Therefore, it might be expected that adherence to the intervention within households recruited for a case-ascertained study would be higher than it would be in the general population during an influenza epidemic or pandemic. However, case-ascertained studies will always suffer from some truncation problems because there will inevitably be a delay between symptom onset in the index case and the implementation of an intervention. Therefore some secondary transmission may occur before the start of the intervention and this has been noted in several case-ascertained studies [13, 14]. Cohort studies may permit more representative estimates of the efficacy of NPIs in the general population, while it should be borne in mind that adherence may be lower than a case-ascertained design due to waning compliance over time. It is also notable that case-ascertained study designs may be a good choice for assessing secondary aims such as the duration and severity of symptoms and viral shedding [42], while cohort designs could also address secondary aims such as annual incidence and risk factors for infection and illness.

Our study has some limitations when considering the application of the results to planning actual studies. First, our estimates of the costs of enrolment and testing are specific, based on our experience in Hong Kong. These may differ in other countries in a manner that would affect the results, and thus, the optimality should ideally be calibrated to other settings. Second, we did not consider self-swabbing of study participants as a strategy for confirmation of influenza virus infections. This strategy has been proposed as a cost-saving and convenient method for collection of respiratory specimens for testing [43–46]. While early evidence is promising and suggests that sensitivity for self-swabbing may be nearly as high as when swabs are collected by trained personnel [43–46], questions remain about participant compliance and overall cost savings of this strategy. Third, we did not consider identification of influenza using serological methods due to uncertainties relating to the costs and accuracy relative to RT–PCR confirmation. However, household studies using serological testing have been previously used to test NPI efficacy [47]. Fourth, we assumed that in case-ascertained studies that index cases presenting to clinics less than 48 h after symptom onset would be eligible to participate. If this time could be shortened, then the cost efficiency of case-ascertained studies would be elevated. Fifth, it is difficult to gauge adherence in cohort studies and the study protocol could affect adherence. Finally, several papers [29, 48] have noted that direct randomization of household members rather than cluster randomization of households is more powerful and can allow for the estimation of the efficacy of an intervention in both reducing the risk of infection for susceptible household members and reducing the infectiousness of an infected household member. However, for NPIs the intervention cannot usually be blinded from subjects, and cluster randomization may be more feasible and acceptable for implementation.

Despite the sensitivity of our models to the choice of parameters and potential limitations, our results serve as useful guidelines for researchers in planning future household studies of influenza interventions. Our findings suggest that none of the designs performed particularly poorly and that secondary considerations might warrant the use of designs not considered optimal in terms of statistical power.