Diagnostic uncertainty of herpangina and hand-foot-and-mouth disease and its impact on national enterovirus syndromic monitoring

Summary

The community burden of enterovirus is often monitored through syndromic monitoring systems based on reported cases of enterovirus-related infection (EVI) diagnoses. The extent to which this is affected by under- and over-diagnosis has not been reported. In Taiwan, children often make more than one healthcare visit during an episode of infection. We used change of diagnosis within an episode of infection as a guide of diagnostic uncertainty in a nationally representative cohort of Taiwanese children (n = 13 284) followed from birth to the 9th birthday through electronic health records. We conducted a nested case-control analysis and estimated cross-diagnosis ratios (CDRs) as the observed proportion of acute respiratory infection (ARI) diagnoses following an EVI diagnosis in excess of background ARI burdens. With 19 357 EVI diagnoses in this cohort, the CDR within 7 days was 1·51 (95% confidence interval 1·45–1·57), confirming a significant excess of ARI diagnoses within the week following an EVI diagnosis. We used age-specific CDRs to calibrate the weekly EVI burden in children aged 3–5 years in 2008, and the difference between observed and calibrated weekly EVI burdens was small. Therefore, there was evidence suggesting a small uncertainty in EVI diagnosis, but the observed EVI burdens through syndromic monitoring were not substantially affected by the small uncertainty.

INTRODUCTION

Enteroviral infections are common in early childhood [1–4]. Although the majority of infections are self-limiting, some subtypes are associated with life-debilitating consequences such as meningitis, encephalitis, or cardiomyopathy, and have posed major public health threats in Asia [5–9], while potentially emerging in other countries [10, 11]. National intelligence of circulating enteroviruses in many countries has relied primarily on laboratory-based surveillance. In Asian countries where the burden of enterovirus to the healthcare systems is high and requires immediate response, this burden is additionally monitored through syndromic reports of cases with enterovirus-associated diagnoses, mostly herpangina and hand-foot-and-mouth disease (HFMD) [1–4]. However, overlapping presentations are common between enterovirus-related infection (EVI) diagnoses and acute respiratory infections (ARIs) [12], and the monitored syndromic burden of EVIs may be affected by over- or under-diagnosis of EVI. The extent of this effect has not been investigated.

In Taiwan, diagnoses of primary and secondary healthcare visits are routinely recorded electronically and stored centrally. These diagnostic records are rich data sources to identify childhood infectious diagnoses, and are used as a tool for real-time monitoring of enteroviral trends [2]. The high accessibility of medical care among young children means a large proportion visit a physician several times during a course of infection. Discrepancies of diagnoses during several visits of an infectious episode can be used to assess the degree of diagnostic uncertainty in this episode. A high degree of uncertainty may indicate poor accuracy in EVI diagnosis that may affect the precision of the enterovirus syndromic monitoring system. In this study, we estimate the extent to which monitored EVI burdens through the syndromic monitoring system might be affected by cross-diagnosis between ARIs and EVIs.

METHODS

Complete ambulatory care medical history of 13 284 Taiwanese children born in 2000 were followed from birth to their 9th birthday through the Taiwan Longitudinal National Health Insurance Research Dataset 2005 [13]. Briefly, this is a nationally representative, random and anonymous sample of 1 million people of the Taiwanese population in 2005 from the National Health Insurance Database, which covered >99% of Taiwanese population. In this database, primary diagnoses of all medical visits were recorded according to the International Classification of Diseases, 9th Revision (ICD-9) since 2000, allowing consistent coding criteria to identify diagnoses of ARI (ICD-9: 460–466) and EVI (ICD-9: 074.0, 074.3) during the follow-up of this birth cohort. For ARI we included most minor acute respiratory illness such as acute pharyngitis, acute sinusitis, etc., which can be caused by any virus, including enteroviruses. For EVI we included herpangina (074.0) and HFMD (074.3) to be consistent with the enterovirus syndromic monitoring system in Taiwan. Therefore, wherever EVI is referred to the species associated with severe consequences were implied.

We calculated the incidence of ARI and EVI by week of age from birth to the 9th birthday in this cohort born in 2000. To examine any findings specific to birth cohort, we also compared the incidence to (the incidence of) another cohort born in 2003 from birth to the 6th birthday, using the same data source and extraction method.

Nested case-control analysis

Because any ARI diagnosis after EVI could simply reflect the higher background incidence of ARI compared to the background incidence of EVI, we conducted a nested case-control study to test whether there were occurrences of ARI after EVI in excess of the background incidence of ARI. The case-control study was nested in the original cohort (born in 2000) to compare occurrences of ARI diagnoses following a case date and a control date. Children of the same sex and born in the same week and the same region (Northern, Central, Southern, Eastern, and others) [13] were selected into comparison groups. Occurrences of ARI diagnoses following the dates on which a child had an EVI diagnosis (case dates) were compared to the occurrences of ARI diagnoses following the same dates of other children (control dates) in the same comparison group. Dates with an EVI diagnosis in the past 7 days, or on which there was an ARI diagnosis, were excluded from analyses. Diagnostic uncertainty of EVI was quantified in terms of cross-diagnosis ratios (CDRs) using conditional logistic regression (SAS v. 9.4; SAS Institute Inc., USA) as the relative risk of case dates vs. control dates of the proportions of ARI diagnoses following the date, stratified by comparison group and index date. Therefore, children contributed to case dates on the dates they had an EVI diagnosis, and contributed to control dates on the dates other children in the same comparison group had an EVI diagnosis.

We calculated CDRs 1–7, 8–14, 15–21, and 22–28 days following the index date. A CDR >1·0 within a short period following an index date suggested that children with an EVI diagnosis were given ARI diagnoses more often than expected according to background ARI burdens after the EVI diagnosis, and may indicate some episodes of ARI were diagnosed as EVI in the first visit of the infectious episode. By contrast, a CDR >1·0 within a longer period may indicate existence of other long-term factors of excessive ARI diagnosis, such as high medical accessibility or an environment with high circulating virus levels among those who were diagnosed with EVI.

χ ² tests were used for heterogeneity of CDRs across different subgroups or conditions, including sex (boys and girls), age (0–2, 3–5, 6–8 years), season (in four 3-month periods starting from December), and the burden of ARI and EVI in the prior week in quartiles, calculated from the Taiwan Longitudinal National Health Insurance Research Dataset 2005. Because all children in this dataset were born in or before 2005 and were aged ⩾4 years by the end of 2009, we used the total number of ARI or EVI diagnoses in children aged 4–6 years to represent prior-week ARI and EVI burdens.

We also used conditioned CDRs to calculate the population attributable fraction [14] of ARI that could have been cross-diagnosed as EVI, and calibrated the observed weekly EVI burdens by subtracting the numbers of cross-diagnosed cases in the week.

RESULTS

Characteristics of children in this cohort are shown in Table 1. There were 1 486 926 ARI diagnoses in this cohort before the 9th birthday (112 ARI per child on average), compared to only 19 357 EVI diagnoses (1·5 per child). The burden of ARI and EVI varied substantially with age. Figure 1(a, b) shows the proportion of children who had ARI or EVI at each week of age from birth to their 9th birthday (in fine dots), plotted against 5-week moving averages (solid lines). We observed two age peaks of ARI, around the first and fifth birthdays. Two age peaks were also observed for EVI, at around the second and fifth birthdays. Due to viral seasonality, these age peaks varied slightly according to time of the year the children were born (Supplementary Appendix P2). The gap between two peaks coincided with the outbreak of severe acute respiratory syndrome in 2002–2003, during which health visits declined substantially, as reported previously [13]. We examined the burden in another cohort of 9383 children born in 2003 (Figure 1c, d ). The burden of ARI by week of age was substantially lower in this cohort, and the peak at around age 5 years was less obvious. By contrast, the burden of EVI by week of age was similar between the two cohorts.

Fig. 1. Disease burden (dots) and 5-week moving average (lines) of acute respiratory infection (ARI) and enterovirus-related infection (EVI) diagnoses by week of age in the study cohort born in 2000 from birth to the 9th birthday (a, b) and in a comparison cohort born in 2003 followed from birth to the 6th birthday (c, d).

Table 1. Characteristics of the study cohort and diagnoses

ARI, Acute respiratory infection; EVI, enterovirus-related infection, including herpangina and hand-foot-and-mouth disease.

* Sex information is missing in <0·1% of diagnoses. Information was obtained from other episodes of the same child in further analyses.

Diagnosis of ARI was common following an EVI diagnosis

The proportions having the same or a different diagnosis within 28 days following a previous healthcare visit are shown by three age groups (0–2, 3–5, 6–8 years) in Figure 2. For both ARI and EVI diagnoses, the proportion having a second visit with the same diagnosis was generally lower in older age groups, but there was qualitative difference between ARI and EVI: unlike ARI for which a substantial proportion of same diagnosis consistently occurred throughout 28 days after the initial diagnosis (Fig. 2a ), most repeated diagnoses for EVI occurred within 7 days (Fig. 2b ). Across all age groups, the proportion having an EVI diagnosis following an ARI diagnosis or other diagnoses was very low (⩽2% within 28 days, Fig. 2c ). By contrast, there were substantial proportions of children diagnosed with ARI within 7 days following an EVI diagnosis (16–28%, Fig. 2d ).

Fig. 2. Cumulative proportions having an acute respiratory infection (ARI) or enterovirus-related infection (EVI) diagnosis within 28 days following a previous same diagnosis (a, b) and following another diagnosis (c, d) in three age groups.

These findings confirmed our speculation that there was indeed high background incidence of ARI in this population, and any ARI occurrence after EVI could be largely attributed to the background incidence of ARI. It would be necessarily to use a nested case-control design to investigate whether there was ARI incidence after EVI in excess of the background incidence of ARI (see below).

The proportion of ARI diagnoses following an EVI diagnosis was in excess of background ARI burdens

Using the nested case-control approach, we observed excessive ARI diagnoses 1–7 days after an EVI diagnosis (CDR 1·67, 95% CI 1·61–1·74). Excessive ARI diagnoses were also observed 8–14, 15–21, and 22–28 days following an EVI diagnosis, although the CDRs were substantially smaller (Table 2, left column). We further adjusted the CDRs for the total number of ARI diagnoses each child had at age 1, 4 and 7 years as a general indication of having an ARI diagnosis. The CDR 1–7 days after an EVI diagnosis was attenuated after adjustment (CDR 1·51, 95% CI 1·45–1·57), and so were the CDRs 8–14, 15–21, and 22–28 days after adjustment (Table 2, right column).

Table 2. Cross-diagnosis ratios by duration after the first enterovirus-related infection diagnosis

ARI, Acute respiratory infection; CDR, cross-diagnostic ratio; CI, confidence interval; EVI, enterovirus-related infection, including herpangina and hand-foot-and-mouth disease.

Case and control dates were matched by week of birth, sex, and region. CDRs and 95% CIs were estimated using conditional logistic regression. Tendency to have ARI diagnosis is indicated by total numbers of acute respiratory infections in the year of ages 1, 4, and 7 years of each child.

The adjusted CDRs did not vary with sex, but did vary with characteristics that may affect an accurate EVI diagnosis (Table 3). Adjusted CDRs were higher in older (age 6–8 years) and younger (age 0–2 years) age groups compared to the age when disease presentation was considered typical (age 3–5 years, P for heterogeneity <0·0001). Adjusted CDRs were also found to be highest in summer (June–August). We then stratified the analyses by community burden of ARI and EVI in the prior week. The Spearman correlation between burden of ARI and EVI in the same week was −0·11, P = 0·01 (522 weeks in total). High adjusted CDRs were found to be associated more strongly with lower (compared to higher) ARI weekly burdens, and less strongly with lower (compared to higher) EVI weekly burdens (P for heterogeneity = 0·0002 for ARI and 0·03 for EVI).

Table 3. Cross-diagnosis ratios by age, season, and community burden of ARI or EVI in the prior week

ARI, Acute respiratory infections; CDR, cross-diagnostic ratio; CI, confidence interval; EVI, enterovirus-related infection, including herpangina and hand-foot-and-mouth disease.

Case and control dates were matched by week of birth, sex, and region. CDRs and 95% CIs were estimated using conditional logistic regression, adjusted for total numbers of acute respiratory infections at ages 1, 4, and 7 years.

Cross-diagnosis between ARI and EVI did not substantially affect enterovirus syndromic monitoring system

To quantify the extent to which observed EVI burdens may be inflated by over-diagnosis, we calculated CDRs specific to age and prior-week ARI burden, and used the CDRs to calculate population attributable fractions of ARI that could have been over-diagnosed as EVI, and then used this fraction to calculate numbers of over-diagnosed EVI (Supplementary Appendix P3). As a visual example, we plotted the observed weekly EVI burden for age 3–5 years in 2008 against the calibrated weekly burden. As shown in Figure 3, the difference between observed and calibrated weekly EVI burdens was small.

Fig. 3. Observed burden of acute respiratory infections (ARI, upper half) and observed (black) and calibrated (grey) burdens of enterovirus-related infections (EVI, lower half) in 2008 in children aged 3–5 years in Taiwan, by weekly burden (dots) and 5-year moving average (lines).

DISCUSSION

In this national representative birth cohort in Taiwan based on routinely collected diagnostic information from primary and secondary healthcare databases, we investigated the extent to which the enterovirus syndromic monitoring system may have been affected by diagnostic uncertainty of EVI. Because it was not possible to verify each diagnosis, we assumed that excessive ARI diagnoses following an EVI diagnosis were due to ARI being over-diagnosed as EVI at the first visit. We quantified these excessive ARI diagnoses in terms of CDRs as a guide of EVI over-diagnosis, and used CDRs to calibrate syndromic monitoring observations. Our analyses confirmed that there were excessive ARI diagnoses following an EVI diagnosis, but the observed burdens of enterovirus was not substantially affected.

Age and seasonality are major factors of ARI and EVI, and this is mostly demonstrated in cross-sectional studies [3, 4, 15, 16]. In this study we followed each child's complete medical history from birth to the 9th birthday in a large and nationally representatively sample of Taiwanese children, and summarized the actual trajectory of ARI and EVI. We were able to compare the trajectory with that of another cohort born in 2003, and the cohort difference of EVI was small. We also demonstrated that most repeated EVI diagnoses in this cohort occurred within 7 days following the previous one, consistent with the natural disease course [16]. Altogether, with the finding that observed enteroviral burden based on EVI diagnosis was not affected substantially by diagnostic uncertainty, our results suggest good credibility in this enterovirus syndromic monitoring system.

One major limitation of this study is residual confounding. Although we have adjusted for number of ARI diagnoses at ages 1, 4, and 7 years, the excessive ARI diagnoses following an EVI diagnosis could still be confounded by residual effect of medical accessibility or health-seeking behaviour, or by other confounding factors. The diagnostic criteria for ARI and for EVI may vary by case, by season, or by diagnosing physician. Cases and controls were matched by week of birth, sex, and region, but we could not exclude the possibility that children who had EVI were more susceptible to other infections or were in an environment where ARI more likely occurred. Nevertheless, the low CDRs 7 days after adjustment, compared to those within 1–7 days, could suggest that any residual confounding is relatively small.

We did not quantify the proportion of EVI under-diagnosis. Because the proportion of EVI diagnoses following an ARI diagnosis was very small within 7 days, the risk of under-diagnosis in this population is relatively small. We also did not quantify the proportion of EVI over- or under-diagnosis related to cross-diagnosis with other diseases. Because these diseases do not cause a burden comparable to the size of ARI, the risk of inflation or deflation caused by these cross-diagnoses may also be small.

Although the study cannot explicitly account for all sources of diagnostic uncertainty, the approach is comparable to using repeated measurements to assess reproducibility regardless of the source of errors. The syndromic monitoring system for enterovirus infections in Taiwan takes the advantage of real-time registration of genuine clinical impressions from clinical settings, but the reliability inevitably relies on general consensus of diagnostic criteria among physicians across the country. Deviation from this general consensus could be a source of confounding or diagnostic uncertainty. Syndromic monitoring uses one or more enterovirus-related diagnoses as indicators of healthcare burden attributed to enteroviruses. Because presentations of enteroviruses could depend on various viral and host factors such as viral subtypes, host age, or immune status, the syndromic monitoring may not always reflect this variation and be prone to errors. We used the nested case-control study design, and observed a small effect of diagnostic inconsistency on the estimated burden of enteroviruses based on syndromic monitoring.

The heterogeneity between CDRs has suggested some host or environmental factors may have contributed to diagnostic uncertainty. The higher CDRs in younger and older children may suggest difficulties in diagnosing EVI in these age groups, perhaps due to non-specific presentations and difficulties in differential diagnosis. The higher CDRs associated with summer season, low prior-week ARI or EVI burden may affect children's exposure history, healthcare-seeking behaviour, or physician's diagnostic preference. Our findings suggest these factors, wherever accountable in our analyses, may not substantially affect our observation of EVI burden at the population level in Taiwan. To which extent these factors affect individual EVI diagnosis is unclear. To which extent our findings can be generalized to other populations requires further investigation.