Evaluating measles surveillance: comparison of sentinel surveillance, mandatory notification, and data from health insurance claims

Summary

Inadequate notification is a recognized problem of measles surveillance systems in many countries, and it should be monitored using multiple data sources. We compared data from three different surveillance sources in 2007: (1) the sentinel surveillance system mandated by the Act on Prevention of Infectious Diseases and Medical Care for Patients Suffering Infectious Diseases, (2) the mandatory notification system run by the Aichi prefectural government, and (3) health insurance claims (HICs) submitted to corporate health insurance societies. For each dataset, we examined the number of measles cases by month, within multiple age groups, and in two categories of diagnostic test groups. We found that the sentinel surveillance system underestimated the number of adult measles cases. We also found that HIC data, rather than mandatory notification data, were more likely to come from individuals who had undergone laboratory tests to confirm their measles diagnosis. Thus, HIC data may provide a supplementary and readily available measles surveillance data source.

INTRODUCTION

Measles remains highly endemic in Japan [1,2]. An estimated 200 000 cases of measles and 88 measles-related deaths (mainly in children) were reported in 2000, making it necessary for the implementation of effective control measures for measles epidemics [3]. In 2007, more than 4000 cases of measles were reported through the sentinel surveillance system in Japan, but it is estimated that the actual number of cases is 10 times this number [3,4]. Measles epidemics continue to occur every several years in Japan [5]; in order to eliminate this infection entirely, it is necessary to develop clear strategies and prioritizations for vaccine policies, both of which can be guided by accurate and long-term measles surveillance [6].

Case reports to the surveillance system are collected passively from medical facilities, especially private clinics. Because, for a variety of reasons, physicians fail to report measles cases, the incidence is substantially underestimated by the surveillance system [7]. Thus, it is necessary to verify that any decline in the reported incidence of measles is due to the actual decline of the disease, rather than a failure of surveillance.

Inadequate notification is a recognized problem of measles surveillance systems in many countries [8–11]. Recently, several countries have replaced their measles surveillance systems or introduced additional systems in order to improve case reporting. In Japan, only sentinel medical facilities reported measles cases until December 2007. In January 2008, it became mandatory for all medical facilities in Japan to report every measles case. In France, because of serious underreporting, mandatory notification for measles was replaced with sentinel surveillance in 1985. However, with the current low incidence of measles, the sensitivity of this new system was found to be insufficient, and mandatory notification was recently reintroduced [11]. In Italy, a paediatric sentinel network for measles was established in 2000 in addition to mandatory notification, which is characterized by major underreporting [9]. In Germany, a sentinel system for measles was launched in 1999, followed by statutory surveillance in 2001 [10].

High-quality reporting, particularly at the beginning of an outbreak, can prevent disease spread by facilitating rapid intervention [12]. For this to happen, it is necessary to periodically evaluate surveillance systems to ensure that they provide accurate information in a timely manner. Despite the fact that the Japanese surveillance system is known to underestimate the actual number of measles cases, only a few studies have described the quality of the country's infectious disease surveillance system [3,4,13]. One method of evaluating underreporting is to examine data that come from sources other than physician notifications to the surveillance system.

The sentinel surveillance system for infectious diseases in Japan was mandated by the Act on Prevention of Infectious Diseases and Medical Care for Patients Suffering Infectious Diseases (Infectious Disease Prevention Law). Until December 2007, data in this system came from two separate sources. Paediatric in-patient and outpatient sentinel facilities reported cases of children (aged <15 years), while sentinel hospitals (hospitals with ⩾300 beds and full-time paediatricians and physicians) reported cases of adult patients (aged ⩾15 years). Facilities without in-patient care were excluded from the sentinel surveillance system for adult measles cases. Since January 2008, all medical facilities were requested to report every measles case.

The case definition of the measles surveillance system in Japan is mainly based on clinical diagnosis, and laboratory data are used as supplementary data. The Infectious Disease Prevention Law defined measles as the presence of a generalized rash and fever (⩾38·5°C), as well as cough, coryza, or conjunctivitis, or a laboratory-confirmed diagnosis of measles. Laboratory confirmation was defined as the detection of measles-specific immunoglobulin M (IgM) antibodies or positive results from measles virus isolation/detection and genotyping procedures. Although the local health authorities suggest that physicians also conduct laboratory tests and submit their results along with clinical diagnostic details, this is not a mandatory part of the reporting process and, therefore, also applies to measles surveillance throughout Japan. The proportion of measles patients who received diagnostic testing to confirm the presence of measles was 11·3% [13]; however, there has been no evaluation of the proportion of measles patients undergoing laboratory tests to confirm a measles diagnosis.

In Japan, health insurance coverage is universal; healthcare providers use a uniform claims format and fee schedule to submit health insurance claims (HICs) to insurers in order to claim the costs of healthcare services (except patient co-payments). Because all the format and data fields recorded on HICs are identical regardless of the insurance provider [14], they can be used to evaluate the effects of health policy changes on patients' behaviours [15] and to measure the effectiveness of clinical procedures [16]. However, HICs have not previously been utilized for evaluating the surveillance system. Here, we have used these data for the first time to evaluate underreporting in the measles surveillance system and to quantify the proportion of measles patients who undergo laboratory tests in order to confirm their measles diagnosis.

METHODS

Data sources

We compared the number of measles cases in Aichi Prefecture, as reported by three different sources: the sentinel surveillance system established by the Infectious Disease Prevention Law, the mandatory notification system for measles run by the Aichi Prefectural government, and HICs filed by employees and dependants insured by corporate health insurance societies.

In Aichi Prefecture, the mandatory notification system for measles was introduced in February 2007, with all local medical institutions requested to provide the following information about every measles case immediately after confirmation of the clinical diagnosis [13]: name of the municipality where the medical facility was located and where the patient lived, date of diagnosis, sex and age of the patient, results of the diagnostic test for measles, history of measles vaccination, frequency of consultation, and, where relevant, type of school (university, high school, junior high school, elementary school, kindergarten, nursery school). Aichi Prefecture defines measles cases using the same criteria as those prescribed by the Infectious Disease Prevention Law. The diagnostic tests used for the confirmation of measles were detection of IgM antibodies in blood or isolation and genotyping of measles virus from any sample. Again, although physicians were asked to provide the results of laboratory data when reporting cases to the local health authorities, the conducting and reporting of laboratory tests are not mandatory.

HICs were obtained from several corporate health insurance societies in Aichi Prefecture. From January to December 2007, we electronically searched for all HICs containing the International Statistical Classification of Diseases and Related Health Problems – 10th Revision (ICD-10) code A71 (measles). Since Japan's health insurance system is based on fee-for-service reimbursement, reimbursement rules dictate that each clinical procedure must be justified by a corresponding diagnosis. Thus, HICs include ‘rule-out diagnoses’, which is a term for conditions that are either measles or measles-like (e.g. rubella, scarlet fever, roseola, dengue fever, drug reactions), that are unconfirmed or disproved by the results of diagnostic procedures. HICs with rule-out diagnoses for measles were also included in the electronic search. Unfortunately, even if the HICs indicate that laboratory tests were performed, they do not contain information on test results. Therefore, the available data did not allow us to distinguish between rule-out diagnoses and other diagnoses; in other words, the definition of measles in HICs is equal to syndromic surveillance for measles-like illnesses in a managed care setting [17].

The included HICs were checked for dates of first consultation, medical expenditure, and diagnostic tests for measles. The diagnostic tests included for study were blood tests for IgM antibodies and isolation and genotyping of the measles virus. In cases where a measles patient was treated at multiple medical facilities, the insurers collated the data into a single file, which prevented any repeated sampling of the same individuals. The consultation rate was defined as the number of cases found in HIC records divided by the total number of insured persons. The MediC4 encoding system® (Japan Medical Data Center Co. Ltd, Japan) was used to remove all personal information from the HICs before they were delivered to researchers by the corporate health insurance societies.

Statistical analyses

To compare data from the three different sources, we made three comparisons. First, we looked at the number of measles cases reported per month by each of the sources; for this analysis, we examined children and adults separately. In the mandatory notification system [13], if a patient with suspected measles consulted a medical facility in January 2007 and measles diagnosis was confirmed and notified after February 2007, we classified the onset of the patient as January 2007.

Second, we compared incidence rates (determined from the mandatory notification dataset) and consultation rates (determined from the HIC dataset) across the total population and across each of nine different age groups.

The incidence rate was defined as the number of reported measles cases divided by the total population of Aichi (7 316 155 individuals as of 1 October 2007, including 1 075 344 individuals aged <15 years, or 14·7% of the population, and 6 240 811 individuals aged ⩾15 years, or 85·3% of the population). The consultation rate was defined as the number of reported measles cases divided by the number of insured person by the corporate health insurance societies. In October 2007, 276 541 (3·8%) residents of Aichi Prefecture were insured by corporate health insurance societies, including 66 579 individuals aged <15 years (24·1% of the insured population and 6·2% of the total population) and 209 962 individuals aged ⩾15 years (75·9% of the insured population and 3·4% of the total population).

Finally, we examined the proportion of measles patients in each dataset (mandatory notification vs. HIC) who received diagnostic testing to confirm the presence of measles. Differences between these two datasets were evaluated with prevalence ratios. All analyses were performed using the Statistical Analysis System, version 9.1 (SAS Institute, USA).

RESULTS

According to all three sources, the number of paediatric measles cases peaked mid-year (Fig. 1). Data from the Aichi mandatory notification system showed that the number of reported measles patients was highest in June, accounting for about 30% of the year's patients. While both surveillance systems showed a peak in June, numbers from the mandatory notification system were more than double those from the sentinel system. Data from the HICs indicated a slightly earlier peak, with the highest number of reported measles patients occurring in May.

Fig. 1. Number of reported paediatric measles cases (patients aged <15 years) in 2007 in Aichi, Japan, as reported by three different data sources. (Health insurance claims data were obtained from 6·2% of residents aged <15 years in Aichi Prefecture.)

There was more variation in adult cases documented by the three different datasets (Fig. 2). For instance, both the HICs and mandatory notification data showed mid-year peaks similar to those observed in paediatric cases; judging from these reports, the epidemic began in February and ended in September, with more than 70% of adult measles cases reported in May or June. Among adults, there were greater differences between the number of cases reported each month by the three datasets. According to the sentinel surveillance data, on the other hand, the highest number of adult cases occurred in May and September. However, four was the maximum in any month in the sentinel surveillance dataset, which contrasts sharply with the much higher values observed in the other two reporting systems. Indeed, the relatively low number of reported cases in the sentinel surveillance dataset makes it difficult to estimate the epidemic pattern of adult measles.

Fig. 2. Number of reported adult measles cases (patients aged ⩾15 years) in 2007 in Aichi, Japan, as reported by three different data sources. (Health insurance claims data were obtained from 3·4% of residents aged ⩾15 years in Aichi Prefecture.)

Table 1 shows the incidence of measles diagnosis (obtained from mandatory notification system data) and consultation rates (obtained from HIC data) across the entire population and within each of nine age groups. Fifty-six cases were reported to the sentinel surveillance system, the majority of whom were aged <15 years (45 individuals, 80·4% of the total). Nearly four times as many cases (n=212) were reported to the mandatory notification system, approximately half of which were patients aged <15 years (89 individuals, 42% of the total). Total numbers in the HIC dataset were similar to those from the mandatory notification system (n=189 cases), although the age distribution differed substantially; only a quarter were patients aged <15 years (45 individuals, 24% of the total), while the remainder were adults (144 individuals, 76% of the total). The two types of dataset did not consistently identify a single age group as being most susceptible to measles. In the mandatory notification dataset, the highest number of cases was found in the youngest age group (⩽4 years; 18% of cases); the HIC dataset identified a slightly older age group (15–19 years) as experiencing the highest number (22%) of cases.

Table 1. Age-specific measles incidence rate (mandatory notification dataset) and consultation rate (health insurance claim dataset) for patients in Aichi, Japan, 2007

Incidence rates (A) were described as frequency per 100 000 people in the population, while consultation rates (B) were described as frequency per 100 000 people insured by the corporate health insurance societies.

Overall, there were significant differences between the mandatory notification and HIC datasets. The mandatory notification data indicated that the measles incidence rate was 2·9 cases/100 000 individuals in the population, which is 23·6 times lower than the consultation rate (68·3 cases/100 000 insured individuals) calculated from the HIC dataset. According to the mandatory data, the highest age-specific incidence rate, recorded in individuals aged ⩽4 years, was 11·1/100 000 people in the population; the second highest rate, 10·4/100 000 people, occurred in those aged 10–14 years. This differs markedly from HIC data, which indicated that the highest consultation rates were in insured individuals aged 15–19 years (202·8/100 000 insured people) and 20–24 years (130·9/100 000 people). According to the mandatory notification dataset, the incidence of measles declined with age in individuals aged ⩾15 years, falling to just 0·48/100 000 people for individuals aged ⩾40 years. A similar trend was found in HIC data, which showed that consultation rates declined with age in groups aged ⩾20 years. The datasets were most similar for patients aged 10–14 years, as shown by the lowest value (4·6) for the ratio between consultation and incidence rates. On the other hand, the least amount of agreement between the two datasets occurred for adults aged ⩾40 years (ratio 74·9). This reflects the general trend found across all data. The ratios between consultation and incidence rates were smaller in paediatric patients than for adults, and tended to increase with age.

Table 2 shows the prevalence of diagnostic tests for confirming measles diagnoses. Measles patients identified by the HIC database were significantly more likely to have received a laboratory examination than patients reported to the mandatory notification system, regardless of whether they were children [prevalence ratio (PR) 4·2, 95% confidence interval (CI) 2·5–7·2] or adults (PR 11·0, 95% CI 6·1–20·0).

Table 2. Prevalence of diagnostic tests for confirmation of measles diagnoses of patients in Aichi, Japan, 2007

CI, Confidence interval.

DISCUSSION

A comparison of the number of measles cases reported by three different sources revealed two major findings. First, the sentinel surveillance system appears to underestimate the true number of measles cases in adults. Second, diagnostic tests were more common in cases obtained from HICs than in those reported to the surveillance system.

The sentinel surveillance system probably underestimated the true number of measles cases because facilities without in-patient care were excluded for adult measles cases until December 2007. Thus, even though sentinel surveillance data are useful for evaluating trends and estimating the impact of the vaccination programme, the system is less likely to detect relatively rare diseases such as adult measles. However, the accuracy of diagnosis and the reporting rate of sentinel events are generally better than those from passive surveillance systems. For instance, data from the mandatory notification systems in Saitama Prefecture [5] and Aichi Prefecture [13] led to the identification of measles outbreaks in 2007. Since 2008, all medical facilities have been required to report measles cases; this may facilitate more precise detection of measles outbreaks, but efforts should continue to minimize underreporting of measles cases to surveillance systems.

The difference in incidence rate between mandatory notification (2·9/100 000 people in the population) and consultation rate (23·6/100 000 insured individuals) is probably explained by the different definitions used in the mandatory notification system and HIC datasets to identify measles. The sentinel and mandatory notification datasets include only measles cases, while the HIC dataset includes patients whose conditions are either measles or measles-like (e.g. rubella, scarlet fever, roseola, dengue fever, drug reactions). This means that the HIC dataset has greater detection sensitivity than the mandatory notification system, although its specificity is lower. Furthermore, this tendency is stronger in adults than children. Our HIC data are consistent with syndromic surveillance for measles-like illnesses in a managed care setting [17], and we were not surprised to find that the consultation rate calculated from HICs exceeds the incidence estimated from the mandatory notification data.

In many countries, the completeness of infectious disease surveillance is influenced by physician reporting [7]. For instance, the incidence of measles may be underestimated if physicians fail to report cases or if patients seek medical care at facilities not included in the sentinel surveillance system. An active surveillance of medical institutions enables an assessment of the quality of passive surveillance and sentinel surveillance systems [13,18]. However, even with active surveillance some physicians fail to complete notifications, leading to an underestimate of the actual number of cases. HICs enable us to obtain the complete number of measles patients by evaluating how many individuals seek medical care, because HICs were designed for processing the reimbursement of healthcare services rendered by the medical facilities in a given calendar month. Our results indicate that the trends reported by HICs are similar to those detected by mandatory notification techniques, and may actually be more effective than sentinel surveillance at detecting epidemics of adult measles.

Another advantage of HICs is that they reflect activity at all medical care facilities across the country, regardless of size or location; this differs from the other surveillance systems, which only gather information from certain medical facilities. In Japan, for example, patients can seek medical care outside the prefecture in which they reside, since Japan's health insurance system guarantees free access. Thus, the Aichi mandatory notification data may not accurately reflect the number of measles patients in Aichi Prefecture if many patients seek medical care elsewhere. Additionally, because insurers can identify whether patients are treated at multiple medical facilities for the same disease, the use of HICs eliminates inadvertent data duplications. This is not possible with passive surveillance unless a personal identifier is assigned to each patient. One final benefit of the HIC database is that it is less costly than performing active data collection from medical facilities, a process that is required for active surveillance.

The government mandates collection of HIC data for statistics using HIC data in Japan, such as those in the Social Insurance Claims Survey or National Health Insurance Medical Benefit Surveys. However, their results are based only on HICs for services provided in May. Further, information about multiple diagnoses on HICs has been ignored because of technical limitations that prevent the assessment of all the information included on a HIC. Further study analysing all the information included on HICs for a full year is necessary for the surveillance potential of this data collection.

We were only able to evaluate data from Aichi Prefecture in this study, although outbreaks of measles were also reported in other Japanese prefectures in 2007 [5]. In the future, additional work should be done to evaluate the quality of the infectious disease surveillance systems in each individual prefecture. Another limitation of the current study is that HICs are not survey forms designed to test a specific hypothesis. Therefore, a diagnostic criterion is not strictly defined. The first step in the detection of a measles case is for the patient to seek medical care [19]. Detecting a cluster of measles cases precedes an outbreak investigation with strict case definition [20]. In a setting in which true measles cases occur very rarely, the positive predictive value of a clinical diagnosis of measles is low [21,22]. With careful consideration regarding the objective of surveillance and information obtained from HICs, the HIC database can be used to evaluate the surveillance system. Finally, it is important to note that data from the HICs analysed in this study were not necessarily representative of the total population of Aichi Prefecture. For instance, the corporate health insurance societies that permitted use of the data from HICs insured only 3·8% of the total population of Aichi Prefecture. Additionally, the proportion of age groups in the HIC dataset did not always mirror the total proportion found across the prefecture (e.g. patients aged<14 years). However, because reimbursement for measles care is guaranteed, it is unlikely that patients would seek less, or different, care just because they were covered by other insurance societies. However, further work is necessary to verify this assumption and determine whether the current results are broadly applicable.

In conclusion, we advocate the use of HIC data for supplementing measles surveillance data collected from physicians – but only as long as researchers and public health practitioners are aware of the strengths and weaknesses of this alternative data source. Continued efforts to reinforce the clinical recognition and reporting of measles cases are warranted.