Evidence-Based Policy Making during the Coronavirus Disease 2019 Pandemic: A Systematic Review

Abstract

Introduction:

The aim of this systematic review was to collect evidence and recommendations for the applicability of the concept of evidence-based policy making (EBPM) during the coronavirus disease 2019 (COVID-19) pandemic and to discuss the implementation of this concept from a medical science perspective.

Methods:

This study was performed according to the guidelines, checklist, and flow diagram of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020. An electronic literature search was conducted on September 20, 2022 using PubMed, Web of Science, Cochrane Library, and CINAHL databases with the following search terms: “evidence based policy making” and “infectious disease.” Study eligibility assessment was performed based on the flow diagram of PRISMA 2020, and risk of bias assessment was performed using The Critical Appraisal Skills Program.

Results:

Eleven eligible articles were included in this review and divided into three groups as follows: early, middle, and late stages of the COVID-19 pandemic. Basics of COVID-19 control were suggested in the early stage. The articles published in the middle stage discussed the importance of the collection and analysis of evidence of COVID-19 from around the world for the establishment of EBPM in the COVID-19 pandemic. The articles published in the late stage discussed the collection of large amounts of high-quality data and the development of methods to analyze them, as well as emerging issues related to the COVID-19 pandemic.

Conclusions:

This study revealed that the concept of EBPM applicable to emerging infectious disease pandemics changed between the early, middle, and late stages of the pandemic. The concept of EBPM will play an important role in medicine in the future.

Introduction

The first cases of coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus-2, were reported from Wuhan, China in December 2019. ¹ Since then, COVID-19 has rapidly spread to other countries across the world, and more than six million people have died as a result. ^1,2 Despite advances in the development and administration of drugs and vaccines, it remains a life-threatening infectious disease. ^2,3

More than two years have passed since the World Health Organization (WHO; Geneva, Switzerland) declared a public health emergency on January 30, 2020. ¹ The causative virus mutated repeatedly, and COVID-19 still affects public health and socioeconomics today. ⁴ In the past two years, there has been much discussion about human life, public health, and socioeconomic aspects, but it is debatable whether appropriate decisions have been made regarding the most prolonged global infectious disease ever experienced by humankind in recent years.

In recent years, the term “evidence-based policy making” (EBPM) has received increasing attention; ^5,6 EBPM is established through systematic data collection on issues or topics, analysis of those collected data, and creation of evidence on which to base policy decisions. ^5–7 In the medical field, “evidence-based medicine” (EBM) is widely accepted, and guidelines have been created for each disease to standardize the medical care provided. ⁸ On the contrary, there may not have been sufficient discussion of the steps involved in creating a policy for the medical region. Policies related to infectious diseases should be developed based on appropriate and high-quality evidence as they affect public health. ^5–7

On September 15, 2022, WHO announced that the end of the COVID-19 pandemic “is in sight.” ⁹ In accordance with this announcement, the authors considered that a verification of the policies and responses regarding COVID-19 by researchers and stakeholders should be initiated.

The aim of this systematic review was to collect evidence and recommendations for the applicability of the EBPM concept during the COVID-19 pandemic and to discuss the implementation of EBPM from a medical science perspective during the COVID-19 pandemic declared by WHO (ie, from January 30, 2020 through September 15, 2022).

Methods

This systematic review was performed according to the guidelines, checklist, and flow diagram of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 (Figure 1; Supplementary Material [available online only]). ^10–12

Figure 1. Flow Diagram of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020.

Inclusion and Exclusion Criteria

The inclusion criteria were as follows: (1) literature associated with EBPM for infectious diseases; (2) literature on COVID-19; (3) literature published from January 30, 2020 through September 15, 2022; and (4) literature published in English.

The exclusion criteria were as follows: (1) case reports, case series, reviews, and conference papers or proceedings; (2) animal experiment trials; (3) unavailability of full text; and (4) articles in languages other than English.

Information Sources and Literature Search Strategy

An electronic systematic literature search was conducted using PubMed (National Center for Biotechnology Information, National Institutes of Health; Bethesda, Maryland USA); Web of Science (Clarivate Analytics; London, United Kingdom); Cochrane Library (Wiley; Hoboken, New Jersey USA); and CINAHL (EBSCO Information Services; Ipswich, Massachusetts USA) databases (Table 1). The electronic searches were performed on September 20, 2022. The search strategy for electronic literature is shown in Table 1.

Table 1. Electronic Literature Search Strategy

Study Selection

The first author performed the electronic literature search using the search strategy described above, and all authors evaluated the studies. Disagreements between reviewers were resolved through discussion and consensus. The authors selected the literature to include in this study based on the PRISMA flow chart; this comprised the removal of duplicates, screening of titles and abstracts, and reviewing of full texts. ^10,12

Risk of Bias Assessment

Considering the problems of quality and the risk of bias, the authors evaluated studies based on The Critical Appraisal Skills Program checklist for qualitative research that consists of ten questions. ¹³

Results

Study Selection

An electronic systematic literature search was conducted using the PubMed, Web of Science, Cochrane Library, and CINAHL databases, and a total of 68 articles were identified (Figure 1). There were no duplicate articles; 25 review articles were excluded. A further 32 articles were excluded on the basis of their title and abstract. Eleven (11) articles were assessed for eligibility, and each article was assessed for risk of bias. ^14–24 Finally, 11 eligible articles were included in this systematic review. ^14–24 In order to discuss advances in policy making and EBPM in the early, middle, and late stages of the COVID-19 pandemic, the authors analyzed these articles separately according to publication year as follows: 2020, 2021, and 2022 (Table 2).

Table 2. Details of Included Studies in this Review

Early Stage of the COVID-19 Pandemic (Published in 2020)

In this stage, basic reproduction number and the time-varying estimate of the effective reproductive number of COVID-19 were reported, and there were differences in these trends between European–North American and Asian countries; furthermore, the behavior of the COVID-19 pandemic was predicted using the model consisted by the items as follows: susceptible, exposed, infectious, and removed. ¹⁴ Until the development of an effective vaccine, non-pharmaceutical interventions such as social distancing, frequent hand washing, personal protective equipment, activity cessation, avoiding mass gatherings, and closing facilities were reported to be considered and implemented to avoid a lockdown policy. ¹⁵ At the same time, a warning was issued against information confusion—including information overload, information uncertainty, and misinformation—with poor evidence regarding the new infectious disease. ¹⁶ The evidence for the allocation of medical facilities for infectious disease control was discussed. ¹⁷ These were reported at a stage when evidence for COVID-19 was poor, and it was difficult to apply concepts of EBPM in the COVID-19 pandemic.

Middle Stage of the COVID-19 Pandemic (Published in 2021)

It was reported that evidence based on timely and detailed transmission risk assessment of COVID-19 could be applied world-wide, and that a global epidemic intelligence network and information sharing should be strengthened. ^18–20 High-quality information provided by these global sharing networks had the potential to allow the application of the EBPM concept to the public health aspect of the COVID-19 pandemic, and to respond effectively to future infectious disease threats. ^18–21 The articles published in 2021 discussed that the collection and analysis of evidence regarding COVID-19 from around the world would be important to establish evidence-based policies against the COVID-19 pandemic.

Late Stage of the COVID-19 Pandemic (Published in 2022)

A data-driven model of infectious diseases, such as COVID-19, spreading in spatiotemporal networks informed by mobility data of individuals, that could be useful for controlling the spread of an infectious disease, was reported. ²² The accuracy of the widely used and cheap COVID-19 tests (ie, lateral flow tests) was validated. ²³ Magnified gender-based disparities due to the COVID-19 pandemic were also discussed. ²⁴ The articles published in 2022 discussed the collection of large amounts of high-quality data and the development of methods to analyze them, as well as emerging issues related to the COVID-19 pandemic.

Risk of Bias Assessment

The authors assessed the risk of bias using the Critical Appraisal Skills Program checklist for qualitative research for the 11 included articles and determined that there was no significant risk of bias that would affect the results. ¹³

Discussion

This systematic review revealed that the concept of EBPM applicable to emerging infectious disease pandemics, such as COVID-19, had changed between the early, middle, and late stages of the pandemic. In this study, it can be concluded that EBPM associated with the COVID-19 pandemic was not feasible in the early stage of the pandemic because there was insufficient evidence to support it. When the scientific evidence is incomplete, and when interventions whose need is inferred have not yet been implemented (ie, early stage), it may be useful to attempt EBPM based on lessons learned in other regions and in other infectious diseases. ²⁵ Ultimately, COVID-19 falls into this category and will serve as an example for future countermeasures against emerging infectious diseases. ²⁵ When the extent of infectious diseases is wide-spread and cannot be controlled locally (ie, middle and late stages of infectious disease pandemics), it is important to collect and analyze data obtained at various locations. This may involve extensive data that eliminates local factors such as race and socioeconomics to build high-quality evidence and support the practice of EBPM.

This is the third year that humanity has experienced the COVID-19 pandemic. Although the basics of its control were reported and suggested in the early stages, it has had a significant impact on health services and the social economy world-wide for too long. ^14,15,26 The authors speculate that this is due to the unexpected multiple mutations of the coronavirus, the variability in the degree of clinical symptoms, and the route of transmission—short-range aerosol and airborne transmission or droplet transmission. ^27,28 This is due in part to the fact that the world is suddenly faced with the difficult and seemingly contradictory task of maintaining socioeconomic activity and controlling an infectious disease pandemic that governments have not had to perform concurrently in the recent past.

It is surprising that no papers were extracted discussing EBPM associated with COVID-19 vaccines and lockdown policy at this time. Clinical guidelines and EBM rely heavily on meta-analyses based on randomized controlled trials (RCTs). ^29,30 However, the lack of sufficient verification of the safety and side effects of the COVID-19 vaccines suggests that these should be given only to those who wish to receive them, and RCTs of the COVID-19 vaccines may violate medical ethics. ³¹ In addition, with regard to lockdown policies, it is probably not possible to make simple comparisons because the political, social, and economic conditions in each country play important roles. ^32,33 It has been reported that lockdown policies had a significant impact on socioeconomic activity and the maintenance of physical and mental health. ^34,35 The following issues should be examined in the future: what kind of evidence was used to select vaccines and apply lockdown policy and whether there was a difference in the resulting social impact in countries that promoted vaccination and applied lockdown as a public policy and those that did not. This issue needs to be examined in developed/developing countries, and high-income/low-income countries, and the equality of distribution of health services during the COVID-19 pandemic should also be discussed. The lessons drawn from this discussion will contribute to the achievement of the third goal of the Sustainable Development Goals 17 proposed by the United Nations. ³⁶

Limitations

As a limitation of this study, first, the authors declare that they have identified articles using search terms such as “evidence based policy making” and “infectious disease” and included articles published from January 30, 2020 through September 15, 2022, taking into account the WHO statement. ^1,9 That is, this paper was written when the end of the COVID-19 epidemic had not been declared world-wide and further spread may still occur. Second, this review focuses on the medical science aspect and does not adequately discuss the socioeconomic aspects. As such, COVID-19 should be discussed when it is recognized as a common disease world-wide as the global economy is always in flux. It should be discussed by bringing together various experts and stakeholders with high-quality knowledge in medical science, economics, politics, and other fields, and should serve as an example for future pandemics of emerging infectious diseases for which there is a lack of evidence. ³⁷

The concept of EBPM will play an even more important role in medicine in the future. The evidence and recommendations in the early, middle, and late stages of the COVID-19 pandemic validated in this paper may have provided important information for the implementation of EBPM in the event of a sudden pandemic of emerging infectious diseases in the future.

Conclusions

This study revealed that the concept of EBPM applicable to emerging infectious disease pandemics changed between the early, middle, and late stages of the pandemic. It can be concluded that EBPM associated with the COVID-19 pandemic was not feasible because there was insufficient evidence to support this in the early stages.