A machine learning algorithm to analyse the effects of vaccination on COVID-19 mortality

Abstract

The coronavirus disease 2019 (COVID-19), with new variants, continues to be a constant pandemic threat that is generating socio-economic and health issues in manifold countries. The principal goal of this study is to develop a machine learning experiment to assess the effects of vaccination on the fatality rate of the COVID-19 pandemic. Data from 192 countries are analysed to explain the phenomena under study. This new algorithm selected two targets: the number of deaths and the fatality rate. Results suggest that, based on the respective vaccination plan, the turnout in the participation in the vaccination campaign, and the doses administered, countries under study suddenly have a reduction in the fatality rate of COVID-19 precisely at the point where the cut effect is generated in the neural network. This result is significant for the international scientific community. It would demonstrate the effective impact of the vaccination campaign on the fatality rate of COVID-19, whatever the country considered. In fact, once the vaccination has started (for vaccines that require a booster, we refer to at least the first dose), the antibody response of people seems to prevent the probability of death related to COVID-19. In short, at a certain point, the fatality rate collapses with increasing doses administered. All these results here can help decisions of policymakers to prepare optimal strategies, based on effective vaccination plans, to lessen the negative effects of the COVID-19 pandemic crisis in socioeconomic and health systems.

Introduction

The coronavirus disease 2019 (COVID-19) is an infectious illness caused by the novel severe acute respiratory syndrome coronavirus 2, which appeared in late 2019 [1–4]. COVID-19 is still circulating in 2022 with mutations of the novel coronavirus that generate new variants of concern¹ driving continuous infections and deaths in manifold countries [5–7]. Seligman et al. [8] show some characteristics of people that are significantly associated with COVID-19 mortality, such as mean age of 71.6 years, non-white race/ethnicity, income below the median, and less than a high school level of education. High numbers of COVID-19-related infected individuals and deaths worldwide have supported the development of different types of vaccines from 2020 based on viral vector, protein subunit and nucleic acid [9–12]. In the presence of the COVID-19 pandemic crisis, the investigation of vaccination plans is a crucial aspect to determine how the novel infectious disease can be controlled and/or eradicated in the population [13]. Vaccination has the potential effect to reduce the diffusion of COVID-19, relaxing non-pharmaceutical measures, and maintain low basic reproduction number; nevertheless, an important point to clarify is the optimal strategy of administering the vaccines during the pandemic evolution wave, to reduce negative effects in society [14]. Akamatsu et al. [15] argue that the vital role of governments is directed to implement an efficient campaign of vaccination to substantially reduce infections and mortality in society and avoid the collapse of the healthcare system. Aldila et al. [13] maintain that higher levels of vaccination rate can eradicate COVID-19 in the population by approaching herd immunity to protect vulnerable individuals [16–19]. Rosen et al. [20] describe socio-economic and organisational factors associated with the success of the vaccination campaign in Israel as well as they show some aspects of misinformation that can reduce the effectiveness of a fruitful vaccination plan over time [21]. In this context, a vital problem in the current COVID-19 pandemic crisis is the effective level of vaccination that supports a drastic reduction of infected individuals and deaths.

The present study confronts this problem by developing a machine learning (ML) algorithm to empirically assess the effects of vaccination on deaths and the fatality rate of the COVID-19 pandemic. The application of ML approaches can provide a new perspective to support appropriate policy responses of policymakers directed contrast the outbreaks of new variants of COVID-19 and similar infectious diseases in the future [22–29].

The goal of this paper is to show the role of ML experiments as one of the significant methods in the research arena for predicting and assessing the effects of vaccination on health and improving crisis management of the COVID-19 pandemic. The results can suggest best practices of optimisation in the vaccination strategy to guide effective and timely policy responses for combatting the novel virus and constraining the negative effects of the COVID-19 pandemic crisis and future epidemics of similar infectious diseases in society. The findings of this study could be also of benefit to countries as they grapple to plan their vaccine plans for the COVID-19 pandemic crisis to minimise the negative effects of the pandemic crisis on the environment and socio-economic systems. This study is part of a large body of research projects directed to explain the drivers of transmission dynamics of COVID-19 and design effective policy responses to cope with and/or prevent pandemic threats [30–36].

The rest of the paper is organised as follows: Section ‘Literature review on artificial intelligence (AI) studies on COVID-19 vaccines effects’ presents the literature on studies that employed AI techniques to explore the effects of COVID-19 vaccines. In Section ‘Materials and methods’ we describe the methodologies applied and the dataset. Section ‘Empirical results’ presents the empirical findings. Finally, Section ‘Conclusions and policy implications’ concludes and gives the limitations of the study together with some policy recommendations.

Literature review on artificial intelligence studies on COVID-19 vaccines effects

Several studies analysed the effects of vaccination campaigns on COVID-19 diffusion through AI approaches.

Several papers have been devoted in the more recent years to provide surveys on literature reviews or systematic reviews regarding the COVID-19 pandemic. Abd-Alrazaq et al. [37] elaborated an extensive bibliometric analysis to offer a comprehensive overview of the literature on COVID-19. Lv et al. [38] showed an extensive survey on the application of AI and ML to defeat the COVID-19 pandemic. Wang et al. [39] performed a systematic review of the application of AI techniques related to COVID-19. The results showed that AI obtained high performance in diagnosis, prognosis evaluation, epidemic prediction and drug discovery for the virus. De Felice and Polimeni [40] conducted a bibliometric analysis using an ML bibliometric methodology. Zyoud and Al-Jabi [41] run a bibliometric analysis to get a plausible scenario of the COVID-19 pandemic crisis.

Lincoln et al. [42] investigated the willingness to get a vaccine over a sample of five advanced countries using an ML algorithm. Vaccination conspiracy belief was found to represent the most relevant predictor.

Many AI applications analysed the public sentiments towards vaccines using social media data. Xue et al. [43] analysed 4 million Twitter messages regarding the COVID-19 pandemic. The Latent Dirichlet Allocation (LDA) findings revealed the anticipation that measures can be taken as the dominant sentiment for the virus spread. Liew and Lee [44] used social media data to understand public sentiments about COVID-19 vaccines through an unsupervised ML approach (structural topic modelling). They found that tweets with negative feelings were about emotional reactions. Lyu et al. [45] identified the main topics present in the public discussion of the COVID-19 vaccines on social media by applying an LDA for topic modelling. The emotion analysis showed the main emotion was trust, followed by anticipation, fear and sadness. Kwok et al. [46] built an LDA topic model to reveal topics and sentiments on COVID-19 vaccination on Twitter in Australia. Three topics emerge: attitudes toward COVID-19 and its vaccination; advocating infection control measures against COVID-19; misconceptions and complaints about COVID-19 control. Lian et al. [47] developed an ML and Natural Language Processing (NLP) approach to discover COVID-19 vaccine adverse events using data from Twitter. The results show that the four most populous states in the US (California, Texas, Florida and New York) recorded the most discussions on Twitter, highlighting a strong correlation between Twitter discussions and vaccination campaigns.

A different strand of literature used AI approaches to inspect the effect of contact tracing apps. Cresswell et al. [48] explored public perceptions of COVID-19 contact tracing apps in the UK using a deep learning (DL) approach, revealing 76% positive sentiments. Hussain et al. [49] analysed public sentiments on social media towards COVID-19 vaccines in the UK and the US by applying NLP and DL-based techniques. The results show a majority of the overall averaged positive sentiments in both countries. Weiß et al. [50] developed an item set to monitor nationally issued COVID-19 contact tracing apps using an Open-Source Intelligence approach. Empirical findings highlighted differences among the countries in the sample.

Bagabir et al. [51] tried to highlight the advantages of AI applications to identify the genomic sequences as well as the development of drugs and vaccines for COVID-19. Monteleone et al. [52] investigated the impact of AI on drug repurposing of therapies for the treatment of COVID-19.

Materials and methods

Source and sample

The sample of this study is based on N = 192 countries worldwide. The period under study is from March to May 2021, using data on vaccines, confirmed cases, and the fatality rate of COVID-19. The list of countries under study is in the Appendix.

Measures

1. – Doses of vaccines administered × 100 inhabitants on 15 March 2021 with N = 114 countries; on 14 April 2021 with N = 154 countries and on 26 April 2021 with N = 190 countries. The number of samples tends to increase over time with the diffusion of vaccines across countries worldwide. Doses of vaccines refer to the total number of vaccine doses, considering that an additional dose may be obtained from each vial (e.g. six doses for Pfizer BioNTech^® Comirnaty), whereas the number of doses administered refers to any individual receiving any dose of the vaccine [53, 54]. The data here considers all types of COVID-19 vaccines used in different countries, i.e. vaccines by Johnson & Johnson, Oxford/AstraZeneca, Pfizer/BioNTech, Sinopharm/Beijing, Sinovac, Sputnik V and Moderna [55]. Of course, every country has been using a different combination of these COVID-19 vaccines to protect the population [56–58]. Source of data: Our World in Data [59].

2. – Number of COVID-19 infected individuals (%) is measured with confirmed cases of COVID-19 divided by the population of countries under study on 20 March 2021 (N = 192 countries), 25 April 2021 (N = 192) and 19 May 2021 (N = 216 countries). Source of data: Johns Hopkins Center for System Science and Engineering [5].

3. – Number of COVID-19 deaths is measured with case fatality rate (%) given by deaths on 25 April 2021 divided by the total infected individuals in each country. Source of data: Johns Hopkins Center for System Science and Engineering [5].

Data analysis procedure

Once we have collected the data and postulated it according to a rigid regime consistent with the ML hypotheses framework, we search for the best algorithm to optimise the dataset. The data computerisation process aims to train the machine to process results without any further intervention by the operator. In other words, the initial computer programming instructions are only necessary for it to carry out an autonomous but at the same time specific behaviour in the processing of data and results. Among the numerous approaches that an ML process can use, we have chosen to use the Artificial Neural Networks (ANNs). The empirical analyses are conducted in Oryx, and for the analysis of the neural network (NN) AD-Designer is used. The choice to use an NN process coincides with the assumption of discrimination. In general, we can say that a discriminant function is a function that receives the input data as input and whose output represents the classification of such data. Furthermore, the discriminant function can be generalised by transforming the result through a non-linear function called the activation function.

So, our experiment began with the activation function. It is responsible for updating the status of the i-th neuron as it transitions from one moment to the next:

(1)$$s_i( t ) \to s_i( {t + 1} ) $$

Thus, the stimulation input of the neuron i of the σ-layer with respect to our n-layer feed-forward network was given by the potential $P_i^\sigma$:

(2)$$P_i^\sigma = \mathop \sum \limits_j^i [ {w_{ij}s_j^{\sigma -1} ( t ) -\theta_i} ] , \;\quad {\rm with}\;\sigma = 2, \;\ldots , \;n$$

The threshold θ _i has been dropped, while an emulated input s_k = 1 is added, to which we have assigned the relative connection weight:

(3)$$\eqalign{ w_{ik} = {-}\theta _i\to P_i^\sigma = \mathop \sum \limits_j^i [ {w_{ij}s_j^{\sigma -1} ( t ) -} ] , \;\;\quad \;{\rm with}\;\sigma = 2, \;\ldots , \;n}$$

The subsequent state of the neuron i, s_i(t + 1), is computed by an appropriate function of the potential P_i, called the activation or transfer function:

(4)$$s_i^\sigma = f( {P_i^\sigma } ) $$

The function f can take various forms, but the one used here is the hyperbolic function of the type: sine, cosine and secant.

Then, the training phase is activated. In the training phase, the NN starts from an initial state characterised by the assignment of arbitrary values for the synaptic weights. It dynamically evolves towards a final equilibrium state corresponding to the learning of the problem in question. In the context of the connectionist paradigm, learning assumes a fundamental importance: it is generally not possible to fix in advance the weights of the connections between neurons, depending on the task that the NN has to perform. These weights must be learned and the NN must behave like an adaptive system. The learning phase is very long. It requires the presence of a set of input vector values with known output (supervised training). This dataset forms the set of examples (training set) that the NN must learn and emulate. The adaptation of the weights of the links was done by using iterative procedures; here retroactive procedures are chosen. They consist in comparing the values computed by the network with those to be emulated. Then, the weights of the connections are modified to minimise the difference in this comparison. This procedure is repeated for each example, i.e. for each vector of the training set. For the NN to reach its final structure, a single training cycle is not sufficient: many iterations are needed for this experiment to reach a satisfactory approximation. Our algorithm for the backpropagation of faults finds a place in numerous applications due to its generalisation ability. However, it has the limitation that it is very slow in the learning process. It requires many cycles before it reaches a sufficiently small global error. However, we have accelerated the algorithm with the following modification of the weight update law:

(5)$$w_{ij}( {t + 1} ) = w_{ij}( t ) -\Delta w_{ij} + \beta ( {w_{ij}( t ) -w_{ij}( {t-1} ) } ) $$

The error function obtained is considered as a function of the energy of the system. The state of minimum energy corresponds to the equilibrium state of the same. The error function, when the transfer (or activation) function is linear, is represented by a hyper paraboloidal surface with an absolute minimum. Learning corresponds to reaching the configuration with minimum energy and is guaranteed since the hyper paraboloid has only an absolute minimum. When the activation function is non-linear but monotonic, like the sigmoid function, it is represented by a deformed hyper paraboloid characterised by the presence of relative minima. Thus, once the best estimation model is defined and its activation function is specified, we can develop a prediction process capable of estimating the number of deaths and the mortality rate with respect to a set of inputs attributable to seven variables: Total Cases, Cases/Population, Vaccine Dose (April), Vaccine Dose per 100 Population (April), Vaccine Dose (15–21 March), Vaccine Dose per 100 Population (15–21 March), Total Vaccination (15–21 March). Since the algorithm needs to read a large dataset with numerous observations, we extended the dataset using various mathematical transformations. In particular, for each variable, we created its logarithm and the first difference transforms. We then trained the machine to generate an NN based on a combination of 6 nodes distributed according to the following scheme: 12-10-7-5-7 and 2 output layers. Therefore, the neural series analysis process is defined through the following scenario: Scaling Layer methods: Mean Standard Deviation; Unscaling layer: Minimum-Maximum; Bounding layer: no apply bounding layer; Maximum iterations: 100 000; Maximum time (hh: mm): 01:00; Maximum failures: 1.

Empirical results

After demonstrating the mathematical results of the newly constructed algorithm, in this section we show the results obtained with the NN. The dataset includes n + 1 countries (i.e. 192) distributed according to the seven variables (plus their mathematical transformations): The architecture of the derived NN is shown in Figure 1. We can see that the NN is of multivariate type. Two targets attributable to the variables deaths (April) and mortality rate (April) were isolated.

Fig. 1. Constructed ANN.

Source: our elaborations in Oryx.

Now, we analyse the ANNs results. In total, the experiment consists of 25 inputs and 2 targets (Fig. 2).

Fig. 2. Variables bars chart.

Source: our elaborations in Oryx.

Through the pie chart in Figure 3, we can observe in detail the use of all instances in the dataset.

Fig. 3. Instances pie chart.

Source: our elaborations in Oryx.

The total number of instances is 192. The number of training instances is 116 (60.4%), the number of selection instances is 38 (19.8%), the number of testing instances is 38 (19.8%), and the number of unused instances is 0 (0%). These results can be explained as follows: the training instances designed the best process for the neural algorithm. It has a design accepted 60 times out of 100 and its value is higher than another algorithm (selection instance) and the same value as the testing instance.

Before analysing the results obtained on the target variables, we observe the behaviour of the data within our NN structure. We interrogate the algorithm to illustrate how the real signals among the numerous nodes are compatible with the predicted ones. The underlying hypothesis is that the NN prediction process coherently reflects the environment and thus the reality of the data with respect to an unsupervised approach. The results are shown in Figure 4.

Fig. 4. Fit model process.

Source: our elaborations in Oryx.

The algorithm starts from a signal optimisation process that one can imagine coinciding with a dated time 1. Subsequently, as the signal (and therefore the data distribution) is different from the respective weights between the nodes, the algorithm begins to choose the best signal optimisation process. This procedure is an automatic learning process resulting from 1796 strings of commands that leave the machine in a situation of quasi-controlled autonomy. Therefore, the figures named with numbers from 2 to 4 result from the signal optimisation process. In fact, we can observe how at time 4 (top left panel) the predicted data coincide perfectly with the predicted ones. The methodological summary of graphs 1–4 is represented in the last figure below. Thus, it is easy to see how the NN model can perfectly predict data coming from the real world. Once we have observed that our NN is able to generate highly reliable predictive values, we estimate which input might explain the desired targets. To this end, according to Mele et al. [4], we use the ‘Plot Directional Output’ function. In other words, we implement the new algorithm through what is commonly called the ‘cut effect’ of the NN. Hence, the following commands into the algorithm have been introduced:

Core commands:

The results of the ‘cut effect’ analysis in the NN are illustrated in Figure 5. In general, the ‘Plot Directional Output’ function analyses numerous combinations of inputs on the outputs, automatically choosing the one that can influence the most network signal process.

Fig. 5. ‘Cut Effect’ based on a new ML algorithm.

Source: our elaborations in Oryx and AD-Designer 2021.

Figure 5 shows the vital result of the experiment here. The cut-off signal in the transmission of the nodes generated the following propagation: from the input ‘D.Total Vaccinated 15 March 21’ to the target ‘Fatality Rate April’ it has been identified at point 12.8 regarding the ordinate y-axis (target) and level 26079511.3 concerning the abscissa x-axis (input). This result requires an interpretation of the data that has been cut during the signal propagation. In particular, considering 192 countries under study here, the respective vaccination plan, the turnout in the participation in the vaccination campaign, and the doses administered, these countries suddenly saw a reduction in the Case Fatality Rate of COVID-19 precisely at the point where the cut effect was generated in our NN.

This result is significant for the international scientific community. It would demonstrate the effective impact of the vaccination campaign on the Case Fatality Rate from COVID-19, whatever the country considered. Indeed, once the vaccination has started (for vaccines that require a booster, we refer to at least the first dose) the antibody response seems to prevent the probability of death from COVID-19. The graph of the cut effect is clear: at a certain point, the Case Fatality Rate collapses with increasing doses administered. To be able to understand the delay effect in the activation of neutralizing antibodies (14 days of the activation of IgG antibodies), we have used a signal amplification process (Fig. 6).

Fig. 6. Signal amplification process.

Source: our elaborations in Oryx and AD-Designer 2021.

According to the Qiagen experiment (Valencia 2007) and the Siemens approach (Tarrytown 2006), our algorithm simulates the signal amplification process used in virology. This technique was introduced to overcome the limitation of polymerase chain reaction tests. However, unlike the signal test in virology, since we did not use the enzyme rotation techniques, we simulated that the neural weights can be amplified in n + 1 cycles, to make the underlying factors more visible.

It was built through a new algorithm implementation that generated the cutting effect. In other words, through this analysis, we have interrogated the machine to illustrate time delays in the propagation of the signal before the algorithm detects the cut effect. Thus, with this methodology, we have created a hypothetical premise to grasp the 14 days of delay necessary for the human body to produce immunity. The result is really interesting, as we can see that the signal undergoes a significant amplification even before the cut relative to Figure 5. Therefore, this observation allows us to limit the gap between those who start vaccination (but immediately produce the IgM) and colours who within and beyond 14 days acquire protective immunity from COVID-19.

Finally, to check for errors in the prediction process, we test our model through two different tools: the Conjugate gradient errors history and the Quasi-Newton method (Fig. 7).

Fig. 7. ML diagnostic tests.

Source: our elaborations in Oryx.

Both methods follow the belief that the best training strategy is the one that allows the best possible loss of information as interactions or epochs grow. The conjugate gradient is used for training. In this algorithm, the search is performed with conjugate directions, which generally leads to faster convergence than the gradient descent directions. The initial value of the training error (orange line) is 49.6111, while the final value after 569 iterations is 0.279563. The initial value of the selection error (blue line) is 4.2048 and the final value after 5 ITE is 0.000243. The analysis of this test confirms that the error in our prediction the error decreases as the iterations increase, and this result confirms the goodness of the selected model.

Instead, the Quasi-Newton method computes an approximation to the inverse Hessian at each iteration of the algorithm using only the gradient information. The blue line represents the training error, and the orange line represents the selection error. The initial value of the training error is 0.065332 and the final value after 1000 epochs is 0.0005. The initial value of the selection error is 112.0454 and the final value after 1000 epochs is 0.0004. This analysis, which confirms the previous one, also shows that the whole selection and training process of our NN has almost no error as the epoch increases.

Conclusions and policy implications

COVID-19 and future epidemics of novel influenza viruses pose more and more a serious threat to the security and public health of nations [1, 6, 30, 31, 32, 34, 35]. The global response to the COVID-19 pandemic has pushed the research for detecting factors and aspects associated with a rapid pandemic response in several areas, including vaccine development, distribution, allocation and administration. This study suggests efficient strategies of vaccination for reducing the impact of the novel viral agent that might not disappear in the short-term because of new variants [60]. We demonstrated, through a new ML algorithm, that the vaccination campaign significantly reduced the negative effects of the COVID-19 pandemic, with a sharp decrease in the fatality rate. In particular, results suggest that, based on the respective vaccination plan, the turnout in the participation in the vaccination campaign, and the doses administered, countries under study suddenly have a reduction in the fatality rate of COVID-19 precisely at the point where the cut effect is generated in the NN. This result is significant for the international scientific community. These results here can help policymakers to design satisfying goals to cope with current infectious diseases with effective vaccination strategies to prevent future outbreaks of new variants of COVID-19 and similar infectious diseases in the future.

Although this study has provided some interesting results, that are of course tentative, it has several limitations. First, a limitation of the study is the lack of data about doses administered and total vaccinations in several countries, mainly in the spring season of the year 2021, and also the difficulty of production and distribution of COVID-19 vaccines worldwide; moreover, country-specific health norms may affect the gathering and transmission of data, such that unreported doses of vaccines and deaths in manifold countries may be present in the database under study here. Antony et al. [61] explained that the similarity in presentation between COVID-19 and influenza can have generated underreported data across countries. Second, not all the possible confounding factors that affect the efficacy of vaccinations are taken into consideration (such as non-pharmaceutical measures applied, health expenditures, numbers of ICUs, equipment of medical ventilators in hospitals, hesitancy to vaccination, etc.) and in the future, they deserve to be controlled for reinforcing results here. Third, the lack of integration of data with the age of vaccinated people among countries (the priority given in many countries to elderly subjects, with a more compromised immune system) may have influenced the results of infected individuals and deaths across countries. Fourth, new variants of COVID-19 may reduce the effectiveness of current vaccines and vaccination campaigns and these aspects have to be considered in the future development of this study. Thus, generalizing the results of this research should be done with caution. Despite these limitations, the results presented here clearly illustrate the critical level of vaccination rollout that has significantly reduced the negative effects of the COVID-19 pandemic in terms of a sharp decrease in the case of fatality rates across countries. These findings can better support the strategies for prevention and/or reduction of negative effects of the pandemic crisis in society [30, 33]. Future research should consider new data when available, and when possible, also examine time series of variables within countries to explain more dynamic relations of the phenomena and relationships under study here over time and space. In fact, as the vaccination campaign progresses around the world, it would be interesting to repeat the analysis conducted here to check if the results we have presented are confirmed through a larger sample. Moreover, the empirical analysis may be conducted using a different AI algorithm or a completely diverse approach [27].

To conclude, there is a need for much more detailed research on these topics and this study encourages further investigations for supporting optimal strategies of vaccination plans, using lessons learned from COVID-19, also considering the interaction between the evolution of pandemics with new variants and vaccination campaigns, and different factors between countries that are not only parameters related to medicine but also to public governance that can clarify results and improve the preparedness of countries to face next pandemic crisis and control negative impact on public health, economy and society.