Development of the COVID-19-Anxiety Questionnaire and first psychometric testing

Summary

The coronavirus disease 2019 (COVID-19) pandemic may cause psychological distress in the general population and has the potential to cause anxiety regarding COVID-19. No validated questionnaires exist for the measurement of specific COVID-19 anxiety. We modified the DSM 5 – Severity Measure for Specific Phobia–Adult scale specifically for anxiety regarding COVID-19, and report the psychometric properties from an online study with 6262 participants from the general population in Germany. We analysed internal consistency as well as concordant validity. The scale showed good internal consistency (α = 0.86) and good concordant validity (r_s = 0.60) with the 2-item Generalized Anxiety Disorder measure and r_s = 0.61 with self-rating of limitations in daily life caused by COVID-19 anxiety.

Background

The current coronavirus disease 2019 (COVID-19) pandemic has the potential to cause severe psychological distress for the general population worldwide.^1–3 The development of public health approaches to combat COVID-19 heavily depend on strategies for social and behavioural change. It is therefore of critical importance to assess and compare behavioural as well as psychological responses to COVID-19.

Although there are a large number of media reports on the psychological consequences of the COVID-19 pandemic substantial research on this topic is lacking. Several cross-sectional studies as well as one longitudinal study, have shown elevated levels of psychological distress, depression and anxiety.^3–9 For example, in a longitudinal study with 1738 participants from the Chinese general population elevated levels of symptoms of post-traumatic stress disorder, depression and anxiety were found, which were in large part stable from the first assessment at the end of January to the second one at the end of February 2020.⁶ In addition to general symptoms of psychological distress, depression and anxiety, the development of symptoms of anxiety specifically focused on COVID-19 seems possible.² With an increasing number of infections, the development of ‘coronaphobia’ in a growing number of individuals has to be expected.²

Aims

Although there are a large number of well-established and validated questionnaires to assess general anxiety as well as disorder-specific anxiety, to our knowledge there are no scales for the assessment of specific anxiety regarding COVID-19. Research undertaken during past outbreaks of infectious diseases show that psychometric questionnaires are able to identify individuals at high risk for the later development of mental health problems.^10,11 Thus, screening questionnaires that enable the incorporation of screen-and-treat approaches in clinical settings, as well as their use in research focusing on anxiety in the context of the current pandemic, are urgently needed.² The purpose of this study was to present a scale that aims to measure COVID-19 anxiety. Therefore, we present a slightly modified version of the DSM-5 – Severity Measure for Specific Phobia–Adult (SP-D)^12,13 questionnaire that focuses on anxiety regarding COVID-19 and we report our results on the psychometric properties from a large online survey. Using our measure research about whether specific forms of anxiety relating to COVID-19 are present in the general populations worldwide should be possible. As we do not know whether such specific forms of anxiety exist, the scale is thought of as an addition to the existing established measures of anxiety rather than a replacement.

Method

Development of the questionnaire

Modifications to the original SP-D questionnaire were as follows. We modified the original instruction: ‘The following questions ask about thoughts, feelings, and behaviors that you may have had in a variety of situations’ to ‘The following questions ask about thoughts, feelings and behaviors that you may have had in connection to Corona’. Furthermore, we added ‘due to Corona’ at the end of item 1; ‘felt regarding Corona …’ at the beginning of item 2; modified ‘in these situations’ to ‘due to Corona’ in item 3; added ‘caused by anxiety regarding Corona…’ at the beginning of item 4; added ‘due to Corona…’ at the end of item 5; modified ‘these situations’ to ‘situations linked with Corona’ at the end of item 6 and 8; and replaced ‘these situations' with ‘Corona' at the end of item 9. Item 7 and 10 remained unchanged. Supplement 1, available at https://doi.org/10.1192/bjo.2020.82, contains a link to the original SP-D questionnaire by the American Psychiatric Association¹³ and also full details of our modifications to the SP-D.

We decided to use the non-scientific term ‘Corona’ as we thought it might be more commonally used in the general population. We named the resulting questionnaire the COVID-19-Anxiety Questionnaire (C-19-A).

Design, recruitment and eligibility criteria

To test its psychometric properties, we used data from a large-scale online survey on psychological distress, anxiety and depression in the current COVID-19 pandemic in Germany. This large trial was approved by the ethics commission of Charité Universitätsmedizin Berlin (Approval number: EA1/071/20) and registered on clinicaltrials.gov (NCT04331106). The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

The questionnaire was delivered online via SoSci Survey; participants were recruited mainly using the official social media channels (Twitter and Instagram) of the Charité University Medicine and the Charité University Medicine website. An invitation to participate in the study was posted on each channel once. We did not use paid advertising. No compensation was offered. Participants gave their informed consent. The recruitment period was from 27 March 2020 to 10 April 2020.

Measurement

Participants answered questions about demographics as well as completing the C-19-A. For assessment of concordant validity regarding general anxiety and depressive symptoms we used the four-item Patient Health Questionnaire 4 (PHQ-4)¹⁴ an ultra-brief screening tool for anxiety and depression, that can be divided into two subscales of depression (PHQ-2) and anxiety (Generalized Anxiety Disorder, GAD-2). For the assessment of concordant validity regarding specific anxiety symptoms that focus on COVID-19, and with the aim of measuring the extent of anxiety, in that specific anxiety can be problematic, we used two self-developed items: ‘My anxiety concerning Corona leads to limitations in my daily life’ as well as ‘My anxiety concerning Corona is exaggerated’ which could be answered on a six-point Likert scale from not true at all to totally true.

Data analysis

Data was analysed using SPSS 25.0. Internal consistency was calculated using Cronbach's alpha. Concordant validity was calculated using Spearman's Rho.

Results

Participants

Of the 6262 participants, 70.5% (n = 4412) reported gender as female, 28.7% (n = 1796) reported gender as male and 0.9% (n = 54) reported gender as diverse. Tables 1 and 2 provide details of the demographics of the participants and the descriptive statistics of the scales use, respectively.

Table 1 Demographics of the sample (n = 6262)

Table 2 Descriptive statistics of the used scales

Internal consistency

The C-19-A scale showed an internal consistency of α = 0.86 (n = 6262). We did not find great differences in internal consistency when calculated for different subgroups (men: α = 0.86; women: α = 0.86; age ≤30: α = 0.86; age >30 and ≤60: α = 0.87, age >60: α = 0.89).

Concordant validity

The C-19-A showed correlations of r_s = 0.58 (n = 6247, P < 0.001) with the PHQ-4; r_s = 0.45 (n = 6248, P < 0.001) with the PHQ-2; and r_s = 0.60 (n = 6247, P < 0.001) with the GAD-2. Figure 1 shows scatterplots for these associations.

Interpolation lines: PHQ-4: y = 4.77 + 1.3 × x, R ²-linear, 0.35; PHQ-2: y = 6.09 + 1.92 × x, R²-linear, 0.22; GAD-2: y = 5.13 + 2.47 × x, R ²-linear, 0.39.

Fig. 1 Scatterplots for the associations of COVID-19-Anxiety Questionnaire (C-19-A) with the 4-item Patient Health Questionnaire (PHQ-4), PHQ-2 and Generalized Anxiety Disorder (GAD)-2.

Furthermore, the C-19-A showed a correlation of r_s = 0.61 (n = 6262, P < 0.001) with the item ‘My anxiety concerning Corona leads to limitations in my daily life’ and a correlation of r_s = 0.32 (n = 6262, P < 0.001) with the item ‘My anxiety concerning Corona is exaggerated’. When analysed for the different subgroups regarding age and gender, correlations did not show substantial differences between the groups.

Discussion

We found higher average values for C-19-A (mean 10.14) than those reported for the original scale (SP-D) in a sample of 102 patients seeking out-patient treatment for a mental disorder in Germany in 2010 and 2011 (mean 8.2).¹⁵ Even though the two measures are not directly comparable, the fact that we found higher average scores on the C-19-A in a sample of the general population than those in a study that assessed a population of patients seeking treatment for a mental disorder seems remarkable, and is in line with the hypothesis that the COVID-19 pandemic causes a large amount of psychological distress in the general population.¹ However, these results should be interpreted with great caution as this study is only a cross-sectional assessment and longitudinal studies are needed to generate more information on this subject.

The C-19-A revealed good internal consistency (α = 0.86), which is somewhat lower but comparable with the results of the validation study of the SP-D where a version with only eight items showed an internal consistency of α = 0.91 in a non-clinical sample with 702 participants, and similar values in the 10-item version (α = 0.93 in a non-clinical sample with 57 participants and α = 0.93 in a clinical sample with 48 patients with anxiety disorders).¹² This demonstrates that the C-19-A is a reliable method to assess specific anxiety about COVID-19. It is, however, important that test–retest reliability is assessed further; we were not able to analyse this in our study. The rapidly changing situation in the COVID-19 pandemic makes it very difficult to assess test–retest reliability.

Our results demonstrate that the C-19-A achieved good concordant validity with depressive and anxiety symptoms as well as with specific anxiety regarding COVID-19. In terms of general anxiety, our scale revealed a correlation of 0.60 with the GAD-2, whereas the original scale showed a correlation of 0.53 with the GAD-7.¹⁵ These results suggest that the anxiety regarding COVID-19 seem to be closely associated with general anxiety symptoms. The correlation of the C-19-A with the PHQ-2 of 0.45 shows that there is also a close relationship with depressive symptomatology, whereas the fact that the correlation is lower than that with general anxiety is in line with our aim to measure specific anxiety symptomatology (discriminant validity). The high correlation of our scale with the item that assessed the amount of limitations that anxiety regarding COVID-19 causes participants in their daily life shows that our scale might be able to assess clinically relevant anxiety regarding COVID-19. This is also supported by the moderate correlation with the item that asks for the extent to which the participants rated their anxiety regarding COVID-19 as exaggerated.

Strengths and limitations

This study presents the first self-rating scale to assess anxiety regarding COVID-19, which seems urgently needed for further research. We used a well-validated measure (SP-D) as the basis and made only slight modifications with the aim of not losing the good psychometric properties of the original instrument. We present data on the psychometric properties from a fairly large sample of over 6000 participants. Nevertheless, there are some limitations.

We were unable to assess test–retest-reliability, the measures used to test concordant validity were very short and it was not possible to assess further the differences between general anxiety and COVID-19 anxiety. Therefore, further studies with longitudinal data and structural equational modelling are needed. Another limitation comes from the fact that we used a convenience sample that might come with sample bias; those familiar with social media such as young participants and also people with higher levels of anxiety might have been more likely to participate in our study. As a result of this, levels of anxiety might have been overestimated in our study. Furthermore, we were not able to assess how many people received the invitation to participate in the study and therefore we were unable to calculate the response rate. The recruitment strategy used may have reduced the generalisability of our results as there are some differences between our sample and the general population in Germany (for example lower age and overrepresentation of women). However, in our sample, no substantial differences in internal validity and concurrent validity were found for different subgroups regarding age and gender.

Implications

In conclusion, we present the first self-rating scale for anxiety relating to COVID-19. The instrument is a modification of the SP-D scale. Our data show that the scale has good internal consistency as well as good concordant validity. This instrument might help researchers identify determinants of adaptive and mal-adaptive psychological responses to COVID-19, such as COVID-19-specific anxiety. Thereby it might help in identifying subgroups of the general population that show a higher risk for anxiety and enable the development of individually targeted treatment options for individuals at higher risk for psychological burden as a result of the COVID-19 pandemic.