A Short Scale for Measuring Political Secularism

Abstract

As religiousness is declining across democracies, scientific interest in secular orientations and their political implications is growing. One specific and particularly important aspect of secular attitudes is political secularism. Political secularism is not merely the absence of religiousness, but rather a world view which holds that religious beliefs should play no role in politics. While there are hundreds of survey instruments that measure the strength and content of religiousness, there is no comparable measure that taps into political secularism. In this research note, I briefly review the concept of political secularism and present a cluster of items which target it. Utilizing data from four large population representative samples taken from the eastern and western states of Germany, I use confirmatory factor analysis to show that these items form a short but internally consistent scale. This scale also displays convergent and discriminant validity. It may be readily used in future surveys.

Introduction

Social scientists have long been aware that religion is a powerful political force and that by implication, significant changes to the religious makeup of a society will have far-reaching political repercussions. The growing number of people who identify as Muslims is one such change that has become a hot button issue in Europe. However, Muslims currently make up just 5% of Europe's population, and even under a high immigration/high fertility scenario, their share will rise to less than 15% over the next three decades (Pew Research Center 2017). At least numerically, secularization is a much more important development in Europe.

Secularization is a multi-faceted term which denotes both social and intrapersonal processes (Sommerville 1998) that diminish the role of religion at the societal, group, and individual levels (Stolz and Tanner 2019, 2). At the micro-level, applied research usefully distinguishes between a decline in membership and practice on the one hand and a decline in individual beliefs on the other. The former may outpace the latter, creating “unchurched” segments of the population that still believe but no longer belong (Davie 1990). However, in their global survey of religious life, Norris and Inglehart (2011) demonstrate that in some parts of the world and most notably in Western Europe, both membership and belief have been on the retreat for decades. These trends continue to the present day, making Western Europe “one of the world's most secular regions” where “many do not describe themselves as Christian” (Pew Research Center 2018, 6).

For many of these citizens (in Europe and elsewhere), being secular is not just the absence of religion: they embrace secularism as a non-religious world-view. In the broadest possible sense, secularism is an idea that “seeks to restrict religious elements” (Stolz and Tanner 2019). While such an outlook has multiple dimensions, one particularly interesting one is political secularism, a preference to limit religion's role in politics (Beard et al. 2013).

One prominent account (Voas 2009) of religious change in Europe that is well supported by empirical data holds that long-running societal secularization brings about individual secularism, via a mechanism of changing socialization patterns and generational replacement. At the same time, individual (political) secularism may lead to demand for policy changes (e.g., full legal recognition of same-sex partnerships) that in turn drive further societal secularization. Somewhat paradoxically, secularization in Europe is also contributing to a re-politicization of religion: it makes societies more culturally diverse and results in polarization between religious organizations and citizens on the one hand and their political secular opponents on the other (Pickel 2017).

To gauge the importance of these issues at the micro-level, a survey measure for political secularism is required. Yet, although there is a plethora of scales for measuring religiousness, secularism as such is rarely studied, and not a single psychometrically evaluated instrument for measuring politically secular attitudes exists.

In this research note, I briefly review the concept of political secularism and present a cluster of items which target it. Using data from four large population representative samples, I show that these items form a short, internally consistent scale, and that this scale also displays convergent and discriminant validity.

What is Political Secularism, and How can it be Measured?

Rates of religious membership and participation have been a concern for social scientists since the inception of sociology in the 19th century. Attempts to complement them by instruments that capture more private religious practice as well as the structure of religious beliefs and attitudes hark back to at least the 1950s. Allport and Ross's distinction between intrinsic and extrinsic orientations (Allport and Ross 1967) is perhaps the most famous example of this early research. Less than two decades later, Gorsuch (1984) reviewed the many instruments already existing at the time and strongly argued against their further proliferation. Yet after another quarter-century, Cutting and Walsh (2008) counted nearly 180 scales for measuring religiousness, and their number is still growing.

Almost all of these scales implicitly follow Sommerville's classic notion that secular attitudes represent a “shift of attention from ultimate (religious) concerns to proximate concerns” (Sommerville 1998, 250), i.e., a lapse from or a neglect of religion. While such “passive secularism” is an important phenomenon in itself, “active secularism,” i.e., “the affirmation of secular identity and beliefs” (Campbell et al. 2018, 553) is arguably a more interesting cluster of attitudes. While passively secular citizens are by definition indifferent toward religion but may still accept its traditional role in society and politics, actively secular citizens are positively opposed to religious arguments and authority, which may spur them into political activity and lead them to adopt more extreme positions (see, e.g., Layman and Weaver 2016; Brockway 2018; Layman et al. 2021).

Just like religion, secularism has several “dimensions” (Campbell et al. 2018) or “facets” (Beard et al. 2013). These include philosophical convictions, social values, secular identities, and negative emotions toward religious authority and doctrine, all of which can have political consequences.

In a narrower sense, political secularism refers to one's attitudes on “the proper role of religious beliefs in political life” (Beard et al. 2013, 758). More succinctly, political secularism holds that religious beliefs should not be used to justify curbing individual freedoms, and should play little, if any role in political debates.

As a political and legal idea, political secularism is well-researched (Bhargava 2008; Cliteur and Ellian 2020). However, there is not one validated scale for political secularism at the micro-level. This is problematic, because politically secular views are believed to become more and more widespread in Europe and even in the United States, and because they represent a particularly disruptive type of active secularism. This is because they are conceptually linked to a host of polarizing “morality policies,” which pit religious views and actors against notions of individual freedom (Mourão Permoser 2019). If one wants to assess the alleged spread of politically secular views as well as their impact on politics, it is therefore desirable to have an instrument that is both reliable and valid and preferably comprises multiple indicators.

However, even single-item questions or short batteries that tap into politically secular views are rare in the extant literature. Beard et al. (2013), who prominently make the case for researching political secularism at the micro-level, propose only two items that fit their own definition: “Do you worry the government is getting too involved in the issue of morality?” and “Should churches and other houses of worship keep out of political matters” (Beard et al. 2013, 760).¹

Brockway (2018), in his useful study of “committed seculars,” relies on a single item that refers to one important but very specific aspect of political secularism, namely the separation of church and state. Similarly, Castle (2015) measures opinions on “religious establishment” using an index comprising four items that refer to government institutions and officials engaging in and supporting religious activities.

Finally, Ribberink et al. (2018) measure “anti-religiosity” by combining four items that were included in the 1998 and 2008 waves of the International Social Survey Programme (ISSP). Two of these items refer to respondents' views of believers and their creeds as intolerant and belligerent. Such opinions may be secular, but they are not politically secular in the aforementioned sense. The two other items tap into attitudes on the role of religious leaders in politics. These items are a better fit for the concept of political secularism but again address a single, rather narrow and specific aspect of it.

Conversely, a valid and reliable scale for measuring political secularism should consist of several items that cover the breadth of the relationship between religion and politics. While developing such a scale from scratch would have its own obvious benefits, re-using existing items from large-scale surveys has at least three important advantages. First, it is already clear that these items “work” in surveys of the general public. Second, replicating single items or group of items opens up the possibility of meaningful comparisons between new and existing surveys. Third, if a whole set of items is replicated that form a scale, its reliability and validity can be assessed across different settings.

The questionnaire of an older survey (“Church and Religion in an enlarged Europe” (CREE)), which ran in 2006 in various European countries and targeted attitudes on “religion, society, and life in general” (Pollack et al. 2007, 20), provides a useful starting point. Its large pool of questions contains five items that closely fit Beard et al.'s (2013) definition of political secularism.

One of these items (Symbols, see below) was taken from the earlier “Religious and Moral Pluralism” survey. Another one (Debates) came from the ISSP survey series and is one of the two that were also used by Ribberink et al. (2018). The remaining three were written by members of the CREE team.²

These five (and also some of the other) items were then subsequently included in the 2012 iteration of Germany's biannual General Social Survey (ALLBUS) in somewhat modified form:

1. (1) “The EU treaties should contain a reference to the Christian god” (God EU) (reversed)

2. (2) “Religious symbols such as crucifixes should be banned from state schools” (Symbols)

3. (3) “Education in state schools should be free of religious elements” (Education)

4. (4) “Scientific research on humans should not be limited by religious norms and values” (Science)

5. (5) “Religious norms and values have no place in public debates on political issues” (Debates)

When the “Problems of Representation in the Domain of Biopolitics” (PRDB) project set out in 2016 to measure political secularism and its effects at the micro-level, it adopted these five items for its nationally representative survey. For comparability with the ALLBUS and because the rewritten items are even better suited for measuring political secularism, the PRDB project also used the new wording.³

The PRDB project chose these items because collectively, they capture different aspects of political secularism as defined by Beard et al. (2013). The first question was highly controversial in European Union politics in the 2000s and is still useful as a supranational variation of the classic question about the separation of church and state. The second and third items relate to the role of religion in state schools, which is another contentious issue for secularists in many societies. The fourth item alludes to the role of religious authority in one specific domain of regulatory politics where appeals to religious doctrine are frequent. More generally, it also taps into secular support for science and rationality. Finally, the fifth item spells out the core tenet of political secularism: that religion should play no role in politics.

Data

The analyses presented in the remainder of this research note are based on data from the 2012 ALLBUS round (GESIS—Leibniz-Institut für Sozialwissenschaften 2019) and the 2016 PRDB survey (Arzheimer 2016). Both datasets are available in their entirety from Germany's national data archive, where they can be located by following their respective DOIs.

Because of the persistent political, social, and economic east-west differences within Germany, both studies drew separate and disproportionately large samples of respondents from Germany's eastern states. For the present research question, this is particularly apt because West German politics was dominated for decades by a Christian Democratic party, whereas Eastern Germany was a self-avowed irreligious state. As a consequence of this research design, the analyses are effectively based on four large (n = 740–2,538), independent samples.

While the ALLBUS usually uses an odd (five, seven, or eleven) number of categories for measuring agreement, they retained the CREE's original four-point format. Conversely, the PRDB survey employed five categories with a neutral mid-point for almost all items and adjusted the items' response options accordingly. Otherwise, the wording of the items is identical across the samples.

Reliability and Dimensionality

Methods

Any presumptive scale must be internally consistent, i.e., its items should tap into a single common dimension. Internal consistency is one important type of reliability, which in turn is a necessary but insufficient condition for validity. Traditionally, internal consistency is assessed by calculating Cronbach's alpha (Cronbach 1951), which can be interpreted as the mean of all possible split-half reliabilities across a given set of items.

Confirmatory factor analysis (CFA) provides a complementary perspective that is more nuanced and can also help mitigate some of the well-known problems⁴ linked to the exclusive reliance on alpha (Viladrich et al. 2017). However, the common practice of establishing unidimensionality mechanically by assessing the fit of a single-factor model may also be misleading, because established fit indices as well as newer measures may be too sensitive to trivial violations under some circumstances while failing to identify clearly multidimensional structures in others (Reise et al. 2013). Therefore, a detailed inspection of the measurement model that takes into account not just various fit indices and newer variations of alpha but also the factor loadings and possibly alternative specifications of the measurement model is required.

One final complexity arises from the ordinal nature of the survey responses. While items with as few as five categories are sometimes treated as if they were continuous (Viladrich et al. 2017), the WLSMV estimator, which was specifically developed for large samples of ordinal measurements, was employed for both surveys. Therefore, the factor loadings are probit coefficients, and variants of alpha that account for the ordinal nature of the data (Green and Yang 2009) were calculated. Threshold parameters are omitted from the main tables but documented in the Appendix.

Findings

Table 1 shows the estimates for a preliminary measurement model for political secularism. All five items display substantial factor loadings in the correct direction. (Ordinal) alpha varies between 0.7 and 0.83, a range that is usually seen as acceptable to good. Because the measurement model assumes that the items are congeneric whereas alpha presupposes tau-equivalent items, (ordinal) omega is a more appropriate measure of internal consistency (Viladrich et al. 2017). The respective row in Table 1 shows that values for omega are indeed slightly higher, but the differences are minuscule.

Table 1. Measurement of political secularism by five items (factor loadings only)

***p < 0.001; **p < 0.01; *p < 0.05. Threshold parameters are omitted from the table for better legibility. Full set of parameters are given in the Appendix.

The evidence from the fit indices, however, is more mixed. The CFI and the SRMR show an acceptable fit in all four samples. Conversely, the TLI and the RMSEA indicate that the model fit is unsatisfactory for all respondents but those interviewed in the east in 2012.

A closer look at the bivariate correlations reveals that there is a particularly close relationship between the two items referring to the role of religion in state schools, which can be explained by their very similar content. This in turn suggests that allowing their error variances to correlate could improve the model fit substantially.

From a substantive point of view, however, it is preferable to remove the Symbols item from the scale. First, having two items that refer to public education gives disproportionate weight to this specific aspect of political secularism. Second, while both items are similar, “religious elements” is more general and already encompasses “religious symbols.” Third, the meaning of the former item is also clearer, because almost all federal states provide some form of religious education in state schools, whereas crucifixes in schools (and other public buildings) are closely linked to the Bavarian Christian Democrats (CSU) and their representation of Bavarian identity. Moreover, because “religious symbols” also include the hijab, they item may also tap into anti-Muslim sentiment. And finally, short(er) scales have clear advantages: they often provide a more realistic assessment of reliability, increase the motivation of and decrease the burden on respondents, and are cheaper to administer (Gogol et al. 2014).

Table 2 shows the estimates for the four-item measurement model. The factor loadings remain high, and their direction is still correct. Omega and alpha are somewhat lower, which is expected because they partly depend on the number of items, but still acceptable. Finally, the CFI, TLI, and SRMR indicate near perfect fit in all four samples. The RMSEA indicates a very good fit for all samples but the eastern one in 2016.

Table 2. Measurement of political secularism by four items (factor loadings only)

***p < 0.001; **p < 0.01; *p < 0.05. Threshold parameters are omitted from the table for better legibility. Full set of parameters are given in the Appendix.

Taken together, these findings suggest that the four items form a consistent scale that works reasonably well across slightly different formats (four versus five scale points) and cultural settings (Germany's staunchly secular eastern states versus the more religiously diverse states in the west). The next section will assess the question whether the scale is also valid.

Validity

Many different types of validity have been proposed in the literature. However, they all relate to the same underlying question: to what degree does an instrument tap into what it is supposed to measure? Thus, they should be seen as “tests” that provide (sometimes conflicting) evidence on the question of validity, which “can never be answered with absolute certainty” (Bollen 1989, 185).

This uncertainty is fundamental: in an ideal world, validity could be quantified simply by the correlation of a scale with some universally accepted external criterion, but in reality, there is usually no such gold standard. Like reliability, validity is therefore best investigated making use of CFA, which opens up many sophisticated possibilities (see, e.g., O'Leary-Kelly and Vokurka 1998).

In this section, I follow a comparatively simple approach (see, e.g., Hayes et al. 2005, 309): I demonstrate that scores on the secularism scale are highly correlated with measurements that they should correlate with (convergent validity), yet remain distinguishable from them (discriminant validity). This provides a more rounded view of the scale's validity than the relationship with a single criterion and is a first step toward a full nomological validation, which would test a whole network of theoretical propositions and auxiliary assumptions (see, e.g., Hagger et al. 2017). There are two constructs available across the four samples that can be used for validation, one rather specific, the other more general.

Attitudes on a specific policy, namely legal access to abortion, form the first of these constructs. The regulation of abortion is widely seen as one of the most prominent “morality issues” (Mooney and Schuldt 2008; Studlar et al. 2013) worldwide. Efforts to limit legal access to abortions are almost universally spearheaded by religious organizations, who explicitly claim that they act on of their beliefs. This suggests that political secularism should be positively associated with more permissive views on abortion. However, because this is a single specific policy, because many other factors may shape one's views on abortion, and because the items used in the surveys make no reference to religious doctrine, this correlation should be far from perfect.

All surveys include four identical or very similar items on the regulation of abortion (see the Appendix for details). A preliminary analysis shows that all items display substantial loadings on the common factor. Ordinal alpha is high and ordinal omega is acceptable across all four samples so that the construct may be used for validation.

The second, more general construct is a composite measure of religiousness that encompasses individual and collective religious practice on the one hand (prayer and attendance) and self-conception as more or less religious on the other. All three items display substantial loadings on the common factor. Alpha is high, and omega is acceptable across all four samples (see the Appendix for details).

Following Beard et al. (2013), I have argued above that political secularism should not be understood as the absence of religiousness but rather as an attitude on the proper role of religion in political life. For the validity of the political secularism scale, its correlation with religiousness is therefore crucial. Overall, a strong negative relationship is to be expected: non-religious citizens should be opposed to religious interference in the public sphere, whereas believers should support institutional arrangements that make outcomes which align with their beliefs more likely. Having said that, many religious citizens (particularly in some protestant traditions) may still favor a clear separation of church and state. Conversely, lukewarm believers and even non-believers may see a societal value in religion. Political secularism and religiousness should hence remain distinguishable.

To investigate the validity of the secularism scale, a joint model of secularism and attitudes on abortion was estimated (see the tables in Appendix for the full results). Table 3 shows that the two-factor model, which treat secularism and abortion attitudes as separate constructs, fit the data very well. In line with expectations, the correlation between the factors is clearly positive (more secular respondents do indeed have more permissive views on legal access to abortion) but far from perfect, with values between 0.35 and 0.53.

Table 3. Secularism and abortion attitudes

***p < 0.001; **p < 0.01; *p < 0.05.

Conversely, assuming that all items load on a single common factor results in a dramatic decline of model fit to values that would be unacceptable by common standards. This provides some initial evidence for the convergent and discriminant validity of the secularism scale.

In the next step, a joint model for secularism and religiousness was estimated (see the tables in the Appendix for the full results). Table 4 shows that a two-factor solution again achieves an acceptable fit across all four samples.⁵ In line with expectations, the estimated correlation between secularism and religiousness is negative and quite substantial, with values ranging from −0.56 to −0.70.

Table 4. Secularism and religiousness

***p < 0.001; **p < 0.01; *p < 0.05.

To ascertain that political secularism and religiousness are indeed closely related but not identical, again all items were forced to load on a single common factor. Once more this results in a dramatic increase in χ ² and a decline in all measures of model fit well below the common cut-off criteria.

Taken together, these findings already provide strong evidence for the convergent and discriminant validity of the political secularism scale. Ideally, they should be complemented by a full nomological validation in future research.

Conclusion

Recent research highlights the importance of political secularism as an individual attitude that affects political behavior. While secularism as a political and philosophical concept has existed for centuries and while there is a plethora of measures for individual religiousness, there is no established instrument for measuring political secularism at the micro-level.

In this research note, I introduce a cluster of items which tap into individual political secularism. Applying CFA, I show that these items form an internally consistent scale. Importantly, good internal consistency can be achieved with just four items. Such short scales are becoming increasingly popular, because they reduce both surveys costs and the burden on respondents.

The secularism scale also displays a moderate to strong positive correlation with permissive views on abortion and a strongly negative correlation with religiousness, yet it remains clearly distinguishable from both. This suggests that the scale is not just a reliable but also a valid instrument for measuring political secularism.

However, there are some limitations. First, the analyses presented in this note rely on surveys that were carried out in Germany only, albeit both in the territory of the former Federal Republic of Germany and the former German Democratic Republic. In future research, they should be complemented by comparative survey data to ascertain that measurements are invariant (Davidov 2009) across cultural, political, and linguistic borders. Ideally, they would be included in large-scale projects such as the European Social Survey.

Second, and relatedly, the item referencing the European Union (EU) treaties should obviously be confined to the EUs 27 member states. In an even broader context, it would have to be adapted, e.g., by instead referring to the national constitution.⁶

Third, while internal consistency is a central aspect of the scale's reliability, it is unrelated to test–retest reliability (McCrae et al. 2011). Political secularism itself should be a relatively stable attitude. If the scale is reliable, measurements of secularism should be relatively stable, too, at least in the short and medium term. Ideally, future surveys should therefore administer the items repeatedly so that we may learn more about the scale's properties.

But ultimately, the usefulness of the scale can only be gauged in applied research: including the scale in rich and substantively interesting models of political attitudes and behavior will yield the true test of nomological validation.