To avoid the extreme consequences of climate change, people need to make collective efforts to reduce greenhouse gas emissions (Gifford, Reference Gifford2011). The present study investigates Dutch residents’ climate concern, preferences for climate policy, interviewer-rated engagement with climate change, and behavior in large face-to-face and online pools collected by the environmental non-profit Milieudefensie.
Public Polling in Environmental Organizations
Describing perspectives over time and among diverse audiences is essential for effective climate change mitigation. Because public support is a substantial driver of political decision-making in participatory democracies (Burstein, Reference Burstein2003), public polling enables policy solutions supported by public opinion (Wlezien & Soroka, Reference Wlezien and Soroka2016). Second, a better understanding of beliefs and behaviors across groups allows the design of more effective communication to increase awareness and behavioral engagement with climate change (Lee et al., Reference Lee, Markowitz, Howe, Ko and Leiserowitz2015; Moser, Reference Moser, Selin and VanDeveer2009; Roser-Renouf et al., Reference Roser-Renouf, Stenhouse, Rolfe-Redding, Maibach, Leiserowitz, Hanson and Cox2015).
Social scientists studying public opinion on climate change and policy preferences can use large, existing international datasets like the World Values Survey, International Social Survey, Eurobarometer, or Latinobarometer, which increasingly include items on environmental issues (Prakash & Bernauer, Reference Prakash and Bernauer2020). Moreover, they can collaborate with environmental organizations that conduct surveys to gather information or engage their audiences with climate change. The second author was a student volunteer at the Dutch environmental non-profit Milieudefensie.
Milieudefensie conducted the public poll ‘Operatie Klimaat’ (Operation Climate) in 2019–2020. Milieudefensie volunteers interviewed 3,000 Dutch residents face-to-face and recruited a larger sample for an online survey from their website and email newsletter. Both the online and face-to-face sample were convenience samples and likely represent a more concerned and engaged segment than the general Dutch public based on their willingness to participate in surveys of an environmental group. Therefore, it is helpful to explore the potential selection bias of this group to inform future research collaborations with organizations such as Milieudefensie. Our first aim was to explore whether attitudes and their association with behavioral engagement were similar to previous studies (Aim 1). This can help inform how to interpret findings from such convenience samples and integrate them into the scientific record.
Moreover, this effortful face-to-face sampling is unique because online surveys are increasingly selected for describing environmental attitudes and behavior (Prakash & Bernauer, Reference Prakash and Bernauer2020). Face-to-face interviews have several advantages over online surveys: reduced non-response, more control of the data collection process, the possibility of clarifying content to the survey taker, and the inclusion of social cues (Doyle, Reference Doyle, Balakrishnan, Colton, Everitt, Piegorsch, Ruggeri and Teugels2014). Advantages of online surveys over face-to-face interviews include more anonymity, broader geographic accessibility, and less interviewer bias (Evans & Mathur, Reference Evans and Mathur2018). Relying on different surveying modes may thus influence results and lead researchers to draw different conclusions (Ansolabehere & Schaffner, Reference Ansolabehere and Schaffner2011). Therefore, contrasting outcomes across surveying methods in similar samples and periods can inform cost-benefit analyses of techniques (Ansolabehere & Schaffner, Reference Ansolabehere and Schaffner2011). For instance, if climate concern were similar for a given population between sampling modes, it might be more efficient and justifiable to use online over costly face-to-face interviews. Therefore, our second aim was to compare attitudes between the online and face-to-face samples (Aim 2).
Climate Concern
We define environmental concern as how much individuals are aware of environmental problems and support efforts to solve them (Dunlap & Jones, Reference Dunlap and Jones2002). In the Netherlands, most residents (74%, European Commission, 2019; 76%, Kloosterman et al., Reference Kloosterman, Akkermans, Reep, Wingen, Veld and van Beuningen2021) saw climate change as a very serious problem, slightly below the EU average (European Commission, 2019). 27% of Dutch residents thought climate change was the single most serious problem the world was facing, slightly above the EU average (23%, European Commission, 2019), and 69% expressed strong concern (Ipsos, 2021). Moreover, 85% found it important that the government focuses on climate policy and 42% said that current policies were insufficient to address the climate crisis (Kloosterman et al., Reference Kloosterman, Akkermans, Reep, Wingen, Veld and van Beuningen2021). The majority supported the development of renewable energy sources, with solar (83%) and wind (72%) as the most popular options (Kloosterman et al., Reference Kloosterman, Akkermans, Reep, Wingen, Veld and van Beuningen2021).
These results suggest that Dutch residents increasingly see the climate crisis as an emergency, consistent with the scientific consensus. A radical and widespread change in behavior towards a system of lower production and consumption is needed (Lorenzoni et al., Reference Lorenzoni, Nicholson-Cole and Whitmarsh2007; Steg & Vlek, Reference Steg and Vlek2009). One domain of behavior change is consumer behaviors that reduce environmental impact.
Environmental concern does not necessarily lead to pro-environmental behavior (Tam & Chan, Reference Tam and Chan2017). Climate change is perceived as a slow, distant, and abstract threat, unlike those humans have evolved to understand and act upon (Gifford, Reference Gifford2011). Several psychological barriers to action, such as limited cognition, feelings of helplessness, and social norms, undermine the reliance on individual action and obstruct climate change mitigation efforts (Gifford et al., Reference Gifford, Kormos and McIntyre2011; Lorenzoni et al., Reference Lorenzoni, Nicholson-Cole and Whitmarsh2007). Moreover, behavior not only emerges from thoughts and intentions but is also a product of social and institutional contexts (Lorenzoni et al., Reference Lorenzoni, Nicholson-Cole and Whitmarsh2007).
Climate Activism
Besides psychological barriers to behavior, the climate crisis is structurally anchored in industrialized ways of life (Jensen & Schnack, Reference Jensen and Schnack1997). Individuals are therefore incapable of attaining sufficient emissions reductions by themselves (Ockwell et al., Reference Ockwell, Whitmarsh and O’Neill2009). Environmental policy and regulation are inevitable for facilitating individual behavior change through large-scale solutions (Lorenzoni et al., Reference Lorenzoni, Nicholson-Cole and Whitmarsh2007; Nielsen et al., Reference Nielsen, Clayton, Stern, Dietz, Capstick and Whitmarsh2021). Besides voting, people can pressure governments through activism: intentional behaviors aimed at a collective and political system change (Alisat & Riemer, Reference Alisat and Riemer2015; Roser-Renouf et al., Reference Roser-Renouf, Maibach, Leiserowitz and Zhao2014). Current governmental efforts are highly inadequate to address the climate crisis, and continued inaction will irreversibly damage the biosphere and exacerbate global inequalities (Hagedorn et al., Reference Hagedorn, Loew, Seneviratne, Lucht, Beck, Hesse, Knutti, Quaschning, Schleimer, Mattauch, Breyer, Hübener, Kirchengast, Chodura, Clausen, Creutzig, Darbi, Daub, Ekardt and Zens2019). Given that the governments are unlikely to act without public pressure, climate activism is more effective than consumer behaviors, like recycling, for achieving emission reductions (Ockwell et al., Reference Ockwell, Whitmarsh and O’Neill2009).
Climate activism can range from more participatory citizenship behaviors like informing oneself about an issue and engaging in conversations with friends to political leadership actions like organizing a protest (Alisat & Riemer, Reference Alisat and Riemer2015).
The more people were worried about climate change, the stronger their preferences for climate policy (Bouman et al., Reference Bouman, Verschoor, Albers, Böhm, Fisher, Poortinga, Whitmarsh and Steg2020; Rauwald & Moore, Reference Rauwald and Moore2002). Similarly, higher environmental concern predicted emphasizing environmental consequences when evaluating trade policy (Bechtel et al., Reference Bechtel, Bernauer and Meyer2012). However, according to the Social Identity Model of Collective Action (Thomas et al., Reference Thomas, Mavor and McGarty2012; van Zomeren et al., Reference van Zomeren, Postmes and Spears2008) and the Encapsulated Model of Social Identity in Collective Action (Thomas et al., Reference Thomas, Mavor and McGarty2012), key predictors of environmental activism are not concern and policy preferences but identifying with an environmental group (social identity), experiencing emotional responses to injustice (i.e., anger and moral outrage), and believing in the effectiveness of group efforts (collective efficacy). Therefore, in line with the value-action gap (Tam & Chan, Reference Tam and Chan2017), in the Netherlands, we expect weak positive associations between both climate concern and preference for climate policy and interviewer-rated engagement with climate change (Hypothesis 1a,b) (face-to-face sample), and preference for climate policy and behavior (providing email and phone number in the online sample) (Hypothesis 1c).
Rural-Urban Differences
Demographic characteristics, like age, gender, and urbanity, predict beliefs and behaviors about climate change (Gifford & Nilsson, Reference Gifford and Nilsson2014; Wolf & Moser, Reference Wolf and Moser2011). As an indicator of urbanity, Milieudefensie recorded postcodes in the face-to-face sample. There are mixed findings regarding rural-urban differences in environmental concern (Ergun et al., Reference Ergun, Khan and Rivas2021). Several studies suggested that rural residents were less concerned about environmental problems than urban residents (Tremblay & Dunlap, Reference Tremblay and Dunlap1978; Yu, Reference Yu2014), possibly because they perceived a higher dependence on natural resources (Lowe & Pinhey, Reference Lowe and Pinhey1982). However, these differences may also be explained by lower education and income in rural areas (Huddart-Kennedy et al., Reference Huddart-Kennedy, Beckley, McFarlane and Nadeau2009). When controlling for socio-economic variables, there were no rural-urban differences in China (Liu et al., Reference Liu, Hao, Portney and Liu2020) and in Pakistan rural residents were more environmentally concerned than urban residents (Ergun et al., Reference Ergun, Khan and Rivas2021). Moreover, rural residents in Poland (Piekarski et al., Reference Piekarski, Stoma, Dudziak, Andrejko and Ślaska-Grzywna2016) and Spain (Berenguer et al., Reference Berenguer, Corraliza and Martín2005) reported concerns about local environmental problems like water shortages, while urban residents reported abstract environmental concerns like climate change. People living in rural areas may hold more anthropocentric environmental concerns (protect the environment to fulfill human needs), while urban residents may have more ecocentric concerns (protect the environment for its own sake) (Gagnon Thompson & Barton, Reference Gagnon Thompson and Barton1994; Huddart-Kennedy et al., Reference Huddart-Kennedy, Beckley, McFarlane and Nadeau2009; Rauwald & Moore, Reference Rauwald and Moore2002). Because of these mixed results, we will compare climate concern in rural versus urban residents in the Netherlands without a strong prediction.
Urban residents in the United States were more supportive of climate action than rural residents, controlling for partisanship and other demographics (Bonnie et al., Reference Bonnie, Diamond and Rowe2020). Moreover, urban residents in Scotland preferred renewable energy projects with low impacts on landscape, wildlife, and air pollution, but job creation was the most important factor for rural residents (Bergmann et al., Reference Bergmann, Colombo and Hanley2008). Rural residents may oppose measures less considerate of rural lifestyles due to their greater reliance on cars (Otto & Gugushvili, Reference Otto and Gugushvili2020). Therefore, we expect that rural residents in the Netherlands will be less supportive of climate mitigation policy than urban residents (Hypothesis 2a).
If rural residents disfavor climate policy compared to urban residents, they may also be less willing to engage in climate activism. However, social identity, collective efficacy, and group-based emotions were stronger predictors of activism behaviors than environmental attitudes (review: van Zomeren et al., Reference van Zomeren, Postmes and Spears2008). Because rural residents in the United States reported less identification with environmentalists than urban residents (Brick & Lai, Reference Brick and Lai2018), we expect that rural residents in the Netherlands will be rated as less engaged with climate change than urban residents (Hypothesis 2b).
Method
Operation Climate
Milieudefensie conducted the ‘Operation Climate’ poll in Dutch residents from September 2019 to March 2020. Their process was based on the principles of Big Organizing, which guides large-scale and decentralized activism (Bond & Exley, Reference Bond and Exley2016). The goal of Operation Climate was to focus the 2021 Dutch elections on climate justice, to raise public awareness, and to create a widespread movement of engaged citizens. Around 200 volunteers participated in groups across the Netherlands. Most groups were managed by a city organizer who coordinated the campaign in a town or region. City organizers worked together with office organizers who coordinated the campaign across the Netherlands.
These 200 interviewers were recruited in different ways: Milieudefensie contacted them by phone or email because they had previously volunteered, joined a local meetup, participated in a survey, or other methods (e.g., through friends). Upon registration as a volunteer, a city or office organizer called them to discuss the campaign’s procedures and objectives. In larger cities, volunteers joined local groups. In smaller towns, volunteers joined the nearest local group or set up a new one.
Respondents
In total, 3,102 Dutch residents were interviewed face-to-face and 30,311 people answered a similar survey online. Participation was voluntary and responses were anonymized apart from postal codes in the face-to-face sample. Contact information (email addresses and telephone numbers) was collected and either deleted from the datasets (face-to-face sample) or anonymized (online sample) by Milieudefensie. Other respondent characteristics like age or gender were not recorded, which limits the comparability between the samples and generalizability to the broader Dutch population.
Conversation Procedure (Face-to-Face Interviews)
Interviews took place in residential areas (interviewers went door-to-door in neighborhoods) and in public spaces like train stations. Respondents in public spaces and households were selected based on convenience (e.g., proximity to the interviewing team). Usually, volunteers went in pairs, with one volunteer asking the questions and the other one entering participant answers through an online application. When ringing a door, a volunteer entered the postal code and sometimes the specific address. If residents agreed to participate, there was no fixed framework for opening a conversation. Typically, volunteers introduced themselves and the organization, and asked for five minutes of the respondent’s time to talk about climate change and policy solutions. There were four other outcomes: The address did not exist, nobody was home, volunteers were asked to come back later, or residents did not want to participate. After the conversation, volunteers thanked respondents for their participation and explained how the data would be used. Participants could enter their email addresses if they wanted to stay updated on the campaign or sign-up as future volunteers. At the end of the conversation, the interviewers rated the participant’s engagement with climate change.
Respondent Recruitment (Online Survey)
The online survey was available on the Milieudefensie website and advertised through their social media channels (Facebook and Instagram) and newsletter. Occasionally, volunteers handed out a note with a prompt to the online survey if people did not have time for a face-to-face interview on the streets or left these notes in mailboxes if people were not at home.
Materials
Interviewers entered data into an online application by Milieudefensie and the online respondents filled in a survey on the Milieudefensie website (English translation in the Supplemental Material). The face-to-face and online surveys were not identical: The face-to-face survey collected items on urbanity, climate concern, preferences for climate policy, and interviewer-rated engagement with climate change, while the online survey included items on preference for climate policy and objective measures of behavior (whether the interviewee provided their email and phone number to the organization). The authors were not involved in the survey design and had therefore no influence on which items were selected or the content of the policies.
Climate concern (quantitative, face-to-face survey only). To measure climate concern, the item “Are you worried about climate change?” was answered on a three-point scale of no (–1), slightly (0), yes (1).
Climate concern (qualitative, face-to-face survey only). To measure the contents of climate concern, interviewers recorded what participants’ responses to: “If yes, what are you particularly worried about? If no, why not?”.
Preference for climate policy (quantitative). To measure preference for climate policy, several items were rated on a scale from completely disagree (1) to completely agree (5), or don’t know. An example item was “I think it is more important that the government puts money in good public transport than in motorways.” The original surveys included six items (online sample) or seven items (face-to-face sample). However, in the online sample, some items were adjusted between the 2019 to the 2020 versions based on feedback on item clarity. This resulted in differences in content and phrasing between the 2019 and 2020 versions. Therefore, only the five items that were present in both versions of the online sample were analyzed here. During the write-up of this research, Milieudefensie also told the authors that some policy items in the face-to-face sample may have been slightly adjusted in phrasing and unfortunately these minor changes were not recorded. Last, several items differed between the face-to-face and online samples. To assess comparability between the two scales, exploratory factor analyses were conducted (see Supplementary Tables S1–S3 for fit statistics and factor loadings). One-factor solutions were adequate for both scales in the face-to-face and online sample after dropping one item (“prevent losses for low-income households”) from both scales. The remaining items (k = 6 for the face-to-face data and k = 4 for the online data) were adapted into composite scales, with acceptable internal consistency both face-to-face (α = .67) and online (α = .67). However, due to the differences described above, the comparisons of the policy composites should be interpreted with caution.
Beliefs about climate policy (qualitative). To assess beliefs about climate policy, three items were used: “How can climate policy become fairer?” (both samples), “If we want to stop further climate change, we must invest in green solutions. That costs money. Who do you think should pay for this? Why or why not?” (face-to-face sample), and “Oil, coal, and gas cause climate change. That is why the Netherlands must switch to sustainable energy. Nevertheless, the Netherlands gives a subsidy of 7.6 billion euros to oil, coal, and gas. Did you know? What do you think of that?” (face-to-face sample). We did not analyze these open-ended items.
Behavior (objective, online survey). To assess behavior towards climate action, two questions were asked: “Can we keep you informed by email?” and “Could we get your telephone number (optional)?”, scored either no (0) or yes (1).
Engagement with climate change (face-to-face interviewer rating). After the interaction, interviewers privately recorded their perception of the person’s engagement (“How engaged with climate change was the person you talked to?”) from not engaged with climate change (0) to engaged with climate change (4). Because of the timing of these ratings, they were likely influenced by two questions just asked: “We visit as many Dutch people as possible. Do you want to be kept informed of our work?” and “Would you like to help with going from door to door and having conversations yourself?”, both scored either no (0) or yes (1). Due to the privacy concerns of Milieudefensie, these items were not connected to the other survey items and therefore cannot be used for testing relationships with other variables.
Urbanity. Participant postal codes were used to calculate urbanity scores from data by Centraal Bureau voor de Statistiek (CBS, Statistics Netherlands) (see Supplemental Material). CBS categorizes postal codes from very urban: > 2,500 addresses per km2 (1) to not urban: < 500 addresses per km2 (5).
Analytic Plan
For the current study, we performed mostly quantitative and descriptive analyses (data and code are available at https://osf.io/kfhu8.
Climate concern, preferences for climate policy, interviewer-rated engagement with climate change, behavior (providing email and phone number), and differences between urban and rural residents were analyzed in RStudio (Version 2022.2.3.492, RStudio Team, 2022). Exploratory analyses of qualitative data using thematic coding are given in the Supplemental Material (see Table S7 for key themes of climate concern). The only inferential statistics are correlations. Power was computed with a sensitivity analysis in the R package pwr (Champely et al., Reference Champely, Ekstrom, Dalgaard, Gill, Weibelzahl, Anandkumar, Ford, Volcic and De Rosario2017) for two-tailed correlations, alpha = .05, power = .80, and the smallest cell used in the correlations (N = 1,956), and this revealed 80% power to detect effect sizes of r = .06 or larger.
Results
Descriptives
Climate concern (Figure 1) and preference for climate policy were high (Table 1a and 1b, Figure 2a and 2b). Interviewer-rated engagement with climate change was high (Figure 3a). Half of online respondents signed up for an email newsletter and 7% provided their phone number (Figure 3b). In the face-to-face sample, 2,560 participants provided their post codes, and most were urban (M = 1.71, SD = 1.01, with 1 indicating high urbanity). Mean urbanity in the Netherlands is 2.77 (Centraal Bureau voor de Statistiek [CBS], 2021).
Table 1a. Descriptive Statistics and Correlations (Face-to-Face Sample, Ns = 1,956 to 2,996)
Note. Kendall’s tau-b (τb) correlation significance (2-tailed): * p < .05. *** p < .001.
Table 1b. Correlations (Online Sample, Ns = 12,305 to 29,796)
Note. Kendall’s tau-b (τb) correlation significance (2-tailed): *** p < .001.
Figure 1. Histogram of Climate Concern in The Face-To-Face Sample (N = 3,102)
Figure 2a. Item-level Descriptives: Preference for Climate Policy In The Face-To-Face Sample (N = 2,846 to 2,959)
Figure 2b. Item-level Descriptives: Preference for Climate Policy In The Online Sample (N = 30,081–30,164)
Figure 3a. Histogram of Interviewer-Rated Engagement with Climate Change In The Face-to-Face Sample (N = 3,102)
Figure 3b. Frequency of Providing Email and Phone Number in The Online Sample (N = 30,311)
Note. Phone numbers were only collected for a subset of the online respondents. This decision was made by the partner non-profit (missing N = 17,779)
Correlational Analyses
As all variables were non-normally distributed, non-parametric tests were performed. Kendall’s tau-b (τb) correlations revealed associations between climate concern and preference for climate policy with engagement with climate change (face-to-face sample, Hypothesis 1a,b), preference for climate policy and behavior (providing email and phone number) (online sample, Hypothesis 1c), and between urbanity and climate concern, preference for climate policy, and engagement with climate change (face-to-face sample) (Hypothesis 2a,b). See Table 1a and 1b for correlation coefficients. To determine effect sizes (Cohen, Reference Cohen2013), τb was transformed to Pearson R (r) (Kendall, Reference Kendall1970).
Exploratory Analyses
Wilcoxon rank sum tests revealed mean differences between the face-to-face (Ns = 2,389 to 3,102) and online samples (Ns = 12,532 to 30,311). The assumption of equality of variances was met based on visual inspection. Face-to-face respondents (M = 4.33, SD = 0.60) reported a higher preference for climate policy than online respondents (M = 4.14, SD = 0.84), z = 8.34, p < .001, yet this difference was negligible, r(32,183) = .05.
Robustness Checks
To test whether two key analytic decisions influenced the results, three robustness checks are reported in the Supplemental Material (Tables S4–S6). We assessed removing “prevent losses for low-income households” from the preference for climate policy-scale, and also not imputing missing data. The robustness tests revealed that all results for both main and exploratory analyses were highly similar.
Discussion
The Dutch environmental non-profit Milieudefensie conducted a study on climate change attitudes and engagement both face-to-face and online in 2019/2020. The aim of this study was to test whether previous findings held in large-scale, face-to-face and online studies by outside groups. We also explored differences in preference for climate policy between the face-to-face and online samples.
The majority of face-to-face respondents (73%) were highly concerned about climate change and strongly supported climate policy. Online and face-to-face respondents widely agreed with strategies to reduce emissions, like investing in circular agriculture or public transport, and disagreed with hindering practices, like not requiring corporations to contribute to climate solutions. Interviewers rated the majority of face-to-face respondents as engaged with climate change (73%). In the online sample, 7% provided their phone number to Milieudefensie, and half signed up for an email newsletter. These findings align with recent reports of high climate concern in the Netherlands (European Commission, 2019; Ipsos, 2021; Kloosterman et al., Reference Kloosterman, Akkermans, Reep, Wingen, Veld and van Beuningen2021). Policy support was somewhat higher than recent reports of the Netherlands (Kloosterman et al., Reference Kloosterman, Akkermans, Reep, Wingen, Veld and van Beuningen2021) and Europe (Kácha et al., Reference Kácha, Vintr and Brick2022), which suggests the current samples might have been biased by willingness to participate in polls by an environmental group.
Based on the value-action gap (Tam & Chan, Reference Tam and Chan2017) and previous work identifying environmentalist identities, group-based emotions, and collective efficacy as key predictors of climate action (Thomas et al., Reference Thomas, Mavor and McGarty2012; van Zomeren et al., Reference van Zomeren, Postmes and Spears2008), we expected weak positive associations between both climate concern and preference for climate policy and interviewer-rated engagement with climate change (Hypothesis 1a,b) as well as between policy preferences and behavior (providing email and phone number) (Hypothesis 1c).
Higher climate concern and preference for climate policy strongly predicted interviewer-rated engagement with climate change. These associations are likely inflated because the observer ratings were not solely based on respondents’ willingness to become engaged with the movement but also on having just heard the participant’s perceived concerns and policy preferences. Unfortunately, objective behavioral measures (willingness to stay informed and signing up as a volunteer) were not connected by Milieudefensie in the face-to-face data and could therefore not be predicted from the other variables. In the online sample, more preference for climate policy moderately predicted email newsletter sign-up and weakly predicted whether participants provided their phone number to movement organizers, which aligns with prior findings (e.g., Thomas et al., Reference Thomas, Mavor and McGarty2012; van Zomeren et al., Reference van Zomeren, Postmes and Spears2008).
Contrary to expectations (Hypothesis 2a,b), higher urbanity did not predict preference for climate policy and interviewer-rated engagement with climate change in face-to-face respondents. On the one hand, this may be due to the high urbanization and availability of infrastructure, like public transport, in the Netherlands. On the other hand, the underrepresentation of rural residents in the sample may have concealed rural-urban differences.
Urbanity did not predict climate concern either, which aligns with previous mixed results (Ergun et al., Reference Ergun, Khan and Rivas2021; Gifford & Nilsson, Reference Gifford and Nilsson2014), but this may also be due to the underrepresentation of rural residents in the sample. Future polls by environmental organizations can strive for more representative samples of the Dutch public, include highly urban and highly rural respondents, and compare the concerns of urban and rural residents (Gagnon Thompson & Barton, Reference Gagnon Thompson and Barton1994; Huddart-Kennedy et al., Reference Huddart-Kennedy, Beckley, McFarlane and Nadeau2009; Rauwald & Moore, Reference Rauwald and Moore2002).
Exploratory Findings
Preference for climate policy did not differ between the face-to-face and online samples. This was surprising as one might expect more socially desirable reporting during in-person interviews than in online surveys (Evans & Mathur, Reference Evans and Mathur2018). In any case, the policy preferences need to be interpreted with caution because the items were not identical between the two samples. To inform cost-benefit analyses of both techniques, future research can test this using identical items in similar populations and timeframes (Ansolabehere & Schaffner, Reference Ansolabehere and Schaffner2011).
Limitations
This study had large sample sizes, included objective behaviors (providing email and phone number), and had a rare face-to-face sample. Yet, except for residence (Netherlands) and urbanity (face-to-face sample only), no demographics were collected, which limits the comparability between the two samples and the generalizability. We suspect that the sample represents a more concerned and engaged segment than the general Dutch public because participation depended on willingness to participate in the survey of an environmental group.
Second, although factor analysis on the policy preference items yielded one-factor solutions, the phrasing of some face-to-face survey items may have been changed by Milieudefensie over time, and these changes were not recorded, which makes it difficult to interpret the results of the policy items. Third, climate concern was assessed by a single item rather than multiple items. Fourth, self-reports are susceptible to social desirability and poor introspection (Brewis, Reference Brewis2014; Demetriou et al., Reference Demetriou, Ozer, Essau, Cautin and Lilienfeld2015). Fifth, interviewer ratings are not objective measures of behavior (Lange & Dewitte, Reference Lange and Dewitte2019). Last, data were collected by over 200 volunteers, who likely did not interact with respondents identically. Respondents were interviewed by one or two interviewers, but which interviewers was not recorded. Interviewer bias can be assessed through multilevel modeling in future studies (Hox, Reference Hox1994) and using independent ratings of two or more interviewers, which allows for calculating interrater reliability of the ratings.
Future Avenues
Partnering with environmental organizations can give social psychologists access to effortful survey data, like face-to-face interviews or phone calls, and help shape their research in ways that are helpful to environmental groups. However, like in the current project, researchers often have low control over study design and data collection, which can reduce the validity and reliability of the results. Building long-term relationships with environmental organizations may enable scientists to become involved earlier in the research cycle.
Moreover, these partnerships enable researchers to observe behaviors more objectively than self-reports of behaviors or intentions. In the current study, behavior was measured by whether people provided their email address or phone number to become engaged with Milieudefensie. However, climate activism is a diverse category, and people may have different expectations of what engagement with an environmental organization would require, e.g., informing oneself about an issue versus organizing a climate march (Alisat & Riemer, Reference Alisat and Riemer2015).
Future research with environmental groups can map the range of possible and helpful behaviors in a given context and study psychological (e.g., policy preferences or environmental identity), behavioral (e.g., previous engagement with an environmental organization), and structural (e.g., income or urbanity) predictors. Such a systematic understanding of climate activism could help design and test targeting and communication strategies and help environmental organizations attract and engage more people.
Rare face-to-face interviews and online polls by the environmental group Milieudefensie revealed high concern, preference for climate policy, and interviewer-rated engagement with climate change in the Netherlands in 2019/20. Providing contact details to engage in climate activism was rare to uncommon. The results provide convergent evidence to conventional online surveys, and these Dutch residents appear slightly more engaged with systemic change to mitigate climate change than in previous polls.
Supplementary Materials
To view supplementary material for this article, please visit http://doi.org/10.1017/SJP.2023.3.
Open access
