Survey experiments are an important tool to measure policy preferences. Researchers often rely on the random assignment of policy attribute levels to estimate different types of average marginal effects. Yet, researchers are often interested in how respondents trade-off different policy dimensions. We use a conjoint experiment administered to more than 10,000 respondents in Germany, to study preferences over personal freedoms and public welfare during the COVID-19 crisis. Using a pre-registered structural model, we estimate policy ideal points and indifference curves to assess the conditions under which citizens are willing to sacrifice freedoms in the interest of public well-being. We document broad willingness to accept restrictions on rights alongside sharp heterogeneity with respect to vaccination status. The majority of citizens are vaccinated and strongly support limitations on freedoms in response to extreme conditions—especially, when they vaccinated themselves are exempted from these limitations. The unvaccinated minority prefers no restrictions on freedoms regardless of the severity of the pandemic. These policy packages also matter for reported trust in government, in opposite ways for vaccinated and unvaccinated citizens.

Conjoint analysis is widely used for estimating the effects of a large number of treatments on multidimensional decision-making. However, it is this substantive advantage that leads to a statistically undesirable property, multiple hypothesis testing. Existing applications of conjoint analysis except for a few do not correct for the number of hypotheses to be tested, and empirical guidance on the choice of multiple testing correction methods has not been provided. This paper first shows that even when none of the treatments has any effect, the standard analysis pipeline produces at least one statistically significant estimate of average marginal component effects in more than 90% of experimental trials. Then, we conduct a simulation study to compare three well-known methods for multiple testing correction, the Bonferroni correction, the Benjamini–Hochberg procedure, and the adaptive shrinkage (Ash). All three methods are more accurate in recovering the truth than the conventional analysis without correction. Moreover, the Ash method outperforms in avoiding false negatives, while reducing false positives similarly to the other methods. Finally, we show how conclusions drawn from empirical analysis may differ with and without correction by reanalyzing applications on public attitudes toward immigration and partner countries of trade agreements.

For the purposes of farm animal welfare assessment, Farm Assurance Schemes and enforcement of animal welfare legislation, a requirement arises for a unitary welfare score which may be the amalgamation of several animal welfare measures. In amalgamating measures, weighting to reflect the importance of the individual measures for animal welfare is desirable. A study is described in which conjoint analysis was used to collect and evaluate expert opinion to weight a number of welfare assessment measures for the importance of each to broiler welfare in UK husbandry systems. The statistically combined opinion of the experts consulted revealed the weighting factors of the welfare assessment measures selected, with respect to the importance for bird welfare, to be: 0.26 for mortality levels on the growing unit; 0.24 for the level of leg weakness; 0.16 for the level of hock burn; 0.14 for stocking density; 0.10 for enrichment provision; and, 0.10 for the level of emergency provision. Criteria for selection of welfare assessment measures for use in the field, and level of agreement between experts consulted for the study, are discussed. It is concluded that weightings of welfare assessment measures by expert opinion, using conjoint analysis, might be used in the construction of a welfare index for assessment of broiler welfare on-farm. Such an index should not be considered as a ‘gold standard’ for welfare measurement but as an evolving standard for welfare assessment, based on current knowledge.

A large range of variables can affect the welfare of the dairy cow, making it difficult to assess the overall ‘level of welfare’ of the individual animal. Two groups of individuals completed a questionnaire based upon the ‘five freedoms’: 26 respondents had expertise either in the field of dairy cow welfare or as practicing veterinary surgeons, and 30 were veterinary students in their penultimate year of study. Conjoint analysis was used to calculate the average importance scores (AIS) for 34 variables presented to the respondents as 52 ‘model cows’ in the form of grouped questions, phrases and pictures. Conjoint analysis identified the most important factors for each ‘freedom’: access to forage, body condition score, foot conformation, hock lesions, and the encouragement required for a dairy cow to walk into the parlour. There was a significant difference between the expert and student groups for seven out of 34 factors, which may be attributed to individual variation of opinion, knowledge, experience and expectation. The factors were ranked within each ‘freedom’ using the experts' AIS but it was not assumed that each freedom had equal ‘weight’; therefore, the factors within each freedom were compared only with factors within the same freedom. These scores produced a weighting scale, which was applied on-farm, in a preliminary exercise comparing ‘model’ and ‘perceived’ welfare scores.

Standard preference models in consumer research assume that people weigh and add all attributes of the available options to derive a decision, while there is growing evidence for the use of simplifying heuristics. Recently, a greedoid algorithm has been developed (Yee, Dahan, Hauser & Orlin, 2007; Kohli & Jedidi, 2007) to model lexicographic heuristics from preference data. We compare predictive accuracies of the greedoid approach and standard conjoint analysis in an online study with a rating and a ranking task. The lexicographic model derived from the greedoid algorithm was better at predicting ranking compared to rating data, but overall, it achieved lower predictive accuracy for hold-out data than the compensatory model estimated by conjoint analysis. However, a considerable minority of participants was better predicted by lexicographic strategies. We conclude that the new algorithm will not replace standard tools for analyzing preferences, but can boost the study of situational and individual differences in preferential choice processes.

Existing research demonstrates that parents are poorly informed consumers of early childhood education and care (ECEC) services. Choosing such services is a complex process shaped by a combination of logistical limitations (e.g., cost/location), informational barriers and ideas about what the goal of care should be (e.g., education of young children or provision of an environment that feels like home). Experimental studies have also demonstrated that when study participants are informed of the importance of a specific decision, they engage in more complex decision-making. In this article, we test whether providing parents with information about the regulatory stringency of ECEC options available influences their choices regarding ECEC. A conjoint survey designed to capture quasi-behavioural choices for ECEC services was completed by 682 parents. Before engaging with the survey, participants were randomly assigned into either a control group or a treatment group that informed them about the stringency of oversight regarding ECEC options available in the province of Ontario, Canada. Receiving information did not meaningfully change the choices of the entire sample. However, a subgroup analysis revealed an important information effect on parent decisions for lower income/lower-education parents.

In many Latin American countries, social policy preferences among economically vulnerable citizens seem largely unpolarized. However, current studies rarely confront citizens with realistic policy options and often lack the required detail to capture the heterogeneity of economic vulnerability. Drawing on the dualization debate, we expect individuals facing different degrees of vulnerability to show distinct social policy preferences. Using original survey data from Mexico and a conjoint experiment, our findings reveal a complex divide, where the most economically vulnerable are least supportive of public solutions. Sharing the home with a formal labor market participant does not seem to mitigate social policy skepticism among the vulnerable. In contrast, magnified vulnerability via household composition reduces support for welfare policy expansion. Social policy preferences become much less distinct when policy design alternatives are introduced, suggesting reduced expectations about the state’s role and a lack of clarity about the tangible benefits of social policy reform.

Numerous studies have demonstrated that the provision of early childhood education and childcare services (ECEC) is associated with higher women’s participation in the labor market. However, many questions about the causal relationship between the supply of childcare and patterns of female employment remain open. In an effort to overcome common endogeneity problems, we conducted a conjoint experiment in Switzerland, which enables us to analyze mothers’ employment intentions in different – and even in some hypothetical – contexts. Our results demonstrate that improving the provision of ECEC services does affect mothers’ intentions to engage in paid labor. Nevertheless, mothers comprise a heterogeneous group. As expected, ECEC services’ effects are limited for mothers with comparatively high levels of employment. In contrast, mothers with low levels of employment are quite reactive to changing policy contexts, especially if external childcare spots for preschoolers become affordable. Notably, elasticity is present not only in the behavior of women with preferences for supplementary, external childcare, but also in that of women with preferences for parental or home-centered childcare. Our study thus highlights childcare policies’ potential to change the patterns of female employment in contexts marked by persistent traditional gender roles and limited childcare provision.

Although consumers' interest in organic products has increased in recent years, the total demand for these products is still small in most countries. This mismatch between the positive perception of these products and their limited final demand is the so-called intention-behavior gap. The aim of this study is twofold. The first aim is to evaluate the effect of self-reported attitudes toward organic foods in the willingness to pay (WTP) for these products. Second, we analyze the effects of these self-reported attitudes in the final purchasing decision when consumers are asked to evaluate several food attributes simultaneously. The results show that self-reported attitudes toward organic products are useful predictors of higher WTP and can be arranged in different categories (range 15.2–20.1%). However, when a conjoint analysis of different food attributes was conducted, the segment of pro-organic consumers reported that the origin of the product was more important for them than the production system. This opens a new debate about the advisability of promoting the joint use of both labels (organic and origin labels) to engage pro-organic consumers.

Citizens that tend to experience political exclusion are often more supportive of direct and participatory forms of decision-making. We empirically verify two competing explanatory logics for such high support: the “anti-establishment” logic, which expects politically excluded citizens to unconditionally express more support than their fellow citizens for democratic innovations (DIs); and the “instrumental” logic, which expects politically excluded citizens to only express more support for DIs than other citizens when these innovations offer procedural control and favorable outcomes. Based on a conjoint analysis of Dutch citizens' preferences for participatory budgeting, we find no support for the anti-establishment logic and partial support for the instrumental logic. We show how measures of citizens' own feelings of exclusion help to explain the results.

Forced-choice conjoint experiments have become a standard component of the experimental toolbox in political science and sociology. Yet the literature has largely overlooked the fact that conjoint experiments can be used for two distinct purposes: to uncover respondents’ multidimensional preferences, and to estimate the causal effects of some attributes on a profile’s selection probability in a multidimensional choice setting. This paper makes the argument that this distinction is both analytically and practically relevant, because the quantity of interest is contingent on the purpose of the study. The vast majority of social scientists relying on conjoint analyses, including most scholars interested in studying preferences, have adopted the average marginal component effect (AMCE) as their main quantity of interest. The paper shows that the AMCE is neither conceptually nor practically suited to explore respondents’ preferences. Not only is it essentially a causal quantity conceptually at odds with the goal of describing patterns of preferences, but it also does generally not identify preferences, mixing them with compositional effects unrelated to preferences. This paper proposes a novel estimand—the average component preference—designed to explore patterns of preferences, and it presents a method for estimating it.

How can we elicit honest responses in surveys? Conjoint analysis has become a popular tool to address social desirability bias (SDB), or systematic survey misreporting on sensitive topics. However, there has been no direct evidence showing its suitability for this purpose. We propose a novel experimental design to identify conjoint analysis’s ability to mitigate SDB. Specifically, we compare a standard, fully randomized conjoint design against a partially randomized design where only the sensitive attribute is varied between the two profiles in each task. We also include a control condition to remove confounding due to the increased attention to the varying attribute under the partially randomized design. We implement this empirical strategy in two studies on attitudes about environmental conservation and preferences about congressional candidates. In both studies, our estimates indicate that the fully randomized conjoint design could reduce SDB for the average marginal component effect (AMCE) of the sensitive attribute by about two-thirds of the AMCE itself. Although encouraging, we caution that our results are exploratory and exhibit some sensitivity to alternative model specifications, suggesting the need for additional confirmatory evidence based on the proposed design.

The article offers an overview of the use of survey experiments in political research by relying on available examples, bibliographic data and a content analysis of experimental manuscripts published in leading academic journals over the last two decades. After a short primer to the experimental approach, we discuss the development, applications and potential problems to internal and external validity in survey experimentation. The article also provides original examples, contrasting a traditional factorial and a more innovative conjoint design, to show how survey experiments can be used to test theory on relevant political topics. The main challenges and possibilities encountered in envisaging, planning and implementing survey experiments are examined. The article outlines the merits, limits and implications of the use of the experimental method in political research.

Conjoint experiments are quickly gaining popularity as a vehicle for studying multidimensional political preferences. A common way to explore heterogeneity of preferences estimated with conjoint experiments is by estimating average marginal component effects across subgroups. However, this method does not give the researcher the full access to the variation of preferences in the studied populations, as that would require estimating effects on the individual level. Currently, there is no accepted technique to obtain estimates of individual-level preferences from conjoint experiments. The present paper addresses this gap by proposing a procedure to estimate individual preferences as respondent-specific marginal component effects. The proposed strategy does not require any additional assumptions compared to the standard conjoint analysis, although some changes to the task design are recommended. Methods to account for uncertainty in resulting estimates are also discussed. Using the proposed procedure, I partially replicate a conjoint experiment on immigrant admission with recommended design adjustments. Then, I demonstrate how individual marginal component effects can be used to explore distributions of preferences, intercorrelations between different preference dimensions, and relationships of preferences to other variables of interest.

Conjoint analysis has become popular among social scientists for measuring multidimensional preferences. When analyzing such experiments, researchers often focus on the average marginal component effect (AMCE), which represents the causal effect of a single profile attribute while averaging over the remaining attributes. What has been overlooked, however, is the fact that the AMCE critically relies upon the distribution of the other attributes used for the averaging. Although most experiments employ the uniform distribution, which equally weights each profile, both the actual distribution of profiles in the real world and the distribution of theoretical interest are often far from uniform. This mismatch can severely compromise the external validity of conjoint analysis. We empirically demonstrate that estimates of the AMCE can be substantially different when averaging over the target profile distribution instead of uniform. We propose new experimental designs and estimation methods that incorporate substantive knowledge about the profile distribution. We illustrate our methodology through two empirical applications, one using a real-world distribution and the other based on a counterfactual distribution motivated by a theoretical consideration. The proposed methodology is implemented through an open-source software package.