Social Preferences: Measuring Private, Public, and Group Preferences through Focus Groups

Abstract

How does the social nature of focus groups shape what researchers learn about preferences? This article delineates three preference types—private, publicly expressed, and group preferences—and introduces a new method for measuring each in a focus group setting. Original data on people’s preferences for punishing rape, wife beating, and theft across 80 focus groups in 20 villages in eastern Democratic Republic of Congo reveal clear differences across preference types, featuring more extreme punishment preferences in the public sphere. A within-subject experiment also shows that focus group discussions affect people’s private preferences by making them more extreme, which has ethical implications for researchers who use focus groups worldwide. The social nature of preferences observed in Democratic Republic of Congo underscores that scholars must adopt clear and transparent approaches to data comparison to learn about sensitive issues in the face of contested norms.

Scholars characterize focus groups as a useful method to ground studies in context because they provide insights into how people think about research questions at hand. Focus groups harness the form and content of conversations among participants in a group discussion to reveal socially informed truths. We know from scholarship on group dynamics and norms that the preferences that people express in group contexts can substantially differ from those they privately hold (Kuran 1997). Despite viewing and analyzing silences, hesitations, and dominance in focus group conversations (Fujii 2017), questions still remain about how the public nature of focus groups affects what researchers learn (Morgan 1996, 136). Even less explored are the ways by which group discussions affect people’s preferences (Zorn et al. 2006, 123), which have important ethical implications for researchers who use focus groups worldwide.

This article delineates three types of preferences—private, publicly expressed, and group preferences—to inquire into and empirically trace the operation of norms and group dynamics within focus groups. In doing so, this article describes how the group nature of focus groups affects the responses that researchers hear. While all preference types are socially constituted and worthy of investigation, delineation and comparison across the three types clarify contributions of focus group data (as compared to more privately collected data) and refine scholarly approaches to data triangulation (Cyr 2019).

I introduce a new method to measure private, public, and group preferences in an inclusive focus group setting. Together with an invaluable team of local researchers, I use the new method to learn about preferences for punishing three locally perpetrated (hypothetical) crimes across 80 focus groups in 20 villages in eastern Democratic Republic of Congo (DR Congo). The large number of focus groups allows for quantitative assessment of hypotheses alongside qualitative work. The method is unique in its ability to measure private preferences within the focus group setting without relying on participant literacy or numeracy. It also allows for consistent presentation of hypothetical scenarios in a controlled setting for measures of private, public, and group preferences across participants of each focus group.

DR Congo is a conflict-affected state with weak rule of law and highly localized dynamics of responding to within-community disputes. Since the onset of armed violence in the 1990s, there has been significant international attention to violence against women and efforts to mitigate it in DR Congo (Autesserre 2012; Baaz and Stern 2013). Yet, little is known about people’s preferences for punishment. I ask questions to uncover private, public, and group preferences for punishing two physical forms of violence against women that have been the focus of many advocacy interventions: rape and wife beating. I also ask questions about theft, a less sensitive and less gender-targeted crime. While substantive findings are tied to eastern DR Congo, the underlying theory of social preferences suggests broad applicability since generally derived.

Analysis of the quantitative focus group data shows that public and group preferences are measurably distinct from private preferences and are often more extreme, underscoring the relevance of theorizing and measuring preferences across multiple dimensions. The findings show that these differences are consistent across the three crimes, suggesting some generalizability in how preferences for punishment are shaped by the public sphere beyond these three crimes.

A within-subject experiment shows that people update their private preferences to be closer to expressed (more extreme) group norms and that there is more consensus around these more extreme forms of punishment after discussions about each crime. These findings have mixed implications, because the effects of focus groups depend on the norms expressed within them. There are positive implications for well-designed norm-change interventions that seek to alter preferences through carefully crafted messages. ¹ Yet, evidence also suggests that harmful norms may become further entrenched in focus groups through group dynamics. Thus, the onus is on researchers to think about and to address the ethical implications of engaging in focus group discussions ex ante. ²

By examining preferences for punishing everyday crimes, this article engages in the important discussion of tolerance for violence against women at a time when there is significant attention to decreasing tolerance for such violent crimes (as in the “Me Too” movement). It pushes readers to consider the social dimensions of preferences in this research area, as well as in their own studies. It also introduces the framework of private and public preference dimensions beyond the Western context, where focus group research is widespread but largely unquestioned. Most broadly, through examining the effects of focus groups on preferences, this article provides insights into the social nature of preference formation; specifically, how existing attitudes emerge and change.

A Social Model of Preferences

Researchers often take the average preference derived from private survey responses from individuals to reflect a population’s preference, noting some standard deviation. Yet, we know that the average preference does not necessarily reflect a community’s preferences, because averaging is only one of many ways in which preferences may aggregate when people act as groups (Austen-Smith and Banks 1996). Studies of deliberation show that—far from coming to moderate, average preferences—group decisions or preferences can be highly polarized aggregations of their component parts (Roux and Sobel 2015).

Focus groups are designed to overcome bias toward the private sphere in survey research and learn directly from social processes by engaging with people as groups (Wilkinson 1998, 120). A focus group is defined broadly as “a research technique that collects data through group interaction on a topic determined by the researcher” (Morgan 1996, 130). Discussion questions can take more or less structured forms. ³ Interaction among focus group participants is thought to elicit insights that might otherwise be inaccessible (Cyr 2019; Liamputtong 2011).

In observing interactions within focus groups, researchers can note who is silent, who contributes, who overpowers others, and who makes whom uncomfortable—and then ask why (Fujii 2017). Yet, researchers tend to analyze and present interactively generated data in ways remarkably similar to more privately collected forms (Liamputtong 2011, 12; Wilkinson 1998). In a meta-analysis of recent focus group research in high-impact journals, Cyr (2015) finds that analyses treating focus group data as a sum of individual viewpoints are most common, whereas interactive data (insights from back-and-forth exchanges across participants) are least analyzed.

Few studies have been designed to investigate whether and how people’s private views may (or may not) differ from those that they express in focus groups. ⁴ Rather, scholars tend to either assume that focus group data are comparable to survey data and triangulate their findings across data types, or they consider focus group interactions as a given, unique form of data not directly comparable to other forms of data (Cyr 2019; Morgan 1996, 136; Zorn et al. 2006). The fact that focus group data tend to be largely qualitative (and often using convenience samples) while survey data tend to be quantitative (and often using representative samples) also leaves comparisons open to criticism about what qualifies as similarity or difference. ⁵ However, scholars more recently have begun to incorporate a mixed-methods approach to focus groups (Shek 2017).

Several studies have implemented pre- and post-focus group questionnaires to assess changes in attitudes. Zorn and colleagues (2006) find that one of two measured attitudes changed over the course of focus group discussions, but they focus on how increased self-efficacy emerges from discussions rather than preference change. In a study of data quality, Wutich and colleagues (2010) find that focus groups elicit more information about sensitive issues than a pre-focus group self-administered questionnaire. Yet, the study does not explicitly examine preference change. Both studies also take place in developed countries. Scholars know even less about differences between private and public preferences in lower-literacy, lower-technology parts of the world where self-administered questionnaires are not readily applied.

In summary, studies that use focus groups remain inattentive to the private dimension of participant preferences and preference change. This leaves important ethical questions about the effects of focus groups on the private sphere unanswered—and unanswerable. It is necessary to differentiate private and public preferences in theory and in measurement to learn about a focus group’s effects.

This study takes the social nature of preferences seriously, exploring how the public shapes the private sphere and the private shapes the public sphere in a focus group setting—using the same form of measurement across all preference types. To facilitate this examination, I categorize preferences into three types: private, publicly expressed, and group preferences. None of these preferences are “true” original preferences, nor are they exogenous from social context; I take private preferences as a starting point furthest from the social sphere.

I introduce a three-step process model that draws from literature on social dynamics to describe important reasons why these three types of preferences may differ as we move between them. The model also explores a circular pathway by which norms change as a result of focus group dynamics, contributing to studies on the evolution of norms.

As depicted in Figure 1, the process model begins with an individual’s private preference. It then asks how that original preference adapts to the social world by describing how it is related to the preference that an individual is willing to share with others. This is the first arrow in the model (private to public). Then, the process moves from public expression of a preference to a group decision that reflects shared preferences (public to group). Finally, the model examines how group norms, as expressed in the group decision, affect private preferences, coming full circle (group to post-discussion private).

Figure 1 Process Model

There are additional pathways relevant to this model that are not fully explored here. It may be the public expression of preferences (Step 2) that drives private preference change, rather than the expression of group preferences (Step 3). For example, an influential person within a group may be more pivotal to driving private preference change (Step 1) than a larger group consensus (Step 3). Additionally, private preferences may influence group preferences (Step 3) directly rather than through the public preference channel (Step 2). The process model provides a framework for thinking about how these preferences not only differ but also change. However, the circular process does not describe all potential pathways of how the private sphere affects the social sphere and vice versa.

Private Preferences

It is the contention of this article that no measure of preference is completely private or asocial. Once a preference is disclosed to someone (including to a researcher), some element of it becomes social. Yet, many researchers seek to approximate private preferences in their research to the fullest extent possible.

By implementing a survey experiment in Kenya, Cloward (2016) shows that, even in private surveys, the preferences that people reveal can be highly dependent on who respondents believe will receive the information. If respondents think that an international aid donor will receive the information, their responses are more likely to conform to international donor expectations; if respondents think that their local community will receive the data, their responses will be more likely to conform to local expectations. This shows that even responses to private survey questions are not viewed as private but are informed by respondents’ understanding of an audience or audiences.

How one perceives privacy when taking a survey has also been shown to affect the responses that one gives. In a study that varies levels of privacy to elicit sensitive information, Scacco (2010) finds that respondents disclose more engagement in sensitive behaviors simply by erecting a physical barrier between enumerators and respondents. Another method for imparting greater privacy is to ask respondents to record their responses on a slip of paper, only later to be examined by the enumerator (Humphreys, Khan, and Lindsey 2015). List experiments are also widely used to get closer to the truth without individuals having to reveal sensitive information to researchers (Aronow et al. 2015). ⁶ In research on violence against women, women are often coupled with female enumerators, based on the understanding that shared characteristics among enumerator and respondent will generate an environment of comfortability and better approximate an individual’s private attitudes or experiences (Garcia-Moreno 2001, 17).

In sum, existing research demonstrates that levels of privacy and understandings about the intended audience have implications for what information a survey respondent reveals in surveys. The more privacy assured to respondents and the less that respondents know about the preferences of the audience, the closer a revealed preference will be to a respondent’s private preference. As such, private preferences are distinct and potentially differentiable from the preferences that people will express in the public sphere.

Public Preferences

In the process model that I explicate here, the first step in any group discussion is bringing preferences from the private sphere into the public sphere. In a focus group setting, this means sharing one’s preference with the focus group leader and other participants in the session. ⁷ In shifting to the public sphere, participants may choose to express a preference that is different from their privately held preference, a process that Kuran (1997) refers to as “preference falsification” or “dissimulation.”

The divergence between an individual’s private preference and the preference that he or she is willing to express to a particular social group is an important subject in the social norms literature (Miller and Prentice 2016; Tankard and Paluck 2016). Kuran (1997) outlines three factors—individual benefits, reputational benefits, and expressive benefits—that determine whether an individual will express a privately held preference in public. Because similarity is often highly prized among groups, individuals thinking about their reputations may choose to hide their private preferences and characterize them as similar to those of respected individuals in the public sphere. ⁸ But individuals will also weigh reputational concerns against the value that they expect to gain from revealing their private preference (such as moving a group decision toward one’s private preference) or the value that they expect to gain directly from self-expression.

When many people choose not to disclose their private preferences in the public sphere, a suboptimal social equilibrium can emerge in which community members largely behave as if they prefer one thing while they privately prefer another. This discordance is known as “pluralistic ignorance” (Bjerring, Hansen, and Pedersen 2014). In a setting of two potential preferences under pluralistic ignorance, most people (or everyone) privately prefer option A but think that others prefer option B. Because their perceptions of others’ private preferences are incorrect, people act as if they prefer option B even while they prefer option A. This behavior confirms people’s incorrect perceptions of others’ private preferences, generating a remarkably stable suboptimal equilibrium.

Despite this stability, theories of norm change suggest that behavior can change quickly when a community’s shared, privately held preference for option A is revealed, because people begin to act in accordance with their private preferences. In some cases, such as the eradication of foot binding in China, communities were able to move away from the behavior by pledging alongside other families that they would not engage in foot binding or allow their sons to marry women whose feet had been bound. By changing incentives posed by the marriage market, people were able to act on their private preferences within a generation (Mackie 1996).

Thus, social norms theories suggest that people will attempt to predict the preferences of others in their social group and will adapt their preference accordingly when revealing their preference to that social group. This means that one’s public preference is not singular or stable but will be differently constituted depending on the individuals present in social interactions.

Scholars have described how more powerful participants influence the preferences of less powerful participants. If members of a focus group expect the preferences of powerful members of their group to be more extreme, they will adapt their preferences to be more extreme. If they expect the preferences of powerful members of their group to be more moderate, they will adapt their preferences to be more moderate. Where there is uncertainty about others’ preferences, people will have difficulty making predictions and may adapt their preferences less when expressing them in public.

Yet, even though the norms framework suggests that people will reveal a different preference in public than in private, other frameworks suggest otherwise. Some scholars contend that people do not hold private preferences as distinct from public preferences. One reason is that respondents often believe that researchers are asking about what happens in a community rather than about personal preferences or what they believe should happen in a community (Schuler and Islam 2008). In many societies, respondents do not think in the individualistic way that researchers often assume, so respondents may not believe that individual preferences are (or should be) distinct from community preferences (Smith 2004). Additionally, a large psychological literature suggests that people tend to believe that others think like them (Ross, Greene, and House 1977). In this case, the difference between public preferences and private preferences given by the same individual should be minimal.

It is clear from the foregoing discussion that the first step in understanding the social nature of preferences in focus groups is accounting for how individuals portray their preferences to others. There are many reasons why an individual’s private and public preferences may differ, as well as reasons why they may not. By empirically assessing whether people hold private preferences that are distinct from the preferences that they express to others, researchers can refine their approach to interpreting silences within focus groups in relation to what is heard.

Group Preferences

Group preferences are substantively important because groups, rather than individuals acting alone, reflect the context in which many decisions are made. In focus group settings, it is common to ask people to work together on a task to learn directly from interactions and understand how preferences aggregate. ⁹ For example, focus group leaders may ask participants to rank the importance of different issue areas while noting how the discussion proceeds to give context and social meaning to the final ranking. If the first step in the social nature of preferences is sharing preferences with others in the focus group, the next step often involves making decisions with others in the group.

Several theories provide insights into dynamics of focus group decisions. As described in the previous section, social norms theories highlight how, within groups, individuals often feel implicit or explicit pressure to conform to the opinions of others in their groups, ultimately decreasing the diversity of opinions for group members to consider. People with social power sway decisions toward their preference, sometimes achieving this by doing no more than voicing their preference. Social power can also be wielded in more direct ways, such as through a glance of disapproval or a reminder about a participant’s relative status (e.g., mocking fellow participants for speaking in a language associated with lower social strata). In sum, social norms theories suggest that group decisions can be driven by a few dominant individuals.

Group polarization occurs when public discussions lead to more extreme rather than more moderate preferences (Myers and Lamm 1976). If dominant group members hold more extreme preferences, group preferences may become more polarized as others conform or adopt those more extreme preferences. Alternatively, extreme preferences may simply tend to be more compelling independent of an individual’s status within a group. Finally, there may also be diffusion of responsibility for decisions among groups, leading people to make more extreme decisions than they would individually (Kogan and Wallach 1967).

The key component of group polarization as defined in social psychology is movement toward an “already preferred pole” (Myers and Lamm 1976, 603–4). Polarization suggests movement in the same direction but beyond the average preferences of individuals. ¹⁰ In this usage, polarization stands in contrast to “extremization,” which describes how people become less neutral when acting as groups.

Group composition is thought to affect the nature of group decisions. More heterogeneous groups begin with more diversity of opinion, but this diversity might be offset as people conform to perceived social pressures imposed by dominant group members. A group’s gender composition is particularly important. Research shows that including one woman in an all-male group has no effect on decision making. Rather, women’s preferences are incorporated in group decisions only when a critical threshold of female participants is achieved (Karpowitz and Mendelberg 2014).

In sum, research shows that group preferences are not simply the mean preference held by individuals within a group. The social sphere leaves ample space for social norms to affect preference expression within focus groups; the same set of underlying private preferences may generate different group outcomes, depending on social dynamics within the group. Norms theories posit several channels by which public and group preferences will become more extreme.

Post-Discussion Private Preferences

The next step in the social preference process is how the public sphere, in turn, shapes the private sphere. When others’ preferences and arguments become known and a group preference is voiced in the context of a focus group, how does that knowledge affect private preferences that individuals hold? This step in the process describes a channel by which social norms, as expressed within focus groups, affect preferences that people then internalize.

The constructivist literature on norms recognizes that the social sphere and the private sphere are mutually constituted. In Finnemore and Sikkink’s (1998) model of international norms cascades, people internalize international norms as a third and final step of norm consolidation. When norms become internalized, people follow the norm unthinkingly and this leads to more consistency between preferences in the private and public spheres. As norms become internalized by more individuals, there will also be more consistency in preferences—“preference convergence”—across a social group. Similarity in preferences across members of a social group can reinforce preference stability because there are fewer actors to introduce new ideas or to dissent.

The process of internalization can be long, and social norms are not always internalized. When people do not change their private preferences to reflect their social group, a gap will remain between public and group preferences on the one hand, and post-discussion private preferences on the other. Take the case of religion. When a group expresses a norm against religion within a focus group discussion, this may lead people to change (1) the religious preference that they reveal to others, (2) their underlying private preference related to religion, (3) neither, or (4) both. The level of internalization may depend on group power dynamics or deliberation. Less explored is the possibility of norm defiers. Bicchieri (2006) highlights that preferences can change with or against expressed norms. Norm compliers will want to adapt their preferences to those expressed by their group, but norm defiers will want to act contrary to preferences expressed by their group. However, norm defiance is unlikely to describe the behavior of a broad population, even though it may aptly capture the behavior of a few.

Thus, there are two frameworks for thinking about whether and how people change their private preferences in social settings. Public and group preferences expressed in focus groups may have no effect on an individual’s private preference, which suggests that private preferences are fairly stable in the face of expressed social norms. Or public and group preferences in focus groups may shift an individual’s “original” private preference toward the expressed group norm or decision, which suggests that social norms become internalized. If there are changes in private preferences, this underscores the model’s depiction of all preferences as innately social.

Hypotheses

Based on the described process model, I propose three sets of hypotheses to examine the effects of the public nature of focus groups on preferences. The hypotheses help to specify what we can learn from focus groups and draw attention to unrecognized effects on human subjects.

How does the social nature of focus groups shape the preferences that researchers learn about in focus groups? I examine several hypotheses of difference to establish the validity of the steps in the process model: from private to public, public to group, and group to post-discussion private preferences. Specifically, I examine whether public preferences, group preferences, and post-discussion private preferences are statistically different from “original” private preferences of participants. If this study finds that private, public, and group preferences differ, it is incumbent on the researcher to elicit the preference type most relevant to the topic at hand.

In what direction does the public sphere shape preferences in focus groups? The process model suggests that preferences not only change but also become more extreme moving from the private to the public spheres. I propose several hypotheses of polarization based on the process model to specify (a) what researchers learn in the context of focus groups and (b) how engaging in focus group research may generate unintended effects. From private preferences to public preferences to group preferences, norms theories suggest that preferences will move unidirectionally to greater extremes. ¹¹ Finally, post-discussion private preferences will be more extreme than “original” private preferences, but potentially not as extreme as group preferences themselves as people shift in line with expressed norms.

Hypotheses of polarization suggest movement in a particular direction (toward extremes), but people’s preferences may also change by becoming more similar to one another because of discussions. I explore hypotheses of convergence to see how the proximity of people’s preferences changes before and after engaging in focus group discussions. Norm internalization suggests that post-discussion private preferences will be more similar to the post-discussion preferences that others hold than the same comparison at the outset of group discussions. Increased similarity across participant preferences (convergence) may be observed independently of whether preferences become more extreme.

These hypotheses are assessed using statistical analyses of a large number of focus groups to establish the existence of multiple, differentiable preference types and direction of preference change. The protocols draw from previous fieldwork and are validated within the focus groups alongside qualitative work. ¹²

Line of Inquiry and Context

Preference expression by both individuals and groups is a fundamental element of democracy. People within democracies ideally believe that the preference they express in the voting booth is private; however, their expressed preference may differ when asked in telephone interviews, family gatherings, or even more public settings such as a protest or social media.

Exploring private, public, and group preferences is essential to understanding sensitive attitudes in the face of competing norms. Violence against women is a highly sensitive research area where there are clear incentives to make bold statements against violence against women—even when those statements fail to reflect the attitudes or preferences one privately holds. Understanding preference divergence across preference types among different populations is key to understanding internalization of internationally propagated norms.

This study considers whether there is evidence of difference, polarization, and/or convergence in people’s private, public, and group preferences for punishing local perpetrators of three hypothetical crimes. This study measures each preference type through a series of 80 focus groups in eastern DR Congo. ¹³

Conflict and Violence against Women

For more than two decades, DR Congo has been a focus of research related to building rule of law after civil war and to decreasing impunity for violence against women. Since the onset of civil war in 1994, the state has been wracked by violence by armed groups. This violence has involved many external armed actors, the extraction of natural resources, and notably high levels of violence against civilians, including sexual violence by armed groups. The civil war formally ended in 2003, but armed violence continues in its eastern provinces.

The primary way in which researchers and advocacy organizations have engaged with DR Congo is through initiatives to address one particular consequence of the armed conflict: rape perpetrated by armed groups (Autesserre 2012; Koos and Lindsey 2022; Lake 2014). At the same time, local dimensions of violence against women have remained largely overlooked. We know that violence against women is an everyday occurrence and is not confined to acts by armed groups or to rape alone. Wife beating is a common practice within households across DR Congo, and rape is perpetrated not only by members of armed groups but also by fellow members of communities. Despite substantial international efforts to end impunity for violence against women, we know little about people’s willingness and preferences for addressing such crimes.

Three Crimes

Violence against women is violence targeted at women specifically because of their gender (True 2012). I examine social dimensions of preferences for punishing two physical forms of violence against women of interest to researchers and advocacy workers in DR Congo: rape and wife beating. Rape is recognized by the state as an international crime and explicitly violates Congolese law. It is punishable with a 20-year prison sentence, and this punishment can apply to all perpetrators of rape, not just rape by armed groups. Lake (2014) has noted the surprising ability of the Congolese justice system to hold perpetrators of rape to account, arguing that this is due to the weak nature of the state combined with the strength and magnitude of international advocacy efforts that have pushed against impunity for this particular crime. ¹⁴

Unlike rape, the perpetration of physical violence by a husband against his wife is common practice, perceived by perpetrators and victims alike as disciplinary action for a wife’s mistakes or wrongdoings. ¹⁵ Despite its common occurrence, wife beating is categorized as a crime that the police can charge and arrest people for. However, punishment is rarely pursued.

By examining social preferences related to these two forms of violence against women, this study can illuminate what is shared or distinct about social preferences for punishing two crimes (which are intertwined with different social dynamics) rather than inferring generalizability across the two forms. However, to improve our understanding of the social nature of preferences for punishment, I also consider preferences for punishing theft, a less sensitive and less gender-targeted crime.

Theft is a prominent concern of residents in DR Congo. Following the quantitative measurements detailed in later sections of this article, focus group participants describe how they view theft. They view it as the result of poverty, increased levels of circulating weapons, and continued conflict since the larger-scale civil war. Because villagers are poor and largely unarmed, there is little defense against armed roaming thieves. Focus group participants describe theft as a harm to community stability and an act that threatens children with starvation. Theft can be punished through means such as jail, fines, or return of stolen goods. But, according to my interviews with village chiefs, villagers often take punishment of theft into their own hands (unless authorities step in to stop them).

Through a comparison of rape, wife beating, and theft, this research is designed to gauge whether the social dimensions of preferences are crime-specific, related to crimes against women or generalizable to a larger set of less sensitive and less gender-specific crimes. This approach provides a nuanced perspective about the social dimensions of preferences for punishment that gains insights from differences in sensitivity, social desirability, and prevalence across crimes.

Contributions of the DRC Case

Policy discussions, research efforts, and advocacy interventions related to rape, wife beating and theft may have already aligned private and public preferences for punishing crimes through norms interventions and research engagements. ¹⁶ In addition, Congolese society is not considered highly individualistic, making it a case in which public and private preferences may be less differentiated in the first place. Shared experiences of civil war and insecurity may have also decreased variation in people’s attitudes toward punishing crimes (Lindsey 2022). Such factors may make DR Congo a difficult case to observe differences in private and public preferences or to observe new focus group effects.

Yet, the heavy inundation of advocacy and research engagement in DR Congo also makes it a critical case to consider the impact of engagements like those that have taken place. Literature on research ethics highlights the importance of considering “research fatigue,” the effect that repeated questioning about traumatizing past events has on respondents (Boesten and Henry 2018). Quantitative work has also revealed an effect of having been a research participant (in any form of research) on subsequent survey responses in this same context (De Juan and Koos 2021). If private preferences change as a result of research engagements, then there needs to be increased oversight and efforts to mitigate any potential harmful effects. This is particularly true for focus group designs that use real community events rather than hypothetical crimes.

Research Approach

Sample

To gather data on private, public, and group preferences for punishing local crimes, I partnered with an invaluable team of local researchers to conduct 80 focus groups across 20 purposively sampled villages in eastern DR Congo. ¹⁷ The sample draws from a survey conducted in 2011, which spanned a wide sample of rural villages in South Kivu. I selected villages from the survey to best approximate the causal identification of armed conflict’s effects defined as village exposure to violence by armed groups in the past five years. ¹⁸ The sampling procedure means that villages recently exposed to armed conflict are overrepresented as compared to the wider sample of rural villages in South Kivu. ¹⁹

Together with the village chief, research teams purposely sampled potential participants for two focus groups with men and two focus groups with women in each village. To mitigate bias from participant selection, focus group leaders asked village chiefs to include individuals from both higher and lower echelons of society. ²⁰

Male enumerators led focus groups with men while female enumerators led focus groups with women. Because discussing crimes of violence against women is a sensitive subject, no focus group included both men and women. The first two focus groups were held simultaneously and were immediately followed by the second two focus groups to avoid cross-contamination of results.

In this context, it is expected that some participants will be survivors of rape, will have experienced or perpetrated wife beating, or will also have experienced theft. Protocols seek to mitigate harm by focusing only on hypothetical narratives. Discussions focus on ways participants prefer to address crimes, emphasizing community agency and potential responses to harms. As a methodology, focus groups also provide a venue to give voice and decrease the power differential between researchers and vulnerable populations (Liamputtong 2011).

The nature of the discussion was described before requesting participant consent, and participants could leave at any time. Focus group leaders engaged in human subjects training for this project and had former experience asking questions about gender-based violence in this context. See supplemental appendix 1 for a full discussion.

Design

Focus group discussions center on three hypothetical crime narratives accompanied by illustrations (figure 2). Each focus group lasted approximately 2 hours, with refreshments offered due to its long duration. The measurements and qualitative insights used in this study draw from the first half of the session.

Figure 2 Crimes

The three crimes were designed to be comparable on many dimensions and reflect the context as derived from previous fieldwork (see discussion of design comparability in supplemental appendix 7.1). The illustrations were drawn for the project by a local artist in Bukavu, a city in eastern DR Congo, to embed the narratives further in context. The artist considered only the narratives and had no further direction as to content. This approach allows the drawings to more fully reflect existing conceptions about the crimes—working with and engaging understandings rather than imposing different ones.

The combination of hypothetical narratives and illustrations fosters greater uniformity in the crimes being discussed across the focus groups. While hypothetical narratives seek to approximate preferences that are relevant to real events, direct behavioral implications are one step removed.

Each focus group begins with a focus group leader verbally collecting background data from participants, including marital status, occupation, age, years living in the village, and frequency of social interactions. These variables are used as controls in the quantitative models. ²¹ After collecting these basic data from focus group participants (which included 11 to 16 members), focus group leaders then divide participants into subgroups of three to four members each, as outlined in a detailed protocol. Dividing into subgroups facilitates the quantitative portion of the research by increasing the number of (sub)group-level observations to more powerfully estimate how the public dimension shapes the private sphere. ²² The protocol yields approximately 230 subgroups from the sample of 80 focus groups with a total of 960 participants.

Each participant is then given a response card (figure 3) with a series of illustrations drawn to represent potential punishment options: counseling by a fellow villager, a fine imposed by the village chief, expulsion by the community, 25 years in prison, and beating to near death. These punishment alternatives were informed by previous rounds of fieldwork in similar communities (see the contextual discussion of these punishment alternatives in supplemental appendix 7.2).

Figure 3 Punishments

These five punishment options were selected to ground an abstract (often numerical) severity scale of 1 to 5, with specific punishments accompanied by illustrations. Abstract numerical scales can be intimidating for less educated participants. Some required encouragement to hold the pen and circle the response, underscoring the importance of this design decision for the context.

After the collection of all quantitative subgroup measurements, participants are asked to rank the punishments in terms of severity together as a (full) focus group. Figure 4 plots the means and standard deviations of the 79 rankings. During this ranking exercise, notes about perceptions of crimes and their punishment are recorded to further enrich the contextual data.

Figure 4 Mean Severity Rankings in 80 Focus Groups

Figure 4 shows the mean severity ranking of the five different punishment options, disaggregated for male and female focus groups. It is notable that there is absolute consensus that counseling is the least punitive. Men and women also have clear and consistent understandings of payment and expulsion in terms of severity. However, women view prison as more severe than beating to near death, whereas men perceive no difference in severity. When participants describe the logic of treating prison and beating as equal in severity (or with prison as potentially more severe), they emphasize that prisoners die during incarceration due to poor prison conditions. This validation exercise underscores the importance of continually checking key metrics in relation to the context over the course of a study.

To reflect perceived severity across the full set of focus groups, prison and beating to near death are collapsed into one category as the highest level of perceived severity, leaving a scale of 1–4. All analyses use the collapsed severity scale.

Hypothetical scenarios of non-intimate partner rape, wife beating, and theft are presented in a randomized order, and participants circle their preferred punishment option for each crime on a card. ²³ Focus group leaders present the crime illustrations each time they reference the crime to facilitate discussions. The order of the preference measurement is summarized in figure 5 and in the following list:

1. 1. Private preference: First, participants circle their single most-preferred punishment on the pictorial card for all three crimes in private. The focus group leaders emphasize that their response will not be shared with anyone. After all crime narratives are read, participants fold the response card and return it to the focus group leader. Respondents thus do not feel that they are being observed by focus group leaders or by other participants in the focus group, approximating private preferences.

2. 2. Public preference: Second, new cards are distributed. Participants repeat the same task for all three crimes, but focus group leaders emphasize that each participant will be asked to share his or her response card with his or her small subgroup of three to four participants. After everyone circles their public preference for each crime, participants then share their response card with other subgroup members.

3. 3. Group preference: Third, focus group leaders ask participants to discuss the punishment options with their subgroup to decide which option is the subgroup’s most-preferred punishment. Focus group leaders then verbally ask each subgroup about their choice and record the group’s preferred punishment accordingly. In the background, focus group leaders also take note of important subgroup dynamics and whether and when there are cross-subgroup dynamics at play. ²⁴

4. 4. Post-discussion private preference: Finally, a third set of response cards are distributed. Participants circle their preferred punishment on the pictorial card for all three crimes in private. The focus group leaders emphasize that these responses will not be shared with anyone. After all crime narratives are reviewed, each participant folds his or her response card and returns it to the focus group leader in order to approximate participant privacy.

By randomizing the order of the crimes for each focus group, crime-specific findings are not subject to ordering effects. However, the order in which preference types are measured remains unvaried. Thus, there are potential ordering effects between private preferences, publicly expressed preferences, group preferences, and post-discussion private preferences. ²⁵

Figure 5 Order of Measurement

To summarize, the design measures private preferences, publicly expressed preferences, group preferences, and a second measure of private preferences (post-discussion private preferences). The structure takes the form of a within-subject experiment, in which participants are “treated” at each stage of measurement. For example, the design well translates into a comparison between private and post-discussion private preferences, where the difference between an individual’s response is attributable to the public and group measurement components of the focus group discussion. However, the design does have limitations. It is not possible to tease apart whether a focus group’s effects on post-discussion private preferences are attributable to expressing a preference in public or to the process of forming a group preference. For such comparisons, iterations can be interpreted as a bundled treatment.

This design contributes a new method for the empirical measurement of preferences that also accounts for group norms. The method is feasible to apply in almost any focus group setting, and researchers may choose a different balance between quantitative and qualitative data to investigate preference types and how they change.

Evidence

This section begins with an examination of the quantitative evidence related to hypotheses of difference. I then evaluate the same evidence in terms of directional change to assess hypotheses of polarization. Finally, I consider whether there is evidence of preference convergence across participants as a result of focus group participation.

I begin by analyzing differences between public, group, and post-discussion private preferences and original private preferences for punishing rape, wife beating, and theft. I calculate the individual-level means, take the difference, and conduct paired t-tests, pooling responses from both male and female focus group participants.

Difference

As suggested by hypotheses of difference, the naive difference in means analyses depicted in table 1 show statistically significant differences between original private preferences and group choices across all three crimes. For rape, the difference between original private preferences and publicly expressed preferences is statistically significant as well as the difference between original private preferences and post discussion private preferences. For theft, the difference between original private preferences and post-discussion private preferences approach statistical significance as well. The results thus demonstrate the validity of differentiating between preference types, particularly for measures of preferences for punishing rape. ²⁶

Table 1 Mean and Difference from Private Punishment Preference

Participants may have less knowledge about others’ preferences on punishing wife beating and theft as compared to rape, which is widely discussed by advocacy groups and others in this context. Without information about the “right” response, participants will be less able to account for the expected preferences of others before expressing their preference, which is consistent with the insignificant difference in private preferences for punishing wife beating and preferences expressed to their groups.

Polarization

Table 1 shows differences between original private preferences and other preference types. But are expressed, group, and post-discussion private preferences different from one another? Consistent with hypotheses of polarization, does the severity of punishment preferences move toward extremes? Table 2 describes the statistical significance of the differences across all private and public measures of preferences for each crime.

Table 2 Difference across All Punishment Preferences

Note: The column variable mean is subtracted from the row variable mean; Significance thresholds indicated by p <= .001***; p <= .05**; p <= .01.*

While there is some variation, table 2 shows that social measures tend to be statistically different from one another. For all three crimes, the difference between publicly expressed and group preferences is statistically significant, suggesting that it is not only the publicly expressed preference that determines how preferences aggregate but also group dynamics. For all three crimes, there is also a statistically significant difference between group measures and the private preferences that people report after group discussions. Thus, even though participants update their private preferences after group decisions, participants do not fully conform with the preferences of their groups.

Is the direction of change consistent with hypotheses of polarization? All preference variables are measured on a scale from 1 to 4, with a midpoint at 2.5. Movement toward extremes is indicated by movement away from this midpoint toward, but beyond the average preference. ²⁷ The mean private preference for wife beating falls below the central threshold of 2.5, meaning a move toward extremes is downward. Accordingly, table 2 shows that preferences for punishing wife beating decrease in severity when moving from publicly expressed to group decisions (-.20). However, the mean private preference for rape and theft is above this threshold, meaning a move toward extremes is in the upward direction. Accordingly, preferences for punishing rape and theft increase in severity when moving from publicly expressed to group decisions (.27 and .14, respectively).

In sum, the data provide support for hypotheses of polarization, with preferences for punishment moving toward extremes in severity (beyond the average private preference) along the trajectory from private to public to group preference. The sign flips in the opposite direction for the difference between group preferences and post-discussion private preferences in all cases. Post-discussion private preferences become more extreme (as compared to original private preferences) but are not as extreme as group preferences. Preferences move unidirectionally through the public sphere toward extremes, but the difference moderates when preferences reenter the private sphere.

Overall, the data in tables 1 and 2 provide support for hypotheses of difference and hypotheses of polarization. The uniquely observed difference in public preference expression for rape suggests that difference depends on information about the “correct” response. However, polarization of group decisions does not seem to depend on the sensitive nature of crimes because polarization is observed across crimes.

When participants explain their (sub)group choice, they point out the grave nature of rape and theft and the need to keep the perpetrator from perpetrating again. When explaining levels of punishment for wife beating, their reasoning suggests that tolerance is a practical response, given this crime’s prevalence and the understanding that this violence does not impact people outside the marriage. Participants also perceive wife beating as a private matter. Rather than sensitivity, the perceived public nature of an issue may be critical to explaining why group dynamics unfold.

Examining Potential Mechanisms

Because powerful participants may dominate or drive public and group preferences in focus groups, researchers suggest recruiting roughly homogeneous sets of participants to gain insights about particular populations (Cyr 2019; Liamputtong 2011).

Observations noted by focus group leaders during (sub)group discussions identify cases in which group decisions were likely affected by powerful, vocal individuals. I quantitatively assess whether group power disparity, on average, affects preference difference and polarization. As a proxy for economic and social power, focus group leaders divided participants into homogeneous and heterogeneous subgroups in terms of education, a relevant indicator in this context.

Figure 6 plots average preferences for heterogeneous, homogeneous (lower status), and homogeneous (higher status) groups to descriptively assess difference across composition types. Error bars indicate standard errors of the means.

Figure 6 Preferences by Group Heterogeneity

For rape, heterogenous and homogeneous low-status groups are primarily driving preference polarization in the social sphere. While this finding somewhat supports the idea that recruiting homogeneous participants reduces polarization, polarization within vulnerable low-status groups is still present. For wife beating, polarization is relevant to all group types, but the clearest group dynamics emerge (again) in decisions made by heterogeneous groups. For theft, heterogeneous groups are also the main drivers of polarization. ²⁸

Social dynamics may also differ by respondent sex (Karpowitz and Mendelberg 2014), which may be pertinent when discussing sensitive gender-based crimes. Figure 7 presents plots of the means for all participants and then disaggregates the results by sex. Error bars indicate standard errors of the means.

Figure 7 Men’s versus Women’s Preferences

The plots reveal little need to disaggregate men’s and women’s preferences for punishing rape, even though it is a highly sensitive gender-based crime. However, sex-specific group dynamics are relevant for wife beating and theft. Women’s preferences are more affected by group dynamics for wife beating, whereas men’s preferences are more affected by group dynamics for theft. ²⁹

During discussions, multiple women voiced their preferences not to punish or jail their husbands, an effect that might explain the polarization of group preferences for wife beating. Men emphasized the harmful impact of theft on “the whole population” and that it “risks our children starving and becoming street children.” Sex-specific polarization seems intertwined with sex-specific concerns about these crimes.

Effects of Discussions

The question of whether and how focus group discussions change people’s preferences raises important ethical considerations related to focus group research, a standard form of research used worldwide. If engaging in discussions has the power to change people’s private preferences, then researchers need to take measures to mitigate potential harm to participants or others that may emerge from preference change.

To investigate the effects of focus group discussions on preferences, I examine the extent to which group preferences affect post-discussion private preferences using a within-subject experimental design. In the model, I regress post-discussion preferences—preferences that have been “treated” with the social influence of an expressed group preference—on group preferences. The models control for a respondent’s original private preference, examines the effect of group preference or “treatment” (1, 2, 3, or 4), and their interaction (to account for how an individual’s private preference plays a part in forming a group’s preference). I present the coefficient estimates from the linear regression models in table 3. ³⁰ All models use fixed effects at the focus-group level, individual-level control variables (such as age, years in the village, frequency of meeting others and education), and design-based controls (such as crime order and subgroup heterogeneity). ³¹

Table 3 Determinants of Post-Discussion Private Preferences

Note: Standard errors clustered at the village level (20 villages). ^*p<=0.1; ^**p<=0.05; ^***p<=0.01.

Across all three crimes, both an individual’s private preference and a group’s preference are positively and significantly related to an individual’s post-discussion private preference. The more severely an individual prefers to punish a criminal, the more he or she wants to punish him after the discussion. The more severely an individual’s group prefers to punish a criminal, the more an individual wants to punish him after the discussion. The interaction between original private preferences and group preferences accounts for how one’s original preference influences the group’s decision. Interaction coefficients are substantively small and do not substantively change point estimates.

In the within-subject experimental framework, exposing individuals to the “treatment” of a group preference affects the severity of people’s preferences for punishment. ³² This treatment involves public expression of preferences, interactions between subgroup members, the subgroup decision itself, and a multitude of group dynamics that remain undifferentiated.

Although this bundled “treatment” is temporally bound and relatively weak, it reflects many social norms interventions designed to change people’s local attitudes toward violence against women through group discussions. It also reflects focus groups held widely for research purposes to learn about local perceptions. However, researchers often fail to consider how discussions—on their own—even without an explicit normative agenda—express norms and thus contain the potential to beget preference change.

The findings are surprising and potentially hold implications for any discussion of norms that may beget harm. This study has focused on hypothetical crime narratives, removing the discussion to some extent from specific instances of crimes. However, consider the many focus groups being held throughout eastern DR Congo on community stigmatization of rape victims (Kelly et al. 2011, 2012). Does engaging in discussions bring about further stigmatization? Does talking about the pervasiveness of wife beating in one’s community encourage more tolerance? This study provides empirical support for cautionary tales suggested by theories of norm change (Tankard and Paluck 2016). While described in the literature, this possibility is rarely accounted for by researchers in their work.

Convergence

While it is clear that private preferences, on average, become more extreme as a result of discussions, the question remains whether there is also more agreement among participants about those (on average) more extreme preferences. Are more extreme preferences driven by outliers? Or is there also preference convergence among focus group members around those more extreme preferences?

To assess this question about preference convergence, I calculated the absolute difference between each individual’s private preference in a focus group and the average private preference in that individual’s focus group. I then calculated the absolute difference between each individual’s post-focus group private preference and the average post-focus group private preference in that individual’s focus group. Figure 8 presents plots of the mean distance from the focus group’s preference for all respondents and then disaggregates the results by sex. Error bars indicate standard errors of the means.

Figure 8 Men and Women’s Preference Convergence

Figure 8 shows that there are similar initial levels of preference similarity (for private preferences) among both male and female focus group participants. Overall, men and women hold a similar potential for preference convergence as a result of discussions. Crimes also begin with a similar potential for preference convergence, with a similar distance between people’s initial private preferences across each of the three crimes (>.7 and <.9). Because the outcome is an ordinal variable (coded 1, 2, 3, or 4), preference distance tends to be a difference of one severity level only.

Comparing the distance between a (sub)group’s mean private preference for punishment with the (sub)group’s mean post-discussion private preference for each crime reveals evidence of preference convergence among both male and female participants. As theories of preference internalization suggest, preferences for punishing rape and theft become more in line with one another after discussions take place—and this is true for both men and women. While directionally consistent, there is only suggestive evidence that the distance between women’s preferences for punishing wife beating is reduced as a result of focus group discussions.

In supplemental appendix 6 (tables A.11–A.13), I include formal analyses of the data presented descriptively in figure 8. The analyses stack the data and use a linear regression model to assess whether the form of each individual’s preference (whether the private preference or the post-focus group private preferences) affects the average preference distance from the relevant (sub)group mean. All analyses cluster standard errors at the village level. The findings confirm that the distance between an individual’s average preference and the focus group mean is significantly less for preferences measured after discussions among their groups. Yet, the difference for wife beating falls beneath standard statistical thresholds (significant at the 0.1 level).

Interestingly, women and men are similar in their likelihood of converging toward (aligning with) preferences of fellow focus group members. This runs somewhat contrary to literature that describes women as more consensus-oriented than men (Brooks and Valentino 2011) and suggests the utility of convergence analysis.

In sum, the findings show that after participation in focus group discussions not only are preferences more extreme, but there is also more agreement about those more extreme preferences. This holds mainly for rape and theft and is consistent with the direction of the estimates for wife beating.

Such analyses of preference convergence are useful ways of considering changes in the extent to which preferences are shared in a community. When more extreme preferences are coupled with more agreement in a community about a preference for a punishment and with more knowledge of that shared preference, the result suggested by the norms literature is a more stable equilibrium or set of preferences. However, at the same time, this research also shows that preferences can be changed through discussions, at least in the near term.

Limitations and Future Research

The study provides quantitative evidence of private near-term preference change measured before and after interactions in focus groups. Although the findings are notable given the duration of the treatment, this study does not examine longer-term preference change. This might be explored in future work through interviews in weeks following the discussions.

This research prioritizes the inclusiveness of people uncomfortable with numbers and literacy by using pictorial representations of five punishments. This leaves a small range of options to form the punishment severity scale. Even given this small scale, the analyses present clear evidence of difference, polarization, and convergence in preference severity; however, the substantive implications of these statistical findings should be further investigated in a context conducive to using a larger scale.

This study’s comparison of social preferences across three crimes shows that public and group preferences become more severe for rape and theft, but less severe for wife beating. The comparison across crimes allows the study to rule out alternative theories that punishment preferences always become more punitive (severe) or that the findings on polarization are specific to violence against women. Without this comparison, false inferences about different crimes or unexamined forms of violence may have been drawn. Exploring preferences for punishing multiple crimes improves generalizability of the findings to other crimes in eastern DR Congo; however, private, public, and group preferences in other research areas and contexts warrant similar study to establish the extent to which observed dynamics hold.

Finally, important questions remain about mechanisms of preference change and polarization. It is unclear whether powerful people are driving the polarization of preferences or whether there is something about the nature, content, and power of extreme preferences themselves. This is a relevant question not only for this research in DR Congo but also for political scientists seeking to understand processes of attitudinal polarization in the contemporary world. Qualitative mapping of arguments and power dynamics within a small set of focus group conversations would help tease apart essential dynamics of preference change.

Discussion

This article explores the social dimension of preferences in focus group discussions about punishment for local crimes in eastern DR Congo. Drawing from the social norms literature, I delineate three preference types—private, public, and group preferences—and introduce a process model to trace how preferences evolve. The framework advances our thinking about the interaction between the public and private spheres in focus groups while also contributing a practical approach by which researchers can measure the impact of the public sphere on the private sphere and vice versa.

I examine several families of hypotheses, positing that (1) there are differences between private, public, group, and post-discussion private preferences; (2) preferences become more polarized when moving from stage to stage; and (3) private preferences will converge (become more similar to the preferences of others) as a result of discussions. Findings from 80 focus group discussions (divided into 230 subgroups) across 20 villages reveal statistical differences between private, public, group, and post-discussion private preferences for punishing the three crimes. Moving from the private to the public sphere and back again also leaves preferences both more extreme and with more agreement overall about these more extreme preferences. This holds substantive relevance in eastern DR Congo and provides a basis for evaluating the pertinence of social preferences in other research areas and contexts.

Quantitative differences across preference types suggest that private, public, group, and post-discussion private preferences should be treated as distinct, yet interrelated, outcomes in measurement and interpretation. Researchers should ask questions in the private or public spheres depending on the substantive outcome they want to explain. In some scenarios, the private preferences of individuals are clearly relevant for behavior, such as voting in secret ballots (so long as people truly believe that these ballots are secret). However, in many scenarios private preferences will be further removed from behavior as people recast their private preferences to conform with their social group. This research contributes rare empirical support for claims that there are differences between individual data and focus group data while recognizing their connection (Fujii 2017; Morgan 1996; Zorn et al. 2006). As called for by Cyr (2019) and Liamputtong (2011), focus group researchers should more fully recognize the social nature of focus group data in writing, analysis, and data triangulation.

Finally, private preference change in focus groups contributes to the flourishing literature on research ethics and respondent research fatigue (Boesten and Henry 2018; Carpenter, Montgomery, and Nylen 2021; Krause 2021). DR Congo has seen frequent engagement in research activities and advocacy campaigns addressing violence against women. However, questions rarely arise about how engagement in group discussions, even without specific messaging, affects research findings and, potentially, people’s lives. Engaging participants as groups is thought of as a useful approach to decreasing the power differential between vulnerable populations and researchers while giving participants voice through open discussions (Liamputtong 2011, ch. 7). However, open discussions are also accompanied by the risk of potentially harmful or polarizing preference change.

Embracing the interactive nature of focus group data and the social nature of preference change means recognizing these risks and addressing them. One approach will be to adapt human subjects protocols to communicate the potential effects to participants before and debriefing after engaging in open discussions of any kind.