Do Gender Quotas Really Reduce Bias? Evidence from a Policy Experiment in Southern Africa

Abstract

A well-documented policy experiment reserving seats for women village leaders in India suggests quotas decrease citizens’ gender stereotypes and increase career aspirations and educational attainment for young women and girls. But do these results generalize to cases outside of India? And how long must quotas be in place to observe the positive effects of women’s leadership? I present data from a similar policy experiment in the southern African nation of Lesotho to test whether citizens express less gender bias after 6-years of exposure to quota-elected women village representatives. Relying on surveys and data from in-field Implicit Association Tests, I find no evidence that the quota reduced average measures of citizen gender bias, although I do find weak evidence that the quota caused young women in particular to exhibit more gender-egalitarian attitudes, both explicit and implicit.

INTRODUCTION

Drawing mostly from a government-initiated policy experiment in which a 1993 law randomly rotates districts reserved for women village leaders in India, scholars have shown that women’s presence in local leadership increases women citizens’ political engagement and the likelihood that women will run in future local elections (Beaman et al., 2011; Bhavnani, 2009). This body of work also suggests that the presence of women leaders decreases men’s gender biases (Beaman et al., 2009), improves the career ambitions and educational attainment of young women and girls (Beaman et al., 2012), increases reporting of violence against women (Iyer et al., 2012), and lowers the rate of sex-selective abortion and female infanticide and neglect (Kalsi, 2017).

But do these effects generalize to cases outside of India? And are the positive effects observed following women’s leadership immediate or do they take several electoral cycles to realize? I add evidence to this line of inquiry through a randomized policy experiment with local-level single-member-districts reserved for only women candidates in the southern African nation of Lesotho. This policy was initiated by the Government of Lesotho for one electoral cycle between 2005 and 2011, and was then repealed.

Specifically, I examine whether 6 years of exposure to quota-elected women councilors caused citizens to reduce explicit (survey-reported) gender biases as well as implicit gender bias, as measured through in-field Implicit Association Tests (IATs) I conducted in Lesotho after the quota’s removal. I find that the quota had little effect in the aggregate. Cumulatively, there is no difference between average measures of citizens’ implicit or explicit biases between formerly reserved and unreserved electoral divisions (EDs). I do find weak evidence suggesting the quota caused young women in particular to exhibit more gender-egalitarian attitudes, both explicit and implicit. Young women in formerly reserved EDs report more gender-egalitarian attitudes about the appropriateness of women in politics and are also less likely to implicitly associate men with leadership tasks and women with domestic tasks than young women in formerly unreserved areas. These findings, however, do not reach conventional levels of statistical significance after adjusting p-value thresholds to correct for multiple tests and are very sensitive to alternative specifications; and thus, are far from conclusive. In light of the evidence amassed from the Indian case, the Lesotho findings suggest that policies to increase women’s representation may take many years or decades to change citizens’ explicit and implicit gender biases. In combination, the Lesotho and Indian cases also suggest that the theorized symbolic benefits of women’s representation may be most immediately experienced by young women.

HOW QUOTAS AFFECT BIAS

Scholars of symbolic representation have argued that increased exposure to women in political decision-making may be one way to combat public gender biases, particularly around preferences for women’s political leadership (Mansbridge, 1999; Morgan and Buice, 2013). This work suggests that quotas may reduce citizens’ gender biases in at least three ways: by changing citizens’ explicit preference for male leadership, by giving citizens more information about women’s political capabilities, and/or by changing implicit stereotypes about the appropriate role of women in society.

First, by challenging the historically constructed belief that women hold a socially inferior place to men, exposure to quota-elected women representatives may reduce citizens’ taste-based discrimination. In its most blatant form, taste-based discrimination occurs when voters promote men over women due to explicitly held sexist preferences (either hostile or benevolent). Because taste-based discrimination relates to citizens’ explicit preferences for their political representatives to have certain ascriptive characteristics, changes along this dimension are likely observed in the way citizens explicitly report gender-(in)egalitarian attitudes.

Second, exposure to women representatives may causes citizens to update their beliefs about the abilities of women to effectively govern (Alexander, 2012). Whereas risk-adverse voters may not have voted for women candidates in the past because they did not know whether women could perform as well as men, being forced to see a woman as a community representative gives new information about the capabilities of women in the political sphere. Changing citizens’ beliefs that women are inexperienced and thus ineffective representatives should cause citizens to reduce their explicit political gender biases.

Third, distinct from explicit discrimination, role congruity theorists attribute individually held biases to implicitly held social stereotypes (Eagly and Karau, 2002). Because certain social groups are stereotyped with characteristics associated with success in particular roles, citizens may form prejudice against out-group members who possess traits that are perceived as incongruous. Prejudice toward women leaders, then, follows from the incongruity individuals perceive between socially constructed ideals of female behavior and the perceived requirements of successful leaders (Eagly and Karau, 2002; Benstead et al., 2015, p. 574). Exposure to women councilors, then, will affect citizens’ implicit gender biases by exposing them to counter-stereotypical women.

THE CASE: A POLICY EXPERIMENT IN LESOTHO

This article takes advantage of a nationwide randomized policy experiment in the southern African nation of Lesotho. In 2005, Lesotho’s Ministry of Local Government divided the country’s 10 main local administrative districts into 129 newly created community councils. Each community council was divided into 9–15 single-member EDs, each of which elected a community councilor through a first-past-the-post election. The community councils were charged with village level maintenance issues such as land allocation, livestock grazing rights, managing the local water supply, and maintaining village markets and local roads (Shale, 2004; Clayton, 2014). The councilors elected from each ED are mandated to represent the villages in their ED at council meetings, typically once a month. The EDs are relatively small, with constituencies consisting on average of around 1,000 adults. Local councilors are well-known local figures. In the original survey data collected for this project, over 90% of respondents could correctly name both their current and previous local councilors without any difficulty.

The experimental nature of the quota was as follows: Between 2005 and 2011, the Local Government Elections Act required that 30% of all newly created single-member EDs be reserved for only women councilors.¹ Reservation status was assigned randomly by Lesotho’s Independent Electoral Commission (IEC) (SADC, 2011, p. 59). To test the veracity of the random assignment, I use the 2004 Afro-Barometer survey to verify that the characteristics of EDs that were reserved a year later were comparable with EDs that went unreserved. The Supplementary material presents these balance diagnostics with standard errors clustered at the ED level. Overall, the reserved and unreserved EDs are comparable across pre-treatment variables (also see Clayton, 2014, 2015).

FINDINGS

Explicit Bias

I use the 2012 Afro-Barometer Lesotho survey to test for changes in explicit citizen bias following exposure to quota-elected representatives. In order to identify which observations were in previously reserved EDs, I merge the Afro-Barometer data with data I collected from the IEC in Maseru, which groups villages by ED and identifies the reservation status of each ED. The 2012 Afro-Barometer survey includes three questions related to gender biases, and asks respondents to state their level of agreement with the following statement pairs:

Explicit bias question 1:

Statement 1: Men make better political leaders than women, and should be elected rather than women.

Statement 2: Women should have the same chance of being elected to political office as men.

Explicit bias question 2:

Statement 1: In my country, women should have equal rights and receive the same treatment as men do.

Statement 2: Women have always been subject to traditional laws and customs, and should remain so.

Explicit bias question 3:

Statement 1: If funds for schooling are limited, a boy should always receive an education in school before a girl.

Statement 2: If funds for schooling are limited, a family should send the child with the greatest ability to learn.

I recode the second statement pair so that higher values are associated with more gender-egalitarian attitudes across all three measures (on 4-point Likert-type scales).² Table 1 shows the average treatment effect of the quota policy with p-values and confidence intervals calculated through randomization inference. In short, the quota appears to not have affected citizens’ self-reported gender bias across all three indicators.

Table 1 Total Sample Reservation ATEs

Notes. Num. obs = 996, Num of groups (EDs) = 135. Confidence intervals and p-values calculated through randomization inference. Standard errors clustered by ED.

It is possible that the null average treatment effects are masking important heterogeneous treatment effects among specific demographic groups. I identify conditional average treatment effects for subgroups partitioned on two theoretically-motivated characteristics: respondent age and gender.³ Previous research indicates that men and women respond to counter-stereotypic women in different ways, but the predicted direction of these effects remains unclear. Studies conducted in Western contexts often find that men exhibit stronger gender biases (Rudman and Kilianski, 2000), but that men may update their beliefs about women leaders more quickly than do women (Eagly and Karau, 2002, p. 589). Yet, other work suggests women respond more positively than men to women’s increased presence in politics (Alexander, 2012).

Age is another salient evaluator feature that might predict the malleability of respondents’ explicit and implicit gender biases. Younger cohorts of both men and women likely hold more tractable gender attitudes, as socially constructed gender roles are not as fully crystallized by social experiences as among older generations. This may be particularly pronounced among young women, as young women and girls may realize the symbolic and substantive benefits of emulating successful women. Indeed, a significant body of research from a diverse set of cases has demonstrated how women in public leadership may act as role models for young women and girls in their communities (see Beaman et al., 2012; Wolbrecht and Campbell, 2007).

In the analyses that follow, I code younger cohorts as respondents who are under 25 years in age.⁴ This choice is motivated by Beaman et al. (2012), who use the random assignment of the Indian quota to identify role model effects specifically among adolescent girls aged 11–15 years. The Afro-Barometer does not survey respondents under 18 years old, but the under 25 threshold allows a subgroup of respondents who experienced quota-elected councilors during their adolescent and teenage years. The youngest respondents in the under 25 cohort experienced the representation of a quota-elected councilor from the age of 11–17 years and the oldest from the age of 17–23 years.

I use treatment by covariate interactions to identify model-based estimates for the quota’s conditional average treatment effects by age and gender. The complete regression results for all three bias indicates are displayed in appendix Table A1. I test for treatment effects in four distinct subgroups: men and women, both older and younger than 25 years. To account for these multiple tests, I employ a Bonferroni correction, which implies a significance threshold of 0.0125 (0.05/4). No subgroup effect reaches significance at this level. The only intersection of age and gender that approaches non-corrected conventional significance levels relates to young women, who, when living in an area previously reserved for women councilors, report more gender-egalitarian attitudes on women’s political rights (p = 0.056, Model 1.3, Table A1).

Descriptively, compared to her counterpart in an unreserved ED, a young woman living in a previously reserved ED is 30% more likely (moving from 57% to 74%) to report that she strongly agrees “women should have the same chance of being elected to political office as men” and 35%t less likely (moving from 17% to 11%) to strongly agree that “men make better political leaders than women, and should be elected rather than women.” Figure 1 displays the quota’s marginal effect by both gender and age cohort on the 4-point Likert-type scale. Substantively, this effect is associated with a movement of 0.34 points on the political bias scale; a standardized effect size of 0.29 standard deviations. However, this finding is not particularly robust. Treatment effects associated with alternative cut-points 2 years above and 2 years below the under 25 specification reach significance at the 10% level or below and a three-way interaction of age cohort, gender, and reservation status is positive, but is even further from traditional significance thresholds (p = 0.13). An interaction with age rather than the under 25 cohort dummy goes in the expected (negative) direction, but also does not reach conventional levels of statistical significance (p = 0.11).

Figure 1 Marginal Effects of Quota on Explicit Gender Bias by Subgroup. Higher Values Associated with More GenderEgalitarian Attitudes. Robust Standard Errors Clustered by ED. Confidence Intervals Around Estimates Calculated at the 98.75% Level to Correct for Multiple Tests. Num. Obs = 498 Men and 498 Women.

Implicit Bias

To test for changes in gendered implicit associations, I use an experimental test typically employed in social psychology and increasingly in behavioral economics, but still rarely used in political science. Following Beaman et al. (2009), I tested for differences in gender stereotypes across reserved and unreserved EDs through laptop-based IATs in 92 EDs across Lesotho in early 2014. Power calculations reveal that despite the small sample size, I have sufficient power to detect relatively small effect sizes (see Supplementary material), largely due to limited variance in implicit biases across respondents.⁵

Specifically, I administered an activity-based IAT to test for implicit bias in gender stereotyping of occupations. This test assesses whether villagers exposed to quota-elected councilors are less likely to associate women with domestic life and men with leadership activities. The IAT records response times as each prompt is presented on the screen to measure strength of association, based on the assumption that making a response is easier when items that the respondent considers as closely related are on the same side of the screen and share the same response key (Nosek et al., 2007). The laptop-based test uses images of a Mosotho man and a Mosotho woman and pictorial or auditory prompts (in Sesotho) associated with leadership or domestic concepts and male or female Sesotho names (see Supplementary material for details).⁶

The IAT measures bias through a D-score, which captures the normalized difference in mean response time between the non-stereotypical and stereotypical test blocks (Nosek et al., 2007). A higher D-score indicates a stronger implicit stereotype—in this case a stronger association of men with leadership and women with domestic life.⁷ Table 2 shows the quota’s average treatment effect between formerly reserved and unreserved EDs as well as the conditional average treatment effects for the split samples of men and women respondents. I again calculate p-values and confidence intervals through randomization inference. I find statistically indistinguishable levels of implicit bias across formerly reserved and unreserved areas, both for the whole sample and for men and women respondents separately. Table 2 also allows a direct comparison to effects identified from the Indian case. Beaman et al. (2009, p. 1529) find that reserved village leadership positions for women caused men respondents to reduce their implicit biases through a 0.076 reduction in D-score (SE = 0.032); an effect size more than seven times larger than any of near zero effects found among either men or women here.

Table 2 D-Score Responses to IATs for Whole Sample and Men and Women Respondents Separately

Notes. Num. obs = 101 (51 women, 50 men), Num of groups (EDs) = 92 Confidence intervals and p-values calculated through randomization inference. Standard errors clustered by ED

As above, I test for potential conditional treatment effects within age cohorts through model-based estimates of treatment by covariate interactions. Model 3 of Table 3 reveals a conditional average treatment effect among women respondents under 25 that is significant at the p = 0.03 level, approaching the 0.0125 threshold to correct for multiple tests. The significance level associated with this result is sensitive to alternative specifications. As above, the three-way interaction on the complete data as well as a specification with a continuous age variable (rather than a young cohort dummy) both go in the expected direction but do not approach conventional significance levels.

Table 3 D-Score Responses to IATS: Model-Based Estimates of Age and Gender CATEs. Robust Standard Errors Clustered by ED

Notes. To correct for multiple sub-group tests, coefficients with p < 0.0125 are marked in bold.

This result provides weak, but suggestive evidence that the quota may have weakened the extent to which young women (but not young men) implicitly associate women with the home and men with public leadership. Substantively, young women in previously unreserved EDs have a mean D-score of 0.29 compared to young women in previously reserved EDs, who have a D-score of 0.18; an effect associated with standardized effect size of 0.22 standard deviations (similar to the effect size on explicit bias). Figure 2 plots the marginal effect of the quota on implicit bias by subgroup type.

Figure 2 Marginal Effects of Quota on Implicit Gender Bias by Subgroup. Lower D-Scores Associated with Less Implicit Bias. Robust Standard Errors Clustered by ED. Confidence Intervals Around Estimates Calculated at the 98.75% Level to Correct for Multiple Tests. Num. Obs = 51 Women and 50 Men.

Model 3 of Table 3 reveals that while the quota appears to reduce implicit bias among young women, this demographic group actually exhibits more implicit bias than older women in unreserved areas. This result may reflect noise in the data and add to a cautious interpretation of significant subgroup effects, but a substantive interpretation is also plausible. Younger cohorts of women likely have less crystallized gender stereotypes, but this does not mean this group will hold more egalitarian beliefs. Young women and girls in predominately rural Lesotho may be most prone to associate women with domestic life and men with leadership if they have yet to fully participate in public life. At the same time, their internal associations may also be more malleable, and may more readily change after exposure to counter-stereotypic women.

CONCLUSION

I find weak evidence that quota-induced increases in women’s representation caused a move toward gender-egalitarian attitudes specifically among young women respondents, but not within other demographic groups. My evidence suggests, then, that growing up with a woman community leader may have signaled to these young women and girls, both explicitly and implicitly, that politics is an appropriate sphere for women. I am somewhat circumspect, however, when interpreting these findings. Although consistent across two distinct data sources, my findings related to young women do not cross significance thresholds that correct for multiple tests and are sensitive to changes in model specification. Indeed, my null findings in this area are much more robust. I find no evidence that quota-mandated exposure to women representatives affected average levels of public gender biases. The quota did not change explicit taste-based discrimination or implicit gender biases on the sample as a whole.

My findings speak to the large body of work on the effects of India’s reservation-based quota system. First, there is one area where my findings comport with this body of work. Beaman et al. (2009, p. 1529) find no evidence that citizens decrease explicit taste-based discrimination against women village leaders even after two electoral cycles (10 years) of reservations. Evidence from the two cases, then, reveal that even after a decade of exposure to women representatives, citizens do not change their explicit preferences for women representatives.

Related to the effects of women’s leadership on young women and girls in particular, Beaman et al. (2012) find that increases in adolescent girls’ aspirations only start to appear after the second reserved electoral cycle. This result suggests tentative patterns consistent with the Lesotho case. I find weak evidence that quotas decrease young women’s explicit and implicit stereotypes around appropriate gender roles, a finding that the Indian case suggests grows stronger and more diffuse over time. Taken together, findings from both cases suggest, somewhat intuitively, that young women may be the first group susceptible to the symbolic benefits of women’s increased presence in political decision-making. Women’s political presence, however, likely needs to be sustained for many years before most citizens change their attitudes toward women in public life.

Finally, this study speaks to previous findings from the Lesotho case (Clayton, 2015, 2014). In the former, I find that Lesotho’s policy experiment caused a backlash among women citizens, who reported less political engagement in reserved areas; whereas, in the latter, I find that the quota challenged the perceived influence of local traditional leaders. Whereas the quota may have been unpopular, it also may have caused citizens to change their perceptions of women’s influence in local governance in ways they did not attribute to the quota. The findings I present here perhaps suggest a similar pattern. Whereas the quota did not change most citizens’ immediate gender biases, the downstream effect of the quota—women’s increased presence in politics—may have caused young women in particular to form more gender egalitarian beliefs.

SUPPLEMENTARY MATERIALS

For supplementary material for this article, please visit Cambridge Journals Online: https://doi.org/10.1017/XPS.2018.8.