Negating the Gender Citation Advantage in Political Science

Abstract

Open-access (OA) advocates have long promoted OA as an egalitarian alternative to traditional subscription-based academic publishing. The argument is simple: OA gives everyone access to high-quality research at no cost. In turn, this should benefit individual researchers by increasing the number of people reading and citing academic articles. As the OA movement gains traction in the academy, scholars are investing considerable research energy to determine whether there is an OA citation advantage—that is, does OA increase an article’s citation counts? Research indicates that it does. Scholars also explored patterns of gender bias in academic publishing and found that women are cited at lower rates in many disciplines. Indeed, in many disciplines, men enjoy a significant and positive gender citation effect (GCE) compared to their female colleagues. This article combines these research areas to determine whether the OA citation advantage varies by gender. Using Wilcoxon-Mann-Whitney (WMW) tests, the nonparametric analog to the independent samples T-test, I conclude that OA benefits male and female political scientists at similar rates. Thus, OA negates the gender citation advantage that typically accrues to male political scientists.

A key line of inquiry in academic publishing is the efficacy of open-access (OA) publishing. Advocates contend that OA articles (i.e., freely available online) “level the playing field” for researchers worldwide. The argument is that OA is egalitarian in that now everyone can use the scholarly resources that previously were reserved for only those scholars and institutions that could afford to purchase access. In turn, authors of the OA articles will benefit from increased exposure because their work is more widely disseminated. Advocates argue that increased accessibility will give OA articles a citation advantage over toll-access (TA) articles; and, although there are mixed results, the research supports that argument (Atchison and Bull 2015; Doty 2013; Lawrence 2001; McCabe and Snyder 2014; Norris, Oppenheim, and Rowland 2008).

A second research area in the study of academic publishing addresses citations and gender bias. Scholars have found that in many disciplines, women are less frequently cited by their male colleagues (Aksnes et al. 2011; Ferber 1988). Also, and most important to this study, researchers have found that in some disciplines, women are cited at lower rates than their male colleagues, which indicates that men in those disciplines receive positive gender citation effect (GCE) simply by being male (Aksnes et al. 2011; Davenport and Snyder 1995). When we consider gender bias in citations in conjunction with the OA citation advantage, it raises the question of whether the advantage applies equally to women and men. To my knowledge, no one has yet asked this question, but it is a critical question. All available evidence indicates that women experience discrimination at almost every level of the academy, from hiring to publishing to promotion and tenure decisions (American Political Science Association 2011; Monroe and Chiu 2010; Monroe et al. 2014). At many institutions, citation counts are an important consideration in promotion and tenure decisions; therefore, if women are cited at lower rates despite the high quality of their research, they are disadvantaged from the outset in the promotion and tenure process (Maliniak, Powers, and Walter 2013). This study is a first step in determining whether OA publishing may be a way to level the playing field for female political scientists.

The article begins with a basic overview of both OA and gender biases in academic publishing. This is followed by a description of the data and methodology used in the study. The results and implications for the discipline are presented next. The article concludes by discussing limitations of the study and possibilities for further research into gender and the OA citation advantage.

OPEN-ACCESS FUNDAMENTALS

The price of academic journal and database subscriptions has skyrocketed in recent decades. As a result, authors and institutions must seriously consider whether the traditional model of academic journal publishing provides the most effective access to scholarly works (Greco et al. 2007). Of the many proposed alternatives, OA publishing has emerged as the most promising option because it allows articles “to be read for free by anyone, anytime, anywhere—as long as they have Internet access” (Crawford 2011, 11). In contrast, any article that is locked behind a paywall is considered TA. Advocates make a strong case for OA publishing as an egalitarian method of information dissemination because it allows everyone and anyone to access scientific research at the click of a button. OA allows researchers in developing states or at poorly funded institutions to access the same resources as their colleagues at well-funded institutions (Arunachalam 2008; Crawford 2011; Guédon 2008).

There are many nuances in OA publishing; however, at the core, all OA is a version of either Gold OA or Green OA. ¹ Within the Gold category, there are several different business models. The point of commonality is that in all forms of Gold OA—whether it is a fully OA journal or an article processing charge (APC) is paid to a TA journal to make the article OA—the route to OA is through the publisher. There is a common (mis)perception that “Gold” means a “pay-to-play” model, which results in scholars’ many concerns about OA publishing (Harnad 2010; Suber 2013). Xia’s (2010) findings indicated that these concerns are likely predicated on the belief that OA publishing is a low-quality model in which research is subjected to lax peer review (if any). Additionally, because most high-ranked journals are not OA, scholars may be concerned that publishing in an OA journal could be detrimental to hiring, promotion, and tenure decisions. Although there are some disreputable OA journals, most Gold OA journals are peer-reviewed with safeguards in place to ensure that financial concerns and APCs are kept separate from the editorial- and article-acceptance process. For example, the American Journal of Political Science has a Gold OA option (i.e., OnlineOpen) that is made available to an author only after an article has been accepted. ² One benefit of Gold OA is that because articles have been made OA by the journal publishers, they are accessible from a general Internet search as well as from traditional academic databases.

The point of commonality is that in all forms of Gold OA—whether it is a fully OA journal or an article processing charge (APC) is paid to a TA journal to make the article OA—the route to OA is through the publisher.

In the Green OA model, the route to OA is through the author herself. She is free to publish with the high-quality journal of her choice; she then self-archives her work in an institutional repository or on her personal website. ³ Provided that an author has chosen to submit her work to a reputable journal, the Green OA model ameliorates all of the concerns mentioned previously. There are no fee-based variants of Green OA publishing; therefore, there are no pay-to-play issues. The work has been peer-reviewed, published in a traditional journal, and only then made publicly available. The OA version of an article is then searchable through a general web search (e.g., Google Scholar); however, it would not be found in a traditional academic database search. ⁴ Although this is a drawback to the model, Green OA is a particularly attractive model for the social sciences and humanities because it is essentially free to an author—and authors in those disciplines are less likely to have grant-funded research projects than colleagues in the natural sciences.

Regardless of the OA model, in theory, an author should see increased citations as a result of making her work freely available—what Doty (2013) called the “open-access citation effect” (OACE). Providing evidence to support that theory has been the main thrust of research about the efficacy of OA. ⁵ In the social sciences, research indicates that there is a positive OACE. For example, Hajjem, Harnad, and Gingras (2005) found that there is an across-the-board citation advantage for each discipline in the study, including economics, political science, and sociology. Similarly, Xia and Nakanishi (2012) found a positive OACE in their study of anthropology research. More recently, Atchison and Bull (2015) found that political scientists receive a positive OACE when they follow the Green OA model and self-archive their work. Despite the large volume of research into the issue, the OACE literature has ignored an important question: Do women and men benefit from OA publishing at similar rates? To put this question into context, it is important to understand the significant differences between men and women in academic publishing.

GENDER AND ACADEMIC CITATIONS

Researchers have explored the possibility of differences between men and women in both citation patterns and citation rates. First, researchers examined the way in which gender affects the pattern of who cites whom, and it is clear that a two-way gendered split in citation patterns exists. The first gendered citation pattern is that women tend to cite women more often and men tend to cite men more often (Baldi 1998; Ferber 1988; McElhinny et al. 2003). There is evidence that this particular gendered citation pattern results from gendered academic networks—networks in which women are more at the margins of several disciplines (Aksnes et al. 2011). The second gendered citation pattern concerns self-citation: the available evidence indicates that women self-cite at far lower rates than men across all disciplines (Wilson 2014). ⁶

Second, scholars also attempted to determine whether men receive a positive GCE in academic publishing (as measured by citation rates). The results are mixed. On one the hand, researchers in several fields found a significant positive GCE for men (Aksnes et al. 2011; Davenport and Snyder 1995; Gonzalez-Brambila and Veloso 2007). On the other hand, other researchers did not identify any gendered difference in citation rates (Bordons et al. 2003; Lewison 2001) or a positive GCE for female researchers (Long 1992; Sonnert and Holton 1995). These contradictory findings indicate that Copenheaver, Goldbeck, and Cherubini (2010, 128) were correct in stating that “Gender differences in citation rate appear to be discipline specific.” This raises the question: What are the gendered effects in political science publishing?

Although there are relatively few studies on gender and publishing in political science in general, Masuoka, Grofman, and Feld (2007) found that female political scientists are not cited at rates proportional to their presence in the discipline. In a related study of more than 3,000 articles from multiple international relations (IR) journals, Maliniak, Powers, and Walter (2013, 19) found that women are cited at significantly lower rates than men, and that “[a]rticles written by female authors are not only being cited less, but authors of the most influential articles are citing them less often.” Furthermore, they found that women are self-citing at significantly lower rates than their male counterparts, and they concluded that this has a significant negative effect on women’s citation rates. Their results provided evidence that there is a significant gender citation gap between men and women in IR, with men receiving an average of 4.8 more citations than their female colleagues (Maliniak, Powers, and Walter 2013, 892). ⁷ As the authors noted, the results could be an artifact of male dominance in the IR fields. However, because less than 30% of all political science faculty are women, it is relatively reliable to state that the discipline is male-dominated (American Political Science Association 2011, 4). When considered with the discipline-wide evidence presented by Masuoka et al. (2007), this indicates that it is logical to expect a gender citation gap in political science generally, not only IR.

COULD OPEN ACCESS CHANGE GENDERED CITATION EFFECTS?

There is no reason to expect that Green OA—by virtue of the fact that it makes information accessible—would expand researchers’ academic networks, thereby affecting gendered citation patterns and who-cites-whom. Consequently, this article focuses on the issue of gendered citation rates. As discussed previously, OA advocates have framed OA—particularly Green OA—as an egalitarian publication model. Clearly, they are referring to Green OA as a more economically egalitarian model. It levels the playing field for institutions and scholars with limited means by reducing both the subscription fees and the APCs. In contrast, my question is whether Green OA is a gender-egalitarian publishing model: Given the known OA citation advantage, does this effect hold equally for men and women? However, to put that question into perspective, it is important to determine whether women are actually using the Green OA model (i.e., self-archiving) at similar rates as men.

If we view self-archiving as a form of self-promotion, the evidence indicates that women are likely to self-archive at lower rates than men.

If we view self-archiving as a form of self-promotion, the evidence indicates that women are likely to self-archive at lower rates than men. Winkler (2000) noted that women’s disinclination to self-promote is a barrier to tenure and promotion. Furthermore, as McIlwee and Robinson (1992, 71) pointed out, “We associate femininity with…avoidance of bragging and self-promotion. Even if a woman knows her work is outstanding, she should not brag about it.” This is borne out by the evidence of Maliniak, Powers, and Walter (2013) that women self-cite (i.e., a form of self-promotion) at far lower rates than men. In her comments in an article in The Chronicle of Higher Education, Walter explained those results, noting that women are reticent to self-cite because they perceive it as somewhat unethical (Wilson 2014). Women’s reluctance to self-promote led me to the first hypothesis, as follows:

• • H1: Female political scientists will self-archive at a lower rate than male political scientists.

Previous research indicates that there is an OA citation advantage in political science (Antelman 2004; Atchison and Bull 2015). However, as noted above, the available evidence in political science also indicates that women are cited at a statistically significant lower rate than their male colleagues (Maliniak, Powers, and Walter 2013; Masuoka et al. 2007). The lower citation rates of female political scientists may well be due to women’s concentrations at lower ranks and nonresearch institutions (American Political Science Association 2011). Green OA has no effect on those structural barriers to citation. Thus, it is logical to expect that even when their work is more available, female political scientists will continue to be cited at lower rates than their male colleagues. As a result, I hypothesized the following:

• • H2: Female political scientists will see lower OA citation effects than male political scientists.

DATA AND METHODS

In this section, I first explain the data collection. I then provide descriptive statistics, and explain my use of the Wilcoxon-Mann-Whitney method to test my hypotheses.

Data

In citation-effect research, it is important to consider both length of time since publication and journal influence (Craig et al. 2007). To account for both of these factors, I started with Atchison and Bull’s (2015) original dataset, which includes 727 articles from the 2007–2008 volumes of the American Political Science Review, American Journal of Political Science, Public Opinion Quarterly, Journal of Conflict Resolution, Political Analysis, Political Geography, Annual Review of Political Science, and Comparative Political Science. ⁸ By including articles only from 2007–2008, I used those that have been available for similar lengths of time and have had time to amass citations. By using articles published in these journals, I used those that have similar Journal Citation Report impact-factor scores. In addition, none of these is a Gold OA journal, which means that any OA articles in the dataset are Green OA. The Atchison and Bull (2015) data included citation counts and accessibility (i.e., OA or TA). I used Google Scholar to find each article, determine the total number of authors, and identify the gender of each author. Gender was determined by examining the personal pronouns in author biographies and/or photographs posted on author or institution websites. I omitted any records for which an author’s gender could not be definitively determined. Also, citation rates can be distorted by a single highly cited article; therefore, to account for this, I excluded articles for which the citation count was three or more standard deviations above the mean. ⁹ This resulted in an N of 704 records, each of which represented a single article and its corresponding citation total.

Descriptive Statistics

In OA research, the descriptive statistics are often revealing. In this case, they provide an overview of the presence or absence of a gender citation gap for political science journals. Table 1 presents the gender distribution of observations for the entire dataset, as well as the distribution across journals. Almost 67% of the 704 articles were written by men, 12.5% by women, and 20.5% by mixed-gender teams. Figure 1 indicates that more than 56% of the observations are OA, having been self-archived in some format. The descriptive statistics reported in figure 1 provide preliminary evidence in support of H1, given that 32% of solo-female-authored papers in the dataset are self-archived compared to 49% of solo-male-authored papers. Indeed, the disparity was even greater between papers with multiple male authors (71%) and multiple female authors (41%).

Table 1 Gender Distribution of Observations

Figure 1 Open-Access Distribution, by Gender

The citation rates shown in table 2 and figure 2 indicate that male-authored articles receive more citations than female-authored articles. As table 2 indicates, this pattern holds whether men’s articles are single- or multi-authored.

Table 2 Citation Rates by Gender, All Articles

Figure 2 Citation Rates, by Gender

However, these statistics are of limited utility in exploring H2, given that they are the citation rates for the full dataset rather than the population of OA articles.

The statistics presented in table 3 provide a better examination of citation rates for the OA articles. As shown, there is mixed descriptive evidence regarding the hypothesis that women will have less of an OA citation advantage than men. Whereas the OA citation rates for papers written by multiple female authors have lower citation rates than those written by multiple male authors, the citation rates for solo-authored papers written by women are higher than those solo-authored by men.

Table 3 Citation Rates of Open-Access Articles, by Gender

Methodology

To test H1 (i.e., female political scientists self-archive at lower rates than their male colleagues), I performed a simple chi-squared test to determine the relationship between gender and self-archiving rates. To test H2 (i.e., female political scientists will experience lower OACE than male political scientists), a difference of means test typically would be used to determine the gender differences in citation rates. However, these data cannot be assumed to be normally distributed because of the count nature of the variable. Following Atchison and Bull (2015), I used the nonparametric Wilcoxon-Mann-Whitney (WMW) test to compare the two populations. The WMW tests H0 (i.e., population one is equal to population two) versus H1 (i.e., population one is not equal to population two). The resulting test statistic (i.e., the Z-score) indicates position from the mean, whether positive and above the mean or negative and below the mean.

RESULTS AND DISCUSSION

The gender differences in self-archiving are reported in table 1. The results of the chi-squared test indicated that the gender difference in self-archiving rates is statistically significant (i.e., Chi-squared with one degree of freedom: 20.6; p = 0.000). This finding provides support for H1 (i.e., female political scientists are self-archiving at much lower rates than their male counterparts). To test H2, I first ran a series of WMW tests to determine whether both genders receive an OA citation advantage. Those results, presented in table 4, indicate that there is a significant and positive OACE for the full dataset, as well as for each gender category. With WMW analysis, the test statistic cannot be interpreted to indicate the number of citations above or below the mean. The number indicates only the certainty that the result is more than (or less than) the mean. ¹⁰

Table 4 Open-Access Citation Effect, by Gender

Note: **p < 0.05, ***p < 0.01

It has been established that self-archived articles receive a citation advantage compared to TA articles regardless of the gender of the author(s); the discussion now turns to an analysis of gender within the OA articles. To determine whether there is a GCE within the OA population, I ran another series of WMW tests. The results, presented in table 5, indicate that there is no statistically significant gender advantage within the OA data. This indicated that the OA papers written by female political scientists are cited at rates similar to OA articles written by their male counterparts; H2 is not supported by the data.

Table 5 Gender Advantage in Open-Access Articles

Note: **p < 0.05, ***p < 0.01

Taken together, these results suggest that OA not only provides a citation advantage to all political scientists, it also negates the positive GCE that typically accrues to male researchers. This finding indicates that when female political scientists self-archive their work, they are cited at the same basic rate as their male counterparts; self-archiving should equalize citation rates between male and female political scientists.

Taken together, these results suggest that OA not only provides a citation advantage to all political scientists, it also negates the positive GCE that typically accrues to male researchers.

This result is surprising given women’s citation disadvantage in political science, and future studies should explore why the OACE appears gender-neutral in this analysis. Contributing factors may include a combination of rising journal costs, budget cuts, and accessibility. Since 2007, academic institutions have faced an almost 35% overall increase in journal subscription costs, and almost all academic libraries in the United States have faced budget cuts (Greco 2015, 14, 21). These cuts were steepest at less prestigious universities with smaller endowments. There are approximately 300 research-focused universities (which typically have larger endowments) and about 3,000 four-year degree-granting institutions in the United States; thus, the majority of faculty in the United States work at institutions that have comparatively limited research resources (Carnegie Foundation for the Advancement of Teaching 2011; US Department of Education National Center for Education Statistics 2015). Consequently, free access to quality research has become increasingly important for most members of the discipline. The results presented in this article may indicate that people use the research to which they have access, regardless of the author’s gender.

OBSTACLES TO SELF-ARCHIVING

The findings presented in this article demonstrate that women in political science are self-archiving at significantly lower rates than men; however, they do not tell us why women are less likely to self-archive. The extant literature on institutional repositories indicates that, in general, reluctance to self-archive stems from technological qualms, uncertainty regarding copyright limitations, and confusion regarding publishers’ self-archiving policies.

First, many scholars neither know how to use the self-archiving resources available to them (e.g., Google Scholar) nor feel able to dedicate the time to set up their own web page. This issue has been solved primarily at the institutional level, with the implementation of institutional repositories at many colleges and universities in the United States and Canada (Dubinsky 2014). Institutional repositories are the easiest method of self-archiving for most scholars. At most colleges and universities, institutional repositories are housed in Library Services and self-archiving is as simple as e-mailing the work to the repository. The staff then handles the technical aspects of posting the article (Dubinsky 2014). If resources do not allow for the creation of an institutional repository, authors can upload their work to an external repository such as the Social Science Research Network (Carling 2012) or create a Google Scholar page. Additionally, researchers can upload their work to sites such as ResearchGate and Academia.edu, which essentially are academic social-networking sites on which researchers can follow one another’s work. These sites have the added benefit of helping researchers to expand their academic networks, but they lack the benefit of copyright-agreement assistance.

A second obstacle to self-archiving is that scholars often are uncertain about what is and is not allowed under a publisher’s copyright agreement (Dubinsky 2014). This can become confusing because one publisher may allow an author to self-archive the formatted and branded publisher PDF, whereas others may allow only self-archiving of the version that was initially submitted to the journal (pre-review); still others will allow self-archiving of the accepted version. ¹¹ This obstacle typically is overcome with help from the institutional repository staff: most institutions have an in-house expert who will assist with interpreting copyright agreements and ensure that authors are self-archiving as allowed under the agreements. Also, whether or not the institution has an institutional repository, faculty can request workshops on publishing, OA, and copyright agreements; these are helpful in clarifying the post-acceptance process. For scholars who want to research a journal’s or a publisher’s self-archiving policy, the SHERPA/RoMEO database is an excellent resource to clarify those policies. ¹²

A final obstacle is that some publishers still do not allow any form of self-archiving. If an author has not determined the publisher’s self-archiving policy before signing the copyright agreement, she may be signing away the ability to self-archive. Again, the institutional repository staff is the best resource for interpreting a copyright agreement. Additionally, if a publisher’s standard agreement indicates that self-archiving is not allowed, an author can petition for an exception, which may be called an OA addendum to the copyright agreement. ¹³

CONCLUSION

The results presented in this article indicate that female political scientists do not self-archive at the same rates as their male colleagues. However, when women make their work freely available online, their research is cited at similar rates. This is a positive finding given the current gender imbalance found in many aspects of the discipline (Mershon and Walsh 2015; Monroe and Chiu 2010). It must be noted, however, that these results should be interpreted with caution. First, the finding that OA can negate the gender citation advantage is surprising in light of previous research on GCEs. Nothing in the data provides solid evidence about why the positive OACE in political science appears to be gender-neutral. This must be researched further to determine whether it is an artifact of the data, whether the pattern holds when other data are used, and whether the pattern holds once self-archiving becomes more commonplace in political science.

Second, as with any single-discipline study, the results may lack generalizability. There is considerable evidence that GCE varies by discipline; therefore, it is important to remember that results from studies of GCE in OA publishing also are likely to vary by discipline. Thus, it will be important to explore this issue in both cross-disciplinary and within-discipline contexts.

Furthermore, the data used in this study do not include author rank, yet the American Political Science Association (2011) reported that women are concentrated at junior ranks, and Maliniak, Powers, and Walter (2013) indicated that untenured women are even less likely to be cited than their tenured colleagues. This may indicate that there will be different effects at different ranks; this is an interesting opportunity for future research. Future studies also could consider other gender-related factors such as article submission and acceptance rates, as well as other OA-related factors including the relative permissiveness of publishers’ self-archiving policies and the social-networking effects of sites such as Academia.edu and ResearchGate. Finally, the data analyzed in this study do not include authors’ race. As the recent PS: Political Science & Politics symposium on diversity in political science made clear, scholars of color continue to be underrepresented and marginalized in political science, and they are leaving the discipline in large numbers (Alexander-Floyd 2015; Mershon and Walsh 2015; Sinclair-Chapman 2015). Although Sinclair-Chapman (2015) noted that further research on retention is necessary, she clarified that conditions in the academy undermine scholars of color—particularly women of color—at all academic ranks. Although citation rates are only one part of the academy, they have a major influence on hiring, promotion, and tenure. To the extent that citation rates vary by race, that variation may perpetuate the underrepresentation of scholars of color in the discipline.

Although this is a limited study, it is an important first step in exploring the intersection of OACE and GCEs. This article provides initial evidence that by making their work OA, female political scientists can reduce the gender-citation gap, and it opens a new line of research into OACE and author gender. Finally, the research presented in this article provides additional insight into gendered patterns in political science, thereby contributing to the ongoing conversation about diversifying the discipline.