Author's reply

In the barrage of recent letters, the sentiments have varied widely and the many supportive arguments presented are worthy of additional comment; however, given space limitations, I have decided to focus on the criticisms to help ensure the results are given the attention deserved.

There are some comments that I believe are without basis and may not have been made with a more careful, less emotional read of the article. For example, Littell & Coyne suggested that scientific standards for systematic reviews were not followed. The protocol employed is detailed in the methodology section and the strategy was in line with recommendations in the Handbook of Research Synthesis and Meta-Analysis. Rather than focus on these types of comments, I address criticisms requiring more information from me to allow readers to make informed decisions regarding the merit of the issues raised.

The studies included in the meta-analysis have a relatively high degree of heterogeneity given the demographic and cultural differences in sampling, the variability in control groups and outcomes, and differences in third variable controls. Counter to the claim by Polis et al, heterogeneity was addressed by employing a random effects model. The random effects model yields an estimate of the mean of a distribution of true effects; whereas in the fixed effects model there is an assumption that all the included studies share one common effect. When assigning weights to studies in a fixed effects model, the smaller ones are afforded less importance, since the same effect is believed to be more precisely assessed in the larger studies. In contrast, in the random effects model, individual studies of varying sizes contribute data from distinct populations, all of which must be considered in the pooled estimate. Weighting is therefore more balanced in the random effects v. fixed effects model, with smaller studies given relatively more emphasis. In recognition of the heterogeneity, I not only employed the random effects model, but I ran separate meta-analyses based on distinct comparison groups and outcomes.

Goldacre & Lee provided a funnel plot analysis and presented it as evidence of publication bias. However, the funnel plot is largely inappropriate for heterogeneous meta-analyses, wherein studies are not likely from a single underlying population, ^1–4 and several investigators have warned that use of funnel plots with meta-analyses derived from heterogeneous samples may result in false-positive claims of publication bias. ^1–4 When funnel plot asymmetry is detected in a heterogeneous meta-analysis, the cause is likely to be essential differences between the smaller and larger studies. For example, the majority of the smaller studies included in my meta-analysis employed substance use outcome variables and these outcomes tend to yield the strongest, most robust effects. ^5,6 In addition, the larger studies were more likely than the smaller studies to include actual diagnoses for disorders, rarer events than cut-off scores on single surveys. In the context of this meta-analysis, the funnel plot most certainly does not provide evidence of publication bias.

My experience attempting to locate unpublished data/studies on abortion and mental health has been very disheartening over the past 15 years, with virtually all requests ignored. I suspect that reluctance to share unpublished data is an attempt to keep results that challenge contemporary views on abortion and indicate significant increased risks for adverse psychological effects out of the public domain. In contrast, I believe energy is likely invested in seeing to it that non-significant findings, suggesting abortion carries no increased psychological risks, find their way into the journals. If there is any topic wherein many editors, researchers and professional organisations are highly motivated to publish non-significant effects, it is this one, rendering publication bias less common than in other areas. Support for this notion can be found in the American Psychological Association’s (APA’s) 42-year history of abortion advocacy.

In 1969, the APA passed a resolution which made the pro-choice political position the organisation’s official stance and declared abortion a civil right. For decades the APA has aligned itself with major organisations with pro-choice social agendas, frequently submitting amicus briefs and providing congressional testimony. Martel ⁷ recently discussed the APA’s position on abortion, among other issues, noting that the organization’s stance has led them to promote psychological research and disseminate data to lawmakers to inform the public and advocate for societal change. Martel further pointed out that the political stance of the APA lacks the strong backing of empirical data. With this long history of abortion advocacy by the strongest professional psychology organisation in the world, politically motivated efforts to publish null findings to support and legitimise their position is logical.

As indicated under the methodology section of the meta-analysis, studies identified using the Medline and PsycINFO databases were included based on sample size, comparison groups, outcome variables, controls for third variables, use of odds ratios, and publication in English in peer-reviewed journals between 1995 and 2009. In an effort to isolate the effect of abortion on mental health, use of comparisons groups and controls for third variables are basic methodological requirements consistent with the Bradford Hill criteria. ⁸ The majority of studies meeting these criteria and incorporated into the meta-analysis also had many other strong methodological features (multiple data points, nationally representative samples, etc.). I purposely avoided selecting from among the many more peripheral methodological criteria that could be argued as a necessary basis for including or excluding studies, when there is not universal agreement regarding strengths necessary to consider a study’s results sufficiently reliable and valid, nor is there consensus on the particular deficiencies necessary for the wholesale dismissal of a study.

Using the criteria outlined above, a significant proportion of the included studies (11/22) were articles that I authored or co-authored. However, having published 33 peer-reviewed articles, I believe I am more widely published on this topic than any other researcher in the world. It makes sense, therefore, that I am a co-author on a significant proportion of the included studies. Moreover, no studies satisfying the inclusion criteria were left out of the analyses.

Curiously the issue of my not including a study by Danish researchers Munk-Olsen et al published in the New England Journal of Medicine ⁹ was raised despite the fact that the paper came out long after my meta-analysis was completed and submitted for publication. Incidentally, their paper is presented as offering more reliable conclusions than the meta-analysis. However, there are several problematic features of this study. To begin with, Munk-Olsen et al note that previous studies lack controls for third variables, but the only third variables they consider are age and parity. There are no controls for pregnancy intendedness, pressure to abort, marital status, income, education, exposure to violence and other traumas, etc. Many studies have been deemed inadequate based on only one of these variables not being accounted for (see APA Task Force Report ¹⁰ ). The data indicated that rates of mental health problems were higher after abortion compared with childbirth (15.2% v. 6.7%); however, the generally comparable rates before and after abortion were used to negate a possible causal link between abortion and mental health. This reasoning is problematic as many women were likely disturbed to the point of seeking help precisely because they were pregnant and contemplating an abortion or they were involved in troubled relationships. These factors may have resolved, yet disturbance rates remained elevated because of the impact of the abortion. Further, the Danish Civil Registration System contains over 40 years of data, but the researchers curiously compressed the study period to 12 years. A more appropriate strategy would have been to include all women experiencing an abortion, a birth, or no pregnancy and then compare pre- and post-pregnancy mental health visits with statistical controls for all psychiatric visits pre-dating conception and all other relevant third variables described above.

A quote from a presentation I gave at the annual meeting of the American Association of Prolife Obstetricians and Gynecologists was used by Goldacre & Lee to label me an ‘anti-abortion campaigner’. This out of context comment was part of a broader call for more concerted efforts to create environments wherein objective scientists are able to make the psychology of abortion a priority. Once strong synopses of the best evidence are published, the data can and should be used to intelligibly inform policy. I am opposed to professional organisations such as the APA creating a culture wherein it is perfectly acceptable for any political position (in this case pro-choice) to drive data collection efforts, restrict grants to researchers committed to a political agenda, serve as journal gatekeepers to block publication of findings that are not consonant with the political agenda, and ultimately use the biased information assembled to back policy.

I do not hold membership in any political organisations and my work has never been funded by any pro-life group. My expertise tends to be called upon by the pro-life community and unfortunately I am never asked to present my research or perspective on the literature to groups committed to a pro-choice political position. As a professor at a public university, what motivates me is simply the desire to foster high-quality research and reach as many people as one individual can with an accurate appraisal of the literature, given the biases that permeate the study of abortion and dissemination of information through the usual channels. I do not have many graduate students working with me or large grants, and it is alarming that a researcher with such modest resources was the first to conduct a major quantitative review.

Rather than hurling unfounded accusations of personal bias, we need to more effectively utilise the well-established methods of science to fairly scrutinise the methodologies of individual studies, expand the empirical investigation of abortion and mental health, and develop a consensus-based standardised set of criteria for ranking studies meriting inclusion in reviews. Without agreement, the selected standards may be used to manipulate conclusions. For example, the ranking system employed by Charles et al ¹¹ ignored two central methodological considerations in prospective research designs: (a) percentage consenting to participate (no information was provided by the authors of the Gilchrist et al ¹² study that this team ranked as ‘Very Good’); and (b) retention of participants over time. In the Gilchrist et al study, only 34.4% of the termination group and 43.4% of the group that did not request termination were retained. A major problem with nearly all the recently published narrative reviews was somewhat arbitrary exclusion criteria at best and the purposeful selection of specific criteria resulting in dismissal of large bodies of evidence with politically incorrect results at worst.

By raising concerns of publication bias and attempting to undermine the credibility of an individual researcher who managed to publish in a high-profile journal, several people have sought to shift attention from the truly shameful and systemic bias that permeates the psychology of abortion. Professional organisations in the USA and elsewhere have arrogantly sought to distort the scientific literature and paternalistically deny women the information they deserve to make fully informed healthcare choices and receive necessary mental health counselling when and if an abortion decision proves detrimental.