Abortion and mental health: guidelines for proper scientific conduct ignored

In her review of research on the mental health effects of abortion, Coleman ¹ stated:

‘In this highly politicised area of research it is imperative for researchers to apply scientifically based evaluation standards in a systematic, unbiased manner when synthesising and critiquing research findings. If not, authors open themselves up to accusations of shifting standards based on conclusions aligned with a particular political viewpoint. Moreover, the results may be dangerously misleading and result in misinformation guiding the practice of abortion.’

However, Coleman failed to follow well-accepted scientific standards for the conduct and reporting of systematic reviews and meta-analyses. Further, Coleman’s failure to state her obvious conflicts of interest in this review raises serious questions about biases in her analysis. Hence, the review is open to serious questions about the author’s scientific standards, methods, political viewpoints, and potentially misleading conclusions.

Widely accepted standards for systematic reviews and meta-analyses are contained in the published AMSTAR, MARS, MOOSE, and PRISMA statements. ^2–5 None of these standards were cited or followed by Coleman. AMSTAR is the only validated instrument for assessing the methodological quality of systematic reviews and meta-analysis. We assessed Coleman’s review according to the AMSTAR statement, and found that it failed to meet any of the eleven basic requirements for systematic reviews and meta-analysis included in AMSTAR.

Following AMSTAR, specific flaws of the Coleman review are as follows:

1. 1 there was no public a priori design

2. 2 there was no duplicate study selection or duplicate data extraction

3. 3 the author did not describe the search strategy in sufficient detail

4. 4 the review was limited to published studies, contrary to all published standards

5. 5 a list of excluded studies was not provided

6. 6 the author did not provide sufficient descriptive information on included studies, including demographic characteristics of participants

7. 7 the scientific quality of included studies was not documented

8. 8 scientific quality of included studies was not considered in formulating conclusions

9. 9 appropriate methods were not used in combining the findings of studies (Coleman clearly violated the rule for avoiding dependencies in meta-analysis, when she synthesised 36 effects from 22 studies in Fig. 1)

10. 10 the likelihood of publication bias was not assessed

11. 11 conflicts of interest and sources of support were not acknowledged (no financial disclosures were made and no other potential conflicts were acknowledged).

An article in the British Journal of Psychiatry ⁶ calls attention to the importance of non-financial conflicts of interest in the psychiatric literature. Coleman has at least two types of conflict of interest here. Among the most important of such conflicts is an agenda-driven bias, by which authors seek to influence legislation and social policy. David Reardon is a co-author with Coleman on seven articles included in the review and an author on an additional study in the review that does not involve Coleman as a co-author. Reardon is quite explicit about his agenda to instil fear of abortion as a way of facilitating passage of anti-abortion legislation. ⁷

Coleman is the first author on 6 studies and co-author on 5 additional studies in her review; thus, she authored or co-authored fully half of the 22 studies included. According to the Cochrane Handbook, ⁸ this is another potential conflict of interest, since it may ‘unduly influence judgements made in a review (concerning, for example, the inclusion or exclusion of studies, assessments of the risk of bias in included studies or the interpretation of results)… This should be disclosed in the review and, where possible, there should be an independent assessment of eligibility and risk of bias by a second author with no conflict of interest.’ Coleman did not obtain an independent assessment of the studies she authored or co-authored, nor did she acknowledge these conflicts in the review.

Coleman’s conclusion that the results of the studies in her review are ‘quite consistent’ (p. 183) is belied by visual inspection of the Forest plots, which include non-overlapping confidence intervals. Coleman should have reported results of heterogeneity tests (chi-squared and I ²), which probably would have shown significant heterogeneity in results across studies (presumably that is why she chose the random effects model).

Some of the commentaries on Coleman’s review appear to be uninformed by current scientific standards for reviews. Comments by Fergusson et al are particularly misleading. Faced with variations in the methodological quality of available studies, it is essential for reviewers to weed out weaker studies. Valid conclusions can only be based on valid studies.

It is unclear how this paper got through peer review at the Journal. It appears that peer reviewers and the Editor ignored published standards for systematic reviews and meta-analyses. Given the serious methodological flaws contained in Coleman’s review and the author’s failure to report obvious conflicts of interest, we believe the article should be retracted.