Global research and effective interventions
Cognitive–behavioural therapy (CBT) and other structured, manualised psychotherapies have revolutionised the treatment of common mental disorders, such as anxiety and depression, panic disorder, agoraphobia, obsessive–compulsive disorders, post-traumatic stress disorder, conversion disorders and psychosexual disorders. CBT may also have value in the care of people with psychoses, specifically to tackle delusions and hallucinations, although the evidence for this is less convincing than that for non-psychotic disorders. As the armoury of psychotherapies has shown benefit, so clinicians and patients are excited by the possibilities and opportunities presented, especially the suggestion that there may be alternatives to purely pharmacological approaches for patients who do not wish to rely on medication, whose illnesses do not fully respond to medication or who experience adverse effects. Choice of treatment modalities in psychiatry is key to maximising the clinician–patient therapeutic alliance and optimising outcomes.
In low- and middle-income countries the treatment gap is so great that the majority of people with mental illnesses, common mental disorders and psychoses, receive no care. This is largely because of significant constraints in resources, including trained professionals. It may also relate in part to concerns about the transferability of new research findings across diverse social, cultural and geo-political contexts, as well as the variations in health literacy and health beliefs of people in different countries.
China is a case in point. China has been bold in modernising mental healthcare, establishing a new mental health act, moving rapidly to community care models and tenaciously adopting psychotherapies in the armoury of interventions. Importantly, China is also increasingly active in undertaking original research including randomised trials suited to local social and cultural contexts that may differ in design and outcome from those of pioneering original studies in high-income countries. The levels of publicly funded services and aspirations for universal coverage also vary, even within high-income countries.
Research is critical in the development of interventions to prevent and treat illnesses, and to reduce associated disability. Patients, caregivers, clinicians, and commissioners of research and services expect research to be undertaken to the highest standards, with careful design and execution, as well as transparency in accurate reporting and considered interpretation.
Research integrity: a case study
Published in BJPsych, following two rounds of peer review and revisions, Guo and colleagues concluded in their paper that brief CBT had a positive effect on Chinese patients with schizophrenia.Reference Guo, Li, Ma, Sun, Guo and Li1 The abstract stated:
‘At the post-treatment assessment and the 12-month follow-up, patients who received brief CBT showed greater improvement in overall symptoms, general psychopathology, insight and social functioning. In total, 37.3% of those in the brief CBT plus TAU group experienced a clinically significant response, compared with only 19.1% of those in the TAU alone group (P = 0.003).’Reference Guo, Li, Ma, Sun, Guo and Li1
Following publication, some readers expressed concerns about the reporting of the findings, suggesting there was undue emphasis on the positive rather than negative findings. Initially, we considered this to be a reporting issue. The editorial board considered the paper in greater detail, and it transpired that there were several sources of concern regarding the conduct of the trial, and how it had been reported.
We offered the authors an opportunity to clarify the concerns we had unearthed, and if appropriate they should issue a corrigendum presenting a more balanced account of the findings and the limitations of the paper, as well as acknowledging the failings in the trial registration, execution, reporting and interpretation. Detailing this further after careful editorial review, we identified the following issues concerning research integrity.
(a) The trial was registered retrospectively: the authors state that the study was conducted between 1 August 2010 and 1 July 2013. Ethics approval was secured on: 11 April 2011. The first patient was recruited in May 2011.
(b) The published trial did not conform to the sample sizes, methods, inclusion criteria and implementation of trial procedures, as set out in the trial registry.
(c) The primary outcomes reported in the paper did not match those listed in the registry entry.
(d) Positive comparisons were reported, whereas numerous negative comparisons of similar importance were not.
After discovering these irregularities and raising our concerns with the authors, they offered several explanations in correspondence; for example, that the systems of trials registry in China were not as strict as in other parts of the world; nor were ethics committee reviews of psychotherapy trials as stringent; that the team were unfamiliar with the need for prospective trials registration and retrospectively registered the trial at the earliest point. The authors apologised and reassured us that they acted in good faith but were essentially inexperienced. However, there were co-authors involved in the trial that were experienced in trials design, and they were based in the UK. They responded that these collaborators and co-authors had been involved in advising the authors in China on trial execution and analyses, but were not acting as guarantors.
The editorial board deliberated upon these matters, the authors' responses and the possibility of retraction as well as taking advice from the Committee on Publishing Ethics. However, despite much discussion with the authors, and several iterations, we were not satisfied that the proposed corrigendum did justice to the concerns. Primarily, this was because the authors continued to emphasise the few positive findings from their study, and overlooked a lack of oversight on the execution of the trial and necessary adherence to a prespecified protocol and publication plan. Therefore, the paper has been retracted.
This matter has been successfully resolved but it raised some important aspects for the journal and our internal processes. We learned some valuable lessons and feel we assured that the integrity of the research published in the journal is maintained by such scrutiny. Going forward we intend to issue clearer guidance that trials (and systematic reviews) must be pre-registered and that the authors present all the findings, positive and negative. Abstracts may now be up to 250 words in length in order to accommodate a more balanced overview of the findings; outcome switching is not permitted and that the International Committee of Medical Journal Editors authorship guidelines must be strictly followed. Our investigation also shows that the desire to encourage and publish research from low- and middle-income countries must be balanced with the need for quality and adherence to the same high standards of research integrity that are expected of all research communities. Trials should not be undertaken lightly, and only by groups that can fully conduct the trial to the highest professional standards, and ensure adherence to ethical and research integrity mandates. Such changes must be taken up by researchers and scholarly societies in equal measure in high- and low-income countries. All must maintain the same standards.
Our investigation also revealed issues about our internal processes for peer review and the need for enhanced editorial scrutiny. Consequently, alongside issuing more detailed guidance for authors, critically, we have made important changes to our editorial review processes as well the level of scrutiny we now give to registered protocols and reporting. Going forward, we will require clear statements on pre-registered protocols and close adherence to these in the conduct and reporting of the study.
It is important to note that this process naturally took some time to implement and follow through and so in the interim we issued a notice of concern in the journal while we investigated the irregularities reported.Reference Guo, Li, Ma, Sun, Guo and Li2 Following a detailed investigation of the research and publication process our overall conclusion is that – we cannot with confidence say that CBT is beneficial for community patients living with schizophrenia in China. Furthermore, although in this instance we identified a cluster of concerns, such as incomplete reporting and outcome switching. These issues are probably more common than we suspect, and better safeguards are needed and need to be applied more widely.
We consider that transparency and integrity must also be modelled in our handling of such matters, and in our reflections on the weaknesses of peer review and editorial review. We hope our processes and the outcome of our investigations are instructive not only for psychiatric research, but more broadly. However, we can only ensure the highest standards of ethics in science through partnership with the research community, and ultimately it is up to individuals conducting research to act with honesty and good faith in all matters. To this end we offer workshops on peer review and publications already, and propose to offer further training for researchers and authors on research integrity and expected standards of conduct and reporting research findings.
eLetters
No eLetters have been published for this article.