Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-02T19:57:34.498Z Has data issue: false hasContentIssue false
  • English
  • Français

Methodological quality of systematic reviews on treatments for depression: a cross-sectional study

Published online by Cambridge University Press:  02 May 2017

V. C. H. Chung ,
X. Y. Wu ,
Y. Feng ,
R. S. T. Ho ,
S. Y. S. Wong  and
D. Threapleton
V. C. H. Chung
Affiliation:
Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong Cochrane Hong Kong, The Chinese University of Hong Kong, Hong Kong
X. Y. Wu*
Affiliation:
Cochrane Hong Kong, The Chinese University of Hong Kong, Hong Kong Hong Kong Institute of Integrative Medicine, The Chinese University of Hong Kong, Hong Kong
Y. Feng
Affiliation:
Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong
R. S. T. Ho
Affiliation:
Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong
S. Y. S. Wong
Affiliation:
Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong
D. Threapleton
Affiliation:
Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong
*
*Address for correspondence: X. Y. Wu, Room 509, JC School of Public Health Building, Prince of Wales Hospital, Shatin, NT, Hong Kong. (Email: [email protected])
Get access Rights & Permissions [Opens in a new window]

Abstract

Aims.

Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.

Methods.

A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.

Results.

A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.

Conclusion.

The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.

Keywords

Depressionevidence-based medicinemeta-analysisrandomised controlled trials as topicreview literature as topic
Type
Original Articles
Information
Epidemiology and Psychiatric Sciences , Volume 27 , Issue 6 , December 2018 , pp. 619 - 627
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

These authors contributed equally to this paper.

References

American Psychiatric Association (1980). Diagnostical and Statistical Manual of Mental Disorders (DSM-III). American Psychiatric Association: Washington, DC.Google Scholar
Burda, BU, Holmer, HK, Norris, SL (2016). Limitations of a measurement tool to assess systematic reviews (AMSTAR) and suggestions for improvement. Systematic Reviews 5, 58.Google Scholar
Busse, JW, Bruno, P, Malik, K, Connell, G, Torrance, D, Ngo, T, Kirmayr, K, Avrahami, D, Riva, JJ, Ebrahim, S, Struijs, PA, Brunarski, D, Burnie, SJ, LeBlanc, F, Coomes, EA, Steenstra, IA, Slack, T, Rodine, R, Jim, J, Montori, VM, Guyatt, GH (2014). An efficient strategy allowed English-speaking reviewers to identify foreign-language articles eligible for a systematic review. Journal of Clinical Epidemiology 67, 547–553.Google Scholar
Chung, VCH, Ho, RST, Wu, XY, Fung, DHY, Lai, X, Wu, JCW, Wong, SYS (2015). Are meta-analyses of Chinese herbal medicine trials trustworthy and clinically applicable? A cross-sectional study. Journal of Ethnopharmacology 162, 47–54.Google Scholar
Cipriani, A, Barbui, C, Butler, R, Hatcher, S, Geddes, J (2011). Depression in adults: drug and physical treatments. BMJ Clinical Evidence 2011, 1003.Google Scholar
Cochrane Community (beta) (2015). Database of Abstracts of Reviews of Effects (DARE).Google Scholar
Cohen, JF, Korevaar, DA, Wang, J, Spijker, R, Bossuyt, PM (2015). Should we search Chinese biomedical databases when performing systematic reviews? Systematic Reviews 4, 23.Google Scholar
Dwan, K, Altman, DG, Arnaiz, JA, Bloom, J, Chan, AW, Cronin, E, Decullier, E, Easterbrook, PJ, Von Elm, E, Gamble, C, Ghersi, D, Ioannidis, JPA, Simes, J, Williamson, PR (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 3, e3081.Google Scholar
Egger, M, Zellweger-Zähner, T, Schneider, M, Junker, C, Lengeler, C, Antes, G (1997). Language bias in randomised controlled trials published in English and German. Lancet 350, 326–329.Google Scholar
Egger, M, Smith, GD, Sterne, JA (2001). Uses and abuses of meta-analysis. Clinical Medicine 1, 478–484.Google Scholar
Fleming, PS, Koletsi, D, Seehra, J, Pandis, N (2014). Systematic reviews published in higher impact clinical journals were of higher quality. Journal of Clinical Epidemiology 67, 754–759.Google Scholar
Hamilton, M (1960). A rating scale for depression. Journal of Neurology, Neurosurgery, and Psychiatry 23, 56–62.Google Scholar
Higgins, J, Green, S (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration. Available from www.cochrane-handbook.org.Google Scholar
Ho, RS, Wu, XY, Yuan, JQ, Liu, SY, Lai, X, Wong, SY, Chung, VC (2015). Methodological quality of meta-analyses on treatments for chronic obstructive pulmonary disease: a cross-sectional study using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. Npj Primary Care Respiratory Medicine 25, 14102.Google Scholar
Jørgensen, AW, Hilden, J, Gøtzsche, PC (2006). Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ 333, 782.Google Scholar
Jüni, P, Holenstein, F, Sterne, J, Bartlett, C, Egger, M (2002). Direction and impact of language bias in meta-analyses of controlled trials: empirical study. International Journal of Epidemiology 31, 115–123.Google Scholar
Lexchin, J, Bero, LA, Djulbegovic, B, Clark, O (2003). Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326, 1167–1170.Google Scholar
Moher, D, Tsertsvadze, A (2006). Systematic reviews: when is an update an update? Lancet 367, 881–883.Google Scholar
Moher, D, Tetzlaff, J, Tricco, AC, Sampson, M, Altman, DG (2007). Epidemiology and reporting characteristics of systematic reviews. PLoS Medicine 4, e78.Google Scholar
Moher, D, Liberati, A, Tetzlaff, J, Altman, DG (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339, b2535.Google Scholar
Moja, LP, Telaro, E, D'Amico, R, Moschetti, I, Coe, L, Liberati, A (2005). Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ 330, 1053.Google Scholar
Morrison, A, Polisena, J, Husereau, D, Moulton, K, Clark, M, Fiander, M, Mierzwinski-Urban, M, Clifford, T, Hutton, B, Rabb, D (2012). The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. International Journal of Technology Assessment in Health Care 28, 138–144.Google Scholar
Mulrow, CD, Cook, DJ, Davidoff, F (1997). Systematic reviews: critical links in the great chain of evidence. Annals of Internal Medicine 126, 389–391.Google Scholar
Mykletun, A, Bjerkeset, O, Overland, S, Prince, M, Dewey, M, Stewart, R (2009). Levels of anxiety and depression as predictors of mortality: the HUNT study. British Journal of Psychiatry 195, 118–125.Google Scholar
Page, MJ, Shamseer, L, Altman, DG, Tetzlaff, J, Sampson, M, Tricco, AC, Catalá-López, F, Li, L, Reid, EK, Sarkis-Onofre, R, Moher, D (2016). Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Medicine 13, e1002028.Google Scholar
Papageorgiou, SN, Papadopoulos, MA, Athanasiou, AE (2011). Evaluation of methodology and quality characteristics of systematic reviews in orthodontics. Orthodontics and Craniofacial Research 14, 116–137.Google Scholar
Remschmidt, C, Wichmann, O, Harder, T (2014). Methodological quality of systematic reviews on influenza vaccination. Vaccine 32, 1678–1684.Google Scholar
Sequeira-Byron, P, Fedorowicz, Z, Jagannath, VA, Sharif, MO (2011). An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS). Journal of Applied Oral Science 19, 440–447.Google Scholar
Shea, B, Grimshaw, J, Wells, G, Boers, M, Andersson, N, Hamel, C, Porter, A, Tugwell, P, Moher, D, Bouter, L (2007). Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology 7, 10.Google Scholar
Shea, B, Hamel, C, Wells, GA, Bouter, L, Kristjansson, E, Grimshaw, J, Henry, D, Boers, M (2009). AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology 62, 1013–1020.Google Scholar
Wegewitz, U, Weikert, B, Fishta, A, Jacobs, A, Pieper, D (2016). Resuming the discussion of AMSTAR: what can (should) be made better? BMC Medical Research Methodology 16, 111.Google Scholar
World Health Organization (2012). Depression. http://www.who.int/mediacentre/factsheets/fs369/en/Google Scholar
Wu, XY, Tang, JL, Mao, C, Yuan, JQ, Qin, Y, Chung, VCH (2013). Systematic reviews and meta-analyses of traditional Chinese medicine must search Chinese databases to reduce language bias. Evidence-based Complementary and Alternative Medicine: eCAM 2013, 812179.Google Scholar
Wu, XY, Du, XJ, Ho, RS, Lee, CC, Yip, BH, Wong, MC, Wong, SY, Chung, VC (2016 a). Characteristics and methodological quality of meta-analyses on hypertension treatments-a cross-sectional study. Journal of Clinical Hypertension 19, 137–142.Google Scholar
Wu, XY, Lam, VC, Yu, YF, Ho, RS, Feng, Y, Wong, CH, Yip, BH, Tsoi, K, Wong, SY, Chung, VC (2016 b). Epidemiological characteristics and methodological quality of meta-analyses on diabetes mellitus treatment: a systematic review. European Journal of Endocrinology 175, 353–360.Google Scholar
Supplementary material: File

Chung supplementary material

Chung supplementary material

Download Chung supplementary material(File)
File 61.9 KB