We appreciate that Firth and colleagues undertook writing a systematic review on nutritional interventions for schizophrenia (Firth et al. Reference Firth, Stubbs, Sarris, Rosenbaum, Teasdale, Berk and Yung2017). Our concern is finding all relevant original studies for a systematic review.
The paper claims electronic searches were made for several databases from inception to July 2016. After identification, screening and assessment for eligibility, 18 studies were included, the oldest from 1990.
As database searches were not restricted to certain years, we notice that some papers published before 1990, which probably fulfilled the authors’ criteria were not included. Thus, we are interested if there may be relevant studies not identified in the data extraction process. We find the search words in Supplement 1 highly relevant, but the full search strategy is not described. As the reader is not informed which studies were excluded, providing an extensive search strategy would have facilitated replications of the search.
We found two reviews with a focus comparable with the Firth review. In 1991 a paper (Kleijnen & Knipschild, Reference Kleijnen and Knipschild1991) reviewed controlled trials of the vitamins B3 and B6 in mental disorders. For schizophrenia 17 trials in adults, published between 1955 and 1978, were included. Nine studies were described as randomized controlled trials and double-blinded (Hoffer et al. Reference Hoffer, Osmond, Callbeck and Kahan1957; Hoffer, Reference Hoffer1962; Ramsay et al. Reference Ramsay, Ban, Lehman, Saxena and Bennet1970; Ananth et al. Reference Ananth, Vacaflor, Kekhwa, Sterlin and Ban1972; Reference Ananth, Ban and Lehmann1973; McGrath et al. Reference McGrath, O'Brien, Power and Shea1972; Altman et al. Reference Altman, Mehta, Evenson and Sletten1973; Wittenborn et al. Reference Wittenborn, Weber and Brown1973; Deutsch et al. Reference Deutsch, Ananth and Ban1977). However, one reference (Hoffer, Reference Hoffer1962) is a book and may not have been eligible for the Firth review. We are concerned whether the other eight original publications, were excluded or not identified, respectively, in the Firth paper.
In 2011 a systematic review (Smedslund & Berg, Reference Smedslund and Berg2011) included 22 studies from 1957 to 2006 with vitamin interventions. Eight of these were the papers referred above from the Kleijnen review. Four studies were not included in the reviews by Firth or Kleijnen. Two of these were indexed in PubMed (Lohr & Caligiuri, Reference Lohr and Caligiuri1996; Vaughan & McCohagny, Reference Vaughan and McCohagny1999), and two were found in Google Scholar (Joshi & Eswaran, Reference Joshi and Eswaran1980; Lerner & Miodownik, Reference Lerner and Miodownik2007).
Thus, we found 12 studies included by Smedslund but not by Firth. Inclusion of any of these original studies may or may not have had implications for the main results. However, to be able to draw sound conclusions on the body of evidence, all relevant original studies meeting a review's criteria have to be retrieved and included in a systematic review. Otherwise, results may be incomplete or false.
Even if librarians or information specialists participate in database searching, as recommended (Higgins & Green, Reference Higgins and Green2008), older studies may not be found for different reasons. Database indexing may be incorrect or non-existent and papers may lack abstract. However, studies should not be discarded because of their age. Systematic reviews are becoming increasingly common, and maintaining a high quality is important.
