15 - Bringing it all together: the future of evidence synthesis

Summary

What Works Centres and other similar organisations, which we’ll call ‘knowledge brokerage intermediaries’, are concerned with research evidence that can be useful to people making decisions, when ‘useful’ means being relevant to the problem being faced, offering reliable findings and being available before decisions are made. Turning to a single study is not enough. An individual study may have flaws or vagaries that, by design or chance, result in misleading findings, or it may be conducted in a setting that differs so much from the decision being faced it is barely relevant. These difficulties can be overcome, at least in part, by relying on collections of studies addressing the same or a similar issue. These are systematic reviews of research, or research syntheses.

What Works Centres all support decision makers by taking on the task of identifying relevant studies and checking their quality before pooling and summarising their findings to conclude what is known from existing studies. They make this work publicly available as systematic reviews for any decision makers to access. Typically, What Works Centres address questions about the effects of intervening in people's lives, for instance with education or social care policies or practices, to find out what works. They gather studies addressing causal questions, mainly controlled trials and particularly randomised controlled trials, and quasi-experimental studies that construct fair comparisons of different policies or practices through statistical means. However, the types of studies available to the different centres vary. Far more randomised controlled trials address questions about what works in medicine than in other fields, such as homelessness or wellbeing. Consequently, What Works Centres need different methods to find studies, appraise their quality or synthesise their findings. Indeed, in newly emerging research areas, where high-quality studies addressing what works questions are really sparse, important prior steps are finding out about the nature and the scale of the problem, before asking what interventions for tackling the problem are well designed, feasible and acceptable. When evidence about what works is plentiful, and policy decisions are made, the next step is learning from existing research what challenges may be faced when implementing policy, and how they can be overcome.