Chapter 3 - Realist Critiques and Manifesto for Evaluation and Reviews

Summary

Realist evaluators argue that evaluations need to ask not just what works but also what works for whom under what conditions. They argue interventions need to be evaluated in terms of the mechanisms they trigger and how these interact with context to generate different outcomes in different settings or populations. Hypotheses should be worded as context–mechanism–outcome configurations (CMOCs). Many realist evaluators argue that randomised trials are not a proper scientific design, do not encompass sufficient variation in contexts to test CMOCs and are inappropriately positivist in orientation. They argue that it is better to test CMOCs using observational designs which do not use randomisation. We welcome the focus on CMOCs but disagree with the view that trials cannot be used for realist evaluation. Trials are an appropriate scientific design when it is impossible for experimenters to control all the factors which have an influence on the result of an experiment. Trials can include sufficient variety of contexts to test CMOCs. Trials need not embody a positivist approach to the science of complex health interventions if they are oriented towards testing hypotheses, draw on theory which engages with deeper mechanisms of causation and use distinctly social science approaches such as qualitative research.