Hostname: page-component-cd9895bd7-gvvz8 Total loading time: 0 Render date: 2024-12-27T12:47:02.108Z Has data issue: false hasContentIssue false
  • English
  • Français

The State of Evidence-Based Policy Evaluation and its Role in Policy Formation

Published online by Cambridge University Press:  26 March 2020

Philip Davies
Philip Davies*
Affiliation:
Oxford Evidentia Limited
*
1 E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper argues that evidence-based policy has clearly made a worldwide impact, at least at the rhetorical and institutional levels, and in terms of analytical activity. The paper then addresses whether or not evidence-based policy evaluation has had an impact on policy formation and public service delivery. The paper uses a model of research-use that suggests that evidence can be used in instrumental, conceptual and symbolic ways. Taking four examples of the use of evidence in the UK over the past decade, this paper argues that evidence can be used instrumentally, conceptually and symbolically in complementary ways at different stages of the policy cycle and under different policy and political circumstances. The fact that evidence is not always used instrumentally, in the sense of “acting on research results in specific, direct ways” (Lavis et al., 2003, p. 228), does not mean that it has little or no influence. The paper ends by considering some of the obstacles to getting research evidence into policy and practice, and how these obstacles might be overcome.

Keywords

Evidence-based policypolicymakingpublic service deliverydelivery trajectoriesH11H43I28I38J62
Type
Research Articles
Information
National Institute Economic Review , Volume 219 , January 2012 , pp. R41 - R52
Copyright
Copyright © 2012 National Institute of Economic and Social Research

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

JEL Classifications: H11; H43; 128; 138; J62

References

Amara, N., Ouimet, M., Landry, R. (2004), ‘New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies’, Science Communication, 26, pp. 76–106.CrossRefGoogle Scholar
Balas, E.A., Boren, S.A. (2000), ‘Managing clinical knowledge for health care improvement’, in Yearbook of Medical Informatics 2000: Patient-Centered Systems, Stuttgart, Germany, Schattauer, pp. 65–70.CrossRefGoogle Scholar
Barber, M. (2007), Instruction to Deliver: Tony Blair, the Public Services and the Challenge of Achieving Targets, London, Methuen Publishing Ltd.Google Scholar
Best, A., Holmes, B. (2010), ‘Systems thinking, knowledge and action: towards better models and methods’, Evidence and Policy, 6, 2, pp. 145–59.CrossRefGoogle Scholar
Beyer, J.M. (1997), ‘Research utilization: bridging the gap between communities’, Journal of Management Inquiry, 6, 1, pp. 17–22.CrossRefGoogle Scholar
BIS (2011a), Impact Assessment Guidance When to do an Impact Assessment, London, Department of Business, Innovation and Skills, available at: http://www.bis.gov.uk/assets/biscore/betterregulation/docs/i/11-1111-impact-assessment-guidance.pdf.Google Scholar
BIS (2011b), IA Toolkit How to do an Impact Assessment, Department of Business, Innovation and Skills, available at: http://www.bis.cov.uk/assets/biscore/better-regulation/docs/i/11-1112-impact-assessment-toolkit.pdf.Google Scholar
Bryson, A., Dorsett, R., Purdon, S. (2002), The Use of Propensity Score Matching in The Evaluation of Active Labour Market Policies, Working Paper Number 4, London, Department for Work and Pensions.Google Scholar
Cabinet Office (1999a), Modernising Government White Paper, London, Cabinet Office.Google Scholar
Cabinet Office (1999b), Professional Policy Making for the Twenty-First Century, London, Cabinet Office.Google Scholar
Cabinet Office (2000), Additing It Up: Improving Analysis and Modelling, London, Cabinet Office.Google Scholar
Cabinet Office (2003), Adding It Up: Improving Analysis and Modelling in Central Government, London, Cabinet Office.Google Scholar
Campbell, D. (2005), ‘Getting a ‘GRIPP’ on the research-policy interface in NSW’, New South Wales Public Health Bulletin, 16, 10, pp. 154–6.Google Scholar
Campbell, S., Benita, S., Coates, E., Davies, P., Penn, G. (2007), Analysis for Policy: Evidence-Based Policy in Practice, London, Government Social Research Unit.Google Scholar
Chowdry, H., Emmerson, C. (2010), An Efficient Maintenance Allowance?, London, Institute for Fiscal Studies, available at: http://www.ifs.org.uk/publications/5370.Google Scholar
CHSRF (2001), ‘Communication notes: reader-friendly writing — 1:3:25’, Ottawa, Canadian Health Services Research Foundation, available at: http://www.chsrf.ca/knowledge_transfer/communication_notes/comm_reader_friendly_wwriting_e.php.Google Scholar
CIHR (2006), Evidence in Action, Acting on Evidence: A casebook of health services and policy research knowledge translation stories, Canadian Institute of Health Services and Policy Research, Ottawa, Canada, available at: www.cihr-irsc.gc.ca/e/documents/ihspr_ktcasebook_e.pdf.Google Scholar
Davies, P.T. (2004), ‘Is Evidence-Based Government Possible?’, Jerry Lee Lecture to the 4th Annual Campbell Collaboration Colloquium, Washington D.C., 19 February.Google Scholar
Dearden, L., Emmerson, C., Frayne, C., Meghir, C. (2001), Education Maintenance Allowance: The First Year — A Quantitative Evaluation, London, Department for Education and Skills (now archived at The National Archives website).Google Scholar
Dearden, L., Emmerson, C., Frayne, C., Meghir, C. (2008), ‘Conditional cash transfers and school dropout rates’, The Journal of Human Resources, 44, 4, pp. 827–57.Google Scholar
Dorsett, R., Campbell-Barr, V., Hamilton, G., Hoggart, L., Marsh, A., Miller, C., Phillips, J., Ray, K., Riccio, J.A., Rich, S., Vegeris, S. (2007), Implementation and First-Year Impacts of The UK Employment Retention and Advancement (ERA) Demonstration, Research Report 412, London, Department for Work and Pensions.Google Scholar
Fiszbein, A., Shady, N. (2009), Conditional Cash Transfers: Reducing Present and Future Poverty, Washington D.C, The World Bank.CrossRefGoogle Scholar
Gabbay, J., Le May, A. (2004), ‘Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care’, British Medical Journal, 329, pp. 1013–8.CrossRefGoogle ScholarPubMed
Greenhalgh, T., Robert, G., Bate, P., Macfarlane, F. and Kyriakidou, O. (eds) (2005), Diffusion of Innovations in Health Service Organisations: A Systematic Literature Review, Oxford, Blackwell Publishing Ltd.CrossRefGoogle Scholar
GSR (2009), Rapid Evidence Assessment Toolkit, London, Government Social Research Service, available at: http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence-assessment.Google Scholar
Hendra, R., Riccio, J.A., Dorsett, R., Greenberg, D.H., Knight, G., Phillips, J., Robins, P.K., Vegeris, S., Walter, J., Hill, A., Ray, K., Smith, J. (2011), Breaking the Low-Pay, No-Pay Cycle: Final Evidence from the UK Employment, Retention and Advancement (ERA) Demonstration, London, Department for Work and Pensions.Google Scholar
Treasury, HM (2003), The Green Book: Appraisal and Evaluation in Central Government, London, HM Treasury.Google Scholar
HM Treasury (2011), The Magenta Book: Guidance for Evaluation, London, HM Treasury.Google Scholar
IFS (1999), Education Maintenance Allowance (EMA) Evaluation, Institute for Fiscal Studies, London (summary available at: http://www.ifs.org.uk/projects/98).Google Scholar
Jowell, R. (2003), The Role of ‘Pilots’ in Policy-Making. Report of a Review of Government, Government Social Research Unit, London, HM Treasury.Google Scholar
Lavis, J., Davies, H., Oxman, A., Denis, J-L., Golden-Biddle, K., Ferlie, E. (2005), ‘Towards systematic reviews that inform health care management and policy-making’, Journal of Health Services Research & Policy, 10, Supplement 1, pp. 35–48.CrossRefGoogle ScholarPubMed
Lavis, J.N., Robertson, D, Woodside, J.M., McLeod, C.B., Abelson, J. and the Knowledge Transfer Study Group (2003), ‘How can research organizations more effectively transfer research knowledge to decision makers?’, The Milbank Quarterly, 81, 2, pp. 221–48.CrossRefGoogle ScholarPubMed
Leigh, A. (2009), ‘What evidence should social policymakers use?’, Australian Treasury Economic Roundup, 1, pp. 27–43.Google Scholar
Lomas, J. (2000), ‘Connecting research and policy’, Canadian Journal of Policy Research, Spring, pp. 140–4.Google Scholar
Lomas, J, Culyer, T., McCutcheon, C., McAuley, L., Law, S. (2005), Conceptualizing and Combining Evidence for Health System Guidance: Final Report, Canadian Health Services Research Foundation, Ottawa.Google Scholar
Marsh, D. (2006), ‘Evidence-based policy: framework, results and analysis from the New Zealand biotechnology’, International Journal of Biotechnology, 8, 3–4, pp. 206–24.CrossRefGoogle Scholar
Martin, J.P. (2000), ‘What works among active labour market policies: evidence from OECD countries' experiences’, OECD Economic Studies, 30, pp. 80–113.Google Scholar
Milani, C.R.S. (2009), Evidence-Based Policy Research: Critical Review of Some International Programmes on Relationships Between Social Science Research and Policy-Making, Paris, France, UNESCO.Google Scholar
Mold, J.W., Peterson, K.A. (2005), ‘Primary care practice-based research networks: working at the interface between research and quality improvement’, Annals of Family Medicine, 3, 1, May/June 2005, S12–S20.CrossRefGoogle ScholarPubMed
NAO (2007), Evaluation of Regulatory Impact Assessments 2006–07, London, National Audit Office.Google Scholar
NAO (2009), Delivering High Quality Impact Assessments, London, National Audit Office.Google Scholar
NAO (2010), Assessing the Impact of Proposed New Policies, Report by the Comptroller and Auditor General, HC 185 Session 2010–2011, London, National Audit Office.Google Scholar
Nutley, S.M., Walter, I., Davies, H.T.O. (2007), Using Evidence: How Research Can Inform Public Services, Bristol, Policy Press.CrossRefGoogle Scholar
Obama, B.H. (2009), Inaugural Address, Washington DC, 20 January, available at: http://www.whitehouse.gov/the-press-office/president-barack-obamas-inaugural-address.Google Scholar
O'Connor, T. (2008), How The Prime Minister Monitors Performance and Assesses Delivery, Presentation to GORSInduction, Tony O'Connor CBE, Chief Operational Research Analyst, Prime Minister's Delivery Unit, 8 May.Google Scholar
Office of the Presidency (2010), Guide To The Outcomes Approach, Pretoria, Office of the Presidency of South Africa.Google Scholar
Ouimet, M., Landry, R., Ziam, S., Bédard, P. (2009), ‘The absorption of research knowledge by public servants’, Evidence and Policy, 5, 4, pp. 331–50.CrossRefGoogle Scholar
Purdon, S. (2002), Estimating The Impact of Labour Market Programmes, Working Paper No. 3, London, Department for Work and Pensions.Google Scholar
Quets, H., Robins, P.K., Pan, E.C., Michalopoulos, C., Card, D. (1999), Does SSP Plus Increase Employment? The Effect of Adding Services to the Self Sufficiency Project's Financial Incentives, Ottawa, Social Research Development Corporation.Google Scholar
Rosenbaum, P, and Rubin, D. (1983), ‘The central role of the propensity score in observational studies for causal effects’, Biometrika, 70, 1, pp. 41–55.CrossRefGoogle Scholar
Spielhofer, T., Golden, S., Evans, K., Marshall, H., Mundy, E., Pomati, M., Styles, B. (2010), Barriers to Participation in Education and Training, Slough, NFER.Google Scholar
Topp, L., McKetin, R. (2003), ‘Supporting evidence-based policy making: A case study of the illicit drug reporting system in Australia’, Bulletin on Narcotics, 55, 1–2, pp. 23–30, United Nations Office on Drugs and Crime, Vienna, Austria.Google Scholar
Townsend, T., Kunimoto, B. (2009), Collaboration and Culture. The Future of the Policy Research Function in the Government of Canada, Policy Research Initiative, Ottawa, Canada, March.Google Scholar
Weiss, C.H. (1982), ‘Policy research in the context of diffuse decision making’, Journal of Higher Education, 53, 6, pp. 619–39.CrossRefGoogle Scholar
Zussman, D. (2003), ‘Evidence-based policy making: some observations of recent Canadian experience’, Social Policy Journal of New Zealand, 20, pp. 64–71, June.Google Scholar