4 - Conducting process evaluation

Summary

• • Understanding the different types of process evaluation and where to use them

• • Distinguishing between process evaluation, self-evaluation and action research

• • The focus of process evaluation – assessing achievements, effectiveness and quality

• • Understanding the main options and choices for process evaluation methodologies

• • Identifying and avoiding common pitfalls in conducting process evaluation.

Introduction

After the groundwork laid down by compilation and composition, evaluators come to conducting evaluation, with each of the three main types of evaluation considered in the following chapters. This chapter looks at process evaluations, which are often a gateway into professional evaluation where there are fewer immediate conceptual hurdles to the non-initiated than in economic or impact evaluations. The demand for process evaluations is widespread, and they remain the bedrock of understanding whether and how policies, initiatives and programmes are working, and how they might be improved.

Different types of process evaluation

As the name suggests, process evaluations concentrate on the working mechanisms of whatever is being evaluated. They are concerned with providing timely information on how well something is being, or has been, implemented against whatever expectations there are of it. Treasury guidance in the UK sees this as:

a detailed description of what interventions are involved in a service or policy, who provides them, what form they take, how they are delivered and how they are experienced by the participants and those who deliver them. It can also provide an in-depth understanding of the decisions, choices and judgments involved, how and why they are made and what shapes this. (HM Treasury, 2011, p 82)

While this definition is geared towards central government civil servants looking at how well larger-scale government-funded actions and policies are working, it can equally apply to any attempt to stocktake how well smaller-scale or localised initiatives are being put in place.

Although there is not a uniform typology, some theorists have found it helpful to define different subtypes of process evaluations, variously in terms of ‘implementation’ evaluations, ‘developmental’ evaluations and ‘managerial evaluations’ (Ovretveit, 1998). Each of these usually takes place in an organisational context and all can be regarded as process evaluations. The differences are subtle and mainly in how the evidence to be generated is to be used.