15 - Evaluating an Injury Intervention or Program

Summary

Introduction

Evaluation is defined by Last's Dictionary of Epidemiology as a “process that attempts to determine as systematically and objectively as possible, the relevance, effectiveness, and impact of activities in light of their objectives. Several varieties of evaluation can be distinguished; e.g., evaluation of structure, process, and outcome.” (Last et al., 1995).

No single definition captures all the aspects of program evaluation. Evaluation research shares some themes with outcomes research and with efforts to measure the quality of healthcare (Petitti, 1998; Petitti and Amster, 1998). With outcomes research, it shares a focus on effectiveness (intervention under everyday circumstances) more often than on efficacy (the effect of the intervention under ideal and tightly controlled conditions). With quality of care research, it shares the use of structural, process, and outcome (health effects) measures to examine program/intervention effects.

We emphasize the importance of linking an intervention with an evaluation plan at the outset. Without this linkage, it is often impossible to design an evaluation post hoc that will provide valid process and outcome information.

Performing Program Evaluation

Purpose of Evaluation

Evaluation is often described as having one or more of three purposes: (1) to establish or clarify program effectiveness; (2) to improve program implementation; and (3) to address administrative needs. This chapter addresses the first two purposes. Generally evaluation indicates how effectively the program was implemented (process), what it has accomplished (outcomes), and how effective it was for health (outcome) (Dannenberg and Fowler, 1998; Thompson and McClintock, 1998).

By working with the program's stakeholders, one should generate a list of the speciic questions to be answered by the evaluation.