Data on the uptake of clinical guidelines into practice are essential to guide and evaluate quality improvement interventions. Organizations responsible for service specification, monitoring and improvement need to consider the practicality of and trade-offs made in different data collection methods. We examined the feasibility of deriving and applying review criteria for clinical guidelines in English primary care.

We selected two sets of guidance, on osteoporosis and depression, and used a consensus process to derive review criteria. We manually extracted data on adherence to review criteria from patient records in 20 general practices from three NHS primary care trusts in northern England. We compared the relative utility of extracted data with that of routinely available data, summarizing feasibility using what we termed a Resource Ratio.

Of 53 proposed review criteria we assessed, 41 were judged clinically important, valid, relevant and measurable. Thirty-one could be assessed in 10% or more of sampled patients, whereas 15 could be readily extracted (resource ratio of 15 or less). Only eight met all desirable attributes for use as review criteria. Resource ratios correlated poorly with local stakeholders’ prior views on feasibility of data collection. We observed wide variations in compliance with review criteria, with notably low levels among self-care standards.

A minority of guideline recommendations were suitable for review criteria development, fewer still when using routinely available data. Local stakeholders tend to underestimate the actual resource requirements of data collection. Although improved design and use of clinical records may facilitate measurement of adherence to recommended practice, detailed assessments are still likely to rely upon some degree of manual data collection in the foreseeable future.