5 - The Assessment of Clinical Expertise (ACE)

Summary

Training in psychiatry has traditionally been based on an apprenticeship model. Many years ago, during my first post as a junior trainee in psychiatry, I was very fortunate that my first educational supervisor was a particularly skilled clinician and educator. For the first three out-patient clinics I undertook for her, she sat with me through my fumbling attempts to take a psychiatric history and conduct a mental state examination. She also checked my notes to ensure they were legible and comprehensive, listened to my discussion with the patient about their diagnosis and the treatment we might offer, and finally checked my letter to the general practitioner. Immediately after the patient left we had a brief discussion about my interview, and diagnostic and therapeutic skills. These practices, although intimidating, proved to be very educational, and I learnt quickly.

This is of course the fundamental approach of the Assessment of Clinical Expertise (ACE), during which an experienced clinician observes and assesses an entire clinical encounter between a trainee and a patient in order to be able to assess the trainee's ability to take a full history, perform a mental state examination and arrive at a diagnosis and management plan. Thus, the ACE as an assessment process has very good face validity, as it directly accesses and assesses key competencies and their underlying attitudes, skills and knowledge.

Background

Alongside adopting workplace-based assessment, the Royal College of Psychiatrists has radically revised its national examinations (Chapter 12). The ACE component of workplace-based assessment most closely resembles the superseded long case. There have always been significant technical concerns about the long case, around interrater reliability, case specificity, and intra-observer reliability. Van der Vleuten et al (1994) reported that the generalisability coefficients (a measure of reliability) of the judgements made using different formats of examination show clearly that even after 8 h of testing, oral examinations could only achieve a coefficient of 0.48, as opposed to the multiple station or Observed Structured Clinical Examination (OSCE) format, which gave a coefficient of 0.86 after 8 h of testing; multiple choice questionnaires (which are more stable and the most time-efficient of all), gave a coefficient of 0.93 after 4 h of assessment.