A clear step in the right direction

Summary

This is a commentary on the recent online survey by the Royal College of Psychiatrists about case-based discussion as a method of revalidation of psychiatrists in practice in the UK. The overall quality and impact of this pilot study are discussed, as well as some comparisons with revalidation processes in the USA and Canada. Some recommendations for future research are suggested.

The study on case-based discussion, ¹ drawing on an online survey conducted by the Royal College of Psychiatrists, renders important new information about the feasibility and user-acceptance of this tool for the process of revalidation of psychiatrists in practice in the UK. This relatively small pilot study demonstrates that practising psychiatrists regard this type of formative assessment a positive learning experience and find it useful in improving their clinical practice. It also suggests that practitioners would welcome case-based discussion as a component of the revalidation process. These are significant findings, with implications for future developments in both training and continuing professional education. Finally, the study highlights the importance of uniform training of assessors - and those being assessed - and the need to establish interrater reliability among assessors.

In a broader context, the relatively inclusive, educational, research-oriented approach of the Royal College of Psychiatrists has been a positive model for recertifying organisations in psychiatry worldwide. One can hope that this approach will continue as revalidation and continuing professional development (or continuing medical education, as it is known in the USA) are further expanded and implemented. The quality of our revalidation processes is of the highest importance, as this self-monitoring is manifest evidence of our commitment to excellence in patient care and our social contract with patients, their families, and the public.

International perspectives on revalidation: USA and Canada

Psychiatric credentialing or certifying bodies in many countries are working to develop processes by which practitioners can demonstrate their ongoing competence in their fields, and it is often helpful to compare notes. The UK, Canada and the USA use different terminology for their processes. ‘Revalidation’ is the term used in the UK to encompass two elements: licensure through the General Medical Council and recertification through the Royal College of Psychiatrists. In the USA and Canada the overall process is called ‘maintenance of certification’ (MOC). ^2,3 In the USA, medical licences are obtained from individual state licensing boards, which are overseen by the Federation of State Medical Boards. Board certification through the American Board of Psychiatry and Neurology (ABPN, a division of the American Board of Medical Specialties) is not currently required for medical licensure, but considered a mark of excellence - and is required by most academic departments of psychiatry and for employment with many federal agencies.

Recertification and practice assessment tools in the USA

The ABPN is engaged in the process of maintenance of certification and recertification of psychiatric practitioners. Since October 1994, all board-certified psychiatrists enter a 10-year cycle of recertification, which involves four separate areas: continuing medical education and self-assessment, cognitive expertise, ‘performance in practice’ activities, and professional standing. The assessments, including the performance in practice assessments requirements, are being introduced in a gradual timetable. Interestingly, the ABPN has designated two methods of assessment that were ranked lowest by two Royal College of Psychiatrists’ focus groups of users and carers in the online preference survey: an audit of practice against evidence-based guidelines and a written exam, ranked 7 and 8 respectively out of eight possible choices. The UK responders’ third choice, ‘anonymous feedback from service users, carers and colleagues’ was also designated as a required component of the APBN performance in practice section. This requirement has been very controversial among psychiatrists in the USA, prompting letters of protest from district branches of the American Psychiatric Association to the ABPN. The ABPN - unlike the Royal College of Psychiatrists - has not surveyed its membership about preferred methods of assessment before designating requirements. ² Although psychiatrists may not prefer tools that rely on the most evidence, and may be influenced by multiple other factors in their preferences, user-friendliness and acceptance by practitioners would appear to be key factors in winning support for the relatively new processes of revalidation/recertification.

The ABPN has not designated a form of case-based discussion with an assessor for its recertification process; it may be that the large numbers of psychiatrists in the USA (compared with the UK) would make training adequate numbers of assessors an unfeasible task. However, the presence of trained assessors adds significant validity to the revalidation/recertification process.

Assessor bias in case-based discussion

Individual self-assessment modules have been shown to be effective in improving clinical care of patients, ⁴ but the peer/assessor component of the case-based discussion format in Mynors-Wallis et al's study may have significant potential advantages. A well-trained assessor can detect and highlight blind spots, bias and faulty reasoning in a practitioner. The authors point out that none of the assessed practitioners was given a score less than ‘3’ in five of the eight sections of the assessment; they wondered whether assessors were reluctant to give colleagues scores indicating less-than-adequate competence in most areas. This is an important question and should be a focus of research and training in the future. Some assessors may have known those being assessed, or been familiar with their reputations before the exercise. Even if they were unknown to each other, many physicians have great difficulty giving negative or constructive feedback to colleagues. ⁵ Mynors-Wallis et al are correct in stating that assessors will need further training to accurately score performances and to give their verbal and written feedback.

This important pilot study demands future investigation in several areas. First, a process should be established to develop interrater reliability among assessors, using Frame of Reference and Performance Dimension Training workshops. ⁵ Second, it is important to seek further evidence that case-based discussions can improve patient care. The individuals undergoing assessment overwhelmingly believed that the case-based discussions were effective in that area; a logical next step would be to examine and measure actual changes in practice in a group of practitioners who have undergone these formative assessments.