The MRCPsych OSCE workshop: a new game to play?

In line with the Royal College of Psychiatrists’ commitment to improve its professional examinations based on the best current evidence, Spring 2003 has seen the introduction of the objective structured clinical examination (OSCE). This has replaced the individual patient assessment (IPA) in the MRCPsych Part I clinical examination. An OSCE consists of a series of time-limited clinical tasks that candidates have to perform in a consecutive series of ‘stations’ or booths. They have the advantage of being able to test clinical competence using a number of different scenarios via a standardised format (Katona et al, 2000). Their use has become widespread over recent years, particularly in undergraduate psychiatry exams (Brewin & Cantwell, 1997), as they have good reliability and validity (Hodges et al, 1998). This was confirmed by the College’s initial pilot OSCE, which had a k score of around 0.8 for the examination as a whole (Oyebode, 2002).

In tackling an OSCE, candidates have to be able to break down and display their clinical skills in several small ‘testable’ tasks (Wallace et al, 2002). This provides a new challenge for clinical tutors across the UK in providing appropriate structured teaching to help trainees prepare for this exam. After an initial pilot, we have introduced a series of ‘MRCPsych OSCE workshops’ locally, which could be realistically incorporated into other UK training programmes to benefit all trainees preparing for the MRCPsych Part I exam.

Background

A survey of regional pre-Part I trainees identified five areas of specific anxiety they had regarding the OSCE that could impact upon their final performance:

1. • how the OSCE stations are designed and set

2. • the specific kind of topic areas that might be examined at a station

3. • being able to accurately display their clinical skills in any given OSCE station

4. • what instructions the simulated patient might have

5. • being able to anticipate what the examiner’s mark sheet might contain for a given station.

An interactive, small-group teaching session plan was designed to help tackle these anxieties (which acted as five relevant learning objectives). The session plan incorporated the aim of dividing the trainees into two groups, allowing each to construct an OSCE station (related to an allocated skill), pilot it in the roles of simulated patient and examiner and attempt the other group’s station in the role of the candidate. Six was considered to be the optimal number of trainees to have in the workshop, allowing the piloting of the stations to be completed in under 1 hour (8 minutes per candidate × 6=48 minutes). The session was successfully piloted locally and modified in response to trainee feedback. The specific structure of a ‘ MRCPsych OSCE Workshop’ was formulated in response to this pilot session.

Structure of the workshops

We currently arrange and run 2-3 half-day workshops every 6 months. The session dates are advertised locally (via the Kingston & Richmond Regional Academic Programme), covering trainees from the St George’s Postgraduate Training Scheme in Psychiatry. This covers a pool of up to 14 SHOs, of which there are usually at least six who are at the pre-Part I stage. The six places on each workshop are allocated on a first-come-first-served basis, dependent upon trainees returning the invitation letter to the session coordinator. Any shortfall in numbers is made up by opening the invitation to SHOs in other parts of the rotation (via the Regional MRCPsych Part I course).

To run the workshop, two teaching rooms (in close proximity to each other), a stop-watch and buzzer and some refreshments are needed. One of the rooms should be able to accommodate eight people, while the other needs to be big enough for four. Each workshop is run by two session facilitators (a consultant and specialist registrar or 2 SpRs) and lasts for 3 hours. It is divided into four stages.

Stage 1 - setting the scene (45 min)

The workshop begins in the larger teaching room. The session facilitators outline the stages of the workshop, summarise the nature of the Part I OSCE (Box 1) and describe the way in which an OSCE station is designed to test specific skills. We use a modified version of the College’s ‘OSCE examination skills’ (Table 1) to illustrate this. The trainees then observe a sample OSCE station, role-played by the two facilitators (in the roles of simulated patient and candidate, respectively). Each trainee is given a copy of the examiner’s mark sheet for the station before observing the role-play, to aid inter-rater reliability. This provides an ideal ‘ice-breaker’ for the workshop, and an observed station to discuss. A different OSCE station is role-played for each of the workshops during each 6-month cycle.

Box 1. The MRCPsych Part I OSCE

1 Each candidate rotates through 12 stations, each of 7 minutes’ duration (with a ‘warning bell’ at 6 minutes). 2 Prior to entering each station, candidates have 1 minute to read the task instructions posted outside the station. 3 Each station consists of a scenario (from general adult or old age psychiatry) and a task to perform. 4 Each station either has a simulated patient, an anatomical model, video scenario or investigation result. 5 A candidate’s ‘overall’ grade (A-E) for any station is calculated from their scores on a set of predetermined weighted objectives for that station.

Table 1. The five skills to be examined in the OSCE

Skill	Examples
History taking/mental state examination	Mini-mental state examination, schizophrenia, risk assessment, substance misuse
Physical examination skills	Fundoscopy, motor examination, cranial nerves, thyroid examination, extrapyramidal side-effects
Practical skills	Electrocardiogram lead or electroconvulsive therapy electrode applications, interpreting blood results/computerised tomography or magnetic resonance imaging brain scan
Emergency management	Neuroleptic malignant syndrome, delirium tremens, resuscitation, rapid tranquillisation
Communication skills	Gaining electroconvulsive therapy consent, explaining a diagnosis/treatment/prognosis (e.g. obsessive-compulsive disorder, clozapine), counselling for alcohol dependence

After a discussion of the simulated station, the trainees are divided into two groups of three (A & B). Group A remain in the larger teaching room with one of the facilitators, while Group B move to the smaller room with the other facilitator. Each group is then given a specified ‘skill to be examined’ by their facilitator (e.g. history taking and eating disorder). Each group is kept ‘blind’ to what the other has been given.

Stage 2 - designing the stations (45 min)

Each group designs an OSCE station related to their ‘allocated skill’, with the help of their facilitator. They are initially asked to think of an appropriate scenario to base their station on. They then design a set of candidate instructions, simulated patient instructions and the examiner mark sheet weighted objectives for their station. Details of a station that was designed in one of our workshops are given in the appendices.

Stage 3 - piloting the stations (50 min)

The larger teaching room is used as the ‘exam room’, while the smaller one acts as the candidates’ ‘waiting room’. Group A pilot their station first, with one of the trainees role-playing the simulated patient, one being the examiner and one acting as an observer. Each trainee from Group B is brought consecutively to attempt the station ‘ cold’, with the remaining members of the group taking a refreshment break in the ‘waiting room’. One of the session facilitators acts as the ‘invigilator’ in the ‘exam room’, structuring the timings (using the stop-watch and buzzer) as per the actual exam. The other facilitator is responsible for bringing consecutive Group B trainees to the exam room every 8 minutes. The process is then reversed with Group B piloting their station and with Group A trainees as the ‘ candidates’.

Stage 4 - feedback and discussion (40 min)

This allows a discussion of the designed OSCE stations, including the marking schemes used by the groups. Each trainee is given a copy of their marks from the station they attempted. To reduce trainee anxiety, but still allow constructive discussion to occur, we find it helpful to use a ‘ problem-based analysis’ approach (Vassilas & Ho, 2000). This allows feedback to be balanced, descriptive and non-judgemental. Each trainee should comment on what they did well, and what could have been done differently, in attempting the mock stations. The group should then be encouraged to suggest solutions to any problems identified in tackling the stations, with input from the facilitators.

Evaluation method

An ‘evaluation form’ is completed by the trainees at the beginning and end of each workshop. The form asks them to rate their subjective confidence in each of the five learning objectives (see earlier) on a 5-point Likert Scale, ranging from 5 (very confident) to 1 (not confident at all). The post-workshop form also allows space for further comments regarding the session. Since introducing the workshops, suggestions for improvement by our local trainees have included incorporating them into the existing academic programme at 8-weekly intervals; encouraging the introduction of the workshops across other regions; setting up a website ‘pool’ of designed OSCE stations across different regions.

Creating a matrix

We have started to develop a matrix (or grid) for the pool of stations generated from the workshops for our region. This allows each of the five ‘ tested skills’ during the OSCE (Table 1) to be mapped against a range of different psychiatric conditions and scenarios. This resembles the process in which the actual Part I OSCE stations are designed, and helps to guide future workshop facilitators.

Methodological considerations

The use of role-play allows candidates to gain a deeper understanding of the examination process, thereby moving the perception of the real exam from an unknown to a familiar situation. Using the ‘skill to be examined’ principle in designing the stations gives the workshop facilitators some control over the final structure of the station, while allowing the trainees to develop relevant questions within that framework. The ‘ matrix’ approach allows the continued expansion of an OSCE knowledge base.

A potential criticism within the workshop setting is that a particular trainee may take on too strong a leadership role, thereby skewing the learning experience for the others. We have noticed a tendency for the group members at each workshop to be familiar with each other, which can lead to a degree of collusion in the level of interrogation within the exam setting. It is important that facilitators recognise the competitive nature of trainees, and manage any potentially destructive interactions.

Meaning and implications

Although some trainees have previously experienced OSCEs as medical students, for many they are a novel and anxiety-inducing experience. We have found that the supervised construction of an OSCE station helps trainees understand the mindset of an examiner. Evaluation of their experience has indicated that it is possible to increase knowledge and confidence and reduce examination anxiety by a significant amount within only one workshop. The OSCE workshop does need commitment from all those involved, and requires facilitators to have viewed the College’s ‘MRCPsych OSCE CD-ROM’ to provide a base knowledge of the OSCE examination method.

Our experience indicates a need for the following:

1. • Good timekeeping by the trainee in the examination setting. Some trainees end their interaction with the patient before the end of the station (despite the warning bell). Practising being able to interact with the patient until the end of the station (responding to their cues and closing the interview correctly) might yield extra marks.

2. • SHOs should be encouraged to incorporate supervised OSCE techniques into their daily practice e.g., explaining lithium treatment to a patient in a ward round.

3. • To reduce anxiety, attempting practical skills (e.g. fundoscopy) outside the actual exam setting.

4. • To maintain, both in the examination setting and everyday clinical work, a good rapport with the patient, by avoiding practising ‘check-list psychiatry’.

The OSCE workshop provides a regular platform for trainees to develop, practise and refine their OSCE technique. However, further practice and evaluation of the workshop is needed to verify what we and the trainees are currently assuming - that increased knowledge and confidence will give good pass rates in the actual exam? Although never able to replicate an in-vivo exam situation, feedback from the workshops indicate that they appear to create an effective learning environment for most trainees. In tackling any new examination format, trainees (and trainers) should remember that ‘chance favours the prepared mind’ (Louis Pasteur).

Appendix 1

OSCE station - communication skills: explaining a diagnosis of depression to a relative.

Candidate instructions - You are asked to see Mr Stevens, whose wife was recently diagnosed with depression by your consultant in the out-patient clinic. The depressive illness was precipitated by the death of his wife’s mother. No formal treatment has yet been started. Mr Stevens would like to know more about his wife’s illness, as she has never had any such problems before. Explain the nature of the illness and the possible treatment options available, and deal with any specific concerns Mr Stevens has.

Simulated patient instructions - You are Mr Stevens, whose wife has been depressed for the last 3-4 months, following the death of her mother in a road traffic accident. She has lost interest in her usual activities, has become socially isolated and is showing signs of poor self-care. She has also been neglecting your two children, aged 3 and 6. This has put considerable strain on the marriage.

You are particularly interested in knowing the following:

1. (1) The nature of her illness, and whether it could get worse.

2. (2) Can it be treated - if so, how? How long would she need treatment?

3. (3) Could the illness come back again? Will it affect your children in future life?

4. (4) Is your wife at risk - could she harm herself or others?

Appendix 2

Examiner mark sheet for a MRCPsych OSCE workshop station.

Station title: Explanation of depression to a relative.
Candidate No./name:
Examiner: please mark one grade for each objective for this station.
Key: A=excellent, B=good, C=average (pass), D=fail, E=severe fail. (score E if the candidate fails to ask about the objective listed)
	A	B	C	D	E
Communication (including rapport) (20%)	□	□	□	□	□
Explaining the diagnosis (20%)	□	□	□	□	□
Drug treatments (20%)	□	□	□	□	□
Other therapies (20%)	□	□	□	□	□
Prognosis (10%)	□	□	□	□	□
Issues of risk & safety (10%)	□	□	□	□	□
Examiner: please mark one grade for the global rating for this station.
	A	B	C	D	E
Global rating	□	□	□	□	□