Response of Chief Examiner

I am pleased to have the opportunity to respond to the letters from Drs Finlayson and Turner. Dr Finlayson's letter relates to how the standard of an examination is set, particularly in the multiple choice question (MCQ) paper. It is, of course, self-evident that the actual mark scored by any candidate is not in itself meaningful without consideration of the relative difficulty of the question paper to which the score relates. Hence it is possible to have a paper that is so easy that a score below 95%, for example, would indicate that the candidate is poor. It is also possible that a paper is so hard that a score of 35% deserves to pass. The case I have been making is that scores need to be set in the context of the actual examination paper and not in the context of the quality of the candidates. This is the method that the College adopts in relation to deciding what standard to set, that is to say, what the pass mark should be. This is technically referred to as criterion-referencing as opposed to norm-referencing and is the currently accepted method for determining the pass mark in MCQ examinations. Standard-setting is thus neither arbitrary nor capricious. There is no evidence that the MCQ paper is any harder than it was 15 years ago. There is also little evidence that the performance of candidates has substantially altered in the same period. In fact the performance of candidates as a group varies, but not significantly from sitting to sitting.

Turning to Dr Turner's letter, the College's aim is to notify candidates at least 4 weeks in advance of the written paper whether their application has been accepted. In very exceptional circumstances, where the candidate's eligibility requires special attention, this may slip. The Associate Deans who deal with eligibility are usually working to a tight time frame; none the less, they keep the candidate's interest to the fore while ensuring that the regulations are adhered to. Flexibility, fairness and justice are the watchwords. Communication with trainees by the College staff and officers is characterised by courtesy. I appreciate that candidates are anxious about the examinations and also that this anxiety may adversely influence their perception of the application and examination process. What is remarkable is the degree to which College staff and officers retain their good nature in the face of ill-tempered behaviour. In general, independent observers of the College examinations remark upon the efficiency and dedication of all who contribute to the process. This is not to say that there is any sense of complacency.