14 - Developing and delivering an online assessment system: Assessments Online

Summary

Developing an online system to support postgraduate medical education presents its own unique challenges, given the diversity of stakeholders across geography and occupational groups. This chapter aims to set out some of these challenges by describing the development of Assessments Online, the Royal College of Psychiatrist's online assessment system.

In 2008, the Postgraduate Medical Education and Training Board (PMETB) set out revised standards for curricula and assessment systems. These standards detailed the ways in which assessment and training should take place, and how it should be monitored and evaluated. To successfully meet these standards would require the development of organised process and systems for monitoring training and assessment at every level. The use of information systems or IT is ideally suited to this kind of scenario, as it allows information to be collected, organised and analysed in ways that are not possible using traditional paper-based methods.

The introduction of a large-scale information system not only relies on successful implementation and delivery, but also on being able to engage the end-users. This would prove to be one of the biggest challenges the Royal College of Psychiatrists would face in implementing their own assessment system, as the memory of the Medical Training Application Service (MTAS) project was fresh in people's minds. Coupled with trainees’ experiences of a rapidly changing landscape of online assessment in the foundation programme it was not surprising that there was at the time a great deal of concern and potentially also resistance to any IT system.

This chapter will discuss not only the rationale but also the process of the development of what is now regarded as a largely successful online system.

Making a case for Assessments Online

There are many benefits at various user levels to having assessments delivered online. From a trainee perspective, it facilitates an efficient way of administering the multisource feedback tool, mini-Peer Assessment Tool (mini-PAT; Chapter 6).