Learning portfolios in psychiatric training

Major changes are taking place in the way doctors are trained and assessed. A new curriculum, devised by the Academy of Medical Royal Colleges on behalf of Modernising Medical Careers, will be implemented for current junior doctors as part of the foundation and specialist training programmes. Every junior doctor will be expected to have a ‘personal learning plan’ to guide professional development and assist appraisal, and to build a portfolio to document their experiences. This marks a shift from traditional summative (i.e. examinations-based) evaluation to the use of more formative methods based on experience and workplace assessment. It reflects greater emphasis on continuing professional development and life-long learning (Wilkinson et al, 2002).

The term ‘learning portfolio’ usually implies two elements: a record of educational experience and a tool to encourage reflective learning (Snadden & Thomas, 1998; Cole, 2005; Rees, 2005). To work effectively for reflective learning, portfolio content needs to include training goals and identify gaps in knowledge or experience, leading to the formation of future goals (Driessen et al, 2005; Rees, 2005). Portfolios should include details of failures as well as successes. Portfolios simply used as a logbook for examination purposes are of limited benefit in this regard (Snadden et al, 1996).

Portfolios have been in compulsory use for learning and assessment in nursing and allied health disciplines for a decade and are regulated by the English National Board for Nursing, Midwifery and Health Visiting. However, there has been comparatively little published concerning their use in postgraduate medical education. In one study conducted in the general practice setting, portfolio use was seen as a tool in promoting reflective learning in addition to aiding supervision and planning future learning goals (Snadden et al, 1996; Challis et al, 1997). However, considerable barriers were cited by participants to continued portfolio use, including resistance to forgoing didactic teaching methods, lack of time and preoccupation with passing examinations (Snadden & Thomas, 1998). These findings were mirrored in a recent study of nursing students (McMullan, 2006). Voluntary use of portfolios was limited and only increased if the process was compulsory (Dornan et al, 2002). Portfolio use tended to decline over time and depended on the learning style and attitude of the individual. Work done in an undergraduate medical setting and in other postgraduate disciplines broadly mirrored the findings in general practice (Finlay et al, 1998; Lonka et al, 2001).

Few studies have examined the benefits of portfolio use, although this has been attempted in the undergraduate setting. Finlay et al (1998) studied two groups of students, one randomised to portfolio use and tutorial support and the other to a standard teaching protocol. There were no significant differences in overall examination marks in the subject of interest (oncology) or in overall degree marks. However, researchers did find a statistically significant benefit of portfolio use in weaker students, who attained higher marks for factual knowledge.

Given the future importance of learning portfolios in medical education, we assessed the current knowledge, attitudes and usage of portfolios among psychiatric trainees in a large London psychiatric training rotation. For those already using them, we wanted to explore issues of attitudes and content.

Method

A self-report postal questionnaire was sent to all 76 permanent senior house officers (SHOs) on the South London and Maudsley training rotation in adult psychiatry employed between January and August 2005. A reminder was sent 4 weeks later to those who did not respond.

The questionnaire requested trainees to provide demographic and training details, and indicate their attitudes to portfolios. Trainees who had a portfolio were asked to provide information on content and sources of advice for compilation, ticking options as appropriate. Those with no portfolio were asked to tick a range of possible reasons exploring barriers to their use. Attitudes towards portfolios were requested from all respondents, using a number of statements scored on a 5-point Likert scale (strongly agree, scored 1; agree, 2; neutral, 3; disagree, 4; strongly disagree, 5). Space was provided at the end for a free-text response. A copy of the questionnaire is available from the authors. Data were analysed using SPSS, version 12 for Windows.

Results

Demographics

Forty-five doctors (59%) returned the questionnaire (23 male). They ranged in age from 26 to 38 years (mean 29.4, s.d.=2.2). The sample came from a broad range of medical schools, with 13 graduating overseas (29%), 11 in London (24%) and 7 in Cambridge (16%). Thirty-one doctors were White (71%), 2 (5%) Black, and 8 (14%) Asian, Indian or Chinese. The ethnic balance was broadly similar to the membership profile of the College (http://www.rcpsych.ac.uk/PDF/05_memfells.pdf). Time spent in psychiatry ranged from 8 to 84 months (mean 30.1, s.d.=16.8), with year of qualification ranging from 1992 to 2003.

Use and content of portfolios

Out of 10 doctors who had compiled a portfolio, 7 were female, 4 had graduated from London and 3 from Cambridge. Ages ranged from 27 to 31 years (mean 28.3, s.d.=1.2), whereas time spent in psychiatry ranged from 9 to 36 months (mean 23.8, s.d.=9.8). The most frequently used source of information in compiling a portfolio was senior advice (60%), followed by Royal College of Psychiatrists’ guidelines (40%) and other published information, peer advice and other sources (30% in total).

Most portfolios contained achievement-focused information; 90% of doctors included their curriculum vitae, General Medical Council certificate and job appraisal; 80% included evidence of research and course attendance; 70% detailed information on audit projects and qualification certificates, in addition to including publications and clinical presentations. Slightly less common were details of teaching experience (60%), awards (50%), conference attendance (40%) and management experience (30%), although this might reflect the relative lack of experience in their career to date. Relatively few included evidence of reflective practice, such as patient feedback (30%) and peer discussions on interesting patients.

Non-users of portfolios

Of those who did not have a portfolio (n=35), 18 (51.4%) had never heard of portfolios, 7 (20%) had thought about making one, 17 (48.6%) would only use one if it was compulsory, 5 (14.3%) considered they did not have the time to compile one and 8 (22.9%) thought a portfolio would not suit their style of learning.

Attitudes to portfolios

Seven attitudinal items were examined separately. Taking the sample together, most doctors gave neutral responses to the question ‘are portfolios helpful’ (n=17), although most agreed portfolios were useful in career planning (n=27), revalidation (n=24) and supervision (n=27). Most disagreed with the statement ‘it is easy to access advice on making portfolios’ (n=18) and felt portfolios were time-consuming to compile and maintain (n=23).

When comparing those who had complied portfolios with those who had not, there was one significant difference between individual items. Those who compiled portfolios disagreed more with the statement that portfolios are something you only do for appraisal (mean 3.80 s.d.=0.92 v. 2.97 s.d.=1.02, t=–2.31, d.f.=41, P=0.03). There was a tendency for those who had compiled portfolios to believe they were helpful in revalidation (mean 1.80, s.d.=0.79 v. 2.36, s.d.=0.86, t=1.85, d.f.=41, P=0.07).

No significant difference was shown when a ‘helpful’ sub-score was calculated from summing five items that were seen to indicate that portfolios were useful (‘portfolios are helpful’, ‘ portfolios are used for appraisal’, ‘can be used to plan career development’, ‘can be used in supervision’, ‘ can be used for revalidation’). Cronbach's α score was acceptable at 0.88.

Free-text responses

There were 17 respondents who included comments on their questionnaire. Of those who had compiled a portfolio (n=8), 3 out of 4 comments were broadly positive, emphasising their usefulness in summarising achievements and planning career development. The negative comment concerned the amount of time needed to compile a portfolio. Of the 9 trainees who did not have a portfolio, 6 made broadly negative comments, focusing on time constraints and uncertainty about potential benefits. There were also comments regarding the need for support and guidance.

Discussion

With the advent of Modernising Medical Careers, learning portfolios are set to become an integral component of medical training. This study shows a low level of portfolio use among psychiatric trainees, with limited understanding of their purpose or content. The content of portfolios within this training rotation suggests they are being used as a record of achievement. There seems to be difficulty in accessing advice on compiling portfolios, with sources of information drawn mainly from informal sources rather than educational bodies. It is worrying that more than half of those who did not have a portfolio had not even heard of them. The publication of Foundation Learning Portfolio (Modernising Medical Careers, 2005) may go some way to alleviate this knowledge gap, complemented by advice from senior colleagues.

Doctors’ attitudes to portfolio development were broadly neutral, both for those who had a portfolio and those who had not. Those who already had compiled portfolios realised that their usefulness extended beyond the appraisal process, with the possibility of ongoing benefits in continuing professional development. This supports previous findings that it is not until you actually start a portfolio that you begin to appreciate its potential benefits (Rees & Sheard, 2004), with the opportunity for reflective learning being developed (Roberts et al, 2002).

Preliminary evidence shows that educational portfolios may benefit the educational process but additional studies are needed to confirm this. Whatever their efficacy, they are here to stay. This survey reinforces the need to make portfolios a compulsory feature of continued learning beyond the foundation years, with clear explanations regarding their content and rationale, otherwise their use may remain low.

Declaration of interest

None.