How to make journal clubs interesting

Abstract

Almost all psychiatric training programmes include a journal club. These clubs are experiencing a revival as a result of the introduction of a critical review paper into Part II of the MRCPsych examinations. Successful clubs are generally those with a single designated leader and mandatory attendance; provision of food is also helpful. A number of possible formats are discussed, designed to make the club an entertaining and stimulating experience.

In a recent survey of all college tutors of psychiatric training rotations in the UK and Ireland (Taylor & Warner, 2000), 97% of respondents stated that their academic programme included a journal club. However, many clubs are characterised by fluctuating levels of apathy and poor attendance. Although poorly functioning clubs run into many different problems, the most successful have certain features in common. Having one person responsible for the club is associated with effectiveness. Longevity and high levels of attendance are associated with the provision of food, mandatory attendance and enthusiastic support from the programme director. Incorporating learning principles such as agreeing goals relevant to participants may enhance learning experiences. There is no single ideal format for a journal club: each needs to respond to the needs of its participants, but innovative formats that have worked elsewhere may spark off ideas for local approaches.

Why journal clubs?

In 1997 Sackett predicted the demise of the journal club (Warner & King, 1997). The fact that they are still running suggests that they are seen as having an important role (Box 1). Originally, journal clubs were valued as means of keeping abreast with the literature. With the explosion in the number of medical journals, this goal has become unrealistic and the emphasis has changed for most clubs to promoting evidence-based practice and teaching critical appraisal. Journal clubs also play a part in fulfilling requirements for continuing professional education for permanent staff. Within psychiatry, the addition of a critical review paper to Part II of the Royal College of Psychiatrists’ Membership examination in 2000 has stimulated interest in evidence-based or critical appraisal journal clubs. Specialist training in psychiatry is unusual in dedicating a day a week to research activity, so the contribution of journal clubs to training in statistics and epidemiology may be more relevant. Finally, psychiatric services are often spread over a large geographical area, and journal clubs may fulfil an important social role in promoting regular contact between colleagues.

Box 1 Journal club goals

1. • Keep up with literature

2. • Promote evidence-based medicine

3. • Demonstrate continuing medical education

4. • Learn critical appraisal skills generally

5. • Improve skills specifically for the critical review paper of the MRCPsych examinations

6. • Promote social contact

Are journal clubs effective?

The evidence is not very strong. A Cochrane review concluded that journal clubs probably do improve knowledge of biostatistics and clinical epidemiology, although the evidence base was small (Parkes et al, 2001). There is conflicting evidence on whether they enhance critical appraisal skills, and objective measurement of skills shows less improvement than does self-assessment (Linzer et al, 1988; Alguire, 1998). It seems unlikely that journal clubs encourage participants to read more. No well-designed study appears to have investigated the impact of journal clubs on patient outcomes, which reflects the difficulty of measuring such a process. As yet, no research has looked at the contribution of particular journal club formats to success in the critical review paper in Part II of the MRCPsych examination.

Research into continuing education suggests that interventions consisting only of didactic sessions do not change physicians’ behaviour (O'Brien et al, 2001). There is some evidence that sessions incorporating principles of adult learning such as those listed in Box 2 are more likely to have an impact not only on knowledge but also on skills and behaviour. Any reorganisation of a journal club needs to take these principles into account.

Box 2 Principles of adult learning

1. • Relate the task to personal goals or to the immediate environment

2. • Present learning objectives as clinical problems

3. • Use problem-solving techniques

4. • Vary teaching approaches to suit different learning styles

5. • Use active learner participation

6. • Provide frequent constructive feedback

The importance of a leader

It is important to have one person who takes responsibility for the club. Having a designated leader correlates with effectiveness (Heiligman & Wollitzer, 1987). The essential attributes of a facilitator are an interest in medical education and a belief that the club has an important role in that process. Given the workload involved, what might attract someone to the role of journal club leader?

Journal clubs can be seen as a rich topic of research. It seems likely that, in order to maintain enthusiasm and effectiveness, clubs will periodically need to be restructured to meet new goals and objectives. Such changes need to be evaluated for their effect on individuals’ learning. Randomised controlled trials of such interventions are difficult to design, but cohort studies may be feasible. Morrison et al(1999) provide a checklist of appropriate questions that would be useful to those designing a study in this area.

Attendance rates may be highest in groups supported by the faculty but run independently of it (Linzer et al, 1987; Sidorov, 1995), indicating the advantages of involving a senior trainee as journal club leader. Trainees may feel less inhibited in asking questions in a club run by one of their peers, and trainees as leaders may be more attuned to the needs of their colleagues. Specialist registrars should be encouraged to see the role of journal club leader as contributing to their own training needs and to career enhancement in terms of managerial and organisational skills. Several descriptive accounts of new formats for journal clubs have been written by trainees and published in the Psychiatric Bulletin in recent years (e.g. Warner & King, 1997; Combe et al, 1999).

The importance of the environment

Running any group that includes junior doctors is difficult. People arrive late owing to pressure of work, and are often paged away, disrupting the meeting. Presenters are delayed by emergency admissions, or present poorly because they have been up half the night. In such a setting, it is easy for commitment to a journal club to wane. Part of the work of the leader is to demonstrate the importance of the club by maintaining boundaries: to make sure that it starts and finishes on time, that breaks for holiday periods are signalled well in advance, that any papers that need distributing are sent out and that the designated room is welcoming. Group theory suggests that clear boundaries contribute to a sense of security, making creative thinking possible.

The room should suit the group. A lecture theatre with tiered seating makes small group work difficult and encourages passivity. Moreover, a lecture theatre that is used primarily for medical students may appear less welcoming to a multi-disciplinary group. Seating participants in a circle or a horseshoe promotes eye contact and increases the chance of active participation.

Providing free food has been shown to correlate both with longevity and high attendance rates in journal clubs (Sidorov, 1995). Given the recent debate in the BMJ (‘Who pays for the pizza?’ Moynihan, 2003), perhaps the timing is right to reconsider whether mental health services should fund journal clubs, to release them from their controversial reliance on the sponsorship of pharmaceutical companies.

There are advantages and disadvantages to making attendance mandatory for trainees. The risk of mandatory attendance is that the club might be seen as being imposed rather than valued as a learning opportunity; this could be overcome by encouraging the group to agree their own rules for holding pagers, starting on time, and so on. On the plus side, having a core group of regular attenders is important for promoting continuity. Mandatory attendance has been found to be an important positive factor in both attendance rates and longevity of journal clubs (Sidorov, 1995).

Part of the environment of any journal club is the organisation within which it is run. If this supports the club, it is far more likely to be successful. Regular attendance by faculty and community physicians correlated with high levels of satisfaction in one study (Heiligman & Wollitzer, 1987). Therefore, even if the main aim of a journal club were to teach trainees, it would be worth considering what factors would stimulate the interest of consultants. Finally, a study of family-practice clubs found that attendance rates were highest when the programme director rated the educational value of a club as ‘vital’ and lowest when it was rated as ‘having no educational value’ (Van Derwood et al, 1991).

Setting up a journal club

Goals and target audience

Who are the target audience of the journal club? What do they see as their needs and interests? The format and the articles chosen will be influenced by the goals set and the audience targeted. If the primary aim of a journal club is to equip psychiatric trainees with skills to pass the the MRCPsych critical review paper, it is likely to be received with enthusiasm by candidates for that examination; it might also encourage research fellows to attend meetings, and improve links with clinical psychology staff. However, such an approach is less likely to provide valued social occasions or enhance team spirit among multi-disciplinary community teams, and might also dampen the enthusiasm of permanent staff, who would be required to sit through basic tutorials on critical appraisal skills every 6 months. If the main aim is to contribute to the application of research, it might be more appropriate to structure each meeting to look at a specific topic rather than an article. This requires significantly more preparation, so that meeting weekly would become less realistic. If participants differ widely in knowledge and skills, it is important to provide access to basic or refresher courses so that people feel competent and confident in presenting in front of their colleagues. If medical students are expected to attend, it might be useful to think in terms of a change in format while they are on rotation through the department.

Critical appraisal journal clubs

Critical appraisal is the systematic weighing up of evidence. Critical appraisal journal clubs are increasingly popular in psychiatry, particularly in view of the changes to Part II of the MRCPsych examinations. They are easier to organise than evidence-based journal clubs and are welcomed by trainees. Generally one article is reviewed: if trainees are encouraged to choose the study themselves, it might be useful to ask the presenter to demonstrate why the chosen study is relevant to the club. Another approach is to review classic papers frequently referred to in the literature, allowing participants to make up their own minds about the validity of the conclusions drawn. However the papers are selected, it is important to include different study designs. Focusing only on randomised controlled trials is not sufficient for the Part II examination.

Journal club sessions may consist of a presenter introducing a paper in a structured way. It can be helpful to use a checklist or reading guide such as the McMaster reading guides (Department of Clinical Epidemiology and Biostatistics, McMaster University, 1981) or JAMA papers (e.g. Oxman et al, 1993). The advantage of this approach is that reviews are more consistent and alert the reader to methodological flaws. Other clubs have tried to increase audience participation by dividing them into small groups (Sackett et al, 1999); the presenter briefly introduces the paper and each group is assigned critical appraisal questions. A general group discussion of the correct answers completes the session. Suitable questions can be taken from books such as The Pocket Guide to Critical Appraisal (Crombie, 1996). It might also be useful to start with worked examples from the critical appraisal papers by Brown & Wilkinson (1998, 2000).

More demanding in terms of preparation, but likely to be useful for trainees preparing for the Part II examination, is the format suggested by Cramer & Mahony (2001) for a family medicine journal club. Over the course of a year, a written test consisting of about ten questions was given to participants before and after each club meeting. The questions were set by the director of the club and were based on critical appraisal of the papers to be discussed. These tests allowed the department and each individual to track objective measures of performance, with the second test representing the learning during the session. The authors suggest that the first test assesses needs, the discussion offers opportunities to practise skills and the second test reinforces change, following principles of adult learning. In both this approach and the one described above, participants are not required to read the papers before the session. Given that a practical problem in most journal clubs is that half the participants have read the paper and the other half have not, this can be seen as an advantage. Moreover, it may mimic examination conditions more closely.

A problem with critical appraisal journal clubs is that participants may be reluctant to join in if they do not feel confident in their critical appraisal skills. Moreover, as few consultants have sat the critical review paper they may not feel competent to offer training to others. The result is that the format can easily slip back into a more familiar, traditional structure. To avoid this, it is important to access resources to support the new format (Box 3). The Public Health Unit's Critical Appraisal Skills Programme (CASP) offers web-based learning and annual courses as well as tailor-made courses for groups. Oxford University's postgraduate diploma course in evidence-based health care looks at organisational difficulties with disseminating this approach as well as the skills necessary for evidence-based medicine (EBM). Another possibility is to seek statistical support, with a statistician attending the club on a regular basis (Swift et al, 2001).

Box 3 Resources for evidence-based journal clubs

1. • Local health librarian

2. • Statistician

3. • National electronic Library for Health's advice on critical appraisal (http://www.nelh.nhs.uk/hth/help4.asp)

4. • JAMA Critical Appraisal User Guides to the Medical Literature (http://www.shef.ac.uk/~scharr/ir/userg.html)

5. • Critical Skills Appraisal Programme website (http://www.phru.org.uk/~casp/casp.htm)

6. • Information on postgraduate evidence-based medicine courses, e.g. Oxford University's website (http://www.conted.ox.ac.uk/cpd/healthsciences/courses/ebhc)

Evidence-based clubs

Evidence-based medicine relies on the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients (Sackett et al, 1997). A journal club based on these principles involves the four key elements of EBM (Box 4), of which critical appraisal is only one. Thus, an EBM journal club is a more extensive undertaking than one devoted specifically to critical appraisal, because it focuses on a topic rather than on a single paper. Such a club is more likely to affect clinical practice and to appeal to senior clinicians.

Box 4 Evidence-based medicine

1. • Question setting

2. • Literature searching

3. • Selecting the relevant papers

4. • Critical appraisal

In terms of question setting, the ideal is to start with a ‘real life’ clinical problem. Badgett et al(1997) refer to ‘teachable moments’, those frequent questions that are generated at the patient's bedside. These opportunities are usually missed because we do not seek answers, or we refer to less reliable but more convenient sources of information. Kehoe (2000) has described one local solution in a mental health setting: a ‘big red book’ to record questions that arise from case conferences and ward rounds.

Literature searching is probably the weakest point in the chain for EBM journal clubs. In a study of an obstetrics and gynaecology journal club, Coomarasamy et al(2001) found that 11 of 55 consecutive appraisals missed the article most relevant to the clinical question. It is unlikely that this would be very different in a mental health setting. Some journal clubs have tried to improve participants’ skills in this area by carrying out a real-time search during meetings, using an internet-linked computer and projector. If there is no one available who feels confident in this skill, consider asking your local trust librarian to help. Most trusts now employ health librarians as training officers, and their skills often include practising and teaching quality filtering and critical appraisal as well as identification of the literature. Once the most relevant article has been identified, critical appraisal then follows as above. In an EBM journal club it may be possible to spend one session formulating the question and identifying the most relevant articles, and the following seesion appraising them. Alternatively, presenters could complete the first three steps in advance with the help of a journal club mentor, and the meeting time could be dedicated to the critical review. A third possibility is to set the question as a debate between two teams. Each side argues its case, with a chairperson mediating.

Other formats: research, publications and medical education

Many variations on formats for journal clubs have been described. Hartlaub (1999) outlines a particularly intriguing problem-solving approach that aims to overcome the nihilism and method-bashing that can plague critical appraisal. The initial steps are as before: posing a question and searching for the most relevant paper. However, this paper is not initially shown to the audience; instead, they are given the question and asked to decide what would be the most appropriate study design to answer it. The design used in the published study is then described, and participants debate its strengths and weaknesses compared with their suggested design. The audience might then be asked to suggest an appropriate control group, and again this is compared with the control group used; and so on. This approach has the merit that participants appreciate the strengths as well as the weaknesses of a particular piece of research, and Hartlaub suggests that they might be more likely to become involved in research as they become more aware of how it can be done.

Several authors have noted the use of journal clubs as a means of generating letters to journals. Sandifer et al(1996) described a public health medicine journal club where participants were encouraged to write letters to the editors after collective appraisal of a recently published paper. Six out of ten submitted letters were published: trainees were less likely to write letters, but more likely to have their letters published. Edwards et al(2001) adopted a similar approach with undergraduates in a public health medicine attachment, with benefits in terms of student enthusiasm, writing skills and an experience of writing for publication.

Journal clubs have also been used as a forum to discuss educational interventions. Centeno et al(1999) noted that physicians are often not interested in theories of teaching. In one medical school in Argentina, accessing an already popular journal club, reduced this problem. Once a month this club was jointly chaired by a physician and a professional educator, and articles relevant to teaching or learning activities were appraised. Similarly, Simpson et al(1997) described the adaptation of relevance criteria from evidence-based medicine to education, to help faculty members interested in medical education to select papers that warranted critical review, as defined by their potential impact on student or trainee learning.

Creativity and journal clubs

In one survey (Taylor & Warner, 2000) respondents complained of the lack of flexibility of evidence-based journal clubs, where philosophical works, personal views and psychodynamic issues were not easily discussed. Roberts (2000) describes a scene from one journal club where discussion of an evidence-based mental health topic was followed by a brief narrative account of a general practitioner visiting an elderly, dying woman; this story

‘created a touching silence, reflections on what the story meant and quiet tears from a colleague … A mature student, previously a professional, said his career change had been prompted by a desire to be more connected and in touch with people's lives. We observed that the current presentation of EBM misses this altogether.’

Combe et al(1999) described an attempt to improve attendance at journal club meetings by alternating the traditional format with a more creative approach, looking at topics as wide-ranging as psychodynamic interpretations of Goldilocks and the Three Bears and recognising and interpreting body language. The alternative presentations were more varied in their choice of media, including videos, story-telling and book reviews, and seemed to promote group cohesion, as members brought more personal material to the group; for example, one showed a video on eating disorders, in which she had taken part some years earlier. The alternative journal clubs were not seen as an easy option, as more thought had to go into the choice of material, but they were rated by the group as having more impact than the traditional format and were better remembered.

Conclusions

The earliest reference to a journal club reflects its potential for combining interest and pleasure. Sir James Paget, describing the situation in St Bartholomew's Hospital in London between 1835 and 1854, reminisced:

‘Some of the pupils, making themselves into a sort of a club, had a small room over the baker's shop near the Hospital Gate where we could sit and read the journals and where some, in the evening, played cards’ (quoted by Linzer, 1987).

Journal clubs will continue to be experienced as interesting if they respond to the needs of participants. With 6-monthly rotations among junior medical staff and high turnover of many others, these needs are liable to change rapidly, so journal clubs need to use regular written surveys to assess goals and monitor satisfaction. Given the wide range of formats possible, these clubs can be both stimulating and entertaining.

Multiple choice questions

1. 1 Journal clubs:

   1. a have little effect on knowledge of clinical epidemiology

   2. b increase participants’ reading of the medical literature

   3. c may enhance critical appraisal skills

   4. d have been around for over 100 years

   5. e may improve card playing skills.

2. 2 Longevity and high attendance are associated with:

   1. a meetings being held in the evenings

   2. b food being provided

   3. c consultants attending

   4. d mandatory attendance

   5. e sponsorship by pharmaceutical companies.

3. 3 In an evidence-based journal club:

   1. a critical appraisal is one of the necessary steps

   2. b literature searching is likely to be the weakest link

   3. c librarians are often willing to become involved

   4. d randomised controlled trials are the only useful source of evidence

   5. e preparation is less time-consuming than for a traditional journal club.

4. 4 In using journal clubs to stimulate interest in research:

   1. a experience shows that undergraduates are not successful in generating published letters

   2. b cohort designs may produce good-quality evidence

   3. c descriptive papers on new journal club formats are of little interest to editors

   4. d evaluation of educational interventions is a worthwhile topic for critical appraisal

   5. e trainees are more likely to have letters accepted for publication than consultants.

5. 5 In terms of resources for journal clubs:

   1. a critical appraisal skills programmes can be run locally to suit participants’ needs

   2. b access to the Cochrane Library is expensive

   3. c website learning is available through the National Health Service

   4. d club leaders must have a postgraduate qualification in EBM before they set up a journal club

   5. e statisticians may be willing to become involved.

MCQ answers

1		2		3		4		5
a	F	a	F	a	T	a	F	a	T
b	F	b	T	b	T	b	T	b	F
c	T	c	F	c	T	c	F	c	T
d	T	d	T	d	F	d	T	d	F
e	T	e	F	e	F	e	T	e	T