National survey of training needs for evidence-based practices

With the increasing importance of evidence-based medicine in psychiatry, the recent inclusion of a critical analysis paper in the MRCPsych Part II Examination and the current emphasis on clinical governance, it is now necessary for all psychiatrists to have some degree of training in evidence-based practices.

Evidence-based journal clubs are a useful method of improving critical appraisal skills (Gilbody, 1996 ; Warner & King, 1997). We aimed to identify how many teaching centres have incorporated a regular journal club that takes the format of evidence-based/critical analysis style presentations into their academic programme. In addition we sought information about how successful local tutors perceived these to be and/or what problems they had encountered in establishing them.

The study

We conducted a cross-sectional survey using a postal questionnaire in October 1999. This questionnaire was sent to all recognised college tutors of psychiatric training rotations in the UK and Ireland as well as to those of related overseas schemes by the college.

Our questionnaire enquired about various aspects of the local academic programme ; in particular whether this included evidence-based style journal clubs which were defined as meaning that the presenter should be involved in the four key elements of evidence-based practice (Sackett et al, 1997) :

1. (a) question-setting ;

2. (b) literature searching ;

3. (c) selecting the relevant articles ;

4. (d) structured critical appraisal.

Further questions sought respondent's opinions about how the journal clubs were running, what assistance and resources were available for trainees and what problems if any had been encountered. In addition, thoughts and comments were welcomed. Data from replies were entered into an SPSS database and results were analysed using simple non-parametric cross-tabulations.

Findings

Sample characteristics

Questionnaires were mailed to all 422 college tutors of basic specialist training schemes. We realised this would result in duplication of questionnaires in some areas, but this strategy ensured all centres were included in the study. One hundred and forty-five questionnaires were returned, a response rate of 35%. All regions were represented in the survey, and replies were received from 95 centres (65% of all schemes). The replies related to 1943 trainees ; 1466 senior house officers and 477 specialist registrars.

Established evidence-based journal clubs

One hundred and forty (97%) respondents stated that their academic programme included a regular journal club with 85 (61%) of these occurring on a weekly basis and 80 (58%) being presented by trainees alone. One hundred (71%) respondents reported evidence-based-style presentations at their journal clubs with 42 (42%) of these occurring weekly, having taken over completely from the traditional journal club format. As shown in Table 1, these are both well attended and felt to be a good forum for teaching evidence-based medicine. Several positive comments were received, including the following :

Table 1 Positive responses to questions about evidence-based medicine journal clubs and resources. Denominator varies between answers

Question	Number (%) of positive responses
Sites with established evidence-based
journal clubs (n=100)
Well attended	97 (97%)
Good forum for teaching	95 (95%)
Problems experienced	82 (60%)
Do trainees receive teaching in
Question-setting	83 (63%)
Literature searches	115 (85%)
Critical appraisal	122 (89%)
Statistics	97 (73%)
Do trainees have easy access to
Library	135 (97%)
Medline	133 (96%)
Embase	67 (52%)
Psychlit	86 (66%)
Cochrane Library	113 (83%)
Internet	113 (82%)
Statistician	35 (26%)
Is there an individual with special interest in evidence-based medicine	67 (52%)

“It is a far more interesting format than previously and as a result 30-40 people attend - the local reps sponsor lunch and queue up for the privilege !… The results frequently reveal no high quality research and therefore help trainees to develop professional judgement instead of the lazier option of looking only at literature”.

Journal clubs run by an individual with a special interest in evidence-based medicine (52%) are significantly more likely to have evidence-based-style presentations (X ²=16.3, P<0.001). However, the presence of such an individual was not linked to teaching literature searches, critical appraisal or statistics.

Several comments were received from individuals running successful evidence-based journal clubs high-lighting the lack of flexibility of this style. Philosophical works, personal views and articles about psychodynamic issues were not thought to be appropriate for the evidence-based medicine format.

“The question often turns out to be rather simplistic. Evidence-based medicine cannot take into account the full range of psychosocial issues in one patient…I miss good review articles that can look at these problems”.

Traditional journal clubs

Forty (27%) respondents stated that their academic programmes did not include any form of evidence-based-style journal club, although 28 (70%) of these indicated that this was planned for the future. Seventeen (61%) of these respondents agreed that reticence of trainees to be involved is a reason for the lack of evidence-based-style presentations.

“Old habits die hard… nursing and paramedical staff are more evidence-based medicine-orientated than Trainees.”

In contrast, a lack of resources was implicated by only 12 (43%) of these respondents and most trainees do have ready access to a library (97%), Medline (96%), the Cochrane Library (83%) and the internet (82%).

Problems with journal clubs

Eighty-two (61%) of all respondents had experienced problems introducing or attempting to introduce evidence-based-style journal clubs. Of the 40 schemes that reported not having regular evidence-based journal clubs, difficulties that accounted for this included reticence of trainees (63%) and lack of resources (43%). Lack of training in evidence-based medicine practices for consultants and educational supervisors was highlighted as another problem. This has left senior staff feeling that they are without the skills and knowledge to partake fully in this style of journal club or to proide teaching and supervision for trainees.

“In the first place, consultants need training in exactly what is involved in critical appraisal and evidence-based medicine”

For some, lack of enthusiasm for evidence-based medicine is a more pertinent issue

“Learning evidence-based medicine is like bitter medicine - I'm sure it will do me good, but I don't much like the experience!” “The problem is with consultants who are used to sleeping through a rehash of their papers and who see evidence-based medicine as a threat.”

Discussion

We were surprised at the large proportion of journal clubs that now have an evidence-based orientation. Nearly all tutors who reported a shift to this format felt it was a good forum for teaching evidence-based medicine skills. The culture of evidence-based practice is becoming established in psychiatry and evidence-based medicine-style presentations at journal clubs could play a fundamental part in this. Most areas appear well-resourced for these activities. However, this process is not problem-free or without opposition. A reluctance to embrace the evidence-based medicine culture and a lack of training for educational supervisors leading to apprehension about to the subject in both themselves and their trainees appear to be major issues.

The interpretation of our results may be hampered by the response rate. Although only 35% of questionnaires were returned, replies were received from most training schemes. Non-respondents are likely to be less interested in evidence-based medicine and our results may be an over-optimistic assessment of current practice.

Adequate training, resources and support are essential for psychiatrists at all levels in order to ensure evidence-based-style journal clubs are successful. We recommend that all training schemes adopt this evidence-based approach, at least in some of their journal club settings.