For 13 years the Royal College of Psychiatrists’ Public Education Committee has been producing award-winning public information leaflets (Fig. 1). There are now over 15 titles in the ‘Help is at Hand’ series (http://www.rcpsych.ac.uk/info/help/index.htm). These cover core disorders, such as schizophrenia, and more general topics, such as bereavement. Over 5 million leaflets have been distributed free of charge to the public, family practitioners, psychiatric units, libraries - even funeral directors. General practices and clinics distribute some, but most have been supplied directly by the College. Some have been translated into other languages such as Chinese and Spanish, although the majority of our materials are still available only in English. More recently it has become clear that:
-
• The leaflets had a traditional, prescriptive, approach; they needed to move towards a more dispassionate provision of information, to serve as a tool to support clients to make better-informed choices.
-
• To ensure that our information is perceived to be unbiased, we need to move away from the pharmaceutical sponsorship which has funded the printing of the leaflets.
Fig. 1. Some of the ‘Help is at Hand’ leaflets.
In 2002 the Editorial Sub-Committee of the Public Education Committee was established to expedite these developments. It consists of three College members and five members of staff. The Sub-Committee's activities are summarised in Box 1.
Box 1. Role of the Editorial Sub-Committee of the Public Education Committee in leaflet development
| The Editorial Sub-Committee: |
|
Although the ‘Help is Hand’ leaflets have always been popular, there are some areas of concern. The Patients’ and Carers’ Liaison Group has vetted our leaflets, but there has been no other direct user or carer input. We have probably omitted significant issues of importance to patients, carers and relatives. Informal feedback in the form of letters to the College has generally been positive, but there has been no structured evaluation. Our ability to revise leaflets has been determined not by the need for updating, but by exhaustion of the stock and the vagaries of sponsorship.
Leaflet ratings
Some limited independent assessment of the ‘Help is at Hand’ leaflets has been carried out. Currie et al (Reference Currie, Dimambro and Joice2002) used the DISCERN criteria to evaluate a number of different leaflets on schizophrenia (DISCERN is an instrument that rates several areas, including relevance to the target audience, the ground covered, the presence of references, and even-handed coverage of alternative views or treatments [http://www.discern.org.uk]; Reference Charnock, Shepperd and NeedhamCharnock et al, 1999). The ‘Help is at Hand’ leaflet was rated as only ‘fair’ in this assessment. The committee now uses the DISCERN criteria to screen all our leaflets at the first draft.
Coulter et al (Reference Coulter, Entwistle and Gilbert1998) assessed seven leaflets on depression using two focus groups, one of people who had had significant depressive episodes and another of specialist experts. The College leaflet was rated top by the specialist group, which felt that it ‘provided the most useful information for patients who wanted to be informed participants in decisions about their care’. The patients group, on the other hand, appreciated the information but did not like the small print size and tone, so ranked it sixth out of the seven publications.
Humfress & Schmidt (Reference Humfress and Schmidt1999) used the Gunning FOG Index. This tests readability by scoring the average sentence length and the number of long words in a piece. The College leaflets had a median score of 13, compared with 15 for the Mind series of leaflets. A score of 15 is roughly equivalent to the level of a BMJ article. To make sure that most people can read our leaflets we should be aiming for a score of around 10. This represents the reading age of 12-13 years, the level at which most tabloid newspapers are written.
Online feedback
Useful though they have been, none of the above assessments has explored how ‘live’ readers of the leaflets were responding to them. The establishment of the College website has presented us with a way of doing this. In fact, ‘Mental Health Information’ has become the most popular part of the College website. The most viewed leaflets get about 160 true visits (as opposed to ‘hits’) a day. On average each visitor spends about 6 min on a leaflet page. We can now receive direct feedback from readers. Each leaflet has an online feedback form and we are currently receiving around 100 responses every week.
Survey of online responses
We analysed all the feedback forms completed online between 18 December 2002 and 7 October 2003. These forms were the original questionnaire, consisting of five forced-choice scales (Fig. 2) and a free text area. An overall score for each leaflet was obtained by awarding points for each response (5, agree strongly; 4, agree; 3, neutral; 2, disagree; 1, strongly disagree); in the case of the statement ‘This leaflet talks down to me’ the points were reversed, so that high scores consistently indicated a favourable ranking.
Fig. 2. Online forced-choice feedback questionnaire.
The overall scores were high (Table 1), although the ‘This leaflet talks down to me’ item performed poorly relative to the other categories. This low score may have been due to the tone of the leaflets. However, it might also have been influenced by the fact that the scoring for this item worked in the opposite direction to other items on the screen. The feedback screen has now been changed so that all items score in the same direction, and we look forward to seeing if there is any overall change in the scores for this item.
Table 1. Online ratings of ‘Help is at Hand’ leaflets
| Feedback scores: mean (s.d.) | |||||||
|---|---|---|---|---|---|---|---|
| Leaflet1 | Responses (n) | Readable | Useful | Talks down | Well designed | Recommended | Overall |
| Manic Depression | 732 | 4.5 (1.1) | 4.5 (1.1) | 3.1 (2.9) | 4.1 (1.2) | 4.2 (1.2) | 4.3 (0.5) |
| Depression | 149 | 4.4 (1.2) | 4.4 (1.1) | 3.7 (2.9) | 4.2 (1.0) | 4.2 (1.3) | 4.2 (0.3) |
| Bereavement | 298 | 4.6 (0.9) | 4.5 (0.8) | 3.3 (3.0) | 4.2 (1.0) | 4.3 (1.0) | 4.2 (0.5) |
| Post Natal Depression | 397 | 4.6 (1.0) | 4.6 (1.0) | 3.1 (2.9) | 4.2 (1.0) | 4.5 (1.0) | 4.2 (0.6) |
| Memory & Dementia | 150 | 4.6 (1.0) | 4.6 (0.9) | 3.1 (2.6) | 4.1 (1.2) | 4.4 (1.1) | 4.2 (0.6) |
| Alcohol & Depression | 101 | 4.6 (1.0) | 4.5 (1.0) | 3.1 (2.9) | 4.0 (1.3) | 4.0 (1.5) | 4.0 (0.5) |
| Men Behaving Sadly | 113 | 4.5 (1.0) | 4.4 (0.9) | 2.9 (2.7) | 4.1 (1.0) | 4.0 (1.3) | 4.0 (0.6) |
| Depression in People with Learning Disability | 47 | 4.3 (1.4) | 4.3 (1.4) | 2.8 (2.7) | 4.0 (1.4) | 4.3 (1.0) | 4.0 (0.6) |
| Schizophrenia | 182 | 4.4 (1.2) | 4.4 (1.2) | 2.7 (2.6) | 4.0 (1.3) | 4.2 (1.4) | 3.9 (0.6) |
| Sleeping Well | 117 | 4.2 (1.4) | 4.1 (1.3) | 2.7 (2.9) | 3.9 (1.3) | 4.0 (1.4) | 3.8 (0.5) |
| Social Phobias | 242 | 4.2 (1.4) | 4.0 (1.4) | 2.9 (2.7) | 3.9 (1.3) | 3.9 (1.6) | 3.8 (0.5) |
| Anxiety & Phobias | 443 | 4.2 (1.4) | 4.1 (1.4) | 2.8 (2.7) | 3.9 (1.3) | 3.9 (1.4) | 3.8 (0.5) |
| Surviving Adolescence | 193 | 4.1 (1.3) | 4.0 (1.3) | 2.7 (2.7) | 3.7 (1.4) | 3.7 (1.6) | 3.7 (0.5) |
| Anorexia & Bulimia | 1070 | 4.1 (1.4) | 4.0 (1.4) | 2.5 (2.5) | 3.5 (1.5) | 3.8 (1.6) | 3.6 (0.6) |
| Total | 4234 | ||||||
If this item is omitted, approval ratings for all categories were over 70%. This held even for the leaflets on schizophrenia and eating disorders, where individuals more often find themselves in conflict with psychiatric notions about their condition. This information is reassuring, but also highlights areas for review. For example, since obtaining this feedback, we have rewritten the adolescence leaflet and retitled it explicitly as A toolkit for parents. We hope that this will reduce the ‘talks down’ score, and are looking at ways of developing a leaflet exclusively for adolescents.
Free-text feedback
This has generally been positive. Here are some responses on individual leaflets.
Manic Depression
“I was struggling to absorb varying information regarding manic depressive [illness]. However, after reading this leaflet I gained a better understanding because all of the components were broken down in a logical manner with language that was easy to understand. Thank you for allowing me to have a greater understanding of this problem.”
“I have just recently been diagnosed as a manic depressive. The leaflet explains a lot of valuable information which I will pass onto my friends and family so they may understand my behaviour over the past few years.’
Alcohol and Depression
“This leaflet is a rarity. An uplifting and refreshing look at an issue that confronts a lot of people. This leaflet is informative, without leaving the reader feeling demoralised and angry. As usual stepping outside of the USA for news and info proves to be enlightening. Thanks.”
Post Natal Depression
“It was as though someone had looked inside my thoughts and put them in writing. A relief to know someone else feels this way.”
Schizophrenia
“Wonderful info. I assist in providing education in a systems management group…and individuals have asked me to help them learn how to educate their families about their illness. Thus occurs to me how little education they have gained about their own illness. This is just what I was looking for - very clear, straightforward and respectful presentation regarding current ideas and knowledge.”
Limitations of our study
We are only hearing from a self-selected group who have chosen to use the feedback tool. They may not be representative of the whole group of individuals reading these leaflets on the website. However, as far as we can gather to date, 1 in every 5-7 readers give us feedback. Most of our materials have not only been available to an English-speaking readership.
Discussion
Overall, all the leaflets in the series did well. The feedback has allowed us to identify those that are performing less well, and these now have priority for rewriting. The feedback forms seem acceptable to our readers and will be attached to all leaflets produced by the College. We will also add further items:
-
• to identify more clearly who the reader is - for example, a service user, a carer, a student or a member of the public;
-
• to identify the national location of the reader, to help identify and quantify our global audience.
The webpage feedback has demonstrated that it is possible to make available a wide range of understandable and useful mental health information directly to the public. Leaflets can now be revised and published on the website without the delays inherent in paper publication. However, paper is still an important medium, especially for use in the context of a consultation. Such information has historically not been available in specialist mental health settings. Having established the broad acceptability of our information products over the web, we are now marketing paper versions of the leaflets to all mental health providers in the UK. Unlike content published on the web, we cannot guarantee that paper leaflets reach their intended readership. So, the next step in our evaluation will be to ascertain the penetration of leaflets purchased by health organisations into settings where they should be available to workers, clients and carers.
Declaration of interest
The authors have all been involved in the writing and editing of the ‘ Help is at Hand’ leaflets, but have received no payment for this activity.
Open access
eLetters
No eLetters have been published for this article.