Lessons learned from a failed research project

There has been a significant emphasis on research experience in the years of specialist registrar training (Royal College of Psychiatrists, 1998). It has also been acknowledged that in reality many specialist registrars find it difficult to start a research project and complete it within the 3 years of specialist registrar training (Allsopp et al, 2002). There have been various suggestions about how the ‘ research day’ can be used more gainfully - in acquiring skills in evidence-based medicine (Ramchandani et al, 2001); or in preparing to be a consultant.

While acknowledging the topical debate, I should like to report on my own experience of undertaking research as a specialist registrar and the difficulties I encountered. I also discuss the learning opportunities that this exercise offered, and how it might help me in my future role as a consultant.

Background

I have personally been interested in the forensic aspects of adolescent mental health. There is anecdotal evidence that young offenders have mental health problems that contribute to their offending behaviour. By their nature, young offenders are difficult to study, and there have been problems in determining the psychiatric morbidity among this vulnerable population (Nicol et al, 2000). Studies of young adult prisoners have found that a third of them suffer from a primary mental disorder (Gunn et al, 1991). Dolan et al (1999) found high rates of psychosocial morbidity, in addition to high rates of substance misuse disorder, in their sample of young offenders attending court.

The Home Office has established multi-agency youth offending teams (YOTs) under the Crime and Disorder Act 1998 (Home Office, 1998). These teams are expected to have input from probation, education, social services and the health authority. There is, however, continuing uncertainty about the nature of mental health needs among young offenders. I therefore decided to address this issue by investigating the psychiatric profile of this group. The questions to be answered were:

1. • Do young offenders have higher levels of psychiatric morbidity than the general population of the same age (Meltzer, 2000)?

2. • What kind of disorders do they have?

3. • Does psychiatric morbidity correlate with offending?

The research process

I held a series of meetings with the local YOT management to obtain their views on the proposed research. The idea met with their wholehearted approval. It was acknowledged that this research would help identify needs and therefore optimise the allocation of resources towards further mental health input into the team. It was also identified as a good example of a truly ‘ needs-based’ service. I also enrolled into a research training programme (for specialist registrars) organised by the Postgraduate Dean of the South Yorkshire and South Humber Deanery, which introduced specialist registrars to research methodology; helped them to identify, develop and critically evaluate their research protocol, and provided the support of an experienced research team, including the services of a statistician. While attending the programme and developing the research protocol, I held meetings with staff of the YOT to incorporate their ideas, suggestions and improvements into the final protocol.

A number of queries and suggestions emerged from the YOT consultative meetings. Team members were particularly keen that the researcher should not have any direct face-to-face contact with their clients for fear of compromising the latter's privacy, as well as to avoid the stigma of seeing a psychiatrist. The staff made it clear that they would be unable to fill in any questionnaires themselves, and requested a change to the title of the study, to remove any reference to mental health.

It was therefore agreed that the questionnaires would be of the self-report type. Three screening tools were identified as suitable for the study: the Strengths and Difficulties Questionnaires (self-report and parent versions; Goodman, 1997), the Adolescent Wellbeing Scale (Birleson, 1980) and the Substance Misuse in Adolescence Questionnaire (Swadi, 1997), focusing on general screening, depression and substance misuse, respectively. Demographic information was to be collected from the ASSET assessment profile (Youth Justice Board, 1997), which contains details of assessment of the young person carried out by the YOT staff.

Statistical analysis

Statisticians advised that 200 would be an adequate sample size for the measures intended. The intention was to see if a correlation exists between emotional problems, substance misuse and offending. With annual client numbers in the YOT being around 400, it was felt that the study could be completed in 6 months.

Post-launch experience

Two weeks after the launch, it became apparent that no client had been recruited to the study. I visited the team base to remind them that the study was underway. Further attempts were made to recruit clients to the study by sending e-mails to all YOT staff, reminding them of the study. Meetings were held with the sub-team leaders to iron out issues, but there did not appear to be any. I was reassured of the team's full support and participation. However, the following 4 weeks enlisted only three participants. The mental health nurse attached to the YOT advised me to be present on specific days and to request more YOT staff to participate while I was there. Yet, there were no further recruits at the end of 8 months. Finally, a meeting was arranged with the YOT team leader who had been supportive of the research from the outset. At that meeting I was informed that another team was doing research of a similar type with the same client group. They were able to offer a £15 voucher as a token of thanks for the young person's participation in the study; as a result, there was a rush to participate in that project, with the opposite effect on my project. I was asked if I could offer a similar token. As this was not within the protocol, and with no recourse to funding on that scale at that late stage, it was decided to abandon the project, more than a year after it was launched.

Reflections on the research experience

Although the research did not yield the expected results, the exercise was a valuable experience in understanding how research is actually conducted and the difficulties that lie therein.

The subject matter chosen was appropriate and useful. The protocol was built after a prolonged consultative exercise with front-line staff at the YOT. The consultative process itself allowed me to understand other professionals’ anxieties about research and their perceived role in it. It gave me an opportunity to study an organisation from the outside, and understand its management structure and style. Timely enrolment on the research training programme allowed formulation of the research protocol with professional help. It also taught me about research methodology in general, thereby enabling me to apply those skills in practice in the future. Undertaking literature reviews and related reading greatly enhanced my knowledge of the subject. A debriefing exercise, conducted after the research project was terminated, attempted to find answers to the following questions:

1. • What could have been done differently with the research?

2. • Why did the research fail, considering that it appeared to have widespread support and the views of the team were taken into consideration in formulating the protocol?

3. • If the project were to be repeated, what are the pitfalls that should be avoided?

The response of the YOT manager was as follows:

1. • The subject matter was too difficult and sensitive an area for young people to engage with easily.

2. • The component parts of the research, including parental involvement, made completion relatively long-winded.

3. • Members of the team had been involved in several research projects in the recent past, which cumulatively affected their enthusiasm for a new study. The service constantly receives requests from researchers and consultants seeking information about service users, and this can become wearing.

4. • There was no incentive for the young people to participate.

5. • Although a great deal of preparation and consultation did take place and the process was reviewed, not enough staff were fully engaged and committed to the exercise.

Role of the supervising consultant

Both of the supervising consultants and the training scheme research coordinator provided support, guidance and encouragement during the research. One of the supervising consultants helped to arrange a meeting with a senior consultant in adolescent forensic psychiatry. Neither of them would have been able to predict and thereby prevent the outcome.

Conclusion

Specialist registrars need to be given the freedom to identify, formulate and launch research projects; it gives them an opportunity to explore a particular subject in greater detail and thereby add to their own knowledge base. Personally, I found the process of undertaking a research project immensely useful. I learnt that, however hard one may try, there are factors in research for which one cannot prepare enough. The process of research, to me, seemed as valuable as the outcome.