Recruitment and retention in clinical trials
Recruitment and retention are two of the biggest challenges in conducting randomised controlled trials (RCTs). For example, a recent review of RCTs funded by the UK's National Institute for Health Research (NIHR) found that recruitment and retention varied widely across trials (N = 151), with only 56% of trials recruiting to their pre-specified target sample size.Reference Walters, Bonacho dos Anjos Henriques-Cadby, Bortolami, Flight, Hind and Jacques1 Problems with low recruitment and retention are significant as they result in a loss of statistical power, unrepresentative samples of participants, biased follow-up data in the case of participant attrition and ultimately, a waste of funders’ money and participants’ time. These challenges are especially prevalent in conducting research with people with severe mental health difficulties, who are often considered a particularly difficult clinical population to recruit and retain in trials.
Systematic reviewsReference Briel, Speich, von Elm and Gloy2–Reference Brintnall-Karabelas, Sung, Cadman, Squires, Whorton and Pao4 have identified barriers to trial recruitment, which include issues related to funding, design, recruiter and participant. Studies focused on barriers to recruiting to trials involving people with severe mental health diagnoses highlight the relevance of factors that are apparent in RCTs in general. These include misconceptions about the different groups in the trial (trial arms), lack of equipoise, variable interpretations of eligibility criteria and lack of staff time and resources to support recruitment. There are also additional factors that are unique to recruiting participants with mental health diagnoses. For example, staff paternalism can lead to excessive gate-keeping, and poor staff–patient relationships may make staff reluctant to approach potentially eligible participants about research opportunities.Reference Howard, de Salis, Tomlin, Thornicroft and Donovan5–Reference Bucci, Butcher, Hartley, Neil, Mulligan and Haddock7
Particular challenges of in-patient trials
Previous reviews demonstrate the benefits of psychological therapies for severe mental health problems, in terms of improvements in symptoms and reduced costs to the National Health Service (NHS).Reference Jones, Hacker, Cormac, Meaden and Irving8,Reference Wykes, Steel, Everitt and Tarrier9 However, recent systematic reviews have found that very few controlled trials of psychological therapies target psychiatric wards.Reference Jacobsen, Hodkinson, Peters and Chadwick10–Reference Wood, Williams, Billings and Johnson12 Although there is good evidence for psychological therapies from high-quality trials with out-patients, existing evidenced-based psychological therapies need to be adapted and then trialled within in-patient settings. The process of delivering therapy in in-patient settings is likely to present unique challenges that require empirical investigation. For example, in-patients are likely to be experiencing higher levels of distress, may be at high-risk of self-harm/suicide,Reference Kapur, Hunt, Windfuhr, Rodway, Webb and Rahman13 and the relatively short length of stay (average 31 days in UK)14 may affect the potential length of therapy. There are also many environmental factors to consider, such as noise and disruption on the ward,Reference Brown, Rutherford and Crawford15,Reference Camuccio, Sanavia, Cutrone, Marella, Gregio and Cabbia16 a lack of appropriate spaces in which to conduct therapyReference Papoulias, Csipke, Rose, McKellar and Wykes17 and interruptions to therapy sessions.Reference Jacobsen18
There is a perception of general reluctance to conducting RCTs on in-patient wards, given that it is often viewed as a chaotic and uncontrollable environment, with in-patients being frequently admitted and discharged. There is also an increased need to consider attitudes of in-patient staff toward the provision of therapy in this setting. For example, staff can be concerned that it's not ‘the right time’ to offer therapy, or that admissions are not long enough to deliver any kind of meaningful therapy.Reference Wood, Williams, Billings and Johnson19
It is clearly important to overcome these barriers to conducting in-patient trials, given that there is an urgent need to increase access to evidence-based psychological therapies in these settings. On the back of significant lobbying by patient and carer groups, the Mental Health Task Force (2016) now advocates a ‘referral to treatment access’ standard for psychological therapy in acute in-patient care, meaning that NHS mental health trusts will need to deliver timely, evidence-based psychological treatments in these settings.20
The aim of the current paper is to describe strategies that we believe have helped us to achieve good recruitment and retention in these trials, with the aim of helping others carrying out trials with similar populations in similar settings.
Method
In this section we present three case studies of trials conducted on NHS psychiatric in-patient wards in the UK, to provide the wider context for our top tips for recruitment and retention for in-patient trials. We chose these three trials based on the senior authors (P.J., K.B., G.H.) experiences of conducting in-patient trials, and as exemplars of in-patient trials that had high recruitment and retention rates.
The TULIPS trial (Talk, Understand, Listen for Inpatient Settings) is a cluster-randomised RCT currently open to recruitment (prospectively registered on Clinicaltrials.gov under identifier NCT039503882, registered on 15 May 2019). The aim of the study is to randomise 34 wards in psychiatric hospitals across the UK to receive either treatment as usual (TAU) or a newly appointed half-time senior clinical psychologist over a 7-month period. The intervention being evaluated is a stepped model of care delivered by the psychologist, including direct therapeutic work with in-patients and indirect work with ward staff. Key outcome measures for the trial are ward level incidents of violence, aggression and self-harm, and self-reported in-patient well-being, social functioning and service use, staff burnout and both staff and in-patient perceptions of ward atmosphere. Both staff and in-patients are required to complete outcome measures at baseline, 6-month and 9-month follow-ups, with the majority of in-patients being followed up in community settings post-discharge. Inclusion criteria for both staff and in-patients are capacity to consent and able to complete self-report measures. In-patients are potentially eligible for participation regardless of diagnosis or main psychiatric symptoms. The trial is ongoing, with staff and in-patient recruitment rates currently on track for at least 80% of the prespecified target.
The amBITION trialReference Jacobsen, Peters, Robinson and Chadwick21 is a completed feasibility RCT of a brief therapy for psychosis, which was conducted on four wards within a psychiatric hospital in London, UK (prospectively registered on the ISRCTN Registry under identifier ISRCTN37625384, registered on 19 August 2015). In addition to TAU, participants were randomly allocated to receive either mindfulness-based crisis intervention or a control intervention (social activity therapy), for one to five sessions. All sessions followed a stand-alone, self-contained format, to accommodate unpredictable lengths of stay and unexpected discharges. The treatment phase was restricted to the duration of the in-patient admission (maximum 5 weeks). Therapy was delivered by a single trial therapist, who was not part of the usual ward team. Primary outcomes were feasibility of recruitment and retention, and secondary clinical outcomes were readmission to hospital, and self-report measures of psychotic and affective symptoms. Measures were taken at baseline, discharge and 3- and 6-month follow-up. Inclusion criteria were people newly admitted to in-patient care, presenting with positive symptoms of psychosis, in the context of a schizophrenia spectrum or affective disorder, and having capacity to consent to participate. Fifty participants were recruited (83% of target sample size of 60), and follow-up rates at 6 months post-discharge were 98% for service use data extracted from clinical notes, and 86% for self-report questionnaire measures. Recruitment and retention rates both exceeded pre-set feasibility benchmarks.Reference Jacobsen, Peters and Chadwick22
The INSITE trial (Inpatient Suicide Intervention and Therapy Evaluation)Reference Haddock, Pratt, Gooding, Peters, Emsley and Evans23 is a completed feasibility and acceptability RCT of a psychological therapy for in-patients who are suicidal, which was conducted on wards within a NHS trust in the North-West UK (retrospectively registered on the ISRCTN Registry under ISRCTN17891026, registered on 22 April 2015). In-patients were randomly allocated to receive either cognitive–behavioural suicide prevention in addition to TAU, or TAU alone. The intervention consisted of TAU plus up to 20 cognitive–behavioural suicide prevention sessions, delivered over 6 months by experienced therapists who were not part of the usual ward team. The intervention began in the in-patient setting but continued into the community if a participant was discharged before they had completed treatment. Primary outcomes were feasibility and acceptability of the intervention, and secondary outcome measures included suicidal thinking, behaviours, functioning, quality of life and service use. Measures were taken at baseline, 6 weeks and 6 months (end of treatment). Inclusion criteria were in-patients experiencing suicidal thoughts or behaviours within the 3 months before admission (any diagnosis), and having capacity to consent to participate. Fifty-one participants were recruited (85% of target sample size of 60), and follow-up rates at both 6-week and 6-month follow-up were 73%, which was slightly lower than the anticipated 80% retention rate.Reference Haddock, Davies, Evans, Emsley, Gooding and Heaney24
Ethics approval and consent to participate
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All procedures involving human patients were approved by the following areas: North West – Greater Manchester East Research Ethics Committee (approval number: 19/NW/0316; TULIPS), London – Camberwell St Giles Research Ethics Committee (approval number: 15/LO/1338; amBITION) and NRES Committee North West – Lancaster (REC reference number: 13/NW/0504; INSITE). All participants gave written consent.
Results
Successful recruitment and retention: ten top tips for success
The first priority should always be to build, and maintain, a strong relationship with all members of the multidisciplinary ward team. Establishing support from senior management (e.g. ward manager and overall service managers) is necessary, but not sufficient if it is not combined with buy-in from the overall ward team (including healthcare assistants, nurses, occupational therapists, psychiatrists and administrative staff) who will be involved in the trial on a more day-to-day basis.
The second tip is that the likelihood of getting strong buy-in from the team is greatly increased if the ward staff have been consulted and involved in the design of the trial and intervention, rather than feeling like it has been imposed on them, if this is possible. To further enhance staff engagement, using terminology such as ‘joint NHS and university project’ can be helpful, as opposed to ‘research trial’, which can be perceived as external to the usual work of the ward.
The third tip is that ward teams are always busy, and they have many competing demands on their time and attention, so make it as easy as possible for them to remember the trial. It is advisable to gather information from the ward manager about the best times to visit the ward, noting down the ward routine (e.g. ward round days).
The fourth tip is that relying solely on busy ward staff to regularly screen and approach in-patients will most likely lead to poor recruitment, and potentially eligible participants will be missed. It is therefore best to ensure that proactive screening of new referrals for the trial is done by research staff, fitting in with usual ward routines and liaising regularly with ward staff.
The fifth tip is that research staff must be prepared to repeat information about inclusion and exclusion criteria on a regular basis, as ward teams can have a high turnover. Conversations in person can be supplemented with simple information sheets tailored specifically for staff, but reliance on written material only should be avoided.
The six tip is that it is important to listen to staff and respect their views about who to approach about the study. However, this needs to be balanced with the need to protect against ‘gate-keeping’, which might inadvertently prevent potentially eligible people from being offered the chance to participate.
The seventh tip is that wards can be chaotic environments, and sometimes in-patients may be taking leave off the ward, may be asleep, with other visitors or not feel up to meeting when research staff have arranged to meet with them. However, it is essential that research staff build a reputation for reliability by always turning up at the agreed time and date for all appointments.
The eighth tip is to randomise participants as close to the start of treatment as possible. Even a 24 h window can lead to losing participants between randomisation and start of treatment because of unplanned discharges or other unforeseen factors.
The ninth tip is that to successfully follow people up after they leave hospital, it is important to be flexible, prepared and willing to go the extra mile. People often change their mobile numbers, sometimes do not have battery or credit, or do not like answering the phone. Therefore, aim to get as many different types of contacts for a person as possible, and seek permission to contact via a trusted third party where appropriate.
The tenth and final tip is that doing follow-up assessments at the same time as routine clinical appointments, such as when people meet with their care coordinator, can be very helpful to save the need for multiple trips. Not all participants will be happy to meet in a healthcare setting – in this case, be flexible and offer alternatives such as home visits (visiting in pairs, or with members of the community team is recommended if there are unknown risk to researchers). Building up a good working relationship with community teams will help facilitate this joint working.
See Table 1 for examples of these approaches across the three case study trials.
Table 1 Summary of top ten tips for successful recruitment and retention
TULIPS, Talk, Understand, Listen for Inpatient Settings; amBITION, Brief Talking Therapies on Wards; INSITE, Inpatient Suicide Intervention and Therapy Evaluation; TAU, treatment as usual; CMHT, community mental health team.
Discussion
Recruiting to target, and retaining participants at follow-up, is challenging but possible for RCTs of psychological interventions conducted on psychiatric wards. This paper aimed to share strategies from three different in-patient trials that successfully recruited and retained participants, to disseminate good practice for the benefit of other researchers planning future research in similar areas.
The themes running through our tips primarily relate to the importance of relationships between the research team and clinical staff; good stakeholder involvement and getting early buy-in from the team; working effectively with ward staff who take on gatekeeper roles; and adapting to the particular demands of the clinical setting. Our findings are consistent with previous literature on successful recruitment to trials involving people with severe mental health problems that have also emphasised the key importance of the relationship between researchers and recruiting clinicians,Reference Peckham, Arundel, Bailey, Callen, Cusack and Crosland25 and support from senior staff to encourage staff participation.Reference Berry, Haddock, Kellett, Awenat, Szpak and Barrowclough26 A large-scale survey of clinical studies officers in mental health research networks (N = 170) found that a positive attitude toward the study among gatekeepers was the most commonly endorsed factor that was seen as facilitating trial recruitment. Conversely, gatekeeper perceptions that the intervention being tested would not likely be effective or appropriate for the target population was one of the most commonly reported barriers to recruitment, as it was seen as leading to staff being paternalistic and overprotective.Reference Borschmann, Patterson, Poovendran, Wilson and Weaver27 This is particularly relevant to recruitment of clinical populations such as people receiving in-patient psychiatric care, as staff may hold unhelpful beliefs about psychological interventions not being helpful, or potentially being destabilising at this point in the care pathway.Reference Wood, Williams, Billings and Johnson19 Similarly, a previous study has found that mental health nurses can be reluctant to refer people with psychosis to take part in trials if they think they would decline to participate, or may lack capacity to consent, even if this is not the case.Reference Mairs, Lovell and Keeley28 However, previous research shows that patients in general appreciate the opportunity to be invited to take part in research trials because they would like to help others with a similar condition, and to advance scientific knowledge in general,Reference Kost, Lee, Yessis, Coller and Henderson29,Reference McCann, Campbell and Entwistle30 and this has also been found to be the case for people diagnosed with severe mental health problems.Reference Thong, Ulph, Barrowclough and Gregg31 Talking to staff about their concerns and providing relevant counter-information can help allay such concerns.
Working in partnership with key stakeholders, including patients, carers and families, service managers and front-line clinical staff, is crucial for successful recruitment. Embedding these processes early on the process, and maintaining them throughout, is important as service managers and staff can feel reluctant to refer people into a trial they feel was approved ‘above their heads’.Reference Patterson, Kramo, Soteriou and Crawford6 This approach also helps to overcome any prior pessimism from staff regarding the likelihood of adopting research into practice following the completion of the research period, based on previous experiences of research.Reference James, Quirk, Patterson, Brennan and Stewart32 Theories from the field of implementation science, such as normalisation process theory,Reference May and Finch33 similarly emphasise the importance of staff ‘buy-in’ when implementing any new practice into a complex system, so that they can make sense of what they are being asked to do, and integrate it smoothly with existing practice. Relationships need to be built interpersonally – winning ‘hearts then minds’, as Patterson and colleagues put it.Reference Patterson, Mairs and Borschmann34 Relationships can be built effectively by establishing common ground between the values and intentions of the research team, and the clinical team (e.g. improving access to effective therapies).
Adapting to the fast-paced and often unpredictable nature of acute psychiatric care is also essential. We found that tailoring the trial design to the clinical setting, such as having more frequent screening visits to the wards and minimising the time window between randomisation and start of treatment, were crucial to success. These are analogous to adaptations already used in clinical trials in other challenging settings, such as hospital emergency departments.Reference Ni, Bermudez, Kennebeck, Liddy-Hicks and Dexheimer35,Reference Cofield, Conwit, Barsan and Quinn36
In writing this paper, we draw from real-world knowledge of conducting clinical trials from three different studies, conducted on different wards across a range of geographical locations. However, as a limitation we fully accept that our top tips are based on our own anecdotal experience, so we may be mistaken in thinking that the factors we identified were in fact responsible for effective recruitment and retention. We may also have been unaware of other factors that actually had a more significant effect. This knowledge gap on recruitment strategies of proven effectiveness has previously been highlighted by two systematic reviews of effective strategies to improve trial recruitment.Reference McDonald, Knight, Campbell, Entwistle, Grant and Cook37,Reference Treweek, Lockhart, Pitkethly, Cook, Kjeldstrøm and Johansen38 The 2006 review by McDonald and colleagues concluded it was unclear what trial factors predicted good recruitment, based on a sample of multi-centre trials (N = 114).Reference McDonald, Knight, Campbell, Entwistle, Grant and Cook37 The later 2013 Cochrane Review of trials testing the effectiveness of various recruitment strategies (N = 45) also found a limited evidence base, and highlighted that some of the studies included in the review had in fact used hypothetical trial scenarios to evaluate various recruitment strategies, so generalisability to real-world trials was questionable. Similarly, a study on loss to follow-up in a large, multi-centre treatment trial (N = 1117 trial participants) concluded they could not identify any strong predictors of loss to follow-up, despite testing multiple participant, clinician, and centre variables.Reference Eichler, Schutzwohl, Priebe, Wright, Adamowski and Rymaszewska39
In future research, one way of addressing this knowledge gap would be to encourage the greater use of SWATs (studies within trials) as routine practice in psychiatric in-patient trials. A SWAT is defined as a ‘self-contained research study that has been embedded within a host trial with the aim of evaluating or exploring alternative ways of delivering or organising a particular trial process’.Reference Treweek, Bevan, Bower, Campbell, Christie and Clarke40 Making the routine inclusion of SWATs into future trials would make a significant contribution to the evidence base on what strategies are effective; for example, testing if it makes a difference who makes the initial approach to a potential participant. A future systematic review could also address the question of how recruitment and retention rates for psychology in-patient trials may be changing over time, following a similar methodology to the Walters et al review of NIHR-funded trials.Reference Walters, Bonacho dos Anjos Henriques-Cadby, Bortolami, Flight, Hind and Jacques1
Ensuring good recruitment and retention for RCTs conducted on psychiatric wards is important to reduce research waste, and to further the evidence base on the effectiveness of psychological interventions within these uniquely challenging settings. When planning and designing future in-patient trials, research teams should aim to draw both on literature from the wider trials evidence base where relevant, in addition to the in-patient-specific literature.
Data availability
Data availability is not applicable to this article, as no new data were created or analysed in this study.
Author contributions
P.J., K.B. and J.R. conceptualised the idea for the paper. P.J. and K.B. wrote the original draft. All authors (P.J., K.B., J.R., G.H., C.P. and R.W.) contributed to reviewing and revising the manuscript. All authors (P.J., K.B., J.R., G.H., C.P. and R.W.) approved the manuscript for submission. All authors (P.J., K.B., J.R., G.H., C.P. and R.W.) read and approved the final manuscript.
Funding
This paper presents independent research funded by the National Institute for Health Research (NIHR), under its Programme Grants for Applied Research (RP-PG-0216-20009; TULIPS), NIHR Doctoral Research Fellowship (DRF-2014-07-003; amBITION) and Research for Patient Benefit Programme (PB-PG-1111-26026; INSITE). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. The funding body had no role in the design of the study, collection, analysis, interpretation of data or writing of the manuscript.
Declaration of interest
None.
Open access
eLetters
No eLetters have been published for this article.