Navigating the challenges of digital health innovation: considerations and solutions in developing online and smartphone-application-based interventions for mental health disorders

Summary

This article presents an analysis of challenges and considerations when developing digital mental health innovations. Recommendations include collaborative working between clinicians, researchers, industry and service users in order to successfully navigate challenges and to ensure e-therapies are engaging, acceptable, evidence based, scalable and sustainable.

Introduction

Digital mental health innovation has seen rapid growth in the past few years, with an increasing range of psychological interventions available online and via smartphone or tablet application (apps). Technology has progressed from computerised cognitive-behavioural therapy (cCBT, for example ‘Beating the Blues’ ¹ ), to mood rater apps and digital interventions that are fully integrated within services, with wearable technology on the horizon (for examples see Harrison et al, ² Richards et al ³ and Lanata et al, ⁴ respectively). The appetite for these ‘e-therapies’ is driven by a multitude of factors, including the extraordinary rate at which society is adopting technology into their everyday lives, ⁵ the pressures on healthcare providers to deliver more for less money, a drive for services to be delivered flexibly in a patient-centred manner and the empowerment that e-therapies can bring to service users by enabling them to make choices about when and how they access psychological care. ⁶

Australia and the USA have been leaders in this field, perhaps unsurprisingly given the need to provide accessible services for their rural and remote communities. However, several government initiatives within the UK have set out to promote digital mental health, including ‘No Health without Mental Health’ ⁷ and ‘Digital First’. ⁸ Now that the UK has woken up to the possibilities that the technological revolution brings within healthcare, there is an exciting opportunity to learn from the lessons learned internationally and take a lead in developing, evaluating and disseminating e-therapies. This is particularly salient for the UK, and many other countries worldwide, as the demand for psychological services goes beyond what is currently provided. As such there is a clear need for mental health services within the National Health Service (NHS) to consider innovative ways to deliver services on a limited budget. Online evidence-based psychological treatments can help improve clinical efficiency, are cost-effective and can ultimately increase access to treatment.

Whereas UK clinical and research communities have been slow at digital mental health innovation, industry has been quick to spot a lucrative and commercially appealing opportunity. Various industry-produced e-therapies have been commissioned across the UK's NHS and apps have been listed on the NHS Choices website (although not currently), despite a lack of scientific evidence underpinning these. ⁹ Although plans are underway to employ an accreditation system for health apps in the UK, led by the National Information Board, ¹⁰ it is unclear whether this will reflect the standards typically adhered to for the recommendation of psychological treatments such as by the UK's National Institute for Health and Care Excellence (NICE).

One of the challenges faced by clinical and research communities is how to maintain sufficient control of the digital mental health revolution to ensure that e-therapies have an appropriate evidence base. This is complicated by the need to balance scientific rigour with the fast pace that technology advances and ultimately to achieve the adoption of evidence-based e-therapies by the NHS and other healthcare providers. Although the evidence base for e-therapies is growing, with sufficient publications to permit various systematic reviews (for example Vallury et al, ¹¹ Reyes-Portillo et al, ¹² Olthuis et al ¹³ ) and a dedicated journal (Internet Interventions), the number of available e-therapies far outstrips the number of evaluations.

Through our experience of digital mental health innovation we have all been struck by the lack of guidance and formalised approaches to development within e-health. As such, this analysis aims to open a discussion about how academics and clinicians can best develop, evaluate and disseminate e-therapies. Although our focus is on navigating these challenges within the UK, the principles apply more broadly and it is hoped that this article will contribute to a framework for how digital mental health innovation can be approached internationally.

Development options

Industry leading development

In the UK, commercial organisations have primarily taken a lead on the development of digital mental health innovations and have produced most of the e-therapies and apps that are available. Examples that are available in the UK include the Big White Wall, IESO Digital Health, ¹⁴ Kooth and Mood District. Commercial organisations, with expertise in online or app-based technology, may have an advantage over healthcare and academic organisations in terms of their skills and experience in both producing and commercialising usable and engaging products. However, they may lack knowledge on the care pathway or the clinical, patient and healthcare system benefits that are required to develop cost- and clinically-effective products. Furthermore, industry may not be best placed to involve service users and healthcare professionals in the co-design of digital products. These considerations highlight the need for effective joint working between healthcare providers and industrial partners.

Some local NHS organisations have commissioned commercially developed e-therapies (such as Big White Wall and Kooth); however, these technologies have not yet been widely adopted across the NHS. A possible reason for this is that these products are reaching the market prior to any formal evaluation of their efficacy and without high-quality peer-reviewed research evidence. An exception to this is ‘Beating the Blues’ which is NICE approved, ¹⁵ although this has recently been found to offer no additional benefit beyond usual general practitioner care. ¹⁶ As many e-therapies are based on CBT, there can be a tendency for developers to claim that they are evidence based as CBT itself may have an evidence base with a particular patient population. Yet clearly the evidence-informed novel e-therapy is not the same as the non-e-therapy version, and may differ in relation to service user engagement, acceptability, therapeutic alliance, processes of change, and ultimately, treatment outcomes. ¹⁷ Although some industrial providers have established academic partnerships to evaluate their products once they have been disseminated, evaluating online treatments retrospectively sits at odds with the principle of evidence-based practice and raises difficulties if the research does not produce favourable results. We recommend involving academic partners from the start to ensure that adequate and appropriate evaluation is built into the development plan.

In-house development within the NHS

Those working within the NHS may have the option of using in-house expertise from their trust's internal website development team. In-house options may seem appealing, as they are likely to appear more affordable than commissioning external organisations. However, this should be balanced with the extensive time required from internal personnel, the need for specific skills and resources and the fit with the organisation's broader strategic vision. Examples of NHS-built innovations that have now been rolled out into clinical services include a moderated social network for service users, first developed for adults with eating disorders (the Support Hope and Recovery Online Network, SHaRON) created in Berkshire Healthcare Foundation Trust, and an SMS texting helpline for school nurses to support young people (ChatHealth) that was set by up Leicestershire Partnership NHS Trust.

As for industry-led developments, rigorous evaluation has not always been central to the development of e-therapies within the NHS, presumably reflecting a lack of time or available resources to undertake research beyond routine service audit. Notably in the most recent reviews of digital interventions for anxiety and depression ^13,18 there were no examples of clinical or health-economic outcomes associated with e-therapies that have been developed within the NHS. Some NHS-led digital innovations for bipolar disorder and psychosis have promising research behind them (for example Simon et al, ¹⁹ Palmier-Claus et al ²⁰ ) but have yet to be adopted at scale across the NHS. ²¹

Individual NHS trusts that develop e-therapies will also need to consider whether and how to disseminate innovations beyond their trust. Disseminating the innovation to other trusts (and healthcare systems) maximises its reach and may provide an income stream for the trust that led the development. Although there may be initial resistance to commission an innovation from other NHS trusts because of a desire to create their own innovations, the growing recognition of the specialist skills and time required to develop and evaluate e-therapies is likely to promote between-trust commissioning going forwards. Effective communication regarding what is already available will be critical to prevent NHS trusts reinventing the ‘digital wheel’, and to enhance learning from both successful and unsuccessful case studies in the NHS.

In-house development within universities

The work of Gerhard Andersson and colleagues in Sweden is arguably the most prolific example of e-therapy development within a university setting. This group have established their own in-house web developer to create psychological interventions ²² that has enabled the rapid and cost-effective development and evaluation of a range of online interventions with demonstrated efficacy compared with a waitlist control (see for example Meyer et al ²³ ) and equivalence to face-to-face treatment (see for example Nordgren et al, ²⁴ Andersson et al ²⁵ ).

An alternative approach that university-based researchers have commonly taken is to use external funding to commission a private company to build, host and maintain a website and/or app. Most of the online treatment programmes for child and adolescent anxiety have taken this approach (for example Vigerland et al ²⁶ Spence et al ²⁷ Morgan et al ²⁸ ). A particular consideration here is that research funding is often fairly slow moving, with lags between bids and awards often being over 12 months. Furthermore, if a tendering procedure is required, this again can be slow, with a UK tender typically taking 6 months and a European Union one up to 9 months. In contrast, digital innovation moves at a fast pace, and what may have been ‘innovative’ at the time the idea was conceived, may become technologically redundant by the time the project starts and technical specifications may need to change. Academics will also need to keep in mind that research funding may cover development, maintenance and hosting of the website/app for a specified period (typically the grant duration) but it is unlikely to cover the costs of maintaining and hosting the online tool beyond this period. Equally, initial funds are unlikely to cover the cost of changes that may be needed to the content or technical specifications, in response to research and development and changes in devices and operating systems. All of these funding limitations create the risk that the e-therapy may become out of date without additional sources of funding being secured. As such, it is important that developers consider scalability and sustainability from the outset of any digital mental health project, including consideration of how the e-therapy will be disseminated within NHS trusts. One approach is to work in partnership with an external company who supply digital mental healthcare solutions and who could take the product to market. This could ultimately result in a revenue stream to maintain and upgrade the system.

Working collaboratively with industry

An alternative approach to digital mental health innovation that has the potential to overcome many of the challenges highlighted above is for clinicians and researchers to work in a collaborative partnership with a website/app development company. Here, clinical and academic partners can contribute clinical insight and research skills while the industrial partner brings technical knowledge and experience of working in the digital health space, and of getting products adopted by the NHS. An example is an online CBT platform that is used routinely across several Improving Access to Psychological Therapies services that was developed collaboratively by Berkshire NHS Foundation Trust, who provided the treatment content, and Silvercloud Health, who developed the technology and market the product to other healthcare providers. Academic partnerships have facilitated research into the efficacy and acceptability of the platform. ^29,30

Collaborations between clinical, academic and industrial partners facilitate the development of innovations that are based on sound psychological theory and clinical practice that has a demonstrable evidence base and that has research and evaluation firmly built into the development phase (i.e. prior to commercialisation). Through a collaborative arrangement all parties are likely to be invested in making a product that is sustainable and scalable. Another advantage of the collaborative approach for industrial partners is that it facilitates access to the views of potential beneficiaries of the product (such as service users) to inform development and to contribute to design and usability testing. Incorporating the views of potential beneficiaries throughout the development process is likely ultimately to lead to the development of a product that is engaging, credible, relevant and valued within services.

Pitfalls of the collaborative approach include potential changes in the direction of the company's business plan that may then deviate from shared goals, or that the company's financial situation may not allow them to continue to support the collaboration, for example if they fold or are taken over. Another potential issue is that the aims of the groups may differ, with the commercial partner likely motivated to take the product in a direction that results in greater financial rewards, whereas the clinical/academic partners may be more concerned with channelling resources into adapting the innovation to maximise effectiveness. Appropriate steps need to be taken when contractual terms are agreed to mitigate these risks.

Development considerations

Across all development options, co-design between clinicians, researchers, industry and service users is crucial. Applying principles from the human factors approach to co-design would help ensure the tool is relevant, usable and engages users. ³¹ Careful consideration should be given to specific populations such as older adults whose ability to work with digital technology is variable, or forensic patients in secure settings who may have restricted access to information technology. There is also an issue about how to best manage and appropriately analyse the enormity of data (‘big data’) generated using digital tools, alongside connectivity complexities, such as ensuring digital tools can integrate into healthcare systems and retain internet connection in different health settings.

Multiagency working

Careful consideration should be given to intellectual property when planning the development of a novel e-therapy. This will be relatively straightforward when developing a digital solution in-house within NHS or academic settings, as the intellectual property will be owned exclusively by that organisation. This will also typically be the case when an external website development company is commissioned to develop a bespoke product. However, there may be limited commercial appeal for industry to work collaboratively in partnership with clinicians/researchers to develop bespoke technologies if they are unable to use this technology in other ventures. Careful identification and assignment of potential intellectual property and the terms under which it will be licensed to the other party/parties is crucial at the outset of any collaboration, and will usually require specialised internal or external legal support.

In the UK, a number of organisations have been set up to encourage the development and evaluation of healthcare innovations such as the NHS England-funded Academic Health Science Networks (AHSNs) and National Institute for Health Research MindTech Healthcare Technology Co-operatives. AHSNs work with a wide group of collaborators, from industry, academia and the NHS to speed up the adoption of proven products and services into the NHS. They also support innovations through the development pathway to ensure that they meet the needs of the NHS and its patients.

Commercialisation

Whichever route is followed to develop and evaluate a novel e-therapy, specific attention should be given to how the product can be disseminated in a way that will generate funds to cover ongoing maintenance, research and development. In the UK, adoption by the NHS is typically the Holy Grail for digital mental health innovation. A clear business model is crucial to achieve this and will need to include consideration of the potential market, what the problem is that needs to be solved in any given care pathway, how the technology or product provides a solution to this problem, the quantifiable health benefits for patients, clinicians, commissioners and the health system itself, how the implementation of this product will take place and the anticipated revenue generation.

E-therapy developers will need to carefully consider the stage in development at which dissemination will occur. The time required to achieve the ‘gold standard’ of evidence from a randomised controlled trial (RCT) contrasts with the fast pace of technological advancements and demand for online access to psychological treatments. Furthermore, for industry, there is limited commercial appeal in waiting years before seeing a return of an investment (particularly if it risks the product going out of date or being pre-empted by a competitor). However, this should not be seen as justification for widespread NHS uptake of products that do not have an evidence base. Instead, developers and funders need to carefully consider ways to evaluate new products efficiently. For example, it may be more efficient to use online technologies to assess and recruit potential participants to an RCT and to evaluate outcomes than to use a face-to-face approach (although this will not be appropriate in all circumstances). There are also a range of study designs, other than the RCT, ³² that may prove useful in evaluating novel e-therapies. Healthcare providers can usefully inform decision-making about what constitutes appropriate evaluation, by clearly specifying the level of evidence that should be required to justify commissioning and insisting that this is met.

Going forward

As the appetite for and provision of e-therapies grows, there is a pressing need for key stakeholders (for example researchers, clinicians, industry, healthcare providers, service users) to join together to consider the issues raised here, and to share information, expertise and experience and to develop collaborations. Indeed, technology, such as virtual conferences and online ‘dating’, can be used to facilitate these interactions. Technological advances such as wearable technology, artificial intelligence and virtual reality also open up a wealth of clinical possibilities. For example, emerging evidence suggests combining experiential virtual scenarios with real-time monitoring using virtual reality and wearable biosensors can reduce anxiety among teachers and nurses. ³³

So far these issues have received limited attention in the academic and clinical literature, although the field has started to utilise key conferences such as the International Society for Research on Internet Interventions as a platform for debate. In navigating the challenges of digital mental health development and dissemination, key stakeholders in the UK could seize the opportunity to learn from the lessons of our international colleagues who have been quick to develop and evaluate online interventions, but are now often struggling to disseminate these into routine clinical practice. Unfortunately, there is still a tendency for clinical academics to focus on the development and evaluation rather than considering how these innovations can be embedded within usual care pathways and ultimately adopted by healthcare providers and their service users.

Recommendations

Based on our analysis of the current issues in the field, our recommendations to clinicians and academics developing online psychological interventions are as follows.

1. • Carefully consider and plan at the start of their project how to scale up and sustain the online intervention beyond the duration of the grant funding.

2. • Develop and commercialise the online intervention collaboratively with industry partners who have the expertise to supply digital health products to healthcare providers.

3. • Develop a business plan for the commercialisation and ensure that health economic analysis is incorporated in research studies.

4. • Consider up front which parties own which aspects of the intellectual property and that this fits with the funding body and organisational stipulations.

5. • Involve service users and other intended end-users in the design.

6. • Work collaboratively with the targeted clinical services to ensure that the online solution meets the needs of the service and can be embedded within their care pathways.

7. • Explore alternative study designs to the RCT that are efficient and provide ongoing evidence for the digital mental health innovation.

8. • Link up with external agencies such as the UK's AHSNs for advice and guidance on commercialisation and adoption by healthcare providers.

We would also argue that there is a pressing need for a dialogue between the key stakeholders and relevant funding bodies in order to raise awareness of the issues highlighted above, and the need to work innovatively to enable successful collaborations with industry that facilitate the adoption of digital mental health innovations by healthcare providers. Furthermore, the UK government should consider applying the approach taken in other countries such as Australia and, more recently, Sweden, in commissioning a national online psychological treatment service. This approach has the potential to ensure that online treatments are developed with a standard, secure platform, have a clear and immediate route to wide-scale dissemination, are delivered by appropriately trained and supervised clinicians with particular expertise in delivering online interventions and provide an unparalleled opportunity for ongoing research and development.

Conclusion

The digital era is undeniably exciting; however, careful planning is required to ensure that e-therapies demonstrate clinical efficacy, cost-effectiveness, acceptability and, crucially, are adopted by healthcare providers and embedded within routine clinical practice to ensure they are available as a treatment option for service users.

Currently digital mental health innovation is expanding beyond the pace of scientific evaluation and it is clear that clinical and research communities need to catch up. Various options for development and routes for dissemination have been presented, alongside their challenges. Although the specifics may become obsolete as e-therapies evolve, we hope that the guiding principles behind our recommendations will remain a useful framework to ensure that the digital era of healthcare is evidence based, loyal to scientific rigour and is informed by and responsive to the needs of services and service users.

Funding

C.H. and C.C. are funded by a National Institute of Health Research (NIHR) Research Professorship () to C.C. J.L.M. is funded by the NIHR MindTech Healthcare Technology Co-operative. The views represented are the views of the authors alone and do not necessarily represent the views of the Department of Health in England, NHS or the NIHR.