The era of digital mental health interventions: we know they can be effective but are they also safe?

Abstract

Over the past two decades, digital mental health interventions (DMHIs) have seen a surge in studies with people experiencing mental ill-health, whether this be via web-based platforms, smartphone applications, text messages or other digital devices. Although DMHIs already demonstrate evidence of their acceptability and some of their effectiveness among different populations, the information about their safety is less clear. This Editorial reflects on a Delphi study by Taher and colleagues that explored the regulation of DMHIs and generated ten safety recommendations. We discuss these recommendations in the context of existing relevant literature and provide suggestions for further steps to advance research and policy on DMHIs in the UK and globally. Further dialog is needed, including the views and experiences of all key stakeholders, and particularly of people with lived experience, to ensure DMHIs are not only an acceptable and potentially effective treatment approach, but also safe for those that use them.

Over the past two decades, we have witnessed a rapid increase in the number of studies using digital mental health interventions (DMHIs), delivered remotely via computers, smartphone applications (apps), text messages or other digital devices such as wearables, aimed at improving various mental health conditions. ¹ Only a decade ago, the acceptability of DMHIs among populations experiencing mental ill-health was still unclear, but systematic reviews ² and individual empirical studies published since then have provided strong evidence of DMHIs being an acceptable and potentially effective treatment option for people with mental ill-health across a range of subpopulations. ³

There is no doubt that such approaches can overcome some of the main limitations of traditional, face-to-face mental health treatments, such as costs associated with face-to-face delivery, restricted access and reach (often tied to clinical settings), along with challenges in supporting continuity of care during critical transition points (e.g. discharge from in-patient treatment). ⁴ However, it is currently less clear if DMHIs present a safe treatment option. The safety of DMHIs, and psychological interventions in general, is typically defined by the occurrence of unwanted effects or negative outcomes, categorised as adverse events or serious adverse events (SAEs). SAEs, which include outcomes like death, disability or hospital admission, are consistently defined because of strict regulatory frameworks. In contrast, there is greater variability in conceptualising and measuring non-SAE, encompassing a broad range of less severe negative effects such as psychological harm (e.g. distress caused by content) and exacerbation of symptoms (e.g. worsening of anxiety or depression). An additional consideration of safety includes risks related to data privacy and security, which are critical in DMHIs, particularly when dealing with sensitive information or in high-risk scenarios, such as suicide prevention apps, where a trade-off between monitoring for safety and ensuring user privacy must be carefully balanced.

One reason for the lack of clarity around the safety of DMHIs is likely related to the fact that, although a small proportion of DMHIs have been tested rigorously in sufficiently powered randomised clinical trials ⁵ (e.g. the Horyzons trial: ⁶ n = 170, found no significant differences compared with treatment as usual in the primary outcome (social functioning), but positive trends in vocational outcomes and use of emergency services; and the EMPOWER trial: ⁷ n = 73, found lower fear of relapse in the EMPOWER group compared with treatment as usual), most DMHIs are available freely on the App store and lack proper evidence of safety and efficacy. As such, the issue is two-fold: first, there is a largely unregulated market supporting the existence of DMHIs, and second, we currently lack a regulatory method that is appropriate for the assessment of DMHIs. Currently, DMHIs are typically classified as ‘medical devices’ and, therefore, are considered to fall under regulatory processes that are responsible for regulating both pharmaceuticals and medical devices (e.g. the Medicines and Healthcare products Regulatory Agency in the UK, Therapeutic Goods Administration in Australia and Digital Services Act in the USA). Merging DMHIs with traditional medical interventions, however, risks overlooking critical aspects specific to mental health contexts.

To address this risk, a recent Delphi study by Taher and colleagues ⁸ involved an international panel of 20 research experts suggesting solutions on how to adapt medical regulatory methods to better suit DMHIs. The strength of this research is in pursuing a regulatory solution that would be tailored to mental health interventions delivered using various technology platforms, given that DMHIs are radically different compared with pharmaceutical interventions, and are characterised by the large heterogeneity of digital approaches and their common use as adjuncts to care, rather than as standalone treatments. Valuable outcomes of the Delphi study are a list of ten safety recommendations that reached consensus and the implementation strategies to support the assessment of the safety of DMHIs (Table 1).

Table 1 Recommendations on how to improve the safety of digital mental health interventions and strategies for their implementation

DMHI, digital mental health intervention.

Reflecting on these recommendations, our own experience with co-production and delivery of DMHIs, ⁴ as well as previous initiatives aiming to capture the strengths and weaknesses regarding the methodology and safety of DMHIs, ⁹ we point out several key elements and steps to advance this research and policy in this field. First, the implementation of a formalised regulatory method is essential for ensuring that DMHIs are both effective and safe, and do not inadvertently increase the risks of harm to their users, ultimately benefiting a broader range of individuals. Second, the safety of DMHIs should be assessed early in pilot feasibility studies, addressing any issues raised by key end users before scaling up. Involving end users throughout the development process of DMHIs early may also improve the implementation of digital treatment approaches, including user engagement with DMHIs, which, in turn, is likely to facilitate positive outcomes. ¹⁰ Third, the need for assessment of the safety of DMHIs early in the research process underscores the importance of regulatory boards such as human research ethics committees being familiar with the guidelines for evaluation of DMHIs, such as the World Health Organization mHealth Evaluation, Reporting and Assessment (mERA) checklist. ¹¹ Such formal regulatory processes would ensure that any risks associated with a specific DMHI to be used in a subsequent study, along with detection and mitigation of these risks, are clearly outlined in the risk evaluation and management plan within a study/trial protocol. Finally, we underscore the need for involvement of people with lived experience, especially those using DMHIs, to tailor these guidelines to specific conditions and risks.

The latter point leads us to a key limitation of the Delphi study by Taher and colleagues: the lack of lived experience, ¹² service user/patient and carer perspectives and an overrepresentation of views from those working in higher education. It is essential that all key end users, including people with lived experience of a specific mental health condition as well as clinicians and other therapists, provide their perspectives on what aspects of safety should be monitored and possibly mitigated when using DMHIs within real-life clinical and community settings. As such, a concept mapping approach may be better suited to capture the perspectives of all of these diverse users.

An example of a successful use of the concept mapping methodology to generate best practice in mental health is the Crisis Resolution Team Optimisation and Relapse Prevention (CORE) study. ¹³ Through involvement of multiple stakeholder groups who prioritised and grouped 72 statements derived from various soruces, ¹⁴ their integrative approach enabled the development of a fidelity scale for crisis resolution teams, which was then piloted widely across UK, ¹⁵ demonstrating feasibility and validity in its use. As such, using a concept mapping approach can ensure a more comprehensive understanding of what constitutes ‘safety’ from diverse perspectives and develop tailored, context-specific measures that reflect the unique features of DMHIs.

Further, involvement of people with lived experience is not only crucial during development, reporting and testing, but also in real-life implementation and monitoring of the intervention usage, ¹² which would provide further reassurance of their safety, or identify needs for further improvement of safety. Additionally, exploration of the role of peer workers/coaches in the delivery and assessment of safety of DMHIs may be an avenue explored further in future research. Adding these voices to the regulations around the safety of DMHIs may be especially important in the context of the increasing complexity of DMHIs, which may involve several end users (e.g. people experiencing mental ill-health, peer coaches, therapists, online peer community).

In line with this next step, it would also be important to include experts and end users from the Global South, ¹⁶ as 85% of experts in the Taher et al study were based in the UK, with the remainder representing countries such as Canada, Australia and the Netherlands. Related to this point is also the need to make regulatory processes more accessible and affordable. As DMHIs developers, we are aware of the high costs and staffing demands associated with proper testing of such treatment approaches. Although national research funding bodies may fund such rigorous testing of a few lucky examples of DHMIs, to ensure the safety of a large majority of existing DMHIs, a different, stepped-level and potentially industry-driven, regulatory approach may be needed. Finally, it is essential to ensure that regulatory standards are proportionate and not unduly stringent for DMHIs, especially compared with pharmacological interventions. For pharmacological treatments, the risk of SAEs can be higher and conceptualised more clearly than in non-pharmacological trials. Although DMHIs do require high-quality safety monitoring, the potential for harm should not be overstated compared with pharmacological trials. Given that these two treatments options involve different risks, safety frameworks should reflect these differences to avoid creating an unfair imbalance in the quality standards between both treatment approaches.

In conclusion, it is evident that the issue of applying regulatory processes pertinent to pharmaceutical and medical devices to DMHIs warrants further dialog. To advance this dialog, considering many more views, including those using DMHIs, would be paramount. This may be especially critical, considering the fast-growing number of innovative DMHI that integrate artificial intelligence and related data collection methods (e.g. photos, posts on social medial platforms). Further, although implementing the recommendations from the Delphi study has the potential to enhance the safety assessment processes for DMHIs, embedding these recommendations into existing frameworks such as World Health Organization mERA ¹¹ checklist can present a useful step toward establishing formal regulatory methods for critical assessment and transparency in reporting of risks associated with digital technologies that aim to improve health outcomes across populations with mental ill-health. Improving the quality of mental health safety data, refining the risk mitigation processes and conducting more accurate risk–benefit analyses have the potential to lead to more effective and safer DMHIs, which, in the current digital era, have a strong potential to become a useful treatment partner across a wide range of mental health settings.