Pragmatic neuroscience for clinical psychiatry

Summary

Mental health and substance use disorders are the leading cause of long-term disability and a cause of significant mortality, worldwide. However, it is widely recognised that clinical practice in psychiatry has not fundamentally changed for over half a century. The Royal College of Psychiatrists is reviewing its trainee curriculum to identify neuroscience that relates to psychiatric practice. To date though, neuroscience has had very little impact on routine clinical practice. We discuss how a pragmatic approach to neuroscience can address this problem together with a route to implementation in National Health Service care. This has implications for altered funding priorities and training future psychiatrists. Five training recommendations for psychiatrists are identified.

Declaration of interest

J.D.S. receives direct funding from MRC Program Grant MR/S010351/1 aimed at developing machine learning-based methods for routinely acquired NHS data and indirect funding from the Wellcome Trust STRADL study. M.P.P. receives payments for an UpToDate chapter on methamphetamine and is principal investigator on the following grants: NIGMS P20GM121312 and NIDA U01 DA041089 and receives support from the William K. Warren Foundation.

Introduction

There is a remarkable need for progress in the practice of clinical psychiatry. Mental illness and substance use disorders are the leading cause of disability worldwide with major depressive disorder (MDD) the commonest cause,^1,2 suicide is a leading cause of death in young adults,² and severe and enduring mental illness is associated with a reduction in lifespan of about a decade.³ However, it is widely recognised that clinical practice in psychiatry has not fundamentally changed for over half a century.^4,5

During this time neuroscience has developed a range of sophisticated techniques allowing brain structure and function to be non-invasively investigated in unprecedented ways.⁶ Magnetic resonance imaging (MRI) allows non-invasive mapping of brain structure and function (fMRI), magnetoencephalography allows mapping of magnetic fields occurring naturally in the brain complementing electroencephalography (EEG) and positron emission tomography allows mapping of neurotransmitters. Countless neuroimaging studies have reported statistically significant differences between groups of patients and healthy controls and findings from ever larger molecular genetics studies are being reported. However, despite these incredible scientific and methodological advances, no progress has been made in applying neuroscience pragmatically to psychiatry.

Responding to this need, the goal of pragmatic psychiatric neuroscience⁷ is to develop neuroscience-based objective quantitative markers that aid clinical decision-making and have implications for individual patients, for example by objectifying diagnosis, quantifying prognosis, supporting treatment selection and yielding objective severity markers for disease monitoring. For example, it can often take months or years of trying different antidepressants to find one that is effective (if any) for a particular patient. There is good evidence for some antidepressants being more effective and tolerable than others,⁸ yet different patients respond to different antidepressants without a clinically obvious pattern.⁹ We would like to be able to say, ‘Mrs Smith, your depression test has come back and with this particular antidepressant, you have a 90% chance of being symptom free in 6 weeks’.¹⁰

Traditional group-level statistical significance framework

There are various reasons for the limited impact of neuroscience on clinical psychiatry.

First, reproducibility of research findings has been a problem.¹¹ Only recently have very large sample sizes become available to provide definitive group-level results: for example the ENIGMA Consortium recently reported a meta-analysis of structural brain scans from 8921 patients with MDD and controls from sites worldwide.¹² Patients had significantly lower hippocampal volumes with the authors concluding their study robustly identified hippocampal volume reductions in MDD.¹² This is a definitive result, however, it is not clinically useful for individual patient management, as will be discussed. Far larger psychiatric genetics studies typically failed to identify significant findings, although a recent study with 480 359 participants with MDD and controls did report significant findings.¹³ Very small genetics effect sizes are unlikely to be useful for decision-making with individual patients; a different statistical framework is required.^14,15

Second, psychiatric disorders are fundamentally multivariate constructs. When taking a psychiatric history and conducting a mental state examination, multiple symptom and social areas are explored before decision-making such as diagnosis and treatment selection. Therefore it should not be surprising that a single univariate measure, such as a single interview question, hippocampal volume or a single genetic measure is insufficient to substitute for this. Indeed, the National Institute of Mental Health, Research Domain Criteria (RDoC), currently has five domains of human function within which there are multiple ‘units of analyses’.^16,17

The main problem with a group-level statistical significance framework is that it only allows inferences about group differences (for example average hippocampal volume in MDD versus controls) and not individual patients. Using this approach with psychiatric disorders, there is invariably substantial overlap in the distribution of a single variable for individual patients and controls, meaning that if such a measure is used in an attempt to make a clinically relevant prediction for an individual patient, the predictive accuracy is so low as to be clinically useless (Fig. 1(a)). No increase in study size will change this and no single measure has ever been identified that is clinically useful. Group-level statistical significance is not the same as individual patient clinical significance.

(a) Representation of the distribution of a single variable, such as hippocampal volume, for controls with mean (mc) volume and patients with a smaller mean (mp) volume. Assuming a hippocampal volume reduction in major depressive disorder (MDD) with Cohen's d = 0.14 and 8921 participants (see ref. 12), the two-group t-test (MDD versus control) difference in volume is highly significant (P = 3.8 × 10^–6). If a cut-off (cut) is defined that balances sensitivity and specificity, then the true positive (TP), false positive (FP), true negative (TN) and false negative (FN) rates can be calculated, indicating very low diagnostic sensitivity, specificity and accuracy of 52%, 53% and 52%, respectively (50% random). (b) Two variables are measured for each participant, both of which have highly overlapping patient and control distributions. Each point represents a patient (black) or control (grey) participant. Machine learning, such as a support vector machine, identifies the maximal distance ‘d’ hyperplane separating the two groups during training. When data from a new participant becomes available, then whichever side of the hyperplane the new data appears, defines the prediction (for example MDD versus control). Only two predictor variables are shown for clarity, normally a large number of variables are used.

Fig. 1 Group-level significance, univariate and multivariate prediction.

Alternative individual patient pragmatic risk-prediction framework

A multivariate risk-prediction framework may provide a better approach to generate clinically useful predictions about individual patients. The strengths of this approach include not being dependent on first understanding psychiatric nosology, which is controversial, and not requiring formulation of illness ‘mechanisms’ that are yet to be identified. Instead it uses predictors (covariates) to estimate the absolute probability or risk that a certain outcome is present (diagnostic) or will occur within a specific period (prognostic) in an individual with a particular predictor profile, so having an immediate impact on clinical practice.¹⁰ Further strengths of this approach are a clear utilitarian approach, sound statistical basis, a framework for iterative improvement and compatibility with a mechanistic understanding of psychiatric disease.⁷ This type of approach is already used in other areas of medicine: such as risk calculators for predicting an individual patient's risk of a vascular event or cancer within a given period.

Different computational methods can be used to implement a risk-prediction framework: ‘machine learning’ is a collective term for a set of methods that can be used to train a predictor to make individual patient predictions. At its simplest, a sample of data is split into two parts and one part used to train a predictor with the other part of the data, which has not been used for training, used to test the predictive accuracy, sensitivity and specificity. This is cross-validation, also known as within-study replication. The testing data is equivalent to data from newly presented patients we want to make predictions for. Crucially, even when the distributions of individual variables strongly overlap, it is possible for a machine learning approach to make accurate individual patient predictions (Fig. 1(b)).

Proof of concept studies for individual patient prediction

Prediction methods require quantitative data and these vary in terms of objectivity, practicality and accessibility to routine clinical services. Subjective quantitative data can easily be generated from clinical rating scales, for example the Hamilton Rating Scale for Depression.¹⁸ These are subjective because patients are asked to match the way they have been feeling to a choice of different statements. There are recent National Health Service (NHS) initiatives to record subjective data for individual patients, such as brief symptom measures acquired as part of the Improving Access to Psychological Therapies (IAPT) in England.¹⁹ Rating scales with machine learning have been used to discover subtle patterns of differential treatment response,^20,21 although more objective measures might be more reliable predictors and provide better insight into abnormal biology.

Objective data includes items such as age and gender but are of limited use because they contain little information. The most readily available ‘information rich’ source of data is likely a T ₁-weighted MRI brain ‘structure’ scan. It is objective because it requires no subjective judgement by a patient and minimal cooperation (lying still). It is practical because it takes ~4 min to acquire and readily available because all NHS radiology departments with an MRI scanner can acquire such data without extra equipment. We provide a few examples of ‘proof of concept’ studies that provide cross-validation estimates for individual patient predictions. If useful for routine clinical practice, further work would be required to develop the technique for an NHS environment.

Example 1: prediction of MDD diagnosis and illness severity

A multicentre study tested if it was possible to make an accurate diagnosis of MDD for individual patients using only T ₁-weighted brain ‘structure’ scans.²² A machine learning technique was used and a high diagnostic prediction accuracy of ~90% for individual patients was reported. An independent study²³ reported 85% accuracy of individual patient diagnostic prediction, and using fMRI data, 97% cross-validation diagnostic accuracy.²⁴ Successful prediction of MDD illness severity using T ₁ brain ‘structure’ scans has also been reported.²⁵

Example 2: prediction of drug-relapse for abstinent methamphetamine users

Accurate prediction of relapse for patients who are currently abstinent but previously drug misusing is not possible using standard clinical measures.⁷ Fifty-eight methamphetamine dependent patients were recruited from an in-patient treatment programme and participated in a stop-signal task during fMRI. These patients were prospectively followed for a year then reassessed. Using fMRI measures of brain activity in combination with a computational model of behaviour and brain activity, relapse was predicted with accuracy ~78% using cross-validation.⁷

Example 3: prediction of cognitive–behavioural therapy (CBT) outcome for anxiety disorders

It is difficult to accurately predict outcome of CBT and such treatment requires access to extended therapist time, which is expensive. Before receiving CBT, adults with generalised anxiety disorder or panic disorder took part in an emotional regulation task during fMRI.²⁶ Standard clinical and demographic variables predicted individual patient clinical outcome with 69% accuracy: fMRI measures predicted individual clinical outcome more accurately at 79%.²⁶

Example 4: prediction of antidepressant outcome for depression

There is no empirically validated method to determine whether a patient will respond to a specific antidepressant. Using STAR*D ratings and a machine learning approach it was possible to predict individual patient symptomatic remission from a 12-week course of citalopram with 60% accuracy.²¹ The predictive model was validated in an independent sample of patients treated with escitalopram. Further development of predictive techniques to select the best antidepressant for individual patients with MDD would be beneficial. This could be achieved by: (a) including a larger number of questionnaires than STAR*D to identify a better list of questions and (b) use of EEG/MRI measures, which are likely to be useful for predicting antidepressant response.²⁷

Example 5: prediction of dementia and predictions for non-psychiatric disease

Predicting the risk of dementia for patients presenting with mild cognitive impairment is important clinically as it would (a) facilitate already available preventative interventions for vascular disease (vascular dementia) and (b) for Alzheimer's disease allow recruitment into clinical trials of novel medications for patients who do not have advanced disease. Proof-of-concept studies have been published for some time: for example Zhang & Shen reported 78% accuracy of dementia diagnosis at least 6 months ahead of baseline scanning.²⁸ Commercial interest in predictive healthcare computing in non-psychiatric areas is booming: such as Google DeepMind's partnership with the Royal Free London NHS Foundation Trust (acute kidney injury detection) and with University College London hospitals (head/neck cancer and retinal imaging at Moorfields).

Summary and implications

These examples show that using a predictive framework with neuroscience techniques it is possible to make objective clinically useful predictions for individual patients: diagnoses, illness severity, drug abstinence relapse, psychotherapy and medication treatment outcomes. It is important to note that machine learning, like any technique, can be misapplied so care is needed in its use.^29–32 Crucially: (a) samples need to be representative of the NHS populations of interest, (b) calculations need to be properly cross-validated and tested against independent samples, (c) different machine learning methods should be compared and (d) careful analyses need to be done to determine whether there are hidden confounders (for example artefacts) driving what the algorithms use to predict.

The ultimate aim of these multivariate approaches is to optimise individual patient care. This means a series of studies are required, starting with relatively inexpensive single studies correctly using cross-validation, then more expensive larger replication studies with independent data, then prospective clinical trials as part of NHS care. There are some initiatives, for example patients with first-episode psychosis in the South London and Maudsley NHS Trust are being assessed using neuroimaging and blood tests as such measures may allow prediction of antipsychotic response.^33,34 In addition, these predictive approaches can also be used to aid illness mechanisms research: it is possible to examine the neurobiological differences between those individuals who were successfully predicted to respond to a treatment versus those that were successfully predicted to not respond to a particular treatment. However, to date, no predictive technique for psychiatry is under development in an NHS environment. NHS implementation issues will now be considered.

Implementation in the NHS

It seems inevitable that machine learning based techniques will start to be introduced into non-psychiatric areas of NHS healthcare before long: for example researchers at John Radcliffe Hospital in Oxford have developed a machine learning technique for cardiac scans that greatly outperformed consultants. The route to adoption of new NHS technologies in the UK is well established, first by establishing sufficient evidence for the benefit of the intervention by suitable clinical trials and economic studies of cost feasibility, then assessment of a sufficient level of cost–benefit evidence by the National Institute for Health and Care Excellence (NICE), then recommendation by NICE of adoption by the NHS and finally implementation by the NHS. However, adoption of new techniques is challenging because of the UK government's ‘austerity’ fiscal policy that restricts spending for routine NHS care and other public sector areas and that has been in place for a decade.

The route to NHS implementation of predictive techniques for psychiatric disorders is not fundamentally different; however, there are additional barriers. Quantitative data is required and can be readily generated, but NHS psychiatric services routinely collect very little quantitative data relevant for individual patients. Only patients in old age services presenting with mild cognitive impairment routinely receive an MRI scan as part of assessment. Even for patients presenting with MDD, it is not part of routine practice in general adult psychiatry to quantify illness severity using an established rating scale despite, for example the Hamilton Rating Scale for Depression being available since 1960¹⁸ although brief symptom measure collection is increasing with IAPT. Instead the UK focus remains qualitative clinical impressions, social interventions and health service reorganisation. Without a change in culture, significant clinical progress seems unlikely.

UK stakeholders, funding and psychiatric research and development (R&D)

UK stakeholders are patients with mental illness and addictions, their families and politically influential organisations representing them. Individual patients are often very supportive of biomedical research into mental illness and addictions, which has allowed recruitment of vast genetics studies and huge neuroimaging studies. However, psychiatry is unique as a medical speciality in having a section of society opposed to a biomedical approach plus staff in community mental health teams do not always accept that mental illness has a biological component. Other than the newly emerged MQ charity, there are no UK mental health charities supportive of biomedical R&D. The Medical Research Council, Wellcome Trust and National Institute for Health Research support biomedical research into psychiatric disorders. However, only approximately 5% of total UK medical funding is spent on mental illness³⁵ and a small number of London Universities, Oxford and Cambridge receive ~46% of all UK public and charity funding,³⁶ meaning funding is very limited in most parts of the UK. In contrast, patients with dementia and their charities strongly support biomedical research and funding is far higher.³⁵ Therefore, although large numbers of individual patients support biomedical R&D into mental illness and addictions, currently, UK progress in this clinical field is likely to be limited.

Recommendations for training future psychiatrists

1. (a) It is important for future psychiatrists to understand the value of quantifying clinically relevant aspects of illness, starting by using clinical rating scales and gaining experience in their use. The objective is not to replace qualitative judgement, but to supplement it as is routine in other areas of medicine.

2. (b) It is crucial for psychiatrists to be able to differentiate statistical significance from clinically meaningful measures useful for individual patient care.

3. (c) It is helpful to be familiar with RDoC domain constructs and reflect on how psychiatric symptoms for individual patients relate to these: negative and positive valence systems, cognitive systems, social processes and arousal-regulatory systems.¹⁶

4. (d) Learning how to synthesise information across ‘units of analyses’ is important (for example symptoms, behaviour, brain circuits and molecular findings)¹⁶ and will help avoid the outdated ‘biology versus psychology’ philosophical dichotomy.¹⁷

5. (e) To gain entry to medical school, all consultant psychiatrists had the necessary qualifications to study undergraduate mathematics. The potential benefit of quantitative methods for patients should be introduced early in undergraduate medical training.

Conclusions

Mental illness and substance use disorders are the leading cause of disability worldwide, suicide is a leading cause of death in young adults, and severe and enduring mental illness is associated with a reduction in lifespan of about a decade. Despite this, clinical practice in psychiatry has not changed fundamentally in over half a century. There is good evidence that clinically useful individual patient predictions of diagnosis, clinical outcome and treatment response, can be made using neuroscience techniques. This predictive approach does not depend on understanding psychiatric nosology or illness mechanisms.

Given the remarkable disability and mortality associated with mental illness and addictions, it is crucial to invest more in UK biomedical research and clinical practice implementation. However, without organised stakeholder support influencing politicians and funding leaders, not much is likely to change. Currently, the area of UK psychiatry that seems most likely to develop and implement these new clinical techniques is old age psychiatry. We hope this will expand to include mental illness and addictions when the potential benefits become better known.