Hostname: page-component-586b7cd67f-rdxmf Total loading time: 0 Render date: 2024-11-23T21:40:10.092Z Has data issue: false hasContentIssue false

Development and psychometric evaluation of the Transdiagnostic Decision Tool: matched care for patients with a mental disorder in need of highly specialised care

Published online by Cambridge University Press:  25 August 2020

Frédérique C. W. van Krugten*
Affiliation:
Erasmus School of Health Policy & Management, Erasmus University Rotterdam, the Netherlands
Christina M. van der Feltz-Cornelis
Affiliation:
Department of Health Sciences, HYMS, University of York, UK
Manon A. Boeschoten
Affiliation:
National Psychotrauma Center, Foundation Centrum ‘45, the Netherlands
Saskia A. M. van Broeckhuysen-Kloth
Affiliation:
Department of Psychosomatic Medicine, Altrecht, the Netherlands
Jonna F. van Eck van der Sluijs
Affiliation:
Clinical Centre of Excellence for Body, Mind and Health, GGz Breburg; and Tranzo Department, Tilburg University, the Netherlands
Elisa van Ee
Affiliation:
Centre for Psychotrauma, Reinier van Arkel Group, the Netherlands
Saskia M. van Es
Affiliation:
Department of Psychopathology, PsyQ Amsterdam, Parnassia Groep, the Netherlands
Maartje Schoorl
Affiliation:
Institute of Psychology, Leiden University; and Department of Psychotrauma, PsyQ Haaglanden, Parnassia Groep, the Netherlands
Lineke M. Tak
Affiliation:
Interdisciplinary Center Psychopathology and Emotion Regulation, University Medical Center Groningen; and Dimence, Institute for Mental Health Care, the Netherlands
Werner B. F. Brouwer
Affiliation:
Erasmus School of Health Policy & Management, Erasmus University Rotterdam, the Netherlands
Leona Hakkaart-van Roijen
Affiliation:
Erasmus School of Health Policy & Management, Erasmus University Rotterdam, the Netherlands
*
Correspondence: Frédérique C. W. van Krugten. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Background

Early identification of patients with mental health problems in need of highly specialised care could enhance the timely provision of appropriate care and improve the clinical and cost-effectiveness of treatment strategies. Recent research on the development and psychometric evaluation of diagnosis-specific decision-support algorithms suggested that the treatment allocation of patients to highly specialised mental healthcare settings may be guided by a core set of transdiagnostic patient factors.

Aims

To develop and psychometrically evaluate a transdiagnostic decision tool to facilitate the uniform assessment of highly specialised mental healthcare need in heterogeneous patient groups.

Method

The Transdiagnostic Decision Tool was developed based on an analysis of transdiagnostic items of earlier developed diagnosis-specific decision tools. The Transdiagnostic Decision Tool was psychometrically evaluated in 505 patients with a somatic symptom disorder or post-traumatic stress disorder. Feasibility, interrater reliability, convergent validity and criterion validity were assessed. In order to evaluate convergent validity, the five-level EuroQol five-dimensional questionnaire (EQ-5D-5L) and the ICEpop CAPability measure for Adults (ICECAP-A) were administered.

Results

The six-item clinician-administered Transdiagnostic Decision Tool demonstrated excellent feasibility and acceptable interrater reliability. Spearman's rank correlations between the Transdiagnostic Decision Tool and ICECAP-A (−0.335), EQ-5D-5L index (−0.386) and EQ-5D-visual analogue scale (−0.348) supported convergent validity. The area under the curve was 0.81 and a cut-off value of ≥3 was found to represent the optimal cut-off value.

Conclusions

The Transdiagnostic Decision Tool demonstrated solid psychometric properties and showed promise as a measure for the early detection of patients in need of highly specialised mental healthcare.

Type
Papers
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - SA
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike licence (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the same Creative Commons licence is included and the original work is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use.
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press on behalf of the Royal College of Psychiatrists

Background

Although the efficacy of psychological interventions for the treatment of a wide range of mental health problems is well established,Reference Chambless and Ollendick1,Reference Hollon and Ponniah2 a significant number of patients require multiple treatment steps to achieve an adequate treatment response.Reference Rush, Trivedi, Wisniewski, Nierenberg, Stewart and Warden3 An inadequate response to initial treatment, in turn, is associated with higher relapse rates, chronicityReference Rush, Trivedi, Wisniewski, Nierenberg, Stewart and Warden3 and substantial societal costs.Reference Baker and Woods4 Against this background, and given the increasing prevalenceReference Whiteford, Degenhardt, Rehm, Baxter, Ferrari and Erskine5 and high associated costsReference Bloom, Cafiero, Jané-Llopis, Abrahams-Gessel, Bloom and Fathima6 of mental health problems, the importance of matching patients to the most appropriate level and type of initial care is increasingly recognised.

The matched-care approach, in which pre-treatment patient characteristics are used to match patients to the level of care that is likely to be most beneficial to them,Reference Van Straten, Tiemens, Hakkaart, Nolen and Donker8 has the potential to improve the clinical effectiveness and cost-effectiveness of treatment strategies.Reference Hill, Whitehurst, Lewis, Bryan, Dunn and Foster9 Matched care has been demonstrated to be an appropriate and effective approach in patients with mental health problems attending the primary care setting,Reference Huijbregts, de Jong, van Marwijk, Beekman, Adèr and Hakkaart-van Roijen10,Reference Goorden, Huijbregts, van Marwijk, Beekman, van der Feltz-Cornelis and Hakkaart-van Roijen11 the occupational healthcare settingReference Goorden, Vlasveld, Anema, van Mechelen, Beekman and Hoedeman12 and the out-patient general hospital setting,Reference Goorden, van der Feltz-Cornelis, van Steenbergen-Weijenburg, Horn, Beekman and Hakkaart-van Roijen13 but is likely to be most beneficial for the subgroup of patients in need of highly specialised mental healthcare. Often, these patients demonstrate low response and high relapse rates after initial treatment,Reference Houtveen, van Broeckhuysen-Kloth, Lintmeijer, Bühring and Geenen14,Reference Van Eck van der Sluijs, de Vroege, van Manen, Cees and van der Feltz-Cornelis15 and require additional treatment steps as the result. The provision of matched care in this subgroup is therefore warranted, but strongly relies on the ability to identify these patients and therefore the availability of pre-treatment assessment tools and decision guidelines to accurately match the initial treatment to the individual patient needs.Reference Haaga16,Reference Mathur and Sutton17

Recent initiatives to inform treatment decisions by pre-treatment patient characteristics include the development of diagnosis-specific decision tools for the following diagnostic groups: personality disorders, eating disorders, unipolar depression and anxiety disorders.Reference Dingemans, Goorden, Lötters, Bouwmans, Danner and van Elburg18Reference Van Krugten, Kaddouri, Goorden, Van Balkom, Berretty and Cath21 Decision tools are brief, clinician-administered instruments, especially designed to identify patients in need of highly specialised care during the diagnostic phase. Decision tools items include pre-treatment patient characteristics such as the absence or presence of psychiatric or somatic comorbidity, and the total score is an indicator of the need for highly specialised care. These diagnosis-specific decision tools have demonstrated solid psychometric properties,Reference Dingemans, Goorden, Lötters, Bouwmans, Danner and van Elburg18Reference Van Krugten, Kaddouri, Goorden, Van Balkom, Berretty and Cath21 and are used in psychiatric specialised centres to enhance the early identification of patients with a highly specialised mental healthcare need.

Aims

The development and psychometric evaluation of these diagnosis-specific decision tools suggested that the allocation of patients to highly specialised mental healthcare settings may be guided by a core set of transdiagnostic patient factors. Building on the theoretical foundations of, and insights from, the development of these diagnosis-specific decision tools, the aim of this study was to explore the possibility of developing a transdiagnostic decision tool for use in heterogeneous patient groups, in patients with a diagnosis for which no diagnosis-specific decision tool is available and in patients without a clear primary diagnosis. Such a tool could enhance the systematic and standardised early identification of patients with a highly specialised mental healthcare need, which, in turn may enhance treatment outcomes in patients with severe and complex mental health problems.

Method

Definition of terms and Transdiagnostic Decision Tool development

Highly specialised mental healthcare (i.e. tertiary mental healthcare) is the care provided by highly trained professionals to individuals with mental health problems that are complex and refractory to interventions provided in specialised (for example secondary) mental healthcare settings such as community mental health centres and general hospitals.Reference Wasylenki, Goering, Cochrane, Durbin, Rogers and Prendergast22,Reference Cochrane, Goering, Durbin, Butterill, Dumas and Wasylenki23 Given the level of necessary staff expertise, assessment and resources, highly specialised mental healthcare is often, but not per definition, provided in mental healthcare centres affiliated with academic medical settings.Reference Wasylenki, Goering, Cochrane, Durbin, Rogers and Prendergast22

In order to enhance the early identification and adequate management of patients with mental health problems in need of highly specialised care, the following four diagnosis-specific decision support algorithms were developed: the Decision Tool Personality Disorders,Reference Goorden, Willemsen, Bouwmans-Frijters, Busschbach, Noomx and van der Feltz-Cornelis19 the Decision Tool Eating Disorders,Reference Dingemans, Goorden, Lötters, Bouwmans, Danner and van Elburg18 the Decision Tool Unipolar DepressionReference Van Krugten, Goorden, van Balkom, van Oppen, Ruhé and van Schaik20 and the Decision Tool Anxiety Disorders.Reference Van Krugten, Kaddouri, Goorden, Van Balkom, Berretty and Cath21 Building on the theoretical foundations of, and insights from, the development and psychometric evaluation of these diagnosis-specific decision tools, the Transdiagnostic Decision Tool was developed for use in heterogeneous patient groups, in patients with a diagnosis for which no diagnosis-specific decision tool is available and in patients without a clear primary diagnosis.

The tool was initially intended for use in the diagnostic phase in specialised mental healthcare centres in order to optimise the clinical decision-making process in the referral of patients with mental health problems to highly specialised care. Its use does not have to be restricted to this setting, however. The Transdiagnostic Decision Tool was developed by the Transdiagnostic Decision Tool Consortium, comprising 16 leading mental health experts (psychiatrists and psychologists), two academics and two patient representatives (see Acknowledgements).

The development process of the Transdiagnostic Decision Tool consisted of three consecutive phases.

  1. (a) In the first phase, the overlapping patient criteria in the diagnosis-specific decision tools were established.

  2. (b) In the second phase, consortium members generated the draft Transdiagnostic Decision Tool through operationalisation of each of the criteria identified in the first phase.

  3. (c) In the third phase, a pilot study was carried out in 34 patients with a DSM-524 diagnosis of post-traumatic stress disorder (PTSD) (n = 10), somatic symptom disorder (SSD) (n = 10), unipolar depression (n = 5), anxiety disorder (n = 2), eating disorder (n = 3), personality disorder (n = 3) or psychotic disorder (n = 1) who were referred for treatment to either a specialised or highly specialised treatment centre in the Netherlands. Clinicians were asked to complete the draft version of the Transdiagnostic Decision Tool and answer questions regarding its feasibility. Feasibility questions included the total time required to complete the tool and the clarity of the item wording and the tool in total.

Evaluation of psychometric properties

Study design and population

In order to evaluate the psychometric properties of the Transdiagnostic Decision tool, a cross-sectional, observational multicentre study was carried out in eight specialised (general psychiatric) and highly specialised (i.e. tertiary) mental healthcare clinics in the Netherlands under routine care conditions. To facilitate the comparison of psychometric properties between diagnoses groups and evaluate the transdiagnostic robustness of the Transdiagnostic Decision tool, the study was carried out in two distinct diagnostic groups. The study population consisted of 505 adult (18 years and older) psychiatric out-patients with either a primary diagnosis of SSD or a primary diagnosis of PTSD according to DSM-5 criteria. Written informed consent was obtained from all patients. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All procedures involving human patients were approved by the Medical Ethical Committee of the Erasmus University Medical Centre Rotterdam, the Netherlands (MEC-2017-051).

Measures

In addition to the Transdiagnostic Decision Tool that was completed by the clinician, participants also completed a number of self-report instruments.

  1. (a) The five-level EuroQol five-dimensional questionnaire (EQ-5D-5L)Reference Herdman, Gudex, Lloyd, Janssen, Kind and Parkin25 is a generic, standardised, self-administered measure of health-related quality of life (HRQoL). The EQ-5D-5L comprises two parts: a descriptive system and a visual analogue scale (EQ-5D-VAS). The descriptive system consists of five items, covering five dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression), each with five response levels (no problems, some problems, moderate problems, severe problems and extreme problems/unable to). The answers on the descriptive system can be converted into a single preference-based summary index score (the EQ-5D-5L index) by applying societal preference weights to the self-classified health states. Based on the Dutch national value set, EQ-5D-5L index scores can range from −0.446 (representing the worst health state) to 1 (representing the best health state).Reference Versteegh, Vermeulen, Evers, de Wit, Prenger and Stolk26 The second part of the EQ-5D-5L, the EQ-5D-VAS, records the respondent's current self-rated health on a 20 cm vertical scale ranging from zero (‘the worst health you can imagine’) to 100 (‘the best health you can imagine’).

  2. (b) The ICEpop CAPability measure for Adults (ICECAP-A)Reference Al-Janabi, Flynn and Coast27 is a generic, standardised, self-administered measure of capability well-being for use in the adult population. The descriptive system consists of five items, covering five dimensions (stability, attachment, autonomy, achievement and enjoyment), each with four response levels. Responses to the descriptive system can be converted into a single summary index by applying societal preference weights to the self-classified capability states. The ICECAP-A index can range from 0 (representing the absence of capability) to 1 (representing full capability).Reference Flynn, Huynh, Peters, Al-Janabi, Clemens and Moody28

Procedures

From March 2017 through March 2018, patients were enrolled in the study at eight specialised (general psychiatric) and highly specialised (i.e. tertiary) mental healthcare clinics in the Netherlands. During the intake interview, clinicians rated each participating patient on the Transdiagnostic Decision Tool and entered the scoring on the Transdiagnostic Decision Tool, as well as demographic and clinical characteristics (gender, age, country of origin, primary diagnosis) and two questions regarding the feasibility of using the Transdiagnostic Decision Tool into web-based case report forms. Feasibility was operationalised as the total administration time of the Transdiagnostic Decision Tool, the clarity of the total set of items (scored as ‘Yes’ or ‘No’) and the percentage of missing values.

In order to evaluate the interrater reliability, a random subsample of 28% of patients was rated on the Transdiagnostic Decision Tool by a second clinician present at the intake interview. During the intake interview, patients completed a three-page questionnaire, including the EQ-5D-5L and the ICECAP-A to assess the convergent validity. Based on the patients’ preference, the EQ-5D-5L was provided in Dutch, English, French or Arabic and the ICECAP-A in Dutch or English.

Criterion validity was evaluated in a random subsample of 59% of patients by comparing the total Transdiagnostic Decision Tool score with the clinical judgement of senior clinicians. Two clinicians independently and masked to the individual scores on the Transdiagnostic Decision Tool rated whether the patient was in need of highly specialised care (scored with ‘Yes’ or ‘No’). An independent researcher verified the consistency between the judgements, and disagreements were resolved by discussion or through third-party consultation.

Statistical analysis

Demographic and clinical characteristics of the study sample and feasibility data were analysed using descriptive statistics. In line with previous Decision Tool research,Reference Van Krugten, Goorden, van Balkom, van Oppen, Ruhé and van Schaik20,Reference Van Krugten, Kaddouri, Goorden, Van Balkom, Berretty and Cath21 criteria for feasibility success were set at a mean administration time of ≤10 minutes, content clarity judged as ‘clear’ in ≥90% of all evaluations, and ≤5% of missing item responses.

To assess the interrater reliability, Krippendorff's alpha reliability coefficientsReference Hayes and Krippendorff29,Reference Krippendorff30 were calculated for each of the individual items, and the total Transdiagnostic Decision Tool score. The minimum acceptable reliability level was set at 0.667.Reference Krippendorff30 Following Shapiro–Wilk tests of normality, non-parametric Spearman's rank correlations between the total Transdiagnostic Decision Tool scores and EQ-5D-5L index, EQ-5D-VAS and ICECAP-A scores were computed to assess convergent validity. Correlations of 0.10–0.29, 0.30–0.49 and ≥0.50 were considered weak, moderate and strong, respectively.Reference Cohen31 Transdiagnostic Decision Tool scores were expected to have a moderate negative correlation with HRQoL (EQ-5D-5L) and well-being (ICECAP-A) scores. Receiver-operating characteristic (ROC) curves were generated to assess the criterion validity and to determine the optimal cut-off score. Areas under the ROC curves (AUCs) were generated to summarise the discriminative accuracy of the Transdiagnostic Decision Tool. In order to determine the optimal cut-off score, a Youden index (J = (sensitivityc + specificityc)−1)Reference Youden32 was calculated for each possible cut-off score. The cut-off score that corresponded to the highest Youden index was selected as the optimal cut-off score.

All statistical analyses were carried out both for the total sample and for each diagnostic group and conducted using IBM SPSS (Statistical Package for the Social Sciences) version 24.0 (SPSS Inc., IBM Corporation, Armonk, New York, USA). Significance levels were set at P < 0.05 (two-tailed).

Results

Scale development and preliminary evaluation of the criterion validity

Analysis of the overlapping criteria in the diagnosis-specific decision tools revealed the following five transdiagnostic criteria to detect patients with a highly specialised care need: high severity level of the primary diagnosis; treatment-interfering psychiatric comorbidity; treatment-interfering somatic comorbidity; treatment-interfering psychosocial dysfunctioning; and previous unsuccessful treatment of the current primary diagnosis in specialised mental healthcare (see Supplementary file A available at https://doi.org/10.1192/bjo.2020.74 for the primary items of the diagnosis-specific decision tools).

In a consensus meeting, consortium members added the criterion ‘Severe or longstanding childhood trauma’ to the initial list of five criteria given the prognostic importance of this criterion in patients with mental health problems.Reference Carr, Martins, Stingel, Lemgruber and Juruena33 In line with the diagnosis-specific decision tools, each of the transdiagnostic criteria was operationalised into a dichotomous (item present or not) scale item, resulting in a six-item draft version of the Transdiagnostic Decision Tool. Based on the data of the pilot study, no adjustments to the wording of items were needed. The items, response options, and scoring system of the Transdiagnostic Decision tool are presented in Table 1. An English translation of the complete Transdiagnostic Decision Tool is presented in Supplementary file B.

Table 1 Items, response options and scoring system of the Transdiagnostic Decision Tool

a. Item text is abbreviated. An English translation of the complete Transdiagnostic Decision Tool is presented in Supplementary file B.

Psychometric evaluation

In total, 505 patients were enrolled in the study. The demographic and clinical characteristics of the study population are presented in Table 2. The mean age of the patients was 41.2 years (s.d. = 12.4; range 18–79), 281 patients (55.6%) were women, and the majority of patients (71.1%) were of Dutch origin. At presentation, 234 (46.3%) patients had a primary diagnosis of SSD, and 271 (53.7%) had a primary diagnosis of PTSD. The mean total Transdiagnostic Decision Tool score was 2.5 (s.d. = 1.8; range 0–6). Mean self-reported HRQoL and well-being scores as measured by the EQ-5D-5L, EQ-5D-VAS and ICECAP-A were 0.40 (s.d. = 0.30; range −0.35 to –1.00), 49.7 (s.d. = 19.7, range 0.0–100.0) and 0.58 (s.d. = 0.20; range 0.00–0.97), respectively.

Table 2 Demographic and clinical characteristics of the study sample

EQ-5D-5L, Five-level EuroQol five-dimensional questionnaire; ICECAP-A, ICEpop CAPability measure for Adults; IRR, interrater reliability; VAS, visual analogue scale.

a. Part of total sample.

Feasibility

Mean administration time of the Transdiagnostic Decision Tool was 6.9 min (s.d. = 4.2; range = 1–30), and the total set items was evaluated as ‘clear’ in a vast majority of the evaluations (96.6%). The mean administration time was significantly lower (P < 0.001) in patients with SSD (5.6 min; s.d. = 3.1) than in patients with PTSD (8.0 min; s.d. = 4.6). The percentage of missing item responses ranged from 0.0% (item 6) to 1.5% (item 5) (mean 0.8%).

Interrater reliability

As shown in Table 3, Krippendorff's alpha values ranged from 0.724 (95% CI 0.581–0.841) for item 4 (‘psychosocial dysfunctioning’) to 0.848 (95% CI 0.731–0.938) for item 5 (‘childhood trauma’) in the total interrater reliability sample. In the SSD subsample, the Krippendorff's alpha values of item 3 (‘somatic comorbidity’) and item 4 (‘psychosocial dysfunctioning’) fell short of the recommended reliability level of 0.667. All other Krippendorff's alpha values of the individual items and the total Transdiagnostic Decision Tool score exceeded the recommended reliability level.

Table 3 Krippendorff's alpha values of the Transdiagnostic Decision Tool

a. Below the recommended level of 0.667.

Validity

As hypothesised, Transdiagnostic Decision Tool sum scores negatively correlated with HRQoL and well-being scores as measured by the EQ-5D-5L (r s(485) = −0.386; P < 0.001), EQ-5D-VAS (rs(485) = −0.348; P < 0.001), and ICECAP-A (rs(485) = −0.335; P < 0.001). As shown in Fig. 1, the AUC in the total criterion validity sample (n = 298) was 0.81 (95% CI 0.76–0.86; P < 0.001). The AUC in the SSD and PTSD subsamples were 0.84 (95% CI = 0.77–0.90; P < 0.001) and 0.78 (95% CI = 0.71–0.86; P < 0.001), respectively. The accuracy indices for various cut-off values of the Transdiagnostic Decision Tool are presented in Table 4. Based on the highest Youden index (Jmax) of 0.474 (sensitivity 72.4%; specificity 75.0%), the optimal cut-off value for the Transdiagnostic Decision Tool was ≥3 in the total criterion validity sample (n = 298). The optimal cut-off value of ≥3 was also found in the SSD (J max = 0.536) and PTSD (J max = 0.436) subsample.

(a) Total criterion validity sample (area under the ROC curve (AUC) = 0.81, 95% CI 0.76–0.86, P < 0.001) (n = 298). (b) Somatic symptom disorder (SSD) subsample (AUC = 0.84, 95% CI 0.77–0.90, P < 0.001) (n = 155). (c) Post-traumatic stress disorder (PTSD) subsample (AUC = 0.78, 95% CI 0.71–0.86, P < 0.001) (n = 143).

Fig. 1 Receiver-operating characteristic (ROC) curves for the Transdiagnostic Decision Tool.

Table 4 Accuracy indices of the Transdiagnostic Decision Tool in the total criterion validity sample (n = 298)

a. Youden index = (sensitivity + specificity) − 1.

Discussion

Main findings

This paper reports on the development and psychometric evaluation of a measure aimed at aiding clinicians in the early identification of patients with mental health problems in need of highly specialised care, the Transdiagnostic Decision Tool. Items of the Transdiagnostic Decision Tool were established through identification of overlapping criteria in previously developed diagnosis-specific decision tools. Overall, the results of the present study suggest that the Transdiagnostic Decision Tool is a psychometrically sound, and with the establishment of a cut-off score, promising tool for the early identification of patients with mental health problems in need of highly specialised care.

Interpretation of the findings

The short mean administration time (6.9 min) and low rate of missing values (mean 0.8%) supported the use of the Transdiagnostic Decision Tool in busy clinical settings. In the total sample, all Krippendorff's alpha values exceeded the recommended reliability level of 0.667,Reference Carr, Martins, Stingel, Lemgruber and Juruena33 demonstrating acceptable interrater reliability. However, the Krippendorff's alpha values of item 3 (‘somatic comorbidity’) and item 4 (‘psychosocial dysfunctioning’) fell short of the recommended reliability level in the SSD subsample. Analyses of the qualitative feedback regarding item 3 suggested that the lower Krippendorff's alpha might be because of the differential classification of medically unexplained physical symptoms across items. In other words, in some instances, clinicians may have classified medically unexplained physical symptoms under item 3 (‘somatic comorbidity’) instead of under items concerning the primary diagnosis, such as item 1 (‘severity’). The provided qualitative feedback provided no explanation for the lower Krippendorff's alpha of item 4. Future studies should evaluate whether further specification and clarification of scoring instructions for items 3 and 4 could improve the interrater reliability of these items in patients with SSD.

The Transdiagnostic Decision Tool demonstrated excellent validity, both in the total sample and within each diagnostic group. Specifically, the total Transdiagnostic Decision Tool score demonstrated meaningful patterns of correlations with total HRQoL and well-being scores, supporting convergent validity. In addition, the AUC in the total criterion validity sample was 0.81, and a cut-off value of 3 or greater on the Transdiagnostic Decision Tool was found to be the optimal cut-off value both in the total sample and within each diagnostic group, indicating that the optimal cut-off value is uniform across these diagnostic groups. Hence, the findings of the present study suggest that although disorder-specific symptoms are the predominant factors defining differential diagnoses, the allocation of patients to highly specialised healthcare may be meaningfully guided by a core set of transdiagnostic patient factors.

Strengths and limitations

This study has a number of strengths, including the large sample size, the population-based design and the examination of important psychometric properties related to the use of the Transdiagnostic Decision Tool in daily clinical practice. However, several limitations should also be noted.

First, in the absence of a reference test for the systematic and standardised early identification of patients with a highly specialised mental healthcare need, the clinical judgement of clinicians was the reference standard for the evaluation of the criterion validity. Although the use of the clinical judgement as the reference standard may have introduced subjective error, effort was made to reduce error by basing the final clinical judgement on dual, independently provided examinations by highly trained clinicians and extensive experience in the treatment of patients with severe and complex mental health problems.

Second, as this study presented a first psychometric evaluation of the Transdiagnostic Decision Tool, future studies are needed to extend these findings. More specifically, future studies are required to evaluate the psychometric properties of the Transdiagnostic Decision Tool in other diagnostic groups, in patients without a clear primary diagnosis and in other settings such as primary care. In addition, although the validity of the Transdiagnostic Decision Tool approximates the validity of the available diagnosis-specific decision tools, future studies are needed to determine whether the Transdiagnostic Decision Tool could be used as a substitute for these available diagnosis-specific decision tools for the diagnostic groups of personality disorders, eating disorders, unipolar depression and anxiety disorders. Given the time constraints and competing clinical demands of clinicians in daily practice,Reference Dragatsi, Norian and Minkoff34 a trade-off should be made between validity (i.e. precision) and feasibility (i.e. ease of use) of application of the Transdiagnostic Decision Tool in all diagnoses groups.

Third, in order to enhance the feasibility of the Transdiagnostic Decision Tool, the scoring system of the tool was constructed as a simple, additive, unweighted sum score. Although this enhances the ease of use in daily clinical practice, it potentially masks differences in the relative importance of individual scale items, which may reduce the precision of the measure. Further work is required to establish the effect of the use of a weighted score on the psychometric properties of the Transdiagnostic Decision Tool.

Fourth, notwithstanding its favourable validity in this first study, the Transdiagnostic Decision Tool is intended to augment rather than replace the clinical decision-making process in the referral of patients with mental health problems to highly specialised care. The Transdiagnostic Decision Tool has the potential to provide indications of highly specialised care need, which, together with an assessment of the patient's individual circumstances, preferences and level of motivation, could motivate a referral to treatment in a highly specialised mental healthcare setting.

Fifth, although the aim of the development of the Transdiagnostic Decision Tool was to facilitate the provision of matched care, the benefit of matched care in patients with a highly specialised mental healthcare has yet to be studied. Use of the Transdiagnostic Decision Tool in daily clinical practice could, however, enhance the assessment of the clinical- and cost-effectiveness of matched care in patients with a highly specialised mental healthcare need. Finally, although the Transdiagnostic Decision Tool was evaluated for its psychometric properties in specialised and highly specialised mental healthcare settings, the Transdiagnostic Decision Tool might also be of value in primary care services. Use of the Transdiagnostic Decision Tool in primary care services may further enhance the early identification and timely referral of patients with mental health problems in need of highly specialised care. Future studies are required to evaluate the benefit of use of the Transdiagnostic Decision Tool in primary care services.

Implications

Despite the limitations, the perceived ease of use, favourable psychometric properties and the transdiagnostic applicability indicate that the Transdiagnostic Decision Tool can be a promising tool for the early identification and adequate management of patients with mental health problems in need of highly specialised care. Its use in daily practice could enhance the systematic and standardised early identification of patients with a highly specialised mental healthcare need, and thereby has the potential to enhance treatment outcomes, reduce recidivism, reduce prolonged quality of life losses and improve the cost-effective use of scarce healthcare resources.

Supplementary material

Supplementary material is available online at https://doi.org/10.1192/bjo.2020.74.

Data availability

The corresponding author has full access to the study data and had final responsibility for the decision to submit the study for publication. The data-sets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Acknowledgements

Members of Transdiagnostic Decision Tool Consortium: A.J.L.M. van Balkom, V. Beffers, M.A. Boeschoten, S.A.M. van Broeckhuysen-Kloth, C.M. van der Feltz-Cornelis, M. Schoorl, J.F. van Eck van der Sluijs, E. van Ee, A.A. van Elburg, S.M. van Es, E.F. van Furth, L. Hakkaart-van Roijen, G.J. Hendriks, R. Jongedijk, F.W. van Krugten, D. Meijnckens, F.P.M.L. Peeters, K. de Ponti, L.M. Tak, A. Timmermans, E. Willemsen.

Author contributions

F.C.W.v.K., C.M.v.d.F.-C., W.B.F.B. and L.H.v.R. were involved in formulating the research question. F.C.W.v.K., C.M.v.d.F.-C., M.A.B., S.A.M.v.B.-K., J.F.v.E.v.d.S., E.v.E., S.M.v.E., M.S., L.M.T., W.B.F.B. and L.H.v.R. designed and carried out the study. F.C.W.v.K. analysed the data and drafted the manuscript. C.M.v.d.F.-C., M.A.B., S.A.M.v.B.-K., J.F.v.E.v.d.S., E.v.E., S.M.v.E., M.S., L.M.T., W.B.F.B. and L.H.v.R. reviewed the drafts of the manuscript. All authors read and approved the final manuscript.

Funding

This study was supported by a financial contribution from the Innovation Fund Health Insurers (in Dutch ‘Innovatiefonds Zorgverzekeraars’, https://www.innovatiefondszorgverzekeraars.nl) (grant number 3266).

Declaration of interest

F.C.W.v.K. and L.H.v.R. report a grant from Innovation Fund Health Insurers during the conduct of the study. The other authors have nothing to disclose.

ICMJE forms are in the supplementary material, available online at https://doi.org/10.1192/bjo.2020.74.

Footnotes

*

Details of the Membership of the Transdiagnostic Decision Tool Consortium are provided in the Acknowledgements.

References

Chambless, DL, Ollendick, TH. Empirically supported psychological interventions: controversies and evidence. Annu Rev Psychol 2001; 52: 685716.CrossRefGoogle ScholarPubMed
Hollon, SD, Ponniah, K. A review of empirically supported psychological therapies for mood disorders in adults. Depress Anxiety 2010; 27: 891932.CrossRefGoogle ScholarPubMed
Rush, AJ, Trivedi, MH, Wisniewski, SR, Nierenberg, AA, Stewart, JW, Warden, D, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR* D report. Am J Psychiatry 2006; 163: 1905–17.CrossRefGoogle ScholarPubMed
Baker, CB, Woods, SW. Cost of treatment failure for major depression: direct costs of continued treatment. Adm Policy Ment Health Ment Health Serv Res 2001; 28: 263–77.CrossRefGoogle ScholarPubMed
Whiteford, HA, Degenhardt, L, Rehm, J, Baxter, AJ, Ferrari, AJ, Erskine, HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013; 382: 1575–86.CrossRefGoogle ScholarPubMed
Bloom, DE, Cafiero, E, Jané-Llopis, E, Abrahams-Gessel, S, Bloom, LR, Fathima, S, et al. The Global Economic Burden of Noncommunicable Diseases. World Economic Forum, 2012.Google Scholar
Diefenbach, GJ, Tolin, DF. The cost of illness associated with stepped care for obsessive-compulsive disorder. J Obsessive Compuls Relat Disord 2013; 2: 144–8.CrossRefGoogle ScholarPubMed
Van Straten, A, Tiemens, B, Hakkaart, L, Nolen, W, Donker, M. Stepped care vs. matched care for mood and anxiety disorders: a randomized trial in routine practice. Acta Psychiatr Scand 2006; 113: 468–76.CrossRefGoogle ScholarPubMed
Hill, JC, Whitehurst, DG, Lewis, M, Bryan, S, Dunn, KM, Foster, NE, et al. Comparison of stratified primary care management for low back pain with current best practice (STarT Back): a randomised controlled trial. Lancet 2011; 378: 1560–71.CrossRefGoogle ScholarPubMed
Huijbregts, KM, de Jong, FJ, van Marwijk, HW, Beekman, AT, Adèr, HJ, Hakkaart-van Roijen, L, et al. A target-driven collaborative care model for Major Depressive Disorder is effective in primary care in the Netherlands. A randomized clinical trial from the depression initiative. J Affect Disord 2013; 146: 328–37.CrossRefGoogle ScholarPubMed
Goorden, M, Huijbregts, KM, van Marwijk, HW, Beekman, AT, van der Feltz-Cornelis, CM, Hakkaart-van Roijen, L. Cost-utility of collaborative care for major depressive disorder in primary care in the Netherlands. J Psychosom Res 2015; 79: 316–23.CrossRefGoogle ScholarPubMed
Goorden, M, Vlasveld, MC, Anema, JR, van Mechelen, W, Beekman, AT, Hoedeman, R, et al. Cost-utility analysis of a collaborative care intervention for major depressive disorder in an occupational healthcare setting. J Occup Rehabil 2014; 24: 555–62.CrossRefGoogle Scholar
Goorden, M, van der Feltz-Cornelis, CM, van Steenbergen-Weijenburg, KM, Horn, EK, Beekman, AT, Hakkaart-van Roijen, L. Cost-utility of collaborative care for the treatment of comorbid major depressive disorder in outpatients with chronic physical conditions. A randomized controlled trial in the general hospital setting (CC-DIM). Neuropsychiatr Dis Treat 2017; 13: 1881–93.CrossRefGoogle Scholar
Houtveen, JH, van Broeckhuysen-Kloth, S, Lintmeijer, LL, Bühring, ME, Geenen, R. Intensive multidisciplinary treatment of severe somatoform disorder: a prospective evaluation. J Nerv Ment Dis 2015; 203: 141–8.CrossRefGoogle ScholarPubMed
Van Eck van der Sluijs, JF, de Vroege, L, van Manen, AS, Cees, A, van der Feltz-Cornelis, CM. Complexity assessed by the INTERMED in patients with somatic symptom disorder visiting a specialized outpatient mental health care setting: a cross-sectional study. Psychosomatics 2017; 58: 427–36.CrossRefGoogle ScholarPubMed
Haaga, DA. Introduction to the special section on stepped care models in psychotherapy. J Consult Clin Psychol 2000; 68: 547–8.CrossRefGoogle ScholarPubMed
Mathur, S, Sutton, J. Personalized medicine could transform healthcare. Biomed Rep 2017; 7: 35.CrossRefGoogle ScholarPubMed
Dingemans, AE, Goorden, M, Lötters, FJ, Bouwmans, C, Danner, UN, van Elburg, AA, et al. Development and validation of a decision tool for early identification of adult patients with severe and complex eating disorder psychopathology in need of highly specialized care. Eur Eat Disord Rev 2017; 25: 366–72.CrossRefGoogle ScholarPubMed
Goorden, M, Willemsen, E, Bouwmans-Frijters, C, Busschbach, J, Noomx, M, van der Feltz-Cornelis, C, et al. Developing a decision tool to identify patients with personality disorders in need of highly specialized care. BMC Psychiatry 2017; 17: 317.CrossRefGoogle ScholarPubMed
Van Krugten, FC, Goorden, M, van Balkom, AJ, van Oppen, P, Ruhé, HG, van Schaik, DJ, et al. The decision tool unipolar depression (DTUD): a new measure to facilitate the early identification of patients with major depressive disorder in need of highly specialized care. BMC Psychiatry 2019; 19: 179.CrossRefGoogle ScholarPubMed
Van Krugten, FC, Kaddouri, M, Goorden, M, Van Balkom, AJ, Berretty, EW, Cath, DC, et al. Psychometric evaluation of the decision tool anxiety disorders: Facilitating the early identification of patients with an anxiety disorder in need of highly specialized care. Eur Psychiatry 2018; 48: S146.Google Scholar
Wasylenki, D, Goering, P, Cochrane, J, Durbin, J, Rogers, J, Prendergast, P. Tertiary mental health services: I. Key concepts. Can J Psychiatry 2000; 45: 179–84.CrossRefGoogle ScholarPubMed
Cochrane, J, Goering, P, Durbin, J, Butterill, D, Dumas, J, Wasylenki, D. Tertiary mental health services: II. Subpopulations and best practices for service delivery. Can J Psychiatry 2000; 45: 185–90.CrossRefGoogle ScholarPubMed
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (5th ed.). APA, 2013.Google Scholar
Herdman, M, Gudex, C, Lloyd, A, Janssen, M, Kind, P, Parkin, D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011; 20: 1727–36.CrossRefGoogle Scholar
Versteegh, MM, Vermeulen, KM, Evers, SM, de Wit, GA, Prenger, R, Stolk, EA. Dutch tariff for the five-level version of EQ-5D. Value Health 2016; 19: 343–52.CrossRefGoogle Scholar
Al-Janabi, H, Flynn, TN, Coast, J. Development of a self-report measure of capability wellbeing for adults: the ICECAP-A. Qual Life Res 2012; 21: 167–76.CrossRefGoogle ScholarPubMed
Flynn, TN, Huynh, E, Peters, TJ, Al-Janabi, H, Clemens, S, Moody, A, et al. Scoring the ICECAP-A capability instrument. Estimation of a UK general population tariff. Health Econ 2015; 24: 258–69.CrossRefGoogle ScholarPubMed
Hayes, AF, Krippendorff, K. Answering the call for a standard reliability measure for coding data. Commun Methods Meas 2007; 1: 7789.CrossRefGoogle Scholar
Krippendorff, K. Content Analysis: An Introduction to its Methodology. Sage, 2012.Google Scholar
Cohen, J. Statistical Power Analysis for the Behavioral Sciences (2nd edn). Erlbaum, 1988.Google Scholar
Youden, WJ. Index for rating diagnostic tests. Cancer 1950; 3: 32–5.3.0.CO;2-3>CrossRefGoogle ScholarPubMed
Carr, CP, Martins, CMS, Stingel, AM, Lemgruber, VB, Juruena, MF. The role of early life stress in adult psychiatric disorders: a systematic review according to childhood trauma subtypes. J Nerv Ment Dis 2013; 201: 1007–20.CrossRefGoogle ScholarPubMed
Dragatsi, D, Norian, I, Minkoff, K. American Association of community psychiatrists position statement: putting patients first by improving treatment planning and reducing administrative and clinical burden of treatment plan documentation. Community Ment Health J 2019; 55: 48.CrossRefGoogle Scholar
Figure 0

Table 1 Items, response options and scoring system of the Transdiagnostic Decision Tool

Figure 1

Table 2 Demographic and clinical characteristics of the study sample

Figure 2

Table 3 Krippendorff's alpha values of the Transdiagnostic Decision Tool

Figure 3

Fig. 1 Receiver-operating characteristic (ROC) curves for the Transdiagnostic Decision Tool.

(a) Total criterion validity sample (area under the ROC curve (AUC) = 0.81, 95% CI 0.76–0.86, P n = 298). (b) Somatic symptom disorder (SSD) subsample (AUC = 0.84, 95% CI 0.77–0.90, P n = 155). (c) Post-traumatic stress disorder (PTSD) subsample (AUC = 0.78, 95% CI 0.71–0.86, P n = 143).
Figure 4

Table 4 Accuracy indices of the Transdiagnostic Decision Tool in the total criterion validity sample (n = 298)

Supplementary material: File

van Krugten et al. supplementary material

van Krugten et al. supplementary material 1

Download van Krugten et al. supplementary material(File)
File 118.2 KB
Supplementary material: File

van Krugten et al. supplementary material

van Krugten et al. supplementary material 2

Download van Krugten et al. supplementary material(File)
File 12.8 MB
Submit a response

eLetters

No eLetters have been published for this article.