Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-17T23:20:02.015Z Has data issue: false hasContentIssue false

Dealing with heterogeneity of cognitive dysfunction in acute depression: a clustering approach

Published online by Cambridge University Press:  01 June 2020

Muriel Vicent-Gil
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain Department of Psychiatry, Hospital de la Santa Creu i Sant Pau, Biomedical Research Institute Sant Pau (IIB-Sant Pau), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Sant Antoni Mª Claret 167, 08025 Barcelona, Catalonia, Spain
Maria J. Portella
Affiliation:
Department of Psychiatry, Hospital de la Santa Creu i Sant Pau, Biomedical Research Institute Sant Pau (IIB-Sant Pau), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Sant Antoni Mª Claret 167, 08025 Barcelona, Catalonia, Spain
Maria Serra-Blasco*
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain
Guillem Navarra-Ventura
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain
Sara Crivillés
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain
Eva Aguilar
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain
Diego Palao
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain
Narcís Cardoner*
Affiliation:
Mental Health Department, Hospital Universitari Parc Taulí, Institut d'Investigació i Innovació Parc Taulí (I3PT), Universitat Autònoma de Barcelona (UAB), Biomedical Research Networking Center Consortium on Mental Health (CIBERSAM), Parc Taulí 1, 08208 Sabadell, Catalonia, Spain
*
Authors for correspondence: Maria Serra-Blasco, E-mail: [email protected]; Narcís Cardoner, E-mail: [email protected]
Authors for correspondence: Maria Serra-Blasco, E-mail: [email protected]; Narcís Cardoner, E-mail: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Background

Heterogeneity in cognitive functioning among major depressive disorder (MDD) patients could have been the reason for the small-to-moderate differences reported so far when it is compared to other psychiatric conditions or to healthy controls. Additionally, most of these studies did not take into account clinical and sociodemographic characteristics that could have played a relevant role in cognitive variability. This study aims to identify empirical clusters based on cognitive, clinical and sociodemographic variables in a sample of acute MDD patients.

Methods

In a sample of 174 patients with an acute depressive episode, a two-step clustering analysis was applied considering potentially relevant cognitive, clinical and sociodemographic variables as indicators for grouping.

Results

Treatment resistance was the most important factor for clustering, closely followed by cognitive performance. Three empirical subgroups were obtained: cluster 1 was characterized by a sample of non-resistant patients with preserved cognitive functioning (n = 68, 39%); cluster 2 was formed by treatment-resistant patients with selective cognitive deficits (n = 66, 38%) and cluster 3 consisted of resistant (n = 23, 58%) and non-resistant (n = 17, 42%) acute patients with significant deficits in all neurocognitive domains (n = 40, 23%).

Conclusions

The findings provide evidence upon the existence of cognitive heterogeneity across patients in an acute depressive episode. Therefore, assessing cognition becomes an evident necessity for all patients diagnosed with MDD, and although treatment resistant is associated with greater cognitive dysfunction, non-resistant patients can also show significant cognitive deficits. By targeting not only mood but also cognition, patients are more likely to achieve full recovery and prevent new relapses.

Type
Original Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © The Author(s) 2020. Published by Cambridge University Press

Introduction

Research over the last decade has been mainly focused on cognitive performance after a depressive episode (Semkovska et al., Reference Semkovska, Quinlivan, O'Grady, Johnson, Collins, O'Connor and Gload2019) so as to explore difficulties in cognition as an independent facet of clinical manifestation of major depressive disorder (MDD). Some studies report improvements in cognitive performance upon remission of depression (Biringer et al., Reference Biringer, Mykletun, Sundet, Kroken, Stordal and Lund2007) whereas others suggest that cognitive impairment persists during clinical remission (Bora, Harrison, Yücel, & Pantelis, Reference Bora, Harrison, Yücel and Pantelis2013; Hasselbalch, Knorr, & Kessing, Reference Hasselbalch, Knorr and Kessing2011). The disparity in these results might be explained by the fact that a number of patients do not show any cognitive impairment over the course of the disorder, and others display significant cognitive difficulties even in a non-symptomatic phase. In this regard, a recent study has even found discrete neurocognitive subgroups suggesting the presence of substantial heterogeneity in neurocognitive performance in MDD patients with current affective stability (Pu, Noda, Setoyama, & Nakagome, Reference Pu, Noda, Setoyama and Nakagome2018). It is noteworthy to mention that although patients were in clinical remission, some of them were classified as globally impaired showing moderate to severe cognitive impairment. Therefore, it would be reasonable to think that unresolved cognitive deficits were already present during the acute phase of MDD, from which it could be inferred that different cognitive profiles existed among patients. Despite such a heterogeneity being frequently proposed as an explanation for disparate findings in cognitive performance during a depressive episode (Hammar & Ardal, Reference Hammar and Ardal2009; Lee, Hermens, Porter, & Redoblado-Hodge, Reference Lee, Hermens, Porter and Redoblado-Hodge2012), it has scarcely been investigated. A recent study by our group has already found two distinguishable cognitive profiles in first-episode patients (Vicent-Gil et al., Reference Vicent-Gil, Keymer-Gausset, Serra-Blasco, Carceller-Sindreu, de Diego-Adeliño, Trujols and Portella2018), whereas other reports used mixed samples of mood disorders (Cotrena, Branco, Kochhann, Shansis, & Fonseca, Reference Cotrena, Branco, Kochhann, Shansis and Fonseca2016; Iverson, Brooks, Langenecker, & Young, Reference Iverson, Brooks, Langenecker and Young2011).

Previous studies have described that cognitive dysfunction could be associated with more severe manifestations of the disease (Serra-Blasco & Lam, Reference Serra-Blasco, Lam, Harmer and Baune2019), such as early/late illness onset, symptom severity (McDermott & Ebmeier, Reference McDermott and Ebmeier2009), number of previous depressive episodes (Semkovska et al., Reference Semkovska, Quinlivan, O'Grady, Johnson, Collins, O'Connor and Gload2019) and a higher level of resistance to antidepressant strategies (Murrough et al., Reference Murrough, Burdick, Levitch, Perez, Brallier, Chang and Iosifescu2015; Pimontel et al., Reference Pimontel, Rindskopf, Rutherford, Brown, Roose and Sneed2016; Serra-Blasco et al., Reference Serra-Blasco, de Vita, Rodríguez, de Diego-Adeliño, Puigdemont, Martín-Blanco and Portella2015). All these studies did not take into account the possible heterogenic cognitive profiles among the included patients, which could have explained the small-to-moderate effect sizes reported so far. Apart from that, other sociodemographic variables, such as years of schooling (Venezia et al., Reference Venezia, Gorlyn, Burke, Oquendo, Mann and Keilp2018) and age (Dotson, Resnick, & Zonderman, Reference Dotson, Resnick and Zonderman2008), could also be associated with lower cognitive performance. In fact, two recent studies on clustering analysis of cognitive functioning have shown that the most affected cognitive profile was characterized by poorer general functioning (i.e. poorer intellectual ability; Pu et al., Reference Pu, Noda, Setoyama and Nakagome2018; Vicent-Gil et al., Reference Vicent-Gil, Keymer-Gausset, Serra-Blasco, Carceller-Sindreu, de Diego-Adeliño, Trujols and Portella2018).

Recent studies have reported an association of cognitive dysfunction with worse psychosocial functioning, both at work performance and in social relationships (Clark, DiBenedetti, & Perez, Reference Clark, DiBenedetti and Perez2016; Cotrena et al., Reference Cotrena, Branco, Kochhann, Shansis and Fonseca2016; Evans, Iverson, Yatham, & Lam, Reference Evans, Iverson, Yatham and Lam2014). Considering the above, it might be useful to identify those patients with cognitive impairment during an acute episode of depression, to try to cope with cognitive deficits while at the same time improving the patients' psychosocial functioning without having to wait to treat those deficits after clinical remission. So far, only one of the studies mentioned in a sample of first episode of depression has analyzed neuropsychological heterogeneity in the acute stage of the illness revealing two distinguishable cognitive profiles (preserved and impaired clusters) (Vicent-Gil et al., Reference Vicent-Gil, Keymer-Gausset, Serra-Blasco, Carceller-Sindreu, de Diego-Adeliño, Trujols and Portella2018). Some patients were classified as cognitively impaired showing significant deficits in attention/working memory and verbal memory and subtle impairment in executive function, whereas the rest did not show any cognitive deficit. Unfortunately, cognition is not routinely evaluated in all MDD patients [as reported by McAllister-Williams et al. (Reference McAllister-Williams, Bones, Goodwin, Harrison, Katona, Rasmussen and Young2017) in the UK], and the presence of cognitive dysfunction in a non-negligible percentage of them still represents an unresolved problem, which derives in increased social and health costs.

Even though there seems to be some evidence about the heterogeneity in cognitive functioning among MDD patients (see Douglas et al., Reference Douglas, Gallagher, Robinson, Carter, McIntosh, Frampton and Porter2018), no studies have attempted to identify subgroups of acute patients considering the factors mentioned above. The aim of the current study is to identify clusters of MDD patients with an acute episode using cognitive, clinical and sociodemographic measures as indicators of grouping. We hypothesize that different cluster groups will emerge based on their cognitive, clinical and sociodemographic characteristics. In addition, we also expect that most cognitively affected patients will show worse psychosocial adaptation.

Methods

Participants

A sample of 174 participants aged 18–65 years old was selected from the outpatient unit at the Psychiatry Department of the Hospital Universitari Parc Taulí, from part of a broader project studying the cognitive functioning in major depression (Serra-Blasco et al., Reference Serra-Blasco, Torres, Vicent-Gil, Goldberg, Navarra-Ventura, Aguilar and Cardoner2019). The patients fulfilled the inclusion criteria of a current episode of MDD (DSM-IV-TR criteria). Diagnosis was double checked by two experienced psychiatrists and validated through clinical reports. Exclusion criteria for all participants were the following: (i) presence of any neurological disease, (ii) medical illness with a known impact on cognitive functioning, (iii) intelligence quotient (IQ) <85, (iv) presence of a comorbid axis I diagnosis with the exception of anxiety disorders and dysthymia, (v) past or current substance abuse or (vi) any axis II diagnosis according to the DSM-IV-TR. Participants were on medication at the time of evaluation. Patients were invited to participate in this cross-sectional study, which included a clinical and neuropsychological assessment conducted by experienced research neuropsychologists. The study was set following the principles of the Declaration of Helsinki and was approved by the Research Ethics Board of the Institut d'Investigació i Innovació Parc Taulí (I3PT) at Hospital Universitari Parc Taulí. All participants gave their written informed consent after a full and comprehensive explanation of the study.

Clinical and demographic assessment

Clinical and demographic variables were obtained during a semi-structured interview, which covered age, sex, years of schooling, age at illness onset, number of episodes and duration of illness. Medication use at the time of evaluation was categorized as: no medication, monotherapy with antidepressants, antidepressants plus benzodiazepines and combination of antidepressants with one or more psychotropic drugs (e.g. antipsychotics, lithium and anticonvulsants). Depressive symptomatology was assessed using the 17-item Hamilton Depression Rating Scale (HDRS-17; Bobes et al. Reference Bobes, Bulbena, Luque, Dal-Ré, Ballesteros and Ibarra2003; Hamilton, Reference Hamilton1960). The Maudsley Staging Method (MSM; Fekadu, Wooderson, Markopoulou, & Cleare, Reference Fekadu, Wooderson, Markopoulou and Cleare2009) was administered to assess the level of treatment resistance. MSM includes information about duration and severity of depression, antidepressant treatments, augmentation strategies and electroconvulsive therapy providing two categories: non-resistant (scores 3–6) and resistant (scores 7–15).

Neuropsychological assessment

The neuropsychological battery included the following tests: Digit Span of the Wechsler Adult Intelligence Scale version IV (WAIS-IV; Wechsler, Reference Wechsler2008); Rey Auditory Verbal Learning Test (RAVLT; Rey, Reference Rey1941); Digit Symbol Substitution Test (DSST; WAIS-IV); Trail Making Test Part A (TMT-A), and Trail Making Test Part B (TMT-B; Tombaugh, Reference Tombaugh2004); Wisconsin Card Sorting Test (WCST; Heaton, Reference Heaton1981); similarities subtest (WAIS-IV); semantic verbal fluency test (category fluency; Benton & Hamsher, Reference Benton and Hamsher1976; Peña-Casanova et al., Reference Peña-Casanova, Quinones-Ubeda, Gramunt-Fombuena, Quintana-Aparicio, Aguilar, Badenes and Blesa2009) and phonemic verbal fluency test (PMR, adapted for Spanish speaking population; Casals-Coll et al., Reference Casals-Coll, Sánchez-Benavides, Quintana, Manero, Rognoni, Calvo and Peña-Casanova2013; Peña-Casanova et al., Reference Peña-Casanova, Quinones-Ubeda, Gramunt-Fombuena, Quintana-Aparicio, Aguilar, Badenes and Blesa2009). Premorbid intelligence (estimated IQ) was assessed with Vocabulary Subtest of the WAIS-IV and it was used to compare cognitive profiles after the clustering analysis.

Functional assessment

Functioning Assessment Short Test (FAST; Rosa et al., Reference Rosa, Sánchez-Moreno, Martínez-Aran, Salamero, Torrent, Reinares and Vieta2007) was used to evaluate autonomy, occupational functioning, cognitive functioning, financial issues, interpersonal relationships and leisure time. The scores range from 0 to 72 with higher values indicating more disability. A score of ⩾12 represents a mild to severe functional impairment (Bonnín et al., Reference Bonnín, Martínez-Arán, Reinares, Valentí, Solé, Jiménez and Rosa2018).

Data analyses

Data were analyzed using the Statistical Package for Social Sciences (SPSS), version 21. Neuropsychological raw scores were converted to z-scores using normative data. To reduce the number of neuropsychological variables (Miskowiak et al., Reference Miskowiak, Burdick, Martinez-Aran, Bonnin, Bowie, Carvalho and Vieta2017; Mur, Portella, Martínez-Arán, Pifarré, & Vieta, Reference Mur, Portella, Martínez-Arán, Pifarré and Vieta2007), cognitive domains were defined using a principal component analysis (PCA). The number of retained components was decided upon the resulting scree plot, admitting eigenvalues above or close to 1 (based on theoretically-driven decision). These components (cognitive domains) were then rotated and used for a cluster analysis.

To identify homogeneous subgroups of patients, a two-step clustering analysis was carried out based on cognitive domains, stage of treatment-resistance (MSM), depressive symptomatology (HDRS-17), number of depressive episodes, age and years of schooling. This two-step analysis is designed to use categorical and continuous variables in large samples. It represents an extra value to previous studies as other relevant variables such as treatment response can be included. During the first step, subjects are preclustered into small subgroups using a sequential-clustering approach. In the second step, subclusters from the first are entered as inputs and grouped into the best number of clusters according to a hierarchical-clustering method (Norusis, Reference Norusis2011). The determination of the optimal numbers of clusters is based on the Akaike Information Criterion (AIC) and the log-likelihood distance, taking as the best solution the large ratio of AIC changes and the large ratio of distance measures. The final model is based on different criteria (Nylund, Asparouhov, & Muthén, Reference Nylund, Asparouhov and Muthén2007): (i) the highest cohesion and separation of the resulted clusters measured with the Silhouette's index, (ii) the best clinical coherence and (iii) an adequate number of patients in each cluster to facilitate statistical analyses of comparison. Also, as a final result, the analyses provide a ranking of the importance of each predictor entered in the model. The greater the importance measure, the more relevant the variable is considered in the formation of the cluster.

Demographic, clinical and functional variables were analyzed among resulting clusters in each group of patients by means of one-way analysis of variances or chi-square when appropriate, and effect-sizes were reported, as well.

Results

The PCA of neuropsychological data extracted four orthogonal dimensions that corresponded to four cognitive domains and explained a 74% of cumulative variance. The four cognitive domains were: (i) Attention/Working Memory domain, which included the forward and backward Digit Span; (ii) Verbal Memory domain, composed of RAVLT first trial, immediate recall and delayed recall; (iii) Executive Function domain, with TMT Part A, TMT Part B, DSST and number of categories from WCST and (iv) Verbal Ability domain which included PMR, semantic fluency and similarities. See online Supplementary Table 1 with a brief summary of outcome measures for each test.

The two-step clustering analysis resulted in a three-cluster solution, which was selected as being the most optimal one based on a fair Silhouette's index (equal to 0.3, see online Supplementary Fig. 1), on the clinical interpretability from previous studies, and on the number of patients in each cluster. As can be observed from Fig. 1, the stage of treatment resistance obtained the highest relevance for clustering with a predictor importance of 1.0, followed by verbal ability, executive function, attention/working memory and verbal memory with values ranging between 0.8 and 0.5. Depressive symptomatology, years of schooling, age and number of depressive episodes obtained the lowest relevance for clustering. Table 1 presents the centroids for the predictors, mean values of quantitative variables and percentage distribution for the categorical variable treatment resistance.

Fig. 1. Clustering summary: relative importance of each indicator.

Table 1. Clustering summary: centroids

MSM, Maudsley Staging Method; HDRS, Hamilton Depression Rating Scale.

a MSM non-resistant (scores 3–6), MSM-resistant (scores 7–15).

Figure 2 illustrates a radar chart that shows the profile of the clusters within the different predictors of the model (quantitative variables were transformed into z-scores for a better comprehension of the figure). The first cluster (C1) was classified as cognitively preserved (n = 68, 39%) and it was characterized by a sample of non-resistant patients. The second cluster (C2) included 66 patients (38%) classified as selectively impaired and all patients were treatment-resistant. And cluster 3 (C3) included globally impaired patients (n = 40, 23%) with significant deficits in all neurocognitive domains, being 23 patients classified as resistant (58%) and 17 as non-resistant (42%).

Fig. 2. Radar chart for the distribution of the indicators of the model. HDRS-17: Hamilton Depression Rating Scale.

Tables 2 and 3 display comparisons among the three clusters for demographic, clinical and functional variables and for cognitive tests. The three clusters differed in terms of cognitive performance (see online Supplementary Fig. 2), in which C1 patients (cognitively preserved) scored within normal range in almost all the tests (with the exception of number of categories). C2 patients (selectively impaired) obtained scores below 1s.d. in specific tests evaluating memory and executive functioning. And C3 patients (globally impaired) showed significant alterations in almost all the neuropsychological tests (with z-scores ranging between −1 and −2). Post-hoc comparisons of C3 (globally impaired patients) to C1 and C2 showed large effect sizes in all cognitive variables (Cohen's d > 0.8, with the exception of first trial in RAVLT in the contrast C2 v. C3). With regard to functional assessment, significant differences were also observed, in which globally impaired patients showed the worst outcomes.

Table 2. Mean scores (standard deviation) for demographic and clinical variables across clusters

IQ, intelligence quotient; HDRS, Hamilton Depression Rating Scale; MSM, Maudsley Staging Method; FAST, Functional Assessment Short Test; AD, antidepressant; BZD, benzodiazepines; others, antipsychotics; lithium or anticonvulsants.

a Cramer's V for categorical variables.

b Analysis of variance or χ2 tests statistically significant after applying Bonferroni correction for multiple comparisons (p < 0.004).

Table 3. Mean scores (standard deviation) for cognitive variables across clusters

WAIS: Wechsler Adult Intelligence Scale, RAVLT: Rey Auditory Verbal Learning Test, TMT: Trail Making Test, DSST: Digit Symbol Substitution Test, WCST: Wisconsin Card Sorting Test.

a Analysis of variance or χ2 tests statistically significant after applying Bonferroni correction for multiple comparisons (p < 0.003).

Discussion

This work explores the existence of empirical clusters for MDD taking into account patients' cognitive performance, stage of treatment resistance, depressive symptomatology, number of depressive episodes, age and years of schooling. Treatment resistance was the variable with the highest importance of clustering, closely followed by cognitive performance (verbal ability, executive function, attention/working memory and verbal memory). The rest of variables obtained the lowest importance on identifying distinct subgroups of patients. Three empirical clusters were determined: cluster 1, characterized by non-resistant patients with preserved cognitive performance; cluster 2, composed of resistant patients with selective impairment and cluster 3, grouped by resistant and non-resistant patients with a global cognitive impairment. As hypothesized, the latest was related to worse clinical and psychosocial outcomes. These findings may indicate the existence of different subgroups of patients, determined by clinical variables – as well-established in the literature – and by cognitive symptoms, which have not received enough attention for decades and may be underlying poor outcomes.

To our knowledge, this study is the first to show different cognitive profiles during an acute phase of the illness, considering not only cognitive performance but also clinical and sociodemographic factors, which are likely to have contributed to divergent results in the last few decades. Hammar and Ardal (Reference Hammar and Ardal2009) already suggested that no single cognitive functioning profile could characterize all depressed patients, and that not all patients were to be impaired in the same degree during the acute phase (Hammar & Ardal, Reference Hammar and Ardal2009). The existence of such heterogeneity among mood disorders' cognitive functioning is supported by previous cluster analyses, especially in bipolar disorder (Burdick et al., Reference Burdick, Russo, Frangou, Mahon, Braga, Shanahan and Malhotra2014; Cotrena, Branco, Ponsoni, Shansis, & Fonseca, Reference Cotrena, Branco, Ponsoni, Shansis and Fonseca2017; Lima et al., Reference Lima, Rabelo-da-Pontea, Bückera, Czepielewskia, Hasse-Sousaa, Telescaa and Rosa2019; Solé et al., Reference Solé, Jiménez, Torrent, del Mar Bonnin, Torres, Reinares and Martínez-Arán2016). These studies found three-cluster solutions based on cognitive performance, corresponding to intact or preserved, selectively impaired and globally impaired patients. Only two studies have been carried out clustering analysis with exclusively MDD patients, and their findings point toward cognitive heterogeneity along the different stages of the disorder. One included patients in partial remission and reported three clusters (Pu et al., Reference Pu, Noda, Setoyama and Nakagome2018) in accordance with the above-mentioned studies, and the other was centered into first-episode patients and showed two clusters, one preserved and one impaired (Vicent-Gil et al., Reference Vicent-Gil, Keymer-Gausset, Serra-Blasco, Carceller-Sindreu, de Diego-Adeliño, Trujols and Portella2018). These previous studies, except for the one with first-episode patients, endorse our current findings of a subgroup of intact or mostly preserved patients, of a globally impaired subgroup with a general cognitive affectation, and of a subgroup with specific domains impaired. Although some of the studies (Burdick et al., Reference Burdick, Russo, Frangou, Mahon, Braga, Shanahan and Malhotra2014; Pu et al., Reference Pu, Noda, Setoyama and Nakagome2018) claim to have found ‘discrete neurocognitive subgroups’, this may be straightforward for preserved and globally impaired patients, because these individuals can be easily detected even in clinical settings. By contrast, the selectively impaired subjects may not constitute a clearly discrete neurocognitive subgroup given that other characteristics may interact with cognition making difficult to detect specific cognitive alterations. At this point, our current study highlights the importance of adding clinical information to the clustering, as it may be crucial for a more comprehensive classification of patients, by capturing other factors that may blur those patients in C2.

Treatment resistance was the most important variable in the clustering process, which embraces a lack of response (and/or remission) and greater severity of clinical symptoms. The totality of participants in the cognitively preserved cluster was non-resistant and similarly, the 100% of the participants in the selectively impaired cluster were resistant. In the group of globally impaired patients, however, up to 42% of patients were non-resistant. A possible explanation is that alterations of memory and executive function are more specifically related to treatment resistance (Pimontel et al., Reference Pimontel, Rindskopf, Rutherford, Brown, Roose and Sneed2016; Rao et al., Reference Rao, Xu, Lu, Liang, Lin and Tang2019), as observed in C2, in consistence with previous evidence of hippocampus and prefrontal cortex alterations in treatment-resistant depression (Ge et al., Reference Ge, Torres, Brown, Gregory, McLellan, Downar and Vila-Rodriguez2019). Thus, a global alteration of cognitive function may be reflecting a different phenotype, in which the main characteristic would be cognitive impairment and not exclusively linked to treatment resistance – this may not be unreasonable as the majority of antidepressant treatments do not target cognitive symptoms. These results might indicate that although treatment resistance would be associated with greater cognitive impairment, the existence of various cognitive profiles has to be taken into account beyond the usual clinical variables, as non-resistant patients can still display cognitive symptoms. In fact, even though all patients of the study were acutely depressed, 39% showed no cognitive impairment. These findings also help to explain the small-to-moderate effects – depending on the domain – of cognitive dysfunction in previous studies that compare patients with MDD with healthy controls, considering the apriorism of cognitive homogeneity among MDD patients may have led to such disperse results. The current study, which is data-driven, demonstrates the relevant role of cognitive functioning in major depression, pointing at the existence of potential cognitive dysfunction in a high percentage of patients who will require more tailored treatments.

Socio-demographic variables have a low relative importance in the cluster formation (Fig. 1). However, years of schooling and IQ significantly differ among clusters, where the most impaired patients – cluster 3 – had on average 2 years less of schooling and almost one standard deviation less of IQ than the cognitively preserved group. Differences in IQ could be tautological as positive associations have been described between IQ and cognitive performance. However, in clinical practice it is usually observed that patients with normal or intact premorbid IQ show relevant cognitive impairment, and the other way around (patients with limited IQ who do not show any cognitive deficits). Although globally impaired patients had the lowest IQ in this study, their intellectual ability fell within normality, and their cognitive performance was below normality (1.5s.d. below, on average). All the above-mentioned variables, which are normally used as proxies for cognitive reserve, showed a significant importance in previous studies of cluster analysis (Pu et al., Reference Pu, Noda, Setoyama and Nakagome2018; Vicent-Gil et al., Reference Vicent-Gil, Keymer-Gausset, Serra-Blasco, Carceller-Sindreu, de Diego-Adeliño, Trujols and Portella2018). And cognitive reserve itself has specifically shown to moderate the relationship between mood and cognition (Opdebeeck et al., Reference Opdebeeck, Matthews, Wu, Woods, Brayne and Clare2017). Thus, cognitive reserve should not be neglected when assessing MDD patients.

By considering both cognitive and other illness-related variables, the three clusters may reflect variations of the disorder due to differences in underlying pathophysiology, in response to treatment or in illness trajectories, and not merely in cognitive subdivisions on a linear continuum (Carruthers, Van Rheenen, Gurvich, Sumner, & Rossell, Reference Carruthers, Van Rheenen, Gurvich, Sumner and Rossell2019). Consequently, the clinical implication of the present results refers to the necessity of considering cognitive functioning in clinical settings in all patients diagnosed with major depression. First, clinicians should seriously contemplate addressing cognitive symptoms when a given patient begins to show resistance to treatment, as cognitive difficulties may be related to worse treatment outcomes. Therefore, the intervention should be directed not only to clinical symptoms, but also to cognitive dysfunction. Second, 42% of patients who respond adequately to antidepressants may also present a global cognitive dysfunction. In clinical practice, some patients with a good response to antidepressant treatment complain about a lack of complete recovery and of difficulties to perform daily activities, partly due to their perceived cognitive problems. Detecting such unidentified cognitive difficulties is of great importance as they are associated with worse psychosocial functioning (Cambridge, Knight, Mills, & Baune, Reference Cambridge, Knight, Mills and Baune2018; Knight & Baune, Reference Knight and Baune2018) and with the low rates of recovery (Groves, Douglas, & Porter, Reference Groves, Douglas and Porter2018). Hence, treating cognitive dysfunction together with clinical symptomatology in an active episode of depression could probably result in a better response to treatment. Different pharmacological strategies (e.g. vortioxetine, duloxetine, modafinil or erythropoietin), non-pharmacological approaches (cognitive remediation or aerobic exercise) and neurostimulation interventions have been shown to be effective in the treatment of cognitive dysfunction while improving psychosocial functioning and quality of life (Salagre et al., Reference Salagre, Solé, Tomioka, Fernandes, Hidalgo-Mazzei, Garriga and Grande2017; Zuckerman et al., Reference Zuckerman, Pan, Park, Brietzke, Musial, Shariq and McIntyre2018).

The current study was subject to some limitations. First, due to cross-sectional design it was not possible to assess the long-term stability of the cognitive profiles. Second, the sample included outpatients treated in a specialized clinical setting, and may not comprise worldwide clinical practice as mild outpatients may be underrepresented. Third, the mean age of the sample was older than other studies, which could have facilitated the inclusion of patients with cognitive deficits caused by other conditions (such as neurodegenerative processes). In any case, the presence of a neurological condition was an exclusion criterion which was strictly evaluated. Fourth, although two-step clustering analysis is one of the most robust techniques to classify individuals (as it combines k-means and hierarchical approaches), the generalizability of the findings is one of the main drawbacks as the results are very sample-specific. The lack of external replication with an independent dataset represents a limitation of these results. In any case, our findings are fairly similar to the scarce literature. Fifth, individual scores were corrected with demographic-adjusted norms from a similar population, which might affect the interpretation of the present findings; therefore, future research should consider the cognitive heterogeneity within the norm samples. To overcome this limitation, composite scores for cognitive performance were used as a more objective patient's cognitive performance outcome compared to the use of single tests (Miskowiak et al., Reference Miskowiak, Burdick, Martinez-Aran, Bonnin, Bowie, Carvalho and Vieta2017). Sixth, the variety of impaired cognitive domains within the ‘selectively impaired cluster’, when compared to other studies, reflects another kind of heterogeneity that cannot be resolved with cluster analysis, as the solution depends on the samples used in each study. However, by using a two-step clustering, we detected two clear and extreme subgroups (i.e. preserved and globally impaired) together with a subgroup of selective impairment, in which other factors, beyond cognition, help to better characterize them. Finally, concomitant medication could be a possible confounder because of its side effects on cognition. But due to clinical reasons and ethical concerns, it was not adequate to discontinue the medication.

In conclusion, the current study shows the existence of distinguishable subgroups in a sample of acute depressed patients, in which treatment resistance and cognitive performance are relevant factors to take into account. The current design provides evidence for the heterogeneity of cognitive dysfunction in MDD. Therefore, future clinical research should consider the existence of potential cognitive dysfunction in all MDD patients, to tailor new strategies to achieve a full clinical and functional recovery and to prevent new relapses.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S0033291720001567.

Acknowledgements

We gratefully acknowledge patients who participated in this study. We also thank the staff of the Department of Psychiatry, Hospital Universitari Parc Taulí for their assistance, especially to Dr Figuereo, Dr Ros, Dr Acebillo and Dr Coronas for their kind cooperation in patient recruitment. To end, we are most grateful to Roser Granero for her helpful comments regarding statistical analysis.

Financial support

This work was supported by Lundbeck España (Institutional ethical committee # 2015630).

Conflict of interest

Dr Maria J. Portella is funded by the Instituto Carlos III through a research contract Miguel Servet II (CP16/00020) and from the Health Research and Innovation Strategic Plan (PERIS SLT006_17_177). She has also received honoraria as academic from Lundbeck. Dr Sara Crivillés has received honoraria as consultant, advisor or CME speaker for Janssen-Cilag, Lundbeck/Otsuka, Lab Esteve and Lab Servier. Dr Eva Aguilar has received grants and served as consultant, advisor or CME speaker for Janssen-Cilag and Lundbeck/Otsuka. Dr Diego Palao has received research funding from the Spanish Ministry of Science and Innovation, Instituto Carlos III and from Health Research and Innovation Strategic Plan (PERIS). Dr Narcís Cardoner declares that he has received funding from the Universitat Autònoma de Barcelona, from the Spanish Ministry of Science and Innovation, Instituto Carlos III and from Health Research and Innovation Strategic Plan (PERIS) 2016–2020. He has also received honoraria as consultant, advisor or CME speaker from Janssen, Lundbeck, Pfizer, Exeltis and MSD within the last 3 years. The rest of authors have nothing to disclose.

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Footnotes

*

Equal contribution authorship.

References

Benton, A. L., & Hamsher, K. (1976). Multilingual aphasia examination. Iowa City: University of Iowa.Google Scholar
Biringer, E., Mykletun, A., Sundet, K., Kroken, R., Stordal, K. I., & Lund, A. (2007). A longitudinal analysis of neurocognitive function in unipolar depression. Journal of Clinical and Experimental Neuropsychology, 29, 879891.CrossRefGoogle ScholarPubMed
Bobes, J., Bulbena, A., Luque, A., Dal-Ré, R., Ballesteros, J., & Ibarra, N. (2003). Evaluación psicométrica comparativa de las versiones en español de 6, 17 y 21 ítems de la escala de valoración de Hamilton para la evaluación de la depresión. Medicina Clinica, 120, 693700.CrossRefGoogle Scholar
Bonnín, C. M., Martínez-Arán, A., Reinares, M., Valentí, M., Solé, B., Jiménez, E., … Rosa, A. R. (2018). Thresholds for severity, remission and recovery using the functioning assessment short test (FAST) in bipolar disorder. Journal of Affective Disorders, 240, 5762.CrossRefGoogle Scholar
Bora, E., Harrison, B. J., Yücel, M., & Pantelis, C. (2013). Cognitive impairment in euthymic major depressive disorder: A meta-analysis. Psychological Medicine, 43, 20172026.CrossRefGoogle ScholarPubMed
Burdick, K. E., Russo, M., Frangou, S., Mahon, K., Braga, R. J., Shanahan, M., & Malhotra, A. K. (2014). Empirical evidence for discrete neurocognitive subgroups in bipolar disorder. Psychological Medicine, 44, 30833096.CrossRefGoogle ScholarPubMed
Cambridge, O. R., Knight, M. J., Mills, N., & Baune, B. T. (2018). The clinical relationship between cognitive impairment and psychosocial functioning in major depressive disorder: A systematic review. Psychiatry Research, 269, 157171.CrossRefGoogle ScholarPubMed
Carruthers, S. P., Van Rheenen, T. E., Gurvich, C., Sumner, P. J., & Rossell, S. L. (2019). Characterising the structure of cognitive heterogeneity in schizophrenia spectrum disorder: a systematic review and narrative synthesis. Neuroscience and Biobehavioral Reviews, 107, 252278.CrossRefGoogle ScholarPubMed
Casals-Coll, M., Sánchez-Benavides, G., Quintana, M., Manero, R. M., Rognoni, T., Calvo, L., … Peña-Casanova, J. (2013). Estudios normativos españoles en población adulta joven (proyecto NEURONORMA jóvenes): normas para los test de fluencia verbal. Neurologia (Barcelona, Spain), 28, 3340.CrossRefGoogle Scholar
Clark, M., DiBenedetti, D., & Perez, V. (2016). Cognitive dysfunction and work productivity in major depressive disorder. Expert Review of Pharmacoeconomics & Outcomes Research, 16, 455463.CrossRefGoogle ScholarPubMed
Cotrena, C., Branco, L. D., Kochhann, R., Shansis, F. M., & Fonseca, R. P. (2016). Quality of life, functioning and cognition in bipolar disorder and major depression: a latent profile analysis. Psychiatry Research, 241, 289296.CrossRefGoogle ScholarPubMed
Cotrena, C., Branco, L. D., Ponsoni, A., Shansis, F. M., & Fonseca, R. P. (2017). Neuropsychological clustering in bipolar and major depressive disorder. Journal of the International Neuropsychological Society, 23, 110.CrossRefGoogle ScholarPubMed
Dotson, V. M., Resnick, S. M., & Zonderman, A. B. (2008). Differential association of concurrent, baseline, and average depressive symptoms with cognitive decline in older adults. American Journal of Geriatric Psychiatry, 16, 318330.CrossRefGoogle ScholarPubMed
Douglas, K. M., Gallagher, P., Robinson, L. J., Carter, J. D., McIntosh, V. V., Frampton, C. M., … Porter, R. J. (2018). Prevalence of cognitive impairment in major depression and bipolar disorder. Bipolar Disorders, 20, 260274.CrossRefGoogle ScholarPubMed
Evans, V. C., Iverson, G. L., Yatham, L. N., & Lam, R. W. (2014). The relationship between neurocognitive and psychosocial functioning in major depressive disorder: A systematic review. Journal of Clinical Psychiatry, 75, 13591370.CrossRefGoogle ScholarPubMed
Fekadu, A., Wooderson, S. C., Markopoulou, K., & Cleare, A. J. (2009). The Maudsley Staging Method for treatment-resistant depression: Prediction of longer-term outcome and persistence of symptoms. Journal of Clinical Psychiatry, 70, 952957.CrossRefGoogle ScholarPubMed
Ge, R., Torres, I., Brown, J. J., Gregory, E., McLellan, E., Downar, J. H., … Vila-Rodriguez, F. (2019). Functional disconnectivity of the hippocampal network and neural correlates of memory impairment in treatment-resistant depression. Journal of Affective Disorders, 253, 248256.CrossRefGoogle ScholarPubMed
Groves, S. J., Douglas, K. M., & Porter, R. J. (2018). A systematic review of cognitive predictors of treatment outcome in major depression. Frontiers in Psychiatry, 9, 382.CrossRefGoogle ScholarPubMed
Hamilton, M. (1960). A rating scale for depression. Journal of Neurology Neurosurgery & Psychiatry, 23, 5662.CrossRefGoogle ScholarPubMed
Hammar, A., & Ardal, G. (2009). Cognitive functioning in major depression – a summary. Frontiers in Human Neuroscience, 3, 17.CrossRefGoogle ScholarPubMed
Hasselbalch, B. J., Knorr, U., & Kessing, L. V. (2011). Cognitive impairment in the remitted state of unipolar depressive disorder: a systematic review. Journal of Affective Disorders, 134, 2031.CrossRefGoogle ScholarPubMed
Heaton, R. K. (1981). Wisconsin card sorting test manual. Odessa: Psychological Assessment Resources.Google Scholar
Iverson, G. L., Brooks, B. L., Langenecker, S. A., & Young, A. H. (2011). Identifying a cognitive impairment subgroup in adults with mood disorders. Journal of Affective Disorders, 132, 360367.CrossRefGoogle ScholarPubMed
Knight, M. J., & Baune, B. T. (2018). Cognitive dysfunction in major depressive disorder. Current Opinion in Psychiatry, 31, 2631.CrossRefGoogle ScholarPubMed
Lee, R. S. C., Hermens, D. F., Porter, M. A., & Redoblado-Hodge, M. A. (2012). A meta-analysis of cognitive deficits in first-episode major depressive disorder. Journal of Affective Disorders, 140, 113124.CrossRefGoogle ScholarPubMed
Lima, F., Rabelo-da-Pontea, F. D., Bückera, J., Czepielewskia, L., Hasse-Sousaa, M., Telescaa, R., … Rosa, A. R. (2019). Identifying cognitive subgroups in bipolar disorder: A cluster analysis. Journal of Affective Disorders, 246, 252261.CrossRefGoogle ScholarPubMed
McAllister-Williams, R. H., Bones, K., Goodwin, G. M., Harrison, J., Katona, C., Rasmussen, J. S., … Young, A. H. (2017). Analysing UK clinicians’ understanding of cognitive symptoms in major depression: a survey of primary care physicians and psychiatrists. Journal of Affective Disorders, 207, 346352.CrossRefGoogle ScholarPubMed
McDermott, L. M., & Ebmeier, K. P. (2009). A meta-analysis of depression severity and cognitive function. Journal of Affective Disorders, 119, 18.CrossRefGoogle ScholarPubMed
Miskowiak, K. W., Burdick, K. E., Martinez-Aran, A., Bonnin, C. M., Bowie, C. R., Carvalho, A. F., … Vieta, E. (2017). Methodological recommendations for cognition trials in bipolar disorder by the International Society for Bipolar Disorders Targeting Cognition Task Force. Bipolar Disorders, 19, 113.CrossRefGoogle ScholarPubMed
Mur, M., Portella, M. J., Martínez-Arán, A., Pifarré, J., & Vieta, E. (2007). Persistent neuropsychological deficit in euthymic bipolar patients: executive function as a core deficit. Journal of Clinical Psychiatry, 68, 10781086.CrossRefGoogle ScholarPubMed
Murrough, J. W., Burdick, K. E., Levitch, C. F., Perez, A. M., Brallier, J. W., Chang, L. C., … Iosifescu, D. V. (2015). Neurocognitive effects of Ketamine and association with antidepressant response in individuals with treatment-resistant depression: a randomized controlled trial. Neuropsychopharmacology, 40, 10841090.CrossRefGoogle ScholarPubMed
Norusis, M. (2011). IBM SPSS statistics 19 procedures companion. Reading, MA: Addison.Google Scholar
Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14, 535569.CrossRefGoogle Scholar
Opdebeeck, C., Matthews, F. E., Wu, Y.-T., Woods, R. T., Brayne, C., & Clare, L. (2017). Cognitive reserve as a moderator of the negative association between mood and cognition: Evidence from a population-representative cohort. Psychological Medicine, 48, 111.Google ScholarPubMed
Peña-Casanova, J., Quinones-Ubeda, S., Gramunt-Fombuena, N., Quintana-Aparicio, M., Aguilar, M., Badenes, D., … Blesa, R. (2009). Spanish multicenter normative studies (NEURONORMA project): norms for verbal fluency tests. Archives of Clinical Neuropsychology, 24, 395411.CrossRefGoogle ScholarPubMed
Pimontel, M. A., Rindskopf, D., Rutherford, B. R., Brown, P. J., Roose, S. P., & Sneed, J. R. (2016). A meta-analysis of executive dysfunction and antidepressant treatment response in late-life depression. The American Journal of Geriatric Psychiatry, 24, 3141.CrossRefGoogle ScholarPubMed
Pu, S., Noda, T., Setoyama, S., & Nakagome, K. (2018). Empirical evidence for discrete neurocognitive subgroups in patients with non-psychotic major depressive disorder: clinical implications. Psychological Medicine, 48, 27172729.CrossRefGoogle ScholarPubMed
Rao, D., Xu, G., Lu, Z., Liang, H., Lin, K., & Tang, M. (2019). Comparative study of cognitive function between treatment-resistant depressive patients and first-episode depressive patients. Neuropsychiatric Disease and Treatment, 15, 34113417.CrossRefGoogle ScholarPubMed
Rey, A. (1941). L'examen clinique en psychologie. Paris: Presses Universitaires de France.Google Scholar
Rosa, A. R., Sánchez-Moreno, J., Martínez-Aran, A., Salamero, M., Torrent, C., Reinares, M., … Vieta, E. (2007). Validity and reliability of the functioning assessment short test (FAST) in bipolar disorder. Clinical Practice and Epidemiology in Mental Health, 3, 110.CrossRefGoogle Scholar
Salagre, E., Solé, B., Tomioka, Y., Fernandes, B. S., Hidalgo-Mazzei, D., Garriga, M., … Grande, I. (2017). Treatment of neurocognitive symptoms in unipolar depression: a systematic review and future perspectives. Journal of Affective Disorders, 221, 205221.CrossRefGoogle ScholarPubMed
Semkovska, M., Quinlivan, L., O'Grady, T., Johnson, R., Collins, A., O'Connor, J., … Gload, T. (2019). Cognitive function following a major depressive episode: a systematic review and meta-analysis. The Lancet Psychiatry, 6, 851861.CrossRefGoogle Scholar
Serra-Blasco, M., de Vita, S., Rodríguez, M. R., de Diego-Adeliño, J., Puigdemont, D., Martín-Blanco, A., … Portella, M. J. (2015). Cognitive functioning after deep brain stimulation in subcallosal cingulate gyrus for treatment-resistant depression: an exploratory study. Psychiatry Research, 225, 341346.CrossRefGoogle Scholar
Serra-Blasco, M., & Lam, R. W. (2019). Clinical and functional characteristics of cognitive dysfunction in major depressive disorder. In Harmer, C. J. & Baune, B. T. (Eds.), Cognitive dimensions of Major depressive disorder (pp. 4558). Oxford: Oxford University Press.CrossRefGoogle Scholar
Serra-Blasco, M., Torres, I. J., Vicent-Gil, M., Goldberg, X., Navarra-Ventura, G., Aguilar, E., … Cardoner, N. (2019). Discrepancy between objective and subjective cognition in major depressive disorder. European Neuropsychopharmacology, 29, 4656.CrossRefGoogle ScholarPubMed
Solé, B., Jiménez, E., Torrent, C., del Mar Bonnin, C., Torres, I., Reinares, M., … Martínez-Arán, A. (2016). Cognitive variability in bipolar II disorder: who is cognitively impaired and who is preserved. Bipolar Disorder, 18, 288299.CrossRefGoogle ScholarPubMed
Tombaugh, T. N. (2004). Trail making test A and B: Normative data stratified by age and education. Archives of Clinical Neuropsychology, 19, 203214.CrossRefGoogle Scholar
Venezia, R. G., Gorlyn, M., Burke, A. K., Oquendo, M. A., Mann, J. J., & Keilp, J. G. (2018). The impact of cognitive reserve on neurocognitive performance in major depressive disorder. Psychiatry Research, 270, 211218.CrossRefGoogle ScholarPubMed
Vicent-Gil, M., Keymer-Gausset, A., Serra-Blasco, M., Carceller-Sindreu, M., de Diego-Adeliño, J., Trujols, J., … Portella, M. J. (2018). Cognitive predictors of illness course at 12 months after first-episode of depression. European Neuropsychopharmacology, 28, 529537.CrossRefGoogle ScholarPubMed
Wechsler, D. (2008). Weschler adult intelligence scale (WAIS-IV) (4th ed.). San Antonio, Texas: Pearson.Google Scholar
Zuckerman, H., Pan, Z., Park, C., Brietzke, E., Musial, N., Shariq, A. S., … McIntyre, R. S. (2018). Recognition and treatment of cognitive dysfunction in major depressive disorder. Frontiers in Psychiatry, 9, 655.CrossRefGoogle ScholarPubMed
Figure 0

Fig. 1. Clustering summary: relative importance of each indicator.

Figure 1

Table 1. Clustering summary: centroids

Figure 2

Fig. 2. Radar chart for the distribution of the indicators of the model. HDRS-17: Hamilton Depression Rating Scale.

Figure 3

Table 2. Mean scores (standard deviation) for demographic and clinical variables across clusters

Figure 4

Table 3. Mean scores (standard deviation) for cognitive variables across clusters

Supplementary material: Image

Vicent-Gil et al. supplementary material

Figure S2

Download Vicent-Gil et al. supplementary material(Image)
Image 414.6 KB
Supplementary material: File

Vicent-Gil et al. supplementary material

Table S1

Download Vicent-Gil et al. supplementary material(File)
File 23.3 KB
Supplementary material: File

Vicent-Gil et al. supplementary material

Table S2

Download Vicent-Gil et al. supplementary material(File)
File 22.1 KB
Supplementary material: Image

Vicent-Gil et al. supplementary material

Figure S1

Download Vicent-Gil et al. supplementary material(Image)
Image 185.9 KB