Screening and measurement of cognitive impairment in psychiatry

Abstract

A key message from the review of cognitive dysfunction in psychiatry published by Millan et al (2012) was not just that cognitive skills are often compromised in patients with psychiatric disorders, but that deficits in specific domains are common to a number of conditions. The review also highlighted that the magnitude of the observed deficits varied across disorders. A helpful element of the Millan et al study was the inclusion of a table in which the authors sought to convey the domains of cognition and a categorization of the magnitude of the observed deficits.

In previous articles, we have considered best practice for the assessment of cognition. In these contributions, we have argued not for the use of specific tests, but instead for measures that meet acceptable standards of reliability, validity, and sensitivity. In the course of our discussions, we have included reference to test validity in the context of considering whether selected measures index appropriate domains of cognition. In this article, we begin with a brief discussion of the requirements for good test selection, especially with respect to issues of sensitivity, reliability, and validity. Thereafter the focus of this article is on the issue of domain validity. We will critically review the specification of the cognitive domains proposed by Millan et al, as well as those selected by authors of meta-analyses characterizing cognitive deficits in major depressive disorders. This focus is solely to make the discussion tractable, though we propose that the issues raised will be applicable across all psychiatric and neurological disorders.

Introduction

Milan et al in their review of cognitive deficits characteristic of psychiatric disorders helpfully described the pattern and magnitude of observed deficits. ¹ In previous articles, we have offered best practice guidance on the assessment of cognitive status, which we define as a judgment regarding an individual’s levels of performance on valid, reliable, and sensitive measures of various cognitive skills. ² Such judgments are made in the context of expectations of performance based on comparison with normative data collected for cohorts appropriate for comparison, typically based on individuals of the same sex, with similar ages and levels of education. We have previously contrasted the question of determining impairment with the assessment of cognitive change, which we define as circumstances in which we wish to determine whether there has been a change in an individual’s cognitive status. ³ In the determination of both cognitive status and cognitive change, we have stressed that selected tests must exhibit the desired characteristics of a suitable measurement instrument, specifically reliability, sensitivity, and validity. We have commented extensively on issues of reliability and sensitivity in previous publications. In this contribution, we focus on the topic of validity, and especially that of the constructs we assess when conducting cognitive assessment. In our discussions of this issue, we will characterize what have been described as core cognitive domains. We will therefore necessarily consider the utility of dichotomies used to distinguish cognitive domains. At a micro-level this analysis extends to distinctions applied in the context of specific constructs, such as “episodic” and “semantic” memory. At a macro-level, we will consider the proposed distinction between so-called “hot” and “cold” cognition. A key feature of these discussions will be an evaluation of these dichotomies and constructs in the context of the additional explanatory value such distinctions bring. Before embarking on these analyses, we will begin by considering why the reliable, sensitive, and valid measurement of cognition has such importance, particularly in the context of assessing cognition in patients with depression.

Assessing Cognition in Patients with Depression

There is an ever-increasing number of interventions available for the management of depression. ⁴ Pharmacological interventions have been a mainstay of disease management, and recently interest in their use has extended from antidepressant approaches to include procognitive effects. ⁵ Nonpharmacological interventions have also had a role to play, and techniques such as cognitive behavioral therapy (CBT), cognitive remediation, and electroconvulsive therapy (ECT) have demonstrated evidence of efficacy. However, the literature regarding cognitive efficacy remains mixed, with some studies reporting no benefits of treatment. ⁶ In this context, it has been our suspicion that on occasion the observation of no treatment benefit has not been demonstrated, not necessarily because of a lack of efficacy, but because of poor measurement. ⁷ As we have here and repeatedly previously stated, cognitive measures employed in trials must exhibit acceptable levels of reliability, sensitivity, and validity. Failure to meet acceptable standards with respect to all 3 of these requirements undermines the credibility of any claim regarding the true status of cognitive abilities in the study population of interest. Tests may be insufficiently sensitive to changes in cognitive status due to a variety of reasons, but key issues are range restrictions and gross measurement intervals. By analogy on this latter point, a ruler with millimeter gradations is capable of making finer distinctions than one with centimeter gradations. With respect to the validity of employed measures, it is essential that the appropriate facets of the disease be investigated. In the context of measuring cognition, this means the measurement of those cognitive skills known to be compromised in patients with depression. This is a topic we will consider in detail later, but for now we propose that studies of possible cognitive treatment effects must at a minimum include measures of attention, episodic memory, working memory, and various facets of executive function. Note the possibly surprising omission from this list of psychomotor speed, a topic we will turn to later.

In the opening section, we described the importance of accurately assessing individuals at specific junctures, as well as over time. Judgments regarding cognitive status at specific time points are usually made in the context of determining whether an individual exhibits evidence of impairment. Such judgments are made statistically, with levels of performance falling 1.96 standard deviations below the population mean representing the threshold for impairment. This criterion is analogous to the employment of an alpha-level of 5% in significance testing, as application of a ±1.96 SD cut-off equates to levels of performance seen in the top 2.5% and the bottom 2.5% of the population. Similar judgments are often required when describing the mean level of performance of cohorts diagnosed with specific psychiatric and neurological disorders. Here, effect size has emerged as a convenient lingua franca for comparing performance between study cohorts. ⁸ These are usually comparisons of “typical controls” with those who meet generally accepted criteria for diagnostic categories, such as can be found in the World Health Organization (WHO) International Statistical Classification of Diseases and Related Health Problems. ⁹ In psychiatry, the Diagnostic and Statistical Manual of Mental Disorders (DSM) is also commonly applied. These classification systems evolve and hence we have available to us the 10th version of the ICD and the 5th edition of the DSM. ¹⁰

Effect size differences between typical control and disease-specific groups offer a means by which we can quantify levels of impairment. Judgments about whether group performance differs significantly can be determined with the use of inferential statistical testing.However, some members of the disease category may not necessarily exhibit levels of cognitive impairment understood to be typical of disease category membership. To take the example of major depressive disorders, group levels of impairment are typically observed to be in the 0.5–0.8 effect size range. ^{11 – 14} Maruff and Jaeger ¹⁵ sought to determine the extent to which individual patients with MDD exhibited evidence of impairment. Their analysis indicates that with the FOCUS study cohort, 60% of patients exhibited evidence of cognitive impairment at baseline. ¹⁵ This is unsurprisingly a substantial proportion of the cohort. However, this analysis indicates that 40% of the FOCUS cohort recorded levels of performance that were essentially normal. A key point of consideration is the methodology employed by the authors. They adopted the previously discussed –2 SD below the mean cut-off criterion. This was supplemented with a second criterion according to which impairment at the level of –1 SD on at least 2 tasks was also taken as evidence of impairment. These twin criteria seem to us a reasonable operational definition of impairment. However, this is not to suggest that those categorized as having no impairment were genuinely cognitively normal. Some members of this cohort may have been performing less well than they might have prior to onset of the disease. In the absence of premorbid data, it is not possible to determine whether performance in these individuals has worsened. Some methods exist with which to estimate premorbid IQ, which can in turn be used to predict premorbid performance on measures such as verbal fluency. However, as observed by the authors of a recent review, there are still “considerable limitations associated with all currently available methods” (p. 2). ¹⁶

Given the foregoing observations, what might we reasonably conclude about the extent and degree of cognitive impairment observed in patients with MDD? As stated, when compared with typical controls, deficits between –0.5 and –0.8 are reported. However, there would appear to be considerable heterogeneity within study cohorts of patients with MDD, with perhaps as many as 40% presenting with normal cognition. One interpretation of these data is that these impaired and unimpaired cohorts represent different subtypes of the disorder and that systematic evaluation of these patients might reveal different predispositions to variants of the disorder.

Cognitive Domains

The totality of human cognition, what we might refer to as “thinking,” has been divided into subcategories. In everyday parlance we refer to memory, paying attention, problem-solving, etc. This folk psychology has correlates with the vocabulary of cognitive science, though extra levels of categorization have been employed. Memory, for example, has been dichotomized several ways. The construct “memory” has been expanded beyond the recollection of information to include evidence of skill-learning, yielding a distinction between “implicit” (the latter) and “explicit” (the former) memory. A further distinction has been made between semantic and episodic memory. Semantic memory is characterized by our knowledge of facts and constructs, whereas episodic memory is our recollection of specific events. The basis of this distinction has been a methodology popular in human cognitive neuropsychology based on dissociation. The logic of this approach is that if an individual becomes impaired in remembering new events (new episodes), but their recollection of facts is intact, then this constitutes a dissociation. These are the circumstances that can be observed in patients with early stage Alzheimer’s disease, whose most obvious difficulty is usually the acquisition of new information. This dissociation can be seen on tests of orientation, on which the patient can exhibit impairments regarding his or her recollection of the date (episodic memory) but can recall their name (semantic memory). However, as the disease progresses semantic memory will also be compromised. This same pattern of selective deficits for new episodic memories has also been seen in patients with specific lesions, for example patient HM. ¹⁷ This patient underwent neurosurgery to help control his hitherto intractable epilepsy. After the procedure, which involved the re-sectioning of medial-temporal lobe structures, HM developed severe anterograde amnesia, while his working memory and implicit memory skills were left intact.

Dissociations of the kind described are a key technique for discerning the components of human cognition. As will have been clear from the cited example, the technique relies on showing how an individual cognitive skill can be selectively impaired. However, there is an alternative possibility, that the task on which the patient appears impaired is altogether more difficult to perform normally than other aspects of cognition. Ellis and Young ¹⁸ considered this issue in the context of stronger evidence for dissociating cognitive domains—what they refer to as a “double dissociation,” circumstances they describe using the examples of (i) disordered face recognition versus (ii), written word recognition. They write that a double dissociation could be said to have occurred if a case of selective written word recognition (in Patient X) had been observed and “If a second patient, Y, could be discovered in whom written word recognition was intact whereas face recognition was impaired. That patient when contrasted with patient X would provide us with a double dissociation between face recognition and written word recognition” (p. 5). ¹⁸ In practice, individual cognitive skills are only rarely entirely extinguished, and, typically, as explained by Ellis and Young, it is “sufficient to show that patient Y performed reliably and significantly better on task 1 than on task 2 whereas patient X performed reliably and significantly better on task 2 than on task 1”(p. 5). ¹⁸

Selective cognitive deficits are often consequent upon brain insults, and there is a tendency to link neurological damage to cognitive deficits as part of the endeavor to localize function, an enterprise that has a controversial history, but one that is beyond the remit of this article. We will instead limit ourselves to functional dissociations, ie, dissociations between cognitive domains, with no reference to their possible neural substrates. Functional dissociations suggest a modular structure for human cognition and offer an architecture for considering the cognitive deficits reported in patients with depression, as well as other CNS disorders.

The literature describing cognitive deficits in patients with depression is substantial, and often authors describe the tests employed as being measures of particular cognitive domains. For our purposes it is sufficient to contemplate a few summary examples. We will therefore limit our consideration to recent meta-analyses of cognitive deficits, specifically those published by Rock et al, ¹¹ Lee et al, ¹² Ahern and Semkovska, ¹³ and Snyder. ¹⁴ Each author, or set of authors, lists the cognitive domains reported, and Table 1 summarizes the domains identified in each meta-analysis.

Table 1 Cognitive domains listed in selected meta-analysis studies.

Where possible we have aligned similar domains on the basis of the tests used to describe individual functions. For example, Lee et al ¹² describe Trail Making Test A performance as a measure of “processing speed,” Ahern and Semkovska ¹³ describe it as a measure of “psychomotor speed,” and Snyder ¹⁴ lists this measure as a test of “shifting.” Snyder ¹⁴ also lists the Wisconsin Card Sorting Test (WCST) as a measure of shifting, whereas for Lee et al ¹² the WCST is a test of “cognitive flexibility.” All 3 of these authors refer to verbal fluency as a distinct domain, though in the wider neuropsychological community tests of fluency are usually considered to be tests of executive function and language (eg, Lezak ¹⁹ ).

The cognitive nomenclature employed by Millan et al ¹ employs the same or similar domains, though they make further specific reference to procedural memory, semantic memory, fear extinction, and social cognition. In their characterization of major depression, they comment that studies of fear extinction were “essentially absent,” “poorly documented, ambiguous, mild and or variable,” and “not clearly evaluated” (p. 144). We shall not therefore refer further to this characteristic. Social cognition was listed as largely “consistently present but not pronounced” (p. 144). Social cognition’s status as an independent cognitive domain remains controversial. However, a detailed discussion of this issue is beyond the scope of this article and it will also not therefore be discussed here in detail.

The specification of domains is not peculiar to issues of cognition in depression. A similar enterprise was engaged in by the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) group. ²⁰ Part of this process was the identification of cognitive domains relevant to presentation and treatment in cases of cognitive impairment associated with schizophrenia (CIAS). ²¹ The cognitive domains identified, together with the measures selected to form their assessment, are shown in Table 2.

Table 2 Summary of cognitive domains and selected measures identified by the MATRICS group.

There is considerable similarity between the MATRICS characterization of cognition and the reported meta-analysis studies and Millan et al’s ¹ review. Attention/vigilance is again represented, as are working memory and social cognition. Verbal and visual memory are in the MATRICS taxonomy referred to as visual and verbal learning, but there is very substantial overlap between these “memory” and “learning” constructs. However, there are 2 distinctions of note. First, tests of symbol coding, category fluency, and simple trail making are described as “speed of processing” tasks. This is essentially the same nomenclature as that employed by Millan et al ¹ and Ahern and Semkovska, ¹³ who use the term “processing speed.” Our view is that processing speed helpfully emphasizes that the described behavior is a process rather than a domain. Similarly, and as we shall later discuss, reaction time is best thought of as an outcome metric and not a cognitive domain or cognitive process. Later work on the MATRICS assessment using confirmatory factor analysis provided support for the cognitive domains structure of the MATRICS assessment as evaluated with the MATRICS Consensus Cognitive Battery (MCCB). ²² The analysis conducted by McCleery et al ²² is of course peculiar to the characterization of cognition in patients with schizophrenia assessed using the MCCB. However, this methodological approach could reasonably be applied to conceptualizations of cognitive domains in other diseases, and to the performance of typical control study participants.

Our commentary on specific cognitive measures as tests of individual cognitive skills is not intended to be a criticism of the authors, but instead a recognition of the challenges in associating tests with function. Helpful context can be gained by deconstructing what is intended by stating that a test is a measure of a specific domain. As an example, let us take the California Verbal Learning Test (CVLT), which both Lee et al ¹² and Ahern and Semkovska ¹³ describe as a test of verbal learning and memory. The CVLT is similar to other auditory verbal learning tests, such as the Rey Auditory Verbal Learning Test (RAVLT), in that the study participant is read a list of words on 5 successive trials and after every trial is asked to recall as many items as possible from the list. A minor difference between the CVLT and the RAVLT is that the word list length is 1 word longer for the CVLT at 16 words. A more significant difference between the 2 tests is that the CVLT word list is arranged as a “shopping list.” The “Monday” list for example contains 4 articles of fruit, 4 items of clothing, 4 types of tools, and 4 different herbs and spices. Lezak ¹⁹ points out that this organization of the “to-be-remembered-items” “provides important information about the subject’s use of learning strategies” (p. 446).

Strategic thinking is often considered to be an element of the executive functions, and so the CVLT, in addition to measuring verbal learning and memory, is also indexing an aspect of executive function. Additionally, the first stage of memory is encoding the To-be-remembered-items, a process reliant on attentional skills. Furthermore, successful test completion is reliant also on verbal comprehension of the task instructions, as well as intact articulatory abilities. Thus, while the CVLT is evidently a test of episodic verbal memory, it is also a test of a number of other key cognitive functions. These facts regarding cognition testing indicate that describing individual tests as measures of specific domains should not be taken too literally.

The development of effective test completion strategies plays a significant role in the performance of other cognitive tests. For example, performance on tests in which rules have to be generated and tested by the study participant, such as “set-shifting” and “problem solving” measures, benefit from the use of strategic thinking. In the Wisconsin Card Sorting Test, the study participant is required to divine that the 3 rules are color, shape, and number. In other set-shifting tests the study participant is required to work out that the basis for achieving the rule is to consider new variants of familiar exemplars (“intradimensional” set shifting), as well switching to previously irrelevant stimulus categories (“extradimensional” set shifting). Even in the successful completion of tests not primarily concerned with strategy, such as the CANTAB Spatial Working Memory (SWM) test and the Rey Osterreith Complex Figure (ROCF), task performance can be improved by developing a search strategy for completing the SWM test and an organizing principle for copying the ROCF. The risk with using tests that rely on a strategy in a repeated measures design is that divination or realization of an effective strategy can thereafter improve performance markedly, just as the sudden use of mnemonics can dramatically improve performance on tests of memory.

Performance improvement can also occur on Stroop tests, in which ink colors must be read out for color words that are either congruent (the ink color and the word name are the same, ie, the word RED written in red ink) or incongruent (the ink color and word do not match, ie, the word RED written in green ink). The latter condition typically requires more time for completion, as the word interferes with reading out the color of the ink. However, study participants who defocus their eyes for this task can complete the incongruent condition as quickly as the congruent condition, as the orthography of the word no longer interferes. The foregoing is not to suggest that these tests are incapable of detecting cognitive change. However, methods of detecting reliable change that include the Standard Error of Prediction (SEP) are directly impacted by the test’s temporal reliability, as test–retest reliability correlations are an element of the SEP calculation. Less reliable tests yield wider confidence intervals, making reliable change more difficult to observe (see Hinton-Bayre ²³ for a useful perspective on reliable change calculations).

Evidence for discrete cognitive modules is largely derived from neuropsychology, as well as, to a lesser extent, both experimental psychology and psychopharmacology. The methodology of dissociation and double dissociation offers a means by which cognitive domains can be established and empirically defended. As such they offer a disciplined means of determining the constructs that comprise human cognition. It is interesting in this context to apply these ideas to a recently proposed dichotomy between “hot” and “cold” cognition. This dichotomy has been a welcome attempt to integrate emotional content into considerations of cognitive processing. Authors of this theory define the 2 variants thus:

Hot cognition is that which involves emotional stimuli with survival implications for the organism, for example positive or negative reinforcements or emotional stimuli such as angry faces (p. 69).

Cold cognition, by contrast, is affectively neutral cognitive processing (eg, remembered spatial locations or numerical calculations) (p. 69). ²⁴

Although offered as a strict dichotomy, it is not hard to imagine instances in which so-called cold cognition involves an emotional component. In fact, a reasonable question is to ask whether any cognitive activity can be entirely free of some emotional content? We might similarly ask whether the use of facial stimuli necessarily evokes an emotional response? It seems likely that rather than representing a strict dichotomy, hot” and “cold” cognitive components represent dimensions along which specific individuals might vary according to the task presented and circumstances. The challenge then is to find a means of determining the extent to which emotions impinge upon task performance. Cognition measurement has typically been achieved by measuring behavioral variables, such as speed of response or accuracy, and then inferring the presence of cognitive processes. Thus, when administering cognitive tests, we record and analyze time, usually in milliseconds, or accuracy, usually expressed as the number correct. The measurement of emotion is problematic in this regard, as we have no agreed-upon metric with which to quantify the emotional content. Also problematic is that we have no established taxonomy of human emotion. The folk psychology of emotion extends to constructs such as fear, anger, joy, etc. However, problematic emotions such as fear and anger are physiologically indistinguishable, leading to problems of classification for scientific purposes. This issue has remained sufficiently controversial that the James–Lange theory, the premise of which is that emotion is experienced only once the brain reacts to information received via the body’s nervous system, has been difficult to disprove. Modern theorists, such as Barrett, ²⁵ tend to the view that individuals make meaning of a physical response based on context, prior experience, and social cues, before they know what emotion is attached to the situation.

It would seem that while we have a defendable taxonomy and measurement system for cognition, neither yet exist in the context of emotion. These deficiencies place limits on the value of a simple dichotomy between “hot” and “cold” cognition. Nevertheless, few would deny that memories contain some “emotional” component, just as a negative attentional bias can be evident in patients with depression. The presence of these phenomena is therefore not in question, but the cognitive substrates remain a point for which we need clarification. A simple interpretation we might invoke is that a negative attentional bias has been learned. Thus, the phenomenon is represented in semantic memory. In the context of CBT, the mechanism for changing this bias is to get the patient to change their behavior (ie, a negative attentional bias) by relearning their responses to stimuli. The disposition to develop a negative attentional bias may provide clues concerning who might be at risk for developing depression.

Is Psychomotor Speed a Discrete Cognitive Domain?

Earlier in this chapter we acknowledged our omission of psychomotor speed from the list of key cognitive domains that comprise a basic assessment. In the context of evaluating cognition in patients with depression, this at first look appears to be a glaring omission. This might seem especially odd given that reference to psychomotor slowing appears in both the DSM-5 and ICD-10 diagnostic criteria. While we would certainly acknowledge that measuring speed of response is an important component of assessing cognition in depression, we believe that psychomotor speed is best considered an outcome measure, but not a cognitive domain. Our own experience of assessing patients with depression includes a strong impression that patients often have difficulties in processing information as quickly as typical controls. This gives the clinical impression of slowness, which has understandably yielded the notion that patients exhibit psychomotor speed impairments. The laboratory analogue of this clinical feature is often considered to be performance on timed tests on which, subject to earlier caveats, patients with MDD seem reliably to exhibit impairment. However intuitive the idea of psychomotor speed impairment might seem, use of the term still begs the question, What do we mean when we invoke use of this expression? One way to think about this issue is to apply an information processing analysis to the tasks on which we observe patients to be slowed. In our view, references to psychomotor slowing are reminiscent of the construct of “bradyphrenia” that has been invoked to describe the cognitive slowing often observed in patients with Parkinson’s disease dementia (PDD). The term “bradyphrenia” was introduced as an analogous reference to bradykinesia and used to describe the cognitive slowing seen in patients with PDD in an analogous fashion to the slowness of movement that is a cardinal sign of the disease. Helpfully, reaction time tasks have been employed as laboratory analogues of cognitive slowing in both MDD and PDD. In the following section, we will consider which specific cognitive deficits might yield impairments in reaction time task performance.

Mental chronometry has often been investigated with the use of reaction time paradigms. The first use of reaction time paradigms to investigate cognition dates to the work of Donders. ²⁶ He employed 3 key paradigms, all of which have a place in modern cognitive assessment, with versions of the Donders tasks to be found in the assessments provided as part of the CogState, CANTAB, and CDR systems. Donders described 3 tasks. Donders A required a single response to a single stimulus, the paradigm that has come to be known as simple reaction time (SRT). The type B paradigm requires the correct selection of a response according to the stimulus that has been presented, the so-called choice reaction time (CRT) paradigm. The most common variant of this paradigm is in which 1 of 2 possible stimuli can be delivered as the imperative signal, which is the participant’s cue to select and deliver the appropriate response. However, the number of possible stimuli and responses has been assessed with multiple stimuli and responses. The third variant is a paradigm that also requires a selective response, but here the decision is not which response of 2 to select as in the CRT, but whether or not to respond. One stimulus requires a response (the “go” condition), whereas the other stimulus requires the participant to withhold a response (the “no go” condition). The ideal study participant in these tasks is seeking to respond as quickly as possible without making too many errors.

Timed performance on choice and simple reaction time tasks has yielded a reliable difference, with mean latency for CRT trials typically longer than that for SRT trials. A common interpretation of choice and simple reaction time performance has relied on a subtractive logic, such that performance in SRT is seen as a subset of those processes required for CRT. However, there is an alternative perspective built on robust empirical evidence. On this interpretation, the fast responses observed in the SRT paradigm, when compared to CRT, are due to the recruitment of an additional cognitive resource. In advance of turning to the empirical evidence, it is helpful to characterize the SRT and CRT tasks in the context of certainty. As will be evident from earlier descriptions of the 2 tasks, in SRT there is certainty regarding the nature of the stimulus to respond, or imperative signal (IS). There is also certainty regarding the required response; all that is unknown is when the IS will be delivered. Put differently the “what” and “where” are known; all that is uncertain is the “when.” Foreknowledge of the IS and the required response theoretically allow for preparation to be engaged in. This contrasts with the circumstances that govern CRT, in which neither the required response, nor the specific IS, are known in advance. In CRT, the study participant must await delivery of the CRT, evaluate its nature, and then select the correct response. In SRT all that is required to elicit the response is presentation of the IS. It seems likely that the speed advantage seen in SRT is thus due to the recruitment of a process via which selective, sustained attention in anticipation of the IS facilitates a quicker response. A further possibility is that the required response can also be maintained and held in a “motor buffer” in preparation for the onset of the IS.

There is empirical support for the notion that the SRT speed advantage is due to processes selective to this task. Frith and Done ²⁷ sought to test the hypothesis that responses can be specified in advance, that the stimulus does not need to be identified, and that “The subject needs merely to register that it has occurred” (p. 169). ²⁷ In their second experiment, they found that the imposition of a secondary task (reading) yielded SRT latencies that were as slow as those for CRT, but there was virtually no impact on CRT latency. This paradigm was later employed by Goodrich et al ²⁸ to investigate the selective impairment of SRT that had been reported in patients with Parkinson’s disease (PD). These investigators observed that patients with PD showed no difference in the degree of interference due to dual-task performance. In contrast, typical controls exhibited far more interference in SRT. ²⁸ These experiments suggest that successful SRT performance cannot be seen solely as a subset of those processes subserving CRT. Instead they suggest that additional processes are recruited when typical controls are engaged in SRT performance. The foregoing information processing analyses of SRT and CRT performance provides a framework in which slowed performance in patients with depression can be interpreted. Impaired attention and/or executive dysfunction can both lead to slowed performance.

So far, we have reviewed the issue of cognitive domain measurement with particular reference to the assessment of cognition in patients with depression. We have suggested that the link between tests and domains is often a “soft” association. We have also queried whether psychomotor speed should be considered as a cognitive domain and offered the view that it is better considered as an outcome measure, poor performance on which can be due to compromised cognitive skills, such as attention. We have also briefly considered a methodology for deciding under what circumstances empirical evidence supports the notion of a standalone domain, ie, dissociation and double dissociation. The evidence considered in the context of dissociation so far has been from the neuropsychology literature, which is typically based on case studies of patients with brain lesions. However, it is also possible to dissociate cognitive functions pharmacologically. Here we refer to circumstances in which a pharmacological intervention selectively impacts a specific domain of function. To take the earlier cited example of SRT and CRT performance on patients with Parkinson’s disease, some investigators have found that levodopa therapy selectively improved CRT performance, though this finding is not consistently observed (see Jahanshahi et al ²⁹ ). A recent feature of the literature on pharmacological treatment benefits in patients with depression has been positive effects of treatment on cognition. For example, a study of duloxetine showed a positive treatment effect on RAVLT performance, but not on Digit Symbol Substitution test (DSST) in patients with MDD, whereas treatment with vortioxetine benefitted performance on both tasks. ³⁰

Conclusions

In this article, we have briefly considered best practice for cognitive test selection when identifying the presence of deficits and when seeking to determine whether cognitive change has occurred. We discussed how cognitive domains have been conceptualized and we presented a framework for determining how the independence of these domains can be established using a dissociative methodology. We have also considered in detail the status of psychomotor speed as a distinct domain and suggested that slowed cognition might be due to deficits in other cognitive domains, especially attention. Standard neuropsychological assessment not unusually extends to around 4 hours of testing. However, in the context of screening, clinicians routinely specify the need for brevity when assessing cognition. ^{31 , 32} An argument is sometimes made for increasing the possibility of detecting effects through the selection of more than 1 test of each domain. However, adding to the length of an assessment is not without its costs, as fatigue, ennui, and loss of motivation have the potential to impact cognitive performance. To take an already considered example, the FOCUS study makes a useful contrast with the later CONNECT trial. ³³ The design of both studies was very similar, even to the extent that all but 1 of the cognition measures employed in FOCUS were also used in CONNECT. However, 2 further tests were added to the CONNECT assessment, 1 of which (the Groton Maze Learning Test) adds substantially the duration of the cognitive testing session. It is notable that in the FOCUS study all bar 1 cognitive test comparison yielded a significant effect of treatment, whereas in CONNECT positive treatment effects were limited to Part B of the Trail Making Test and to the DSST. Given these observations, it seems prudent to us to avoid overburdening study participants with lengthy assessment protocols.

When assessing cognitive change in the context of a busy study protocol limits opportunity, our standard practice has been to develop assessments that take no longer than 30 minutes to administer. Given these constraints, we would suggest that a core set of domains for assessment comprise measures of (i) attention, (ii) episodic memory, (iii) working memory, and (iv) aspects of executive function. This 4-domain recommendation is perhaps unsurprising in that it exhibits considerable overlap with the recommendations and descriptions described in the work reviewed in this article. While we do not believe that psychomotor speed represents a discrete cognitive domain, we subscribe entirely to the view that measures so described can be usefully employed in screening for the presence of cognitive dysfunction and measuring cognitive change. Therefore, in addition to selecting tests that index the 4 domains listed above, we see value in adding measures of processing speed. This was the logic that was applied to the selection of measures for the FOCUS study of vortioxetine. We have previously reported the effects of treatment by cognitive domain, but specified performance on the Digit Symbol Substitution Test as a separate analysis in recognition of the fact that intact performance on this task is reliant on a variety of cognitive skills. ³⁴ By not specifying modality-specific measures of learning and memory, we have departed from the view common to the many authors we have reviewed. We have instead suggested the inclusion of an episodic memory measure, though when test selection can be informed by knowledge of a therapeutic intervention’s specific effect on episodic memory, we would support modality-specific assessment. A further context for modality-specific assessment is in instances where sensory impairment would otherwise preclude or disadvantage the study participant.

A further issue for consideration is the assessment of executive function. This construct is variously defined, but common to most interpretations is the idea that to assess executive function, tests must index cognitive skills including flexibility (often referred to as the ability to “shift” or “switch”) and planning. Further elements of executive function are the capacity to inhibit a prepotent response and deal with novelty. Maze and trail tests are often considered to be measures of executive function, as are problem-solving paradigms such as Tower of Hanoi–type tasks. An area of overlap between Tower of Hanoi and maze navigation tests is the need to foresee the consequences of current decisions for future task completion, hence they can be described as tests of planning. Switching or shifting is a characteristic of tests such as Part B of the Trail Making Test, in which study participants are required to complete a trail through circles labeled with letters and numbers. Both classes of stimuli must be connected in ascending order, but a further test requirement is that the study participant must alternate between the 2 stimulus sets. Switching or shifting is also a characteristic of rule-ascertainment tasks, such as the Wisconsin Card Sorting Test and the CANTAB Intra-Extra Dimensional set-shifting test. However, it is not certain whether successful performance on trail completion and rule ascertainment tasks converge on a common cognitive mechanism, damage to which would impair performance on both classes of test.

Disclosures

Dr. Harrison reports personal fees from AbbVie, AlzeCure, Amgen, Anavex, Astra Zeneca, Avonex, Avraham, Axon Neuroscience, Axovant, Biogen Idec, Boehringer Ingelheim, Bracket, Catenion, Compass Pathways Ltd, CRF Health, DeNDRoN, Eisai, Eli Lilly, Enzymotec, ePharmaSolutions, Forum Pharma, G4X Discovery, GfHEU, Heptares, Janssen AI, Johnson & Johnson, Kaasa Health, Kyowa Hakko Kirin, Lundbeck, MedAvante, Merck, Mind Agilis, MyCognition, Neurim, Neurocog, Novartis, Nutricia, Orion Pharma, Pfizer, Pharmanet/i3, Prana Biotech, PriceSpective, Probiodrug, Prophase, Prostrakan, Regeneron, Reviva, Roche, Sanofi, Servier, Takeda, TransTech Pharma, and Velacor; and personal fees and other from Neurotrack, outside the submitted work. In addition, Dr. Harrison has a patent Cognition training system pending to MyCognition.