Introduction
The number of hospital admissions for older adults is growing as the population ages and the prevalence of cognitive problems increases (Li et al., Reference Li, Li, Lu, Li, Mo, Chen, Peng, Guo, Lin, Qiu, Yang, Liu and Xu2020). Although a defining feature of dementia, cognitive decline is common in many neurodegenerative disorders (e.g., Parkinson’s disease, motor neuron disease, Cui et al., 2015; Mihaescu et al., 2019) and may precede other core disease-specific diagnostic criteria (Fields, Reference Fields2017). This cognitive decline can affect a person’s independent functioning, leading to an increase in the demand for assessments of mental capacity (APA & ABA, 2008).
In many countries, there is a distinction between mental capacity, which is a clinical evaluation of a person’s ability to independently make an informed decision or perform a specific task, and competence, which is a legal determination by an administrative tribunal (also termed conservatorship/guardianship board or probate court) regarding a person’s ability to make their own decisions or perform activities (Darby & Dickerson, Reference Darby and Dickerson2017). Although comprehensive evidence-based capacity assessments are required to inform legal determinations about competence (APA & ABA, 2008), these assessments do not inevitably lead to a formal hearing or legal decision, with informal living-supports often trialled first (e.g., assistance with cleaning, shopping, and meals; McSwiggan et al., Reference McSwiggan, Meares and Porter2016). The legislation guiding administrative tribunals differs between countries and jurisdictions (Tsoh et al., Reference Tsoh, Peisah, Narumoto, Wongpakaran, Wongpakaran, O’Neill, Jiang, Ogano, Mimura, Kato and Chiu2015) but typically prioritizes a person’s independence and their continued involvement in decision-making (United Nation’s General Assembly, 2007). Indeed, the appointment of a surrogate decision-maker or guardian to act on a person’s behalf is the least preferred option when mental capacity is compromised (Davidson et al., Reference Davidson, Kelly, Macdonald, Rizzo, Lombard, Abogunrin, Clift-Matthews and Martin2015).
Capacity assessments are conducted by a variety of health professionals (e.g., neuropsychologists, geriatricians, medical practitioners, psychiatrists) and require considerable clinical expertise. Decision-making capacity (DMC; Kolva & Rosenfeld, Reference Kolva, Rosenfeld and Demakis2012) refers to a person’s ability to make independent and informed decisions, which is reliant on their ability to understand relevant information, different opinions and possible outcomes, and to express a clear and consistent choice (Appelbaum & Grisso, Reference Appelbaum and Grisso1988). DMC has many aspects, including the ability to make lifestyle (where to live, necessary supports), financial (manage financial affairs), testamentary (make/alter a will), sexual (consent to sexual relations), driving (safely operate a motor vehicle), medical (consent to/refuse treatment), and research (consent to research) decisions. Lifestyle DMC (LS-DMC), which is the focus of this study, includes decisions regarding independent living and self-care (Demakis, Reference Demakis2012).
LS-DMC assessments are very common in hospital settings (Brindle & Holmes, Reference Brindle and Holmes2005) and are often associated with longer and more complex admissions (Chen et al., Reference Chen, Finn, Homa, St. Onge and Caller2016; Miller et al., Reference Miller, Petrick, Wall and Arcona1999; Torke et al., Reference Torke, Sachs, Helft, Montz, Hui, Slaven and Callahan2014; see Figure 1 for flowchart). Clinical interviews are used to initially identify a specific lifestyle issue (e.g., provision of in-home supports or placement in residential care), the risks associated with not receiving supports, the persons’ preferences, and any existing supports. Clinicians often also administer cognitive screens at this stage to investigate whether cognitive impairment may be affecting a person’s independent living and decision-making. The Mini-Mental Status Examination (MMSE) is commonly used for this purpose (Pachet et al., Reference Pachet, Astner and Brown2010; Shibu et al., Reference Shibu, Rowley and Bartlett2020) and is often supplemented with the Frontal Assessment Battery (FAB) in order to assess the “executive” functions that underpin decision-making (Darby & Dickerson, Reference Darby and Dickerson2017). If both the interview and cognitive screens support the need for additional living-supports, but the patient rejects the recommended supports, a comprehensive LS-DMC assessment may be required to determine whether they lack or do not-lack LS-DMC. As seen in Figure 1, a LS-DMC assessment can result in several different outcomes, all of which are designed to facilitate patient discharge.
Figure 1. Stages involved in determining which older adult inpatients need a comprehensive assessment of lifestyle decision-making capacity and the possible outcomes from this process
Sorting tests are also often used in DMC assessments, with 51% to 75% of Australian psychologists using the Colour Form Sort for this purpose (Mullaly et al., Reference Mullaly, Kinsella, Berberovic, Cohen, Dedda, Froud, Leach and Neath2007). Sorting tests assess multiple cognitive domains (Schneider & McGrew, Reference Schneider, McGrew, Flanagan and McDonough2018), including “executive” functioning, and detect the cognitive decline caused by common neurodegenerative disorders (e.g., dementia; Foran et al., Reference Foran, Mathias and Bowden2020). Although not traditionally used during initial interviews to investigate the need for living-supports, sorting tests – and, more particularly, the QuickSort – may prove useful for this purpose. The QuickSort is a brief standardized sorting test (healthy older adults take <2 mins) in which stimuli are sorted by color/shape/number and the category underpinning the sort is identified. It includes multiple improvements to existing sorting tasks that enhance its usefulness (e.g., reduced administration & scoring time, early discontinuation for intact performance, larger range of impaired scores, freely available stimuli and A4 record form that provides instructions for administration and records all scores). The fact that the QuickSort has good inter-rater and test–retest reliability, has Australian norms for cognitively healthy older adults, and successfully predicts impairment on the MMSE and FAB, additionally supports its use as a brief cognitive screen (Foran et al., Reference Foran, Mathias and Bowden2021). Whether the QuickSort is useful when interviewing patients with questionable LS-DMC, however, has yet to be determined.
The present study was designed to validate the QuickSort for use during the initial clinical interview with inpatients who have questionable LS-DMC. Sensitivity and specificity values were used to determine which QuickSort cut-score (and score range/categories) differentiated between inpatients who lacked and did not-lack LS-DMC. Likelihood ratios (LRs) quantified how more- or less-likely inpatients were to lack LS-DMC if they had certain QuickSort scores and, when combined with the local prevalence of inpatients lacking LS-DMC, were used to calculate the probability that an inpatient in the current clinical setting lacked LS-DMC.
Method
Participants
Participants comprised 124 inpatients, aged 60 years and over, who were consecutive referrals to the Royal Adelaide Hospital (RAH) Neuropsychology Service for an assessment of LS-DMC. Inpatients were identified in one of two ways: (1) retrospectively, by auditing recent Neuropsychology Service records, hereafter referred to as Retrospective-inpatients (n = 48), and (2) prospectively, by recruiting new inpatients for a study investigating the QuickSort, hereafter referred to as Prospective-inpatients (n = 76). An additional three Prospective-inpatients had their requests for LS-DMC assessments retracted and six were eligible but declined to take part in the study, although unfortunately the reasons for declining were not recorded.
Measures
Patient characteristics were obtained from clinical interviews or medical records and included (i) demographic details (age, sex, education, nationality), (ii) relationship status (partnered/not partnered), (iii) living-supports when admitted to hospital (none, in-home supports provided by family/friends/community organizations, living in a residential facility/with family), (iv) history of a stroke or dementia and delirium (yes/no), (v) alcohol intake (daily or almost daily: yes/no), (vi) the presence of challenging behaviors (e.g., self-neglect, noncompliance with medications, placing themselves in dangerous situations, or verbal or physical abuse toward others: yes/no), and (vii) their scores on the MMSE (<24 considered impaired; Hancock & Larner, Reference Hancock and Larner2011) and FAB (<11 considered impaired; Kim et al., Reference Kim, Huh, Choe, Jeong, Park, Lee, Lee, Jhoo, Lee, Woo and Kim2010).
The QuickSort, which was only administered to the Prospective-inpatient group (n = 76), required participants to sort nine cards by color, shape and number within a maximum of six trials. The QuickSort manual, stimuli and psychometric properties are published elsewhere (Foran et al., Reference Foran, Mathias and Bowden2021). Total scores (range: 0–18) were calculated by summing: (1) a “Sorting” score, which aggregated the number of correct sorts (0 = correct without prompt; 1 = prompted, but correct; 2 = incorrect), sorting errors (repetition, set-loss, grouping or completion errors) and verbal prompts (range: 0–12); and (2) an “Explanation” score, which assessed the ability to verbalize the reason for the correct sort (0 = incorrect; 1 = concrete; 2 = correct, range: 0–6). Total scores <10 have previously been found to optimize sensitivity (82%) and specificity (78%) when trying to detecting impairment on either the MMSE, FAB, or both screens (Foran et al., Reference Foran, Mathias and Bowden2021).
The primary outcome for this study was whether inpatients lacked or did not-lack LS-DMC, which was determined after independent medical and neuropsychological assessments. Medical assessments were completed by general physicians, often with the assistance of psychiatrist and geriatricians, based on information obtained during a clinical interview. The latter included questions about mood and safety, cognitive screens (MMSE and FAB), which was supplemented by other reports (e.g., nursing or occupational therapy). Neuropsychological assessments involved: patient and informant interviews; comprehensive assessments of patients’ cognition (e.g., the Weschler Adult Intelligence and Memory Scales); and a review of functional and medical reports (including MMSE and FAB scores) and any prior neuropsychological assessments (to identify cognitive changes). Inpatients were classified as: (1) lacking LS-DMC, when the medical and neuropsychology teams both agreed that the inpatient did not have the capacity to make an informed and reasoned decision regarding a lifestyle matter, or (2) not-lacking LS-DMC, if one or both teams indicated that either (a) the inpatient had the capacity to make lifestyle decisions or (b) the available evidence was inconclusive. This dichotomous classification (lacking vs. not-lacking LS-DMC) aligns with international legislation, which states that people should be assumed to have capacity unless there is clear and convincing evidence to the contrary (Davidson et al., Reference Davidson, Kelly, Macdonald, Rizzo, Lombard, Abogunrin, Clift-Matthews and Martin2015).
A number of secondary outcomes were also examined (see Figure 1), namely the number of inpatients who: (1) underwent a competency hearing with the South Australian Civil and Administrative Tribunal; (2) had a surrogate decision-maker appointed after a tribunal hearing, or re-instated based on an existing tribunal determination with/without a formal hearing; (3) accepted the recommended living-supports; (4) had family/friends agree to provide additional living-supports; (5) received no additional living-supports, and (6) experienced another event that negated the need for a LS-DMC assessment outcome (e.g., death or other medical event that prevented hospital discharge). Whether living-supports were increased at discharge was also recorded, as was the final level of support (which combined previous supports with new supports). Finally, the length of hospital stay and the number of inpatients who were readmitted (within the local public health network) within 1 year of their initial admission were recorded.
Procedure
The RAH Human Research and Ethics Committee approved the inclusion of inpatients with questionable capacity if they provided written consent. A neuropsychologist or postgraduate assistant administered the QuickSort prior to the neuropsychological LS-DMC assessment and before accessing MMSE and FAB scores from medical records in order to blind the assessors. Medical and neuropsychology staff were not privy to patients’ QuickSort performance when determining capacity.
Data analysis
Data was analysed using the Statistical Package for the Social Sciences (IBM, 2017). Missing data were excluded list-wise and p < .05 determined statistical significance.
Data for the Retrospective-inpatient and Prospective-inpatient samples were initially combined (hereafter referred to as the “combined sample”) in order to characterize the inpatients who were referred to the Neuropsychology Service for LS-DMC assessments. The primary (LS-DMC: lacking or not-lacking) and secondary outcomes for the combined sample were also recorded, as was summary information for all inpatients who had an administrative tribunal hearing in order to determine how they differed from those who did not require a hearing.
The combined sample was also used to investigate the variables that were relevant to inpatients’ LS-DMC. Inpatients were classified into one of two groups, based on whether they lacked or did not-lack LS-DMC, after which t tests and chi-square analyses examined their comparability. Although the MMSE and FAB scores of these groups were expected to differ because this information was used when determining capacity, they provide useful benchmarks against which to determine whether the QuickSort (not part of the capacity assessment) better differentiated between those who lacked and did not-lack LS-DMC (Hedges’ g). A minimum sample size of 42 was required in each of the two groups (lacking/not-lacking LS-DMC) in order to detect a large difference (Cohen’s d = .80) with 95% power at p = .05 (Cohen, Reference Cohen1988).
A logistic regression then identified which variables best differentiated between those who lacked/did not-lack LS-DMC (dependent variable) in the combined sample, with inpatient demographics, relationship status, living-supports, history of dementia, stroke or delirium, alcohol use, challenging behaviors and the cognitive screens (MMSE, FAB, QuickSort) being the independent variables. The “forward” method was used, which enters the most significant independent variable first, then adds others based on their contribution to predicting the dependent variable. An additional logistic regression with simultaneous entry of all variables then determined the contribution that each variable made to the LS-DMC determination.
The remaining analyses investigated the QuickSort for informing LS-DMC in the Prospective-inpatient sample. A final logistic regression was used to identify the QuickSort (independent variable) cut-score that correctly classified the largest number of inpatients according to whether they lacked/did not-lack LS-DMC (dependent variable). Having statistically identified a cut-score for the QuickSort, the CAT-maker (Badenoch et al., Reference Badenoch, Sackett, Straus, Ball and Dawes2004) computed sensitivity, specificity and LRs. The sensitivity and specificity values for the recommended cut-scores on the MMSE (<24) and FAB (<11) were also calculated for comparative purposes (Prospective-inpatient sample). Sensitivity was of particular interest because the priority was to minimize the chance of missing someone who lacked capacity (Larner, Reference Larner and Larner2013).
Although useful, cut-scores can be misleading for small samples because any cases that do not conform to the general pattern (e.g., low QuickSort score for someone with intact capacity) disproportionately affect the sensitivity and specificity of specific scores. As an alternative, the number of inpatients who lacked/did not-lack LS-DMC for each QuickSort score were entered into the CAT-maker in order to identify scores that could be grouped (categories), with minimal or no changes to sensitivity and specificity. Multiple level LRs were then generated for each category by entering the number of inpatients who lacked/did not-lack capacity into the CAT-maker. Multiple LRs cannot be computed for scores or categories that have a frequency of zero, which occurred in the group that did not-lack LS-DMC (no one scored 0 or 1). A nominal low value of 0.4 was therefore used to compute multiple LRs (Straus et al., Reference Straus, Pattani and Veroniki2019), while also avoiding rounding errors (which occur with a value of 0.5). QuickSort scores that fell within a category that had a LR > 1 were more likely to occur in inpatients who lacked LS-DMC and scores within a category with a LR < 1 were more likely in inpatients who did not-lack capacity (Deeks & Altman, Reference Deeks and Altman2004). Clinically, scores with LRs >3 or <0.3 are considered particularly useful because they substantially change the likelihood that a patient has/doesn’t have a target condition, which in this study was LS-DMC (McGee, Reference McGee2016).
As with the QuickSort, the number of Prospective-inpatients who lacked/did not-lack LS-DMC for each MMSE score were entered into the CAT-maker in order to identify scores that could be grouped into an equivalent number of categories as the QuickSort, with minimal or no changes to sensitivity and specificity. These MMSE categories were expected to be more sensitive and specific than the QuickSort because the medical and neuropsychology staff used MMSE scores when determining whether an inpatient lacked/did not-lack LS-DMC. Screens were deemed to differ significantly if the QuickSort LRs for the low-, middle-, and high-score categories fell outside of the 95% confidence intervals for the corresponding MMSE LRs.
Finally, EBM modeling customized the QuickSort for investigating LS-DMC in older adults referred to the current Neuropsychology Service (Straus et al., Reference Straus, Pattani and Veroniki2019). This modeling calculated the post-test probability that inpatients with a given QuickSort score would lack LS-DMC after taking into consideration the client-base for that service (local prevalence or pre-test probability of patients lacking LS-DMC). The pre-test probability of patients lacking LS-DMC was estimated from the Retrospective-inpatients, who were not administered the QuickSort, having established that they were demographically comparable to the Prospective-inpatients, who were administered the QuickSort. The CAT-maker used the LRs for each QuickSort score category and the pre-test probability to calculate the post-test probability of an inpatient lacking LS-DMC. This modeling was additionally repeated using 10% and 25% pre-test probabilities to make it applicable to other clinical settings with different prevalence rates, such as community medical practices and medical inpatient units.
Results
Inpatients referred for LS-DMC assessments
Inpatient characteristics
Table 1 provides summary information for all inpatients (combined sample) who underwent a LS-DMC assessment. On average, inpatients were almost 76 years of age and had completed some high school education (M = 10.6, SD = 3.2). Most were male (62%) and Australian (70%), and relatively few were partnered (22%). On admission, many had no living-supports (42%) or received some in-home support from friends, family and/or community organizations (42%), with many fewer living in a residential care facility or with family (16%). A history of dementia or stroke was documented in less than half of all inpatients (40%) and 20% had experienced a delirium. Almost one in four drank alcohol daily or almost daily (26%) and many exhibited challenging behaviors (81%). On average, inpatients fell within the impaired range on the MMSE (<24; M = 23.1, SD = 4.3) and just above the cut-score for impairment on the FAB (<11; M = 11.6, SD = 3.4). The average QuickSort score in the Prospective-inpatient sample was <10 (M = 6.5, SD = 5.7), which has previously been found to occur in as few as 4% of cognitively healthy older adults and has also proved to be the optimal cut-score for detecting impairment on the MMSE and FAB (Foran et al., Reference Foran, Mathias and Bowden2021).
Table 1. Summary characteristics for inpatients referred to the neuropsychology service for an assessment of lifestyle decision-making capacity (LS-DMC)
LS-DMC = lifestyle decision-making capacity.
Primary and secondary outcomes
Of the 124 inpatients who underwent LS-DMC assessment, 63% were deemed to lack capacity and 37% did not-lack capacity (see Table 1, Primary & secondary outcomes). In total, 40% (n = 49) of inpatients had an administrative tribunal hearing. Comparable numbers of inpatients had a surrogate decision-maker appointed/re-instated (35%) or accepted the recommended living-supports (37%). Many fewer had family and friends who could provide new informal supports (<8%), had no changes to their supports (<12%), or experienced another outcome that negated the LS-DMC assessment (8%). Of note, most inpatients (86%) had their living-supports increased from their pre-admission levels following their LS-DMC assessment, with the majority living in residential care or with family (62%), some having in-home supports organized (32%) and many fewer not requiring support (6%). On average, inpatients were hospitalized for approximately 49 days (SD = 37.8), with 62% readmitted to hospital within one year of their discharge.
Inpatients who had an administrative tribunal hearing
Table 2 provides the characteristics and outcomes for the 40% of inpatients who had (n = 49) and 60% who did not have (n = 75) an administrative tribunal hearing. Prospectively recruited inpatients were less likely to have a tribunal hearing, however this coincided with increased staffing, which meant that less-urgent higher-functioning inpatients, who may have been more likely to accept the recommended supports, were assessed sooner. Of note, inpatients who had and did not have a hearing were comparable in terms of their: demographic characteristics, relationship status, living-supports, medical history, alcohol use and challenging behaviors. However, inpatients were more likely to have an administrative tribunal hearing if they had low MMSE and FAB scores and lacked LS-DMC (88%). As expected, inpatients who had a hearing were more likely to have a surrogate decision-maker appointed (73%) and less likely to accept the recommended living-supports (12%). They also had significantly longer hospital stays than those who did not require a hearing; however their 1-year readmission rates were comparable.
Table 2. Inpatient characteristics and outcomes (combined sample) for those who did/did not have an administrative tribunal hearing
LS-DMC = lifestyle decision-making capacity.
aOnly inpatients aged over 59 years of age were investigated in this study.
bFormal education in years.
Comparison between inpatients who lacked/did not-lack LS-DMC
Inpatient characteristics
Table 1 also provides information for inpatients who were deemed to lack (n = 78) and not-lack (n = 46) LS-DMC. Both groups were comparable in terms of their demographic characteristics, relationship status, living-supports, medical history, and alcohol use. However, inpatients who lacked LS-DMC were significantly more likely to exhibit challenging behaviors and have poorer cognition (MMSE, FAB, QuickSort). Of the three cognitive screens, the QuickSort showed the largest group difference (Hedges’ g = .65), followed by the MMSE (g = .43) and FAB (g = 0.43). This was unexpected because, unlike the QuickSort, MMSE and FAB scores were used by clinicians when assessing LS-DMC.
Secondary outcomes
Table 1 additionally reveals that the secondary outcomes for inpatients differed according to whether they were deemed to lack or not-lack LS-DMC. Not surprisingly, inpatients who lacked LS-DMC were significantly more likely to have an administrative tribunal hearing (55%) and a surrogate decision-maker appointed or re-instated (51%), with very few being discharged with new informal supports from family or friends (3%) or with no additional living-supports (5%). The majority of inpatients who lacked LS-DMC (92%) had their living-supports increased from pre-admission levels and were more likely to be discharged to live in residential care facilities or with family (71%), and less likely to receive in-home supports (27%) or have no supports (2%). The length of hospitalization and 1-year readmission rates did not differ between those who lacked and did not-lack LS-DMC, although there was a trend toward longer hospitalizations for those who lacked capacity (p = .06).
Variables related to inpatient LS-DMC
A logistic regression revealed that the QuickSort was the variable that best differentiated between those who lacked and did not-lack LS-DMC (B = −0.11, SE = .05, β = −0.90, Wald = 5.19, p = .02). None of the remaining independent variables (demographic, relationship, living-supports, medical history, alcohol use, challenging behaviors, MMSE, FAB) made a significant contribution, once the QuickSort entered the model (see Table 3). Surprisingly, the MMSE and FAB were not significant predictors of capacity (although the MMSE approached significance), despite being used by clinicians when determining LS-DMC.
Table 3. Logistic regression examining whether demographic information, living-supports, medical history, alcohol use, challenging behaviors, and cognition influenced the lifestyle decision-making capacity of retrospective and prospective-inpatients
B = unstandardized beta; SE = standard error; β = standardized beta; Wald = Wald test; p = p-value; aonly inpatients aged over 59 years of age were investigated in this study; bpartnered compared to being single, divorced, separated or widowed; cthere were three levels of supports with living at admission, no supports, in-home supports, and in-facility supports or living with family; dchallenging behaviors could be to self, such as self-neglect, or verbal or physical abuse of others; MMSE = Mini-Mental Status Examination; FAB = Frontal Assessment Battery.
QuickSort cut-scores and categories to inform LS-DMC
The Prospective-inpatients who lacked LS-DMC performed quite differently on the QuickSort than those who did not-lack LS-DMC, with 86% of inpatients who lacked LS-DMC having scores <10 (see Figure 2). A logistic regression, which examined what QuickSort cut-score most accurately differentiated between inpatients who lacked and did not-lack LS-DMC, found that Total scores <13 correctly classified 44 of the 50 inpatients who lacked LS-DMC (88% sensitivity) and scores ≥13 correctly classified 10 out of the 26 inpatients who did not-lack capacity (38% specificity). Of interest, the QuickSort cut-score <13 had better sensitivity than the recommended cut-scores for the MMSE (<24; 50% sensitivity: detected 22 of 44 Prospective-inpatients who lacked LS-DMC) and FAB (<11; 39% sensitivity: detected 17 of the 44 Prospective-inpatients who lacked capacity). However, the MMSE and FAB had better specificity (MMSE: 67% specificity; identified 16 of 21 Prospective-inpatients who did not-lack capacity; FAB: 73% specificity; identified 16 of 22 Prospective-inpatients who did not-lack capacity).
Figure 2. Histogram showing the distribution of QuickSort Total scores for Prospective-inpatients who lacked and did not-lack LS-DMC.
Next, the QuickSort scores for the Prospective-inpatient sample were grouped into three categories to improve the accuracy of interpretation and multiple LRs were then computed for each category (Table 4). Two score categories were most informative: one defined by the lowest two scores (0–1), which increased the likelihood that an inpatient lacked LS-DMC by a factor of 65.26 (95%CI: 2.91–1463.90), and one defined by the highest six scores (13–18), which reduced the likelihood that an inpatient lacked LS-DMC by a factor 0.32 (95%CI: 0.18–0.57). Scores 2 to 12 were not clinically informative because equivalent numbers of inpatients who lacked (62%) and did not-lack (62%) LS-DMC scored within this range. Although the 95% CIs were extremely large – probably due to the small sample – it is important to note that the lower CIs approached the target LRs of >3 (scores < 2) and <0.3 (scores ≥ 13), both of which substantially change the likelihood of a patient lacking and not-lacking LS-DMC (McGee, Reference McGee2016). The upper CIs far exceed what is considered clinically useful.
Table 4. Total score categories for the QuickSort when predicting inpatients (prospective inpatient sample) who lacked/did not-lack lifestyle decision-making capacity
LR = likelihood ratio; 95% CI = 95 percent confidence interval; apre-test probability of inpatients lacking LS-DMC who were referred to the Neuropsychology service was estimated to be 58%, based on the Retrospective-inpatients group; b25% hypothetical prevalence of LS-DMC; c10% hypothetical prevalence of LS-DMC; dno inpatient who did not-lack LS-DMC scored 0 or 1 on the QuickSort; therefore, a nominal value of 0.4 was used to compute the multilevel likelihood ratios.
The MMSE scores for the Prospective-inpatient sample were also grouped into three categories to enable a direct comparison with the findings from the QuickSort. However, these findings should be treated cautiously because the medical and neuropsychology teams used MMSE scores when determining LS-DMC, and the sensitivity and specificity of the MMSE may have been optimized with more score categories. The lowest MMSE score category (scores: 0–17) increased the likelihood that an inpatient lacked LS-DMC by a factor of 5.42 (95%CI: 0.22–134.72), which was less sensitive than the lowest score category on the QuickSort, but the difference was not significant due to the large CIs (see online Supplementary Table A for details). The highest MMSE score category (scores: 28–30) reduced the likelihood that an inpatient lacked LS-DMC by a factor 0.23 (95%CI: 0.07–0.82), which was more specific than the highest category on the QuickSort, but the difference was not significant. Like the QuickSort, the middle category for the MMSE (scores: 18–27) was not clinically informative, although more inpatients who lacked LS-DMC (84%) achieved these scores than those who did not-lack capacity (70%).
Customization of the QuickSort for specific clinical services
Modeling was undertaken to improve the accuracy of the QuickSort for use in the current Neuropsychology Service by additionally using the LRs for the three QuickSort score categories and the pre-test probability that an inpatient referred to this service would lack LS-DMC. The latter probability was estimated from the Retrospective-inpatient sample because it was demographically comparable to the Prospective-inpatient sample but was not included in the QuickSort analyses (see Table 5). Of the 48 Prospective-inpatients, 28 lacked LS-DMC, resulting in a pre-test probability of 58%.
Table 5. Demographic differences between the retrospectively and prospectively recruited inpatients
n = Number of participants; M = mean; SD = standard deviation; t = t test; χ 2 = chi square; p = p-value. aonly inpatients aged over 59 years of age were investigated in this study; bformal education in years.
Modeling revealed that the lowest (<2) and highest (≥13) QuickSort categories were most useful clinically, because they substantially changed the probability of an inpatient lacking LS-DMC (from 26% to 99% for QuickSort scores <2 and from 12% to 30% for scores ≥13; see Table 4). Figure 3 provides a nomogram to illustrate how the QuickSort can be interpreted by the Neuropsychology Service using the three score categories. The first example (solid line) depicts inpatients scoring within the lowest category (<2), which increases the likelihood of them lacking LS-DMC by a factor of 65.26 (95% CI: 2.91–1463.90), resulting in a 99% chance (post-test probability) that they will lack LS-DMC. The second example (dashed line) depicts inpatients scoring in the middle category (2–12), which is associated with a LR of 1.01 (95% CI: 0.81–1.25) and does not change the probability of them lacking LS-DMC from the pre-test level of 58%. The third example (dotted line) depicts inpatients scoring within the highest category (≥13), which reduces the likelihood of them lacking LS-DMC by a factor of 0.32 (95% CI: 0.18–0.57), resulting in a 30% probability that they will lack capacity.
Figure 3. Nomogram customizing the QuickSort for screening lifestyle decision-making capacity in the RAH Neuropsychology service.
Clinical settings vary in terms of their patient profiles (age, education, referral problem etc) and, consequently, the prevalence of patients who lack LS-DMC. In generalist settings, for example, many fewer patients are likely to lack LS-DMC. Table 4 therefore additionally models the QuickSort for other clinical settings where the pre-test probability a person lacks LS-DMC is either 25% or 10%: In settings where the pre-test probability is 25%, the post-test probability that a person lacks capacity increases to 96% for QuickSort scores <2 and decreases to 9% for scores ≥13. In settings where the pre-test probability is 10%, the post-test probability increases to 25% for QuickSort scores <2 and decreases to 3% for scores ≥13. Once again, there was no change in the probability of inpatients lacking or not-lacking LS-DMC if they scored within the middle category of the QuickSort (2–12). Modeling of the MMSE for clinical settings with 58% (current study setting), 25% and 10% pre-test probability of inpatients lacking LS-DMC, although only tentative, indicated the lowest category (0–17) produced inferior post-test probabilities compared to the lowest QuickSort score category (0–1; see online Supplementary Table A).
Discussion
LS-DMC assessments are increasing in hospital settings as the population ages and more people suffer from cognitive decline, which can impact on their ability to live independently and make lifestyle decisions (Usher & Stapleton, Reference Usher and Stapleton2019). In busy settings, the initial clinical interviews that explore patients’ need for living-supports often also include cognitive screens to investigate some of the abilities that underpin DMC (Pachet et al., Reference Pachet, Astner and Brown2010). This study investigated whether the QuickSort can provide efficient and accurate information regarding patients’ LS-DMC during the initial clinical interview.
In the current setting, 63% of the hospital inpatients who were referred to the Neuropsychology Service for a LS-DMC assessment were deemed to lack LS-DMC. Administrative tribunal hearings were often avoided because inpatients accepted the recommended supports or the appointment/reinstatement of a surrogate decision-maker. Accordingly, living-supports increased for many, with almost two-thirds discharged to live in residential care or with family. Community living-supports and residential care placements are notoriously difficult to arrange (Bai et al., Reference Bai, Dai, Srivastava, Smith and Gill2019), which may have extended inpatients’ hospital stays, especially for those who had a tribunal hearing and/or needed significant additional living-supports. In general, inpatients referred for LS-DMC assessment are hospitalized for up to five times longer than other geriatric patients (Basic & Khoo, Reference Basic and Khoo2015), increasing their risk of hospital-acquired infections and death (Bai et al., Reference Bai, Dai, Srivastava, Smith and Gill2019). These risks emphasize the importance of conducting clinical interviews to ascertain patients’ need for additional living-supports early in their admission to assist in care-planning and to avoid unnecessary delays in accessing resources. Not only are patients with questionable LS-DMC at risk of having extended hospital admissions, but they were also twice as likely to be readmitted to hospital than older adult trauma inpatients (Crijns et al., Reference Crijns, Caton, Teunis, Davis, McWilliam-Ross, Ring and Sanchez2018), which suggests they are a high-risk group who would benefit from continued monitoring.
This study found that it is worthwhile administering cognitive screens when initially interviewing patients with questionable LS-DMC because they better differentiated between those who lacked/did not-lack LS-DMC than a range of other patient characteristics (including demographic, relationship status, living-supports at admission, history of stroke, dementia, delirium, alcohol use, and challenging behaviors). Sensitivity is important when trying to avoid missing inpatients who lack LS-DMC (Larner, Reference Larner and Larner2013) and, of the three cognitive screens, the QuickSort had better sensitivity than the MMSE and FAB using a cut-score of <13 (88% sensitivity, 38% specificity). However, inpatients rarely scored within the middle range of the QuickSort and, importantly, equivalent numbers lacked and did not-lack LS-DMC. Greater certainty was associated with low (<2) and high (≥13) QuickSort scores, which increased or reduced the likelihood that a person lacked LS-DMC by a factor of 65.26 and 0.32, respectively. These categories also proved useful for detecting cognitive decline, with psychologically and cognitively-healthy older adults rarely scoring <2 and most (98%) scoring ≥13 (Foran et al., Reference Foran, Mathias and Bowden2021). As is the case for many cognitive screens, it is therefore recommended that score categories be used to improve the accuracy with which the QuickSort is interpreted. This is because scores in the mid-range, which fail to discriminate between the two clinical groups (e.g., those lacking/not-lacking LS-DMC), are more readily identified (Pachet et al., Reference Pachet, Astner and Brown2010; Bowden & Loring., Reference Bowden and Loring2009).
Finally, modeling enhanced the usefulness of the QuickSort by considering the local prevalence of inpatients lacking LS-DMC (which can be estimated from local or similar service audits) together with the LR for a patient’s QuickSort score (current LRs are suitable if patients are demographically comparable to the Prospective-inpatient group). In the current setting, inpatients had to show some decline in their functioning in order to be referred to the Neuropsychology Service for a LS-DMC assessment. Consequently, the prevalence of them lacking LS-DMC was high (pre-test: 58%), but this increased to 99% (post-test) if their QuickSort scores were <2 and remained at 30% (post-test) if they scored ≥13.
One of the main limitations of this study relates to the fact that LS-DMC is not solely dependent on cognition. For example, the supports that are available to patients, and their risk of not accepting the recommended supports, are also considered when evaluating LS-DMC. In addition, mood/depression was not objectively assessed in this study but may have influenced inpatients’ willingness to accept living-supports and the determination of their LS-DMC. Brain damage affecting specific regions may also influence individuals’ DMC (see Damasio’s somatic marker hypothesis; Damasio, 1998; Dunn et al., Reference Dunn, Dalgleish and Lawrence2006) but was not investigated in this study. These factors may help to explain why some patients who performed adequately on the QuickSort lacked capacity, and vice versa, which impacted on the LRs for a few individual scores. Categories were therefore created to highlight a more general pattern in the LRs, whereby the likelihood of lacking LS-DMC increased with low QuickSort scores and reduced with high scores. Score categories may not have been required if a larger sample was examined because the impact of patients who did not conform to the general pattern would be counteracted by a greater number of patients who did. Larger samples could be achieved by collecting data over-time using electronic health records and routinely using the QuickSort when interviewing patients with questionable LS-DMC.
In conclusion, the QuickSort is freely accessible and provides a more time-efficient and sensitive source of information regarding an inpatient’s LS-DMC, than the MMSE or FAB, making it a viable alternative cognitive screen in settings where clinical resources are limited. The likelihood that a patient lacked LS-DMC increased in a clinically meaningful way when QuickSort scores were <2 and reduced substantially when they were ≥13. Moreover, the accuracy of detecting those lacking capacity increased when modeling additionally took into account the local prevalence of older adults lacking LS-DMC.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1355617722000479
Funding statement
This work was supported by a Royal Adelaide Hospital Grant (grant number: 7425) and A M Foran received an Australian Government Research Training Stipend.
Conflicts of interest
None.
