Hostname: page-component-586b7cd67f-rdxmf Total loading time: 0 Render date: 2024-11-24T09:07:58.708Z Has data issue: false hasContentIssue false
  • English
  • Français

The Other Side of the Bell Curve: Multivariate Base Rates of High Scores on the Delis-Kaplan Executive Function System

Published online by Cambridge University Press:  15 November 2019

Justin E. Karr Open the ORCID record for Justin E. Karr [Opens in a new window] ,
Mauricio A. Garcia-Barrera ,
James A. Holdnack  and
Grant L. Iverson
Justin E. Karr*
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA02215, USA Department of Physical Medicine and Rehabilitation, Harvard Medical School, Boston, MA02129, USA Spaulding Rehabilitation Hospital, Spaulding Research Institute, Charlestown, MA02129, USA Home Base, A Red Sox Foundation and Massachusetts General Hospital Program, Boston, MA02129, USA
Mauricio A. Garcia-Barrera
Affiliation:
Department of Psychology, University of Victoria, Victoria, British Columbia V8P 5C2, Canada
James A. Holdnack
Affiliation:
Research and Statistics Consultant, Bear, DE 19701, USA
Grant L. Iverson
Affiliation:
Department of Physical Medicine and Rehabilitation, Harvard Medical School, Boston, MA02129, USA Spaulding Rehabilitation Hospital, Spaulding Research Institute, Charlestown, MA02129, USA Home Base, A Red Sox Foundation and Massachusetts General Hospital Program, Boston, MA02129, USA
*
*Correspondence and reprint requests to: Justin E. Karr, 79/96 Thirteenth Street, Charlestown Navy Yard, Charlestown, MA 02129, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

Previous researchers have examined the frequency at which healthy participants obtain one or more low scores on neuropsychological test batteries, proposing five psychometric principles of multivariate base rates: (a) low scores are common, with their frequency contingent on (b) the low score cutoff used, (c) the number of tests administered/interpreted, and (d) the demographic characteristics and (e) intelligence of participants. The current study explored whether these principles applied to high scores as well, using the Delis-Kaplan Executive Function System (D-KEFS).

Method:

Multivariate base rates of high scores (≥75th, ≥84th, ≥91st, ≥95th, and ≥98th percentiles) were derived for a three-test, four-test, and full D-KEFS battery, using the adult portion of the normative sample (aged 16–89 years; N = 1050) stratified by education and intelligence. The full D-KEFS battery provides 16 total achievement scores (primary indicators of executive function).

Results:

High scores occurred commonly for all batteries. For the three-test battery, 24.1% and 12.4% had 1 or more scores ≥95th percentile and ≥98th percentile, respectively. High scores occurred more often for longer batteries: 61.6%, 72.9%, and 87.8% obtained 1 or more scores ≥84th percentile for the three-test, four-test, and full batteries, respectively. The frequency of high scores increased with more education and higher intelligence.

Conclusions:

The principles of multivariate base rates also applied to high D-KEFS scores: high scores were common and contingent on the cutoff used, number of tests administered/interpreted, and education/intelligence of examinees. Base rates of high scores may help clinicians identify true cognitive strengths and detect cognitive deficits in high functioning people.

Keywords

Executive functionNeuropsychological testsAssessmentNorms/normative studiesPsychometricsBase rates
Type
Regular Research
Information
Journal of the International Neuropsychological Society , Volume 26 , Issue 4 , April 2020 , pp. 382 - 393
Copyright
Copyright © INS. Published by Cambridge University Press, 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Baron, I.S. (2004). Delis-Kaplan Executive Function System. Child Neuropsychology, 10(2), 147–152. doi: 10.1080/09297040490911140CrossRefGoogle Scholar
Binder, L.M., Iverson, G.L., & Brooks, B.L. (2009). To err is human: “Abnormal” neuropsychological scores and variability are common in healthy adults. Archives of Clinical Neuropsychology, 24(1), 31–46. doi: 10.1093/arclin/acn001CrossRefGoogle Scholar
Brooks, B.L., Holdnack, J.A., & Iverson, G.L. (2011). Advanced clinical interpretation of the WAIS-IV and WMS-IV: Prevalence of low scores varies by level of intelligence and years of education. Assessment, 18(2), 156–167. doi: 10.1177/1073191110385316CrossRefGoogle ScholarPubMed
Brooks, B.L., Iverson, G.L., Feldman, H.H., & Holdnack, J.A. (2009a). Minimizing misdiagnosis: Psychometric criteria for possible or probable memory impairment. Dementia and Geriatric Cognitive Disorders, 27(5), 439–450. doi: 10.1159/000215390CrossRefGoogle ScholarPubMed
Brooks, B.L., Iverson, G.L., & Holdnack, J.A. (2013). Understanding and using multivariate base rates with the WAIS-IV/WMS-IV. In Holdnack, J.A., Drozdick, L.W., Weiss, L.G., & Iverson, G.L. (Eds.), WAIS-IV, WMS-IV, and ACS: Advanced Clinical Interpretation (pp. 75–102). San Diego, CA: Elsevier Science. doi: 10.1016/B978-0-12-386934-0.00002-XCrossRefGoogle Scholar
Brooks, B.L., Iverson, G.L., Holdnack, J.A., & Feldman, H.H. (2008). Potential for misclassification of mild cognitive impairment: A study of memory scores on the Wechsler Memory Scale-III in healthy older adults. Journal of the International Neuropsychological Society, 14(3), 463–478. doi: 10.1017/S1355617708080521CrossRefGoogle ScholarPubMed
Brooks, B.L., Iverson, G.L., Lanting, S.C., Horton, A.M., & Reynolds, C.R. (2012). Improving test interpretation for detecting executive dysfunction in adults and older adults: Prevalence of low scores on the Test of Verbal Conceptualization and Fluency. Applied Neuropsychology: Adult, 19(1), 61–70. doi: 10.1080/09084282.2012.651951CrossRefGoogle ScholarPubMed
Brooks, B.L., Iverson, G.L., & White, T. (2007). Substantial risk of “Accidental MCI” in healthy older adults: Base rates of low memory scores in neuropsychological assessment. Journal of the International Neuropsychological Society, 13(3), 490–500. doi: 10.1017/S1355617707070531CrossRefGoogle ScholarPubMed
Brooks, B.L., Iverson, G.L., & White, T. (2009b). Advanced interpretation of the neuropsychological assessment battery with older adults: Base rate analyses, discrepancy scores, and interpreting change. Archives of Clinical Neuropsychology, 24(7), 647–657. doi: 10.1093/arclin/acp061CrossRefGoogle ScholarPubMed
Brooks, B.L., Sherman, E.M.S., & Iverson, G.L. (2010). Healthy children get low scores too: Prevalence of low scores on the NEPSY-II in preschoolers, children, and adolescents. Archives of Clinical Neuropsychology, 25(3), 182–190. doi: 10.1093/arclin/acq005CrossRefGoogle ScholarPubMed
Chodosh, J., Reuben, D.B., Albert, M.S., & Seeman, T.E. (2002). Predicting cognitive impairment in high-functioning community-dwelling older persons: MacArthur studies of successful aging. Journal of the American Geriatrics Society, 50(6), 1051–1060. doi: 10.1046/j.1532-5415.2002.50260.xCrossRefGoogle ScholarPubMed
Cook, N.E., Karr, J.E., Brooks, B.L., Garcia-Barrera, M.A., Holdnack, J.A., & Iverson, G.L. (2018). Multivariate base rates for the assessment of executive functioning among children and adolescents. Child Neuropsychology, 25(6), 836–858. doi: 10.1080/09297049.2018.1543389CrossRefGoogle ScholarPubMed
Crawford, J.R., Garthwaite, P.H., & Gault, C.B. (2007). Estimating the percentage of the population with abnormally low scores (or abnormally large score differences) on standardized neuropsychological test batteries: A generic method with applications. Neuropsychology, 21(4), 419–430. doi: 10.1037/0894-4105.21.4.419CrossRefGoogle ScholarPubMed
Crawford, J.R., Garthwaite, P.H., Sutherland, D., & Borland, N. (2011). Some supplementary methods for the analysis of the Delis-Kaplan Executive Function System. Psychological Assessment, 23(4), 888–898. doi: 10.1037/a0023712CrossRefGoogle ScholarPubMed
Davis, F.B. (1959). Interpretation of differences among averages and individual test scores. Journal of Educational Psychology, 50(4), 162–170. doi: 10.1037/h0044024CrossRefGoogle Scholar
Deary, I.J., Starr, J.M., & MacLennan, W.J. (1998). Is age kinder to the initially more able?: Differential ageing of a verbal ability in the healthy old people in Edinburgh study. Intelligence, 26(4), 357–375. doi: 10.1016/S0160-2896(99)00005-7CrossRefGoogle Scholar
Delis, D.C., Kaplan, E., & Kramer, J.H. (2001). The Delis-Kaplan Executive Function System: Technical manual. San Antonio, TX: The Psychological Corporation.Google Scholar
Heled, E., Hoofien, D., Margalit, D., Natovich, R., & Agranov, E. (2012). The Delis-Kaplan Executive Function System sorting test as an evaluative tool for executive functions after severe traumatic brain injury: A comparative study. Journal of Clinical and Experimental Neuropsychology, 34(2), 151–159. doi: 10.1080/13803395.2011.625351CrossRefGoogle ScholarPubMed
Holdnack, J.A., Tulsky, D.S., Brooks, B.L., Slotkin, J., Gershon, R., Heinemann, A.W., & Iverson, G.L. (2017). Interpreting patterns of low scores on the NIH toolbox cognition battery. Archives of Clinical Neuropsychology, 32(5), 574–584. doi: 10.1093/arclin/acx032CrossRefGoogle ScholarPubMed
Homack, S., Lee, D., & Riccio, C.A. (2005). Test review: Delis-Kaplan executive function system. Journal of Clinical and Experimental Neuropsychology, 27(5), 599–609. doi: 10.1080/13803390490918444CrossRefGoogle ScholarPubMed
Huey, E.D., Goveia, E.N., Paviol, S., Pardini, M., Krueger, F., Zamboni, G., Tierney, M C., Wassermann, E.M., & Grafman, J. (2009). Executive dysfunction in frontotemporal dementia and corticobasal syndrome. Neurology, 72(5), 453–459. doi: 10.1212/01.wnl.0000341781.39164.26CrossRefGoogle ScholarPubMed
Ivins, B.J., Lange, R.T., Cole, W.R., Kane, R., Schwab, K.A., & Iverson, G.L. (2015). Using base rates of low scores to interpret the ANAM4 TBI-MIL battery following mild traumatic brain injury. Archives of Clinical Neuropsychology, 30(1), 26–38. doi: 10.1093/arclin/acu072CrossRefGoogle ScholarPubMed
Karr, J.E., Garcia-Barrera, M.A., Holdnack, J.A., & Iverson, G.L. (2017). Using multivariate base rates to interpret low scores on an abbreviated battery of the Delis-Kaplan Executive Function System. Archives of Clinical Neuropsychology, 32(3), 297–305. doi: 10.1093/arclin/acw105CrossRefGoogle ScholarPubMed
Karr, J.E., Garcia-Barrera, M.A., Holdnack, J.A., & Iverson, G.L. (2018). Advanced clinical interpretation of the Delis-Kaplan Executive Function System: Multivariate base rates of low scores. The Clinical Neuropsychologist. doi: 10.1080/13854046.2017.1334828CrossRefGoogle Scholar
McKinlay, A., Grace, R.C., Dalrymple-Alford, J.C., & Roger, D. (2010). Characteristics of executive function impairment in Parkinsons disease patients without dementia. Journal of the International Neuropsychological Society, 16(2), 268–277. doi: 10.1017/S1355617709991299CrossRefGoogle ScholarPubMed
Mistridis, P., Egli, S.C., Iverson, G.L., Berres, M., Willmes, K., Welsh-Bohmer, K.A., & Monsch, A.U. (2015). Considering the base rates of low performance in cognitively healthy older adults improves the accuracy to identify neurocognitive impairment with the Consortium to Establish a Registry for Alzheimer’s Disease-Neuropsychological Assessment Battery. European Archives of Psychiatry and Clinical Neuroscience, 265(5), 407–417. doi: 10.1007/s00406-014-0571-zCrossRefGoogle Scholar
Oakes, H., Lovejoy, D., Tartar, S., & Holdnack, J.A. (2013). Understanding index and subtest scatter in healthy adults. In Holdnack, J.A., Drozdick, L.W., Weiss, L.G., & Iverson, G.L. (Eds.), WAIS-IV, WMS-IV, and ACS: Advanced clinical interpretation (pp. 103–169). San Diego, CA: Elsevier Science. doi: 10.1016/B978-0-12-386934-0.00003-1CrossRefGoogle Scholar
Palmer, B.W., Boone, K.B., Lesser, I.M., & Wohl, M.A. (1998). Base rates of “impaired” neuropsychological test performance among healthy older adults. Archives of Clinical Neuropsychology, 13(6), 503–511. doi: 10.1016/S0887-6177(97)00037-1Google ScholarPubMed
Rabin, L.A., Paolillo, E., & Barr, W.B. (2016). Stability in test-usage practices of clinical neuropsychologists in the United States and Canada over a 10-Year Period: A follow-up survey of INS and NAN members. Archives of Clinical Neuropsychology, 31(3), 206–230. doi: 10.1093/arclin/acw007CrossRefGoogle Scholar
Scarmeas, N., & Stern, Y. (2004). Cognitive reserve: Implications for diagnosis and prevention of Alzheimer’s disease. Current Neurology and Neuroscience Reports, 4(5), 374–380. doi: 10.1007/s11910-004-0084-7CrossRefGoogle ScholarPubMed
Schretlen, D.J., Testas, S.M., Winicki, J.M., Pearlson, G.D., & Gordon, B. (2008). Frequency and bases of abnormal performance by healthy adults on neuropsychological testing. Journal of the International Neuropsychological Society, 14(3), 436–445. doi: 10.1017/S1355617708080387CrossRefGoogle ScholarPubMed
Shunk, A.W., Davis, A.S., & Dean, R.S. (2006). Review of Delis-Kaplan Executive Function System (D-KEFS). Applied Neuropsychology, 13(4), 275–27. doi: 10.1207/s15324826an1304_9CrossRefGoogle Scholar
Strong, C.A.H., Tiesma, D., & Donders, J. (2011). Criterion validity of the Delis-Kaplan Executive Function System (D-KEFS) fluency subtests after traumatic brain injury. Journal of the International Neuropsychological Society, 17(2), 230–237. doi: 10.1017/S1355617710001451CrossRefGoogle ScholarPubMed
Swanson, J. (2005). The Delis-Kaplan Executive Function xSystem: A Review. Canadian Journal of School Psychology. doi: 10.1177/0829573506295469CrossRefGoogle Scholar
The Psychological Corporation. (1999). Wechsler Abbreviated Scale of Intelligence. San Antonio, TX: Psychological Corporation.Google Scholar