Hostname: page-component-586b7cd67f-rdxmf Total loading time: 0 Render date: 2024-12-02T19:22:15.589Z Has data issue: false hasContentIssue false
  • English
  • Français

Operationalizing Impaired Performance in Neuropsychological Assessment: A Comparison of the Use of Published Versus Sample-Based Normative Data for the Prediction of Dementia

Published online by Cambridge University Press:  11 December 2019

Brandy L. Callahan Open the ORCID record for Brandy L. Callahan [Opens in a new window]  and
for the Alzheimer’s Disease Neuroimaging Initiative
Brandy L. Callahan*
Affiliation:
Department of Psychology, University of Calgary, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada Hotchkiss Brain Institute, 3330 Hospital Drive NW, Calgary, Alberta T2N 4N1, Canada Mathison Centre for Mental Health Research & Education, 3280 Hospital Drive NW, Calgary, Alberta T2N 4Z6, Canada
for the Alzheimer’s Disease Neuroimaging Initiative
Affiliation:
Department of Psychology, University of Calgary, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada Hotchkiss Brain Institute, 3330 Hospital Drive NW, Calgary, Alberta T2N 4N1, Canada Mathison Centre for Mental Health Research & Education, 3280 Hospital Drive NW, Calgary, Alberta T2N 4Z6, Canada
*
Correspondence and reprint requests to: Brandy L. Callahan, Department of Psychology, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada. Phone: +1 403 220 7291. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives:

To compare the sensitivity, specificity, and predictive value of published versus sample-based norms to detect early dementia in the Uniform Data Set (UDS).

Methods:

The UDS was administered to 526 nondemented participants from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Baseline scores were standardized using published norms and healthy control data from ADNI corrected for age, education, and sex. Subjects obtaining two scores < −1 SD (determined separately using published and sample norms) were labeled “at risk for dementia.” Both methods were compared on sensitivity, specificity, and positive/negative predictive value (PPV/NPV) for dementia at follow-up.

Results:

Risk scores derived from published data had 86.1% sensitivity, 62.0% specificity, 68.6% accuracy, 46.1% PPV, and 92.2% NPV. Those from sample norms were more sensitive (91.0%), less specific (52.9%), and less accurate (63.3%), with worse PPV (42.1%) and similar NPV (94.0%). Sample norms were better at identifying incident dementia cases with relatively lower education than those with higher education. Discrepancies between both methods were more common in women.

Conclusions:

Sample norms are marginally more sensitive than published norms for predicting dementia, while published norms are slightly more accurate. Accuracy of risk estimates for women and those with lower education may be increased using locally generated norms.

Keywords

Uniform Data SetDementia risk predictionOlder peopleCognitiveSensitivitySpecificityAccuracyAlzheimer
Type
Brief Communication
Information
Journal of the International Neuropsychological Society , Volume 26 , Issue 6 , July 2020 , pp. 624 - 631
Copyright
Copyright © INS. Published by Cambridge University Press, 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf

References

REFERENCES

Alzheimer’s Disease Neuroimaging Initiative. (2005). ADNI General Procedures Manual. Retrieved from http://adni.loni.usc.edu/wp-content/uploads/2010/09/ADNI_GeneralProceduresManual.pdf.Google Scholar
Anderson, S.J. (2001). On the importance of collecting local neuropsychological normative data. South African Journal of Psychology, 31(3), 29–34. doi: 10.1177/008124630103100304.CrossRefGoogle Scholar
Arsenault-Lapierre, G., Whitehead, V., Belleville, S., Massoud, F., Bergman, H., & Chertkow, H. (2011). Mild cognitive impairment subcategories depend on the source of norms. Journal of Clinical and Experimental Neuropsychology, 33(5), 596–603. doi: 10.1080/13803395.2010.547459.CrossRefGoogle ScholarPubMed
Babulal, G.M., Quiroz, Y.T., Albensi, B.C., Arenaza-Urquijo, E., Astell, A.J., Babiloni, C., Bahar-Fuchs, A., Bell, J., Bowman, G.L., Brickman, A.M., Chételat, G., Ciro, C., Cohen, A.D., Dilworth-Anderson, P., Dodge, H.H., Dreux, S., Edland, S., Esbensen, A., Evered, L., Ewers, M., Fargo, K.N., Fortea, J., Gonzalez, H., Gustafson, D.R., Head, E., Hendrix, J.A., Hofer, S.M., Johnson, L.A., Jutten, R., Kilborn, K., Lanctôt, K.L., Manly, J.J., Martins, R.N., Mielke, M.M., Morris, M.C., Murray, M.E., Oh, E.S., Parra, M.A., Rissman, R.A., Roe, C.M., Santos, O.A., Scarmeas, N., Schneider, L.S., Schupf, N., Sikkes, S., Snyder, H.M., Sohrabi, H.R., Stern, Y., Strydom, A., Tang, Y., Terrera, G.M., Teunissen, C., Melo van Lent, D., Weinborn, M., Wesselman, L., Wilcock, D.M., Zetterberg, H., O'Bryant, S.E., & International Society to Advance Alzheimer’s Research and Treatment, Alzheimer’s Association. (2019). Perspectives on ethnic and racial disparities in Alzheimer’s disease and related dementias: Update and areas of immediate need. Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association, 15(2), 292–312. doi: 10.1016/j.jalz.2018.09.009.CrossRefGoogle ScholarPubMed
Bondi, M.W., Edmonds, E.C., Jak, A.J., Clark, L.R., Delano-Wood, L., McDonald, C.R., Nation, D.A., Libon, D.J., Au, R., Galasko, D., & Salmon, D.P. (2014). Neuropsychological criteria for mild cognitive impairment improves diagnostic precision, biomarker associations, and progression rates. Journal of Alzheimer’s Disease: JAD, 42(1), 275–289. doi: 10.3233/JAD-140276.CrossRefGoogle ScholarPubMed
Brooks, B.L., Iverson, G.L., Holdnack, J.A., & Feldman, H.H. (2008). Potential for misclassification of mild cognitive impairment: A study of memory scores on the Wechsler Memory Scale-III in healthy older adults. Journal of the International Neuropsychological Society, 14(03), 463–478. doi: 10.1017/S1355617708080521.CrossRefGoogle ScholarPubMed
Callahan, B.L., Ramirez, J., Berezuk, C., Duchesne, S., & Black, S.E. (2015). Predicting Alzheimer’s disease development: A comparison of cognitive criteria and associated neuroimaging biomarkers. Alzheimer’s Research & Therapy, 7(1), 68. doi: 10.1186/s13195-015-0152-z.CrossRefGoogle ScholarPubMed
Chertkow, H., Feldman, H.H., Jacova, C., & Massoud, F. (2013). Definitions of dementia and predementia states in Alzheimer’s disease and vascular cognitive impairment: Consensus from the Canadian conference on diagnosis of dementia. Alzheimer’s Research & Therapy, 5(Suppl 1), S2. doi: 10.1186/alzrt198.CrossRefGoogle ScholarPubMed
Christensen, H., Hadzi-Pavlovic, D., & Jacomb, P. (1991). The psychometric differentiation of dementia from normal aging: A meta-analysis. Psychological Assessment, 3(2), 147–155. doi: 10.1037/1040-3590.3.2.147.CrossRefGoogle Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences, Vol. 2, (pp. 273–406). Hillsdale, New Jersey: Lawrence Erlbaum Associates.Google Scholar
Daugherty, J.C., Puente, A.E., Fasfous, A.F., Hidalgo-Ruzzante, N., & Pérez-Garcia, M. (2017). Diagnostic mistakes of culturally diverse individuals when using North American neuropsychological tests. Applied Neuropsychology: Adult, 24(1), 16–22. doi: 10.1080/23279095.2015.1036992.CrossRefGoogle ScholarPubMed
Edmonds, E.C., Delano-Wood, L., Jak, A.J., Galasko, D.R., Salmon, D.P., & Bondi, M.W. (2016). “Missed” mild cognitive impairment: High false-negative error rate based on conventional diagnostic criteria. Journal of Alzheimer’s Disease, 52(2), 685–691. doi: 10.3233/JAD-150986.CrossRefGoogle ScholarPubMed
Harrison, A.G., Armstrong, I.T., Harrison, L.E., Lange, R.T., & Iverson, G.L. (2014). Comparing Canadian and American Normative Scores on the Wechsler Adult Intelligence Scale-Fourth Edition. Archives of Clinical Neuropsychology, 29(8), 737–746. doi: 10.1093/arclin/acu048.CrossRefGoogle ScholarPubMed
Heilbronner, R.L., Sweet, J.J., Attix, D.K., Krull, K.R., Henry, G.K., & Hart, R.P. (2010). Official position of the American Academy of Clinical Neuropsychology on serial neuropsychological assessments: The utility and challenges of repeat test administrations in clinical and forensic contexts. The Clinical Neuropsychologist, 24(8), 1267–1278. doi: 10.1080/13854046.2010.526785.CrossRefGoogle ScholarPubMed
Holtzer, R., Verghese, J., Wang, C., Hall, C.B., & Lipton, R.B. (2008). Within-person across-neuropsychological test variability and incident dementia. JAMA, 300(7), 823. doi: 10.1001/jama.300.7.823.CrossRefGoogle ScholarPubMed
Loewenstein, D.A., Acevedo, A., Potter, E., Schinka, J.A., Raj, A., Greig, M.T., Agron, J., Barker, W.W., Wu, Y., Small, B., Schofield, E., & Duara, R. (2009). Severity of medial temporal atrophy and amnestic mild cognitive impairment: Selecting type and number of memory tests. The American Journal of Geriatric Psychiatry, 17(12), 1050–1058. doi: 10.1097/JGP.0b013e3181b7ef42.CrossRefGoogle ScholarPubMed
Mehta, K.M., Yaffe, K., Pérez-Stable, E.J., Stewart, A., Barnes, D., Kurland, B.F., & Miller, B.L. (2008). Race/ethnic differences in AD survival in US Alzheimer’s Disease Centers. Neurology, 70(14), 1163–1170. doi: 10.1212/01.wnl.0000285287.99923.3c.CrossRefGoogle ScholarPubMed
Palmer, B.W., Boone, K.B., Lesser, I.M., & Wohl, M.A. (1998). Base rates of “impaired” neuropsychological test performance among healthy older adults. Archives of Clinical Neuropsychology: The Official Journal of the National Academy of Neuropsychologists, 13(6), 503–511. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/14590634.Google ScholarPubMed
Prince, M., Bryce, R., Albanese, E., Wimo, A., Ribeiro, W., & Ferri, C.P. (2013). The global prevalence of dementia: A systematic review and metaanalysis. Alzheimer’s & Dementia: The Journal of the Alzheimer’s Association, 9(1), 63–75.e2. doi: 10.1016/j.jalz.2012.11.007.CrossRefGoogle ScholarPubMed
Rivera, D., & Arango-Lasprilla, J.C. (2017). Methodology for the development of normative data for Spanish-speaking pediatric populations. NeuroRehabilitation, 41(3), 581–592. doi: 10.3233/NRE-172275.CrossRefGoogle ScholarPubMed
Shirk, S.D., Mitchell, M.B., Shaughnessy, L.W., Sherman, J.C., Locascio, J.J., Weintraub, S., & Atri, A. (2011). A web-based normative calculator for the uniform data set (UDS) neuropsychological test battery. Alzheimer’s Research & Therapy, 3(6), 32. doi: 10.1186/alzrt94.CrossRefGoogle ScholarPubMed
Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A Compendium of Neuropsychological Tests. New York: Oxford University Press.Google Scholar
Thygesen, L.C., Gimsing, L.Nø, Bautz, A., Hvidt, N.C., & Johansen, C. (2017). Chronic neurodegenerative illnesses and epilepsy in Danish adventists and baptists: A Nationwide Cohort Study. Journal of Alzheimer’s Disease, 56(4), 1429–1435. doi: 10.3233/JAD-160710.CrossRefGoogle ScholarPubMed
Van Breukelen, G.J.P., & Vlaeyen, J.W.S. (2005). Norming clinical questionnaires with multiple regression: The Pain Cognition List. Psychological Assessment, 17(3), 336–344. doi: 10.1037/1040-3590.17.3.336.CrossRefGoogle ScholarPubMed
Weintraub, S., Salmon, D., Mercaldo, N., Ferris, S., Graff-Radford, N.R., Chui, H., Cummings, J., DeCarli, C., Foster, N.L., Galasko, D., Peskind, E., Dietrich, W., Beekly, D.L., Kukull, W.A., & Morris, J.C. (2009). The Alzheimer’s Disease Centers’ Uniform Data Set (UDS): The neuropsychologic test battery. Alzheimer Disease and Associated Disorders, 23(2), 91–101. doi: 10.1097/WAD.0b013e318191c7dd.CrossRefGoogle ScholarPubMed