Hostname: page-component-6bf8c574d5-2jptb Total loading time: 0 Render date: 2025-02-20T05:31:28.124Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing Bias in Cognitive Testing for Older Adults with Sensory Impairment: An Analysis of Differential Item Functioning in the Baltimore Longitudinal Study on Aging (BLSA) and the Atherosclerosis Risk in Communities Neurocognitive Study (ARIC-NCS)

Published online by Cambridge University Press:  26 April 2021

E. Nichols Open the ORCID record for E. Nichols [Opens in a new window] ,
J.A. Deal ,
B.K. Swenor ,
A.G. Abraham ,
N.M. Armstrong Open the ORCID record for N.M. Armstrong [Opens in a new window] ,
M.C. Carlson ,
M. Griswold ,
F.R. Lin ,
T.H. Mosley  and
P.Y. Ramulu
...Show all authors
E. Nichols*
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA
J.A. Deal
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA Cochlear Center for Hearing and Public Health, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD21205, USA
B.K. Swenor
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA Wilmer Eye Institute, Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD21287, USA
A.G. Abraham
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA Department of Epidemiology, School of Public Health, University of Colorado Anschutz Medical Campus, 1635 Aurora Ct, Aurora, CO 80045, USA
N.M. Armstrong
Affiliation:
Department of Psychiatry and Human Behavior, Brown University Warren Alpert Medical School, 700 Butler Dr, Box G-BH, Providence, RI02906, USA
M.C. Carlson
Affiliation:
Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway St, 8th Floor, Baltimore, MD21205, USA
M. Griswold
Affiliation:
Memory Impairment and Neurodegenerative Dementia Center, University of Mississippi Medical Center, 2500 North State St, Jackson, MS39216, USA
F.R. Lin
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA Cochlear Center for Hearing and Public Health, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD21205, USA Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway St, 8th Floor, Baltimore, MD21205, USA
T.H. Mosley
Affiliation:
Memory Impairment and Neurodegenerative Dementia Center, University of Mississippi Medical Center, 2500 North State St, Jackson, MS39216, USA
P.Y. Ramulu
Affiliation:
Wilmer Eye Institute, Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD21287, USA
N.S. Reed
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA Cochlear Center for Hearing and Public Health, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD21205, USA
S.M. Resnick
Affiliation:
Laboratory of Behavioral Neuroscience, National Institute on Aging, 251 Bayview Blvd, Suite 101, Baltimore, MD21224, USA
A.R. Sharrett
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA
A.L. Gross
Affiliation:
Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, W6508, Baltimore, MD, 21205, USA
*
Correspondence and reprint requests to: Emma Nichols, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 2024 E Monument St Suite 2-700, Baltimore, MD, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives:

Vision and hearing impairments affect 55% of people aged 60+ years and are associated with lower cognitive test performance; however, tests rely on vision, hearing, or both. We hypothesized that scores on tests that depend on vision or hearing are different among those with vision or hearing impairments, respectively, controlling for underlying cognition.

Methods:

Leveraging cross-sectional data from the Baltimore Longitudinal Study of Aging (BLSA) and the Atherosclerosis Risk in Communities Neurocognitive Study (ARIC-NCS), we used item response theory to test for differential item functioning (DIF) by vision impairment (better eye presenting visual acuity worse than 20/40) and hearing impairment (better ear .5–4 kHz pure-tone average > 25 decibels).

Results:

We identified DIF by vision impairment for tests whose administrations do not rely on vision [e.g., Delayed Word Recall both in ARIC-NCS: .50 logit difference between impaired and unimpaired (p = .04) and in BLSA: .62 logits (p = .02)] and DIF by hearing impairment for tests whose administrations do not rely on hearing [Digit Symbol Substitution test in BLSA: 1.25 logits (p = .001) and Incidental Learning test in ARIC-NCS: .35 logits (p = .001)]. However, no individuals had differences between unadjusted and DIF-adjusted measures of greater than the standard error of measurement.

Conclusions:

DIF by sensory impairment in cognitive tests was independent of administration characteristics, which could indicate that elevated cognitive load among persons with sensory impairment plays a larger role in test performance than previously acknowledged. While these results were unexpected, neither of these samples are nationally representative and each has unique selection factors; thus, replication is critical.

Keywords

CognitionNeuropsychological testsEpidemiologyBiasHearing lossLow vision
Type
Regular Research
Information
Journal of the International Neuropsychological Society , Volume 28 , Issue 2 , February 2022 , pp. 154 - 165
Copyright
Copyright © INS. Published by Cambridge University Press, 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Alzheimer’s Association. (2019). 2019 Alzheimer’s Disease Facts and Figures (No. 15(3); pp. 321–387). Retrieved from https://www.alz.org/media/documents/alzheimers-facts-and-figures-2019-r.pdf CrossRefGoogle Scholar
Asparouhov, T. & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495–508. doi: 10.1080/10705511.2014.919210.CrossRefGoogle Scholar
Benton, A.L. (1962). The visual retention test as a constructional praxis task. Confinia Neurologica, 22, 141–155. doi: 10.1159/000104348.CrossRefGoogle ScholarPubMed
Benton, A.L. & Hamsher, K.D. (1976). Multilingual examination. Iowa City, IA: University of Iowa.Google Scholar
Blackburn, H.L. & Benton, A.L. (1957). Revised administration and scoring of the digit span test. Journal of Consulting Psychology, 21(2), 139.CrossRefGoogle ScholarPubMed
Bollen, K.A. (1989). Confirmatory factor analysis. In Structural Equations with Latent Variables (pp. 226–318). John Wiley & Sons, Ltd. doi: 10.1002/9781118619179.ch7.Google Scholar
Brookmeyer, R., Johnson, E., Ziegler-Graham, K., & Arrighi, H.M. (2007). Forecasting the global burden of Alzheimer’s disease. Alzheimer’s & Dementia, 3(3), 186–191. doi: org/10.1016/j.jalz.2007.04.381.CrossRefGoogle ScholarPubMed
Cella, D., Eton, D.T., Lai, J.-S., Peterman, A.H., & Merkel, D.E. (2002). Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales. Journal of Pain and Symptom Management, 24(6), 547–561. doi: 10.1016/s0885-3924(02)00529-8.CrossRefGoogle ScholarPubMed
Chen, S.P., Bhattacharya, J., & Pershing, S. (2017). Association of vision loss with cognition in older adults. JAMA Ophthalmology, 135(9), 963–970. doi: 10.1001/jamaophthalmol.2017.2838.CrossRefGoogle ScholarPubMed
Deal, J.A., Betz, J., Yaffe, K., Harris, T., Purchase-Helzner, E., Satterfield, S., … Lin, F.R. (2017). Hearing impairment and incident dementia and cognitive decline in older adults: the health ABC study. The Journals of Gerontology: Series A, 72(5), 703–709. doi: 10.1093/gerona/glw069.Google ScholarPubMed
Delis, D., Kramer, J., Kaplan, E., & Thompkins, B. (1987). CVLT: California verbal learning test–adult version: manual. San Antonio, TX: Psychological Corporation.Google Scholar
Dupuis, K., Pichora-Fuller, M.K., Chasteen, A.L., Marchuk, V., Singh, G., & Smith, S.L. (2015). Effects of hearing and vision impairments on the Montreal Cognitive Assessment. Aging, Neuropsychology, and Cognition, 22(4), 413–437. doi: 10.1080/13825585.2014.968084.CrossRefGoogle ScholarPubMed
Ehrlich, J.R. & Langa, K.M. (2019). Potential for dementia prevention in Latin America and Africa based on population-attributable fraction estimates. The Lancet Global Health, 7(10), e1323. doi: 10.1016/S2214-109X(19)30329-8.CrossRefGoogle ScholarPubMed
Eton, D.T., Cella, D., Yost, K.J., Yount, S.E., Peterman, A.H., Neuberg, D.S., … Wood, W.C. (2004). A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. Journal of Clinical Epidemiology, 57(9), 898–910. doi: 10.1016/j.jclinepi.2004.01.012.CrossRefGoogle Scholar
Ferris, F.L. III, Kassoff, A., Bresnick, G.H., & Bailey, I. (1982). New visual acuity charts for clinical research. American Journal of Ophthalmology, 94(1), 91–96.CrossRefGoogle ScholarPubMed
Goman, A.M., & Lin, F.R. (2016). Prevalence of hearing loss by severity in the United States. American Journal of Public Health, 106(10), 1820–1822. doi: 10.2105/AJPH.2016.303299.CrossRefGoogle ScholarPubMed
Gussekloo, J., de Craen, A.J.M., Oduber, C., van Boxtel, M.P.J., & Westendorp, R.G.J. (2005). Sensory impairment and cognitive functioning in oldest-old subjects: the Leiden 85+ study. The American Journal of Geriatric Psychiatry, 13(9), 781–786. doi: 10.1097/00019442-200509000-00006.CrossRefGoogle ScholarPubMed
Hebert, L.E., Weuve, J., Scherr, P.A., & Evans, D.A. (2013). Alzheimer disease in the United States (2010-2050) estimated using the 2010 census. Neurology, 80(19), 1778–1783. doi: 10.1212/WNL.0b013e31828726f5.CrossRefGoogle ScholarPubMed
Jones, R.N. (2006). Identification of measurement differences between English and Spanish language versions of the mini-mental state examination: detecting differential item functioning using MIMIC modeling. Medical Care, 44(11), S124–S133. JSTOR. Retrieved from JSTOR.CrossRefGoogle ScholarPubMed
Jones, R.N. & Gallo, J.J. (2002). Education and sex differences in the mini-mental state examination effects of differential item functioning. The Journals of Gerontology: Series B, 57(6), P548–P558. doi: 10.1093/geronb/57.6.P548.CrossRefGoogle ScholarPubMed
Lin, Frank R. & Albert, M. (2014). Hearing loss and dementia – who is listening? Aging & Mental Health, 18(6), 671–673. doi: 10.1080/13607863.2014.915924.CrossRefGoogle ScholarPubMed
Lin, F.R., Ferrucci, L., An, Y., Goh, J.O., Doshi, J., Metter, E.J., … Resnick, S.M. (2014). Association of hearing impairment with brain volume changes in older adults. NeuroImage, 90, 84–92. doi: 10.1016/j.neuroimage.2013.12.059.CrossRefGoogle ScholarPubMed
Lin, Frank R., Ferrucci, L., Metter, E.J., An, Y., Zonderman, A.B., & Resnick, S.M. (2011). Hearing loss and cognition in the Baltimore longitudinal study of aging. Neuropsychology, 25(6), 763–770. doi: 10.1037/a0024238.CrossRefGoogle ScholarPubMed
Lin, Frank R., Niparko, J.K., & Ferrucci, L. (2011). Hearing loss prevalence in the United States. Archives of Internal Medicine, 171(20), 1851–1853. doi: 10.1001/archinternmed.2011.506.CrossRefGoogle ScholarPubMed
Lin, H., Zhang, L., Lin, D., Chen, W., Zhu, Y., Chen, C., … Chen, W. (2018). Visual restoration after cataract surgery promotes functional and structural brain recovery. EBioMedicine, 30, 52–61. doi: 10.1016/j.ebiom.2018.03.002.CrossRefGoogle ScholarPubMed
Lin, M.Y., Gutierrez, P.R., Stone, K.L., Yaffe, K., Ensrud, K.E., Fink, H.A., … Mangione, C.M. (2004). Vision impairment and combined vision and hearing impairment predict cognitive and functional decline in older women. Journal of the American Geriatrics Society, 52(12), 1996–2002. doi: 10.1111/j.1532-5415.2004.52554.x.CrossRefGoogle ScholarPubMed
Liu, Y., Yin, H., Xin, T., Shao, L., & Yuan, L. (2019). A comparison of differential item functioning detection methods in cognitive diagnostic models. Frontiers in Psychology, 10. doi: 10.3389/fpsyg.2019.01137.Google ScholarPubMed
Lord, F. (1980). Applications of item response theory to practical testing problems. New York: Routledge: Taylor & Fracis Group.Google Scholar
Morales, L.S., Flowers, C., Gutierrez, P., Kleinman, M., & Teresi, J.A. (2006). Item and scale differential functioning of the mini-mental state exam assessed using the differential item and test functioning (DFIT) framework. Medical Care, 44(11 Suppl 3), S143–S151. doi: 10.1097/01.mlr.0000245141.70946.29.CrossRefGoogle ScholarPubMed
Muthén, B. & Asparouhov, T. (2014). IRT studies of many groups: the alignment method. Frontiers in Psychology, 5. doi: 10.3389/fpsyg.2014.00978.Google ScholarPubMed
Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52(3), 431–462. doi: 10.1007/BF02294365.CrossRefGoogle Scholar
Nitsan, G., Wingfield, A., Lavie, L., & Ben-David, B.M. (2019). Differences in working memory capacity affect online spoken word recognition: evidence from eye movements. Trends in Hearing, 23, 2331216519839624. doi: 10.1177/2331216519839624.CrossRefGoogle ScholarPubMed
Peelle, J.E., Troiani, V., Grossman, M., & Wingfield, A. (2011). Hearing loss in older adults affects neural systems supporting speech comprehension. Journal of Neuroscience, 31(35), 12638–12643. doi: 10.1523/JNEUROSCI.2559-11.2011.CrossRefGoogle ScholarPubMed
Penfield, R.D. & Lam, T.C.M. (2000). Assessing differential item functioning in performance assessment: review and recommendations. Educational Measurement: Issues and Practice, 19(3), 5–15. doi: 10.1111/j.1745-3992.2000.tb00033.x.CrossRefGoogle Scholar
Pichora-Fuller, M.K., Kramer, S.E., Eckert, M.A., Edwards, B., Hornsby, B.W.Y., Humes, L.E., … Wingfield, A. (2016). Hearing impairment and cognitive energy: the Framework for Understanding Effortful Listening (FUEL). Ear and Hearing, 37, 5S. doi: 10.1097/AUD.0000000000000312.CrossRefGoogle Scholar
Pickles, J. (2012). An introduction to the physiology of hearing (4th ed.). Bradford: Emerald Group Publishing Limited.Google Scholar
Plassman, B.L., Langa, K.M., Fisher, G.G., Heeringa, S.G., Weir, D.R., Ofstedal, M.B., … Rodgers, W.L. (2007). Prevalence of dementia in the United States: the aging, demographics, and memory study. Neuroepidemiology, 29(1–2), 125–132.CrossRefGoogle ScholarPubMed
Prevention of Blindness and Deafness Grades of Hearing Impairment. (n.d.). Retrieved February 25, 2020, from WHO Website: http://www.who.int/deafness/hearing_impairment_grades/en/ Google Scholar
Reitan, R.M. (1958). Validity of the trail making test as an indicator of organic brain damage. Perceptual and Motor Skills, 8(3), 271–276. doi: 10.2466/pms.1958.8.3.271.CrossRefGoogle Scholar
Reitan, R.M. (1986). Trail Making Test: Manual for administration and scoring. Tucson, AZ: Reitan Neuropsychological Laboratory.Google Scholar
Roberts, K.L. & Allen, H.A. (2016). Perception and cognition in the ageing brain: a brief review of the short- and long-term links between perceptual and cognitive decline. Frontiers in Aging Neuroscience, 8. doi: 10.3389/fnagi.2016.00039.CrossRefGoogle Scholar
Rogers, M.A.M. & Langa, K.M. (2010). Untreated poor vision: a contributing factor to late-life dementia. American Journal of Epidemiology, 171(6), 728–735. doi: 10.1093/aje/kwp453.CrossRefGoogle ScholarPubMed
Rönnberg, J., Lunner, T., Zekveld, A., Sörqvist, P., Danielsson, H., Lyxell, B., … Rudner, M. (2013). The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances. Frontiers in Systems Neuroscience, 7. doi: 10.3389/fnsys.2013.00031.CrossRefGoogle Scholar
Rucker, D.D., McShane, B.B., & Preacher, K.J. (2015). A researcher’s guide to regression, discretization, and median splits of continuous variables. Journal of Consumer Psychology, 25(4), 666–678. doi: 10.1016/j.jcps.2015.04.004.CrossRefGoogle Scholar
Ryan, J.J. & Lopez, S.J. (2001). Wechsler adult intelligence scale-III. In Dorfman, W.I. & Hersen, M. (Eds.), Understanding psychological assessment (pp. 19–42). Boston, MA: Springer US. doi: 10.1007/978-1-4615-1185-4_2.CrossRefGoogle Scholar
Shock, N., Greulich, R., Arenberg, D., Costa, P., Lakatta, E., & Tobin, J. (1984). Normal human aging: the Baltimore longitudinal study of aging. Washington, DC: US Department of Health and Human Services.Google Scholar
Smith, A. (1968). The symbol-digit modalities test: a neuropsychologic test of learning and other cerebral disorders. In Hellmuth, J. (Ed.), Learning disorders (pp. 83–91). Seattle, WA: Special Child Publications.Google Scholar
Swenor, B.K., Ramulu, P.Y., Willis, J.R., Friedman, D., & Lin, F.R. (2013). The prevalence of concurrent hearing and vision impairment in the United States. JAMA Internal Medicine, 173(4), 312–313. doi: 10.1001/jamainternmed.2013.1880.CrossRefGoogle ScholarPubMed
Taljaard, D.S., Olaithe, M., Brennan-Jones, C.G., Eikelboom, R.H., & Bucks, R.S. (2016). The relationship between hearing impairment and cognitive function: a meta-analysis in adults. Clinical Otolaryngology, 41(6), 718–729. doi: 10.1111/coa.12607.CrossRefGoogle ScholarPubMed
Tay, T., Wang, J.J., Kifley, A., Lindley, R., Newall, P., & Mitchell, P. (2006). Sensory and cognitive association in older persons: findings from an older Australian population. Gerontology, 52(6), 386–394. doi: 10.1159/000095129.CrossRefGoogle ScholarPubMed
The ARIC Investigators. (1989). The Atherosclerosis Risk in Communities (ARIC) study: design and objectives. American Journal of Epidemiology, 129(4), 687–702.CrossRefGoogle Scholar
Tun, P.A., McCoy, S., & Wingfield, A. (2009). Aging, hearing acuity, and the attentional costs of effortful listening. Psychology and Aging, 24(3), 761–766. doi: 10.1037/a0014802.CrossRefGoogle ScholarPubMed
Vitale, S., Cotch, M.F., & Sperduto, R.D. (2006). Prevalence of visual impairment in the United States. JAMA, 295(18), 2158–2163. doi: 10.1001/jama.295.18.2158.CrossRefGoogle ScholarPubMed
Wechsler, D. (1987). The Wechsler memory scale—revised. San Antonio, TX: Psychological Corp, Harcourt.Google Scholar
Williams, B.W., Mack, W., & Henderson, V.W. (1989). Boston naming test in Alzheimer’s disease. Neuropsychologia, 27(8), 1073–1079. doi: 10.1016/0028-3932(89)90186-3.CrossRefGoogle ScholarPubMed
Yost, K.J., Sorensen, M.V., Hahn, E.A., Glendenning, G.A., Gnanasakthy, A., & Cella, D. (2005). Using multiple anchor- and distribution-based estimates to evaluate clinically meaningful change on the Functional Assessment of Cancer Therapy-Biologic Response Modifiers (FACT-BRM) instrument. Value in Health, 8(2), 117–127. doi: 10.1111/j.1524-4733.2005.08202.x.CrossRefGoogle ScholarPubMed
Zheng, D.D., Swenor, B.K., Christ, S.L., West, S.K., Lam, B.L., & Lee, D.J. (2018). Longitudinal associations between visual impairment and cognitive functioning: the Salisbury eye evaluation study. JAMA Ophthalmology, 136(9), 989–995. doi: 10.1001/jamaophthalmol.2018.2493.CrossRefGoogle ScholarPubMed
Supplementary material: File

Nichols et al. supplementary material

Appendix

Download Nichols et al. supplementary material(File)
File 994.5 KB