Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-08T00:02:17.739Z Has data issue: false hasContentIssue false
  • English
  • Français

Effects of using same- versus alternate-form memory tests during short-interval repeated assessments in multiple sclerosis

Published online by Cambridge University Press:  21 October 2005

RALPH H. B. BENEDICT
RALPH H. B. BENEDICT
Affiliation:
State University of New York (SUNY) at Buffalo School of Medicine, Department of Neurology, and the Jacobs Neurological Institute, Buffalo General Hospital, Buffalo, New York
Get access Rights & Permissions [Opens in a new window]

Abstract

Repeated neuropsychological testing gives rise to practice effects in that patients become familiar with test material as well as test-taking procedures. Using alternate forms prevents the learning of specific test stimuli, potentially mitigating practice effects. However, changing forms could diminish test-retest reliability coefficients. Our objective was to examine test-retest effects in multiple sclerosis (MS) patients randomly assigned to same- (SF) or alternate-form (AF) conditions. Thirty-four MS patients underwent neuropsychological evaluation. The battery included the California Verbal Learning Test II (CVLT-II) and the Brief Visuospatial Memory Test–Revised (BVMT-R), memory tests recommended by a recently convened consensus panel. Patients were randomly assigned to SF or AF groups and then tested at baseline and follow-up examination 1 week later. Analysis of variance tests (ANOVAs) revealed significant group × time interactions, with SF patients showing greater gain than AF patients. SF practice effects were often large, compromising test validity. Reliability coefficients were either equivalent or higher in the AF group, a finding attributed to ceiling effects and reduced variance in the SF group at retest. The generalizability of the findings may be limited to short test-retest intervals and the MS population. Nevertheless, I conclude that the use of CVLT-II and BVMT-R alternate forms likely helps preserve test validity without compromising test-retest reliability. (JINS, 2005, 11, 727–736.)

Keywords

Multiple sclerosisMemoryAlternate formsPractice effectsReliabilityValidity
Type
Research Article
Information
Journal of the International Neuropsychological Society , Volume 11 , Issue 6 , October 2005 , pp. 727 - 736
Copyright
© 2005 The International Neuropsychological Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Amato, M.P., Ponziani, G., Siracusa, G., & Sorbi, S. (2001). Cognitive dysfunction in early-onset multiple sclerosis: A reappraisal after 10 years. Archives of Neurology, 58, 1602–1606.Google Scholar
American Psychiatric Association (1994). Diagnostic and Statistical Manual of Mental Disorders. (4th ed.) Washington, DC: American Psychiatric Association.
Archibald, C.J. & Fisk, J.D. (2000). Information processing efficiency in patients with multiple sclerosis. Journal of Clinical & Experimental Neuropsychology, 22, 686–701.Google Scholar
Beatty, W.W. (1996). Memory disturbance in multiple sclerosis: Reconsideration of patterns of performance on the Selective Reminding Test. Journal of Clinical & Experimental Neuropsychology, 18, 56–62.Google Scholar
Beatty, W.W., Goodkin, D.E., Monson, N., & Beatty, P.A. (1989). Cognitive disturbances in patients with relapsing remitting multiple sclerosis. Archives of Neurology, 46, 1113–1119.Google Scholar
Beck, A.T., Steer, R.A., & Brown, G.K. (2000). BDI–Fast Screen for Medical Patients: Manual. San Antonio, TX: Psychological Corporation.
Benedict, R.H.B. (1997). Brief Visuospatial Memory Test–Revised: Professional manual. Odessa, FL: Psychological Assessment Resources, Inc.
Benedict, R.H.B., Cox, D., Thompson, L.L., Foley, F.W., Weinstock-Guttman, B., & Munschauer, F. (2004a). Reliable screening for neuropsychological impairment in MS. Multiple Sclerosis, 10, 675–678.Google Scholar
Benedict, R.H.B., Fischer, J.S., Archibald, C.J., Arnett, P.A., Beatty, W.W., Bobholz, J., Chelune, G.J., Fisk, J.D., Langdon, D.W., Caruso, L., Foley, F., LaRocca, N.G., Vowels, L., Weinstein, A., DeLuca, J., Rao, S.M., & Munschauer, F. (2002). Minimal neuropsychological assessment of MS patients: A consensus approach. Clinical Neuropsychologist, 16, 381–397.Google Scholar
Benedict, R.H.B., Fishman, I., McClellan, M.M., Bakshi, R., & Weinstock-Guttman, B. (2003). Validity of the Beck Depression Inventory–Fast Screen in multiple sclerosis. Multiple Sclerosis, 9, 393–396.Google Scholar
Benedict, R.H.B., Schretlen, D., Brandt, J., & Groninger, L. (1998). Revision of the Hopkins Verbal Learning Test: Reliability and normative data. Clinical Neuropsychologist, 12, 43–55.Google Scholar
Benedict, R.H.B., Schretlen, D., Groninger, L., Dobraski, M., & Shpritz, B. (1996). Revision of the Brief Visuospatial Memory Test: Studies of normal performance, reliability, and validity. Psychological Assessment, 8, 145–153.Google Scholar
Benedict, R.H.B., Weinstock-Guttman, B., Fishman, I., Sharma, J., Tjoa, C.W., & Bakshi, R. (2004b). Prediction of neuropsychological impairment in multiple sclerosis: Comparison of conventional magnetic resonance imaging measures of atrophy and lesion burden. Archives of Neurology, 61, 226–230.Google Scholar
Benedict, R.H.B. & Zgaljardic, D.J. (1998). Practice effects during repeated administrations of memory tests with and without alternate forms. Journal of Clinical and Experimental Neuropsychology, 20, 339–352.Google Scholar
Benton, A.L., Sivan, A.B., Hamsher, K., Varney, N.R., & Spreen, O. (1994). Contributions to neuropsychological assessment. (2nd ed.) New York: Oxford University Press.
Boringa, J.B., Lazeron, R.H., Reuling, I.E., Ader, H.J., Pfennings, L., Lindeboom, J., de Sonneville, L.M., Kalkers, N.F., & Polman, C.H. (2001). The brief repeatable battery of neuropsychological tests: Normative values allow application in multiple sclerosis clinical practice. Multiple Sclerosis, 7, 263–267.Google Scholar
Brandt, J. & Benedict, R.H.B. (2001). The Hopkins Verbal Learning Test Revised: Professional manual. Odessa, FL: Psychological Assessment Resources, Inc.
Camp, S.J., Stevenson, V.L., Thompson, A.J., Miller, D.H., Borras, C., Auriacombe, S., Brochet, B., Falautano, M., Filippi, M., Herisse-Dulo, L., Montalban, X., Parrcira, E., Polman, C.H., De Sa, J., & Langdon, D.W. (1999). Cognitive function in primary progressive and transitional progressive multiple sclerosis: A controlled study with MRI correlates. Brain, 122, 1341–1348.Google Scholar
Chelune, G.J., Naugle, R.I., Lueders, H., & Awad, I.A. (1991). Prediction of cognitive change as a function of preoperative ability status among temporal lobectomy patients seen at 6-month follow-up. Neurology, 41, 399–404.Google Scholar
Chelune, G.J., Naugle, R.I., Lueders, H., & Sedlak, J. (1993). Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7, 41–52.Google Scholar
Christodoulou, C., Krupp, L.B., Liang, Z., Huang, W., Melville, P., Roque, C., Scherl, W.F., Morgan, T., MacAllister, W.S., Li, L., Tudorica, L.A., Li, X., Roche, P., & Peyster, R. (2003). Cognitive performance and MR markers of cerebral injury in cognitively impaired MS patients. Neurology, 60, 1793–1798.Google Scholar
Cohen, J. (1988). Statistical power for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Collie, A., Darby, D.G., Falleti, M.G., Silbert, B.S., & Maruff, P. (2002). Determining the extent of cognitive change after coronary surgery: A review of statistical procedures. Annals of Thoracic Surgery, 73, 2005–2011.Google Scholar
Crawford, J.R., Stewart, L.E., & Moore, J.W. (1989). Demonstration of savings on the AVLT and development of a parallel form. Journal of Clinical and Experimental Neuropsychology, 11, 975–981.Google Scholar
Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
Cutter, G.R., Baier, M.L., Rudick, R.A., Cookfair, D.L., Fischer, J.S., Petkau, J., Syndulko, K., Weinshenker, B.G., Antel, J.P., Confavreaux, C., Ellison, G.W., Lublin, F., Miller, A.E., Rao, S.M., Reingold, S., Thompson, A., & Wiloughby, E. (1999). Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain, 122, 871–882.Google Scholar
DeCarli, C., Mungas, D., Harvey, D., Reed, B., Weiner, M., Chui, H., & Jagust, W. (2004). Memory impairment, but not cerebrovascular disease, predicts progression of MCI to dementia. Neurology, 63, 220–227.Google Scholar
Delis, D.C., Kaplan, E., & Kramer, J.H. (2001). Delis-Kaplan Executive Function System. San Antonio, TX: Psychological Corporation.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A. (2000). California Verbal Learning Test Manual: Second edition, Adult version. San Antonio, TX: Psychological Corporation.
Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A. (1987). California Verbal Learning Test: Adult version. San Antonio, TX: Psychological Corporation.
DeLuca, J., Gaudino, E.A., Diamond, B.J., Christodoulou, C., & Engel, R.A. (1998). Acquisition and storage deficits in multiple sclerosis. Journal of Clinical and Experimental Neuropsychology, 20, 376–390.Google Scholar
DeLuca, J., Johnson, S.K., & Natelson, B.H. (1993). Information processing efficiency in chronic fatigue syndrome and multiple sclerosis. Archives of Neurology, 50, 301–304.Google Scholar
DeLuca, J., Barbieri-Berger, S., & Johnson, S.K. (1994). The nature of memory impairments in multiple sclerosis: Acquisition versus retrieval. Journal of Clinical and Experimental Neuropsychology, 16, 183–189.Google Scholar
Demaree, H.A., DeLuca, J., Gaudino, E.A., & Diamond, B.J. (1999). Speed of information processing as a key deficit in multiple sclerosis: Implications for rehabilitation. Journal of Neurology, Neurosurgery & Psychiatry, 67, 661–663.Google Scholar
Duff, K., Westervelt, H.J., McCaffrey, R.J., & Haase, R.F. (2001). Practice effects, test-retest stability, and dual baseline assessments with the California Verbal Learning Test in an HIV sample. Archives of Clinical Neuropsychology, 16, 461–476.Google Scholar
Fischer, J.S. (1988). Using the Wechsler Memory Scale–Revised to detect and characterize memory deficits in multiple sclerosis. Clinical Neuropsychologist, 2, 149–172.Google Scholar
Fischer, J.S., Priore, R.L., Jacobs, L.D., Cookfair, D.L., Rudick, R.A., Herndon, R.M., Richert, J.R., Salazar, A.M., Goodkin, D.E., Granger, C.V., Simon, J.H., Grafman, J.H., Lezak, M.D., O'Reilly-Hovey, K.M., Perkins, K.K., Barilla-Clark, D., Schacter, M., Shucard, D.W., Davidson, A.L., Wende, K.E., Bourdette, D.N., & Kooijmans-Coutinho, M.F. (2000). Neuropsychological effects of interferon beta-1a in relapsing multiple sclerosis. Multiple Sclerosis Collaborative Research Group. Annals of Neurology, 48, 885–892.Google Scholar
Fischer, J.S., Rudick, R.A., Cutter, G.R., & Reingold, S.C. (1999). The Multiple Sclerosis Functional Composite Measure (MSFC): An integrated approach to MS clinical outcome assessment. National MS Society Clinical Outcomes Assessment Task Force. Multiple Sclerosis, 5, 244–250.Google Scholar
Franklin, G.M., Heaton, R.K., Nelson, L.M., Filley, C.M., & Seibert, C. (1988). Correlation of neuropsychological and MRI findings in chronic/progressive multiple sclerosis. Neurology, 38, 1826–1829.Google Scholar
Grant, I., McDonald, W.I., Trimble, M.R., Smith, E., & Reed, R. (1984). Deficient learning and memory in early and middle phases of multiple sclerosis. Journal of Neurology, Neurosurgery & Psychiatry, 47, 250–255.Google Scholar
Green, M.F., Kern, R.S., & Heaton, R.K. (2004). Longitudinal studies of cognition and functional outcome in schizophrenia: Implications for MATRICS. Schizophrenia Research, 72, 41–51.Google Scholar
Gronwall, D.M.A. (1977). Paced auditory serial addition task: A measure of recovery from concussion. Perceptual and Motor Skills, 44, 367–373.Google Scholar
Hageman, W.J.J.M. & Arrindell, W.A. (1993). A further refinement of the reliable change (RC) index by improving the pre-post difference score: Introducing RC(ID). Behaviour Research and Therapy, 31, 693–700.Google Scholar
Harvey, P.D., Palmer, B.W., Heaton, R.K., Mohamed, S., Kennedy, J., & Brickman, A. (2005). Stability of cognitive performance in older patients with schizophrenia: An 8-week test-retest study. American Journal of Psychiatry, 162, 110–117.Google Scholar
Hawkins, K.A. & Wexler, B.E. (1999). California Verbal Learning Test practice effects in a schizophrenia sample. Schizophrenia Research, 39, 73–78.Google Scholar
Heaton, R.K., Grant, I., Butters, N., White, D.A., Kirson, D., Atkinson, J.H., McCutchan, J.A., Taylor, M.J., Kelly, M.D., & Ellis, R.J. (1995). The HNRC 500—Neuropsychology of HIV infection at different disease stages. HIV Neurobehavioral Research Center. Journal of the International Neuropsychological Society, 1, 231–251.Google Scholar
Heaton, R.K. (1985). Neuropsychological findings in relapsing-remitting and chronic-progressive multiple sclerosis. Journal of Consulting & Clinical Psychology, 53, 103–110.Google Scholar
Heaton, R.K., Temkin, N., Dikmen, S., Avitable, N., Taylor, M.J., Marcotte, T.D., & Grant, I. (2001). Detecting change: A comparison of three neuropsychological methods, using normal and clinical samples. Archives of Clinical Neuropsychology, 16, 75–91.Google Scholar
Iverson, G.L., Lovell, M.R., & Collins, M.W. (2003). Interpreting change on ImPACT following sport concussion. Clinical Neuropsychologist, 17, 460–467.Google Scholar
Jacobs, L.D., Cookfair, D.L., Rudick, R.A., Herndon, R.M., Richert, J.R., Salazar, A.M., Fischer, J.S., Goodkin, D.E., Granger, C.V., Simon, J.H., Alam, J.J., Bartoszak, D.M., Bourdette, D.N., Braiman, J., Brownscheidle, C.M., Coats, M.K., Cohan, S.L., Dougherty, D.S., Kinkel, R.P., Mass, M.K., Munschauer, F.E., Priore, R.L., Pullicino, P.M., Scherokman, B.J., Weinstock-Guttman, B., Whitham, R.H., & Multiple Sclerosis Collaborative Research Group. (1996). Intramuscular interferon beta-1a for disease progression in relapsing multiple sclerosis. Annals of Neurology, 39, 285–294.Google Scholar
Jacobs, L.D., Wende, K.E., Brownscheidle, C.M., Apatoff, B., Coyle, P.K., Goodman, A., Gottesman, M.H., Granger, C.V., Greenberg, S.J., Herbert, J., Krupp, L., Lava, N.S., Mihai, C., Miller, A.E., Perel, A., Smith, C.R., & Snyder, D.H. (1999). A profile of multiple sclerosis: The New York State Multiple Sclerosis Consortium. Multiple Sclerosis, 5, 369–376.Google Scholar
Jacobson, N.S. & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.Google Scholar
Johnson, K.P., Brooks, B.R., Cohen, J.A., Ford, C.C., Goldstein, J., Lisak, R.P. et al. (1995). Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: Results of a phase III multicenter, double-blind placebo-controlled trial. The Copolymer 1 Multiple Sclerosis Study Group. Neurology, 45, 1268–1276.Google Scholar
Krupp, L.B., Christodoulou, C., Melville, P., Scherl, W.F., MacAllister, W.S., & Elkins, L.E. (2004). Donepezil improves memory in multiple sclerosis in a randomized clinical trial. Neurology, 63, 1579–1585.Google Scholar
Kujala, P. (1994). Automatic and controlled information processing in multiple sclerosis. Brain, 117, 1115–1126.Google Scholar
Kurtzke, J.F. (1983). Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS). Annals of Neurology, 13, 227–231.Google Scholar
Litvan, I., Grafman, J., Vendrell, P., & Martinez, J. (1988). Multiple memory deficits in patients with multiple sclerosis: Exploring the working memory system. Archives of Neurology, 45, 607–610.Google Scholar
Lovell, M., Collins, M., & Bradley, J. (2004). Return to play following sports-related concussion. Clinics in Sports Medicine, 23, 421–441.CrossRefGoogle Scholar
McCaffrey, R.J., Cousins, J.P., Westervelt, H.J., & Martynowicz, M. (1995). Practice effects with the NIMH AIDS abbreviated neuropsychological battery. Archives of Clinical Neuropsychology, 10, 241–250.Google Scholar
McCaffrey, R.J., Ortega, A., & Haase, R.F. (1993). Effects of repeated neuropsychological assessments. Archives of Clinical Neuropsychology, 8, 519–524.Google Scholar
McCaffrey, R.J., Ortega, A., Orsillo, S.M., & Nelles, W.B. (1992). Practice effects in repeated neuropsychological assessments. Clinical Neuropsychologist, 6, 32–42.Google Scholar
McDonald, W.I., Compston, A., Edan, G., Goodkin, D.E., Hartung, H., Lublin, F., McFarland, H.F., Paty, D.W., Polman, C.H., Reingold, S.C., Sandberg-Wollheim, M., Sibley, William A., Thompson, A., van der Noort, S., Weinshenker, B.Y., & Wolinsky, J.S. (2001). Recommended diagnostic criteria for multiple sclerosis: Guidelines from the international panel on the diagnosis of multiple sclerosis. Annals of Neurology, 50, 121–127.Google Scholar
Medaer, R., De Smedt, L., Swerts, M., & Geutjens, J. (1984). Use of rating scales to reflect cognitive and mental functioning in multiple sclerosis. Acta Neurologica Scandinavica, Supplementum, 101, 65–67.Google Scholar
Parker, E.S., Eaton, E.M., Whipple, S.C., Heseltine, P.N.R., & Bridge, T.P. (1995). University of Southern California repeatable episodic memory test. Journal of Clinical and Experimental Neuropsychology, 17, 926–936.Google Scholar
Paty, D.W., Li, D., UBC MS/MRI Study Group, & IFNB Multiple Sclerosis Study Group (1993). Interferon beta-1b is effective in relapsing-remitting multiple sclerosis. Neurology, 43, 662–667.Google Scholar
Rao, S.M. (1991). A manual for the brief, repeatable battery of neuropsychological tests in multiple sclerosis. New York: National Multiple Sclerosis Society.
Rao, S.M., Hammeke, T.A., McQuillen, M.P., Khatri, B.O., & Lloyd, D. (1984). Memory disturbance in chronic progressive multiple sclerosis. Archives of Neurology, 41, 625–631.Google Scholar
Rao, S.M., Leo, G.J., Bernardin, L., & Unverzagt, F. (1991). Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology, 41, 685–691.Google Scholar
Rao, S.M., Leo, G.J., Haughton, V.M., Aubin-Faubert, P.S., & Bernardin, L. (1989). Correlation of magnetic resonance imaging with neuropsychological testing in multiple sclerosis. Neurology, 39, 161–166.Google Scholar
Rapport, L.J., Axelrod, B.N., Theisen, M.E., Brines, D.B., Kalechstein, A.D., & Ricker, J.H. (1997). Relationship of IQ to verbal learning and memory: Test and retest. Journal of Clinical and Experimental Neuropsychology, 19, 655–666.Google Scholar
Rey, A. (1964). L'examen clinique en psychologie. Paris: Press Universitaire de France.
Ruff, R.M., Light, R.H., Parker, S.B., & Levin, H.S. (1996). Benton Controlled Oral Word Association Test: Reliability and updated norms. Archives of Clinical Neuropsychology, 11, 329–338.Google Scholar
Sawrie, S.M., Chelune, G.J., Naugle, R.I., & Lueders, H.O. (1996). Empirical methods for assessing meaningful neuropsychological change following epilepsy surgery. Journal of the International Neuropsychological Society, 2, 556–564.Google Scholar
Smith, A. (1982). Symbol Digit Modalities Test: Manual. Los Angeles: Western Psychological Services.
Temkin, N.R., Heaton, R.K., Grant, I., & Dikmen, S.S. (1999). Detecting significant change in neuropsychological test performance: A comparison of four models. Journal of the International Neuropsychological Society, 5, 357–369.Google Scholar
Woodard, J.L., Benedict, R.H.B., Roberts, V.J., Goldstein, F.C., Kinner, K.M., Capruso, D.X., & Clark, A.N. (1996). The reliability and validity of alternate short forms for the Benton Judgement of Line Orientation Test. Journal of Clinical and Experimental Neuropsychology, 18, 898–904.Google Scholar
Zgaljardic, D.J. & Benedict, R.H.B. (2001). Evaluation of practice effects in language and spatial processing test performance. Applied Neuropsychology, 8, 218–223.Google Scholar