Hostname: page-component-745bb68f8f-d8cs5 Total loading time: 0 Render date: 2025-02-05T08:35:31.834Z Has data issue: false hasContentIssue false
  • English
  • Français

The Ecological and Construct Validity of a Newly Developed Measure of Executive Function: The Virtual Library Task

Published online by Cambridge University Press:  20 February 2012

Belinda Renison ,
Jennie Ponsford ,
Renee Testa ,
Barry Richardson  and
Kylie Brownfield
Belinda Renison
Affiliation:
Monash University, School of Psychology and Psychiatry, Melbourne, Australia
Jennie Ponsford*
Affiliation:
Monash University, School of Psychology and Psychiatry, Melbourne, Australia Monash-Epworth Rehabilitation Research Centre (MERRC), Melbourne, Australia
Renee Testa
Affiliation:
Monash University, School of Psychology and Psychiatry, Melbourne, Australia Melbourne Neuropsychiatry Centre, The University of Melbourne, Melbourne, Australia
Barry Richardson
Affiliation:
Monash University, School of Humanities, Communications and Social Sciences, Melbourne, Australia
Kylie Brownfield
Affiliation:
Monash University, School of Psychology and Psychiatry, Melbourne, Australia
*
Correspondence and reprint requests to: Jennie Ponsford, School of Psychology and Psychiatry, Monash University, Clayton, 3800, Victoria, Australia. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Virtual reality (VR) assessment paradigms have the potential to address the limited ecological validity of pen and paper measures of executive function (EF) and the pragmatic and reliability issues associated with functional measures. To investigate the ecological validity and construct validity of a newly developed VR measure of EF, the Virtual Library Task (VLT); a real life analogous task—the Real Library Task (RLT); and five neuropsychological measures of EF were administered to 30 patients with traumatic brain injury (TBI) and 30 healthy Controls. Significant others for each participant also completed the Dysexecutive Questionnaire (DEX), which is a behavioral rating scale of everyday EF. Performances on the VLT and the RLT were significantly positively correlated indicating that VR performance is similar to real world performance. The TBI group performed significantly worse than the Control group on the VLT and the Modified Six Elements Test (MSET) but the other four neuropsychological measures of EF failed to differentiate the groups. Both the MSET and the VLT significantly predicted everyday EF suggesting that they are both ecologically valid tools for the assessment of EF. The VLT has the advantage over the MSET of providing objective measurement of individual components of EF. (JINS, 2012, 18, 440–450)

Keywords

Executive functionBrain injuriesNeuropsychological testsValidation studiesCognitive functionVirtual systems
Type
Research Articles
Information
Journal of the International Neuropsychological Society , Volume 18 , Issue 3 , May 2012 , pp. 440 - 450
Copyright
Copyright © The International Neuropsychological Society 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alderman, N., Burgess, P.W., Knight, C., Henman, C. (2003). Ecological validity of a simplified version of the multiple errands shopping test. Journal of the International Neuropsychological Society, 9(1), 31–44.CrossRefGoogle ScholarPubMed
Axelrod, B.N., Goldman, R.S., Heaton, R.K., Curtiss, G., Thompson, L.L.., Ghelune, G.J., et al. (1996). Discriminability of the Wisconsin Card Sorting Test using the standardization sample. Journal of Clinical & Experimental Neuropsychology Society, 18, 338–342.CrossRefGoogle ScholarPubMed
Bennett, P.C., Ong, B., Ponsford, J. (2005a). Assessment of executive dysfunction following traumatic brain injury: Comparison of the BADS with other clinical neuropsychological measures. Journal of the International Neuropsychological Society, 11(05), 606–613. doi:10.1017/S1355617705050721CrossRefGoogle ScholarPubMed
Bennett, P.C., Ong, B., Ponsford, J. (2005b). Measuring executive dysfunction in an acute rehabilitation setting: Using the dysexecutive questionnaire (DEX). Journal of the International Neuropsychological Society, 11(4), 376–385.CrossRefGoogle Scholar
Benton, A. (1968). Differential behavioural effects on frontal lobe disease. Neuropsychologia, 6, 53–60.CrossRefGoogle Scholar
Brooks, B.M., Rose, F.D., Potter, J., Jayawardena, S., Morling, A. (2004). Assessing stroke patients’ prospective memory using virtual reality. Brain Injury, 18(4), 391–401.CrossRefGoogle ScholarPubMed
Burgess, P.W., Alderman, N., Evans, J., Emslie, H., Wilson, B.A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 10(4), 547–558.CrossRefGoogle Scholar
Burgess, P.W., Alderman, N., Forbes, C., Costello, A., Coates, L.M., Dawson, D.R., Channon, S. (2006). The case for the development and use of ‘ecologically valid’ measures of executive function in experimental and clincial neuropsychology. Journal of the International Neuropsychological Society, 12(2), 194–209 .CrossRefGoogle Scholar
Burgess, P.W., Alderman, N., Volle, E., Benoit, R.G., Gilbert, S.J. (2009). Mesulam's frontal lobe mystery re-examined. Resotrative Neurology and Neuroscience, 27(5), 493–506.CrossRefGoogle ScholarPubMed
Burgess, P.W., Shallice, T. (1996). The Hayling and Brixton Tests: Test manual. Edmunds, England: Themes Valley Test Company Limited.Google Scholar
Busch, R.M., McBride, A., Curtiss, G., Vanderploeg, R.D. (2005). The components of executive functioning in traumatic brain injury. Journal of Clinical & Experiemental Neuropsychology, 27, 1022–1032.CrossRefGoogle ScholarPubMed
Chan, R.C. (2001). Dysexecutive symptoms among a non-clinical sample: A study with the use of the Dysexecutive Questionnaire. British Journal of Psychology, 92(3), 551–565.CrossRefGoogle Scholar
Chan, R., Manly, T. (2002). The application of “dysexecutive syndrome” measures across cultures: Performance and checklist assessment in neurologically healthy and traumatically brain-injured Hong Kong Chinese volunteers. Journal of the International Neuropsychological Society, 8(6), 771–780.CrossRefGoogle ScholarPubMed
Chan, R., Shum, D., Toulopoulou, T., Chen, E. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23, 201–216.CrossRefGoogle ScholarPubMed
Chaytor, N., Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13(4), 181–197.CrossRefGoogle ScholarPubMed
Chaytor, N., Schmitter-Edgecombe, M., Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21(3), 217–227.CrossRefGoogle ScholarPubMed
Coakes, S.J., Steed, L.G. (2003). SPSS Analysis Without Anguish. Queensland: John Wiley & Sons Australia.Google Scholar
Crawford, J.R., Henry, J.D. (2005). Assessment of executive dysfunction. New York, NY: Oxford University Press.CrossRefGoogle Scholar
Cushman, L.A., Duffy, C.J. (2008). Detecting navigational deficits in cognitive aging and Alzheimer disease using virtual reality. Neurology, 71, 888–895.CrossRefGoogle ScholarPubMed
Damasio, A.R., Anderson, S.W. (2003). The frontal lobes. In K.M. Heilman & E. Valenstein (Eds.), Clinical Neuropsychology (4th ed.). New York: Oxford University Press.Google Scholar
Dawson, D.R., Anderson, N.D., Burgess, P., Cooper, E., Krpan, K.M., Stuss, D.T. (2009). Further development of the Multiple Errands Test: Standardized scoring, reliability, and ecological validity for the Baycrest version. Archives of Physical Medicine and Rehabilitation, 90(11, Suppl. 1), S41–S51.CrossRefGoogle ScholarPubMed
Draper, K., Ponsford, J. (2008). Cognitive functioning ten years following traumatic brain injury and rehabilitation. Neuropsychology, 22(5), 618–625.CrossRefGoogle ScholarPubMed
Duncan, J. (2005). Frontal lobe function and general intelligence: Why it matters. Cortex, 41, 215–217.CrossRefGoogle ScholarPubMed
Duncan, J., Johnson, R., Swales, M., Freer, C. (1997). Frontal lobe deficits after head injury: Unity and diversity of function. Cognitive Neuropsychology, 14(5), 713–741.Google Scholar
Elkind, J.S., Rubin, E., Rosenthal, S., Skoff, B., Prather, P. (2001). A simulated reality scenario compared with the computerized Wisconsin Card Sorting Test: An analysis of Preliminary results. CyberPsychology & Behavior, 4(4), 489–496.CrossRefGoogle ScholarPubMed
Evans, J., Chua, S., McKenna, P., Wilson, B. (1997). Assessment of the dysexecutive syndrome in schizophrenia. Psychological Medicine, 27(3), 635–646.CrossRefGoogle ScholarPubMed
Greve, K.W., Brooks, J., Crouch, J.A., Williams, M.C., Rice, W.J. (1997). Factorial structure of the Wisconsin Card Sorting Test. British Journal of Clinical Psychology, 36, 283–285.CrossRefGoogle ScholarPubMed
Heaton, R.K. (2005). Wisconsin Card Sorting Test-64: Computer version 2 – research edition (WCST-64CV2). Lutz, FL: Psychological Assessment Resources.Google Scholar
Jansari, A., Agnew, R., Akesson, K., Murphy, L (2004). Using virtual reality to create an ecologically valid measure of real world problems in patients with dysexecutive syndrome. Paper presented at the Symposium on Neurological Rehabilitation: A Satellite Symposium to the Joint 27th Conference of the Australian Society for the Study of Brain Impairment and the 27th Mid-year Meeting of the International Neuropsychological Society Uluru, Australia.Google Scholar
Jovanovski, D. (2004). Cognitive set shifting using functional magnetic reasonance imaging and virtual reality: A comparison between traditional and a novel ecologically valid exectuive function task. Master of Arts, University of Toronto, Toronto.Google Scholar
Joyner, K.B., Silver, C.H., Stavinoha, P.L. (2009). Relationship between parenting stress and ratings of executive functioning in children with ADHD. Journal of Psychoeducational Assessment, 27(6), 452–464. doi:10.1177/0734282909333945CrossRefGoogle Scholar
Katz, N., Tadmor, I., Felzen, B., Hartman-Maeir, A. (2007). Validity of the Executive Function Performance Test in individuals with schizophrenia. OTJR: Occupation, Participation and Health, 27(2), 44–51.Google Scholar
Klinger, E., Chemin, I., Lebreton, S., Marie, R.-M. (2006). Virtual action planning in Parkinson's Disease: A control study. CyberPsychology & Behaviour, 9(3), 342–347.CrossRefGoogle ScholarPubMed
Knight, C., Alderman, N. (2002). Development of a simplified version of the Multiple Errands Test for use in hospital settings. Neuropsychological Rehabilitation, 12(3), 231–255.CrossRefGoogle Scholar
Knight, R.G., Titov, N. (2009). Use of virtual reality tasks to assess prospective memory: Applicability and evidence. Brain Impairment, 10(1), 3–13.CrossRefGoogle Scholar
Lamberts, K.F., Evans, J.J., Spikman, J.M. (2010). A real-life, ecologically valid test of executive functioning: The executive secretarial task. Journal of Clinical and Experimental Neuropsychology, 32(1), 56–65.CrossRefGoogle ScholarPubMed
Lezak, M.D., Howieson, D.B., Loring, D.W. (2004). Neuropsychological assessment (4th ed.). New York: Oxford Univerity Press.Google Scholar
Long, C.J., Collins, L.F. (1997). Ecological validity and forensic neuropsychological assessment. In R.J. McCaffrey, A.D. Williams, J.M. Fisher, & L.C. Laign (Eds.), The practice of forensic neuropsychology: Meeting challenges in the courtroom. New York: Plenum Press.Google Scholar
Manchester, D., Priestley, N., Jackson, H. (2004). The assessment of executive functions: Coming out of the office. Brain Injury, 18(11), 1067–1081.CrossRefGoogle ScholarPubMed
Manly, T., Hawkins, K., Evans, J., Woldt, K., Robertson, I.H. (2002). Rehabilitation of executive function: Facilitation of effective goal management on complex tasks using periodic auditory alerts. Neuropsychologia, 40(3), 271–281.CrossRefGoogle ScholarPubMed
Matheis, R.J. (2004). Expanding the boundaries of neuropsychology: The application of VR for memory assessment. Doctoral Thesis, Fairleigh Dickinson University, New Jersey.Google Scholar
McGeorge, P., Phillips, L.H., Crawford, J.R., Garden, S.E., Della Sala, S., Milne, A.B. (2001). Using virtual environments in the assessment of executive dysfunction. Presence, 10(4), 375–383.CrossRefGoogle Scholar
Morganti, F. (2004). Virtual interaction in cognitive neuropsychology. In G. Riva, C. Botella, P. Legeron, & G. Optale (Eds.), Cybertherapy: Internet and virtual reality as assessment and rehabilitation tolls for clinical psychology and neuroscience. Amsterdam: IOS Press.Google Scholar
Moriyama, Y., Mimura, M., Kato, M., Yoshino, A., Hara, T., Kashima, H., Watanabe, A. (2002). Executive dysfunction and clinical outcome in chronic alcoholics. Alcoholism: Clinical and Experimental Research, 26(8), 1239–1244 .CrossRefGoogle ScholarPubMed
Morris, R.G., Kotitsa, M., Bramham, J., Brooks, B.M., Rose, D. (2002). Virtual reality investigation of strategy formation, rule breaking and prospective memory in patients with focal prefrontal neurosurgical lesions. Paper presented at the International Conference of Disability, Virtual Reality & Associated Technology, Hungry.Google Scholar
Norris, G., Tate, R.L. (2000). The Behavioural Assessment of the Dysexecutive Syndrome (BADS): Ecological, concurrent and construct validity. Neuropsychological Rehabilitation, 10(1), 33–45.CrossRefGoogle Scholar
Ord, J.S., Greve, K.W., Bianchini, K.J., Aguerrevere, L.E. (2009). Executive dysfunction in traumatic brain injury: The effects of injury severity and effort on the Wisconsin Card Sorting Test. Journal of Clinical and Experimental Neuropsychology, 32(2), 132–140.CrossRefGoogle ScholarPubMed
Ponsford, J., Kinsella, G. (1991). The use of a rating scale of attentional behaviour. Neuropsychological Rehabilitation, 1(4), 241–257.CrossRefGoogle Scholar
Rand, D., Basha-Abu Rukan, S., Weiss, P.L., Katz, N. (2009). Validation of the Virtual MET as an assessment tool for executive functions. Neuropsychological Rehabilitation, 19(4), 583–602.CrossRefGoogle ScholarPubMed
Renison, B., Ponsford, J., Testa, R., Richardson, B., Brownfield, K. (2008). Virtual library task. Melbourne: Monash University.Google Scholar
Shallice, T., Burgess, P.W. (1991). Deficits in strategy application following frontal lobe damage in man. Brain, 114(2), 727–741.CrossRefGoogle ScholarPubMed
Shores, E.A., Marosszeky, J.E., Sandanam, J., Batchelor, J. (1986). Preliminary validation of a clinical scale for measuring the duration of posttraumatic amnesia. The Medical Journal of Australia, 144, 569–572.CrossRefGoogle Scholar
Spooner, D.M., Pachana, N.A. (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology, 21(4), 327–337.CrossRefGoogle ScholarPubMed
Stuss, D.T. (2007). New approaches to prefrontal lobe testing. In B. L. Miller & J. Cummings (Eds.), The human frontal lobes: Functions and disorders (2nd ed., pp. 292–305). New York: Guilford Press.Google Scholar
Testa, R., Bennett, P.C., Ponsford, J. (2011). Factor Analysis of Nineteen Executive Function Tests in a Healthy Adult Population. Manuscript accepted for publication to Archives of Clinical Neuropsychology. In Press.Google Scholar
Titov, N., Knight, R.G. (2005). A computer-based procedure for assessing functional cognitive skills in patients with neurological injuries: The virtual street. Brain Injury, 19(5), 315–322.CrossRefGoogle ScholarPubMed
Tyson, P.J., Laws, K.R., Flowers, K.A., Mortimer, A.M., Schulz, J. (2008). Attention and executive function in people with schizophrenia: Relationship with social skills and quality of life. International Journal of Psychiatry in Clinical Practice, 12(2), 112–119.CrossRefGoogle ScholarPubMed
Wilson, B.A., Alderman, N., Burgess, P.W., Emslie, H., Evans, J.J. (1996). Behavioural assessment of the dysexecutive syndrome: Test manual. England: Thames Valley Test Company.Google Scholar
Wilson, B.A., Evans, J.J., Emslie, H., Alderman, N., Burgess, P.W. (1998). The development of an ecologically valid test for assessing patients with dysexecutive syndrome. Neuropsychological Rehabilitation, 8(3), 213–228.CrossRefGoogle Scholar
Zhang, L., Abreu, B.C., Seale, G.S., Masel, B., Christiansen, C.H., Ottenbacher, K.J. (2003). A virtual reality environment for evaluation of a daily living skill in brain injury rehabilitation: Reliability and validity. Archives of Physical Medicine and Rehabilitation, 84(8), 1118–1124.CrossRefGoogle ScholarPubMed