Hostname: page-component-6bf8c574d5-w79xw Total loading time: 0 Render date: 2025-03-01T02:41:49.875Z Has data issue: false hasContentIssue false
  • English
  • Français

Equivalency of In-Person Versus Remote Assessment: WISC-V and KTEA-3 Performance in Clinically Referred Children and Adolescents

Published online by Cambridge University Press:  27 September 2021

Taralee Hamner Open the ORCID record for Taralee Hamner [Opens in a new window] ,
Cynthia F. Salorio Open the ORCID record for Cynthia F. Salorio [Opens in a new window] ,
Luther Kalb  and
Lisa A. Jacobson Open the ORCID record for Lisa A. Jacobson [Opens in a new window]
Taralee Hamner
Affiliation:
Department of Neuropsychology, Kennedy Krieger Institute, Baltimore, MD, USA Department of Psychiatry and Behavioral Sciences, Johns Hopkins School of Medicine, Baltimore, MD, USA
Cynthia F. Salorio*
Affiliation:
Department of Neuropsychology, Kennedy Krieger Institute, Baltimore, MD, USA Department of Physical Medicine and Rehabilitation, Johns Hopkins School of Medicine, Baltimore, MD, USA
Luther Kalb
Affiliation:
Department of Neuropsychology, Kennedy Krieger Institute, Baltimore, MD, USA Department of Mental Health, Johns Hopkins School of Public Health, Baltimore, MD, USA
Lisa A. Jacobson
Affiliation:
Department of Neuropsychology, Kennedy Krieger Institute, Baltimore, MD, USA Department of Psychiatry and Behavioral Sciences, Johns Hopkins School of Medicine, Baltimore, MD, USA
*
*Correspondence and reprint requests to: Cynthia Salorio, PhD, ABPP, Department of Neuropsychology, Kennedy Krieger Institute, 1750 E. Fairmount Avenue, Baltimore, MD21231, USA. Phone: 443-923-9440. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

Teletesting has the potential to reduce numerous barriers to patient care which have only become exacerbated during the COVID-19 pandemic. Although telehealth is commonly utilized throughout medicine and mental health practices, teletesting has remained limited within cognitive and academic evaluations. This may be largely due to concern for the validity of test administration via remote assessment. This cross-sectional study examined the equivalency of cognitive [Wechsler Intelligence Scales for Children – Fifth Edition (WISC-V)] and academic [Kaufman Test of Educational Achievement – Third Edition (KTEA-3)] subtests administered via either teletesting or traditional in-person testing within clinically referred youth.

Method:

Chart review using a retrospective, cross-sectional design included a total of 893 children and adolescents, ranging from 4 to 17 years (Mean age = 10.2 years, SD = 2.9 years) who were administered at least one subtest from the aforementioned cognitive or academic assessments. Of these, 285 received teletesting, with the remaining (n = 608) receiving in-person assessment. A total of seven subtests (five from the WISC-V and two from the KTEA-3) were examined. A series of inverse probability of exposure weighted (IPEW) linear regression models examined differences between groups for each of the seven subtests after adjustment for numerous demographic, diagnostic, and parent-reported symptom variables.

Results:

Only two significant differences were found, such that WISC-V Visual Puzzles (p < .01) and KTEA-3 Math Concepts (p = .03) scores were slightly higher in the teletesting versus in-person groups. However, these differences were quite small in magnitude (WISC-V Visual Puzzles, d = .33, KTEA-3 Math Concepts, d = .18).

Conclusions:

Findings indicate equivalency across methods of service delivery without clinically meaningful differences in scores among referred pediatric patients.

Keywords

TelemedicineTelehealthCognitiveAcademicNeuropsychologyPediatric
Type
Research Article
Information
Journal of the International Neuropsychological Society , Volume 28 , Issue 8 , September 2022 , pp. 835 - 844
Copyright
Copyright © INS. Published by Cambridge University Press, 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596 Google Scholar
Austin, P.C. & Stuart, E.A. (2015). Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Statistics in medicine, 34(28), 3661–3679.CrossRefGoogle ScholarPubMed
Bashshur, R.L., Shannon, G.W., Bashshur, N., & Yellowlees, P.M. (2016). The empirical evidence for telemedicine interventions in mental disorders. Telemedicine and e-Health, 22(2), 87–113.CrossRefGoogle ScholarPubMed
Berryhill, M.B., Culmer, N., Williams, N., Halli-Tierney, A., Betancourt, A., Roberts, H., & King, M. (2019). Videoconferencing psychotherapy and depression: a systematic review. Telemedicine and e-Health, 25(6), 435–446.CrossRefGoogle ScholarPubMed
Brearly, T.W., Shura, R.D., Martindale, S.L., Lazowski, R.A., Luxton, D.D., Shenal, B.V., & Rowland, J.A. (2017). Neuropsychological test administration by videoconference: A systematic review and meta-analysis. Neuropsychology Review, 27(2), 174–186.CrossRefGoogle ScholarPubMed
Cullum, C.M., Weiner, M.F., Gehrmann, H.R., & Hynan, L.S. (2006). Feasibility of telecognitive assessment in dementia. Assessment, 13(4), 385–390.CrossRefGoogle ScholarPubMed
Cullum, C.M., Hynan, L.S., Grosch, M., Parikh, M., & Weiner, M.F. (2014). Teleneuropsychology: Evidence for video teleconference-based neuropsychological assessment. Journal of the International Neuropsychological Society: JINS, 20(10), 1028.CrossRefGoogle Scholar
Daniel, M.H., Wahlstrom, D., & Zhang, O. (2014). Equivalence of Q-interactive® and Paper Administrations of Cognitive Tasks: WISC®–V. Q-Interactive Technical Report, 8.Google Scholar
DuPaul, G.J., Reid, R., Anastopoulos, A.D., Lambert, M.C., Watkins, M.W., & Power, T.J. (2016). Parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms: Factor structure and normative data. Psychological Assessment, 28(2), 214.CrossRefGoogle ScholarPubMed
Ebesutani, C., Bernstein, A., Nakamura, B.J., Chorpita, B.F., & Weisz, J.R. (2010). A psychometric analysis of the revised child anxiety and depression scale—parent version in a clinical sample. Journal of Abnormal Child Psychology, 38, 249–260.CrossRefGoogle Scholar
Egede, L.E., Acierno, R., Knapp, R.G., Lejuez, C., Hernandez-Tejada, M., Payne, E.H., & Frueh, B.C. (2015). Psychotherapy for depression in older veterans via telemedicine: a randomised, open-label, non-inferiority trial. The Lancet Psychiatry, 2(8), 693–701.CrossRefGoogle ScholarPubMed
Fabiano, G.A., Pelham, J., William, E, Waschbusch, D.A., Gnagy, E.M., Lahey, B.B., … Burrows-MacLean, L. (2006). A practical measure of impairment: Psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35(3), 369–385.CrossRefGoogle ScholarPubMed
Farmer, R.L., McGill, R.J., Dombrowski, S.C., McClain, M.B., Harris, B., Lockwood, A.B., … Loethen, E. (2020a). Teleassessment with children and adolescents during the coronavirus (COVID-19) pandemic and beyond: Practice and policy implications. Professional Psychology: Research and Practice, 51(5), 477–487.CrossRefGoogle Scholar
Farmer, R.L., McGill, R.J., Dombrowski, S.C., Benson, N.F., Smith-Kellen, S., Lockwood, A.B., … Stinnett, T.A. (2020b). Conducting Psychoeducational Assessments During the COVID-19 Crisis: the Danger of Good Intentions. Contemporary School Psychology, 25(1), 27–32.CrossRefGoogle ScholarPubMed
Galusha-Glasscock, J.M., Horton, D.K., Weiner, M.F., & Cullum, C.M. (2016). Video teleconference administration of the repeatable battery for the assessment of neuropsychological status. Archives of Clinical Neuropsychology, 31(1), 8–11.CrossRefGoogle ScholarPubMed
Harder, L., Hernandez, A., Hague, C., Neumann, J., McCreary, M., Cullum, C.M., & Greenberg, B. (2020). Home-based pediatric teleneuropsychology: A validation study. Archives of Clinical Neuropsychology, 35, 1266–1275.CrossRefGoogle ScholarPubMed
Hewitt, K.C., Rodgin, S., Loring, D.W., Pritchard, A.E., & Jacobson, L.A. (2020). Transitioning to telehealth neuropsychology service: considerations across adult and pediatric care settings. The Clinical Neuropsychologist, 34(7–8), 1335–1351.CrossRefGoogle ScholarPubMed
Hodge, M.A., Sutherland, R., Jeng, K., Bale, G., Batta, P., Cambridge, A., … Silove, N. (2019). Agreement between telehealth and face-to-face assessment of intellectual ability in children with specific learning disorder. Journal of Telemedicine and Telecare, 25(7), 431–437.CrossRefGoogle ScholarPubMed
Jenco, M. (2020). AAP awarded $6 million to improve access to care via telehealth models in response to the COVID-19 pandemic. Retrieved 23 March 2021. American Academy of Pediatrics. https://www.aappublications.org/news/2020/04/30/covid19grant043020.Google Scholar
Joffe, M.M., Ten Have, T.R., Feldman, H.I., & Kimmel, S.E. (2004). Model selection, confounder control, and marginal structural models: review and new applications. The American Statistician, 58(4), 272–279.CrossRefGoogle Scholar
Kaufman, A.S. & Kaufman, N.L. (2014). Kaufman Test of Educational Achievement (3rd ed.). Bloomington, MN: Pearson.Google Scholar
Koterba, C.H., Baum, K.T., Hamner, T., Busch, T.A., Davis, K.C., Tlustos-Carter, S., … Slomine, B.S. (2020). COVID-19 issues related to pediatric neuropsychology and inpatient rehabilitation–challenges to usual care and solutions during the pandemic. The Clinical Neuropsychologist, 34(7–8), 1380–1394.CrossRefGoogle ScholarPubMed
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in psychology, 4, 863.CrossRefGoogle ScholarPubMed
Marcin, J.P., Shaikh, U., & Steinhorn, R.H. (2016). Addressing health disparities in rural communities using telehealth. Pediatric Research, 79(1), 169–176.CrossRefGoogle ScholarPubMed
Penny, A.M., Waschbusch, D.A., Klein, R.M., Corkum, P., & Eskes, G. (2009). Developing a measure of sluggish cognitive tempo for children: Content validity, factor structure, and reliability. Psychological assessment, 21(3), 380.CrossRefGoogle ScholarPubMed
Peterson, R.K., Ludwig, N.N., & Jashar, D.T. (2020). A case series illustrating the implementation of a novel tele-neuropsychology service model during COVID-19 for children with complex medical and neurodevelopmental conditions: A companion to Pritchard et al., 2020. The Clinical Neuropsychologist, 35(1), 99–114.CrossRefGoogle Scholar
Pritchard, A.E., Sweeney, K., Salorio, C.F., & Jacobson, L.A. (2020). Pediatric neuropsychological evaluation via telehealth: Novel models of care. The Clinical Neuropsychologist, 34(7–8), 1367–1379.CrossRefGoogle ScholarPubMed
National Association of School Psychologists (2020). Virtual Service Delivery in Response to COVID-19 Disruptions.Google Scholar
Ragbeer, S.N., Augustine, E.F., Mink, J.W., Thatcher, A.R., Vierhile, A.E., & Adams, H.R. (2016). Remote assessment of cognitive function in juvenile neuronal ceroid lipofuscinosis (Batten disease) a pilot study of feasibility and reliability. Journal of child neurology, 31(4), 481–487.CrossRefGoogle ScholarPubMed
Raiford, S.E. (2017). Essentials of WISC-V integrated assessment. Hoboken, NJ, USA: John Wiley & Sons.Google Scholar
Ransom, D.M., Butt, S.M., DiVirgilio, E.K., Cederberg, C.D., Srnka, K.D., Hess, C.T., … Katzenstein, J.M. (2020). Pediatric Teleneuropsychology: Feasibility and Recommendations. Archives of Clinical Neuropsychology. https://doi.org/10.1093/arclin/acaa103 CrossRefGoogle ScholarPubMed
Temple, V., Drummond, C., Valiquette, S., & Jozsvai, E. (2010). A comparison of intellectual assessments over video conferencing and in-person for individuals with ID: preliminary data. Journal of Intellectual Disability Research, 54(6), 573–577.CrossRefGoogle ScholarPubMed
United States Census Bureau (2019). Quick Facts Population Estimates. Retrieved 11/05/2020 from https://www.census.gov/quickfacts/fact/table/US/PST045219 Google Scholar
Vahia, I.V., Ng, B., Camacho, A., Cardenas, V., Cherner, M., Depp, C.A., … Agha, Z. (2015). Telepsychiatry for neurocognitive testing in older rural Latino adults. The American Journal of Geriatric Psychiatry, 23(7), 666–670.CrossRefGoogle ScholarPubMed
Wadsworth, H.E., Dhima, K., Womack, K.B., Hart, J. Jr, Weiner, M.F., Hynan, L.S., & Cullum, C.M. (2018). Validity of teleneuropsychological assessment in older patients with cognitive disorders. Archives of Clinical Neuropsychology, 33(8), 1040–1045.CrossRefGoogle ScholarPubMed
Waite, M.C., Theodoros, D.G., Russell, T.G., & Cahill, L.M. (2010). Internet-based telehealth assessment of language using the CELF–4. Language, Speech, and Hearing Services in Schools.CrossRefGoogle Scholar
Wang, Q. & Rao, J. (2001). Empirical likelihood for linear regression models under imputation for missing responses. Canadian Journal of Statistics, 29(4), 597–608.CrossRefGoogle Scholar
Wechsler, D. (2014). Wechsler Intelligence Scale for Children (5th ed.). Bloomington, MN: NCS Pearson, Incorporated.Google Scholar
Weinstein, R.S., Lopez, A.M., Joseph, B.A., Erps, K.A., Holcomb, M., Barker, G.P., & Krupinski, E.A. (2014). Telemedicine, telehealth, and mobile health applications that work: opportunities and barriers. The American journal of medicine, 127(3), 183–187.CrossRefGoogle ScholarPubMed
Willcutt, E.G., Boada, R., Riddle, M.W., Chhabildas, N., DeFries, J.C., & Pennington, B.F. (2011). Colorado Learning Difficulties Questionnaire: validation of a parent-report screening measure. Psychological assessment, 23(3), 778.CrossRefGoogle ScholarPubMed
Wolraich, M.L., Lambert, W., Doffing, M.A., Bickman, L., Simmons, T., & Worley, K. (2003). Psychometric properties of the Vanderbilt ADHD diagnostic parent rating scale in a referred population. Journal of pediatric psychology, 28(8), 559–568.CrossRefGoogle Scholar
Wright, A.J. (2016). Equivalence of remote, online administration and traditional, face-to-face administration of the Woodcock–Johnson IV cognitive and achievement tests (online white paper).Google Scholar
Wright, A.J. (2020). Equivalence of remote, digital administration and traditional, InPerson administration of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V). Psychological Assessment. Advance online publication. http://dx.doi.org/10.1037/pas0000939 CrossRefGoogle Scholar