Hostname: page-component-669899f699-tpknm Total loading time: 0 Render date: 2025-04-26T22:32:17.672Z Has data issue: false hasContentIssue false
  • English
  • Français

Health technology assessment for digital technologies that manage chronic disease: a systematic review

Published online by Cambridge University Press:  26 May 2021

Amy von Huben Open the ORCID record for Amy von Huben [Opens in a new window] ,
Martin Howell ,
Kirsten Howard ,
Joseph Carrello  and
Sarah Norris
Amy von Huben*
Affiliation:
School of Public Health, Faculty of Medicine and Health, University of Sydney, Camperdown, New South Wales, Australia
Martin Howell
Affiliation:
School of Public Health, Faculty of Medicine and Health, University of Sydney, Camperdown, New South Wales, Australia
Kirsten Howard
Affiliation:
School of Public Health, Faculty of Medicine and Health, University of Sydney, Camperdown, New South Wales, Australia
Joseph Carrello
Affiliation:
School of Public Health, Faculty of Medicine and Health, University of Sydney, Camperdown, New South Wales, Australia
Sarah Norris
Affiliation:
School of Public Health, Faculty of Medicine and Health, University of Sydney, Camperdown, New South Wales, Australia
*
Author for correspondence: Amy von Huben, E-mail: amy.vonhuben@sydney.edu.au
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective

A growing number of evaluation frameworks have emerged over recent years addressing the unique benefits and risk profiles of new classes of digital health technologies (DHTs). This systematic review aims to identify relevant frameworks and synthesize their recommendations into DHT-specific content to be considered when performing Health Technology Assessments (HTAs) for DHTs that manage chronic noncommunicable disease at home.

Methods

Searches were undertaken of Medline, Embase, Econlit, CINAHL, and The Cochrane Library (January 2015 to March 2020), and relevant gray literature (January 2015 to August 2020) using keywords related to HTA, evaluation frameworks, and DHTs. Included framework reference lists were searched from 2010 until 2015. The EUNetHTA HTA Core Model version 3.0 was selected as a scaffold for content evaluation.

Results

Forty-four frameworks were identified, mainly covering clinical effectiveness (n = 30) and safety (n = 23) issues. DHT-specific content recommended by framework authors fell within 28 of the 145 HTA Core Model issues. A further twenty-two DHT-specific issues not currently in the HTA Core Model were recommended.

Conclusions

Current HTA frameworks are unlikely to be sufficient for assessing DHTs. The development of DHT-specific content for HTA frameworks is hampered by DHTs having varied benefit and risk profiles. By focusing on DHTs that actively monitor/treat chronic noncommunicable diseases at home, we have extended DHT-specific content to all nine HTA Core Model domains. We plan to develop a supplementary evaluation framework for designing research studies, undertaking HTAs, and appraising the completeness of HTAs for DHTs.

Keywords

Health technology assessmentDigital healthRemote monitoringEvaluation frameworksChronic disease
Type
Assessment
Information
International Journal of Technology Assessment in Health Care , Volume 37 , Issue 1 , 2021 , e66
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Australian Institute of Health and Welfare. Chronic disease overview. 2020 [cited 2021 Feb 10]. Available from: https://www.aihw.gov.au/reports-data/health-conditions-disability-deaths/chronic-disease/overview.Google Scholar
Eysenbach, G. Consort-ehealth: Improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. 2011;13:e126.CrossRefGoogle ScholarPubMed
Andalusian Health Quality Agency (ES) [Internet] Complete list of recommendations on design, use and assessment of health apps. Seville (ES). 2012 [cited 2020 Aug 16]. Available from: www.calidadappsalud.com/en/listado-completo-recomendaciones-app-salud/.Google Scholar
Kidholm, K, Ekeland, AG, Jensen, LK, Rasmussen, J, Pedersen, CD, Bowes, A, et al. A model for assessment of telemedicine applications: MAST. Int J Technol Assess Health Care. 2012;28:44.CrossRefGoogle Scholar
Haute Autorité de Santé [French National Authority for Health]. Methodological choices for the clinical development of medical devices. Paris (FR): The Authority; 2013.Google Scholar
Khoja, S, Durrani, H, Scott, RE, Sajwani, A, Piryani, U. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed e-Health. 2013;19:48–53.CrossRefGoogle ScholarPubMed
Lewis, TL, Wyatt, JC. Mhealth and mobile medical apps: A framework to assess risk and promote safer use. J Med Internet Res. 2014;16:e210.CrossRefGoogle ScholarPubMed
Bergmo, TS. How to measure costs and benefits of ehealth interventions: An overview of methods and frameworks. J Med Internet Res. 2015;17:e254.CrossRefGoogle ScholarPubMed
Mohr, DC, Schueller, SM, Riley, WT, Brown, CH, Cuijpers, P, Duan, N, et al. Trials of intervention principles: Evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. 2015;17:e166.CrossRefGoogle ScholarPubMed
Mookherji, S, Mehl, G, Kaonga, N, Mechael, P. Unmet need: Improving mhealth evaluation rigor to build the evidence base. J Health Commun. 2015;20:1224–9.CrossRefGoogle ScholarPubMed
Steventon, A, Grieve, R, Bardsley, M. An approach to assess generalizability in comparative effectiveness research: A case study of the whole systems demonstrator cluster randomized trial comparing telehealth with usual care for patients with chronic health conditions. Med Decis Making. 2015;35:1023–36.CrossRefGoogle ScholarPubMed
Ruck, A, Wagner Bondorf, S, Lowe, C (Consard Limited). Second draft of guidelines, EU guidelines on assessment of the reliability of mobile health applications. European Commission, Directorate-General of Communications Networks, Content & Technology; 2016.Google Scholar
Gorski, I, Bram, JT, Sutermaster, S, Eckman, M, Mehta, K. Value propositions of mhealth projects. J Med Eng Technol. 2016;40:400–21.CrossRefGoogle ScholarPubMed
McMillan, B, Hickey, E, Patel, MG, Mitchell, C. Quality assessment of a sample of mobile app-based health behavior change interventions using a tool based on the national institute of health and care excellence behavior change guidance. Patient Educ Couns. 2016;99:429–35.CrossRefGoogle ScholarPubMed
McNamee, P, Murray, E, Kelly, MP, Bojke, L, Chilcott, J, Fischer, A, et al. Designing and undertaking a health economics study of digital health interventions. Am J Prev Med. 2016;51:852–60.CrossRefGoogle ScholarPubMed
Murray, E, Hekler, EB, Andersson, G, Collins, LM, Doherty, A, Hollis, C, et al. Evaluating digital health interventions: Key questions and approaches. Am J Prev Med. 2016;51:843–51.CrossRefGoogle ScholarPubMed
Rojahn, K, Laplante, S, Sloand, J, Main, C, Ibrahim, A, Wild, J, et al. Remote monitoring of chronic diseases: A landscape assessment of policies in four European countries. PLoS ONE. 2016;11:e0155738.CrossRefGoogle ScholarPubMed
Young, M. IRBs could address ethical issues related to tracking devices: Mobile devices raise new concerns. IRB Advisor. 2017;17:8–9.Google Scholar
Lennon, MR, Bouamrane, MM, Devlin, AM, O'Connor, S, O'Donnell, C, Chetty, U, et al. Readiness for delivering digital health at scale: Lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. 2017;19:e42.CrossRefGoogle ScholarPubMed
Maar, MA, Yeates, K, Perkins, N, Boesch, L, Hua-Stewart, D, Liu, P, et al. A framework for the study of complex mhealth interventions in diverse cultural settings. JMIR MHealth UHealth. 2017;5:e47.CrossRefGoogle Scholar
Michie, S, Yardley, L, West, R, Patrick, K, Greaves, F. Developing and evaluating digital interventions to promote behavior change in health and health care: Recommendations resulting from an international workshop. J Med Internet Res. 2017;19:e232.CrossRefGoogle ScholarPubMed
Philpott, D, Guergachi, A, Keshavjee, K. Design and validation of a platform to evaluate mhealth apps. Stud Health Technol Inform. 2017;235:3–7.Google ScholarPubMed
Drury, P, Roth, S, Jones, T, Stahl, M, Medeiros, D. Guidance for investing in digital health. Manila (PH): Asian Development Bank (ADB); 2018.CrossRefGoogle Scholar
European Commission. Synopsis report, consultation: Transformation health and care in the digital single market. Luxembourg: The Commission; 2018.Google Scholar
Hogaboam, LS. Assessment of technology adoption potential of medical devices: Case of wearable sensor products for pervasive care in neurosurgery and orthopedics [PhD]. Ann Arbor: Portland State University; 2018.CrossRefGoogle Scholar
Jurkeviciute, M. Planning of a holistic summative ehealth evaluation: The interplay between standards and reality [Licentiate]. Ann Arbor: Chalmers Tekniska Hogskola (Sweden); 2018.Google Scholar
Nielsen, S, Rimpiläinen, S. Report on international practice on digital apps. Glasgow (UK): Digital Health and Care Institute; 2018.Google Scholar
Sax, M, Helberger, N, Bol, N. Health as a means towards profitable ends: Mhealth apps, user autonomy, and unfair commercial practices. J Consumer Policy. 2018;41:103–34.CrossRefGoogle Scholar
Academy of Medical Sciences (UK). Our data-driven future in healthcare: People and partnerships at the heart of health related technologies. London (UK): The Academy; 2018.Google Scholar
Wyatt, JC. How can clinicians, specialty societies and others evaluate and improve the quality of apps for patient use? BMC Med. 2018;16:225.CrossRefGoogle ScholarPubMed
Beintner, I, Vollert, B, Zarski, AC, Bolinski, F, Musiat, P, Gorlich, D, et al. Adherence reporting in randomized controlled trials examining manualized multisession online interventions: Systematic review of practices and proposal for reporting standards. J Med Internet Res. 2019;21:e14181.CrossRefGoogle ScholarPubMed
Caulfield, B, Reginatto, B, Slevin, P. Not all sensors are created equal: A framework for evaluating human performance measurement technologies. npj Digital Med. 2019;2:7.CrossRefGoogle ScholarPubMed
Department of Health & Social Care (UK) [Internet] Code of conduct for data-driven health and care technology. London (UK): The Department; 2019 [updated 2019 Jul 18; cited 2020 Aug 18]. Available from: https://www.gov.uk/government/publications/code-of-conduct-for-data-driven-health-and-care-technology/initial-code-of-conduct-for-data-driven-health-and-care-technology.Google Scholar
Haute Autorité de Santé [French National Authority for Health]. Guide to the specific features of clinical evaluation of a connected medical device (CMD) in view of its application for reimbursement. Paris (FR): The Authority; 2019.Google Scholar
Haute Autorité de Santé [French National Authority for Health]. Public consultation on the draft analysis grid intended for use by CNEDiMTS to contribute to its evaluation of medical devices embedding decision systems based on automatic learning processes (“artificial intelligence”). Paris (FR): The Authority; 2019.Google Scholar
Huckvale, K, Torous, J, Larsen, ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Network Open. 2019;2:e192542.CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence (UK). Evidence standards framework for digital health technologies. London (UK): The Institute; 2019.Google Scholar
NHS Digital (UK) [Internet] How we assess health apps and digital tools. London (UK): NHS Digital; 2019 [updated 2019 May 17; cited 2020 Apr 13]. Available from: https://digital.nhs.uk/services/nhs-apps-library/guidance-for-health-app-developers-commissioners-and-assessors/how-we-assess-health-apps-and-digital-tools.Google Scholar
Rajan, B, Tezcan, T, Seidmann, A. Service systems with heterogeneous customers: Investigating the effect of telemedicine on chronic care. Manag Sci. 2019;65:1236–67.CrossRefGoogle Scholar
Australian Commission on Safety and Quality in Health Care. National safety and quality digital mental health standards - Consultation draft. Sydney (AU): The Commission; 2020.Google Scholar
Dick, S, O'Connor, Y, Thompson, MJ, O'Donoghue, J, Hardy, V, Wu, TJ, et al. Considerations for improved mobile health evaluation: Retrospective qualitative investigation. JMIR MHealth UHealth. 2020;8:e12424.CrossRefGoogle ScholarPubMed
Federal Ministry of Health (DE). Regulation on the procedure and requirements for testing the eligibility for reimbursement of digital health applications in the statutory public health insurance (Digital Health Applications Ordinance - DiGAV) (Draft bill). Bonn (DE): The Ministry; 2020.Google Scholar
Health Information and Quality Authority (IE). International review of consent models for the collection, use and sharing of health information. Cork (IE): The Authority; 2020.Google Scholar
Medical Services Advisory Committee (AU). Draft guidelines for preparing assessment reports for the medical services advisory committee. Canberra (AU): The Committee; 2020.Google Scholar
Moshi, MR, Tooher, R, Merlin, T. Development of a health technology assessment module for evaluating mobile medical applications. Int J Technol Assess Health Care. 2020;36:252–61.CrossRefGoogle ScholarPubMed
EUnetHTA Joint Action 2, Work Package 8. HTA Core Model® version 3.0. [Pdf]; 2016. Available from: www.htacoremodel.info/BrowseModel.aspx.Google Scholar
World Health Organization. WHO guideline: Recommendations on digital interventions for health system strengthening. Geneva (CH): The Organization; 2019.Google Scholar
Medical Device Coordination Group. Guidance on qualification and classification of software in regulation (EU) 2017/745 – MDR and Regulation (EU) 2017/746 – IVDR. European Commission; 2019.Google Scholar
O'Rourke, B, Oortwijn, W, Schuller, T. The new definition of health technology assessment: A milestone in international collaboration. Int J Technol Assess Health Care. 2020;36:187–90.CrossRefGoogle ScholarPubMed
Australian Government Department of Health and Ageing. Review of health technology assessment in Australia. Canberra (AU): Commonwealth of Australia; 2009.Google Scholar
Regulation (EU) 2017/745 of the European Parliament and of the Council. Official Journal. 2017;L117:1–175.Google Scholar
Regulation (EU) 2016/679 of the European Parliament and of the Council. Official Journal. 2016;L119:1–88.Google Scholar
Moshi, MR, Tooher, R, Merlin, T. Suitability of current evaluation frameworks for use in the health technology assessment of mobile medical applications: A systematic review. Int J Technol Assess Health Care. 2018;34:464–75.CrossRefGoogle ScholarPubMed
Iribarren, SJ, Cato, K, Falzon, L, Stone, PW. What is the economic evidence for mhealth? A systematic review of economic evaluations of mhealth solutions. PLoS ONE. 2017;12:e0170581.CrossRefGoogle ScholarPubMed
Kidholm, K, Kristensen, MBD. A scoping review of economic evaluations alongside randomized controlled trials of home monitoring in chronic disease management. Appl Health Econ Health Policy. 2018;16:167–76.CrossRefGoogle ScholarPubMed
Vukovic, V, Favaretti, C, Ricciardi, W, de Waure, C. Health technology assessment evidence on e-health/m-health technologies: Evaluating the transparency and thoroughness. Int J Technol Assess Health Care. 2018;34:87–96.CrossRefGoogle ScholarPubMed
Moher, D, Liberati, A, Tetzlaff, J, Altman, DG. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009;6:e1000097.CrossRefGoogle ScholarPubMed
Grey matters: A practical tool for searching health-related grey literature [Internet]. Ottawa (CA): CADTH; 2018 [updated 2019 Apr; cited 2020 Apr 4]. Available from: https://www.cadth.ca/resources/finding-evidence.Google Scholar
Hailey, D, Ohinmaa, A, Roine, R. Study quality and evidence of benefit in recent assessments of telemedicine. London (UK): SAGE Publications; 2004. p. 318–24.Google ScholarPubMed
Gagnon, M-P, Scott, R. Striving for evidence in e-health evaluation: Lessons from health technology assessment. J Telemed Telecare. 2005;11:S34–6.CrossRefGoogle ScholarPubMed
Reardon, T. Research findings and strategies for assessing telemedicine costs. Telemed e-Health. 2005;11:348–69.CrossRefGoogle ScholarPubMed
Shiell, A, Hawe, P, Gold, L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ. 2008;336:1281–3.CrossRefGoogle ScholarPubMed
Dávalos, ME, French, MT, Burdick, AE, Simmons, SC. Economic evaluation of telemedicine: Review of the literature and research guidelines for benefit-cost analysis. Telemed e-Health. 2009;15:933–48.CrossRefGoogle ScholarPubMed
Rickles, D, Hawe, P, Shiell, A. A simple guide to chaos and complexity. J Epidemiol Community Health. 2007;61:933–7.CrossRefGoogle ScholarPubMed
Supplementary material: File

von Huben et al. supplementary material

Tables S1-S5 and Figure S1

Download von Huben et al. supplementary material(File)
File 98.3 KB