Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-26T08:16:49.295Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing the performance of health technology assessment organizations: A framework

Published online by Cambridge University Press:  24 January 2008

Louise Lafortune ,
Lambert Farand ,
Isabelle Mondou ,
Claude Sicotte  and
Renaldo Battista
Get access Rights & Permissions [Opens in a new window]

Abstract

In light of growing demands for public accountability, the broadening scope of health technology assessment organizations (HTAOs) activities and their increasing role in decision-making underscore the importance for them to demonstrate their performance. Based on Parson's social action theory, we propose a conceptual model that includes four functions an organization needs to balance to perform well: (i) goal attainment, (ii) production, (iii) adaptation to the environment, and (iv) culture and values maintenance. From a review of the HTA literature, we identify specific dimensions pertaining to the four functions and show how they relate to performance. We compare our model with evaluations reported in the scientific and gray literature to confirm its capacity to accommodate various evaluation designs, contexts of evaluation, and organizational models and perspectives. Our findings reveal the dimensions of performance most often assessed and other important ones that, hitherto, remain unexplored. The model provides a flexible and theoretically grounded tool to assess the performance of HTAOs.

Keywords

Health technology assessmentOrganizational performanceEvaluationHTA agencyTheoretical models
Type
GENERAL ESSAYS
Information
International Journal of Technology Assessment in Health Care , Volume 24 , Issue 1 , January 2008 , pp. 76 - 86
Copyright
Copyright © Cambridge University Press 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

1. Agència d'Avaluacio de Technologica i Recerca Mèdiques (AATRM) Annual Report 2004. Generalitat de Catalunya, Barcelona; CatSalut. Departamente de Sanitat i Seguretat Social; 2005.Google Scholar
2. Banta, D, Oortwijn, W. Conclusion: Health technology assessment and health care in the European Union. Int J Technol Assess Health Care. 2000;16:626-635.Google ScholarPubMed
3. Banta, D, Jonsson, B. Commentary on: Expanding the scientific basis of health technology assessment: A research agenda for the next decade. Int J Technol Assess Health Care. 2006;22:280–283.CrossRefGoogle Scholar
4. Banta, D, Oortwijn, W. Health technology assessment in the European Union. Int J Technol Assess Health Care. 2000;16:299–635.CrossRefGoogle ScholarPubMed
5. Battista RN. Expanding the scientific basis of health technology assessment: A research agenda for the next decade. Int J Technol Assess Health Care. 2006;22:275–280.CrossRefGoogle Scholar
6. Battista RN, Feeny DH, Hodge MJ. Evaluation of the Canadian coordinating office for health technology assessment. Int J Technol Assess Health Care. 1995;11:102–116.CrossRefGoogle Scholar
7. Battista RN, Lance JM, Lehoux P, et al. Health technology assessment and the regulation of medical devices and procedures in Quebec: Synergy, collusion, or collision? Int J Technol Assess Health Care. 1999;15:593–601.CrossRefGoogle Scholar
8. Bennis, WG. Toward a truly scientific management: The concept of organization health in changing organizations. New York: Society for Organizational Learning; 1966.Google Scholar
9. Benson JK. The organizational network as a political economy. Adm Sci Q. 1975;20:229–249.CrossRefGoogle Scholar
10. Black, N. Public accountability: One rule for practitioners, one for scientists? J Health Serv Res Policy. 2001;6:131–132.CrossRefGoogle ScholarPubMed
11. Black, N. Evidence based policy: Proceed with care. Br Med J. 2001;323:275–279.CrossRefGoogle ScholarPubMed
12. Blancquaert I. Managing partnerships and impact on decision-making: The example of health technology assessment in genetics. Community Genet. 2006;9:27–33.Google Scholar
13. Berg, M, Van Der Grinten, T, Klazinga, N. Technology assessment, priority setting, and appropriate care in Dutch health care. Int J Technol Assess Health Care. 2004;20:35–43.CrossRefGoogle ScholarPubMed
14. Busse, R, Orvain, J, Velasco, M, et al. Best practice in undertaking and reporting health technology assessments: Working group 4 report. Int J Technol Assess Health Care. 2002;18:361–422.CrossRefGoogle Scholar
15. Bodeau-Livinec, F, Simon, E, Montagnier-Petrissans, C, et al. Impact of CEDIT recommendations: An example of health technology assessment in a hospital network. Int J Technol Assess Health Care. 2006;22:161–168.CrossRefGoogle Scholar
16. Brown, N, Micheal, M. From authority to authenticity: Governance, transparency and biotechnology. Health Risks Soc. 2002;4:259–272.CrossRefGoogle Scholar
17. Cameron, K. Effectiveness as paradox: Consensus and conflict in conceptions of organizational effectiveness. Manage Sci. 1986;32:539–553.CrossRefGoogle Scholar
18. Carlsson, P, Jorgensen, T. Scanning the horizon for emerging health technologies: Conclusions from a European workshop. Int J Technol Assess Health Care. 1998;14:695–704.CrossRefGoogle ScholarPubMed
19. Chinitz D. Health technology assessment in four countries: Response from political science. Int J Technol Assess Health Care. 2004;20:55–60.CrossRefGoogle Scholar
20. Cole DC, Robson LS, Lemieux-Charles L, et al. Quality of working life indicators in Canadian health care organizations: A tool for healthy, health care workplaces? Occup Med (Lond). 2005;55:54–59.CrossRefGoogle Scholar
21. Cookson, R, Maynard, A. Health technology assessment in Europe: Improving clarity and performance. Int J Technol Assess Health Care. 2000;16:639–650.CrossRefGoogle ScholarPubMed
22. Coulter A. Perspectives on health technology assessment: Response from the patient's perspective. Int J Technol Assess Health Care 2004;20:92–96.CrossRefGoogle Scholar
23. Culyer AJ. Involving stakeholders in healthcare decisions–the experience of the National Institute for Health and Clinical Excellence (NICE) in England and Wales. Healthc Q. 2005;8:56–60.CrossRefGoogle Scholar
24. Danish Centre for Evaluation and Health Technology Assessment (DACEHTA). External evaluation of DACEHTA. Copenhagen: National Board of Health; 2003.Google Scholar
25. Davies, L, Drummond, M, Papanikolaou, P. Prioritizing investments in health technology assessment: Can we assess potential value for money? Int J Technol Assess Health Care. 2000;16:73–91.CrossRefGoogle ScholarPubMed
26. Davies HTO, Nutley SM. Developing learning organizations in the new NHS. Br Med J. 2000;320:998–1001.CrossRefGoogle Scholar
27. Dixon, S, Coleman, P, Nicholl, J et al. , Evaluation of the impact of a technology appraisal process in England: The South and West Development and Evaluation Committee. J Health Serv Res Policy. 2003;8:18–24.CrossRefGoogle ScholarPubMed
28. Dodgson M. Organizational learning: A review of some literature. Organisational Studies. 1993;14:375–394.CrossRefGoogle Scholar
29. Douw, K, Vondeling, H, Oortwijn, W. Priority setting for horizon scanning of new health technologies in Denmark: Views of health care stakeholders and health economists. Health Policy. 2006;76:334–345.CrossRefGoogle ScholarPubMed
30. Draborg E, Gyrd-Hansen D, Poulsen PB, et al. International comparison of the definition and the practical application of health technology assessment. Int J Technol Assess Health Care. 2005;21:89–95.CrossRefGoogle Scholar
31. Drummond, M, Weatherly, H. Implementing the findings of health technology assessments: If the CAT got out of the bag, can the TAIL wag the dog? Int J Technol Assess Health Care. 2000;16:1–12.CrossRefGoogle ScholarPubMed
32. Drummond, M, Manca, A, Sculpher, M. Increasing the generalizability of economic evaluations: Recommendations for the design, analysis, and reporting of studies. Int J Technol Assess Health Care. 2005;21:165–171.CrossRefGoogle ScholarPubMed
33. Ehlers, l, Vestergaard, M, Kidholm, K et al. Doing mini-health technology assessments in hospitals: A new concept of decision support in health care? Int J Technol Assess Health Care. 2006;22:295–301.CrossRefGoogle ScholarPubMed
34. Eskola, J, Höckerstedt, K, Mäkäräinen, H et al. The future of FinOHTA – an external review. FinOHTA Report. Helsinki: Finnish Office of Health Care Technology Assessment; 2004.Google Scholar
35. Evers, S, Goossens, M, de Vet, H et al. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care. 2005;21:240–245.CrossRefGoogle ScholarPubMed
36. Farand, L, Champagne, F. An integrative model of emergency department performance based on social action theory. International Interdisciplinary Conference on Emergencies. Montreal, Canada; 2005.Google Scholar
37. Farmer, J, Chesson, R. Nice lessons to be learned. Int J Technol Assess Health Care. 2001;17:222–235.CrossRefGoogle ScholarPubMed
38. Garcia-Altes A, Ondategui-Parra S, Neumann PJ. Cross-national comparison of technology assessment processes. Int J Technol Assess Health Care. 2004;20:300–310.CrossRefGoogle Scholar
39. Giacomini, M, Miller, F, Browman, G. Confronting the “grey zones” of technology assessment: Evaluating genetic testing services for public insurance coverage in Canada. Int J Technol Assess Health Care. 2003;19:301–316.CrossRefGoogle Scholar
40. Guisset, A-L, Sicotte, C, Leclercq, P et al. , W. Définition de la performance hospitalière: une enquête auprès des divers groupes d'intérêts au sein des hôpitaux. Sci Soc Sante. 2002;20:65–104.CrossRefGoogle Scholar
41. Haddad, S, Roberge, D, Pineault, R. Comprendre la qualité: en reconnaître la complexité. Ruptures – Revue transdisciplinaire en santé. 1997;4:59–78.Google Scholar
42. Hailey, D. Elements of effectiveness for health technology assessment programs. HTA Initiative #9. Edmonton: Alberta Heritage Foundation for Medical Research; 2003.Google Scholar
43. Hailey D. Toward transparency in health technology assessment: A checklist for HTA reports. Int J Technol Assess Health Care. 2003;19:1–7.CrossRefGoogle Scholar
44. Hailey, D, Juzwishin, D. Managing external risks to health technology assessment programs. Int J Technol Assess Health Care. 2000;22:429–435.CrossRefGoogle Scholar
45. Hanney SR. The utilization of health research in policy making: Concepts, examples and methods of assessment. Health Res Policy Syst. 2003;1:2–30.CrossRefGoogle Scholar
46. Hivon, M, Lehoux, P, Denis, JL et al. , Use of health technology assessment in decision making: Co-responsibility of users and producers? Int J Technol Assess Health Care. 2005; 21:268–275.CrossRefGoogle Scholar
47. INATHA. Available at: http://www.inatha.org.Google Scholar
48. Institute, L.B. Annual Report 2006. In: Ludwig Boltzmann Gesellschaft, editor. Vienna: Ludwig Boltzmann Institute – Health Technology Assessment; 2006.Google Scholar
49. Jacob, R, McGregor, M. Assessing the impact of health technology assessment. Int J Technol Assess Health Care. 1997;13:68–80.CrossRefGoogle ScholarPubMed
50. Jacob, R, Battista, RN. Assessing technology assessment: Early results of the Quebec experience. Int J Technol Assess Health Care. 1993;9:564–572.CrossRefGoogle ScholarPubMed
51. Kuruvilla, S, Mays, N, Pleasant, A et al. , Describing the impact of health research: A research impact framework. BMC Health Ser Res. 2006;6:134.CrossRefGoogle ScholarPubMed
52. Landry, R, Amara, N. The impact of transaction costs on the institutional structuration of collaborative academic research. Res Policy. 1998;27:901–914.CrossRefGoogle Scholar
53. Lavis JN, Ross S, McLeod CB. Measuring the impact of health research. J Health Serv Res Policy. 2003;8:165–170.CrossRefGoogle Scholar
54. Lavis JN. A political perspective on evidence-based decision-making. In: Using knowledge in health care: Multidisciplinary perspectives. Toronto: University of Toronto Press; 2004:70–85.Google Scholar
55. Lehoux, P, Battista, RN, Granados, A et al. , International Master's Program in health technology assessment and management: Assessment of the first edition (2001-2003). Int J Technol Assess Health Care. 2005;21:104–112.CrossRefGoogle ScholarPubMed
56. Lehoux, P, Tailliez, S, Denis, JL et al. , Redefining health technology assessment in Canada: Diversification of products and contextualization of findings. Int J Technol Assess Health Care. 2004;20:325–336.CrossRefGoogle ScholarPubMed
57. Leys, M. Health technology assessment: The contribution of qualitative research. Int J Technol Assess Health Care. 2003;19:317–329.CrossRefGoogle ScholarPubMed
58. Martin, DK, Giacomini, M, Singer, PA. Fairness, accountability for reasonableness, and the views of priority setting decision-makers. Health Policy. 2002;61:279–290.CrossRefGoogle ScholarPubMed
59. Minvielle, É, Sicotte, C, Champagne, F et al. , Hospital performance: Competing or shared values? Health Policy. In press.Google Scholar
60. Maynard, A, McDaid, D. Evaluating health interventions: Exploiting the potential. Health Policy. 2003;63:215–226.CrossRefGoogle ScholarPubMed
61. McGregor M, Brophy JM. End-user involvement in health technology assessment (HTA) development: A way to increase impact. Int J Technol Assess Health Care. 2005;21:263–267.CrossRefGoogle Scholar
62. McKee, M, Figueras, J, Lessof, S. Research and policy: Living on the interface. Eurohealth. 2006;12:26–29.Google Scholar
63. Mears R, Taylor R, Littlejohns P, et al. A review of international health technology assessments (IHTA). London: National Institute of Clinical Excellence; 2000.Google Scholar
64. Menon, D, Marshall, D. The internationalization of health technology assessment. Int J Technol Assess Health Care. 1996;12:45–51.CrossRefGoogle ScholarPubMed
65. Menon, D, Fung, F, Harstall, C et al. , The development of a health technology assessment program: The case of Alberta. Int J Technol Assess Health Care. 1995;11:93–101.CrossRefGoogle ScholarPubMed
66. Mowatt, G, Bower, DJ, Brebner, JA et al. , When is the ‘right’ time to initiate an assessment of a health technology? Int J Technol Assess Health Care. 1998;14:372–386.CrossRefGoogle ScholarPubMed
67. Mowatt, G, Grant, AM, Bower, DJ et al. , Timing of assessment of fast-changing health technologies. In: Stevens, A, Abrams, K, Brazier, J et al. , eds. The advanced handbook of methods in evidenced-based health care. London: Sage Publications; 2001;471–484.CrossRefGoogle Scholar
68. Nutley, SM, Davies, HTO. Developing organizational learning in the NHS. Med J. 2001;35:35–42.Google ScholarPubMed
69. Parsons, T. The social system. New York: Free Press; 1951.Google Scholar
70. Parsons, T. Social systems and the evolution of action theory. New York: Free Press; 1977.Google Scholar
71. Pelz, DC. Some expanded perspective on the use of social science in public policy. In: Yiner, M, Cutler, SJ, eds. Major social issues: A multidisciplinary view. New York: Free Press; 1978:346–357.Google Scholar
72. Price, JL. The study of organizational effectiveness. Sociol Q. 1972;13:3–15.CrossRefGoogle Scholar
73. Raftery, J. NICE: Faster access to modern treatments? Analysis of guidance on health technologies. Br Med J. 2001;323:1300–1303.CrossRefGoogle ScholarPubMed
74. Ross, J. The use of economic evaluation in health care: Australian decision makers” perceptions. Health Policy. 1995;31:103–110.CrossRefGoogle ScholarPubMed
75. Scott, T, Mannion, R, Davies, HT, Marshall, MN. Implementing culture change in health care: Theory and practice. Int J Qual Health Care. 2003;15:111–118.CrossRefGoogle ScholarPubMed
76. Scott, T, Mannion, R, Davies, H, Marshall, M. The quantitative measurement of organizational culture in health care: A review of the available instruments. Health Serv Res. 2003;38:923–945.CrossRefGoogle Scholar
77. Scott, T, Mannion, R, Marshall, M, Davies, H. Does organisational culture influence health care performance? A review of the evidence. J Health Serv Res Policy. 2003;8:105–117.CrossRefGoogle ScholarPubMed
78. Sculpher, M, Drummond, M, Buxton, M. The iterative use of economic evaluation as part of the process of health technology assessment. J Health Serv Res Policy. 1997;2:26–30.CrossRefGoogle ScholarPubMed
79. Senge, PM. The fifth discipline: The art and practice of the learning organization. New York: Currency, Doubleday; 1994.Google Scholar
80. Sicotte, C, Champagne, F, Contandriopoulos, AP et al. , A conceptual framework for the analysis of health care organizations’ performance. Health Serv Manage Res. 1998;11:24–48.CrossRefGoogle ScholarPubMed
81. Sicotte, C, Champagne, F, Contandriopoulos, AP. La performance organisationnelle des organismes publics de santé. Ruptures - Revue transdisciplinaire en santé. 1999;6:34–46.Google Scholar
82. Smirchich L. Concepts and culture in organizational analysis. Adm Sc Q. 1983;28:328–358.Google Scholar
83. Tsui AS. A multiple-constituency model of effectiveness: An empirical examination at the human resource subunit level. Adm Sc Q. 1988;35:458–483.Google Scholar
84. Vallesco-Garrido, M, Busse, R. Health technology assessment: An introduction to objectives, role of evidence and structure in Europe. European Observatory on Health Systems and Policies. Policy brief series. Copenhagen; 2005.Google Scholar
85. Wanke, MI, Juzwishin, D, Thornley, R. An exploratory review of evaluations of health technology assessment agencies. Alberta: Alberta Heritage Foundation for Medical Research-Health Technology Assessment Unit; 2006.Google Scholar
86. Yuchtman E, Seashore ES. A system resource approach to organization effectiveness. Am Sociol Rev.1967;32:891–903.CrossRefGoogle Scholar
87. Zarcadoolas, C, Pleasant, A, Greer, D. Health literacy: An expanded model. Health Promot Int. 2005;20:195–203.CrossRefGoogle ScholarPubMed
Supplementary material: File

Lafortune supplementary material

Lafortune supplementary material

Download Lafortune supplementary material(File)
File 78.3 KB