Expertise, experience, and excellence. Twenty years of patient involvement in health technology assessment at NICE: an evolving story

Abstract

From its inception in 1999, the National Institute for Health and Care Excellence (NICE) committed to including the expertise, experiences, and perspectives of lay people, patients and carers, and patient organizations in its health technology assessments (HTAs). This is our story of patient involvement in HTA: from early methods designed for use when assessing medicines, widening to address the different requirements of HTAs for interventional procedures, medical technologies, and diagnostic technologies. We also chart the evolution and development of all our patient involvement methods over the past 20 years through regular evaluation and by responding to external challenge. However, we know that processes and methods alone are not enough. Through case studies we demonstrate the value of patient involvement in HTA and highlight the unique perspectives and experiences that patients bring to HTA committees. Finally, we discuss the underpinning principles and commitments that have made NICE a world leader in delivering meaningful and legitimate patient involvement.

Introduction

The National Institute for Health and Care Excellence (NICE) was founded in 1999 with a remit to create consistent clinical guidelines and to end “postcode rationing” (1) by providing guidance on new drugs and technologies for use in the National Health Service (NHS) in England.

NICE's first health technology assessment (HTA) provided guidance on the extraction of wisdom teeth (1)—guidance which remains active to this day. Twenty years on “technology appraisals” as they are known remain high profile, using methods and processes that have contributed significantly to the international development of HTAs.

Involving people with lived experience of the health condition or treatment under consideration, and their carers and families, has been a core principle (2) in our HTA development. This ensures that our decision making and guidance reflects their needs and recognizes the outcomes they value most.

In this commentary, we chart the evolution of patient involvement in HTAs at NICE over the past 20 years. We discuss how the role of patient evidence and lay membership of the committees who develop NICE guidance has evolved, and the value and impact that has had. We have sought to uphold the values and standards for patient involvement in HTA (3), which is reflected in the progress we have made.

Developing the Role of Individuals in HTA

Lay people as Decision Makers

Our initial method of patient involvement in HTA was to recruit lay members to our HTA committees. This follows best practice (4), and lay membership of NICE committees has remained a constant throughout NICE's work. Lay members bring the perspectives of people who use health services to our HTA decision making. They do not bring condition-specific expertise (although they may have that experience) but take a general view on the topic at hand, reflecting on the patient evidence presented to them and highlighting that evidence in committee discussions.

In 2001, having one lay member involved with NHS committees was seen as good practice (5). From its inception NICE went further than this, stating every committee at NICE will have a minimum of two lay members, as full voting members of the committee, “who play a crucial role by providing a patient/carer/public perspective to the discussions and decisions taken” (6).

Lay members are recruited via an advertisement on the NICE website. They complete an application form detailing their relevant experience and knowledge, as well as declaring any interests which might conflict with the work of the committee. As with all committee members, lay members undergo shortlisting and interview in order to be appointed to a NICE committee. Lay members sit alongside the other members (academic, health professional, and management) as equals with full voting rights.

Enhancing the Role of Lay Members

In 2008, we piloted expanding the HTA committee lay member remit by incorporating a lay member as a member of the “lead team” for each topic that technology appraisal committees were considering. The lead team comprised three committee members who focused on clinical effectiveness, cost-effectiveness, and patient and carer evidence, respectively (7). Having a lay member focusing specifically on patient and carer evidence meant that this evidence was treated with parity alongside the other evidence. The lay members took turns being the “lay lead” for each topic.

As part of the lead team, the lay members participated in work before committee meetings. They also presented evidence summarized from patient organization submissions at committee meetings (8). The role helped lay members to be more engaged and informed about the topic and the evidence they were considering, and be seen as an integral part of the team.

In 2010 and 2012, we evaluated this enhanced lay member role. Semi-structured interviews with lay members and committee chairs in 2010 identified five key themes: overall experiences, practicalities, expectations, potential improvements, and impact. All agreed the pilot should become routine procedure. Improvements included: setting out expectations of the role more clearly, especially during new lay member inductions; and enhancing the template that patient organizations used to submit patient evidence to give the lay member taking part in the lead team better information to draw from.

The 2010 themes informed the 2012 survey sent to all committee members (n = 117; 88 responses, including all lay members and all chairs). Key findings corroborated 2010 results. The presence of the lay member on the lead team impacted both visibility of patient evidence: 66 percent believed it increased the committee's awareness of patient issues and 74 percent said it improved how NICE addressed patient issues. There was also an impact on the lay members themselves—especially their understanding of topics—and engagement within the committee: “the climate within which a lay person works has greatly improved” [lay member] (unpublished data, 2010).

In addition to their role on NICE committees, lay members also help us to develop HTA methods and processes, and our approach to patient involvement. They act as a critical friend, working collaboratively with us to address issues either as part of working groups or individually, and provide feedback to us via an exit survey when their work with us comes to an end.

The Challenges of Diagnostic Assessments—a New Approach

In 2009, NICE expanded its HTAs to include evaluations of medical and diagnostic technologies (1).

Understanding the long-term clinical and cost-effectiveness of diagnostic technologies presented a challenge for NICE given that the impact of a technology on disease progression, for example, may not be known for some time after the technology is used. Extensive modeling of post-diagnostic care pathways was required to understand how the information provided by a diagnostic technology influenced care and outcomes, compared to standard care.

In addition to the two committee lay members, a patient (or carer) with specific knowledge of the condition or technology being considered is recruited to the committee for that specific topic with full voting rights as a committee member. Patients provide input at the early stages of assessment by participating in scoping workshops and offering their experiences of using a technology (if applicable) or identifying the key clinical and quality of life outcomes for patients which may not be captured in research evidence. The patient who has been appointed to the committee helps shape the technical assessment, including identifying and reviewing the evidence related to the technology, and presents patient evidence at the committee meetings. Patients are supported in this via an induction meeting, an information pack, and support from NICE's Public Involvement Programme and the committee team.

Case Study 1: Putting Diagnostic Technology in the Patient's Hands. Atrial Fibrillation and Heart Valve Disease: Self-Monitoring Coagulation Status Using Point-of-Care Coagulometers (the CoaguChek XS System)

The contributions of patients can be particularly important when the technology being assessed is used directly by patients themselves. Point-of-care coagulometers for people with atrial fibrillation or heart valve disease who take a vitamin K antagonist are an example of this. Patients “provided information on the benefits of self-monitoring on psychological wellbeing in having a sense of control over the condition, reducing the need to attend clinics or hospital, allowing patients to travel, visit and care for other family members” (9).

Contributing Evidence and Views

The Role of the Patient Expert

While lay members take a generalist view across many topics, we need to ensure that the experiences, views, and perspectives of people affected by the specific condition or technology that NICE is considering are captured. Since 2001, NICE has invited two individuals (known as patient experts) with that experience to submit a written statement and give testimony at committee meetings for medicines HTAs (8). Patient organization stakeholders nominate and support the patient experts with additional support being provided by NICE. Ideally patient experts balance the breadth and depth of experience of patients with a particular condition. For example, one patient expert may have experience as a member of a patient organization, bringing a range of perspectives, and another may have direct personal experience of the intervention being considered, or a comparator.

Case Study 2: Vedolizumab for Treating Moderately to Severely Active Ulcerative Colitis

The importance of including personal experiences of an intervention or condition was keenly illustrated when assessing vedolizumab for treating moderately to severely active ulcerative colitis (10). Two patient experts with ulcerative colitis were able to convey to the committee the very real difficulties, concerns, and anxieties experienced by patients, and the potential benefit of vedolizumab. These included:

1. (1) Effect on quality of life: being unable to work, needing to remain at home and often needing hospitalization;

2. (2) The impact of ulcerative colitis particularly on young people: affecting “their ability to study, find work, socialise and find a partner” (11); and

3. (3) Concerns around surgery to treat ulcerative colitis: the patient experts acknowledged that surgery was likely to be required at some point in their lives but noted concerns “on fertility, its irreversibility, its risks and the potential for a life-long impact on lifestyle” (11), meaning that options to delay surgery were valued by patients.

From the strength of the patient evidence NICE's HTA committee “concluded that a drug treatment that improves or brings the disease into remission would have a major effect on quality of life, and that avoiding surgery was important to people with ulcerative colitis” (11).

Improving How We Work With Patient Experts

We regularly gather feedback from patient experts about their experiences of working with us, to improve how we support them and provide the information they need to participate fully in HTA committee meetings. Evaluation projects, in 2008 (n = 61; 44 responses) and 2012 (n = 62; 49 responses), both recorded patient experts’ overall experience as either “excellent” or “good” in over 80 percent of responses, some “fair”, a few “poor”, and no “awful” responses. One respondent said they “felt [their] presence worthwhile not just tolerated. My contribution listened to with interest. Treated with consideration” [Patient Expert]. Other respondents however reflected a more negative experience: "I do not think that meetings are designed for really listening to patients or patient experts. For this reason I did not feel that I had the opportunity to raise some of the issues that should have been; and that NICE themselves state they want to hear" [Patient Expert].

The 2008 project identified the need for greater involvement of lay people in HTA processes (see above), and other improvements were also taken forward. Since 2012, we have sought feedback immediately after people's participation, to enable a more timely response to issues raised than is possible with large retrospective evaluations. The reported experiences of patient experts have continued to improve, with 96 percent (n = 26) of patient experts stating that their experience was “good” or “excellent” in 2019–2020.

In 2014, we started to involve patient experts in our early dialog (Scientific Advice) program. NICE advises sponsor companies on their evidence generation plans during clinical trial stages, aiming to ensure the best evidence is then available for regulators and HTAs later. Involving patients, or their advocates, during products’ clinical development allows patients to identify outcomes most important to them and ensure acceptability of clinical trial protocols.

Patient Involvement for Novel Interventional Procedures

In addition to medicines HTAs, NICE has developed HTA guidance on novel interventional procedures since 2003 (1). Interventional procedures guidance (IPG) considers the safety and efficacy of procedures used for diagnosis or treatment, and can include things such as making an incision to gain access to the inside of a patient's body; gaining access to a body cavity without cutting; or using electromagnetic radiation.

The procedures considered are normally new, and not yet established in clinical practice. This gives interesting challenges for evidence collection and interaction with stakeholders, as our experience suggests that knowledge and awareness is generally lower than for that of medicines. In the early years of the program, the main patient inputs were comments made at draft guidance consultation, by patient organizations and individual patients and carers.

In 2006, noting the unique insight that patients with experience of specific procedures have (12), NICE enhanced the input of patient evidence to IPGs by seeking their views to directly inform the committee's decision making.

This insight is obtained through anonymous questionnaires, tailored to each topic, distributed to patients via clinicians performing the procedure, and more recently, patient organizations. Questionnaires are returned directly to NICE and a summary report produced for consideration by the committee. Input from patients is not sought for procedures used only in research, and when it is sought it is not always possible to obtain it. An evaluation in 2014 showed NICE obtained patient input for twenty-four topics (69 percent of suitable topics). This input was valuable to cross-reference with other evidence sources considered by the committee, obtain patient quality of life information, and alert the committee to issues not raised in other types of evidence.

The Right Method for the Job—the Challenge of Medical Technologies

Medical technologies in this context include medical devices, active medical devices (powered devices), active implantable medical devices, and in vitro diagnostic medical devices (13).

Approaches to patient involvement needed to adapt to the new challenges posed by medical technologies. We recognized at an early stage that some medical technologies being evaluated would not lend themselves readily to meaningful involvement of individual patients. These are particularly those medical devices that a patient would be unaware of, for example, devices used during surgery where a patient is unconscious, where the main benefit of the device is to the user (such as a surgeon) or where the patient is seriously ill (such as esophageal Doppler monitors). Conversely, there are medical devices about which the views and experiences of individual patients are critical to our understanding of the benefits or disbenefits of the device. These can be devices that are used by patients or their carers themselves, devices which have a direct impact on a person's quality of life, or devices which have clinical outcomes of importance to patients.

We took a flexible approach to individual patient involvement, securing patient expert input or using patient surveys where we and our committees thought that input could be meaningful.

Case Study 3: PleurX Peritoneal Catheter Drainage System for Vacuum-Assisted Drainage of Treatment-Resistant, Recurrent Malignant Ascites

NICE published guidance about the use of the PleurX peritoneal catheter drainage system in March 2012. The PleurX system allows for drainage of malignant ascites from the patient's peritoneal cavity, outside of a hospital setting, usually in the patient's home. Standard care at the time was large volume paracentesis performed in an outpatient clinic. Given the outcomes of using this technology for patients would be predominantly around comfort and quality of life, NICE decided that hearing directly from a patient expert about their experiences of ascites would be valuable.

Supported by a patient organization, the patient expert told our committee about their personal experience of ascites and the discomfort and body image issues (14) associated with waiting for fluid build-up sufficient to require large volume paracentesis (usually in the region of 5l). The final guidance noted that “improvement in quality of life is mainly a result of avoiding regular hospital visits and inpatient stays associated with large-volume paracentesis, and alleviation of symptoms associated with massive ascites through the frequent drainage of small volumes of ascitic fluid” (15).

Case Study 4: gammaCore for Cluster Headache

gammaCore is a noninvasive device which is held against the patient's neck. It uses an electric current to stimulate the vagus nerve to try and treat, and prevent, cluster headaches. It is used by a patient or a carer directly and is intended to be used multiple times per day (16). For NICE's evaluation of gammaCore, the experiences of people who had used the device directly were crucial, therefore we applied the questionnaire-based methods used in the development of IPGs to capture a range of views.

Our online survey received 80 responses—46 from people with chronic cluster headaches and 12 from people with episodic cluster headaches. All had used gammaCore. Of the 58 people who indicated that they had used gammaCore to prevent headaches, 48 respondents agreed that gammaCore had reduced the frequency or severity of their headaches, with 10 people disagreeing (17). The committee considerations noted the “life changing effects” (16) of using gammaCore for many patients who submitted their experiences.

Evidence from the patient survey also contributed insights into potential cost savings of using gammaCore, noting that “more than half the people in the patient survey had reduced their sumatriptan use since starting treatment with gammaCore” (16). This reduced medication use was another key driver of the cost savings for gammaCore in the economic model.

These examples demonstrate the benefits and impact of taking a flexible approach to patient involvement in medical technology evaluations, tailoring the approach to the technology under consideration and the evidence HTA committees needed from patients.

The Role of Patient Organizations in HTA

Patient organizations have played a crucial role in NICE's HTA activities from the inception of NICE. Patient organizations who participate in HTAs are usually charities regulated by the Charity Commission in England and Wales. They bring the perspectives, experiences, and evidence from the patient community that they represent and also draw on the individual experiences of people living with a particular condition or disease.

NICE invites patient organizations with an interest in the disease area or intervention being considered in an HTA to participate in the assessment (Figure 1). Although methods vary depending on the type of HTA being conducted, patient organizations can contribute to:

1. (1) scope consultations or workshops at the beginning of an HTA;

2. (2) submitting patient evidence for consideration by NICE HTA committees;

3. (3) nominating patient experts to give their individual experiences and perspectives at committee meetings;

4. (4) commenting on draft HTA guidance, and appealing final guidance for medicines HTAs; and

5. (5) supporting and helping to put final guidance into practice.

Figure 1. Stages of patient involvement at NICE. Diagram showing the types of patient involvement at the different stages of a typical medicines HTA. HTA, health technology assessments; NICE, National Institute for Health and Care Excellence.

Patient organizations also play a broader role by acting as a critical friend to NICE. Over NICE's 20-year history this has included commenting on NICE process and method guides and working collaboratively with NICE to improve how we involve patients and carers in our work (18).

Responding to Feedback: Improving How We Work With Patient Organizations

We understand that, for patient organizations, working with NICE has not always been easy. In 2012, a representative from a patient organization gave evidence to the Health Select Committee (19) (of Parliament) noting uncertainty about what NICE is asking for when we invite submissions from patient organizations and for evidence from patient experts. In 2015, the Triennial Review of NICE noted that “overall there was a sense from [patient organization] stakeholders that patients felt second-best, that their input was not valued as highly as that of clinicians or specialist stakeholders and that NICE could do more to support patients who do get involved” (20).

In response, NICE has iteratively improved the resources and advice we give to patient organizations and evaluated the effectiveness of those changes. In response to the Health Select Committee recommendations, we updated the template patient organizations are asked to complete to submit evidence for medicines HTAs. In 2016 we evaluated this, using three parallel online surveys: patient organizations who had used the updated template, the committee lay members who had used submissions from patient organizations using the new template in their lead team role, and other committee members and chairs.

The survey results showed that a new section in the template was really useful. This asked the person completing the template to highlight five key points to draw to the attention of the HTA committee. Respondents said the template was generally easy to use and understand for both organizations completing it and decision makers. Committee members felt that the revised template had driven up the quality of patient evidence submissions.

Challenges remained for patient organizations in finding the evidence to contribute to the submissions; for decision makers in terms of the variability in the submissions; and for all in developing a shared understanding of the purpose and utility of patient evidence.

NICE has continued addressing these challenges, most recently in seeking comment from committee members about the impact of patient evidence for our Highly Specialized Technologies' HTA program. This assesses treatments for very rare diseases. A critical element of this work involved “completing the feedback loop.” We drew on the experience of our colleagues at the Canadian Agency for Drugs and Technologies in Health (CADTH) to develop feedback letters (21) for patient organizations about where their evidence submission had an impact or revealed new information (22). By providing this feedback, patient organizations can see how the HTA committees have used their evidence and can focus on key areas in future submissions.

In recent years, we have further enhanced patient input into Interventional Procedures assessments by introducing submissions of evidence from patient organizations, using a template proforma, tailored to each topic.

As well as improving and evaluating our own resources, NICE has contributed learning, expertise, and staff resource to helping to develop international patient organization evidence submission templates for medicines, non-medicines (MedTech), and diagnostics HTAs (23), through our membership of the Health Technology Assessment International Patient and Citizen Interest Group.

In 2019, we further expanded our engagement with patient organizations by using a co-production approach to improving patient involvement in NICE HTAs. A core patient working group oversaw the project, and we captured a broad range of patient organizations’ views, experiences, and suggestions for improvement through an event and survey (unpublished data, 2020).

Supporting Patient and Public Involvement in HTA

Since 2002, NICE has had a dedicated team providing advice on effective patient involvement methods; recruiting and supporting individuals who work with us; identifying and supporting patient organizations through the course of an HTA; and providing training specifically for lay people in line with best practice (1;5). We provide a named team member for each HTA. This person works directly with patient organizations and patient experts—the value and importance of this named contact is consistently reflected in exit survey responses and anecdotal feedback.

We know that principles alone are not enough, therefore in pursuit of continual quality improvement, development, and evaluation, we have periodically taken a step back to review the entirety of our patient involvement activities. In 2016, we conducted a strategic review of patient and public involvement at NICE. A literature review and a stakeholder survey informed a public consultation (24) on key areas for improvement.

In 2019 (25), we mapped our patient involvement methods and processes across the different types of HTA at NICE, to identify where we could improve consistency and when patient involvement could be most effective. We found starting patient involvement early in the development process and continuing throughout increased effectiveness of patient organizations’ contributions. Meeting with patient organizations and experts prior to a committee meeting, to discuss their evidence submission, was also thought to improve NICE's use of patient evidence. We are testing this assertion now, having implemented such a meeting in medicines HTAs.

Conclusions

Our experience over 20 years has demonstrated that NICE is a world leader in patient involvement in HTA. Through continual development, expansion, and evaluation of our patient involvement methods in HTA, we have sought to ensure that patient involvement in HTA at NICE is comprehensive, effective, and ensures that the views and experiences of those most affected by the technologies and conditions we are considering are strongly heard. As NICE has grown and evolved, our involvement methods have too. We continue to regularly evaluate our methods to ensure they remain fit for purpose. We have demonstrated the value and impact of patient involvement in HTA, including case studies and feedback from participants.

Our experience of designing and implementing patient involvement methods in HTA at NICE over 20 years has provided much learning which, by sharing our evolution, we hope will be of value to other HTA agencies undertaking similar work. We have demonstrated that patient involvement in HTA can be done comprehensively and at scale, and that it can have a measurable impact, but there is always room for improvement. We look forward to entering our third decade with new patient involvement initiatives and even greater impact.