Strengthening patient outcome evidence in health technology assessment: a coproduction approach

Abstract

The National Institute for Health and Care Excellence (NICE) worked with patients and staff from six patient organizations to review existing health technology assessment (HTA) methods and coproduce proposals to improve the following: patient involvement, how patient evidence is identified and considered by committees, and the support offered to patient stakeholders. This engagement identified important factors that HTA bodies need to understand to enable meaningful patient and public involvement (PPI), such as having clearly documented processes, appropriate evidence submission processes, transparent decisions, and suitable support. This work demonstrated the benefits of HTA bodies working collaboratively with patient stakeholders to improve PPI. By doing so, HTA bodies can increase their knowledge and understanding of the barriers faced by patient stakeholders to develop appropriate solutions to remove them. The coproduction approach improved stakeholder engagement methods, provided a better analysis of data, supported the development of meaningful conclusions, and improved stakeholder relationships.

Introduction

The National Institute for Health and Care Excellence (NICE) develops evidence-based recommendations to improve outcomes for people using the National Health Service (NHS) and other services in England. Patient and public involvement (PPI) is a core principle governing NICE's work (1), reflecting policy initiatives to involve patients, service users, carers, and the public across the NHS and social care in England (2–5). PPI ensures that those with lived experience of a health condition or health technology can contribute valuable information such as benefits, risks, and side effects to improve health outcomes (6). This information enables health technology assessment (HTA) committees to understand the impact a technology has for patients and the outcomes important to them.

In December 2018, the Government of the United Kingdom (UK) and the Association of the British Pharmaceutical Industry (ABPI) agreed the Voluntary Scheme for Branded Medicines Pricing and Access (7). This committed NICE to reviewing its HTA methods and processes with the aim of

1. (1) speeding up patient access to new and promising health technologies,

2. (2) supporting better market access, and

3. (3) simplifying the HTA process.

HTA bodies require information and expertise from a range of stakeholders to develop guidance (8). One key challenge is designing meaningful approaches to involving stakeholders and patients (9), ensuring that PPI is not tokenistic but adds value. Effective PPI approaches cannot be developed by HTA bodies in isolation; instead, they should be coproduced with stakeholders. This approach (10) encourages equal partnership, using different types of knowledge and skills, such as lived experience and professional learning. This ensures that approaches are mutually appropriate and beneficial (8).

To ensure the review reflected PPI best practice, a patient working group (PWG) was established. The role of the PWG was to review NICE HTA PPI methods and processes and coproduce recommendations to improve the following: PPI, how patient evidence is identified and considered by committees, and the support offered to patient stakeholders.

Methods

The Patient Working Group

The PWG consisted of representatives from six UK patient organizations (11). Representatives included patient experts with direct lived experience (N = 2), patient experts without direct lived experience (N = 3), and patient organization staff (N = 5). The representatives were invited to join the PWG due to their experience working with NICE to develop HTA guidance, such as sharing their expertise as a patient at committee meetings, being a committee lay member, and submitting patient evidence.

To address potential power imbalances between NICE and patient organization representatives, the PWG agreed for equal membership and equal voting rights between all members.

The PWG designed a strategy to engage with patient stakeholders to review and improve PPI in NICE HTAs. The PWG held five meetings between January 2019 and September 2019, as shown in Figure 1.

Figure 1. Approach taken by the PWG. Diagram showing the PWG's approach to reviewing NICE HTA methods.

Stakeholder Event

Thirty patient stakeholders who had contributed to NICE HTAs were invited to a stakeholder event in January 2019. The aims of the event were to identify issues with current PPI processes and methods, understand the limits and parameters of the broader HTA processes, identify learning from other HTA bodies, and codesign proposals for improved involvement. This information was elicited through presentations by patient stakeholders and in group discussions facilitated by an independent consultant. A report and thematic analysis was produced summarizing the learning from the day. This report was checked for accuracy with the attendees and is available upon request.

Stakeholder Survey

The PWG designed an online survey using the thematic analysis from the stakeholder event to anonymously capture views from patient stakeholders. The survey was developed using Snap Surveys (Build 11.28.0) and made available in Microsoft Word 2016 (Version 16.13) format. The survey covered the following eleven themes:

1. (1) experience of working with NICE (one item),

2. (2) confidence in performing different aspects of the process (nine items),

3. (3) involvement (four items),

4. (4) support and training for patient stakeholders (five items),

5. (5) support and training around evidence submissions (eight items),

6. (6) support and training for patient experts (six items),

7. (7) information collection and synthesis (nine items),

8. (8) committee chairing (three items),

9. (9) committee culture change (five items),

10. (10) committee decision making (seven items), and

11. (11) impact and feedback (six items).

To limit bias, an open question was added after each theme to elicit suggestions for improvement. A final open question was included to gather areas for further improvement and ideas for solutions. A five-point Likert scale (“1”—strongly disagree, to “5”—strongly agree) was used for all themes except for theme two which included “1”—not confident at all, to “5”—very confident. The survey is available on request.

The survey was e-mailed to 631 patient stakeholders who had previously participated in a NICE HTA. The survey was promoted via social media and at patient stakeholder events. The survey was live for 4 weeks, with an e-mail reminder sent after 2 weeks.

Engagement Results

Stakeholder Engagement Event

Eighteen (60% of those invited) patient stakeholders attended the event. The key themes identified for improving how NICE finds and uses patient evidence included the following:

1. (1) providing clear information on how patient evidence fits into health economic modeling and influences committee decisions,

2. (2) providing training and support for participants to increase their understanding of the HTA process and submitting evidence, and

3. (3) providing templates, tools, and resources to enable patient stakeholders to provide effective patient evidence.

Stakeholder Survey

Fifty-two (8%) responses were received. The PWG developed eighteen recommendations and fifty-nine proposals to improve PPI approaches. Four main themes emerged:

Documenting processes: Ensure that processes are written in plain English, with opportunities for PPI clearly marked. This enables patient stakeholders to understand their role and contribution to the HTA, and to explain the process to their members.

Presenting evidence: Have a range of methods for presenting patient information at committee meetings, including:

1. (1) reviewing when information is presented during a committee meeting, for example, before or after clinical and economic evidence

2. (2) identifying, with patient stakeholders, the needs and appropriate method of participation for patient experts providing evidence. For example, using video conference or video submissions for topics involving end-of-life patients or where participants have reduced mobility.

Transparency of decisions: Provide feedback to patient stakeholders explaining how their evidence submission influenced the HTA. This increases transparency and provides patient stakeholders with a clearer understanding of what information is required for future involvement.

Support: Provide tailored support and structured training to help participants navigate and understand HTA documents and processes. For example:

1. (1) having a named support contact in the HTA organization,

2. (2) hosting regular webinars,

3. (3) providing e-learning modules,

4. (4) providing information about committee members, logistics, and room layout, and

5. (5) enabling patient experts to speak with lay committee members before the meeting.

Discussion

This article describes a novel coproduction approach to improving PPI in HTAs using a PWG. Several important themes (e.g., presenting evidence) emerged from the PWG's engagement with stakeholders, which support published literature (6;12). Using this information, the PWG reviewed existing NICE HTA methods and coproduced recommendations to improve PPI. The coproduction approach with external stakeholders and NICE enabled the PWG to utilize the knowledge and experience of all parties, ensuring that proposals were mutually appropriate and beneficial.

Although there is no single best practice coproduction method, there are principles to help support the practice (13), which were considered in the PWG approach. Equal membership enabled patient stakeholders to have shared planning and decision-making responsibilities with NICE, which promoted empowerment and coproduction between all members. Roles and responsibilities were shared between NICE and patient stakeholders, which enabled the group to share, control, and develop mutual responsibilities.

There were many advantages to the approach used by the PWG which correlate with existing research (14):

Better Engagement

The expertise of the PWG improved engagement. It enabled the process to be more inclusive by

1. (1) identifying and engaging with key patient stakeholders,

2. (2) identifying and incorporating variables into the engagement time frame, such as stakeholders’ other work demands,

3. (3) ensuring that the language used in communications was framed correctly using plain English, and

4. (4) ensuring that communications accounted for stakeholder needs and knowledge. This was important for the framing of the online survey to ensure that respondents understood the questions to generate accurate responses.

Better Analysis

Multiple perspectives in the PWG enabled the data to be better analyzed and understood. Patient stakeholders helped NICE to understand the issues and their reasons. For example, understanding stakeholders’ additional commitments, their varying resources, and their differing knowledge and confidence helped explain the range of opinions gathered. In turn, NICE was able to provide a rationale to current processes or methods and the requirements for a committee to reach a decision.

Meaningful Conclusions

The coproduction approach enabled the PWG to develop realistic, achievable, and mutually beneficial proposals. Patient stakeholders identified the key needs of external stakeholders and the barriers they face, ensuring that proposals would improve the effectiveness of PPI. NICE ensured that the proposals were realistic and would enhance the availability, quality, and impact of PPI in HTA.

Improved Relationships

Involving patient stakeholders as equal decision makers showed that the engagement was not tokenistic. It enabled NICE to demonstrate its commitment to PPI and transparency in decision making. As part of the methods review, a formal public consultation took place, with the work of the PWG forming part of the consultation evidence. This enabled patient stakeholders to identify the rationale for the proposals and the impact of the engagement.

Limitations

To ensure that the PWG's recommendations influenced the review, short time frames were necessary. Participants were identified for their knowledge and experience, but with additional time, an open recruitment process would have improved transparency. Although two representatives were patient experts with lived experience and three were patient experts without lived experience, only one representative was not patient organization staff. NICE HTAs aim for half of our experts to have direct experience and half with broader experience to bring their different perspectives to the committee.

The PWG was able to provide an insight to support the NICE methods review, but it was not able to provide definitive conclusions. With more flexibility, the PWG could have conducted additional awareness-raising, a series of stakeholder events and interviews, and an extended data collection time frame to increase the response rate.

The survey did not collect identifier data to indicate if a response was from an individual or an umbrella stakeholder. To identify accurate participation rates, future surveys should gather data to identify the number of stakeholders that contributed to each response.

The PWG was limited to patient stakeholders. They are the primary route of engagement in NICE HTA PPI, and so could bring an organizational (rather than personal) perspective of participating in NICE HTAs. To capture additional perspectives, future engagement could also target individual patients with HTA experience.

Including international organizations to assist the identification and evaluation of PPI by other HTA bodies would have generated additional data, providing a stronger understanding of PPI.

Future Direction

The recommendations and proposals were published on the NICE Web site (15) and shared with the 631 patient stakeholders who were invited to participate in the online survey.

As a consequence of the work NICE has

1. (1) introduced monthly meetings with patient stakeholders and patient experts to provide additional support,

2. (2) revised templates, including additional information about what committees require,

3. (3) initiated adoption of the International Summary of Information for Patients (16),

4. (4) included two patient (lay) members as decision makers on the HTA Topic Selection Oversight Panel,

5. (5) published feedback from patient experts on their committee experience quarterly in committee newsletters, and

6. (6) published an action plan [revised quarterly (15)] to communicate comprehensive updates of the impact of the recommendations.

Conclusions

In response to the Voluntary Scheme for Branded Medicines Pricing and Access, NICE reviewed their HTA methods and processes. A PWG was established using a coproduction approach, which ensured that the review reflected smarter PPI approaches. By engaging patient stakeholders, the PWG was able to identify themes and recommendations to support and improve PPI in HTA development, and these have begun to be implemented.

This work demonstrates the benefits of HTA bodies working collaboratively with patient stakeholders to improve PPI. By doing so, HTA bodies can increase their knowledge and understanding of the barriers faced by patient stakeholders and work to remove them.