Integrating Early Economic Evaluation into Target Product Profile development for medical tests: advantages and potential applications

Abstract

Target Product Profiles (TPPs) outline the characteristics that new health technologies require to address an unmet clinical need. To date, published TPPs for medical tests have focused on infectious diseases, mostly in the context of low- and middle-income countries. Recently, there have been calls for a broader use of TPPs as a mechanism to ensure that diagnostic innovation is aligned with clinical needs, yet the methodology underpinning TPP development remains suboptimal. Here, we propose that early economic evaluation (EEE) should be integrated within the TPP methodology to create a more rigorous framework for the development of “fit-for-purpose” tests. We discuss the potential benefits that EEE could bring to the core activities underpinning TPP development—scoping, drafting, consensus building, and updating—and argue that using EEE to help inform TPPs provides a more objective, evidence-based, and transparent approach to defining test specifications.

Background

Translating scientific discoveries from bench to bedside is often challenging for new healthcare technologies (1), particularly for medical tests. Despite extensive research efforts, many innovative tests ultimately fail to enter clinical practice (2). This is in part due to their failure to address a clear unmet clinical need (3), as well as a common lack of understanding as to how tests will fit into existing clinical pathways (4). This is further exacerbated by the slow adoption of new tests by end users (5). Therefore, there is a need for new methods that ensure research efforts are focused on technologies that fulfill a specific unmet clinical need and have the potential to be cost-effective.

Target Product Profiles (TPPs), typically drafted in the early stages of technology development, describe the necessary properties of a new technology to meet an unmet clinical need (6). TPP development is ideally an iterative process, with initial technology requirements being updated as additional evidence becomes available (7). The aim of TPPs is to focus innovation on “fit-for-purpose” technologies that address a specific clinical need. Although past TPPs of tests have focused on addressing infectious diseases in low- and middle-income countries (7), the broader utility of TPPs is being increasingly recognized (8). In the wake of the COVID-19 pandemic, for example, the UK Medicines & Healthcare products Regulatory Agency (MHRA) developed a series of TPPs for COVID-19 tests to promote the development of clinically useful diagnostics (9).

In a recent systematic review, we described the methodology that is currently being used to develop TPPs of medical tests (7). Existing TPPs largely focus on summarizing necessary specifications across five key evidence domains: (i) analytical performance (e.g., sample requirements), (ii) clinical validity (e.g., diagnostic sensitivity and specificity), (iii) infrastructural requirements (e.g., transport and storage conditions), (iv) human factors (e.g., device format and complexity), and (v) costs (7). Several methodology limitations were also identified. These concerned: (i) the subjectivity of data informing test requirements (e.g., heavy reliance on expert opinion); (ii) poor transparency in reporting the methodology underpinning TPP documents; (iii) a lack of explicit consideration for clinical utility when defining test characteristics; and (iv) an oversight of cost-effectiveness considerations.

Economic evaluation compares the costs and benefits of alternative healthcare strategies to support technology adoption and reimbursement decisions. Whilst economic evaluation is usually conducted at the later stages of the evaluation pathway, early economic evaluation (EEE) is increasingly performed at the proof-of-concept or pre-regulatory-approval stage as a means of directing stop/go decisions and informing the optimal trajectory of future research (10;11). A standard approach used in economic evaluations—early or otherwise—is decision analytic modelling: a framework of analysis that uses mathematical tools and schematics to provide a simplified representation of the decision problem, as a series of uncertain events.

In this opinion paper, we discuss how EEE could strengthen the methodological rigor of TPP development for tests. With TPP development being framed around unmet clinical needs, EEE in this context should be regarded as a technology-agnostic methodology that is done independently of any new or existing technology. Based on our collective experience, we describe the possible advantages that EEE methods could afford across four core activities of TPP development: scoping (see Section "Scoping: Pathway Mapping to Define Test Role, Purpose, and Potential Impact"), drafting and consensus building (Section "Drafting and Agreeing Test Specifications"), and updating (Section "Updating TPP specifications"). By placing EEE at the heart of TPP development, we describe how the methodological limitations of TPPs (highlighted above) could be effectively addressed. The key arguments raised below are summarized in Figure 1.

Figure 1. Overview of some expected benefits that early economic evaluation (EEE) can bring to each Target Product Profile (TPP) development activity, alongside relevant methods.

Integrating EEE Methodology within TPP Development

Scoping: Pathway Mapping to Define Test Role, Purpose, and Potential Impact

The scoping phase defines the focus of a TPP in terms of the unmet clinical need being addressed and which test characteristics should be included (7). Typically, this is achieved via literature reviews and consultations with experts and stakeholders (7). Based on current practice, it is unclear if any formal pathway analysis and/or assessment of a test's placement and impact is typically undertaken at this stage (7); this risks overlooking key mechanisms of impact on patient outcomes.

The first step in economic modelling is to map the care pathway in which the new technology will sit. This typically involves reviewing clinical guidelines and consulting clinicians, patients, and carers on their individual experiences of the care pathway. The result is a clear schematic of the existing clinical pathway, detailing the main activities and events that may occur for the patient population under consideration. This ensures that the processes involved in standard care are clearly understood and also forces test developers to specify a test role (i.e., whether a test is a replacement, triage, or add-on) and purpose (e.g., screening, diagnosis, prognosis, prediction, or monitoring) within the specified pathway. This is crucial in the context of TPP development, because selecting a different test role and/or purpose can result in significantly different test requirements (12). Where the optimal placement of a new test is unclear, additional comparators can be added to an EEE to explore alternative options (12).

Pathway mapping further helps to identify the possible downstream consequences (harms and benefits) of a new test, including expected impacts on: decision making (e.g., change in treatment decisions), patient health outcomes, clinical workflow (e.g., time-to-treatment), and economic outcomes (13). This is vital for TPPs, because the expected harms and benefits associated with a test should directly influence which test specifications are included within a TPP. For EEEs, this step further helps to identify which test properties should be captured and varied within the economic model (diagnostic accuracy, test price, turnaround time, etc.). Although pathway mapping is a core process of economic modelling, it should be noted that this activity can be conducted independently of any formal economic evaluation.

Drafting and Agreeing Test Specifications

The TPP drafting phase estimates the specifications for new tests, whereas consensus building aims to attain consensus on those requirements through stakeholder surveys and consensus meetings. Typically, test specifications are presented at two levels (“desirable” and “acceptable”) based on expert consultations and literature findings, and without explicitly considering the dependencies between different test specifications (e.g., the dependence between diagnostic sensitivity and specificity), or other factors related to the clinical context (e.g., the required sensitivity and specificity will be dependent on disease prevalence) that could jointly impact the overall utility of a given testing strategy (7). This apparent failure to consider parameter dependencies risks over- or underestimating test requirements.

Here, we outline how EEE can: (i) allow exploration of dependencies between test specifications and features of the clinical context; (ii) facilitate optimization of quantitative test specifications based on a willingness-to-pay (WTP) threshold; and (iii) increase the objectivity of data informing TPP requirements.

Exploring Dependencies Between Test Specifications and Features of the Clinical Context

Desirable ranges for test specifications may vary when considering different factors underlying the care pathway or other test properties. It might, therefore, be helpful to distinguish upfront between:

• • Properties inherent to a test—test properties (e.g., diagnostic accuracy and test turnaround time) that could have a direct/indirect effect on the utility of a test.

• • Factors relating to the clinical context—aspects exogenous to a test that could influence how a test impacts upon relevant outcomes (e.g., prevalence of disease, natural disease progression, efficacy, and cost of treatment).

With EEE, by synthesizing multiple test attributes and factors relating to the clinical context within a single modeling framework, the dependencies between different parameters can be effectively captured and explored. One-way or multiway sensitivity analyses can be used to explore the impact of one or more parameter(s) on the clinical utility or cost-effectiveness (14–19). Scenario analyses can also be used to explore the impact of a group of parameter changes that represent a specific clinical scenario (14;16).

Estimating Ranges for Quantitative Test Characteristics

Outside the context of TPPs, EEE has previously been used to identify acceptable ranges for test characteristics or components (12;14–20). In the absence of evidence for a new test, hypothetical values for test properties can be assigned and, for each specification or combinations of specifications, the downstream costs and benefits can be computed. Based on a specified WTP threshold, it is possible to then back-calculate the maximum costs and minimum specifications for a new test to be cost-effective (10).

Key relevant methods that may be useful in this phase are:

• • Headroom analysis—The headroom price of new tests represents the maximum price at which a test is considered cost-effective, assuming perfect clinical accuracy. This method can be used to determine the maximum cost for a new test, while calculating the financial room for improvement for a hypothetical test assuming perfect conditions. See (12;15) for examples.

• • Threshold analysis—Identifies the critical value for input parameters above or below which a reimbursement decision is expected to change. This approach could be used within TPPs to estimate the maximum price or minimum diagnostic accuracy at which a test remains cost-effective given a certain WTP threshold. See (12;15–18;20) for examples.

• • Sensitivity analysis—In the absence of a clear WTP threshold, sensitivity analysis could be used to identify ranges of minimal test properties, while exploring changes in cost-effectiveness results due to altering one or more model parameters. See (14;15;19) for examples.

Objective Derivation of Test Specifications

The early stages of technology development are characterized by high uncertainty, making it challenging for experts to form accurate judgments around the desirable specifications that a new test should possess.

EEE methodology facilitates the generation of more objective data, which can be combined with subjective evidence (i.e., clinical and stakeholder judgment) to inform TPP specifications. EEE results can also support the elicitation of expert opinion, with clinical experts being asked to review and discuss the modelling findings. Presenting the results of EEE modelling within consensus meetings could further help to narrow down the ranges for performance benchmarks that a new test should hit to be both clinically effective and cost-effective.

Were EEEs to be integrated into TPP development, the methods, model code, and analysis underpinning TPPs should be clearly reported and made freely available, in line with international modelling guidelines (21). This would provide a more transparent and objective approach to setting performance benchmarks, while, in turn, allowing others to inspect what data have informed TPP development.

Updating TPP Specifications

Although TPPs are dynamic documents that should be updated as new evidence is found (6), no formal or standardized approach for updating TPP specifications is currently available (7). We believe that this activity is important as a means of ensuring that test development is driven by accurate, up-to-date information. Here, we discuss some expected advantages that EEE can bring to this core activity.

Incorporating New Evidence

EEEs can be designed so as to be sufficiently flexible to allow for the model to be iteratively updated as evidence is accrued (4). This means that different TPPs could be developed based on the level of evidence available: early TPPs could focus on defining the unmet clinical need and key requirements for a new test to address that need, whereas later iterations could identify more precise test specifications.

Abel et al. (14), for example, published their model code to allow others to examine and explore additional questions. For TPPs, this approach would help to ensure that performance specifications could be validated with each iteration of a TPP, thus further increasing the methodological rigor underpinning TPPs.

Highlighting Areas for Future Research

Findings from sensitivity analyses could be used to communicate areas of greatest uncertainty requiring further research. Alongside this, when more evidence and information about a test is available (i.e., when parameter distributions reflect uncertainties rather than “unknowns”), Value of Information (VOI) analysis may be useful to assess the value of gathering more information on model parameters to reduce current decision uncertainty (12;18).

Conclusion

TPPs are increasingly of interest as a means of focusing the innovation pipeline for new tests on areas of clinical need (8;22). Here, we argue that some of the methodological limitations with TPPs (7) could be addressed by integrating EEE into TPP development. In particular, pathway mapping would provide clarity on the mechanisms by which a test could impact on patient health, and decision analytic modelling could increase the objectivity and transparency of the data informing TPP specifications. Transparent and flexible early economic models would also allow test specifications to be updated as new evidence becomes available, helping to prioritize areas for future research.

There are, however, potential limitations and complexities with integrating EEE into TPP development. For example, conducting a wide range of sensitivity analyses with EEE risks overwhelming TPP developers with information, making the estimation of clearly defined test performance requirements challenging. Such analyses should always be kept within the confines of clinical plausibility, and clinical experts can support the identification of clinically relevant scenarios. Nevertheless, given that uncertainty in the technology development process is unavoidable, we believe that the value in such analyses lies in explicitly communicating those key areas of uncertainty, identifying trade-offs between test characteristics, and highlighting priorities for future research.

Economic evaluation also requires the adoption of a particular jurisdiction (e.g., UK NHS), which dictates what country-specific clinical pathways and costs are included in the model and what prespecified decision criterion is used to inform the analysis (e.g., the UK WTP threshold per quality-adjusted life year). In case a particular decision criterion is not based on cost-effectiveness, the EEE methods discussed here may still be useful to explore scenarios around clinical utility or cost. The applicability of modelling findings to other jurisdictions is, therefore, often limited, and multiple model versions may be required to derive test specifications across different jurisdictions. This is true of any formal evaluation of downstream clinical or cost outcomes, however (where country-specific clinical pathways and costs will come in to play), and is not specific to EEE.

Several of the issues raised here (transparency, uncertainty, notion of value, etc.) have been discussed in previous papers (23–25); interested readers may wish to consult these commentaries for further aspects to consider when using EEEs to inform the technology development process.

Although we have argued that EEE is a valuable methodology to support TPP development, we recommend this approach as an adjunct rather than a replacement for the existing TPP development methodology. Expert input and consensus-building discussions with relevant stakeholders will be key to ensuring that EEEs capture the nuances of a clinical context, and that the results are clinically meaningful. Ultimately this will ensure the efficient development of tests that provide greater utility for both patients and healthcare services.