If you were a policymaker, which treatment would you disinvest? A participatory value evaluation on public preferences for active disinvestment of health care interventions in the Netherlands

Abstract

Introduction

Currently, it is not known what attributes of health care interventions citizens consider important in disinvestment decision-making (i.e. decisions to discontinue reimbursement). Therefore, this study aims to investigate the preferences of citizens of the Netherlands toward the relative importance of attributes of health care interventions in the context of disinvestment.

Methods

A participatory value evaluation (PVE) was conducted in April and May 2020. In this PVE, 1143 Dutch citizens were asked to save at least €100 million by selecting health care interventions for disinvestment from a list of eight unlabeled health care interventions, described solely with attributes. A portfolio choice model was used to analyze participants' choices.

Results

Participants preferred to disinvest health care interventions resulting in smaller gains in quality of life and life expectancy that are provided to older patient groups. Portfolios (i.e. combinations of health care interventions) resulting in smaller savings were preferred for disinvestment over portfolios with larger savings.

Conclusion

The disinvestment of health care interventions resulting in smaller health gains and that are targeted at older patient groups is likely to receive most public support. By incorporating this information in the selection of candidate interventions for disinvestment and the communication on disinvestment decisions, policymakers may increase public support for disinvestment.

1. Introduction

In many countries, the large increase in health care expenditures (OECD, 2020) has put a considerable pressure on public financing (van der Horst et al., 2011; de la Maisonneuve and Martins, 2015). To limit the increase in health care expenditures, policymakers have increasingly and more strictly been assessing the reimbursement of new health care interventions, defined in this paper as a broad range of curative care, such as medicines, therapies and medical devices.

Although the criteria for the reimbursement of new health care interventions differ between countries and settings (Vuorenkoski et al., 2008; Seixas et al., 2021), previous studies have shown that clinical effectiveness (i.e. health gain of the intervention) is an important criterion in many countries (Vuorenkoski et al., 2008; Stafinski et al., 2011). Nevertheless, it has been estimated that 20–30% of health care costs is spent on interventions that are not effective (Brody, 2012; OECD, 2017). This may, for instance, be due to the less strict application of effectiveness as a criterion in past reimbursement decisions or to new evidence indicating that health care interventions may be less effective than thought before (Elshaug et al., 2007; Ruano Raviña et al., 2007). The large amount of ineffective health care interventions currently being used and reimbursed suggests that a lot can be gained both in terms of health and health care expenditures from the structured, evidence-based reassessment of these interventions, which has been called health technology reassessment (HTR) (Leggett et al., 2012; MacKean et al., 2013). That a lot can be gained from HTR has also been noted in two recent, important policy reports in the Netherlands that have flagged HTR as an important policy measure to curb the growth in health care expenditure in the future (Inspectie der Rijksfinanciën, 2020; Wetenschappelijke Raad voor het Regeringsbeleid, 2021).

HTR, just as reimbursement decision-making, has often been based on more considerations than effectiveness alone, such as the economic, social and ethical aspects of an intervention (Vuorenkoski et al., 2008; MacKean et al., 2013). If, as a result of HTR, it is concluded that a health care intervention does not meet the applicable criteria, it may be decided to withdraw the reimbursement of this intervention. The withdrawal of reimbursement is called disinvestment (Elshaug et al., 2007; Daniels et al., 2013). Despite the potential merit of HTR with regard to health gains and containing health care expenditures, the actual reassessment of health care interventions is rare. Over a period of 10 years, 34 health care interventions were reassessed in the Netherlands (Rotteveel et al., 2021a). This is just a very small fraction of the health care interventions that are currently provided to patients. A possible explanation for the lack of HTRs can be found in the difficulty to disinvest health care interventions (Robinson et al., 2011; MacKean et al., 2013). Support from health care providers, policymakers and the general public has been shown to be essential for successful disinvestment (Daniels et al., 2013; Rotteveel et al., 2021a). In this study, we focus on public support. Insight in which attributes (i.e. characteristics) of health care interventions citizens find most important for policymakers to consider in disinvestment decisions may help policymakers to propose disinvestment decisions that receive more public support.

Much evidence is available on the relative importance of attributes in the context of the assessment of new health care interventions for reimbursement (investment decisions). A systematic review concluded that studies consistently found that health gain was considered most important by citizens. Both gain in quality of life and gain in life expectancy have been shown to be important, with citizens making trade-offs between them (Whitty et al., 2014). The importance attached to health gains may be different, depending on the health state before and after treatment (Whitty et al., 2014; Gu et al., 2015). In addition to health gain, age of the patient group has also shown to be important (Whitty et al., 2014). Previous research on disinvestment policy processes indicated that different considerations may be relevant in the context of disinvestment compared to the context of investment (Ruano Raviña et al., 2007; Rotteveel et al., 2021a). Therefore, it is not clear whether these attributes are also considered important in the context of disinvestment.

Evidence on attributes that citizens find relevant in the context of disinvestment is limited to four studies (Coast, 2001; Street et al., 2015; Bentley et al., 2019; Costa et al., 2019; Rotteveel et al., 2021b). These studies found that citizens consider it important to take medical necessity, effectiveness, real-world cost(-effectiveness) data, availability of treatment alternatives, disease severity and externalities into account in disinvestment decisions (Street et al., 2015; Bentley et al., 2019; Rotteveel et al., 2021b). Moreover, while one study found that citizens are reluctant to disrupt the current treatment of patients (Costa et al., 2019), no indication for this was found in another study (Rotteveel et al., 2021b). Although these studies already provide some insight in the attributes that citizens find relevant in the context of disinvestment, it is not known what the relative importance of these attributes is.

The aim of this study is to investigate the preferences of citizens of the Netherlands toward the relative importance of attributes of health care interventions in the context of disinvestment. To this end, we asked a representative sample of citizens to make disinvestment decisions by selecting two or more health care interventions from a list of eight candidate interventions for disinvestment using the participatory value evaluation (PVE) approach. The PVE approach was used as it allows participants to make a decision as if they were a policymaker (i.e. it mimics the decision-making context), while it provides insight in the preferences citizens have with regards to the attributes of health care interventions as well as the allocation of public money (Bahamonde-Birke and Mouter, 2019) in the context of disinvestment. As previous research in the investment context showed that the importance attached to gains in health is affected by the health state before and after treatment (Whitty et al., 2014), the secondary aim of this study is to assess what the effect of the health state before and after treatment is on the importance attached by Dutch citizens to a gain in health in the disinvestment context.

2. Methods

Here, we have used PVE, a novel stated preference method in which participants are requested to choose the best portfolio (i.e. combination) of policy options within a given budget (Mouter et al., 2021b). The policy options, in this case, the candidate interventions for disinvestment, are specified in terms of attributes and levels. From the portfolios that participants choose, preferences with regards to the attributes and the amount of public money spent/saved can be obtained (Mouter et al., 2021b).

2.1 Design

In an online survey, participants were presented with (1) an introduction to the PVE task, (2) the PVE task itself and (3) questions on the sociodemographic characteristics of participants and how participants experienced the PVE task. The questions on the sociodemographic characteristics of participants were included to check the representativeness of the sample. The questions on how participants experienced the PVE task were included to learn more on the feasibility and usefulness of PVE.

In the introduction to the PVE task, participants were familiarized with the decision-problem and the task, and received instructions on the web-based environment (see Appendix A). Participants were told that, because health care expenditures are rising, the government asks citizens for their advice on savings in health care. They were asked which treatments they would disinvest in order to save at least €100 million from the health care budget to be able to reimburse new/other treatments. The threshold for the minimum required savings was set at €100 million, to ensure that the savings were substantial enough to make an impact (±1% of Dutch health care budget), while still being feasible through the disinvestment of a limited number of health care interventions. In the PVE task (see Appendix B), participants were shown a list of eight hypothetical treatments that could be disinvested to achieve the €100 million of savings. These treatments were not labelled, but randomly numbered and described with attributes and levels.

2.1.1. Attributes and levels

To assess whether similar attributes are relevant in the investment and disinvestment context, while keeping the PVE task manageable for participants, we selected the most important attributes as identified in the investment context as attributes in the current study: gain in quality of life, gain in remaining life expectancy and age of the patient group (Whitty et al., 2014; Gu et al., 2015). In addition, an attribute that may be specifically important in the disinvestment context was also included: availability of an alternative treatment (Street et al., 2015; Rotteveel et al., 2021b). Finally, budget-impact was included as this is a required attribute for PVE. Participants were explained that gains in quality of life and remaining life expectancy as well as the budget-impact were respective to the alternative treatment, if available. To assess whether the value attached to gains in health (i.e. quality of life and life expectancy) is affected by the health state (i.e. quality of life and life expectancy) before and after treatment, these attributes were also included in the PVE. A description of how the attribute levels were determined is included in Appendix C. All attributes and levels are displayed in Table 1.

Table 1. Attributes and attribute levels

^a The budget-impact, as well as the number of patients concerned and the costs per patient were presented to participants.

^b Quality of life before and after treatment are both displayed as separate attributes as well as combined in the attribute ‘improvement in quality of life’. Quality of life before treatment ⩽ quality of life after treatment with the difference between the two being <50% (an improvement from 35% to 85% was considered unrealistic in general and if available, to not be a serious candidate for disinvestment).

^c Life expectancy before and after treatment are both displayed as separate attributes as well as combined in the attribute ‘improvement in life expectancy’. Life expectancy before treatment ⩽ life expectancy after treatment.

^d This attribute level could only be combined with a remaining life expectancy (before and after treatment) ⩽ 15 years.

2.1.2. Experimental design

To increase the information that can be obtained from the choice models, participants must face different combinations of attribute levels for each of the eight treatments. However, collecting data for all possible combinations of attribute levels (350,208 possibilities) is unfeasible. Therefore, an experimental design containing 60 different versions was selected through a three-stage process. First, the feasible set of combinations of attribute levels was determined. Second, a candidate initial design was constructed by randomly selecting 60 combinations of feasible attribute levels, within the restrictions as defined in Table 1. Third, attribute levels of the design were iteratively changed to reduce the correlation between attributes. Through this process, combinations of attribute levels were selected that minimize the correlation between attributes, increasing the efficiency of the model estimates. A more detailed description of the evaluation metrics and algorithms used to construct the experimental design can be found in Appendix 1 of Mouter et al. (2021a).

2.1.3 Pilot testing

The PVE was pilot tested in five face-to-face sessions with 20 participants in total. Participants were recruited by a commercial panel organization. Groups that participate less often in panels were oversampled to assess whether the PVE method was also feasible for these participant groups. Based on the pilot tests, study materials were improved to increase the feasibility of the PVE. For instance, participants were provided with the possibility to have the text read to them (instead of reading the text themselves) and more instructions were added to the instruction video.

2.2 Data collection

Data were collected among a representative sample of the adult Dutch population with regard to age (>18 years), gender and educational level. Participants were recruited by a commercial panel organization. Prior to entering the questionnaire, they received an information letter containing information on the study purpose, questionnaire content and length, data storage, the investigator and who had access to their data, and written informed consent was obtained. Participants received a standard participation fee, following the incentive system of the panel organization. Ethical approval was obtained from the internal review board of the Erasmus School of Health Policy & Management (IRB 20-09). The research team did not have access to participants' contact information and handled the remaining participant information confidentially.

2.3 Data analysis

The econometric approach is based on the portfolio choice model (Bahamonde-Birke and Mouter, 2019) to estimate the preferences for attributes in PVE, and adapted for this study. Under this framework, it is assumed that participants choose to disinvest the portfolio (i.e. combination) of treatments that is least desirable to them from all other possible portfolios of treatments, given the minimum imposed savings of €100 million. The desirability of the different portfolios depends on the perceived societal value of the treatments that are not disinvested, as well as the societal value attached to the monetary savings in addition to the imposed minimum savings of €100 million.

Let a participant n face J treatments to disinvest in the PVE. In turn, each treatment is composed by a set of K attributes and generates savings equal to the budget-impact, denoted by c _nj. If a treatment is chosen by the participant for disinvestment, utility decreases since the treatment is no longer available to society. However, if the participant disinvests a set of treatments that generates savings above the minimum goal explicitly imposed in the PVE – in this case ‘saving at least €100 million’ – denoted by B, we assume that the utility lost by disinvesting the treatments is compensated by the utility gained from the additional savings of disinvestment. Hence, the utility a participant derives from the portfolio p of non-disinvested treatments and additional savings is given by:

$$U_{np} = V_{np} + \varepsilon _{np} = \mathop \sum \limits_{\,j = 1}^J y_{nj}\cdot U_{nj} + \alpha \left({\mathop \sum \limits_{\,j = 1}^J ( {1-y_{nj}} ) \cdot c_{nj}-B} \right) + \varepsilon _{np}$$

where y _nj is a binary variable that is equal to one if the treatment is not disinvested and zero otherwise, U _nj is the utility derived from treatment j, α is an estimable parameter that accounts for the preferences for additional savings, and ɛ _np is an extreme-value error term. The utility of each treatment j is a function of their K attributes:

$$U_{nj} = \mathop \sum \limits_k^K \beta _k\cdot x_{njk}$$

where x _njk is the attribute level k of treatment j, and β _k is an estimable parameter that accounts for the preferences for attributes.

The choice probability of keeping the portfolio p takes the form of a multinomial logit function:

$$P_{np} = P( {U_{np} \ge U_{nq}, \;\forall p\ne q} ) = \displaystyle{{e^{V_{np}}} \over {\mathop \sum \nolimits_{q\in Q} e^{V_{nq}}}}$$

where Q is the set of all feasible portfolios of treatments. A maximum likelihood approach is used to estimate α and β, in a similar way as a standard multinomial logit model (McFadden, 1974).

To assess whether quality of life and life expectancy before and after treatment affected the value attached to a gain in quality of life and life expectancy, two additional analyses were conducted: (1) an analysis containing both quality of life and life expectancy before treatment, and gain in quality of life and life expectancy as attributes, and (2) an analysis containing both quality of life and life expectancy after treatment, and gain in quality of life and life expectancy as attributes. As the gain in quality of life/life expectancy and the quality of life/life expectancy before and after treatment are interrelated, it was not possible to include all in the same analysis.

All analyses were performed using R, version 4.0.2 (R Core Team, 2020).

3. Results

3.1 Sample

Data were collected between 9 April and 5 June 2020. In total, 3387 participants provided informed consent. Of these participants, 1143 (33.7%) completed the questionnaire. People who started but did not complete the questionnaire were on average older (50.4 vs 43.7 years), more often female (66 vs 56%) and more often had a low educational level (35 vs 25%) than people who did complete the questionnaire.

Table 2 displays the age, gender and educational level of the sample. It shows that, compared to the Dutch general population, in our sample, people older than 65 years of age, people with a low educational level and males were slightly underrepresented.

Table 2. Descriptive characteristics of the sample

SD, standard deviation.

Source: CBS Statline: https://opendata.cbs.nl/statline/#/CBS/en/.

^a Education levels correspond to the SOI 2016 and the ISCED 2011 classifications.

The answers to the questions concerning how participants experienced the PVE are included in Appendix D.

3.2 Main analysis

The results of the portfolio choice model are presented in Table 3. These results show that age of the patient group, gain in remaining life expectancy, gain in quality of life and savings in addition to the imposed minimum savings of €100 million have a statistically significant effect on the likelihood of disinvesting a treatment. The larger the health gain of the treatment, the less likely participants are to disinvest the treatment. From the coefficients, it can be calculated that a gain in life expectancy of one year provides similar utility to participants as a gain in quality of life of 3%. The negative sign of the additional savings coefficient indicates that participants prefer not to save more than the minimum required savings of €100 million.

Table 3. Portfolio choice model

SE, standard error.

*** p < 0.001, **p < 0.01, *p < 0.05.

As previous studies found the preference for age of the patient to have an inverse U-shaped pattern (Gu et al., 2015), we included age of the patient squared to check whether this was also the case in our study. The statistical significance of the age attributes may indicate that the preference for age of the patient group in this study also has a U-shaped pattern.

To illustrate the findings in Table 3, we have calculated the relative utility lost by disinvesting three randomly selected portfolios that participants could compile in version 1 of the design of our PVE and that meet the required minimum savings of €100 million. The results of these calculations can be found in Appendix E. Of the three portfolios in Appendix E, participants would choose to disinvest portfolio 1 as disinvesting this portfolio results in the smallest loss in utility.

3.3 Quality of life and life expectancy before and after treatment

The results of the analyses to assess the association between the preference for the quality of life and life expectancy before and after treatment and the preference for the gain in quality of life and life expectancy are displayed in Tables 4 and 5. Table 4 shows that both a larger gain in remaining life expectancy and a larger remaining life expectancy before treatment increase the likelihood to maintain the reimbursement of the treatment. However, the negative coefficient of the interaction effect indicates that if the remaining life expectancy before treatment is higher, people attach less value to the same gain in life expectancy. Quality of life before treatment does not have a statistically significant effect on the likelihood to disinvest a treatment.

Table 4. Portfolio model including quality of life and remaining life expectancy before treatment

SE, standard error.

*** p < 0.001, **p < 0.01, *p < 0.05.

Table 5. Portfolio model including quality of life and remaining life expectancy after treatment

SE, standard error.

*** p < 0.001, **p < 0.01, *p < 0.05.

Table 5 shows that the value attached to the same gain in remaining life expectancy is smaller if the remaining life expectancy after treatment is higher. Table 5 also shows that if the quality of life after treatment is better, then the value attached to the same gain in quality of life is smaller. The utility lost by disinvesting the three randomly selected portfolios in Appendix E was also calculated based on the additional analyses. The results of these calculations are displayed in Appendix F.

4. Discussion

The main objective of this study was to investigate the preferences of Dutch citizens toward the relative importance of attributes of health care interventions in the context of disinvestment. We found that participants prefer to disinvest treatments with limited to no gain in quality of life and life expectancy and that are targeted at older patient groups. Furthermore, we found that participants prefer to disinvest portfolios (i.e. combinations of health care interventions) with smaller additional savings (i.e. savings in addition to the minimum required savings) over portfolios with larger additional savings. This implies that they aimed to save no more than the minimum required savings of €100 million. As we told participants that additional savings would be spent elsewhere in the health care system, this finding may indicate that participants value the maintained treatments over other means in health care the money could be spent on. On the other hand, it may also indicate that people did not want to save more than the minimum required savings because they already considered this amount too high, or it may reflect a general dislike of the disinvestment task.

The secondary objective of this study was to assess what the effect of the quality of life and life expectancy before and after treatment is on the importance attached by Dutch citizens to a gain in quality of life and life expectancy in disinvestment decisions. We found that the value attached to a gain in quality of life is affected by quality of life after treatment: if the quality of life after treatment is better, less value is attached to the same gain in quality of life, compared to when quality of life after treatment is worse. Furthermore, we also found that the value attached to a gain in remaining life expectancy is affected by the remaining life expectancy before as well as after treatment. The model results showed that the value attached to such a gain is smaller if life expectancy before/after treatment is higher.

4.1 Comparison with literature

4.1.1. Comparison with disinvestment literature

Our results confirm the findings of two previous studies on disinvestment in which effectiveness of the treatment was considered an important attribute (Street et al., 2015; Bentley et al., 2019; Costa et al., 2019). Street et al. found that this attribute was rarely considered in isolation from other attributes (Street et al., 2015), which is also confirmed by the findings of our study. For instance, just as Street et al., we found that participants trade the gain in health against disease severity/health status. However, contrary to the previous literature (Street et al., 2015; Bentley et al., 2019), the availability of an alternative treatment did not significantly affect the likelihood to disinvest a treatment. Hence, the other attributes were clearly considered to be more important than availability of alternative treatment in disinvestment decision-making.

4.1.2 Comparison with the investment literature

Our finding that health gain is an important attribute for disinvestment corresponds with the findings of two systematic reviews on allocation preferences for investment (Whitty et al., 2014; Gu et al., 2015). These reviews found that health gain is consistently highly valued in previous studies and can, therefore, be considered one of the most important attributes. As in this study, the reviews found that larger health gains are preferred over smaller health gains, but that these gains are weighted against disease severity and age of the patient group (Whitty et al., 2014; Gu et al., 2015).

Hence, the results of this study on preferences with regards to disinvestment are very much aligned with the findings of previous studies on preferences with regards to investment. A possible explanation for this alignment of preferences may be the choice not to include information on the exact treatments and patient groups concerned in the current study (i.e. to use a non-labelled design). As this information was not included, the patients who would potentially be affected by disinvestment were not identifiable, which may have made the task more abstract to participants, making it a different trade-off from when the patients potentially affected by disinvestment would have been identifiable (Jenni and Loewenstein, 1997; MacKenzie et al., 2008; Robertson et al., 2013). Hence, more differences between the preferences that are considered relevant for investment and disinvestment may have been found if the patient group and treatment concerned would have been known. We recommend future research to explore whether this hypothesis is true.

4.2 Reflections on the participatory value evaluation method

PVE is a novel stated preference method that resembles other stated preference methods such as discrete choice experiments (DCE). To our knowledge, this study is the first PVE in the context of the allocation of health care budgets. However, numerous DCEs have already been conducted on this topic (Luyten et al., 2015; Shah et al., 2015; Skedgel et al., 2015; van de Wetering et al., 2016; Bourke et al., 2018; van der Aa et al., 2018). There are two main differences between DCE and PVE. First, in DCE, participants express their preferences through selecting a single health care intervention out of two or more health care interventions, whilst in PVE, participants select a bundle of health care interventions out of a larger bundle of health care interventions (Mouter et al., 2021a). Second, in PVE, the total societal budget impact is part of the experiment, while in DCE this is not the case. By incorporating this budget in the experiment, participants can express their preferences toward (the attributes of) specific health care interventions as well as the extent to which scarce health care resources should be allocated to these health care interventions (Mouter et al., 2021a). Because of these two differences, PVE more closely aligns with the practice of societal decision-making than DCE does. This close alignment to the practice of policymaking allowed us to involve citizens in the complexity of making decisions on the disinvestment of health care interventions. Nevertheless, as DCE requires participants to make multiple choices, DCE may be more efficient in obtaining information on the relative importance of attributes. Therefore, when deciding between PVE and DCE to measure public preferences, researchers should determine whether they are interested in (1) involving participants in the complexity of societal decision-making, (2) the amount of budget participants want to (re-)allocate or (3) efficiently measuring the preferences for a number of attributes. In the former two situations, PVE may be more suitable, while in the latter situation, DCE may be more suitable.

In this study, we used a non-labeled PVE approach. Alternatively, a labeled approach may have been used in which the candidate interventions for disinvestment would be given identifying labels such as ‘statins’, ‘contraceptives’ and ‘cognitive behavioral therapy’. An advantage of this labeled approach is that it more closely aligns with the practice of disinvestment decision-making, as in that case, the exact intervention and patient group is also known. Hence, in a labeled approach, the differences between DCE and PVE are more distinct, and, as such, the advantages of using PVE are clearer. However, we anticipated that if we would have labeled the treatments, these labels may have dominated the preferences participants have. This is something that has also been observed in PVEs on infrastructure projects (Mouter et al., 2021b). A consequence of this is that we would not be able to assess the importance of the different attributes of health care interventions in disinvestment decisions, which was the main aim of this study. Moreover, another advantage of the non-labeled approach is that the findings are applicable to a broad range of disinvestment decisions, instead of only the labeled cases included in a labeled PVE. Nevertheless, in transferring the findings of this study to the practice of disinvestment decision-making, policymakers are recommended to also take the context of disinvestment into account as previous research has shown that contextual aspects, such as view on the disinvestment case, support and financial incentives, are very important in disinvestment decision-making (Rotteveel et al., 2021a).

4.3 Context

The data for this study were collected at the time the first wave of the Covid-19 epidemic took place in the Netherlands, which resulted in tremendous pressure on the Dutch health care system. Regular health care was delayed and it was feared that the intensive care capacity in the Netherlands would be insufficient (van Giesen et al., 2020). Health care providers were under much pressure, working long hours. Their efforts were very much appreciated by Dutch citizens. At the same time, the country was in partial lock-down with schools, bars and many shops being closed, and citizens being asked to work from home. Hence, data were collected in exceptional circumstances. This may have affected the findings of this PVE. For instance, because of the large appreciation of Dutch citizens for health care providers they may either have been more reluctant (e.g. consider it important to let health care providers determine what is valuable care) or less reluctant to disinvest treatments (e.g. to get some weight of the shoulders of health care providers). To limit the effect of this unique situation on the findings of our study, we specifically told participants that acute health care, such as health care targeted at people suffering from infectious diseases like Covid-19 would never be disinvested. Despite this, our findings may still be affected by the exceptional circumstances.

4.4 Strengths and limitations

Because of the structured, evidence-base selection of attributes and attribute levels, this study can be compared with previous studies using other methods to assess health care allocation preferences. Moreover, by including both remaining life expectancy as well as age, we were able to disentangle the preferences for life expectancy from the preferences for age (Whitty et al., 2014; Gu et al., 2015). Furthermore, the face-to-face pilot test enabled us to assess and increase the feasibility of the PVE method.

Nevertheless, this study also has limitations. First, drop-out was quite high with only a third of participants who started the questionnaire also completing it. Obviously, the task given was not a popular one, no one likes to think about necessary savings on health care. Moreover, the task was cognitively demanding, as participants had to compare characteristics of eight different imaginary treatments in one single task. As discussed in the Results section, drop-out may be selective with participants dropping-out being on average older, lower educated and more often female. Moreover, drop-out may also be selective in the sense that only people who were most motivated completed the PVE task. This may be especially problematic if participants completing the PVE task have a different perspective on disinvestment than non-completers have. It is not possible to assess whether this is the case, as only descriptive information of participants who dropped-out was available. However, it is important to take this into account when interpreting the results of this study. Second, this study has been conducted in the Dutch context. As preferences may be time and context-specific, researchers are recommended to take the context into account when transferring these results to their own setting.

4.5 Policy implications

The results of this study have implications for disinvestment decision-making. First, the model results of this study show that citizens prefer to disinvest treatments that result in limited or no health gain and that are targeted at older patient groups. This information can be used in the selection of candidate interventions for disinvestment. Furthermore, it stresses the need to de-implement low-value care [i.e. health care interventions that provide little to no clinical benefit (Bhatia et al., 2015)], as has also been suggested by international de-implementation campaigns such as Choosing Wisely (Bhatia et al., 2015; Grimshaw et al., 2020). In addition, this information can be used in the communication about disinvestment decisions. Based on the results of this study, we expect that if policymakers explain to the public that a disinvested treatment does not have any or only limited health effect, this will increase societal support. Note, however, that such an explanation alone may not be sufficient to overcome public opposition to disinvestment. For instance, it may also be necessary to better involve citizens in the necessity and complexities of disinvestment decision-making, for example, through citizen forums (Bijlmakers et al., 2020), and to ensure procedural legitimacy of disinvestment decisions (Daniels and Sabin, 2008).

Second, the PVE method shows that subgroups of the general public are able to make disinvestment decisions. This indicates that it is feasible to involve this subgroup of citizens in disinvestment processes. For instance, their advice can be asked on the prioritization of disinvestment candidates. However, the strong dislike of disinvestment of pilot test participants, the large number of selective drop-outs and the feedback provided by participants showed that there was also a significant subgroup of the general public who are not able to or dislike making disinvestment decisions to such a degree that they will not be willing/able to advice policymakers on these difficult decisions. This implies that additional efforts are needed to also fully engage these subgroups of the Dutch population in disinvestment decisions. It is important to engage all subgroups in disinvestment decisions in order to obtain broader societal support for disinvestment. This is especially timely given the attention given to disinvestment as a policy measure to curb the growth in health care expenditure and to the engagement of citizens in health care policy decision-making in two recent, important Dutch policy reports (Inspectie der Rijksfinanciën, 2020; Wetenschappelijke Raad voor het Regeringsbeleid, 2021).

Finally, responses to the follow-up questions in this PVE, displayed in Appendix D, showed that the majority of participants feel that the advice of experts deserves more weight than the advice of citizens with regards to disinvestment decisions. Nevertheless, 92% of participants who answered the question (72% of total) feel that the advice of citizens should be considered as well. Therefore, policymakers are recommended to think of ways to incorporate the preferences of citizens in disinvestment decision-making, in addition to the advice of experts. The majority of participants (58% of those who answered this question, 50% of total) suggest that PVE may be a good method to involve citizens in this type of complicated and sensitive policy decisions. For instance, the organization responsible for reimbursement decision-making (in the Netherlands: Zorginstituut Nederland) may regularly (e.g. each year) ask citizens through PVE for their advice on a number of health care interventions that have been selected by that organization as potential candidates for disinvestment. The advice of citizens can then be incorporated in the decision which of the selected disinvestment candidates will actually be disinvested. However, as discussed in a previous paragraph, the PVE method may only be suitable for a part of the general population or requires additional efforts to engage a representative sample of the general population. Therefore, it may also be valuable to explore whether other methods (e.g. citizen juries or PVEs with researcher assistance) are better suitable to involve a representative sample of the general population in disinvestment decision-making.

4.6 Conclusions and recommendations

In this study, we found that citizens prefer to disinvest treatments with limited to no health gain and that are targeted at older patient groups. The weight attached to the gain in health depends on the health status before and/or after treatment. Policymakers are recommended to take these preferences into account in the selection of candidate interventions for disinvestment as well as in the communication on disinvestment decisions to increase the support for these decisions.