Participatory Modeling in Sustainability Science: The Road to Value-Neutrality

Abstract

Participatory modeling in sustainability science allows scientists to take stakeholders’ interests, knowledge, and values into account when designing a model-based solution to a sustainability problem, by incorporating stakeholders in the model-building process. This improves the chance of generating socially robust knowledge and consensus on solutions. Part of what helps in this regard is that scientists, through involving stakeholders, limit their own values from influencing the outcome, thus achieving some level of value-neutrality. We argue that while it might achieve this to some extent, it comes at a cost to the reliability of the outcomes, which is ethically problematic.

1. Introduction

Sustainability science faces the challenge of solving complex problems that cross societal and natural boundaries. Over the past fifteen years many sustainability scientists have turned to an approach known as participatory modeling, in which stakeholders participate in the modeling design process, on the theory that stakeholder involvement improves the social robustness of outcome, or the willingness of stakeholders to take up scientific solutions and abide by the outcomes. Sustainability scientists have discussed a range of challenges in stakeholder participation in scientific processes (e.g., Polk 2014). As far as philosophers of science are concerned, one thing that is interesting about participatory modeling is that it can be understood as a systematic attempt to resolve the problem of values in science. By bringing participants into the process and allowing them to represent their beliefs and preferences, scientists avoid the problems for example which arise in inductive risk contexts, in which they implicitly set standards assuming one set of values over others. In this article, we aim to introduce participatory modeling to a philosophical audience and evaluate it with respect to the goals of social robustness and value-neutrality. We find that while participatory modeling does provide a potential solution to value-neutrality issues, achieving this in practice raises substantial replicability and reliability issues with which the field has yet to grapple. As such, generating consensus through the use of such a model process produces difficult ethical problems. There are, as a result, reasons to maintain some level of distance between scientific work in sustainability science and its users or stakeholders.

2. Participatory modeling

Participatory modeling (PM) is part of a family of broader approaches that are “participatory” (Kasemir et al. 2003), “transdisciplinary” (Polk 2014), or “co-productive” (Turnhout et al. 2020) that are becoming increasingly mainstream in sustainability science. Participatory modeling in sustainability and the environmental sciences dates back to the early 2000s and has an origin in practices used in management science and operations research (Voinov and Bousquet 2010; Voinov et al. 2016, 2018). It has now evolved into a major modeling strategy, and different strands or streams that differ in terms of the types of models they favor and the types of participatory activities they prefer (Moallemi et al. 2021). In all cases, however, the basic motivation is the same. Stakeholders should contribute directly to the model-design process to arrive at a shared scientifically based decision. This can occur at numerous points in the model-building process. For instance, it can occur through discussion on setting goals for the modeling process; the choice of modeling tools; the collection of data; the articulation of local knowledge, which stakeholders may have specific access to; the initial production of a conceptual model; or discussion and evaluation of results. Voinov et al. (2016) in a large-scale review of participatory modeling approaches report that stakeholders are involved to the greatest frequency in goal setting (establishing a conceptual model and setting parameters and variables to include), the collection of data and checking of data, and evaluation of model results through running the model as well as exploring the implications of those results. In the last case stakeholders can play a role in determining which scenarios are relevant to analyze for the purposes of decision making and reaching a policy consensus. Ideally, stakeholders would be involved at all levels, although this is not always practical or possible. During this process the modeler or sometimes a separate facilitator acts as a guide to the process of discussion and negotiation. The process is often iterative—requiring redesign of a model if stakeholders find the results unrealistic or implications unworkable.

To illustrate PM, we draw on a case of Companion Modeling (ComMod) (Étienne, 2014), a prominent methodology that combines role-playing games involving stakeholders with agent-based modeling (ABM) for quantitative modeling. ComMod has been mainly used for conservation and renewable natural resource management that feature coupled complex systems, both natural (e.g., dynamics of harvest stock) and social (e.g., stakeholders who interact with each other and with the natural systems). The core of the methodology is simulation modeling, both numerical and experimental. As a theoretical resource, game theory is used not only to understand and predict a system behavior of interactive agents but sometimes to construct one (Redpath et al. 2018). That is, “the players are given freedom to explore a range of possible outcomes in strategic situations such that they can reframe the problem and the game, and create new options not initially contemplated by the research team” (418). We illustrate how it works drawing on an example from Worrapimphong et al. (2010); see also Nagatsu (2021).

Worrapimphong et al. (2010) applied the ComMod approach in the context of a razor clam fishery in Don Hoi Lord, a coastal wetland in the upper gulf of Thailand. The context is typical of common pool management, which experimental economists have extensively studied both in the lab and in the field. But the main goal of this study includes collective learning and the promotion of discussion among stakeholders. To this end, the researchers first built a biological state-distribution model of razor clams, which was coupled with a role-playing common pool resource management game. The game was played by the fishermen and local government representatives, who then discussed the outcomes and came up with alternative scenarios such as the establishment and rotation of closed zones for harvesting and quota systems. These proposed scenarios were played out in an agent-based model, and the findings were further discussed by the stakeholders. The main results include the identification of missing scientific information (such as factors affecting clam recruitment processes) as well as of coping management scenarios (such as the quota system), which is not dependent on the missing information about the recruitment process.

The primary purpose of these kinds of processes is to ensure that scientific information has a more immediate impact on public decision making and, importantly, in the context of sustainability science to build consensus for scientifically based solutions and strategies. Sustainability and environmental scientists are often acutely aware of the difficulties and frustrations of trying to get policy makers and others to trust and apply scientifically generated results in their decision making. The science provides the means to measure the various environmental and social consequences of different actions. Sustainability science can be conceived of as a “management science” in this regard (Nagatsu and Thorén 2022), explicitly generating scientific solutions to specific environmental or sustainability situations and issues. However, it is often not considered sufficient to these ends to independently calculate optimal strategies. For all kinds of reasons stakeholders might reject or ignore studies and scientific advice, such as a lack of trust in, or understanding of, scientific studies, as well as political and legal constraints. Similarly, scientists’ models might fail to incorporate crucial pieces of information that stakeholders have. Because these sustainability policies often involve multiple stakeholders with multiple interests and perspectives on what is important and feasible in a given situation, scientific results alone are unlikely to generate consensus particularly if only a subset of those interests are represented. Sustainability scientists acknowledge that fairly representing all interests, while managing various power relations between parties, in a scientific decision-making framework is difficult to do (Turnhout et al. 2020) and working with existing frameworks alone will usually not achieve this. Indeed, just extracting information from stakeholders, even using systematic processes (such as monetary valuation and a multicriteria decision analysis), is often considered inferior to a more directly participatory approach in which stakeholders interact on the model building. Such an “extractive use” (Voinov and Bousquet 2010) preserves a distance between scientists and stakeholders, and provides less potential for corrections to the available, but not necessarily representative or neutral, approaches for which scientists might reach.

A phrase sometimes used to describe the target of participatory modeling (and indeed sustainability science in general) is “social robustness.” Socially robust results (analyses, recommendations, policies, etc.) are ones “acceptable within a wide spectrum of diverse interests and value commitments” (Carrier 2010, 207). This includes interests that may change over time. Socially robust results create opportunities for social consensus on strategies or policies.

To increase the social robustness of decisions participatory modeling operates through two principal mechanisms. In the first place, involving stakeholders in the model production process at these various levels has a colearning function. Stakeholders collaboratively manipulate the modeling framework to explore different assumptions and scenarios. By doing so, they come to better understand the causal relationships at work in the system being studied (if the model is reliable) and how their particular choices and preferences play out in combination with those of others (e.g., in ABM in the ComMod example previously mentioned). As such, with a better understanding of the constraints and trade-offs of pursuing different objectives stakeholders can find a workable solution to a problem that is considered acceptable to all parties. They also come to understand the sources of the model’s reliability, as well as contribute to the latter: “The PM approach is based on the assumption that those who live and work in a system may be better informed about its processes and may have observed phenomena that would not be captured by scientists” (Voinov and Bousquet 2010, 1276). In the preceding example, the participants and scientists colearnt that the quota system would work even though they lacked a detailed understanding of the fish recruitment process.

In the second place, PM has a representative function, in terms of the values and interests the models capture. Stakeholders can be involved in many aspects of modeling including model selection and setting the modeling goals—in terms of what they wish to measure or understand, which assumptions seem warranted to them, and the particular preferences they feel should be represented. In our example, during the discussion of the first role-playing game session, all participants (i) came to see the duration of the rotation as an important factor to explore further, (ii) expressed their hesitation toward closing a fishery area for too long, and (iii) suggested including fishermen from another village who harvest in the same area as players. In this respect stakeholders are in a better position to see how their particular interests and values are represented by the model.

Both mechanisms contribute directly to the social robustness of the modeling outcome. Stakeholders are in a position to better understand the implications of their actions and the real-world constraints governing them; perceive and judge the model’s reliability, in their view at least; and ensure their particular interests and values are represented. PM is thus a vehicle for consensus on the solution of complex problems.

3. Value neutrality and participatory modeling

Philosophers of science have been concerned with the role of nonepistemic or contextual values, interests, and indeed biases in scientific reasoning and scientific decisions for many years. Both the value-free ideal and value-neutrality thesis have been widely challenged through the aid of case studies from across the sciences. The value-free ideal maintains that scientists should strive to minimize the role of nonepistemic values in their decision making, for instance when gathering evidence and assessing results. The value neutrality thesis is the claim that this is in principle possible (Reiss and Sprenger, 2020). With respect to the latter, “inductive risk” studies in particular have illustrated that basic choices for confidence intervals for hypothesis testing invariably requires scientists to fall back on their value preferences, particularly for values related to the consequences of a false negative or false positive. Unequal welfare implications of any decision given the heterogeneity of an affected population also make it inevitable to draw on certain value judgments. Douglas (2000) has argued that these evidential judgements are not the only place in a scientific process at which contextual values matter. Any aspect of a scientific process from study design to methodological choice, to statistical data analysis technique, and so on can be influenced by contextual values. Some of these may be explicit; however, some may be implicit, encoded over time in a field’s standard practice or preference for one type of approach over another.

Much of this research is presented on the basis that the scientific practitioners are not necessarily aware of the value-based decisions or biases inherent in their actual decision making. For example, Winsberg (2018, ch. 3) criticizes climate economists who use particular Integrated Assessment Models for this reason. Scientists—at least in certain domains—may think they work in a value neutral way, agreeing that values might be involved in deciding which questions are valuable and worthy of research, but not with respect to scientific decisions over how best to pursue those questions or determine the reliability or truth of the outcomes (which follows the received view in philosophy of science). Hence there is important work for philosophers to correct misunderstandings scientists may have about science or may express with respect to value neutrality.

However, in the case of sustainability science broadly and participatory modeling in particular scientists are often explicitly aware of the role values and biases can play in scientific research (see Glynn 2017). Part of this stems from a simple understanding of the degree of uncertainty governing many scientific calculations in this area that involve complex coupled human and natural systems. Complexity requires decisions on what to include and what not to include, which often requires priorities be set on what is important to manage uncertainty. Ethical values such as the precautionary principle provide a basis for deciding how to do this and how to weigh the results. Part of it also stems from the fact that sustainability science invariably deals with subjects that often involve many different individuals and groups who might be affected, and the interests or preferences of these people might well be unaligned or in conflict. This fact promotes sensitivity amongst sustainability scientists to their own value-preferences and biases, which can be a problem if unaccounted for, insofar as they can affect the trust of stakeholders in the objectivity and fairness of their results and, overall, the possibility of social consensus on results and policies deriving from them (Kahan 2012). In fact, the role of biases in environmental science and sustainability modeling, not only cognitive but also social and political, has been discussed extensively and in detail (see Hämäläinen 2015; Hämäläinen and Lehtinen 2016; Glynn 2017).

The scientists who have proposed and developed participatory modeling have done so, at least partly as a response then to problems of their own value-ladenness and bias. In principle participatory modeling might be seen as a procedural approach to achieving value-neutrality, or to at least increasing value plurality so as to alleviate problems deriving from a lack of value-neutrality. The way it attempts this should be evident from what we have described previously. The modeling process tries to use direct interactions with stakeholders to both include and represent stakeholder values and preferences directly in a model, and also to help correct the biases modelers themselves might have, including biases deriving from a lack of local knowledge. Having stakeholders participate in setting modeling goals helps modelers avoid setting these goals themselves or employing methods—such as strictly biological ones—that narrow down the sets of possible goals to be considered. This applies to model selection as well. By having stakeholders determine which scenarios to explore, scientists are hopefully prevented from choosing models that are biased in some way or another against any one group’s interests. If stakeholders collect data, a contact point of potential data collection bias amongst scientists is also controlled for. This is not to say that biases are thus removed, but it creates a mechanism that helps ensure that scientists’ own values are not imposed at the expense of those of the other groups.

As such participatory modeling (and other participatory methods) can be understood as a response to the lack of value neutrality in science and engineering decisions. Sustainability scientists—being explicit about the goal- and solution-oriented nature of their field—do not as a rule think that value-neutrality is obtainable, at least in the sense that scientists could work fully independently of contextual values. They do think, however, that within a collective problem-solving environment it will be much harder for their values to dominate the process if exposed to critical reflection and argument. In this way, while not recovering value neutrality or the value free ideal for researchers, PM would nonetheless seem to provide procedural solutions to the problems generated by the fact that these ideals are unachievable. PM helps prevent scientists from, implicitly or otherwise, biasing solutions in unjustified or nonrepresentative ways.

4. The risks of consensus

While PM and other participatory approaches seem on their face innovative and reasonable responses to problems arising from a lack of value neutrality from the scientists and their decisions, this strategy seems to us to come with costs and risks particularly in the context of sustainability science. There are relatively unavoidable trade-offs in the use of these methods that compromise the overall purpose of PM, and raise ethical problems with respect to the use of PM.

To begin with, the goal of representativeness in model choice is difficult and likely unrealistic in many cases. According to PM, “Model selection should be driven by the goals of the participants, the availability of data, the project deadlines and funding limitations, rather than by scientists’ preferred modeling platform and methodology” (Voinov and Bousquet 2010, 1276). In practice, however, various constraints likely produce availability biases in the field. PM is an intensive, often complex time-consuming process, and scientists can find it difficult trying to use tools they are not skilled with. PM involves managing multiple stakeholders who often have variable interests, levels of understanding, and enthusiasm. Producing a model with them can require multiple sessions. At the same time, the natural system to be modeled (e.g., fish stock dynamics, epidemic spread, and climate change) are often complex and uncertain. Modelers should be able to incorporate new information and perspectives from stakeholders into these complex models in a conceptually consistent way. Due to these cognitive and methodological constraints, modelers who publish multiple reports of different PM studies tend to use the same modeling framework without exploring others (Voinov et al. 2016). Because modeling frameworks, such as soft system dynamics modeling and ComMod, work as research paradigms, it is very difficult to be in a position to adapt a wide set of different modeling options to a variety of different circumstances, as opposed to just relying on a single framework a scientist happens to be very skilled with, be it cognitive mapping, Bayesian belief network, systems dynamics, or optimization.

In addition, given colearning and transparency requirements there is an epistemic bias toward simple representations. This is not necessarily a problem insofar as simple models can often perform well on tasks of understanding and communication compared to more complex models. But this does become an issue in complex environmental contexts insofar as the need for understanding and clarity leads to a trade-off in the reliability of the modeling. PM often uses highly simplified representations because the bar on understanding is low (Voinov and Bousquet 2010) or because model-coupling has strong constraints. In our case, Worrapimphong et al. (2010) managed to utilize a detailed resource dynamics model but admitted that coupling it with simulation models in a conceptually consistent manner was a substantial methodological challenge (1336). More simplistic representations, however, do, in general, lose in accuracy and precision compared to more complex representations, and this is always a risk when modeling environmental systems for which it can be fundamentally difficult to produce reliable predictions (Haag and Kaupenjohann 2001).

This raises another more pressing issue. Although the field of PM is aware of issues generally related to complex modeling, and the need for instance to represent uncertainty well to stakeholders, there is little evident discussion of the need for epistemic robustness (as opposed to social robustness) in PM results. Sustainability sciences like climate science in general face issues stemming from the fact that different modeling frameworks will not necessarily give the same answer to similar questions due to underlying biases in the framework (Hämäläinen 2015). Unless results are replicated using different modeling frameworks it is hard to perceive this. Moreover, there are deeper potential robustness problems beyond just these. As already mentioned, PM is supposed to be an iterative process involving direction, mediation, and negotiation between parties and the scientific team. And, indeed, the scientific team often plays a role in managing this process and moving it toward consensus. In this vein scientists play roles they would not normally, which go beyond the modeling alone. However, these processes are likely sensitive to decisions and discussions made during the modeling process. Participants may not necessarily account for their interests consistently. Further, while not uncontroversial, psychology has produced evidence of path-dependence and contingency in “groupthink,” particularly when consensus is a goal. The results are nonreplicable group outcomes (see e.g., Olbrecht and Bornmann 2010). Lastly, even if the participants trust scientists qua facilitators, participants with partially conflicting interests may have incentive to misrepresent their true beliefs and preferences for strategic reasons. Given these social psychological and strategic dynamics running the same process again using the same model does not guarantee the same outcomes will be reached. And it certainly does not guarantee it will be reached when a different model is involved requiring a different set of decisions, or different data collection approaches and so on (see Hämäläinen and Lehtinen, 2016).

These sets of issues raise a relatively stark set of ethical concerns. Namely, without replication studies using different modeling frameworks and different processes, led perhaps by different scientific groups, there is a risk that consensus is being generated using processes that are both context-dependent and unreliable. ¹ Although any decision-making process is likely contingent, in this case the authority of scientific models is thrown into the mix. Stakeholders reach decisions they presumably think are guided and constrained by reality as the models demonstrate, and thus have more force. This is indeed part of the point of bringing stakeholders into the process in the first place. But currently there is no good evidence available that different decisions might not be reached using different models or different processes or running the same process on another day. Nor is there a principled way to proceed when participatory research findings happen to confront the initial claims of some participants (Redpath et al. 2018, 419, describe such a case as “bittersweet”). Should researchers insist that the participants change their opinion and practice, or give them some strategic wiggle room to negotiate what the fact is given model uncertainties? In available PM literature there is little acknowledgment or discernible discussion of this problem currently. And it seems genuinely ethically problematic for scientists (or engineers) to engage in a process of generating consensus through these means, as opposed to, say, for facilitators whose main mandate is to build consensus rather than an epistemically robust ground for social decision making.

5. A case for keeping the boundaries clear

Many discussions in science policy and science and technology studies have over the past twenty or so years pushed for greater, even complete integration between science/engineering and the stakeholders affected by science-based decisions. This has been in the hope of ensuring that scientific results have more meaningful impact and fairness through citizen control. Participatory methods put strong emphasis on having stakeholders deeply involved in the scientific process to ensure that stakeholders’ values play a primary role. Our analysis here would suggest, however, that there are situations or cases in which it is not necessarily desirable to try to eliminate or drastically close the boundary between science and society, at least for the sake of helping to reclaim some kind of scientific value-neutrality for the science involved. This is especially true when dealing with complex natural and social phenomena, in which results are subject to uncertainty and bias, and for which the replication of group-based decisions is a difficult and intensive process. In such cases having participants involved may give legitimacy to decisions and bind participants in ways which are risky or unfair given the unreliability of the scientific information and the nonreplicability of the processes used to reach consensus. Arguably then in such circumstances it would be better to maintain a separation between the scientific production of results and their evaluation and use. For example, such a separation may be achieved by context-rich field experiments (e.g., Samsura et al. 2015), in which scientists and relevant stakeholders can colearn, for example, about how information affects bargaining processes, without directly applying the results or monopolizing the framing of actual social decisions. At the very least, reliability and replicability need to be better factored into discussions about when, how, and which participatory modeling should be used in an ethically sound manner to improve the impact and representativeness of scientific results.