Strategic robustness in bi-level system-of-systems design

Abstract

Robust designs protect system utility in the presence of uncertainty in technical and operational outcomes. Systems-of-systems, which lack centralized managerial control, are vulnerable to strategic uncertainty from coordination failures between partially or completely independent system actors. This work assesses the suitability of a game-theoretic equilibrium selection criterion to measure system robustness to strategic uncertainty and investigates the effect of strategically robust designs on collaborative behavior. The work models interactions between agents in a thematic representation of a mobile computing technology transition using an evolutionary game theory framework. Strategic robustness and collaborative solutions are assessed over a range of conditions by varying agent payoffs. Models are constructed on small world, preferential attachment and random graph topologies and executed in batch simulations. Results demonstrate that systems designed to reduce the impacts of coordination failure stemming from strategic uncertainty also increase the stability of the collaborative strategy by increasing the probability of collaboration by partners; a form of robustness by environment shaping that has not been previously investigated in design literature. The work also demonstrates that strategy selection follows the risk dominance equilibrium selection criterion and that changes in robustness to coordination failure can be measured with this criterion.

1. Introduction

Robust design methods seek to minimize system sensitivity to uncertain events or conditions. Extensive work in the field has developed methods to minimize the impacts of noise factors, design variable uncertainty (Chen et al. 1996; Park et al. 2006), model uncertainty (Choi et al. 2005; Allen et al. 2006) and design process uncertainty (Seepersad et al. 2004). However, fundamentally different sources of uncertainty in systems-of-systems (SoS) arise from interactions among multiple design actors. Distributed decision authority and interdependent utility functions create a game-like dynamic between actors based on available alternatives and expected payoffs. The resulting strategic dynamic influences each actor’s decision, most simply represented as a binary choice between collaborating on a joint system or pursuing an independent system. Robust design methodologies to address multiactor dynamics in SoS are comparatively under-studied yet influential to understand the formation and dissolution of joint system architectures across organizational boundaries.

A successful SoS requires coordination on a common collaborative strategy where each constituent system contributes to the joint system which, in turn, provides greater utility for each member than alternative strategies like independence. A coordination failure occurs when one or more constituent system actors choose or default to (e.g., through a technical failure) a strategy that differs from the common collaborative strategy. Strategic decisions to pursue a collaborative or independent strategy are distinct from design or operational decisions because they are recursively influenced by anticipated strategic decisions of other system actors as well as technical factors that affect system performance.

This work defines strategic stability as a desirable SoS-level property that describes a particular strategic outcome as invariant to small perturbations. As used here, it is closely related to the concept of a Nash equilibrium, a game condition in which no actor can unilaterally improve their outcome by changing strategies (or, in the case of mixed strategies, the frequency of each strategy) (Nash 1951). A strategically stable Nash equilibrium can withstand perturbations to the payoffs (within limits, relatively greater stability allows greater perturbations), namely a strategically stable equilibrium remains an equilibrium over a range of conditions.

Consequently, this work defines strategic robustness as a system-level design property that preserves utility under strategic uncertainty, namely, coordination failure. Robustness and stability work in concert to preserve utility by (a) reducing the potential downside effects of others’ noncollaborative strategies (robustness) and (b) influencing strategy dynamics so that others are more likely to choose collaborative strategies (stability). A system design methodology to mitigate the negative effects of other actor’s decisions and shape the strategic environment to support desirable outcomes requires understanding of each goal and corresponding measurement instruments to support the design process.

As a thematic application representing an SoS, this work considers a transition between existing and next-generation technology alternatives. Technology transitions exhibit operational and managerial independence, key features of an SoS, at both the producer and consumer levels. For example, many organizations and individuals develop and use interacting products that implement an interface standard (e.g., USB) but no central authority governs whether or when a particular manufacturer will transition a product to a particular standard. Furthermore, technology transitions can occur organically, without a common, acknowledged purpose. The interaction between systems composing the technology transition SoS may occur at the technological level, for example, maintaining interface compatibility; at the market level, for example, system actors benefiting from economies of scale and in expanded markets as a technology becomes more ubiquitous; or both.

A fictional mobile computing industry (e.g., smart phone, laptop and peripheral device) whose market functions as an SoS is used as the thematic application for this work. All industry actors are potentially impacted by the introduction of a new wireless communication technology, ‘Greycloak’.¹ Greycloak offers advantages in both range and power consumption over Bluetooth, but is not backwards compatible. System actors must choose between releasing the latest versions of their products using Bluetooth or Greycloak. If commonly interacting devices are both released with the new technology, then the manufacturers increase profits through increased sales volume and prices for the more capable devices. However, if commonly interacting devices do not coordinate on the same technology, then Greycloak implementers will lose sales revenue relative to what they would have earned with a Bluetooth implementation. Bluetooth implementers do not lose revenues in the event of a coordination failure due to the large and established market for such devices.

A portion of the industry has the option of releasing a product with both technologies at a slight penalty to maximum profit under successful coordination due to marginally increased device cost. Such devices mitigate losses under coordination failure via implementation of Bluetooth and Greycloak transceivers, but do not fully recover losses due to greater manufacturing expense and reduced sales (because consumers are unwilling to pay a premium for a device with an interface that does not work with their other devices). However, manufacturers of small peripherals, for example, earbuds, must choose between technologies due to size and weight constraints. Each actor’s goal is maximum revenue.

Technology transitions exhibit tipping points that can be characterized by coordination games with payoffs that grow with adoption (Sarkar 1998; Farrell & Klemperer 2007; Keser, Suleymanova & Wey 2012; Zeppini, Frenken & Kupers 2014); motivating its use in this work as the example SoS. This work differs from other research into technology transitions by focusing on how a strategically robust technology implementation by a subset of the system actors, manifested in the system design, affects selection of a collaborative strategy by all system actors. Consequently, the findings should extend to other SoS that exhibit similar dynamics.

The core contributions of this article are (a) validation of a proposed game-theoretic measure of strategic stability in the context of multiactor networked design games and (b) identification of a relationship between system-level strategic robustness and SoS-level strategic stability in a multiactor agent-based model whereby increasing individual system robustness also increases network-wide collaboration. Results help to establish key theoretical insights about strategically robust design in SoS settings as a precursor to new methodologies for constituent system and SoS communication network design.

The remainder of this article is structured as follows: Section 2 summarizes relevant background information from the fields of robust design, SoS, game-theoretic equilibrium selection, and evolutionary games. Section 3 describes the common elements of the evolutionary game theory model used to answer the research questions. Section 4 presents unique model elements, results and discussion addressing the first research question. Section 5 presents the model, results and discussion for the second research question. Section 6 discusses broader implications for design researchers and practitioners. Finally, Section 7 synthesizes the overall findings and shares concluding remarks.

2. Background

Strategic robustness draws from many bodies of research. This section presents and unites important concepts from biological and artificial system robustness, SoS engineering, game-theoretic equilibrium selection and evolutionary game theory.

2.1. Definitions and applications of robustness

Robustness preserves a system characteristic in the presence of variation of system components (Carlson & Doyle 2002). Designers, researchers and users are often interested in system functionality for both engineered and biological systems (Kitano 2004; Clausing & Frey 2005). Furthermore, Doyle & Csete (2011) assert that a large fraction of design effort and system complexity is attributable to features that enhance system robustness. Clausing & Frey (2005) support this assertion, reporting that many patent claims improve robustness rather than new functionality. Robustness is studied across disciplines ranging from functionality under extreme or accidental loads in structural engineering (Baker, Schubert & Faber 2008) to preserving future options or opportunities under attempts by others to cut off those options in political science (Padgett & Ansell 1993). Robustness clearly plays an important role across disciplines, and especially in engineering and design, as it is a critical property of effective systems.

Research over the last several decades has identified four types of design robustness for engineered systems, each corresponding to a type of uncertainty that can affect a significant system characteristic. Taguchi proposed a design philosophy that reduces variation in performance due to exogenous sources of uncertainty (Type I) (Chen et al. 1996). Chen et al. (1996) developed methods to simultaneously design for Type I uncertainty and endogenous variation such as in design variable values (Type II uncertainty). Choi et al. (2005) augment Chen et al.’s methodology to account for variability in the system performance model used to evaluate candidate designs (Type III uncertainty). Seepersad et al. (2004) identify changes in design specifications and variables and the ‘combined effect of analysis tasks’ as sources of design process uncertainty (Type IV), resembling a combination of Type II and Type III uncertainty. In summary, the objective of these four types of robustness is to preserve system performance or functionality under uncertainty from exogenous sources, design variable values, system performance models and system design processes.

Past research treats designers or system decision-makers (collectively referred to as system actors) as a source of uncertainty in technical factors such as design variable values (Chen & Lewis 1999; Kalsi, Hacker & Lewis 2001; Allen et al. 2006). This perspective does not address strategic decisions, such as system actor participation in a team or collaborative structure, and is an appropriate approach only for design of a single system where there is little strategic uncertainty.

2.2. System-of-systems collaboration

System actors designing a single system ostensibly share a common goal. While their design decisions affect the utility delivered by each other’s design elements, project managers can theoretically resolve conflicts within project constraints to achieve an effective design and mitigate decision conflict. However, unity of purpose does not similarly characterize all SoS because they are comprised of complete, independently managed and operated systems (Maier 1998).

SoS types exhibit different levels of management centralization. Maier (1998) categorizes SoS as directed, collaborative or virtual. OUSD AT&L (2008) adds ‘acknowledged’ SoS to Maier’s list. Only directed SoS are built and managed for a particular collective purpose or set of purposes. Many defense SoS are directed systems. Constituents of an acknowledged SoS share a common purpose managed separately from each system’s goals. However, the constituents retain managerial independence. Collaborative and virtual SoS form voluntarily, and without SoS-specific management, because each constituent system can increase its utility by doing so, either by increasing performance, reducing cost or both. Virtual SoS lack a shared purpose, whereas collaborative SoS retain this feature. The upside benefit of participation depends on mutual collaboration of other actors to overcome initial resource expenditure to enable interoperability.

The combination of upside benefit generated through the SoS and downside risk tied to upfront investments creates a large variation in potential outcomes relative to an independent system. Awareness of these risks, especially as they affect potential collaborative partners, may create uncertainty about the actions of other system actors. These conditions create an environment, particularly for collaborative and virtual SoS, in which system actors desire successful collaboration, but are not certain it will occur and are unlikely to unconditionally commit to collaboration.

Consequently, each system actor is motivated to influence others to adopt a locally favorable policy and to simultaneously pursue a policy that yields the highest utility in response to other actors’ unknown decisions. The risk of coordination failure may be mitigated through contractual agreements which penalize defection and spread risk across the parties or through information exchange which increases confidence that collaboration will be successful. However, contracts are not always viable or effective options, for example, when no formal relationship exists between the actors. Furthermore, contracts generally alter strategy dynamics to reduce the favorability of defection, not increase the favorability of collaboration. Information sharing alone is insufficient to ensure collaboration. For example, the shared information may be negative, and inhibit SoS formation, or it might not be trusted.

The aim of robust SoS design is to reduce the consequences of coordination failures (increasing the favorability of collaboration, conforming to traditional Type I robust design) and to shape the collaboration environment to reduce the probability of coordination failure. Existing design space exploration and optimization methods alone are inadequate to achieve these goals because they do not include the interactive decision models required to assess strategic issues or the effect of design on strategy dynamics.

2.3. Game-theoretical equilibrium selection

Grogan & Valencia-Romero (2019) developed a system design model which represents the collaborative dynamic as a type of bipolar game commonly referred to as stag hunt. Bipolar games are two-strategy normal-form games with two strict Nash equilibria (Selten 1995). Applied to SoS, the strategy space $ {\mathcal{S}}_i=\left\{{\phi}_i,{\psi}_i\right\} $ denotes actor $ i $’s ‘independent’ and ‘collaborative’ strategies respectively. SoS formation requires that successful collaboration is mutually beneficial. This work further assumes that independence is strictly superior to investing in a failed collaboration, completing the conditions required for a stag hunt. Table 1 shows normal-form payoffs $ {V}_i^{s_i,{s}_j} $ for an example stag hunt game using the Bluetooth and Greycloak technology transition labels where numeric payoff values encapsulate actor utilities including other factors like risk aversion. To note, the game payoffs in this section only illustrate equilibrium selection concepts and the bi-level system design model. They are not intended to represent a particular technology transition.

Table 1. Normal-form payoffs for a symmetric two-actor technology transition game modelled after a stag hunt

The equilibria for two actors $ \mathcal{A}=\left\{1,2\right\} $ are $ \phi =\left\langle {\phi}_1,{\phi}_2\right\rangle $, all independent, and $ \psi =\left\langle {\psi}_1,{\psi}_2\right\rangle $, all collaborate. As an analogy for SoS design, each actor has a choice between a safe strategy $ {\phi}_i $ with a relatively low payoff that does not require collaboration (canonically referred to as ‘hare hunting’), and a riskier strategy $ {\psi}_i $ that requires successful collaboration to achieve a higher payoff than hare hunting (canonically referred to as ‘stag hunting’). Stag hunters with defecting partners suffer a loss relative to what could be obtained if they had hunted hare.

In the context of the mobile computing technology transition example, the hare hunting strategy retains the Bluetooth technology and is implemented by a Mk I product design. The stag hunting strategy transitions to the Greycloak technology with superior technical potential and is implemented by a Mk II product design. A pair of interacting system actors successfully collaborate when both adopt the Greycloak technology.

Technology transitions in a commercial environment are also influenced by competitive forces, that is, actors may adopt a new technology to gain or maintain a long-term competitive advantage. Therefore, competition exerts forces on system actors beyond the direct value of short-term gains in utility. Furthermore, the interaction between complementary and substitute goods produces complex dynamics that may accelerate or inhibit transition to a new technology. This work focuses strictly on the complementary relationship between system actors, rather than competitive forces, to reduce assumptions with overlapping impacts and improve the interpretability of the results.

Design and strategy choices are linked in the context of the application case, that is, implementing a Mk II design indicates an actor intends to collaborate in the market with others adopting the Greycloak technology. However, this is not universally true of SoS. For instance, diverse defense systems may adopt a set of common communication protocols to enable combined operations in an SoS but the systems may intermittently or never be used as an SoS. In such a context, the design enables a strategy but it does not determine it.

Rational actors select a strategy in games such as a stag hunt on the basis of two criteria: payoff dominance and risk dominance (Harsanyi & Selten 1988). The payoff-dominant strategy is an element of a Pareto efficient equilibrium, yielding the highest possible payoff for each actor. In the examples used here, the Greycloak strategy $ {\psi}_i $ is always payoff-dominant. The risk-dominant strategy exhibits the lowest risk when considering interactive effects and, as subsequent sections demonstrate, has the greatest basin of attraction, meaning actors tend to select it in repeated games on a multiactor network (Kandori, Mailath & Rob 1993; Grogan & Valencia-Romero 2019).

Risk dominance is analogous to expected value maximization under specific conditions. The normalized deviation loss $ {u}_i $, in Eq. (1), denotes the threshold probability $ {p}_j $ of actor $ j $ selecting strategy $ {\psi}_j $ that equalizes the expected value of $ {\phi}_i $ and $ {\psi}_i $ for actor $ i $. For symmetric games, $ {u}_i={u}_j $, and therefore $ \phi $ (conversely, $ \psi $) maximizes expected value and is risk dominant for $ {u}_i>0.5 $ (conversely, $ {u}_i<0.5 $) if actors play strategic alternatives with equal probability.

(1)$$ {u}_i=\frac{V_i^{\phi_i,{\phi}_j}-{V}_i^{\psi_i,{\phi}_j}}{\left({V}_i^{\phi_i,{\phi}_j}-{V}_i^{\psi_i,{\phi}_j}\right)+\left({V}_i^{\psi_i,{\psi}_j}-{V}_i^{\phi_i,{\psi}_j}\right)} $$

The weighted average log measure (WALM) of risk dominance, $ R $, proposed by Selten (1995), extends the concept of risk dominance to asymmetric and many actor games with linear payoff functions by accounting for interactive effects

(2)$$ {\displaystyle \begin{array}{rcl}R& \equiv & \sum \limits_{i=1}^n{w}_i(A)\ln \frac{u_i}{1-{u}_i}\end{array}} $$

where $ {w}_i(A) $ measures the weight of actor $ i $ based on an influence matrix $ A $, noting that $ {w}_i(A)=\frac{1}{2} $ for all two-actor games. See Selten (1995) for a general derivation of these values. $ R>0 $ indicates that $ \phi $ is risk-dominant and $ \psi $ is risk-dominated, while $ R<0 $ indicates the opposite, namely $ \psi $ risk-dominates $ \phi $. Furthermore, the magnitude of $ R $ indicates the strength of dominance.

Using payoffs from Table 1, Figure 1 illustrates $ {u}_i=\frac{2}{3} $ as the intersection between expected value curves for the $ {\phi}_i $ (blue solid line) and $ {\psi}_i $ (red dashed line) strategies. Following symmetry, the risk dominance criterion evaluates to $ R=2\cdot \frac{1}{2}\ln \left(\frac{2/3}{1/3}\right)=0.69 $, indicating that the Bluetooth equilibrium $ \phi $ is risk-dominant. In general, symmetric games with $ {u}_i>0.5 $ result in $ R>0 $ but asymmetric games depend on the relative magnitude of all $ {u}_i $ values and the influence weights of each actor.

Figure 1. Expected value of actor $ i $’s strategies under uncertainty in actor $ j $’s strategy for the baseline scenario in Table1 and the scenario with the Mk II-B design in Table 3.

Whether payoff or risk dominance should ultimately determine equilibrium selection has been a matter of inquiry for game theorists. Harsanyi & Selten (1988) defined the concepts of payoff and risk dominance, asserting at the time that payoff dominance is the relevant equilibrium selection criterion for stag hunt games. However, Harsanyi later reversed his position, concluding that risk dominance should be the governing selection criterion (Harsanyi 1995). Carlsson & van Damme (1993) likewise concluded that risk dominance is the only rational strategy for incomplete information bi-stability games, such as stag hunt.

The importance of risk dominance also makes sense in systems design settings because using payoff dominance to guide decisions would simply optimize solutions to maximize utility under successful collaboration, ignoring downside risks. Payoff dominance tells a designer nothing about the stability of collaboration or strategic robustness of the system. Such designs are likely to be over-optimized and fragile. Instead, system actors may use risk dominance, specifically the risk dominance criterion $ R $, to guide payoff-dominant collaborative solutions to a more robust state.

2.4. Strategically informed design

Design can influence a system actor’s payoffs under the possible strategic outcomes, thereby modifying the relative risk and payoff dominance of available strategies. For example, actions to reduce losses under a coordination failure can increase risk dominance of a payoff-efficient strategy. Such a design achieves robustness by directly mitigating the impact of a coordination failure but also by reducing the probability of coordination failure if partners also use risk dominance to inform strategy selection. Therefore, an emanation of Selten’s risk dominance criterion for engineering is that system design can reduce losses under coordination failure and influence strategy dynamics to reduce defection by system actors as a form of environment modification (Whitacre 2012) that increases utility robustness. Grogan & Valencia-Romero (2019) used this relationship to develop a bi-level model of SoS design in which system actors each select an upper-level strategy $ {s}_i\hskip1.5pt \in \hskip1.5pt {\mathcal{S}}_i $ and a lower-level design $ {d}_i\hskip1.5pt \in \hskip1.5pt {\mathcal{D}}_i $. The value function $ {V}_i^{s_1,\dots, {s}_n}\left({d}_1,\dots, {d}_n\right) $ maps decisions to payoff values for each system actor.

Consider a new game in which each system actor has two decisions. The first is an operational design decision: a choice between technical implementations in design space $ {\mathcal{D}}_i=\left\{{\alpha}_i,{\beta}_i,{\gamma}_i\right\} $ with assigned labels Mk II-A, Mk II-B and Mk I. The second is, as before, a choice between strategies in strategy space $ {\mathcal{S}}_i=\left\{{\phi}_i,{\psi}_i\right\} $. Table 2 shows the payoffs $ {V}_i^{s_i,{s}_j} $ for each strategy pair-design combination. To simplify the presentation, payoffs are only a function of the design and strategic context, not the partner’s design. For comparison, the prior game in Table 1 adopts the Mk I design $ {\gamma}_i $ under the Bluetooth strategy $ {\phi}_i $ and the Mk II-A $ {\alpha}_i $ design under the Greycloak strategy $ {\psi}_i $.

Table 2. Actor $ i $’s strategy-specific design payoffs for a technology transition design game

In a trade typical to design (Doyle & Csete 2011), adopting the Mk II-B design $ {\beta}_i $ under the Greycloak strategy sacrifices a small amount of upside utility relative to Mk II-A for a large reduction in downside risk, enhancing robustness to coordination failure. Despite being dominated in both the $ \left\langle {\phi}_i,{\phi}_j\right\rangle $ and $ \left\langle {\psi}_i,{\psi}_j\right\rangle $ strategic contexts, the Mk II-B design reduces the normalized deviation loss to $ {u}_i=\frac{1}{4} $, as shown in the normal form game in Table 3 and the green dash-dot line in Figure 1. If the Mk II-B design is selected by both actors, the risk dominance criterion decreases to $ R=2\cdot \frac{1}{2}\ln \left(\frac{1/4}{3/4}\right)=-1.10 $, indicating that the payoff-dominant Greycloak equilibrium $ \psi $ is also risk-dominant.

Table 3. Normal-form payoffs for a symmetric technology transition design game instance

An asymmetric scenario in Table 4 combines the two cases. If one actor’s strategy-design pairs are ($ {\phi}_i $, $ {\gamma}_i $) and ($ {\psi}_i $, $ {\alpha}_i $) and the others are ($ {\phi}_j $, $ {\gamma}_j $) and ($ {\psi}_j $, $ {\beta}_j $), normalized deviation loss alone suggests a different ‘best’ strategy for each actor. However, neither $ \left\langle {\psi}_i,{\phi}_j\right\rangle $ nor $ \left\langle {\phi}_i,{\psi}_j\right\rangle $ are equilibria. Risk dominance resolves the decision dilemma by accounting for the influence each actor has on the other’s utility and the relative preference of each strategy for both actors. In this example $ R=\frac{1}{2}(0.69)+\frac{1}{2}\left(-1.10\right)=-0.20 $, indicating that the Greycloak equilibrium $ \psi $ is risk dominant.

Table 4. Normal-form payoffs for an asymmetric technology transition design game instance

Assuming risk dominance drives strategy selection in the bi-level model reveals a valuable insight for strategic robustness and stability. The risk dominance criterion $ R $ indicates which equilibrium has the larger basin of attraction and its relative strength. Specifically, the magnitude of $ R $ captures the stability of the risk-dominant strategy, namely to what degree equilibrium selection (and thereby strategy selection) resists change due to uncertainty or perturbation. As the preceding bi-level design example illustrates, lower-level design decisions that trade system efficiency for robustness can influence strategy dynamics by changing the utility resulting from a particular strategic outcome, thereby increasing the basin of attraction of the collaborative strategy (Valencia-Romero & Grogan 2020). Strategic stability further increases utility robustness by reducing the probability of coordination failure in a positive feedback loop. Consequently, system actors can use $ R $ as an indicator of both the stability of a strategy and the strategic robustness of constituent system designs in an SoS.

However, these observations naturally lead to questions regarding the viability of $ R $ to describe strategy selection and the influence of design decisions on strategy dynamics in more realistic environments. The effect of robust design implementation on strategy selection by system actors with fragile design options is of particular interest (e.g., the effect of one actor selecting the Mk II-B design on the strategy selection of another actor). The following subsections present models and research questions to investigate the effect of strategically robust designs on actors’ strategy selections in multiactor networks.

2.5. Evolutionary games and strategy selection

Evolutionary game theory provides methods to determine the stability of strategies and to recommend or evaluate equilibrium selection criteria for multiactor, repeated games as more representative settings for decision-making than two-actor single-shot games as in the preceding section. Agent-based (actor-based) evolutionary game theory models simulate the fitness and spread of strategies (or other actor attributes) by representing actors as nodes in a graph and relationships among actors as edges between nodes (Szabó & Fáth 2007). Evolutionary game theory models actor interactions as games and cumulative or single-round payoffs indicate the fitness of each actor. That is, evolutionary games use relatively many actors in a network, interactions restricted by the network topology, and actor property updates based on a fitness measure. Researchers study different phenomena by adjusting network topology, strategy update rules, frequency and synchronicity, and payoff structures to represent various real-world conditions. Studies generally investigate the distribution or prevalence of strategies under equilibrium conditions.

Kandori et al. (1993) found that large, well-mixed populations of simulated actors converge to the risk-dominant equilibrium in coordination games (of which stag hunt is a type) with infrequent myopic best-response strategy updates and low probability mutations of actors’ strategies. Consistent convergence to the risk-dominant equilibrium with small strategy selection perturbations demonstrates that it is the stochastically stable equilibrium, also known as evolutionarily stable. Without mutations, convergence is path dependent and determined by the mixed strategies and the proportion of each strategy in the initial population. Referencing the normalized deviation loss $ {u}_i $ computed from payoff values, Roca, Cuesta & Sánchez (2009b) demonstrated that a well-mixed population without mutations using a best-response update rule, playing a symmetric stag hunt game, will converge to the $ \psi $ equilibrium if the proportion of actors playing $ {\psi}_i $ is greater than $ {u}_i $.

Roca, Cuesta & Sánchez (2009a) performed a comprehensive study of the effects of temporal, spatial, and strategy update rules on strategy selection and stability in evolutionary games, relaxing the well-mixed population and slow update assumptions of Kandori et al. (1993). Many works prior to Roca et al. (2009a) attempted to draw universal conclusions about the effect of individual features, such as topology or selection pressure on strategy distributions. However, Roca et al. (2009a) demonstrated that the effects caused by a given feature are generally convolved with other factors. The results presented by Roca et al. (2009a) extend the findings of Kandori et al. (1993) to confirm that strategy selection in stag hunt games is described by risk dominance on a variety of network topologies, including random, scale free and lattice networks, when a best-response strategy update rule is used. Therefore, the combinations of details such as the network topology, update rule, and payoff structure are critical for achieving meaningful results.

Braha (2020) further supports the importance of graph topology. His work reports that real-world problem-solving networks are often characterized by high frequencies of three- and four-node directed subgraphs that accelerate convergence to a network-wide solution state, that is, coordination, in a stochastic problem-solving model. These coordination-facilitating graph motifs work to reduce cycling between states. The findings of Braha (2020) may play an important role in establishing the system actor communication networks of future SoS to maximize coordination.

2.6. Research objectives

The bi-level design model presented in the preceding section assumes that risk dominance, as expressed in $ R $, accurately models strategy selection in the stag hunt game for utility-maximizing actors. Classic game-theoretical works such as Carlsson & van Damme (1993) and Harsanyi (1995) assert that risk dominance is the relevant equilibrium selection criterion in a single shot stag hunt game. Furthermore, evolutionary game theory, subject to specific model features, also indicates that risk dominance is the governing equilibrium selection criterion in stag hunt games with returns that increase linearly with the number of collaborating neighbors. The support from Roca et al. (2009a) is particularly valuable, as their models examine a variety of network types and policy update rules that may be applicable to SoS development and operations. However, the application of risk dominance to evolutionary games with increasing or decreasing marginal returns to collaboration, a common attribute of real-world SoS, has not been examined. This work poses the following question, testing an expanded application of risk dominance as a measure for subsequent work:

1. RQ1. How does risk dominance predict selected strategies in single-level, multiactor evolutionary games representing SoS with payoff functions exhibiting (a) constant, (b) increasing and (c) decreasing marginal returns to the number of collaborators?

Subquestions (a–c) address scenarios when each additional collaborator is of equal value to the focal actor, increasing value to the focal actor, and decreasing value to the focal actor, respectively.

Therefore, the following hypothesis is made for RQ1:

1. H1. Strategy selection in single-level, multiactor network games representing SoS follows the risk dominance criterion, with Pareto efficient strategies selected with greater frequency as $ R $ decreases.²

Further refined, the majority of system actors are hypothesized to select the Pareto efficient strategy if $ R<0 $ (the threshold value of $ R $) for constant marginal returns to the number of collaborators. The threshold value of $ R $ is less than zero for increasing marginal returns due to the relatively low value of initial collaborators. Lastly, the threshold value of $ R $ is greater than zero for decreasing marginal returns due to the high value of initial collaborators. Examining all three scenarios is important to establish the relationship between $ R $ and strategy selection across a spectrum of payoff functions representative of SoS returns.

The bi-level model and the relationship between strategic robustness and stability critically depend on the concept that design decisions in an SoS context that increase robustness to others’ decisions also reduce the probability of a coordination failure by reinforcing others’ favorable strategy selection. Conclusions drawn by Grogan & Valencia-Romero (2019) based on the work of Selten (1995) support this concept. However, no published research has tested the idea in a dynamic setting representative of SoS. Furthermore, direct support from the body of literature in evolutionary game theory provides insufficient support because relatively little research has been done on equilibrium selection in asymmetric games or with actors with asymmetric options, especially in an engineering context. This gap leads to the second research question:

1. RQ2. Does adoption of strategically robust designs by a subset of a population in an evolutionary bi-level design game increase collaboration amongst all system actors, including those with relatively fragile designs?

Testing RQ2 requires games with asymmetric payoffs and asymmetric options, as expressed in the design decision layer. Combining the findings of Roca et al. (2009a) with the theory presented by Selten (1995), the following hypothesis is made for RQ2.

1. H2. Designs that increase strategic robustness reinforce favorable strategic behavior from others, increasing the likelihood of collaboration by all actors in a multiactor bi-level design game representation of SoS.

The research proposes two computational simulation experiments based on a common multiactor model to evaluate the hypotheses. The first experiment evaluates how well risk dominance describes equilibrium selection in a multiactor network, confirming the foundation for the second experiment. The second experiment tests whether designs that increase strategic robustness (system robustness to coordination failure), lowering $ R $, increase collaboration among all actors, including those with relatively fragile system designs.

3. Multi-actor SoS model

This section describes the SoS used as the context for this research and common components of a multiactor simulation model with network topology, update rule, and payoff structure representative of SoS. All games are played with payoffs that produce stag hunt dynamics. Simulations are performed with NetLogo and leverage several existing models (Wilensky 2005a,b).

3.1. Model constructs

The fundamental features of each model are a set of actors, a game with payoffs, and a network connecting the actors. Each actor’s neighborhood is defined by the network and includes all actors with whom they are connected directly by an edge. Actors play the game with each neighbor simultaneously, using the same strategy and design (as applicable) in all interactions within the same round of play. Actors may update their design and strategy decisions between each round of play solely on the basis of past decisions (their own and neighbors’), corresponding payoffs, and possible future payoffs as a function of strategy and design options. Rounds of play continue until termination criteria are met.

Each round of the simulation represents the outcome of a complete set of design and strategy decisions. Simultaneous move games are a proxy for a scenario in which actors commit resources to a particular policy based on beliefs about the future actions of other actors, but only discover what path has been taken after some time has passed. In real-world SoS, this may occur because actors signal the intent to collaborate, but do not follow through, or because collaboration breaks down before system realization. The simulations do not represent the design process itself. In other words, all design and strategy decisions culminating in system deployment are combined into one round of the simulation. Therefore, simulated actors learn their best one-shot actions, not the best action sets for a repeated game representation of an SoS design process. Strategy is used here to refer to the nontechnical decision to collaborate (embodied in the application case as a decision to adopt the Greycloak technology) in the SoS or not. Actor action spaces are defined by their design and strategy decision spaces. An actor’s policy, composed of a design and strategy choice, is updated using the policy update rule described in a later subsection.

Design choices in this context determine strategy, that is, if an actor implements Greycloak, then it is also collaborating in the market with others that do the same. However, this is not universally true of SoS. For instance, diverse defense systems may adopt a set of common communication protocols, enabling combined operations in an SoS, but the systems may intermittently, or never be used as an SoS. In such a context the design may enable a strategy (whether to collaborate), but it does not determine it.

Additionally, strategies are evaluated and selected on the basis of their utility under the possible strategic outcomes, and not with a view to long-term business objectives, or competitive concerns that are typically included in an organization’s long-term planning activities. These considerations could be included in a utility measure used to evaluate design and strategy options, but are excluded from this work to increase simplicity and interpretability of the results.

3.2. Network topology

Each model implements three network topologies, each with 30 actors, as a contextual variable. Including network topology helps to test the hypotheses over a broad range of conditions, thereby increasing the generalizability of common conclusions. The effect of topology itself is not a subject of interest for this work, and is therefore not considered one of the independent variables in the experiments to test each hypothesis.

The three network topologies include a random network, a random network with preferential attachment, and a small-world network (Watts & Strogatz 1998; Barabási & Albert 1999). All three networks are prevalent in the literature on multiactor simulations and evolutionary game theory (Roca et al. 2009a). A random network randomly places edges between nodes with equal probability (Erdös & Rényi 1959); in the case of this work, until each node has a minimum number of edges. Random networks with preferential attachment are constructed by adding nodes to a minimal network (as few as two nodes and one edge) by connecting the new node to an existing node with probability equal to the ratio of the existing node’s edges and total number of edges in the network (Barabási & Albert 1999). Small-world networks are formed from a ring lattice in which each node is connected to specified number of nearest neighbors. Edges are randomly rewired to a new node with a specified probability, without forming self-loops or duplicate connections (Watts & Strogatz 1998). Small-world networks increase node clustering relative to random graphs, without large average path lengths between nodes. The random networks are simulated with minimum node degree between 1 and 2.

Braha & Bar-Yam (2007) studied new product development networks for a vehicle, operating system software, a pharmaceutical facility, and a 16-story hospital and found that all had networks with high-clustering and short path lengths, typical of small-world networks. Therefore, this work assumes that small-world networks are the most representative of those typical in SoS development. However, as relevant network structures likely vary between industries and specific projects, comparisons across the three topologies bound outcomes and help compare to previous results.

3.3. Update rule

The policy update rule is a critical feature of any evolutionary game to update decisions in response to environmental stimuli. The policy update rule determines which actions actors make in generations subsequent to the seed generation. This work uses the myopic best response (MBR) update rule proposed by Blume (1993) and Kandori et al. (1993) with an implementation similar to Sysi-Aho et al. (2005). Montanari & Saberi (2010) also used the best response update rule in their study of the spread of innovations using Markov chain models.

Consider the set of actors $ \mathcal{A}=\left\{1,\dots, n\right\} $. Every actor $ i $ is connected to a finite set of other actors (its neighborhood) denoted by $ {\nu}_i $. The number of collaborating actors in actor $ i $’s neighborhood, $ \mid \left\{j\hskip1.5pt \in \hskip1.5pt {\nu}_i:{s}_j=\psi \right\}\mid $, is denoted by $ {c}_i $. Actor $ i $ chooses a strategy $ {s}_i^{(t)}\hskip1.5pt \in \hskip1.5pt {\mathcal{S}}_i $ and a design $ {d}_i^{(t)}\hskip1.5pt \in \hskip1.5pt {\mathcal{D}}_i $ from a set of alternatives at time $ t $, plays the game with each neighbor, and receives total payoff $ {G}_i^{(t)}\left({s}_i^{(t)},{d}_i^{(t)}\right) $.

$ G $ takes one of three functional forms: for constant marginal returns with $ {c}_i $:

(3)$$ {G}_i^{(t)}=\sum \limits_{j\hskip1.5pt \in \hskip1.5pt {\nu}_i}\;{V}^{s_i^{(t)},{s}_j^{(t)}}\left({d}_i^{(t)}\right), $$

for increasing marginal returns with $ {c}_i $

(4)$$ {\displaystyle \begin{array}{rc}{G}_i^{(t)}=& {V}^{s_i^{(t)},{\phi}_j}({d}_i^{(t)})+[{V}^{s_i^{(t)},{\psi}_j}({d}_i^{(t)})-{V}^{s_i^{(t)},{\phi}_j}({d}_i^{(t)}){+1]}^{c_i^{(t)}/|{\nu}_i|}-1,\end{array}} $$

or for decreasing marginal returns with $ {c}_i $

(5)$$ {G}_i^{(t)}={V}^{s_i^{(t)},{\phi}_j}\left({d}_i^{(t)}\right)+\left[{V}^{s_i^{(t)},{\psi}_j}\left({d}_i^{(t)}\right)-{V}^{s_i^{(t)},{\phi}_j}\left({d}_i^{(t)}\right)\right]\cdot {\log}_{\mid {\nu}_i\mid +1}\left({c}_i^{(t)}+1\right). $$

The three payoff function forms represent possible network effects in the SoS. Eq. (3) represents the case when an actor’s payoff grows linearly with the number of collaborators in the SoS. Eqs. (4) and (5) represent more complex network effects in which many or few potential collaborators must participate in the SoS for collaboration to be profitable. Real-world SoS may also be characterized by networks with interactions with varied weights, that is, the collaboration of some actors is more valuable than that of others, regardless of whether marginal returns are constant, increasing, or decreasing.

Actors select new $ {s}_i $ and $ {d}_i $ at time $ t+1 $ with probability $ p $ using MBR by

(6)$$ {s}_i^{\left(t+1\right)},{d}_i^{\left(t+1\right)}=\left\{\begin{array}{ll}\underset{s\hskip1.5pt \in \hskip1.5pt \mathcal{S},d\hskip1.5pt \in \hskip1.5pt \mathcal{D}}{\arg \hskip0.1em \max }{G}_i^{(t)}\left(s,d\right),& \mathrm{if}\;k<p\\ {}{s}_i^{(t)},{d}_i^{(t)},& \mathrm{otherwise}\end{array}\right., $$

where $ k $ is a uniform (0,1) distributed random variable. In other words, each actor adopts the design and strategy to those that would have maximized $ G $ in the previous round of play with probability $ p $. The probabilistic formulation disrupts simultaneous updates and mitigates the possibility of cycling between strategies or designs. The MBR update rule rationally, but short-sightedly, maximizes payoff by selecting strategies as if each round of play were the last (Kandori et al. 1993).

As an example, consider the strategic network state in Figure 2 where dark blue actors play the Bluetooth strategy $ {\phi}_i $ and yellow actors play the Greycloak strategy $ {\psi}_i $ (ignoring design decisions in this example). Assume that marginal returns are constant and actor $ i $ realizes payoffs per Table 3 with $ {s}_i^{(t)}={\phi}_i $, receiving a total payoff $ {G}_i^{(t)}=0.4\cdot 2+0.8\cdot 2=2.4 $ for this network state. Alternatively, if actor $ i $ had selected $ {s}_i^{(t)}={\psi}_i $, then the corresponding payoff would have been $ {G}_i^{(t)}=0.35\cdot 2+0.95\cdot 2=2.6 $. Therefore, the MBR rule updates the strategy to $ {s}_i^{\left(t+1\right)}={\psi}_i $ with probability $ p $.

Figure 2. Example game graph. Yellow actors play $ {\psi}_i $ and dark blue actors play $ {\phi}_i $.

MBR has several features that make it well suited for application in an SoS design model and justify its implementation here. Simple imitation rules like ‘follow the best’ require actors to know the best actor with highest payoff, either in the neighborhood or globally, and the best actor’s selected design and strategy. MBR requires only local information; namely, an actor’s own actions and the effect of its neighbors’ actions. Actors’ strategies and the effect of their actions on their neighbors are synonymous in most evolutionary game theory models. In the model used in this work, only neighbors’ strategies affect actor payoffs, and are therefore visible and possible to imitate. Neighbors’ design decisions are not known and cannot be imitated. Furthermore, real-world interactions seldom reveal complete and perfect information about a design actor’s strategy, design, or resulting payoffs, even if neighbors’ designs do affect one’s payoff. Lastly, MBR represents innovative, rational, but limited strategic behavior of the type expected from a process such as decision making in business or design, while maintaining simplicity, continuity with previous research, and interpretable results.

3.4. Model limitations

The implementation of each model feature discussed in this section introduces limitations to the application of results to engineering design. Several are discussed here and others are left for discussion in context of the experiments presented in the following sections.

The basic model construct introduces several limiting features. MBR is used because it produces actor behavior that is short-sightedly rational and requires only local information. However, as van Duinen et al. (2016) demonstrate, an actor selecting new technology strategies may switch between policy update processes using repetition, deliberation, social comparison and imitation depending on the actor’s level of satisfaction and uncertainty. MBR is a deliberative policy update rule. As cited earlier, imitation (and social comparison, which is imitative) requires knowledge of neighbors’ past policies. Strategy update rules representative of behavior amongst design actors in an SoS setting is an area requiring additional research. Therefore, this work employs MBR as an adequate proxy for human decision making rather than introducing a complex rule that is not empirically supported.

Second, the model only considers populations with homogeneous payoff functions. Actors in real-world SoS will often have different payoff functions and be influenced to varying degrees by the other actors in the network. Heterogeneous payoff functions and varied actor weight would also introduce additional assumptions that complicate the interpretation of the results, and are, therefore, not considered in this work.

Lastly, the structure of the networks also influences the dynamics by creating clusters that can protect risk-dominated solutions, effectively biassing the local probability of collaboration. Real-world networks are sometimes dynamic (Braha & Bar-Yam 2006), which could minimize or amplify the impacts of clustering or transfer them temporally and spatially to a new network region. Like other excluded features, the inclusion of dynamic networks would increase model complexity and required assumptions and therefore are reserved for future work.

4. Risk dominance and strategy selection

This section presents unique design elements, results, and discussion for the experiment testing H1.

4.1. Experimental design

The first experiment uses the network topologies and strategy update rule presented in the preceding section to evaluate the effect of risk dominance on strategy selection. Variable payoffs represented by $ S\hskip1.5pt \in \hskip1.5pt \left[-2,0\right] $ and $ T\hskip1.5pt \in \hskip1.5pt \left[\mathrm{0,0.975}\right] $ in Table 5 form the independent experimental variables. The notation for $ S $ and $ T $ aligns with social dilemma game theory literature to represent the payoffs known as the ‘sucker’s payoff’ and ‘temptation to defect’. Varying $ S $ and $ T $ over the corresponding domains produces a range of risk dominance $ R $ values in Figure 3.

Table 5. Normal-form payoffs for a symmetric technology transition game with variable payoffs $ S $ and $ T $

Figure 3. Risk dominance ($ R $) contours as a function of variable payoffs $ S $ and $ T $.

The dependent variable is the density of collaborative strategies selected in equilibrium, measured as the ratio of the number of actors playing the collaborative strategy $ {\psi}_i $ to the total number of actors.

Each combination of topology and payoffs is simulated 1000 times with each actor initially equally likely to implement either strategy. Each of the 1000 trials is initialized with a new graph and a new set of actors, that is, the graph structure is unique for each trial; decisions and dynamics do not carry over from trial to trial. Actors play their neighbors, collect payoffs according to the specified form of $ G $, and then update their strategies using MBR with probability $ p=0.9 $. Each trial is terminated when the number of actors playing each strategy is unchanged in four consecutive rounds of the game. Results for each trial and experimental condition are output as a CSV file with a record of the dependent and independent variables. Theory and past research suggest that, for an initial probability of each actor selecting $ \psi $ of $ 0.5 $, games with $ R>0 $ will converge to $ \phi $ while games with $ R<0 $ will converge to $ \psi $ when the payoff function is linearly increasing with the number of collaborating neighbors.

4.2. Results

Figure 4, row A shows contours representing the threshold values of $ S $ and $ T $ for which the majority of actors implement the collaborative strategy for constant, decreasing and increasing marginal return payoff functions. Red line overlays represent the iso-$ R $ contours for values $ -1 $, 0 and 1. The heat maps in Figure 4, rows B–D show the mean final percentage of collaborators for the range of $ S $ and $ T $. Each cell represents the mean dependent variable value over 1000 trials for one combination of independent variables. The strategy percentage plots visually characterize equilibrium selection for each $ \left(T,S\right) $ game configuration. Figure 4, columns 1–3 displays results for the small world, random with minimum node degree $ =1 $, and random with minimum node degree $ =2 $ networks, respectively. Preferential attachment network results are very similar to the small-world network results. Therefore, they are excluded from the presentation.

Figure 4. Final collaborator ($ {\psi}_i $) density for columns (1) small-world networks, (2) random networks with minimum node degree $ =1 $ and (3) random networks with minimum node degree $ =2 $. Row A shows contour lines for threshold values of $ S $ and $ T $ above which more than half of the actors collaborate (red lines are iso-$ R $ contours for values $ -1 $, 0 and 1). Other rows show heat maps of final collaborator density for variable $ S $ and $ T $ when marginal returns are: B, constant; C, decreasing and D, increasing.

All plots in Figure 4 show a clear increase in collaboration as $ R $ decreases. A unique linear threshold exists for each combination of payoff function and graph topology, above which collaboration is prevalent. For constant marginal returns, the 50% collaboration threshold is on the line representing $ R=0 $ for all topologies. Striations visible in regions above and below the 50% collaboration threshold contour correspond to graph regions where the network structure enables a strategy to persist despite not being the best response to an equal density of actors using each strategy in networked games.

Consider an example converged game state in Figure 5 on a preferential attachment network with $ S=-1 $ and $ T=0.2 $ ($ {u}_i=0.555,R=0.223 $) and constant marginal returns where yellow cells represent actors playing $ {\psi}_i $. The $ R $ value indicates that $ \phi $ is risk-dominant and is the expected strategy for all actors, that is, all the nodes should be dark blue. However, the structure of the network allows for exceptions to this rule. Case in point, actor $ i $ plays $ {\psi}_i $ and has four neighbors that play $ {\psi}_j $ and one that plays $ {\phi}_j $; so its payoff is $ 1\cdot 4-1\cdot 1=3 $. If $ i $ played $ {\phi}_i $ its payoff would be $ 0.2\cdot 4+0\cdot 1=0.8 $. Therefore, actor $ i $ will persist playing $ {\psi}_i $.

Figure 5. Preferential attachment network in equilibrium with $ S=-1.0 $ and $ T=0.2 $. Yellow nodes are actors playing $ \psi $, blue nodes are actors playing $ \phi $.

Actor $ i $ can persist playing $ {\psi}_i $, despite it being risk-dominated, as long as $ S\ge 4\left(T-1\right) $. Similar conclusions can be observed in a single-shot two-actor game if actor $ i $ can reliably estimate a probability of collaboration $ {p}_j=\frac{4}{5} $ whereby selecting $ {\psi}_i $ is rational for games with $ {u}_i\le \frac{4}{5} $. Random and small-world networks can preserve risk-dominated strategies by the same mechanism with a wide variety of configurations, manifesting as a graduated transition between regimes with no and complete collaboration.

4.3. Discussion

Risk dominance clearly influences strategy selection in the conditions evaluated, supporting H1. While the sign of $ R $ guides strategy selection for constant marginal returns, its magnitude describes the relative stability of strategy selection more generally. Games on all studied networks show variability in strategy selection for small-magnitude risk dominance values but reliable strategy selection for larger magnitude values.

The persistence of risk-dominated strategies in certain regions is attributable to local network structure and initial conditions which have an effect akin to biasing the probability that a strategy is selected. This result supports findings of Braha (2020) as well as others such as Gianetto & Heydari (2015, 2016) and Mosleh & Heydari (2017) that network structure and motifs influence collaborative behaviors. Rather than disproving H1, this observation suggests that approaches such as Bayesian games which apply a prior probability distribution over the ‘types’ of other system actors may prove useful for collaborative design scenarios with uncertain bias towards a strategy or uncertainty in the outcome of a strategy pair (Harsanyi 1967). In the context of the technology transition application case specifically and SoS design generally, this means that factors such as business alliances or previous actions by potential partners should be factored into priors over other system actors’ payoffs when using $ R $ as a measure of design robustness.

Lastly, the payoff function form has a large, but consistent and foreseeable effect on collaboration levels as a function of $ R $. When only a small number of participating systems are required to produce large benefits from collaboration, collaboration can dominate when $ R $ indicates unfavorable dynamics. Conversely, for SoS with dynamics requiring relatively large numbers of participating systems to accrue benefit, defection can dominate in regions where $ R $ indicates that it should be rare. However, the relative magnitude of $ R $ (within each case) remains a reliable predictor of collaboration rates, even with nonlinear incentive structures. Furthermore, it is likely that the payoffs in the normal form games used to compute $ R $ can be transformed to include both asymmetric economic importance of potential partners (a problem not studied here) and the value of potential collaborators as a function of the ratio of realized partners versus the number of partners required for full benefit. Such a transformation would enable the retention of $ R=0 $ as the switching point between regimes of mostly collaboration and mostly defection, subject to an accurate characterization of the strategy dynamics.

5. Risk dominance for robust design

This section presents the experimental design, results, and discussion for the test of H2. The emphasis is the study of design options on system actor equilibrium selection in simulations of the technology transition SoS.

5.1. Experimental design

The second experiment uses the same network topologies and update rule as the first experiment but the update rule affects both strategy and design selection simultaneously. Simultaneous update is an intentional simplification of real-world collaborative design, in which strategy selection or modification may occur asynchronously with design modification. The ease of updating either strategy or design and the resulting rate at which updates are possible varies between design efforts. This work starts with the simplest case, simultaneous update. Furthermore, because each generation represents the complete sequence of events leading to system deployments, asynchronous updates do not increase realism.

The experiment introduces one design for Bluetooth strategy and two designs for the Greycloak strategy with a variable payoff for one design and initial probability of collaboration as two independent factors. Whereas the first experiment demonstrates how $ R $ affects strategy selection in a population initially composed of actors equally likely to start as defectors or collaborators by varying $ S $ and $ T $ for all actors, the second experiment investigates how varying options for a single payoff for a subset of the population and varying the probability that each actor is a collaborator in the initial population affects collaboration in the equilibrium state. The goal of the second experiment is to understand how design updates by a subset of a population affect collective dynamics under initial conditions (the probability that actors are collaborators) that vary the favorability of collaboration.

The population contains two types of actors assigned with equal probability. Both types have two strategies $ {\mathcal{S}}_i=\left\{{\phi}_i,{\psi}_i\right\} $. Type 1 actors have design spaces with two alternatives $ {\mathcal{D}}_1 $ = {MkI, MKII –A} associated with the Bluetooth and Greycloak technologies respectively. Type 2 actors have design spaces with three alternatives $ {\mathcal{D}}_2 $ = {MkI, MKII –A, MkII – B}. Type 1 actors are technically unable to implement the Mk II-B design because it uses the Bluetooth and Greycloak transceivers and is, therefore, too large for their products.

Design payoffs are given in Table 6; all are fixed values except for the Mk II-B design in context $ \left\langle {\psi}_i,{\phi}_j\right\rangle $ which takes on variable value $ C $ between 2.0 and 4.9. With this formulation, games always exhibit the stag hunt dynamic.

Table 6. Mobile technology transition design options

Assuming the Mk I design is used with $ {\phi}_i $, pairing the Mk-IIA design with $ {\psi}_i $ yields $ {u}_i=\frac{2}{3} $ while pairing the Mk II-B design with $ {\psi}_i $ yields $ {u}_i $ values between $ \frac{1}{16} $ and $ \frac{2}{3} $. Risk dominance reaches a threshold value $ R=0 $ at $ C=4.25 $ for Type 2/Type 1 pairs and $ C=3.5 $ for Type 2/Type 2 pairs.

The initial probability of collaboration $ {p}_{\psi } $ varies between 0 and 1. Decreasing $ {p}_{\psi } $ increases the severity of the payoff perturbation experienced by an actor playing $ {\psi}_i $. The probability of any actor initially using the Mk II-B design is always 0. All actors starting with the $ {\phi}_i $ strategy select the Mk I design and those starting with the $ {\psi}_i $ strategy select the Mk II-A design. Type 2 actors may select the Mk II-B design in subsequent rounds of play.

The dependent variable is the fraction of Type 1 actors that select strategy $ {\psi}_i $ in the equilibrium state. While the fraction of strategy $ {\psi}_i $ among Type 2 actors is also of interest, the result is expected to follow the patterns observed in Experiment 1. Of greater interest is the effect that robust design implementation has on strategic behavior of neighbors with relatively fragile designs (Type 1 actors). The risk dominance strategy selection concept predicts that actors are more likely to collaborate if their neighbors have lower $ {u}_i $ values such as that provided by robust design solutions in an engineering context.

Each combination of topology, $ C $, and $ {p}_{\psi } $ is simulated 1000 times following the same process as the first experiment. Each simulation is terminated when every actor plays the same strategy and design in four consecutive rounds of the game.

5.2. Results

Figure 6 displays the results of the second experiment simulations as the percentage of collaborators amongst Type 1 actors as a function of the initial probability of collaboration $ {p}_{\psi } $ and the Mk II-B upside payoff $ C $. Results from the preferential attachment networks are again excluded due to similarity with the small-world network results. High initial probability of collaboration directly increases $ {p}_j $ to facilitate collaboration for all actors (including Type 1). High values of $ C $ decreases $ {u}_i $ for Type 2 actors, which makes collaboration more favorable; however, its effect on Type 1 actors is more complex.

Figure 6. Final collaborator ($ {\psi}_i $) density among Type 1 actors for columns (1) small-world networks, (2) random networks with minimum node degree $ =1 $ and (3) random networks with minimum node degree $ =2 $. Row A shows contour lines for threshold values of $ {p}_{\psi } $ and $ C $ below which more than half of the Type 1 actors collaborate (red contour lines display lowest possible $ R $ for an interaction between Type 2 actors). Other rows show heat maps of final Type 1 collaborator density for variable $ {p}_{\psi } $ and $ C $ when marginal returns are: B, constant; C, decreasing and D, increasing.

Type 2 actors that adopt the Mk II-B design are more robust to defection than those that do not (other Type 2 with Mk II-A) or cannot (Type 1). Consequently, they are more likely to collaborate, even in the presence of some defectors. Increased collaboration from Type 2 actors adopting Mk II-B effectively increases $ {p}_j $ from the perspective of their connected neighbors for whom collaboration may be much riskier, stabilizing the collaborative strategy. In this way, the Mk II-B design serves as a robust design by mitigating the effects of coordination failure for the actors that implement it (strategic robustness) and by increasing the probability of collaboration amongst potential partners (strategic stability).

Figure 7 further reinforces the conclusion by showing increasing selection of the Mk II-B design with increasing $ C $ and decreasing $ {p}_{\psi } $. The Mk II-B design option both mitigates downside risk to coordination failure and enables collaboration when there are few initial collaborators. Selection of the Mk II-B design disappears, along with collaboration, when the initial population is composed strictly of defectors and continued defection is the only rational response.

Figure 7. Final Mk II-B design density for columns (1) small-world networks, (2) random networks with minimum node degree $ =1 $ and (3) random networks with minimum node degree $ =2 $. Row A shows contour lines for regions of $ {p}_{\psi } $ and $ C $ within which more than 2% of actors implement the Mk II-B design, equating to $ \approx $4% of Type 2 actors (red contour lines display lowest possible $ R $ for an interaction between Type 2 actors). Other rows show heat maps for the percentage of Mk II-B design users for each combination of $ {p}_{\psi } $ and $ C $ when marginal returns are: B, constant; C, decreasing and D, increasing. Note that the heat map range is from 0 to 50%.

The high variance in the prevalence of the Mk II-B design between the three payoff function forms is noteworthy. Collaboration is generally low risk with decreasing marginal returns. Consequently, the Mk II-B design is relatively uncommon in the equilibrium state because it is inefficient and unnecessary when there are large numbers of collaborators. It only persists for mid-range values of $ C $, and is replaced in the equilibrium state by the more efficient Mk II-A design after playing the role of converting the population to collaborators for high values of $ C $. At the opposite extreme, the Mk II-B design is almost universally adopted in equilibrium by Type 2 actors for increasing marginal returns. Collaboration is persistently risky for increasing marginal returns, but the Mk II-B design enables large numbers of Type 2 actors to collaborate and, as Figure 6 shows, even enables small numbers of Type 1 actors to profitably collaborate and implement the Mk II-A design. For all three payoff function forms, the Mk II-B design is critical for the transition to increased levels of collaboration.

5.3. Discussion

The results of the second experiment demonstrate a mechanism for an individual design actor to achieve utility robustness through strictly technical means. While real-world design often focuses on robustness to exogenous noise, component variation, etc. through mechanisms such as system control, redundancy, degeneracy and modularization, results show that a strategically robust design (which may be achieved by the aforementioned mechanisms) also creates a positive feedback loop for simulated SoS by reducing the probability that any neighboring design actor will select a noncollaborative solution, supporting H2. The robust design feature modifies the system environment to reduce the probability of perturbation occurrence, increasing the stability of the collaborative strategy and utility robustness for all system actors.

6. Broader discussion of results

6.1. Limitations to generalizability

In accordance with strategy selection following risk dominance, the positive effect of strategically robust designs on the collaborative behavior of system actors observed in results depends on the design actors possessing or developing a sufficiently accurate knowledge of the strategy dynamics. The selected model informs actors through repeated interactions. While repeated interactions between multiple system actors are characteristic of SoS design, decisions, particularly strategic decisions governing SoS formation, are often separated by months or years, limiting the opportunities for learning in repeated interactions with a fixed set of partners. Furthermore, early interactions may not produce rewards and penalties consistent with the overarching coordination game dynamic, limiting the learning that can be achieved from those interactions. Instead, it is more likely that learning occurs over a longer, multiprogram or multisystem timescale. Therefore, initial decisions informed by an assessment of SoS strategy dynamics are critically important because strategic exploration may be prohibitively costly.

Whilst strategically robust designs minimize the impacts of coordination failure independent of the strategy dynamic, incomplete information about the dynamic in real-world applications limits the hypothesized stabilizing effects. Lack of understanding of the level of robustness needed for a particular application may lead to overly conservative designs that diminish upside potential gains or overly optimistic (fragile) designs that expose a high risk of losses. Likewise, system actors are not encouraged to collaborate unless they are aware that others pursue strategically robust designs. This issue is exacerbated by the generally asymmetric positions of the constituent system actors. Each will usually have a unique payoff function and will differentially value the collaboration of the other system actors. While the three payoff function forms studied here begin to explore this issue, they do not replicate the actor-to-actor variability present in real systems. Furthermore, additional information, such as risk or loss aversion, reference points, and probability weights are required to transform payoffs into actor-specific utilities. Collecting such information about potential participants in an SoS presents difficulties, as has been the experience of many economists in other domains (Barberis 2013). Further research is required to determine the necessity and practicality of assessing SoS strategy dynamics using actor-specific utility functions.

Two methodological adjustments are required to close this gap and enable practitioners to implement strategically robust designs that also enhance the stability of collaboration. The first is a method for estimating the true strategy dynamics from incomplete information and characterizing the uncertainty of the estimate. Bayesian games (Harsanyi 1967) and team reasoning (Bacharach 1999) are promising methods to characterize payoffs under incomplete information and can be used to formulate games with probability distributions of utility as a function of a design, business alliance or other factors creating uncertainty. Significant research is required to develop and validate design methods, which incorporate such features.

The second is a method to communicate one’s strategic robustness to potential partners with the aim of increasing their likelihood of committing to collaboration. In intra-organization collaborations, such as those common within agencies of a government, signalling one’s strategic robustness may be relatively simple because information can be shared more freely. Signalling strategic robustness in commercial endeavors, particularly those with collaborators that are also competitors, is comparatively difficult due to concerns about maintaining or gaining competitive advantage. Additional research building on relevant economics literature (e.g., Crawford & Sobel 1982; Spence 2002) may help understand how to communicate strategic intent in different environments.

Finally, it must be stated that it is not yet known if human designers are more likely to collaborate when their collaborative partners adopt strategically robust designs, that is, whether human decision-makers use risk dominance as a decision criterion in a collaborative design context from both descriptive and normative perspectives. This is a critical knowledge gap that should be investigated by design experiments with human subjects building to real-world case studies to test the key ideas presented in this work.

6.2. Implications for engineering design

Notwithstanding the limitations in the preceding section, there are a few key implications of this research for design practitioners. The first experiment demonstrates that, subject to local network effects, simulated system actors following myopic decision rules maximize utility by selecting the risk-dominant strategy in a stag hunt game as a model of technology transition SoS with uncertain collaborative behavior. Design practitioners, especially policy makers, can learn from this simple experiment that the mere existence of a Pareto-efficient SoS design is insufficient grounds for each actor to commit to collaboration. Even in cases with the prospect of mutual benefit for all parties, the spectre of a technical failure or discovery of a superior independent strategy by one actor could lead to a coordination failure and corresponding utility loss. Risk dominance represents a significant single factor that differentiates strategic stability of equilibria.

The second experiment builds on the first, demonstrating that designs which are robust to coordination failures increase collaboration by all system actors, thereby reinforcing their expected utility. However, because strategy selection for each actor is still driven by risk dominance and the initial probability of collaboration, even actors with robust collaborative designs sometimes maximize value through defection. Two lessons can be drawn from this result. First, decisions to form an SoS should be supported by analysis of strategy dynamics to determine if collaboration is favorable and, if so, whether options exist to increase its stability. Second, even if it fails to achieve a sufficient stabilising effect, pursuing strategically robust designs reduces losses in the event of a coordination failure. Implementing strategically robust designs maintains the operational independence characteristic of SoS (Maier 1998).

For design researchers, the second experiment clearly demonstrates that a strategically robust design for one or more constituent systems enhances collaboration amongst all constituents in a robustness feedback loop, assuming that each is aware of the robustness of the others’ designs and influenced by risk dominance (limitations previously cited). Intuitively, the robustness feedback loop is built on the idea that system actors with little to lose and much to gain through collaboration are likely to do so. Design actors with a riskier value proposition can collaborate with higher confidence knowing that others are committed. The extent to which this robustness feedback loop is operative amongst human designers is a potentially fruitful area of research.

7. Conclusion

Robust systems must be designed to be resistant to high-risk perturbations. Robust design has traditionally focused on reducing the consequence of variation in design variables, uncontrolled parameters, system models, and design processes (Types I, II, III and IV robustness, respectively). Design for SoS requires consideration of an additional type of robustness, robustness to unfavorable strategic decisions by potential collaborators.

Strategic uncertainty in collaboration can be modelled as a stag hunt game in which all actors increase their utility by collaborating, but are penalized if the other actor(s) defect. A stag hunt game has two pure-strategy equilibria: when all actors collaborate, and when all defect. Equilibrium selection can follow either payoff dominance or risk dominance. The collaborative strategy is always payoff-dominant in stag hunt games, just as collaboration is assumed to be mutually-beneficial in SoS. However, coordination failure can result in significant losses, a consideration that is absent from payoff dominance. Risk dominance, however, indicates which strategy has the greatest basin of attraction and accounts for all possible game outcomes for all actors.

This work demonstrates that risk dominance accurately describes strategy selection in evolutionary technology transition games as a thematic representation of SoS. Significantly, design modifications that reduce losses due to coordination failure (strategic robustness) increase the probability of collaboration for all system actors, including those with designs that are relatively fragile to coordination failures, thereby increasing the stability of the collaborative strategy. That is, system designs that increase robustness by mitigating perturbations directly also increase robustness by modifying the environment to make coordination failure less probable. A change in one actor’s strategy selection as a function of another’s payoff-altering design decisions is a consequence of strategy dynamics and precisely what risk dominance measures.

Future steps for this work should attempt to replicate results with real-world designers and design teams to better understand human design decision-making in collaborative settings. Additionally, supporting design methodologies that implement risk dominance as a measure of a design’s robustness to variations in the strategic environment will help guide robust design efforts. Specific areas of methodological research include probabilistic models of system actor utility functions, implementation of Bayesian games to characterize uncertainty, and domain-specific conceptual design methods incorporating risk dominance to assess strategy dynamics.