NOMENCLATURE
- AI
artificial intelligence
- CONOPS
concept of operations
- DSL
domain-specific language
- DSM
design structure matrix
- IRMA
interactive reconfigurable matrix of alternatives
- LoI
level of interest
- MBSE
model-based systems engineering
- MDDSM
multi-domain DSM
- MDO
multidisciplinary (design) optimisation
- NN
neural networks
- QFD
quality functional deployment
- RDF
resource description framework
- SE
systems engineering
- SoS(E)
system of systems (engineering)
1.0 INTRODUCTION
Product development in aerospace has been affected by very long lead times. Predicting the future is impossible, and forecasts become more uncertain the further into the future they need to stretch, thus leading to high levels of uncertainty regarding for instance available technologies or market assumptions. Especially within military development, a typical envisioned usage (concept of operation (CONOPS)) of a complex system will certainly change during its life cycle, due to changing, emerging or other unforeseen external factors that significantly influence the validity of the system. Traditional product development approaches based on an optimisation with respect to a fixed set of requirements fail to provide resilience in a constantly changing world. The problem becomes even worse when considering the long product life cycle that aerospace systems are designed for. Furthermore, since today's aerospace products are often part of a larger integrated system, a system of systems (SoS), it is important for the system manufacturer to be able to understand the relationships that lead from SoS needs, to required SoS capabilities, to requirements placed on single constituent systems. Customers may have performed detailed SoS analyses to produce a specification document for a constituent system to be developed. However, the product manufacturer needs to fully understand the customers’ specifications and the underlying reasons. To engage in requirement discussions and negotiations, suggesting trading certain requirements while demonstrating that the overall needs will still be met, the manufacturer must to be able to carry out similar analyses to those carried out by the customer. Additionally, the manufacturer may wish to trade some requirements to achieve a better alignment of the future product to its own business strategy, to the overall product portfolio, to technology development plans and to the currently available and future-planned in-house competence. Also, the same product may be developed for different customers at the same time, imposing a more holistic view, since particular needs may diverge and simply producing a union of different requirement may lead to suboptimal solutions.
1.1 SoS engineering
According to the definition offered by Maier(Reference Maier1), in this paper a SoS is assumed to possess five characteristic properties that sets it apart from conventional complex systems:
Operational independence of the components
Managerial independence of the components
Geographic distribution of the components
Emergent behaviour of the system
Evolutionary development of the system
The importance of the last point listed above with respect to product development is also highlighted in the INCOSE SE handbook(Reference Walden, Roedler, Forsberg, Hamelin and Shortell2), which lists 31 product development processes for product life cycle engineering, which may be required concurrently in a huge SoS with its underlying systems in different life cycle stages and parallel system upgrades. Unlike classical (model-based) product development, huge efforts have to be invested to address the SoSs’ evolutionary and emergent behaviour, which can occur at various levels(Reference Holland3). Extensive forecasting and foresighting methods may be applied to analyse the system of interest (SoI) ramifications due to changes in the surrounding environment, external factors and other conditions (see Section 3.4). The used technology assessments should have the capability to identify disruptive technologies that may lead to disruptive events and emergent behaviours. One method for addressing the latter is disruptive technology assessment games (DTAG)(Reference Andersson, Norsell, Svantesson and Andersson4), but more conventional approaches like matrix-based assessment methods (see Section 3.2) may also be applied for technology assessment(Reference Amadori, Bäckström and Christopher5).
In order to distinguish between different SoS-specific characteristics, Gideon et al.(Reference Gideon, Dagli and Miller6) proposed a taxonomy classifying every SoS by three type subsets only. Applying Gideon's taxonomy to large, complex aerospace SoSs, the following classification may apply: The SoSs are of physical domain type, most probably of a dedicated acquisition type and could be of any of the three operational types, directed, collaborative or chaotic. This work does not address a particular SoS within this classification, but rather tries to identify and define the different phases needed to approach the development of SoSs regardless of the type or the operational domain.
1.2 SoS research
Work performed by ASDL at Georgia Tech(Reference Engler, Biltgen and Mavris7–Reference Roberts, Griendling, Gray and Mavris9) has proposed methodologies to tackle SoS in the context of defined scenarios and requirements. SEAri at MIT(Reference Rhodes, Ross and Nightingale10,Reference Rhodes and Ross11) has chosen a different approach to the problem, focusing instead on epoch influences on development of complex systems and SoS (see Section 3.6). When addressing SoSs, expanding to a larger scope also implies that traditional engineering domains may not be sufficient. Stakeholders’ needs may be dependent on socio-economical changes, and therefore a broader set of domains must be understood and integrated.
From literature reviews such as that presented by Axelsson(Reference Axelsson12), it can be noticed that SoS engineering (SoSE) is not yet fully defined as a scientific discipline, and therefore no clear and holistic handbook, guidelines or best practices addressing the whole design process exist. For this reason, this paper tries to offer a complete mapping of all perspectives within an overall SoS design process (as depicted in Fig. 1), including potential methods and tools that may support each phase. The goal is to outline a set of heuristics for SoS engineering and resilient design, but without proposing or developing deeper analytical methods.
1.3 The SoS engineering paradigm shift
While conventional product development is primarily a technical-focused process within established domains, modern approaches like DARPA FANG(Reference Eremenko13) propose instead to tackle product development based on cyber-physical simulations and model integration by means of some kind of a multidisciplinary design optimisation (MDO) framework (e.g. AGILE(14)). These approaches still belong to the mechanical engineering domain, where huge progress has been made with respect not only to model implementation and modelling languages (like Modelica, Catia, Python, etc.), but also to available computational power and industry standards for model exchange and co-simulation such as the functional mock-up interface (FMI)(15). The primary concern of such solving frameworks is the early integration of physics-based models or methods of higher fidelity levels into the design process for design space exploration and optimisation. Generally, the foundation of such frameworks relies on a parametric geometry model that serves as the central node to which domain-specific models are connected as functional extensions(Reference Staack16).
In order to add higher fidelity and include non-mechanical engineering domains, the field of study has to be extended to an interdisciplinary systems engineering (SE) approach. This paradigm shift adds several new domains and concepts to the design process, the most important of which are addressed in Section 3. These extensions not only expand the design process upstream and downstream, but also introduce new domains and features to the design task such as business aspects, requirements and stakeholder needs handling, together with technology selection including technology maturation planning.
2.0 HOLISTIC PRODUCT DEVELOPMENT IN THE SOS CONTEXT
A holistic approach to product development in the context of SoS is proposed and illustrated in Fig. 1. The goal of this phase-based process decomposition is to identify the main areas of interest in order to tie needs, capabilities, and system requirements in the initial phases of product development. Five main levels of interests (LoI) have been identified, as follows:
Needs and boundary conditions
SoS capabilities
SoS design space
Constituent systems design space
Sub-systems design space
The breakdown is recurrent and the main links between them are described in Fig. 1. The following section gives a brief overview of each phase and the connection to the adjacent levels:
Level of Interest 1 – Needs and Boundary Conditions
Within the product needs analysis, the needs related to the end-user needs are being analysed. The end-user needs and boundary conditions arising from a predicted environment are considered and analysed to determine the needs affecting the product. It is important to stress that such needs and boundary conditions are not intended to be limited to stakeholder requirements. Typical high-level frames of interest at this level are:
Geopolitics, doctrine, laws and regulations
Business cases
Customer needs
Threats and technologies
Time frame (history, now, future), needs and boundary conditions
Figure 1. Overview of the proposed system of systems holistic design process.
These analyses can be related to a fixed period or to different time frames, meaning that all of those inputs will be characterised by different levels of uncertainties and may vary within the different time frames. From a holistic perspective, those initial conditions and boundaries have to be varied in order to understand the main required capabilities in response to changing needs. The output is a set of different scenario-representing needs. These scenarios should be agnostic to any solution to understand the main capabilities required by the SoS.
Level of Interest 2 – SoS Capabilities
The SoS capabilities are defined by scenario analyses. The underlying task is to figure out the impact of changes in the boundary conditions and the needs on the overall SoS capabilities. This analysis process leads to a balanced definition of the overall requirements on the SoS. Here, the capabilities design space is explored with the aim of understand it and providing decision support for strategical choices. The output will provide the main capabilities to be considered in the subsequent SoS design space exploration phase.
Level of Interest 3 – SoS Design Space
With help of the architecture design space exploration, the SoS capabilities are transformed into a SoS design space containing all valid solutions that achieve the desired capabilities. Out of this pool, possible SoS concepts – including type and number of the constituent systems, collaboration and tactical models – are generated, responding to the different identified capabilities. Each SoS concept is represented by one entry in the SoS design space. This design space is then down-selected by benchmark processes to a short list of designs, each made of a set of constituent systems. As an output, each constituent system will have a set of individual requirements.
Level of Interest 4 – Constituent Systems Design Space
Based on the individual constituent systems requirements, the design space for each constituent system is generated. Conceptual design of each constituent system is then performed based on the requirements provided by the SoS design space analyses. This phase will validate the feasibility of each envisioned constituent system of the short-listed design space. It corresponds to the traditional product development process where one product or system is designed from a set of requirements.
Level of Interest 5 – Sub-Systems Design Space
Sub-systems are the systems that constituent systems are made of. The sub-system design task includes alternative architectures, system dimensioning and characterisation, and compatibility and integration into complete constituent systems. Typically, the sub-system analyses will consist of models for each discipline within a constituent system. The process can be interpreted as a whole (classic) conceptual design phase for each system, preferably implemented in a highly integrated model-based system engineering (MBSE) approach, enabling the analysis of a large number of different architectures and configurations.
2.1 Comparison to established product development processes
The proposed holistic SoS approach extends the usual design and product development processes to also include all boundary conditions (from geopolitics to regulations, from environment to time relations), as described in the LoI 1-3 in Fig. 1. Unlike established design-to-cost (DTC)(Reference Michaels and Wood17) driven or design-to-value (DTV) driven approaches, the emphasis is more on a thorough system capabilities analysis (in LoI 2) that defines the SoS design space, and not the product setup. Taking into account the whole life cycle analysis of the system as defined by ISO 15288(18), the proposed approach goes far beyond classic multidisciplinary design optimisation (MDO) or CAD/CAE/CAM tools or frameworks usually applied for product development within industry. These tools are tailored for analysing and exploring constituent systems’ and subsystems’ design spaces (see LoI 4 and 5 in Fig. 1).
Another issue the reader should be aware of is that the lack of a universally accepted definition of SoS leads to confusion about whether a product can be seen as a complex system (that might contain several subsystems of different domains, e.g. a military aircraft) or a SoS. Applying the rather strict and distinct SoS definition proposed by Maier(Reference Maier1) (see Section 1.1), many as SoS denoted systems do not match this classification. Industrial examples for such pseudo-SoS systems include interface and (software) architecture concepts like the AUTOSAR architecture(19) for automotive applications.
3.0 KEY ENABLER FOR HOLISTIC DESIGN ENGINEERING
Each process phase described in Section 2 involves its own challenges if it is to be realised; some of them are more mature in methods and tools than others. A higher level of abstraction will be necessary to combine the different phases into one framework. This section presents a selection of different methods, research results and fundamental techniques that are identified by the authors as key enablers or available solutions in order to realize the envisioned process.
3.1 Meta-modelling and common language
In order to connect the different domains of the design phases illustrated in Fig. 1, a common language and semantics is required. Using ontology to describe a complex system or a complete SoS may be the way forward(Reference Benali, Ben Saoud and Ben Ahmed20). A complete ontology (description) of the system of interest might theoretically represent a complete information input for a SoS design process beside the requirements. While examples of ontology for aeronautical applications can be found in publications(Reference Ast, Glas and Roehm21–Reference Glas23), the usefulness of this approach for complex system/SoS engineering has yet to be proven. In a similar way, the DARPA FANG(Reference Eremenko13) and DARPA AVM(Reference de Weck24) projects focused on decreasing the product development time through component-based design and efficient cross-domain modelling. There was a great emphasis on the development of a model integration language, CyPhyML(Reference Sztipanovits, Bapty, Neema, Koutsoukos and Jackson25), and an semantic backplane OpenMeta(Reference Simko, Levendovszky, Neema, Jackson, Bapty, Porter and Sztipanovits26,Reference Simko, Lindecker, Levendovszky, Neema and Sztipanovits27) . The selected tool for formal meta-modelling was FORMULA from Microsoft Research, a framework for formally specifying domain-specific languages (DSLs) and model transformations(Reference Jackson, Levendovszky and Balasubramanian28).
From a mathematical point of view category and sheaf theory(Reference Kashiwara and Schapira29) could be the foundation for an axiomatic description of the problem or the design space. This mathematical foundation seems promising and despite the fact that more applied research is needed to prove its usefulness, it has recently been acknowledge by DARPA as a cornerstone of the DARPA FUN design(30). Another approach to represent large and complex SoSs has been applied by military organisations through the usage of system modelling language (SysML); creating an enterprise architecture approach to capture the information about the business to identify the processes and resources required to deliver the vision expressed in the strategy. Different variants of these architecture frameworks, depending on their origin, are available(Reference Klein and van Vliet31). Prominent examples are DoDAF(32)/MODAF(33)/NAF(34) and IDEAS for military applications. The different architectures contain different viewpoints(32):
Strategic viewpoint
Operational viewpoint
Service orientated viewpoint
Systems viewpoint
Acquisition viewpoint
Technical viewpoint
All viewpoints
These standards have the advantage of being based on a universal system modelling language, but have not yet been proven to be usable within product development as a main backbone for the execution of model-driven design processes (unlike in the software engineering domain with e.g. executable UML (xtUML) models). Combinations of such framework- and service-oriented architectures may enable the execution of SoS within its different viewpoints. Such a framework will serve as the link between viewpoints and models. The creation of domain-specific models, however, will still need to be performed in other frameworks/languages.
3.2 Matrix-based approaches
Matrix-based information arrangement is a common and natural choice of representing (any type of) relationship between different entities. Introduced in 1985 by Steward(Reference Steward35) for product (development) modelling, it is usually denoted as design structure matrix (DSM). In a certain arrangement, linking the customer needs to the system characteristics, it is called the quality functional deployment (QFD), also known as the house of quality. One implementation strategy of a user-in-the-loop matrix-based product development process is the interactive reconfigurable matrix of alternatives (IRMA) process (see e.g. Engler et al.(Reference Engler, Biltgen and Mavris7)). While the mathematics/logic relations in these (usually 2-dim) matrices are simple, the applied processes on the matrices – namely sorting and clustering – are not; each of these processes represents a local optimisation problem, fighting with the inherent problem of the sheer unfathomable number of combinations in already small-/medium-sized matrices(Reference Staack16). The combinatoric growth is at least exponential with the power of two over the matrix size (number of matrix fields), but becomes significantly higher for clustering operations (e.g. with the target of sorting and detecting bus/integrator instances within a matrix(Reference Helmer, Yassine and Meier36)). Inherent problems of 2-dim matrices in the n -dim design space include the fragmentation of clusters and acausal relationshipsFootnote 1. Due to the break-up into a forward and a backward part of interconnected entities/modules, the matrix-based representation becomes difficult to read; this effects not only large and complex systems but also low complexities as low as triple or tetrahedron cluster formations(Reference Yu, Goldberg, Sastry, Lima and Pelikan37).
Some single-domain DSM drawbacks can be mitigated by adding more domains to the DSM, extending the usual square 2-dim (NxN) matrix into a composite 3-dim (NxNxD) matrix with D different domain matrices. However, due to the absence of a natural diagrammatic (2-dim) representation of a multi-domain 3-dim structure, graphical solutions to represent multidomain DSMs (MDDSMs) have to be found. A possible decomposition of a 3-dim space into a 2-dim space can be achieved by cascading the data and presenting the higher dimension within the cells of the first and second dimensions. Abstraction can be achieved by the application of rating schemes e.g. those devised by Pimmler and Eppinger(Reference Pimmler and Eppinger38), and extended later by Helmer et al.(Reference Helmer, Yassine and Meier36).
A significant difference between intra-system and intra-SoS relations is that most systems’ relationships within a SoS are communication channels for information exchange while physical system relationships often deal with the exchange of matter such as materials, fluids, energy, forces and heat. Consequentially, suitable modelling approaches (and tools) differ for both applications such as UML and SysML for the former and Modelica or Simulink for the latter.
3.3 Relational/graph-based
With the named disadvantages of matrix representations at hand, one solution to describe the system of interest is a graph network. With the help of 3−dim rendering, colour schemes, arrows and entities/cluster size, several domains can be represented in a human-understandable manner on a 2−dim screen provided that the network entities have been arranged (and if necessary clustered or sorted) with the help of suitable layout (positioning) algorithms such as Fruchterman-Reingold(Reference Fruchterman and Reingold39) or Hu(Reference Hu40). Schaeffer(Reference Schaeffer41) lists the different mathematical approaches for graph clustering that can be applied for product modelling.
The advent of huge social networks and the associated data mining and analysis needs triggered the development of various tools, relational database systems and data formats for graph structures (such as Gephi(Reference Bastian, Heymann and Jacomy42)). Defining a relational network and editing can be carried out without any knowledge of the residue data unlike in a hierarchical databases approach such as the classic product tree structure. Every relation in the relational database/network is a resource-trait/aspect-resource triplet that establishes the relationship between two entities. These relational entities are data triples similar to the resource description framework (RDF) triples used to model ontology within the semantic web approach, originally invented by Berners-Lee et al.(Reference Kuosa44) (see also Section 3.1).
3.4 Forecasting and foresighting methods
To define aerospace needs in future scenarios, forecasting or foresighting (see Section 3.4) must be performed. The goal of forecasting is to provide a prediction of highly probable future events, often based on an extrapolation of known facts. In contrast, foresighting does not aim to predicting the future but rather:
“…to explore the range of plausible futures that may emerge and to help identify assumptions and strategies that are robust in preparing for an uncertain future.”(Reference Kuosa44)
Several different forecasting and foresighting methods exist and have been summarised by Kindvall et al.(Reference Kindvall, Lindberg, Trane and Westman45), Cho and Daim(Reference Cho and Daim46). The selective data collection process (typically executed by subject matter experts) will lead to recommendations for technologies and scenarios that have been identified as the most influential ones, see example from Silfverskiöld et al.(Reference Silfverskiöld, Liwång, Hult, Sivertun, Bull, Sigholm, Lundmark, von Gerber, Andersson and Sturesson47). One inherent drawback of these methods is the subjective judgement that may affect the results. One key to using the findings from such methods would be to transform these scenarios and technology recommendations into models that can be part of the framework described in Fig. 1. The application of foresighting within a framework for SoS engineering has been presented by Ross and Rhodes(Reference Ross and Rhodes48) and will be addressed further in Section 3.6.
3.5 Value-driven and robust design
Value-driven design aims to shift the focus from the requirements only to understand and analyse the value for the customer brought into the SoS by different parts of the design(Reference Collopy and Hollingsworth49). Underlying resectioning is required to tie customer needs to the added value created by the different solutions. Methods proposed by Isaksson and Bertoni(Reference Bertoni, Bertoni and Isaksson50–Reference Bertoni, Bertoni, Isaksson, Amnell and Johansson52) within aerospace applications show promising results and could be a valuable asset within the envisioned holistic product development process.
3.6 Epoch analysis
Traditional systems engineering tends to focus on meeting technical requirements. However, in a dynamic world, assumptions will probably change over time, affecting both technical and non-technical factors(Reference Collopy and Hollingsworth49). One method to address these changes over time is the epoch analysis proposed by Rhodes and Ross(Reference Rhodes, Ross and Nightingale10,Reference Ross and Rhodes48) . Beesemyer et al.(Reference Beesemyer, Ross and Rhodes53) defines an epoch as:
“…a period of time, defined by a fixed set of context and needs, which impacts the ultimate success of a system. A long-lived system may face a large number of epochs over its lifetime.”
The work performed by the Rhodes SEAri group at MIT has shown the practicality of epoch analyses on various applications ranging from aerospace(Reference Beesemyer, Ross and Rhodes53–Reference Fitzgerald and Ross57) to maritime(Reference Gaspar and Erikstad58). Application has mainly been on large complex systems, with some extensions to SoS(Reference Rhodes and Ross11,Reference Parker, Ross and Rhodes59) . The authors of this paper feel confident that epoch analysis methods can be a key enabler for setting up the first LoI in the proposed holistic development approach.
3.7 Data-driven design and tradespace exploration
Tradespace exploration is not only a way of assessing more design solutions than current methods allow for. It is also envisioned to be an interactive visual environment, enabling live what-if questioning to cover more criteria than are commonly applied in early conceptual design phases. The goal is to provide resilient system solutions in a changing context and a long-term perspective inherent to future aerospace SoSs. To perform such tradespace analyses, a data-driven approach is mandatory to enable an unremitting evaluation and analysis of alternatives. Data-driven methods rely on large computations with sensitivity analyses performed on all relevant variables. In contrast to current approaches, where requirements are considered as the primary input for product development, the aim of tradespace exploration is to generate the system requirements(Reference Rhodes and Ross60). Tradespace exploration techniques and diverse applications have been presented to a large extent by the MIT SEAri group(Reference Ross, McManus, Rhodes and Hastings61–Reference Ross66). The U.S. Department of Defense (DoD) funded recently the Engineered Resilient Systems (ERS) project(Reference Holland67) to explore more efficient methods for military acquisition. As a result of the ongoing effort, the DoD also wants to leverage data-driven design.
3.8 Visual analytics and big data
The authors recognise the need to incorporate big data handling coupled with efficient interactive visualisation as a key capability. The different design spaces within each phase of the proposed process will lead to a very large set of data that needs to be managed and understood to support well-informed decision making. Georgia Tech has for a long time advocated using visual analytics as an assistive technology for decision support(Reference Mavris, Balchanos, Sung and Pinon68,Reference Mavris, Pinon and Fullmer69) to make large SoS design space explorations and uncertainty quantifications possible. Also within military applications, visual analytics and big data are being identified as key enablers for efficient acquisition of military products in the future (see the previously mentioned ERS project of the US DoD). The Swedish Defence Research Agency (FOI) recently published a comprehensive summary of the current research state of visual analytics methods(Reference Jandel, Bivall, Hammar, Johansson, Kamrani and Quas70).
3.9 Other domains
Most of the identified SoS enablers in this section originate from engineering domains. However, to realise the envisioned holistic development approach, additional domains have to be investigated and understood to benchmark their impact and capabilities concerning the design space exploration. Some key thoughts are presented here. They should not be seen as a definitive list but rather as the current status of the authors’ knowledge.
Economic decision-making studies performed by the economist Thaler and co-workers(Reference Mullainathan and Thaler71–Reference Barberis and Thaler73) incorporates psychologically realistic assumptions, limited rationality, social preferences and lack of self-control of the stakeholders. These studies show that external factors have a large (non-rational) influence on decision making. Consequently, similar methods and assumption must be incorporate into the product development process, where customer preferences may certainly be influenced by similar factors.
The availability and recent progress of artificial intelligence (AI) can be an opportunity for decision support and large data analyses within the context of trade studies. It may also support better domain-specific understanding as well as helping to identify advantageous and disadvantageous emergent cross-domain coupling effects. Further understanding of current research (from the authors’ point of view, all with an engineering background) is needed to incorporate the non-engineering disciplines such as geopolitical modelling and assessment into future implementations. Application examples that may largely benefit from ongoing machine learning and natural language research are meta-modelling and socio-economical domain representations.
4.0 CURRENT LIMITATIONS AND THE WAY FORWARD
This work describes a holistic approach to system of systems engineering (SoSE) in the context of aerospace. To solve the specific SoS needs and challenges during (SoS) product development, the authors of this work suggest a five-level process (explained in Section 2 and Fig. 1), largely expanding the traditional product development processes. Our research identifies various methods and techniques applicable to these five phases. An overview of the domains and application areas of these methods, described in detail in Section 3, is summarised in Table 1.
Table 1 illustrates common areas of application (such as aerospace, maritime products and doctrine descriptions) and system levels of application, and shows in which phase of product development the techniques are used. The table indicates the environment in which the methods are applied: academia, research institutes, industry and governmental organisations. Serving as a qualitative maturity indicator, the product development phase columns in the table are only filled in if the methods have been found to be used as state-of-the-art within the industryFootnote 2.
Paying heed to all named domains within one holistic SoS approach currently appears to be infeasible due to the overwhelming complexity and the different modelling approaches within each field. Another reason for this diversity, besides the broad variety of work scopes, is the lack of an established holistic SoS research and education field. Consequently, existing solutions are biased by the research groups’ backgrounds such as mechanical engineering, computer science, social psychology, mathematics and so forth. With today’s knowledge, it is not clear whether a distributed (master-master) framework of different domain experts will be the solution or a single master domain has to be found to take charge of the whole orchestra. Can a symphony lead by different domain conductors produce the desired outcome?
A central point in the implementation strategy has to be the decision between a machines first or a humans first approach(74). How much of the design process can and has to be understood by the person involved? How can the output be actively influenced? How can the operator be integrated into the tool machinery? Can it be ensured that the tradespace reduction process (prior to the actual system development process) is correct and does not exclude any relevant areas of interest? And finally, which methods and processes are able to identify and foresee emergent behaviours of the SoS? A purely AI-like behaviour might be not acceptable due to sensitivity and traceability requirements for trade-off analyses. Relying on training-based AI methods – also denoted as big data mining – such as neural networks (NN) may not be the ideal solution for SoSs due to the lack of relevant training data, although it appears appealing to make use of an NN algorithm analysing the ontology description of the system of interest. The general absence of empirical data in SoS publications is also criticised by Axelsson(Reference Axelsson12). It is possible that applying inspirational ideas from the software engineering domain will lead to widely accepted systematic methods and best practices within the SoS research community.
Table 1 Overview of the domain-specific methods.
Are there any further research fields SoS can be inspired by? Compared to SoS research classic fields of science are generally more mature and consequently their standards and best practices are widely accepted at an international level. Examples where complex collaborative work has largely contributed to scientific success include experimental physics, medicine, economic sociology (irrational behaviour) and climate change research. While the proposed process has not yet been realised, similar approaches are under development driven by the DoD (see DoD digital engineering(75)) with an estimated time frame of full scale deployment within the coming decade.
