Addressing the envisioned world problem: a case study in human spaceflight operations

Abstract

The construction of future technological systems in work domains that do not yet exist, known as the envisioned world problem, is an increasingly important topic for designers, particularly given the rapid rate of technological advancement in the modern era. This paper first discusses the theoretical underpinnings of using cognitive work analysis (CWA) for developing a decision support system (DSS) situated within the envisioned world problem and recasts the problem as pathway-dependent processes. Using this pathway-dependent framework, each stage of the envisioning process is described to reveal how human factors experts can link existing work domains to envisioned instances. Finally, a case study example of the envisioning process that incorporates CWA modelling is demonstrated as it pertains to the advancement of the human spaceflight domain. As a result, this paper provides a unified treatment of the envisioned world problem with an end-to-end example of one approach to designing future technologies for future work domains.

1 Introduction

Cognitive work analysis (CWA) (Rasmussen, Pejtersen & Goodstein 1994; Vicente 1999) is an analytical framework used to ‘characterize the constraints that define the cognitive requirements and challenges, and the knowledge, skills, and strategies that underlie both expert performance and the error-vulnerable performance of domain practitioners (Bisantz & Roth 2008, p. 31)’. CWA has been applied to a number of complex sociotechnical systems to support interface design (Hall, Shattuck & Bennett 2012; Mazaeva & Bisantz 2014), team development (Naikar et al. 2003) and system evaluation (Naikar & Sanderson 2001). However, CWA as a framework is oftentimes insufficient to yielding design solutions, requiring additional approaches or methodologies to support the design process (Read, Salmon & Lenne 2015).

In this paper, we describe an approach using CWA for developing a decision support system (DSS) situated within a domain that does not yet exist, a situation known as the envisioned world problem (Dekker & Woods 1999; Woods & Dekker 2000). While CWA has been applied to the development of interfaces, a process for incorporating CWA to inform DSS development within the broader context of the envisioned world problem has not been addressed.

In developing this approach, our applied research sought to develop a DSS for a future version of an already existing human spaceflight work domain at the NASA Johnson Space Center. While this effort might appear at first to be constructing first-of-a-kind systems (Roth & Mumaw 1995), we contend there is a subtle but important distinction. The degree to which a work domain rooted in decades of institutional experience can incorporate vastly improved functionality that modern technologies may provide is limited. Therefore, a more concerted effort must be made to establish intrinsic work domain constraints in future settings – as they may differ from those of current settings. As a result, prototyping both the DSS designs as well as the future work domain itself becomes a critical component to overcoming the envisioned world problem.

The intent of this paper is to fill a gap in the literature between CWA and simultaneously contending with DSS and future work domain prototype design and testing. We describe how CWA modelling can fulfil a crucial role within the envisioning process by yielding DSS requirements derived from the existing domain which can then support the ideation process of future DSS and work domain prototype designs. This particular aspect of the envisioned world problem has a very limited set of literature and almost no theoretical basis. Ironically, much of what is covered in this paper may seem familiar as this early conceptualization phase, so overlooked by academics, is commonplace in industry and must be accomplished for almost every design activity. Yet, while attempting to associate appropriate theory to the process, we hope to provide additional insight and guidance about a potential new twist on how to approach this age old problem.

In summary, this paper first examines and recasts the existing body of literature on the envisioned world problem to illustrate more clearly the envisioning process. Using this framework, this paper then briefly examines the challenges and opportunities that exist in addressing the envisioned world problem. Finally, a demonstrated application of this framework that leverages CWA is provided to support the development of a DSS that meets the needs of future human spaceflight operations. The case study itself provides a much needed example missing from the current literature on how to combine and associate micro-world studies and laboratory based simulations with natural settings. Specifically, this paper attempts to unite the affordances of CWA requirements with various stages of work domain instantiations to provide one complete representation of how to approach the collective envisioning process.

2 Framing the envisioned world problem

How can results of studies and analyses that characterize cognitive and cooperative activities in the current field of practice inform or apply to the design process, since the introduction of new technology will transform the nature of practice, what it means to be an expert, and the paths to failure (Woods & Dekker 2000, p. 5)?

The envisioned world problem as originally described by Woods & Dekker (2000), Dekker, Woods & Mooij (2002) introduces the following challenges associated with designing technology for a future incarnation of a current work domain:

Plurality:

There are multiple versions of how the proposed changes will effect the character of the field of practice in the future.

Ungrounded:

Envisioned concepts can easily be disconnected or even contradict, from the research base, the actual consequences of the changes on people, technology and work.

Underspecification:

Each envision concept is vague on many aspects of what it would mean to function in that field of practice in the future; in other words, each is a simplification, or partial representation of what it will mean to practice when that envisioned world becomes concrete.

Overconfidence:

Advocates are miscalibrated and overconfident that, if the systems envisioned can be realized, the predicted consequences and only the predicted consequence will occur.

While these four challenges are well articulated overarching considerations, there are no obvious approaches to address them. Thinking about the challenges abstractly, one can formulate two representative pathways to enable more targeted applications of mechanisms of control for the envisioning process: a technology-driven pathway and a work-driven pathway.

Figure 1. Decomposition of the envisioned world problem along the dimensions of technological capability and work domain states.

Figure 1 illustrates an extension of Woods & Dekker (2000)’s seminal work on the envisioned world problem by representing the envisioned world problem as a vector $\mathbf{R}$ that connects the existing work domain state (A) with an envisioned future state (B), enhanced by new technological capabilities. Vector $\mathbf{R}$ is decomposed along two dimensions as originally described by Woods & Dekker (2000): on the x-axis, the work domain, and on the y-axis, technological capabilities. Technological capabilities are defined here as technologies intended to be designed and employed within the work domain that either add to, replace, or modify the existing technologies (e.g. replacing paper medical records with electronic medical records (EMRs) or developing an astronaut electronic cuff checklist to replace existing paper-based checklists).

From a systems engineering perspective, the technological capabilities dimension should be considered separately from the surrounding technologies already existing (e.g. heritage) within the domain. The work domain dimension in this context refers to agents, organizational and cultural structure, and the ‘as-practised’ demands of the work that already exists. The leap between states (A) and (B) along vector $\mathbf{R}$ is a complicated and highly coupled pathway. Predicting the impact of new technological capabilities within a work domain is difficult for multiple reasons: lack of definition of the envisioned work environment; unfamiliar work context, demands, and expertise, unintended consequences those technologies may impose on the work (Sarter, Woods & Billings 1997; Woods & Dekker 2000; Vicente, Roth & Mumaw 2001). However, instead of trying to immediately jump from state (A) to (B), opportunities exist along the various constituent pathways ( $\mathbf{P}_{\mathbf{1}}$ and $\mathbf{P}_{\mathbf{2}}$ ) that yield more incremental insights and perspectives that may be leveraged to advance the envisioning process.

The extension from State A to B exhibits pathway dependency. Throughout the remainder of this paper, we contend that the two pathways shown in Figure 1 are not equivalent and that pathway $\mathbf{P}_{\mathbf{2}}$ is more conducive to yield tenable work domain enhancement and desirable technological designs. This paper consolidates the breadth of considerations and assumptions that should be included to tackle any envisioned world problem. It then proceeds to ground the theoretical discussion in a case study example of how to approach the envisioned world problem with the expectation that the depth of discussion and characteristics presented here will help shape other envisioned world problem research efforts. This paper in effect attempts to link both the theoretical and practical opportunities that exist to overcome the envisioned world problem.

2.1 Pathway 1: the ‘technology-driven’ pathway

Pathway 1 ( $\mathbf{P}_{\mathbf{1}}$ ) as shown in Figure 1 is an idealized depiction of how new technologies are first built and then deployed in the domain. However, the resultant consequence of these properties is that vector $\mathbf{R}_{\mathbf{1}}$ in Figure 2 is inherently unattainable. Any degree of technological enhancement will inevitably shift the characteristics of the work domain (e.g. change in work goals or introduce undesired work demands) to some degree as represented by vector $\mathbf{R}_{\mathbf{1}}^{\prime }$ . Vector $\mathbf{R}_{\mathbf{1}}^{\prime }$ can shift only to the right to signify that the existing domain becomes novel to some extent but does not necessarily imply that the shift is entirely intentional or desired. This shift highlights the inherent coupling that exists between new technologies and the inability of the work domain to fully utilize/accommodate them.

Figure 2. Pathway ( $\mathbf{P}_{\mathbf{1}}$ ): the ‘technology-driven’ approach to addressing the envisioned world problem. The dashed vectors represent the idealized design intentions. Solid vectors represent the realistic progress made via the technology-driven approach. Dashed arrows indicate idealized vectors and solid arrows indicate actual vectors.

New technologies may provide innovative capabilities that appear to align with work domain goals, but in practice may fail to provide meaningful support to domain operators. Some examples of this phenomenon include the under utilization and resistance of electronic health records by health professionals (Declerck & Aimé 2014), the rejection of electronic flight strips in air traffic control (Mackay 1999), and the limited adoption of electronic checklists by astronauts (Simonds & Chen 1991). As depicted in Figure 2, once a new technology is fielded in the work domain, a resulting shift in domain structure of intended aim and consequence occurs as represented by vector $\mathbf{R}_{\mathbf{2}}^{\prime }$ to compensate for the presence of the new technology.

Challenges exist in aligning the resultant state $\mathbf{B}^{\prime }$ with the desired end state $\mathbf{B}$ across the horizontal axis. Often times the extent to which the work domain can adapt with the new technology is a function of a number of compounding factors such as: the way the technology was implemented, the training of users, operator willingness to adopt the technology, and the ‘work-arounds’ developed to overcome technological deficiencies. A more alarming characteristic is that the desired end state was only partially defined or not defined at all at the onset of the technological development. Britcher (1999) provides an example of the consequences of when the end state is not clearly defined and a $5 billion, 14 year automated air traffic control system development effort fails to deploy. We argue this result is due to the lack of adequate envisioned work domain definition and a narrow focus on only technology-driven solutions.

As a consequence of trying to resolve actual and ideal end states, the work domain settles for what the technological capabilities afford. Human operators develop ‘work-arounds’ as coping mechanisms to overcome technological deficiencies or use the technology in unintended ways (Vicente et al. 2001; Flanagan et al. 2013). The difference between the desired and actual modifications in work domain states (e.g. the difference between $\mathbf{R}_{\mathbf{2}}-\mathbf{R}_{\mathbf{2}}^{\prime }$ ) represents the compensation that human operators must dedicate to utilize (or not) the new technological capabilities. As a result, the difference between vector $\mathbf{R}$ and $\mathbf{R}^{\prime }$ is a visual representation of the moving target problem that plagues the envisioning process (Woods & Dekker 2000). The magnitude of unexplained variance introduced under this notional schematic highlights the resultant iterative nature of a ‘technology first’ development perspective.

In summary, pathway $\mathbf{P}_{\mathbf{1}}$ involves a familiar process of first developing new technological capability based on the premise of solving today’s problems by applying technology while ignoring broader system perspectives. The work domain incorporates the new technology and is enhanced to some extent. The unintended consequences are realized and operators develop work-arounds to adopt the technology or the technology is outright rejected. Subsequent iterative development efforts are invoked and the cycle continues ad infinitum to reach the desired end state or until budgets restrict development efforts. The application of EMRs in the health care work domain highlights some of the challenges that can accompany pathway $\mathbf{P}_{\mathbf{1}}$ such as increased time required for data entry, mismatches between user interfaces and clinical workflow, interferences with physician–patient face-to-face conversation, hampered information exchange, information overload, and deterioration of clinical documentation (Declerck & Aimé 2014). However, a movement towards an alternative approach within the health care domain has recently begun; one that involves first capturing the necessary work domain demands to then inform technological development (Declerck & Aimé 2014; Hettinger, Roth & Bisantz 2017). Additionally, our own case study exemplar adopts the perspectives of pathway $\mathbf{P}_{\mathbf{2}}$ as described in the subsequent section.

2.2 Pathway 2: the ‘work-driven’ pathway

Rather than emphasizing technological capabilities at the onset of addressing the envisioned world problem, we contend that first envisioning the work domain in a future context is a more useful first step. Under this perspective, we view the work domain as ‘the system being controlled, independent of any particular worker, automation, event, task, goal, or interface’ (Vicente 1999, p. 10). An important aspect of this perspective is that tasks (e.g. actions to be performed by agents within the domain to accomplish a goal) are heavily influenced by hypothesized technological systems and physical artefacts being utilized. The artefacts themselves influence the specific tasks to be performed. Therefore, the problems and constraints of the future domain should be articulated. Formative modelling efforts such as CWA provide a viable avenue to systematically examine the work domain via constraint definition that can be more readily extended into a future context (Rasmussen et al. 1994; Vicente 1999; Miller & Vicente 2001; Salmon et al. 2010). To the best of our knowledge, the CWA framework has yet to be fully integrated with the envisioned world problem as we have posited.

We contend that the constraints that shape the existing work domain will more often than not also be present in a future context and therefore provide a valuable starting point for envisioning. More specifically, the CWA insights gained from the first three levels of the work domain abstraction hierarchy (AH) models, combined with decision ladders provide a critical linkage between the existing and hypothesized future domains (Miller, McGuire & Feigh 2017). Nevertheless, work domain examination at the onset of the envisioning process is necessary, but not sufficient. The extension of current work domain understanding should be clearly linked to the future context. This perspective leads us to explore how the necessary work domain insights might be derived to place operators in a realistic future work context.

Figure 3. Pathway $\mathbf{P}_{\mathbf{2}}$ : the ‘work-driven’ approach to addressing the envisioned world problem.

Under this ‘work-driven’ perspective, as shown in Figure 3, the existing work domain attributes (e.g. constraints, problems, expectations and technologies) are extended to a novel envisioned setting by defining the problem(s) and constraint(s) that are likely to exist. An important component of this process is to convey the likely shifts in work domain structure and distribution of work functions within the future context. Examination of the domain from a formative perspective allows for the identification of problems and constraints on the work domain. Once identified, these issues that link and/or change between the current and future domains can be analysed more thoroughly.

The physical constraints of the domain are often times more tangible to articulate, define and justify, given their physical nature. For example, future human spaceflight operations will contend with a communication delay constraint between crew and Earth-based support personnel (Love & Reagan 2013; Rader et al. 2013) (e.g. one-way light time communication between Mars and Earth varies from 4 to 20 minutes depending on relative planetary positions). Examining the application and consequences of such constraints is a valuable part of the envisioning process. Furthermore, establishing the presence of new or altered constraints enables the discussion of subsequent implications and work demands those constraints will impose within the future work domain.

A choice exists regarding what technological capabilities may have a place in the future domain. An important assumption made along vector $\mathbf{R}_{\mathbf{3}}^{\prime }$ in Figure 3 is that all existing technological artefacts are first considered, where applicable, within the future context. Asking questions such as: what would that resultant system look like if the similar artefacts are utilized? Would these artefacts provide the desired capabilities to fulfil the expected work functions? provide an initial step towards progressing along vector $\mathbf{R}_{\mathbf{3}}^{\prime }$ . These incremental shifts must be made explicit so that the existing domain can be more clearly mapped to an envisioned setting.

These translational efforts help establish a baseline definition of the future work domain and offer two benefits: (1) the apparent deficiencies in existing technologies are more readily visible when examined from a future context, and (2) potential desirable technological enhancements can be more readily identified and described to reach the desired end state. Note here the separation of identifying deficiencies and potential solutions. All too often technological solutions are proposed without fully understanding the intended work to be supported. Vector $\mathbf{R}_{\mathbf{4}}^{\prime }$ is shown in Figure 3 in the grey region as a range of potential directions which is in contrast with vector $\mathbf{R}_{\mathbf{1}}^{\prime }$ found in the ‘technology-driven’ approach.

The examination of existing technologies within a future context, rather than an existing context, enables a more detailed examination of what capabilities may be desired to support the work domain. Some existing domain technologies may not need enhancement to remain useful in a future context. Technologies should be hypothesized from a vantage point that more accurately encapsulates the context of intended use. The inclusion of new technologies from a future context where a better understanding of work demands can be represented provides a more narrow solution space of hypothesized design solutions thereby increasing the likelihood of effective design solutions.

2.3 Summary

In an idealized sense, technology-driven pathway $\mathbf{P}_{\mathbf{1}}$ represents a familiar and commonly adopted approach to the envisioned world problem: a new technology is developed and installed in a work domain and a new desired state is reached by operators compensating for the deficiencies or burdens the new technologies impose on the work being performed. Contemporary challenges of pathway $\mathbf{P}_{\mathbf{1}}$ include the digital revolution of EMRs in the medical field (Buntin et al. 2011; Ano 2015b ), next generation automated systems in air traffic control (Sarter & Amalberti 2000; Durso & Manning 2008; Ano 2015a ), military command and control (Jenkins, Walker & Rafferty 2012), and rail transport (Bearman et al. 2013). However, as discussed in the previous section, there exist systematic limitations to the affordances provided by pathway $\mathbf{P}_{\mathbf{1}}$ and challenges that must be addressed.

But there is another way to approach the envisioned world problem that first de-emphasizes technological development and prioritizes work domain definition. Work-driven pathway $\mathbf{P}_{\mathbf{2}}$ emphasizes the transition of the current work domain into a hypothesized future work domain ( $\mathbf{R}_{\mathbf{3}}$ ) to first consider the envisioned work to be expected in that domain as a means to then determine what technological capabilities might be necessary ( $\mathbf{R}_{\mathbf{4}}$ ) in that setting. The translation from State A to B can, and should, be performed by first emphasizing the definition of the future work domain where the current technologies, actors, environment, and problems are considered and translated to a future context. Under this concept, the focus is not on what new technologies could afford but rather how might the future domain resemble (or not) the existing domain.

For example, if NASA aims to extend a crew of 4 or 6 people from low-Earth orbit into deep space where they are effectively isolated from Earth-based support personnel, an understanding of the shifts in the current domain of human spaceflight operations (e.g. redistribution of work functions, responsibility and authority) must be acquired. A key aspect of vector $\mathbf{R}_{\mathbf{3}}$ is that the future context is considered with heritage technological capabilities. In other words, the already existent technological artefacts remain constant while the work domain characteristics (e.g. roles, responsibilities and work domain functions) are allowed to shift to meet hypothesized new work conditions, expectations and environment. We contend that this distinction offers a more realistic and meaningful way to reach the desired target (State B) as represented by vector $\mathbf{R}_{\mathbf{4}}$ . Some recent examples that implement this approach along pathway $\mathbf{P}_{\mathbf{2}}$ include the development efforts in the domains of human spaceflight (Miller et al. 2017) (to be used as a specific case study example in this paper) and health care informatics (Pennathur et al. 2010; McGeorge et al. 2015; Hettinger et al. 2017). In an era where technological capability is ever increasing, systems engineers and designers of technology must understand the realistic work demands before hypothesizing technological solutions.

Both pathways shown in Figure 1 constitute a spectrum of various approaches to addressing the envisioned world problem. However, the envisioned world problem has yet to be fully explored along these pathways, and furthermore there exists a lack of theoretical or practical guidance as to what methods and considerations could be leveraged to define, develop, and ultimately advance envisioned world problems along these various pathways. We argue that work-driven pathway $\mathbf{P}_{\mathbf{2}}$ offers a unique and promising avenue that could overcome some of the traditional challenges faced along the technology-driven pathway $\mathbf{P}_{\mathbf{1}}$ . No longer can technological advancement be thought about in isolation from realistic work domain expectations (Carroll 1991; Stary & Peschl 1998; Lintern 2012; Dekker, Hancock & Wilkin 2013). Additionally, cognitive systems engineering (CSE) practitioners will play an increasingly important role in the envisioning process, acting as the arbiter of design insight early in the design process that links both work domain and technological attributes to yield effective designs solutions (Feigh et al. 2018).

We propose that it is imperative to correctly identify and articulate the necessary functions within a domain as a necessary starting point for appropriately envisioning a future domain. It is from this functional understanding of the domain that the envisioning process can take place. As the requirements articulate, functions (that are already identified and desired) must be allocated on the basis of appropriateness and capacity with deliberate design reasoning. The application of technological capabilities should be thought of as a design hypothesis about how best to complete and overcome work domain demands. Therefore, speculating on the utility of new technological capabilities should not necessarily be the place to dedicate time, attention, and resources. The envisioning process at the onset requires an intimate understanding of the functions and constraints to be accommodated within the envisioned work domain, and should be where attention and efforts are devoted.

To this point, we have elaborated and discussed the envisioned world problem from two different pathways. We have argued that a work-driven pathway is a more desirable and useful approach for the envisioning process. In the following section, we will discuss how to follow the work-driven approach of $\mathbf{P}_{\mathbf{2}}$ by acquiring and articulating the constraints in the form of CSE requirements that shape domain behaviours.

3 Bridging work domain context from ‘the wild’ to the laboratory

Central to the envisioning process is acquisition of the constraints that shape work domain behaviour. Traditionally, work domain investigations have been divided into two distinct stages of research: studies made within the actual (or natural) work setting (also known as ‘the wild’) (Hutchins 1995; Patterson, Woods & Watts-Perotti 1999) and those made within a laboratory setting (Egan et al. 1989; Brehmer & Dörner 1993). The natural work setting contains the work as-practised context necessary for domain understanding, whereas the laboratory setting offers the mechanisms of control to examine targeted questions in a repeatable fashion. Unfortunately, these two settings often exist in isolation and there is limited literature attempting to connect these vantage points (Flach, Dekker & Stappers 2008; Brown, Reeves & Sherwood 2011; Lintern 2012; Rooksby 2013).

Linking the stages of observation is integral to the envisioning process and can be accomplished using three stages of observations, as shown in Figure 4. While the stages of observation themselves are not new concepts (see Woods (2003) for a theoretical discussion), the integration of such concepts within the envisioning process itself is. As explained throughout the remainder of this section, the sequence of staged observations are framed in a similar vein to Norman’s (1986) seminal work on bridging the gaps in computer interface design to explore the gulfs that exist (and are experienced in practice) between each stage (Norman 1986). More importantly, this approach incorporates the derivation and articulation of CSE requirements to formally capture the intrinsic work domain demands from which technological solutions may be hypothesized as the primary mechanism to traverse these gulfs. Collectively, these stages and the transitions between them offer one avenue to the envisioning process along pathway $\mathbf{P}_{\mathbf{2}}$ .

Figure 4. Transitioning along Pathway 2 – building insight for the future through the natural history, staged, and spartan lab world observations (observational stages adapted from Woods 2003).

3.1 Natural history

The ‘natural history’ stage of observation refers to the examination of traits exhibited by the existing (or historical) work domain. At this stage, the focus should be on ‘discovering or identifying the processes that drive performance and adaptation (Woods 2003, p. 43)’ within the work domain. Additionally, establishing how and for what purposes should the domain be studied is important to consider at the onset of investigation. The work domain under this context includes not only psychological and technical components/limitations that already exist, but also the social and relationships within the domain (see (Bisantz & Burns 2009, Ch. 8) for more in-depth discussions of work domain considerations.)

Fortunately, many methods already exist to examine an existing work domain; spanning from ethnographic investigations to applied CSE methods that we will not discuss in detail in this paper. Cognitive systems engineering methods collectively strive to ‘identify the basic requirements for how to support work that must be met if new technology will be useful to practitioners in context (Woods & Hollnagel 2006, p. 178).’ Cognitive systems engineering methods in particular provide a host of appropriately defined vantage points to conduct the examination of a current/historical domain that emphasize the characterization of work and the interplay between people, technologies and organizational factors. For example, contextual inquiry, provides a set of representations to help articulate aspects of work that range from the physical artefacts to the cultural arrangement of the work environment. Cognitive work analysis elicits through a set of modelling tools the underlying constraints that shape work domain behaviour (Rasmussen et al. 1994; Vicente 1999). For a comprehensive review of CSE methods and their applications, see Bisantz & Roth (2008), Bisantz & Burns (2009), Read, Salmon & Lenne (2012), Jiancaro, Jamieson & Mihailidis (2013). We acknowledge other methods outside of the CSE literature are likely applicable to the envisioning process but we constrain our discussions to the applicability of CSE methods, and more specifically CWA, in this paper.

While CSE methods provide an abundance of modelling tools that aim to capture the work as-practised, limited guidance is currently provided for progressing work domain insight into actionable envisioning efforts. More specifically, limited methodological support exists to help traverse what we identify here as the gulf of conception. In other words, how do we bring work as-practised understanding into a future context? To answer this question, we propose a greater emphasis on requirements definition using CWA modelling to overcome the challenge defined here as the gulf of conception, or translation transition along vector $\mathbf{R}_{\mathbf{3}}$ in our envisioning efforts as shown in Figure 1.

3.2 Gulf of conception

The challenge of the gulf of conception is to adequately translate existing/historical work domain understanding into a future setting. CWA methods to-date still leave practitioners with the responsibility of ‘crafting their own approach to design’ (Read et al. 2015, p. 171). Therefore, inherent work domain demands should be grounded in past/current domain experience. This premise is slightly different yet complimentary to existing approaches such as the future incidents methods (Smith et al. 1998) or ecological interface design (Vicente 2002). Rather than immediately explore what the envisioned work space might entail in particular hypothesized scenarios or propose design solutions in isolation, we contend the appropriate mechanism to translate existing domain characteristics to the future setting is the articulation of requirements similar to those traditionally used by systems engineers to specify things like power, weight, and performance of an engineered system. Here we propose work domain requirements that reflect the cognitive demands and information relationships inherent to the work domain as defined below:

1. (i) Cognitive Work Requirement (CWR): Specifies the cognitive demands, tasks, and decisions that arise in the domain and for which the operator requires support.

2. (ii) Information Relationship Requirement (IRR): Specifies the proper context for the required data, turning it into information that the decision maker requires.

These particular requirements were first developed by Elm et al. (2003), Potter, Gualtieri & Elm (2007) to achieve a similar goal of informing the systems design process. As a result, these requirements reflect the intrinsic demands associated to the work domain and are thus requirements that must be satisfied in any future hypothesized work domain. Note that traversing the gulf of conception does not involve proposing any specific design solutions, but rather focuses on the specification of requirements at some higher level of abstraction. With regards to the envisioning process, all envisioned aspects of both work domain characteristics (e.g. how it is arranged, the envisioned operators, allocation of responsibility and authority) should be considered hypotheses that strive to meet the previously stated requirements. In effect, these work domain requirements offer a standard by which each hypothesis can be prioritized, assessed, and ultimately validated.

At this point, we have explored opportunities and challenges that exist with studying the natural world and overcoming the gulf of conception. The following sections examine the challenges and perspectives relevant to the development of more controlled instances of the envisioned domain.

3.3 Staged world

The staged world provides an opportunity to both construct and situate observations within an envisioned context to examine the nature of practice and to help reveal what would (could) be useful (Woods & Roth 1988). Some research efforts provide guidance in constructing the staged world including scenario-based design (Carroll 2000), the future incidents technique (Smith et al. 1998), and synthetic task environment design (Flach et al. 2012). The important aspect here is that the future context should be imagined in as many ways and from as many, relevant stakeholder perspectives as possible.

This stage should be considered fluid so that it can accommodate a variety of perspectives from which the envisioning process can be scoped within the larger community of stakeholder perspectives. Additionally, this stage helps discern aspects that are similar or dissimilar from the existing work domain. Finally, the staged world enables the exploration of the variability that arises from situated operations (Turvey, Shaw & Mace 1978; Vicente 2000). This opportunity not only allows subjects to become familiar with the envisioned context, but also enables the familiarization of the implications of new/future demands.

In summary, the staged world provides an opportunity to gain operational experience within the future context while positioning their work within the larger body of development that may be taking place amongst the team of stakeholders. This observational opportunity allows contrasts to be made between the existing domain and what the future might entail and helps identify areas for more targeted research objectives to be examined in a more controlled setting, known as the spartan laboratory.

3.4 Spartan laboratory

The spartan laboratory encapsulates the more traditional experimental environments that abound within the academic literature. Commonly known as micro-worlds, spartan labs provide researchers a platform with extensive control to explore theoretical model development and technology evaluation. Historically, spartan labs have suffered from a lack of work context by operating under a simplified environment which makes results difficult to generalize to the natural world (Brehmer & Dörner 1993; Omodei & Wearing 1995; Brehmer & Elg 2005; Gonzalez, Vanyukov & Martin 2005), or limited advancement beyond paper-based conceptual designs (Naikar et al. 2003).

If the laboratory setting is viewed as part of the envisioning process, where the relevant constraints and problems found in the natural world are extended via the staged world to the spartan settings, then a strategically scoped and relevant environment can be generated. The spartan laboratory under this context still utilizes artefacts as a tool of discovery, but within a contextually relevant setting that enables a more detailed data collection and synthesis effort. In a spartan lab setting, the opportunity to explore more targeted evaluations of new technological capabilities can be made, without sacrificing the important contextual demands of the work domain. Furthermore, the construction of the spartan laboratory itself is an excellent exercise in defining the unavoidable assumptions (e.g. specific work practices, goals, constraints) that face the envisioned domain.

3.5 Gulf of extent

Consider for a moment that the gulf of conception is traversed and the development of the staged and spartan environments are under way. The challenge now shifts to being able to effectively iterate between the staged and spartan environments. Some useful simulation descriptions exist that describe aspects of the domain that are important to consider (Harvey et al. 2003; Pennathur et al. 2010; Williams, Medicine & Administration 2014); these resources however remain embedded within their own specific domains and do not address the envisioned world problem directly. The envisioned context in some ways must maintain traits that resemble the existing domain to promote subject-matter expert (SME) adoption, but in some ways there will be significant departures.

Coping with this extensibility challenge is defined here as the Gulf of Extent where work domain aspects must be prioritized and adopted by the community at large. Individuals embarking on the envisioning process will quickly realize that the ability to prioritize work domain aspects (e.g. specific problems or demands) will be a necessity. Conveying these priorities and in some cases convincing the community at large that these perspectives are important will be a sizeable task. Questions such as: what are the important problems to simulate; what are the scenarios (with appropriate demands) worth simulating; how do we capture objectifiable insight from those experiences? must be answered.

The central tenant here is that the staged and spartan worlds provide spaces for design hypothesis generation and exploration to help support the envisioning process by trial/error and subsequent refinement. Assumption management becomes important to articulate and control what is and is not represented in these work settings. Unfortunately, mechanisms for controlling the domain assumptions are not well defined in the literature. Furthermore, the development and implementation of appropriate measures by which to quantify simulation fidelity and assessing work performance is lacking. Some literature does exist to assist with measures development (Patterson & Miller 2010); however, the ability to apply performance measures that span both spartan and staged worlds is an open area for investigation.

3.6 Summary

By unpacking the envisioning process, system behaviours can be observed and documented that capture the realities of the observed stages as opposed to only measuring ‘successful’ system performance (Brown et al. 2011). Furthermore, by contemplating the envisioned domain at various stages of definition, a more clear articulation of ‘what is being argued against in order to be able to articulate what is in favour of [in terms of technological capabilities] (Rooksby 2013, p. 19:11)’ can be made. We argue these collective perspectives provide a tenable approach to traverse along vector $\mathbf{R}_{\mathbf{3}}^{\prime }$ as shown in Figure 4. However, to this point, limited theoretical discussion exists to more fully elaborate on the gulfs that presently exist.

To this point, we have emphasized a lens through which the envisioned world problem can be approached. The perceived benefits of technologies must be weighted against the demands and needs of the work domain in a more explicit manner. It is through this process of acquiring work domain insight and perspective that how work domains operate can be understood (Hutchins 1995; Woods 2003). In doing so, technological capabilities can then be better hypothesized to meet those work demands. By systematically approaching an envisioned world domain state, via the various aforementioned stages of observation and simulation, we contend that a more tenable process for work domain and technology development can be reached. It is important to note that up to this point in the paper we have offered one theoretical approach to tackling the envisioning process, i.e. the $\mathbf{P}_{\mathbf{2}}$ pathway. This paper goes one step further to provide a concrete example of how we applied this approach and to link it to other CWA literature. The remainder of this paper explores the application of the $\mathbf{P}_{\mathbf{2}}$ pathway approach to the envisioned world problem in practice.

4 Envisioning the future of human extravehicular activity – a case study

To demonstrate the envisioning process as a whole, this work attempts to construct a DSS for use in a future extravehicular activity (EVA) work domain. Extravehicular activity is a mission-critical component of human spaceflight with over 50 years of history in spacecraft and payload inspection, repair, and construction (McBarron 1994; Portree & Treviño 1997; Wilde et al. 2002). As NASA aims to send humans into deep space, EVA will remain a critical component of future missions (Ano 2009a , b ; Augustine et al. 2009); however, the ecological and technological landscape of future operations will differ substantially from the present-day work domain. One key challenge stems from the one-way light time communication delays between Earth and future destinations such as near-Earth objects (NEO) and Mars, thereby effectively removing the wealth of knowledge and resources provided by Earth-bound support personnel during EVA execution.

Considering that nearly 1 out of every 3 EVAs (28%), performed up to July, 2016, have encountered significant incidents such as systems and operational issues during execution that required the immediate intervention of Earth-based support specialists to rectify (Packham & Stockton 2016), the redistribution of the EVA work domain for deep space is a pressing challenge. What will the future EVA work domain contain that enables future crew to successfully execute operations without the immediate intervention of ground-based support personnel? We presuppose that the inclusion of automated systems will indeed benefit future EVA operations. And, since the work domain of future EVA is unknown, its subsequent development falls within the envisioned world framework.

Prior to this study, limited formal examination had been given to the EVA work domain, making it an ideal complex work domain to examine under the context of the envisioned world problem and to demonstrate CSE theory and methods to address the envisioned world problem. This section details the methodological considerations of how we approached the EVA envisioned world problem and provides some practical implications and limitations of executing such approaches.

4.1 Addressing the envisioned world problem in practice

Designable futures,  $\ldots$  , can result if we succeed in describing people’s work in terms that let designers proactively understand, even anticipate, the challenges of that work (Dekker, Nyce & Hoffman 2003, p. 5).

Figure 5 shows the various data collection, model development, and in situ observation efforts performed at each stage of the EVA envisioning process via Pathway 2 for this study. These efforts spanned four years of research and offer one collective perspective that we found successful to design and test a DSS for use in a future EVA setting. The various milestones indicated in Figure 5 were guided by questions that pertained to how the current work domain could inform the design of new technological capabilities to be used in future EVA operations. The list below describes the motivating objectives used to guide this research along with how each objective relates to the stages of observations. Additionally, published manuscripts associated with each stage of research are provided for the interested reader to elaborate on the details that supported our envisioning process. The following sections depict the advances and challenges experienced during this envisioning process and highlight various aspects of the process that we believe others will find useful.

Figure 5. The research activities involved in our envisioning process that follows Pathway 2.

Natural History:

Identify the system constraints on EVA operations (Miller, McGuire & Feigh 2015a , b ; Miller et al. 2016a ; Greenlund, Miller & Feigh 2017; Miller et al. 2017a ).

Natural History/Gulf of Conception:

Develop the requirements for a DSS for EVA operation within an envisioned context (Miller et al. 2017) (Miller 2017, Appendix A2).

Staged World:

Identify the characteristics of the DSS design that are likely to fulfil the DSS requirements in an envisioned context (Chappell et al. 2016; Miller et al. 2016b , 2017b ; Beaton et al. 2017; Miller, Pittman & Feigh 2017) (Miller 2017, Ch. 4).

Spartan Lab:

Assess how well the prototyped DSS performs in envisioned EVA operations (Miller 2017, Chs 5 and 6) (Miller et al. 2017).

The remainder of this section is a revisit of recently published work that applies CWA for the development of work requirements (Miller et al. 2017). We briefly discuss these efforts from an envisioned world perspective.

4.1.1 Natural history examination of EVA

We did not presuppose any solutions or hypotheses about what might ‘improve’ or ‘benefit’ the EVA work domain, but rather leveraged the constraint-based perspectives emphasized in the first two phases of CWA to understand what shaped behaviours within the domain with the desire to satisfy the research goals stated in the prior section. The first two phases, work domain analysis (WDA) and controlled task analysis (ConTA), aligned with our aims of characterizing the ‘Natural World’ of EVA operations by providing methods to study, articulate and organize the underlying constraints that shaped existing domain behaviour.

To support the initial modelling efforts of WDA and ConTA, we performed a variety of studies. A snow-ball sampling technique was used to seek out and interview SMEs within the EVA domain. These SMEs consisted of a variety of certified EVA flight controllers and managers. Semi-structured interviews utilizing a compilation of CWA decision ladder model probes as described in (Miller et al. 2017, p. 153) were used to acquire targeted work domain understanding and to orient ourselves to the various stakeholder communities that exist. Individual responses were compiled and synthesized into an aggregate set of content descriptions that mapped to specific stages within the WDA and ConTA models. The semi-structured nature ensured adequate progress was being made towards building the WDA models as well as provided the flexibility to allow the SMEs to convey their own domain perspectives.

Supplementing our interview data, archived data analysis was also performed. This effort involved examining archived video and audio footage of EVA of both current operations and training simulations on the International Space Station (ISS), as well as historical footage from previous spaceflight programs such as Shuttle and Apollo. Fortunately, a large archived data base was accessible for this study. Other domains that lack extensive archived footage/materials may need to rely more on SME interviews and in situ observations to acquire similar levels of domain understanding. By studying these archives, we became familiar with the language and expectations of the domain. We found our interviews and interactions with SMEs became more productive and insightful because there was less of a language barrier.

Final model validation was performed by reviewing the models with SMEs over a series of interviews, often in a concurrent fashion due to SME availability. The WDA and ConTA models were then used to generate work requirements that reflected the cognitive demands and information relationships exhibited within the EVA work domain. Requirements within the traditional systems engineering process provide the criteria to which system designs can be derived, validated, and verified; therefore, we aimed to provide CSE requirements in a format compatible with the systems engineering process.

The WDA and ConTA phases were well supported by the literature and their modelling applications were useful to manage the data collection and model generation process. To give a sense of scale to acquire this domain understanding, this modelling effort took the better part of two years of active research engagement within the EVA work domain. Acquiring a comprehensive familiarity with the challenges operators faced within the domain required time and patience, while also being opportunistic to SME availability. The stated research objectives were used to bound and steer this envisioning process by focusing on the construction of the intermediate WDA and ConTA models.

However, the existing literature only described how to build the models and required supplemental modelling efforts to satisfy explicit requirements definition. The following sections briefly describe these supplemental efforts and perceived utility of the overall CWA modelling efforts.

4.1.2 WDA – information flow model

Similar to the efforts made by Cummings & Guerlain (2003), Cummings (2004), we first developed a work domain information flow model to understand the domain as it exists today (e.g. what actors were involved in EVA operations and what was their expected work?). An information flow model describes the personnel, work artefacts, and information exchange present during operations. The flow model is not typically performed at the onset of a WDA. However, we had no prior experience with the domain and found it helped identify both additional SMEs worth consulting as well as highlighted what components of the domain were and were not already understood and published.

In this case, we restricted ourselves to only examining certified flight controllers within the EVA flight control group as opposed to considering the multitude of other controllers that also support spaceflight operations; see Patterson et al. (1999) for a full description of flight control groups. The key here was discerning what knowledge about the domain was already formally documented, whether it be in internal documentation or in the public domain. For example, much of the EVA hardware was well documented, whereas examinations of distributed responsibilities and teamwork involved during EVA operations were limited.

The information flow model was found to be useful when describing the domain to both SMEs and non-SMEs alike. Stakeholders did not require extensive briefings in order to understand the intent of the model and the structural elements it contained. One striking feature of the EVA domain was that while many people were familiar with the existence of EVA flight controllers, stakeholders and others were not always aware of the extent of influence that ground personnel had during operations. In other words, the existing domain had extensive familiarity with how the suited astronauts performed their tasks and the tools they used; however, a more complete picture of the domain that included ground support during EVA operations was missing even amongst otherwise knowledgeable individuals. Finally, the structure of the information flow model could be manipulated because it established a baseline to compare and contrast with potential future EVA operations later in the envisioning process.

4.1.3 WDA – abstraction hierarchies

The AH model relates work domain purposes to the physical hardware through means-end relationships. (See Rasmussen (1985) for a theoretical discussion of AH models and Naikar, Hopcroft & Moylan (2005), Naikar (2013) for practical applications of AH models.) We developed two AHs: one that described the collective set of EVA operators (e.g. MCC, IV and EV crew) and one that depicted the environment within which EVA operations take place. This decomposition effort was influenced by Burns, Bryant & Chalmers (2005) and Torenvliet, Jamieson & Chow (2008), where the domain elements were divided into various objects to help cope with the complexity of the domain. By establishing boundary objects within the domain, we were able to identify and decompose the main domain objects that shaped operations found in the information flow model while also articulating what was to be excluded from our analyses. We recognized that our development efforts could not address all components of the work domain, and we felt it was imperative our modelling efforts reflected the breadth of elements omitted for subsequent development efforts.

Special attention to completeness was given to the top three levels of AHs: Functional Purposes, Abstract Functions, and Generalized Functions. The top three levels of the AH are consistent regardless of the changes in underlying technologies used by the work domain. The Functional Purposes help describe why the domain exists in the first place; it was important to understand these purposes and assess whether or not those purposes would persist in the future context. The Abstract Functions level defined more domain specific constraints to be considered. Most importantly, the EVA domain as depicted at the Generalized Function level helped shape what specific functions to study and refine in subsequent analyses (Miller et al. 2017).

When considering what aspects to focus on for our envisioning process, we identified two functions as highest priority based on the evidence provided by our model development and observations made in the existing domain: timeline and life support system management. First, the domain is organizationally structured to specifically promote these two functions, as shown in the information flow model (Miller et al. 2015a ). Second, when considering other EVA generalized functions, SMEs simultaneously have to consider timeline and life support system implications. This trend is an inherent behaviour of EVA operations, where decisions are based on the most up-to-date understanding of timeline and life support system performance; therefore, the management of these aspects is paramount to ensure successful operations (Miller et al. 2017).

4.1.4 ConTA – contextual activity template

The notions of ‘nominal’ and ‘off-nominal’ are important to understand in complex operations such as EVA. Additionally, the specific aspects of operations to examine and support need to be defined early in the envisioning process. The contextual activity template, which maps phases of operation to the generalized functions that we identified in the AH models, provides a useful representation of the work that needs to be performed at the different phases of operation (Naikar 2013).

The ConTA modelling was useful for two reasons: (1) it helped convey to the larger design community the range of operations to be expected during operations that were objective agnostic. All EVA operations will experience the specified phases of operation at some point during execution and (2) it provided additional motivation to justify what functions were likely worth closer inspection, based on how many phases of operation those functions influenced. The two generalized functions, life support system and timeline management, again, were identified as constant functions actively engaged by a community of flight controllers during operations to support EVA operations. At this stage of our analysis efforts, we began to consider how these specific work functions during these particular phases operations might differ in a future operational setting, thereby providing some focus for our envisioning process. The activity template also helped further refine the boundaries of our envisioning to clearly delineate what was omitted from subsequent analyses.

4.1.5 ConTA – decision ladder model

Building upon the insight of our information flow, AH and activity template models, we found the decision ladder model to be useful point of departure from traditional CWA modelling efforts to traverse the gulf of conception (see Figure 4). Timeline and life support system management functions, which were identified in the prior models, were applied directly as the goals of the decision ladders to guide SME interviews for the construction of parts of the subsequent decision ladder. The existing literature provided a variety of useful templates to guide our model development. We refer to the following references for model description and application examples: Vicente (1999), Naikar, Moylan & Pearce (2006), Bisantz & Burns (2009). Some useful aspects regarding the application of our decision ladder model are discussed below:

1. (i) We did not emphasize the data-processing activity stages as we were not as interested in how the SMEs currently do their work as these activities are highly moderated by the tools at their disposal. Instead, we focused our efforts on understanding the intermediate states of knowledge (SoKs) they attempted to obtain throughout EVA execution. Since the specific tasks of the current domain are rooted in existing artefacts and procedures, we wanted to understand the SoKs the SMEs were trying to reach so that we could explore how those specific processing activities could be reached in the envisioned domain.

2. (ii) We first constructed the SME SoKs based on the specific SoKs observed from the in situ observations. This initialization of the DL provided a useful starting point for targeted SME interviews to critique the content. Additionally, the SoKs were written in the form of questions and were iteratively assessed individually by SMEs for comprehensiveness and correctness.

3. (iii) Finally, each individual SoK was used to derive two levels of early DSS design requirements. The SoKs represent the breadth of concerns and work as-practised demands to be expected during EVA operations; therefore, the DL model lends itself to be an ideal vantage point to derive DSS requirements.

The DL model was a useful tool for eliciting specific SME domain insight regarding the constraints and problems that exist when performing specific work functions. However, as described within the next section, these SME SoKs alone were not sufficient to derive requirements for envisioned technological capabilities.

4.2 Traversing the gulf of conception

One of the critical components of the envisioning process itself is to articulate existing knowledge about a work domain in a way that is relatable to a future context. To support this effort, we distilled the volume and variety of observations made during the natural history stage of observation by constructing cognitive work and information relationship requirements based on the SME SoKs derived directly from the decision ladders. In doing so, we compiled a comprehensive set of requirements that we then prioritized to explore in the envisioned future context.

Throughout our progression of pathway $\mathbf{P}_{\mathbf{2}}$ , we aimed to integrate our CSE insight into the larger systems engineering process which expects requirements to guide system development; see Elm et al. (2008) for a review. Previous examples of the requirements derivation process utilized AHs as a tenable way to derive these types of requirements (Burns et al. 2005). However, we found that using the decision ladders proved more useful in terms of minimizing overhead in data management on our part and facilitating fruitful discussion with SMEs. Additionally, AHs can be difficult to explain to SMEs, whereas the Decision Ladder model is a more straightforward model to utilize in an interview setting. Each SoK was assigned at least one cognitive work requirement (CWR) and information relationship requirement (IRR) pair that recast the SoK questions into a more standardized requirement format. Figure 6 below shows an excerpt of this process for the Set of Observations stage of the DL.

Figure 6. Requirements example for the set of observations stage of the DL to depict the SoK, CWR, and IRR statements for timeline management as the overall DL goal.

We assessed the requirements based on the standard requirements format expected by the traditional engineering community as stated by Turk (2006). Requirements are expected to be necessary, correct, unambiguous, traceable, prioritisable, results oriented, verifiable, and feasible. Our initial assessments of our derived early design requirements meet these requirements with only verifiable and feasible attributes left as outstanding characteristics to be more concretely examined at later stages of the design process. The envisioning process requires the identification of what is necessary and correct to examine early in the design process. The WDA and ConTA help distil this necessary information in a traceable and prioritisable way and the instantiation of requirements provided a useful mechanism for extending our domain understanding into a future context.

Three main insights were gained from the natural history examination: (1) the identification of two key work domain functions (life support and timeline management) to be examined in a future context; (2) we distilled the high level requirements associated with performing those functions as they exist in the current domain to help define what constraints and considerations need to be accounted for in the future; (3) the intravehicular operator was identified as the likely user for an advanced DSS system. The IV operator will likely take at least some of the current work expectations found in the EVA flight controller community when we shift the operational context to a future, time-delayed communication setting. By narrowing a broad natural history investigation to a few key intrinsic domain attributes, we now had the necessary prerequisite understanding to develop, explore and study future EVA operational settings in the staged and spartan laboratory settings.

4.3 Situating design solutions within envisioned settings along the $\mathbf{P}_{\mathbf{2}}$ pathway

At this stage of the envisioning process along Pathway 2, we explored what the envisioned EVA work domain might look like using both the large-scale simulated staged and controlled spartan laboratory settings. By leveraging both environments, we could compare and contrast envisioned domain components with heritage domain characteristics, and begin to explore the impact of new technologies in an envisioned setting. We engaged in a range of research activities that capitalized on already active development of future EVA operations.

The following sections describe our envisioned staged and spartan world research activities and discuss what we found important while we were engaged in such activities. For comparison, Figure 7 shows a summary depiction of our activities made across all three stages of our envisioned world problem. We studied and built an information flow model of the existing domain which we then used to compare our staged and spartan models. With respect to our DSS development efforts, we highlight in Figure 7 the intravehicular operator who is our key user of our envisioned system. It was from this user perspective that we joined the research teams of various on-going NASA analogue research programs that aligned with the staged world phase of the envisioning process. These efforts provided the opportunity for us to (1) build familiarity with complementary research activities, (2) contrast these observations with insight gained from the natural history examination of the existing domain, (3) enabled an understanding of where current research interests are within the community at large.

Figure 7. Domain understanding elicitation among the various stages of observation with the intended goal of enabling the Intravehicular (IV) operator under different degrees of simulation fidelity.

4.3.1 Staged worlds – becoming embedded in the future context

Multiple concurrent efforts were performed by NASA to better understand future human spaceflight operations in the form of simulated mission analogues. We were able to leverage these efforts instead of creating them from scratch. Figure 8 shows the Earth-based analogue research programs we participated in as part of our envisioning process.

Figure 8. Staged world participation within three NASA analogue research programs: PLRP: Miller et al. (2016b ), NEEMO: Chappell et al. (2016), BASALT: Beaton et al. (2017), Deans et al. (2017).

We found it important to treat these staged worlds as an opportunity to observe and learn what the community at large was interested in understanding. Therefore, we integrated our research efforts for developing an IV-specific DSS within the scope of the larger research objectives pursued by these analogue programs. Below are a few important characteristics and insights that shaped our envisioning process within the staged worlds:

1. (i) We found it important to actively seek out experiences that had relevance to the staged/spartan stages of the envisioning process. The opportunities available to us predominately resided in the staged world phase of the envisioning process which we leveraged for our subsequent spartan laboratory development efforts.

2. (ii) The staged world allowed us to assess what aspects of the requirements derived from the natural world were actively being pursued in the envisioned context. We internally assessed the requirements in relation to what was observed. Table 1 shows a comparison of our resultant spartan laboratory setting to these larger-scale staged world experiences.

3. (iii) The staged world enabled us to observe what physical and software artefacts currently exist. This is important because up to this point in the design process, we purposefully limited generating design hypotheses so that we could gain familiarity with the future context. In our specific example, the construction of IV support systems had limited formal investigation prior to our own research efforts. Therefore the staged world helped us confirm that our spartan lab and DSS design efforts were in fact novel and we were not duplicating previous works.

4. (iv) The staged world enabled the identification of what current measures and metrics of EVA execution were deemed valuable. The natural history observations revealed the meticulous nature of EVA timeline execution and life support system scrutiny, whereas, the staged world was interested in quantifying EVA performance and assessing the timeline designs. These comparisons were useful to identify what was worth considering for our own spartan laboratory setting.

Table 1. Staged world and spartan lab support system comparisons (adapted from Miller et al., 2017b)

As it related to our EVA work domain, we recognized from our WDA modelling that EVA operations are fundamentally influenced by the mission objectives. Consequently, the specific research objectives being pursued in these staged worlds had profound impacts on what could be observed in the staged work domain. For instance, the staged worlds shown in Figure 8 emphasized incorporating realistic scientific field investigations within the constraints of EVA operations. But from our natural history investigations, we recognized that the pursuit of scientific objectives has not been incorporated in the EVA domain since the Apollo program. Therefore, we aimed to incorporate available data from the Apollo program since that appeared to be a more representative domain example of future operations.

Finally, we needed to be strategic in contributing to the staged world development to avoid over-committing our time and resources. In our case, since we were interested specifically in timeline and life support system management from the IV operator perspective, we were actively engaged in all timeline development efforts and provided the staged worlds a simulated life support system software derived from present-day management displays. Furthermore, since we were interested in supporting EVA execution, we made sure to embed ourselves in the execution either within simulation subject to acquire a first-person perspective or as a silent observer for a third person point of view.

4.3.2 Spartan laboratory – targeted construction of the future context

A spartan laboratory simulation environment was built to test both a baseline and an advanced support system to examine the utility of each within relevant hypothesized operations (Miller & Feigh 2019). These support system designs are shown in Figure 9 with how each design element was linked to relevant operational data. Both designs provide the same underlying functional support to meet a prioritized set of requirements derived from the natural world, but the way in which that functionality is supported differs. Most notably, aspects of life support system and timeline management were performed using both static paper-based tools within the baseline design to represent present-day work domain practices. The advanced DSS supported the same calculations but required interfacing with a novel software tool. Most importantly, the design features found in each of these work stations could be traced to specific CWR and IRR requirements derived from the decision ladder models. (See Ch. 4 of Miller (2017) and Miller & Feigh (2019) for more detail.)

Figure 9. Baseline and advanced DSS design configurations and the various information relationships expected to be performed by the IV operator. The arrows show how the information was originally scattered across many locations and artefacts; it is consolidated in the second design. Equations represent higher level operational data that is necessary to support EVA operations. ETA: estimated time of arrival, PET: phased-elapsed time, LSS: life support system.

Below are a few key aspects that we gleaned from our spartan laboratory design and testing campaign:

1. (i) An important question to be addressed is: what is important enough to simulate/emulate in the envisioned context? Building a contextually relevant environment is a time and resource intensive endeavour and we found that starting with the present-day work artefacts from the existing domain provided a useful starting point to build the simulations from scratch.

2. (ii) Another important component of our spartan laboratory was appropriately simulating the natural variability exhibited during operations. Crew timeline execution can vary both in terms of time and tasks performed. Therefore, to better understand the magnitude of this variability, we performed a detailed examination of Apollo lunar surface EVA operations as a surrogate to Mars surface operations. In doing so, we quantified the variability exhibited both to shape our spartan scenarios (see Miller et al. (2016a , 2017a ) for our operational assessment of the Apollo program).

3. (iii) The construction of the simulation elements, both hardware, paper products, and software is a non-trivial effort. We replicated many of the existing artefacts first in addition to designing an envisioned software tool that could support the same sort of work – timeline tracking and life support system monitoring. To conduct the simulation itself required the involvement of one researcher, two surrogate astronauts and the participant. See Figure 7 for a depiction of our spartan laboratory.

4. (iv) We also had to define the measures and metric definitions and incorporation of those data collection efforts into the scenario development. These data types needed to be domain specific to adequately capture performance. Complementary to this effort is the challenge of setting appropriate measures of control to implement during simulations to maintain simulation repeatability while still allowing for envisioned natural variability throughout the scenarios. For example, our simulation included both scripted and unscripted communication periods to ensure certain actions and information was exchanged as precise periods throughout the simulations.

Given the lack of formal work domain definition of possible future Mars surface operations, we assumed that utilizing novice EVA personnel provided the opportunity to examine the limits of crew performance within our spartan lab. As a result, we provided a one-to-one ratio of training and testing time to ensure the participants were adequately trained to perform the job functions specific to our requirements derived from the natural world.

4.3.3 Modulating the gulf of extent

The ability to compare and contrast the staged and spartan laboratory relies on the ability to adequately articulate and apply prioritized design requirements. The spartan laboratory setting by definition should not attempt to incorporate the context experienced in the staged setting. Therefore, we emphasized building a spartan laboratory that focused on the altering and sets of observation SoKs required to initiate the decision ladder for timeline and life support system management simultaneously. In doing so, we could reduce the number of requirements actually being tested in the spartan laboratory to examine two DSS prototypes that we hypothesized could meet the specified requirements. As a result, we examined two different prototype designs; one that leveraged existing artefacts and one that represented a departure from existing tools in the form of a new digital software system. Both systems represent novel ways of performing work in a future setting (extension along $\mathbf{R}_{\mathbf{3}}$ ) while utilizing different technologies (extension along $\mathbf{R}_{\mathbf{4}}$ ).

To supplement our efforts along vector $\mathbf{R}_{\mathbf{3}}$ , we developed scenarios that were derived from both existing work domain examples as well as staged world observations. This allowed us to situate our spartan laboratory within flight-relevant tasks and realistic objectives. Each of these derivation processes helped us modulate how far we extended the spartan laboratory from the existing domain context (Miller et al. 2017b ). Our specific case study did not fully attempt to iterate from the spartan laboratory back to the staged world, but this step is a near-term challenge that will need to be addressed if spartan laboratory studies hope to influence larger-scale simulations in future research.

In summary, coping with the variety of assumptions and scale of the simulations between the staged and spartan environments is a non-trivial activity. We found the extensive list of requirements derived from the natural world provided a grounded reference point to focus our hypothesized design solutions. In other words, through prioritizing the requirements, we could target a few for evaluation in the spartan laboratory setting. Furthermore, to more appropriately demonstrate the utility of advanced software features, we also generated a baseline workstation solution that incorporated present-day work artefacts and practices to make a relative comparison of performance.

4.4 Case study limitations

It is important to consider a number of limitations of the present study. The first is that this paper includes a single example in a single work domain application. While much of the presented work is largely domain agnostic, these insights were shaped by our experiences and interactions with our specific domain. Additional work is required to fully validate this envisioning process using other complex sociotechnical work domains, such as efforts demonstrated by Burns, Bisantz & Roth (2004).

Second, this integrative approach is not a complete theoretical description of the entire envisioning process. The collective envisioning process described in this paper attempted to integrate three theoretical components by incorporating aspects of CWA with adaptations of Wood’s observational stages of the envisioning process (Woods & Dekker 2000; Woods 2003) along with Norman’s approach to bridging gaps in the design process (Norman 1986). In particular, the latter staged and spartan stages and modulating characteristics between them have ample room for improvement. Contributions from fields such as Naturalistic Decision Making (Hoffman & Klein 2017), and Macrocognition (Patterson & Miller 2010) are likely to add more resolution to the envisioning process and is an area of future work.

Finally, this study was limited in that we were only able to explore a small subset of the design space and out of necessity used a substantial number of assumptions, especially during the staged and spartan phases, to scope our desired research outcomes. The challenge still remains in how to manage the variety of assumptions required for work domain construction and how to scale these assumptions to include more complex DSS capability and more sophisticated and representative simulation scenarios. This research demonstrated how to incorporate a small subset of derived DSS requirements across the envisioning process as a useful starting point but the design hypotheses and associated assessment criteria are still at the research practitioner’s discretion.

5 Conclusions

This paper sought to demonstrate the application of CSE methods to the envisioned world problem. Our intended goal was to provide a theoretical foundation to a problem common to CSE practitioners while simultaneously advocating a shift in perspective from a technology-driven to a work-driven pathway. We recast the envisioned world problem along two different pathways, technology-driven and a work-driven, to illustrate opportunities and existing challenges associated with the envisioning process. We argued that there is pathway dependency and that a work-driven envisioning process should be followed that emphasizes associating hypothesized design solutions to cognitive work and information relationship requirements inherent to the work domain under investigation.

Using this framework, the challenges and opportunities that exist in addressing the envisioned world problem were examined by decomposing pathway $\mathbf{P}_{\mathbf{2}}$ into five stages of research. The natural world was prioritized to first capture the constraints that shape existing work domain behaviour. To overcome the gulf of conception, we contended that natural world studies should aim to derive cognitive work and information relationship requirements that articulate the inherent cognitive demands of the domain. We describe in detail one way of leveraging the CWA framework to achieve this goal. Subsequently within the staged and spartan laboratory settings, these requirements can be prioritized and managed to trace and validate their hypothesized work domain and technological designs.

Envisioning the future of an existing work domain can be a slow process. For each departure from existing work practices and technologies, ample explanation, demonstration, and justification are required to make progress clear among the larger set of domain experts. Additionally, what might not appear as a radical departure from existing practices as a researcher, (e.g. the idea that a DSS for an IV operator is likely a critical component of future deep-space operations) can in fact be a substantial shift in domain thinking. For example, the contributions of this research were formally recognized by the NASA community in our efforts to successfully articulate to domain experts the utility and importance of considering the IV operator work and the development efforts and considerations that will need to be included in the overall EVA systems development process (for news coverage of the 2016 NASA@Work Mars EVA Gap Challenge, see https://www.nasa.gov/feature/nasawork-february-2016-monthly-winners). Further adoption of these research perspectives was demonstrated by the adoption of the DSS software product and numerous other analogue research efforts that are now beginning to align with the overall envisioning process.

Finally, a demonstration of the application of these perspectives was provided that highlighted aspects of the envisioning process as they pertain to advancing the human spaceflight work domain. By following the envisioning process described in Figures 2 and 3, we argue that a more realistic and desired envisioned world can be instantiated for the development of technology and work practice refinement. In doing so, we provide one end-to-end example of navigating the envisioning process that prioritizes a hypotheses-driven process that matches both the ecological and technological advancements to the inherent cognitive challenges present in the domain.