Exploring how design guidelines benefit design engineers: an international and global perspective

Abstract

A multitude of design guidelines that are intended to support design engineers with knowledge and information during the embodiment design of physical products have been developed back in the 1980s and 1990s. However, since then, the setting in which products are developed and designed has changed and associated tasks and activities are carried out globally rather than locally. Yet, knowledge on the benefit of such design guidelines and their impact on the performance of multinational design engineers from different regions and with different levels of experience is still lacking. To address this, a mobile eye tracking study has been developed and carried out with 47 differently experienced practitioners from Germany, Eastern Europe and Asia. The results show differences in how design engineers from different regions with different levels of experience may benefit from design guidelines and how design guidelines may impact experts’ and novices’ performance, indicate beneficial ways of using them and point out the kind of information and the way of representation that attracts the most attention within a design guideline. The paper concludes that the improvement and development of design guidelines that are intended to support the embodiment design of physical products is needed and proposes to rethink current engineering design guidelines both content-wise and representation-wise.

1 Introduction

The emergence of global markets and the globalisation and internationalisation of R&D has begun to have an impact on industry’s product development practice. The increase in globally distributed engineering design activities (Chiesa 2000; Hansen, Zhang & Ahmed-Kristensen 2013), i.e. the major transformation from co-located product development to global product development (Eppinger & Chitkara 2006, 2009; Taylor et al. 2018), has led to new challenges and is a trend in industry today; tasks and activities are continuously globalised, from the early conceptual stage to the final testing of prototypes (Taylor et al. 2018).

In association with global product development, companies are facing challenges which may have been underestimated, like studies indicate (Eppinger & Chitkara 2006, 2009). In a survey, for instance, the Aberdeen Group (2005) emphasises that companies strategically decide for global product development in order to reduce product development costs by hiring low-cost, skilled engineers that are distributed globally. Such cost reductions turn out to be the prime motivation, as reaffirmed by multiple studies (Eppinger & Chitkara 2006, 2009; Taylor et al. 2018). However, despite cost reductions, Zhou, Wu & Shi (2018) also state the need for engineering capabilities to be present locally in order to develop products tailored to the local market by either adapting existing products or developing new ones, i.e. having a ‘local-for-local’ rather than a ‘centre-for-global’ strategy (Bartlett & Ghoshal 1990). Also in association with global product development, researchers stress an increase of time required for coordination issues (Littler, Leverick & Bruce 1995) and, in general, greater project management challenges (McDonough III, Kahn & Barczak 2001). Maintaining global product development collaborations has become the main objective instead of developing the product itself (Littler et al. 1995).

Designing physical products, no matter whether locally or globally, involves the application of technical knowledge (Wallace & Burgess 1995) and remains a non-trivial, largely knowledge-based (Tomiyama 2003) and information-rich activity (Anthony 1985; Hubka & Eder 1990). To come up with any reasonably realistic design, profound domain knowledge (Ullman 2010), as well as a broad basis of capabilities (Matousek 1963), knowledge in multiple fields of the product lifecycle (Albers & Burkardt 1998; Ehrlenspiel & Meerkamm 2013) and the ability of combining different disciplines (Pugh 1991), is required. Even more, it is crucial to understand the details in order to achieve qualitatively good and well-functioning designs (Matthiesen 2017, 2019). This points out the need to support design engineers within their daily work, as it cannot be expected that the engineer becomes an expert in all relevant fields (Leake & Wilson 2001) and is aware of all knowledge and information related to the task (Ehrlenspiel & Meerkamm 2013), especially to the design task itself, which is to answer the questions on ‘what’ to be achieved, i.e. what functions are to be fulfilled, and ‘how’ to achieve it, i.e. the physical embodiment (Suh 1990). The way to the physical embodiment, by applying a mapping procedure which is characterised by an interplay of recurring analysis–synthesis cycles (Matthiesen 2016, 2019), is mostly the most challenging step (Albers & Matthiesen 2002) and, yet, one of the less clarified ones (Hoenow & Meißner 2016; Matthiesen 2016). Thus, besides the need to support design engineers with knowledge and information sources, especially within the embodiment design phase, further research is still required to understand the design engineer’s behaviour and needs.

Though a lot of work has been undertaken to evaluate design engineers’ behaviour in the past, the majority of those studies that were intended to understand the design engineers’ behaviour more deeply was carried out to investigate mainly the early conceptual stage (Dinar et al. 2015; Yu et al. 2016) rather than the embodiment design phase. Researchers considered different levels of experience and describe how experienced and novice designers behave, e.g. Cross & Cross (1998), Atman et al. (1999), Ahmed, Wallace & Blessing (2003), Kavakli & Gero (2003), etc. Others were also interested in the information-seeking behaviour of design engineers, e.g. Kuffner & Ullman (1991), Court, Ullman & Culley (1998), Hertzum & Pejtersen (2000), Aurisicchio, Bracewell & Wallace (2010), Cash, Stanković & Štorga (2013), etc. The largest amount thereof investigated what types of knowledge are used and where the knowledge is stored (Reed et al. 2011). However, throughout the design research community, the value of design knowledge, as well as that its reuse benefits the design engineer, is commonly accepted (Busby 1998; Reed et al. 2011) and stresses the need to capture, store and reuse knowledge (Ahmed 2005).

To support the reuse of design knowledge, design guidelines were developed, introduced and are widely established nowadays (Ponn & Lindemann 2011; Feldhusen & Grote 2013). It is believed that applying and using design guidelines potentially leads to reasonable and improved designs, though the application and usage does not inevitably result in better products (Verein Deutscher Ingenieure 2004–01). Moreover, authors state that design guidelines are especially valuable for novice design engineers (Verein Deutscher Ingenieure 2004–01; Hoenow & Meißner 2016). Furthermore, researchers believe that, if structured in a systematic way, design guidelines can bring benefits to the design of new products (Gouvinhas & Corbett 1999). Thus far, however, there is still a lack of research that corroborates the benefit of design guidelines, and the clear link to design engineers’ levels of experience and performance is still missing. It is furthermore stated that design guidelines are often poorly structured and incomplete (Nowack 1997). A uniform specification that describes how to document knowledge in design guidelines, to ensure their benefits, still does not exist (Butenko & Albers 2018). This leads to multiple documentation forms, structures and depths of detail (Butenko & Albers 2018).

Industry has been aware not only of the potential benefits of design guidelines but also of the challenging embodiment design phase. Consequently, industry itself developed design guidelines that were intended to support design engineers by providing them with reusable knowledge and information on embodiment designs. Different documentation forms, structures and depths of detail, as raised in the literature, do reflect the current state of design guidelines in industry. Some of these design guidelines were published, e.g. in terms of comprehensive design handbooks (E.I. du Pont de Nemours and Company 2000; BASF Corporation 2007a; LANXESS Corporation 2007), cover specific techniques only, e.g. joining techniques (LANXESS Corporation 2005), or deal with single joining elements, e.g. BASF Corporation (2007b). Some design guidelines were for internal use only. Most of these design guidelines that were intended for internal use only were developed back in the 1980s and 1990s. However, since then, the setting in which products are developed and designed has changed. Knowledge on the benefit of design guidelines and their impact on the performance of multinational design engineers with different levels of experience, as it is today’s common practice, is still lacking.

To tackle this, the present paper presents a study with 47 differently experienced practitioners from three different regions and explores the benefit of design guidelines, which are intended to support the embodiment design phase, used in industry. To do so, we formulate one central and overall guiding research question (RQ) that focuses on embodiment designs that fulfil functions that are typically common to multiple applications, a key requirement for a design to be reusable (Girczyc & Carlson 1993), like single engineering design elements, as the design of such elements, no matter whether locally or globally, is of fundamental importance and expected to be part of a design engineer’s professional proficiency:

RQ 1-0

How do design guidelines impact the design engineer’s performance while designing single design elements?

To address the factors of successful usage and the poor, non-uniform structure of design guidelines raised in the literature, we decompose the overall guiding research question into three further research questions, which all together contribute to answering RQ 1-0 as well:

RQ 1-1

How do design engineers use design guidelines for designing single design elements?

RQ 1-2

On what kind of information do design engineers focus while designing single design elements?

RQ 1-3

How does the information have to be represented in order to catch the design engineers’ attention?

Motivated by recently reported studies in the field of engineering design using research methods that allow obtaining quantitative data, we use mobile eye tracking to address the topic of engineering design guidelines, expertise and human behaviour in an international and global context, as introduced above, and to investigate and answer the stated research questions. In order to explore this topic, it is the first step to reveal how these areas have been treated in engineering design research. Next, we describe the methodology of our research including the used materials and methods, present the results of our studies and answer the stated research questions, discuss those results and finally draw the conclusion.

2 Background and related work

This section gives an overview of past research that investigated engineering design in an international and global context, presents related and most relevant work done by other researchers in the field of sharing and reusing explicit engineering design knowledge by means of design guidelines and outlines the potential of mobile eye tracking as a suitable method for analysing human behaviour in engineering design. The aim of this section is to show the necessity and potential of bridging the research gap.

2.1 Engineering design in an international and global context

Many companies have recognised how the development of products has been shifting to global product development. The increase in globally distributed engineering design activities (Hansen & Ahmed-Kristensen 2011) and global product development has also played a major role in academia (Eppinger & Chitkara 2006, 2009). Researchers focused on distributed teams, e.g. Payne et al. (2002), Seki & Nishimura (2011), how to lead and manage them, e.g. Barczak & McDonough III (2003), Salomo, Keinschmidt & de Brentani (2010), explored their communication, e.g. Tavčar et al. (2005), Eris, Martelaro & Badke-Schaub (2014), and shared understanding, e.g. McComb, Green & Compton (1999), Cash, Dekoninck & Ahmed-Kristensen (2017), their collaboration, e.g. Harvey & Koubek (2000), Hammond, Koubek & Harvey (2001), and investigated impacts and barriers, such as time zones and culture, e.g. Cummings & Teng (2003), Larsson (2007). Within global design research, cross-culture collaboration and distributed teamwork has become the prime focus (Ma 2005; Ono 2006; Hansen & Ahmed-Kristensen 2011), which, in association with the multitude of literature arisen in the recent years, in turn, may reaffirm the findings of Littler et al. (1995) that developing the product itself has not been the main objective.

In times of internationalisation and globalisation, and in association with the research efforts in regard to communication, knowledge sharing (McDermott & O’Dell 2001; Wallace, Ahmed & Bracewell 2005; Zammit, Gao & Evans 2016) and reuse was also a topic. In order to reuse knowledge, Hansen, Nohria & Tierney (1999) describe codification as the most promising knowledge sharing strategy to be pursued. And in case knowledge is not codified, companies that are globalising their product development process are facing challenges and difficulties with knowledge sharing, as found by Hansen & Ahmed-Kristensen (2011) in their study.

Though past work was concerned with globalisation and its impact on engineering design practice, the focus was still more on managing internationally and globally collaborating teams. Indeed, knowledge sharing came to the fore and the literature suggests a strategy to be pursued. However, the development of knowledge sources for and the needs of multinational design engineers is rarely considered. Proposals of how a piece of codified knowledge might be used and structured in order to benefit multinational design engineers are still lacking.

2.2 Design knowledge reuse by means of design guidelines

Associated with the reuse of design knowledge, Busby (1999) states the issue that reuse is facing challenges, such as design knowledge being replicated inappropriately and that errors are likely to be introduced when adapting existing designs to new purposes. Design guidelines are well suited to tackle the issue of reusing design knowledge stated by Busby (1999), especially in an international and global environment where codification is the most promising strategy for knowledge sharing.

A literature review shows only a few studies, cf. Edwards (1994), Nowack (1997), Butenko et al. (2017) and Butenko, Wilwer & Albers (2018), for instance, that explored the need for, as well as the development and improvement of, design guidelines. Nowack (1997) carried out research to improve particularly manufacturability guidance to designers and focused more on the early stages of the design process. He concluded that, in particular in those early phases he focused on, guidance can be improved through the use of structured guidelines (Nowack 1997). Edwards (1994) also found this usefulness in the early phases, i.e. the conceptual design phase, by presenting a case study that involves a previously developed collection of design guidelines; cf. Edwards, Wallace & Aguirre-Esponda (1993). He further states that the usefulness of those guidelines tended to weaken during the embodiment design phase and concludes that the specific needs and uses of guideline type information have to be investigated more fundamentally (Edwards 1994). By using the example of fibre-reinforced components and by focusing on various users, Butenko et al. (2018) presented requirements on design guidelines. They concluded that design guidelines like those used currently are not suitable for their application in industry and that, for the development of new design guidelines in the future, industry’s requirements have to be considered (Butenko et al. 2018). The fact that Butenko et al. (2018) did not focus solely on one target group, however, limits their findings and conclusions so that they are not directly transferable to and valid for design engineers.

Though past work points out that structure seems to be key for the benefit of a design guideline, the findings also show that design guidelines were more useful in the early conceptual stage rather than during the embodiment design. Furthermore, proposals and recommendations of how design engineers should use design guidelines in a human-centric and user-specific manner, as well as how what kind of information should be represented, are still lacking. In addition, an international and global perspective, which involves multinational design engineers with different backgrounds and levels of experience, is rarely considered. This is, in turn, also a key motivation for further fundamental research on the benefit of design guidelines and their impact on design engineers’ performance during embodiment design; especially research on the benefit of those design guidelines developed by industry that are intended to support this specific design phase in particular.

2.3 Eye tracking in the field of engineering design

Besides the classical empirical research methods used for research in design for decades (Ahmed 2007), researchers are currently interested in finding methods from different fields that can be adopted in experimental design research (Lohmeyer & Meboldt 2016). Eye tracking, which was formerly adopted in the cognitive sciences (Rayner 1998; König et al. 2016) and found as a method that can act as a window to cognitive processes (König et al. 2016), is applied in multiple fields nowadays (Bojko 2013; Bol et al. 2016; Mussgnug et al. 2017) and, recently, has also been found as a potential research method in the field of engineering design. There exist a few studies that adopt eye tracking as a research method and highlight its potential, e.g. Boa, Hicks & Nassehi (2013), Lohmeyer, Meboldt & Matthiesen (2013), Matthiesen et al. (2013), Meboldt, Lohmeyer & Mussgnug (2013), Lohmeyer et al. (2014), Ognjanovic et al. (2014), Bi et al. (2015), Ruckpaul, Nelius & Matthiesen (2015), etc. For instance, Matthiesen et al. (2013) highlight the potential, though there are also various limitations of the method, cf. Holmqvist et al. (2011), Duchowski (2017), and propose the use of eye tracking technologies for studies on how design engineers build up understanding of technical systems. Meboldt et al. (2013) conclude that eye tracking data might allow the human-centred development and improvement of engineering design tools and methods. Thus, the findings and conclusions of the reported and reviewed studies declare eye tracking as a promising method to analyse human behaviour in engineering design and show the opportunity to uncover hidden and promising insights. Therefore, eye tracking may be a suitable method to explore the benefit of design guidelines and their impact on design engineers’ performance during embodiment design.

3 Materials and methods

The following sections describe the study and how we collected and subsequently evaluated the data, including all used materials and methods. The research reported in this paper was carried out in close collaboration with industry. Our research partner was an internationally and globally acting company that develops, designs and manufactures power tools. The study was conducted three times in total. The study was first conducted at the company’s site in Germany, cf. Reimlinger et al. (2019), then in Eastern Europe and Asia. The methods, the materials and the setup were the same. The materials we used were materials owned and provided by the company, except the device as well as the software to record and analyse the data.

3.1 Task description

An engineering design task had to be performed in the study. The goal of the task was to connect the housing parts of an original and existing product by means of screw joints. The task had to be performed using an MCAD system. During the task, the participants had access to a design guideline dealing with screw joints as well as to a white sheet of paper and a pen. Subtasks included were defining the number and position of screw joints and designing the screw joints to be implemented in the product.

The product was a formerly developed and designed laser ranger finder already launched in the market. The originally implemented and existing screw joints were removed, and some parts that were attached to the outer shape of the product were deleted. The deleted parts were battery cover, type label, keypad and protective glass. Those parts that were not deleted and remained were the housing parts, parts inside the housing parts and the battery contacts. The remaining housing parts were the top housing and the housing base and the remaining inner parts were printed circuit board assembly and optics carrier assembly.

The design guideline, to which the participants had access, was a printable 13-page pdf file containing knowledge and information on the design of screw joints. The design guideline was developed by the company a couple of years before. Knowledge and information was represented in terms of figures and tables as well as text. According to the space used within the design guideline, the original ratio of text to figures and tables equals 1.70. The design guideline was originally written in German. Due to the international research setting, the design guideline had to be translated for the participants from Eastern Europe and Asia. Only the participants in Eastern Europe had to deal with an English edition of the design guideline; all the other participants got the design guideline presented in their mother tongue.

3.2 Participants

A total of 47 design engineers participated in the study, thereof 17 from Germany (GE), 15 from Eastern Europe (EE) and another 15 from Asia (AS). All the participants were employed by the company during the period of time the study took place. The participant selection was based on the precondition that none of the participants has designed a laser range finder yet, though they were usually designing similar products, and that none of the participants was aware of how the housing parts were connected in the original and existing product prior to the study. The participants were divided into two groups, namely experts and novices. We declared design engineers with more than eight years of relevant working experience as experts and design engineers with less than eight years as novices, as commonly done in design research (Ahmed et al. 2003), and did not divide according to the 10 years mark like in other fields (Simon & Chase 1973; Hayes 1981; Ericsson, Krampe & Tesch-Römer 1993). Thus, 25 novice design engineers and 22 experienced design engineers participated in the study. Table 1 depicts the descriptive statistics of each group at each site.

Table 1. The participants’ characteristics

3.3 Procedure

The study is decomposed into four parts (i–iv), cf. Figure 1, which were carried out in chronological order. First, there was an opening interview. Second, the participants received a short briefing to be prepared for the engineering design task to be performed in the experiment. Third, the actual experiment was carried out. Finally, there was a closing interview. The study was conducted by one moderator who was present during each time slot. The moderator used a developed moderator’s guide to ensure the same procedure and information for each participant. Both the opening and closing interview in Germany were carried out in German and those in Eastern Europe and Asia in English. Merely in Asia, there were some participants who did not feel very comfortable with the English language. In this case, a colleague from the local engineering department who was quite familiar with the English language was asked to assist and to translate both the questions of the moderator and the answers of the participant.

Opening interview (i). The study’s first part was the semi-structured opening interview which was intended to gain the participants’ detailed characteristics. The characteristics the moderator asked the participants for were age, years of experience as a design engineer and years with the company.

Briefing (ii). The second part was the briefing part. The briefing part was carried out to provide background knowledge on the state of problem and to give information that directed and constrained the following task. The intention was to familiarise the participants with the task and the product, as the participants neither knew any details about the product nor were allowed to prepare themselves specifically for the study in advance. To conduct the briefing, the moderator used a sheet that depicted the basic requirements, the product architecture, the functions of the housing parts and an exploratory view of the product. The product’s exploratory view contained all components, including those that were not presented in the following experiment.

Figure 1. Experimental procedure consisting of four parts (i–iv).

Engineering design task (iii). The third part was the actual experiment in which the participant carried out the engineering design task. The whole experiment lasted 60 min. None of the participants, however, was aware of that time frame in order to avoid any time pressure. In any case, the moderator shortly intervened after 30 min and asked the participant to focus from this point forward on one single screw joint with all the details it should have. In case the participant has already started, he had to decide for one position and screw joint; in case the participant has not started yet, he had to choose a position and start. Once one screw joint has been finished and the design satisfied all requirements and functions from the participant’s point of view, the participant was asked to design a further screw joint.

Closing interview (iv). The fourth and last part was the semi-structured closing interview which was intended to wrap up shortly and to investigate how the participant experienced the recently conducted experiment. In addition, due to the time frame of 60 min, the participant was asked for the potential next steps.

3.4 Experimental setup

Figure 2 shows the experimental setup. At each site, i.e. in Germany, Eastern Europe and Asia, the experiment was set up identically in a separate room. To ensure the same conditions for each participant, the door and shutters were closed, lights were switched on. The experimental setup consisted of table, chair, docking station, notebook, two screens of same size, mouse, keyboard, a number of cables to connect the devices as well as a white sheet of paper and a pen. The screen on the right-hand side showed the simplified MCAD model in PTC Creo Parametric 2.0 and the screen on the left-hand side showed the presented design guideline on screw joints in Adobe Acrobat DC in Two Page View mode. The participant was sitting on the chair and equipped with mobile eye tracking glasses.

Figure 2. Same experimental setup at three different sites.

3.5 Data recording

Data recording was done by using SMI SensoMotoric Instruments Eye Tracking Glasses 2 Wireless with a sampling rate of 60 Hz binocular, in combination with the SMI ETG 2W Smart Recorder. The Smart Recorder was connected via cable to the Eye Tracking Glasses and placed on the table right next to the participant during the experiment. A three point calibration was run prior to the experiment. To do so, a simple slide showing three crosses on the left screen was used.

3.6 Data analysis and metrics

Semantic Gaze Mapping (Vansteenkiste et al. 2015) was used to analyse the eye tracking data. The analysis via Semantic Gaze Mapping was performed in BeGaze by SensoMotoric Instruments. During the analysis, the participant’s gaze was mapped from a scene video onto a reference view covered by the defined areas of interest (Tien et al. 2015; Erridge et al. 2018). The areas not covered with an area of interest are defined as whitespace (Holmqvist et al. 2011). Areas of interest, cf. Holmqvist et al. (2011), Duchowski (2017), were defined based on the three stimuli involved in the experiment. The stimuli are design guideline, MCAD system and sheet of paper. In case of the stimuli MCAD system and sheet of paper, we defined one area of interest each. The stimulus design guideline was decomposed into 10 areas of interest, according to the design guideline’s chapter structure, as chapters generally frame thematic areas. Table 2 lists and describes the defined areas of interest.

Table 2. Areas of interest definition and description

An analysis via Semantic Gaze Mapping provides various metrics. We used fixations visualised in terms of sequence charts and the dwell time to evaluate the eye tracking data. Further metrics used were the goal achievement rate and the benefit-to-effort ratio.

Sequence charts. Sequence charts relate fixations to areas of interest, illustrate these fixations over time and are commonly used to evaluate eye tracking data, e.g. Zammitto & Steiner (2014), Hareide & Ostnes (2017).

Dwell time. First, the dwell time (Holmqvist et al. 2011) was determined for each area of interest. Second, the dwell time was determined for the stimulus design guideline. Third, the dwell time was determined for the time spent on figures and tables, as well as for the time spent on text, in the design guideline. Throughout the paper, the dwell times are total dwell times.

Goal achievement rate. As each individual experiment had an output in terms of an MCAD model, these models were assessed. This assessment was based on a catalogue of criteria gathered through the closing interview carried out at the end of the experiment, through data of three independent expert reviews of the original, existing and launched product as well as through the aspects stated in the design guideline. This assessment defines the goal achievement rate. The goal achievement rate is a numerical value on a scale from 1 (very low) to 5 (very high).

Benefit-to-effort ratio. The benefit-to-effort ratio is defined by the participant’s personal and individual assessment of the design guideline’s benefit for this design activity. The benefit-to-effort ratio is a numerical value on a scale from 1 (very low) to 5 (very high).

For each participant, metrics were determined. The plotted sequence charts were analysed qualitatively and independently by eight raters. We calculated the interrater agreement by using Fleiss’s Kappa and referred to Landis & Koch (1977) to interpret the results. Further, statistical analysis was performed to either compare experts and novices, different sites and different groups across the sites or to find potential correlations. We first used a Kolmogorov–Smirnov test to check for normality of data and a Levene’s test to check for homogeneity of variances. In case normality of data as well as homogeneity of variances could be assumed, we carried out an Analysis of Variance. In the case of no normality of data, we used a Kruskal–Wallis test. We calculated, if the data was distributed normally, the Pearson correlation and, if the data was not distributed normally, the Spearman correlation. A $p$-value of less than 0.05 was considered statistically significant.

4 Results

The following sections are intended to present the results on the RQs stated in the very beginning. Starting with answering RQs 1-1, 1-2 and 1-3, we finally show the results on the overall guiding research question RQ 1-0. Here, we also point out the key aspects of RQs 1-1, 1-2 and 1-3 briefly, as those RQs contribute to answering RQ 1-0 as well.

4.1 The participants’ strategies to approach the task

This section presents, first, the results of the qualitative analysis of the sequence charts. Based on that, we identified three different patterns which are described here. Second, we show what kind of pattern was adopted by what kind of group. Third, we found significant effects of the pattern, as well as of the group, on the goal achievement rate at the site in GE.

The upper part of Figure 3 shows three plotted sequence charts of one single participant each. In general, the sequence charts contain information at which point of time (cf. abscissa; the duration of the experiment is 3600 s in total) someone is gazing at. More specifically, sequence charts give information on when and how often someone is fixating which area of interest (cf. ordinate). This information, these so-called fixations, is represented by short vertical lines, which correspond to the respective area of interest.

Figure 3. Three patterns based on the strategies the participants pursued.

There are areas within the sequence charts marked with different shapes; cf. Figure 3. The vertical line marked with a square represents that point of time (always at 1800 s) when the moderator intervened and gave the instruction to start with the synthesis of the screw joint. It may be that some participants started earlier, but there were also participants who started after the moderator’s intervention. That point of time when the participants actually started is represented by the vertical line marked with a circle. The phase marked with a triangle is defined by the participant’s first interaction with the design guideline as well as by the interaction’s duration. Whereas both the moderator’s intervention (square mark) and the start of synthesis (circle mark) are added information, the sequence charts show the information on the interaction and its duration (triangle mark) directly.

Each of the plotted sequence charts in Figure 3 represents a pattern. A pattern is defined by the strategy the participants pursued to approach the task to be accomplished in the experiment. The information that defines the strategy itself is, first, the first interaction and, second, the start of synthesis. We first started with analysing the sequence charts of GE before analysing those of EE and AS. The three patterns were found at each site and tested for reliability ($\unicode[STIX]{x1D705}_{GE}=0.82$; $\unicode[STIX]{x1D705}_{EE}=0.83$; $\unicode[STIX]{x1D705}_{AS}=0.86$).

Pattern 1 is defined by an analysis phase of the MCAD model in the beginning, followed by an interaction phase with the design guideline and, finally, the start of synthesis. Participants in pattern 2 start with interacting with the design guideline, continue with analysing the product and then start with the synthesis of the screw joints. Pattern 3 is characterised by an analysis phase of the MCAD model in the beginning, followed by the start of synthesis without having interacted with the design guideline at all before.

The lower part of Figure 3 contains information on what kind of group (experts or novices) in Germany, Eastern Europe and Asia adopted what kind of pattern. When calculating the percentages within each row, and thus within each pattern, 70% of the participants in pattern 1 are experts and 30% are novices. Referring to row 2, 37% of the participants who adopted pattern 2 are experts and 63% are novices. Within row 3, and therefore within pattern 3, 67% of the participants who adopted this pattern are experts and 33% are novices.

Further, when calculating the percentages within each site, 43% of the experts in GE adopted pattern 1, 43% pattern 2 and 14% pattern 3. In contrast to that, 20% of the novices in GE adopted pattern 1, 70% pattern 2 and 10% pattern 3. At the site in EE, pattern 1 is adopted by about 29%, pattern 2 by 43% and pattern 3 by about 29% of the experts. Whereas none of the novices in EE adopted pattern 1, 86% adopted pattern 2 and 14% pattern 3. Referring to the site in AS, about 13% of the experts adopted pattern 1, 75% pattern 2 and about 13% pattern 3. Pattern 1 is adopted by 14% of the novices in AS, pattern 2 by 86% and pattern 3 by none of the novices.

Overall, the calculated percentages of the groups across all sites show that 32% of the experts adopted pattern 1, 50% pattern 2 and 18% pattern 3. Whereas, pattern 1 was adopted by 13%, pattern 2 by 79% and pattern 3 by 8% of the novices.

When assessing the effect of the group on the goal achievement rate within each pattern as well as of the pattern on the group’s goal achievement rate, we find significant effects at the site in GE. That is, experts in pattern 1 performed significantly better than novices ($p=0.003$) and, further, novices who adopted pattern 2 performed significantly better than novices who adopted pattern 1 ($p=0.001$). Within EE and AS, respectively, we cannot find any significant effects (each $p>0.05$) either of the group on the goal achievement rate within each pattern or of the pattern on the group’s goal achievement rate.

Next, after assessing the effects within the sites separately, we investigate the differences among GE, EE and AS. There, no significant effect on the goal achievement rate was found (each $p>0.05$).

Further, we compare the separate groups of each site (experts versus novices) and look for potential effects. There is a significant effect amongst the novices, that is, novices in GE within pattern 2 achieved a significantly higher goal achievement rate than novices in EE ($p=0.043$). Amongst the experts, we cannot find a significant effect on the goal achievement rate (each $p>0.05$).

To answer RQ 1-1

Based on a qualitative analysis of the sequence charts, we found three patterns that represent three different strategies the participants pursued to approach the task. Thus, we found three different ways design engineers use design guidelines for designing single design elements. Novices prefer to adopt pattern 2, as 79% of the novices ($n_{\text{no}}=25$) adopted this pattern, i.e. they first interacted with the design guideline before analysing the product and starting with the synthesis of the screw joints. Experts do not show this strong preference, as amongst the experts ($n_{\text{ex}}=22$), pattern 2 was adopted by 50% of this group’s participants, followed by 32% who adopted pattern 1. But within pattern 1, there were 70% experts and 30% novices. Further, pattern 1 is a pattern where the experts at the site in GE performed significantly better than the novices there and, also, novices in GE were significantly more successful by adopting pattern 2. Starting with the synthesis without considering the design guideline in advance characterises pattern 3, which is adopted by 18% of the experts and 8% of the novices and, thus, by only a few design engineers of each group.

4.2 Information that catches the participants’ attention

This section consists of two parts. First, we show on what kind of information within the design guideline the participants spent the most of their time. Second, we describe the results of our analysis regarding the information’s representation.

4.2.1 Content-wise

Initial impressions on what kind of information the participants were focusing on during the experiment can be gained by the sequence charts like those presented in Figure 3. Sequence charts may be quite suitable for an initial qualitative analysis. However, we decided to conduct a quantitative analysis and used the dwell time to determine the areas of the participants’ interest and, thus, to figure out on what kind of information the participants spent the most of their time.

Figure 4 shows two plots which depict the dwell time in s (cf. abscissa) that is related to the respective area of interests (cf. ordinate). The plot on the left-hand side shows the experts’ and the one on the right-hand side shows the novices’ data. There are three bars related to each area of interest representing the data of one site each.

Figure 4 shows that both the experts’ and the novices’ attention is mainly dedicated to the areas of interest such as screws, screw bosses, assembly and design hints. Both experts and novices, no matter from which site they are, spent the most of their time on screw bosses, followed by screws, design hints and assembly. The dwell time of the remaining areas of interests – cover page, contents, general, literature and internet links – is below a mean value of 17.91 s (sd 15.14 s) for the experts and 24.93 s (sd 25.93 s) for the novices. In particular, literature, internet links and software are not of interest for the participants; cf. Figure 4. For evaluating the data statistically, we therefore focus on the areas of interest such as screws, screw bosses, assembly and design hints.

Assessing the effect of the group at each site on the dwell time within each area of interest, there is no significant effect (each $p>0.05$). Also, there are no significant effects of the site, i.e. experts and novices together, on the dwell time within each area of interest (each $p>0.05$), except within the area of interest assembly. Here, participants from AS spent significantly more time on this information than participants from GE ($p=0.030$). Furthermore, we compare the results within the groups across the sites. There, experts from AS spent significantly more time on the area of interest assembly as well ($p=0.028$). Apart from these significant effects, no further significant effects were found either amongst the experts or amongst the novices (each $p>0.05$).

Figure 4. Areas of the participants’ major interest.

To answer RQ 1-2

Information on which the participants focus the most is about how to dimension specific elements of the screw joints, no matter whether, first, the participant is an expert or a novice or, second, the participant is from GE, EE or AS. Not of interest are the areas of interest such as cover page, contents, general, literature, internet links and software.

4.2.2 Representation-wise

To analyse the way the information is represented, we analyse the dwell time for figures and tables as well as for text. The results are depicted in Figure 5, which shows two plots to separate the data of the experts, cf. plot on the left-hand side, from the data of the novices, cf. plot on the right-hand side, graphically. Each plot shows how the dwell time for figures and tables, as well as for text, is distributed at each site.

Figure 5. The way of representation that catches the participants’ attention.

When assessing the effect of the participants on the dwell time for figures and tables, as well as on the dwell time for text, we cannot find significant effects (each $p>0.05$) within the groups at each site, across the sites, amongst the experts across the sites or amongst the novices across the sites.

When assessing the effect of the way of representation on the dwell time for each group at each site separately, we find that all the experts, as well as novices in GE and EE, spent significantly more time on figures and tables than on text (each $p<0.05$). However, we cannot find this effect amongst the novices from AS ($p=0.064$).

To answer RQ 1-3

Design engineers focus significantly more on information that is represented in terms of figures and tables than information that is represented in terms of text. Here, merely the novices from Asia differ from the other participants. Those novices’ attention is not caught significantly more by information represented in terms of figures and tables.

4.3 The design guideline’s impact on the participants’ performance

This section describes how the participants performed and how the design guideline may impact their goal achievement rate. First, we present the results on the participants’ individual assessment of the design guideline’s benefit for this specific effort and investigate the effect on the goal achievement rate. Second, we show how the dwell time and the goal achievement rate correlate.

Figure 6. Goal achievement rate in relation to individual benefit-to-effort ratio.

Figure 6 shows three plots representing the same kind of data for each site separately. As such, each plot on its own shows, first, how the participants (note that each plot distinguishes between experts and novices as well; cf. Figure 6) assessed the design guideline’s benefit for this design activity (cf. abscissa) and, second, how their individual assessment is related to the goal achievement rate (cf. ordinate).

At the site in GE, cf. plot at the top of Figure 6, 43% of the experts assessed the benefit with low (2), 29% with medium (3), 14% with high (4) and another 14% with very high (5). Amongst the novices, 30% assessed the benefit with medium (3), 20% with high (4) and 50% with very high (5).

Considering the site in EE, cf. plot in the middle of Figure 6, the results show that 14% of the experts assessed the benefit with low (2), 14% with high (4) and about 72% with very high (5). With respect to the novices, 25% assessed the benefit with medium (3), 50% with high (4) and 25% with very high (5).

Whereas at the site in AS, cf. plot at the bottom of Figure 6, about 13% of the experts assessed the benefit with low (2), 25% with medium (3), 50% with high (4) and about 13% with very high (5). Amongst the novices, 43% assessed the benefit with high (4) and 57% with very high (5).

Comparing experts and novices at each site separately, there is a significant effect at the site in GE ($p=0.031$), as well as at the site in AS ($p=0.049$), of the group, i.e. the experience level, on the benefit-to-effort ratio. Those novices assessed the benefit-to-effort ratio significantly higher than those experts. However, we cannot find this effect at the site in EE ($p=0.203$). Investigating this kind of effect further, we cannot find significant effects (each $p>0.05$) across the sites, amongst the experts across the sites or amongst the novices across the sites.

When assessing the effect of the group on the goal achievement rate at each site, we find a significant effect at the site in GE ($p=0.033$). There, experts performed significantly better than the novices. This effect, however, cannot be found at the site in EE and AS (each $p>0.05$), respectively. But across the sites, we find that the participants in GE performed significantly better than those in EE ($p=0.013$). Also, the experts in GE achieved a significantly higher goal achievement rate than the experts in EE ($p=0.013$) and AS ($p=0.025$). Apart from the significant effects found, there are no further significant differences (each $p>0.05$) across the sites, amongst the experts across the sites or amongst the novices across the sites.

Figure 7. Goal achievement rate in relation to dwell time.

Furthermore, also solely at the site in GE, we find a significant correlation between the novices’ personal assessment of the benefit-to-effort ratio and the goal achievement rate ($r=0.664$; $p=0.036$). The higher they assessed the design guideline’s benefit, the better they performed. Here as well, we cannot find any further significant correlations either at the sites or within the groups (each $p>0.05$).

Figure 7 shows three scatter plots representing the same kind of data for each site separately as well. Each separate plot shows how the dwell time (cf. abscissa) is related to the participants’ (experts are marked with a square and novices with a triangle) goal achievement rate (cf. ordinate). Also, the respective regressions were computed and depicted.

The results show different dwell times for different sites and different groups. The time the experts spent on the design guideline ranges from 100 to 657 s in GE (mean 398 s; sd 189 s), from 96 to 1198 s in EE (mean 452 s; sd 396 s) and from 208 to 848 s in AS (mean 492 s; sd 241 s). In contrast to that, the novices in GE spent between 144 and 1168 s (mean 488 s; sd 367 s) on the design guideline, those in EE between 18 and 1355 s (mean 758 s; sd 515 s) and those in AS between 299 and 1363 s (mean 581 s; sd 412 s).

When assessing the effect of the participants on the dwell time, we cannot find significant effects (each $p>0.05$) at the sites, across the sites, amongst the experts across the sites or amongst the novices across the sites.

The computed regressions are slightly decreasing and increasing, respectively. We cannot find any significant correlation either within one single site or within the groups at each site (each $r$ and $\unicode[STIX]{x1D70C}$, respectively, with a $p>0.05$). Novices in GE tended to achieve better designs by interacting with the design guideline more intensively and frequently; whereas a more intense and frequent interaction with the design guideline tended to weaken the experts’ goal achievement rate. By taking also into account the sites in EE and AS, the results show, compared to the results in GE, some differences here. At the sites in EE and AS, experts tended to achieve better designs by interacting with the design guideline more intensively and frequently, like the novices in GE. In contrast, the novices’ goal achievement rate tended to decrease the more frequently and intensively they interacted with the design guideline, like the experts’ goal achievement rate in GE.

To answer RQ 1-0

We have to consider the different dimensions this research implies. First, we can see that there are findings that allow answering this research question for design engineers in general. Second, there are findings that force us to differentiate more precisely in regard to this research question.

The results show that the novices in GE and AS assessed the benefit-to-effort ratio significantly higher than the experts at those sites, not so in EE. We also see that those novices in GE who assessed the benefit higher even performed significantly better, not so in EE and AS. In addition, the novices in GE who interacted more intensively and frequently with the design guideline tended to achieve better designs, not so in EE and AS. Considering the overall impact, novice design engineers in GE tended to perform better by interacting more intensively and frequently with the design guideline. In contrast, the performance of novice design engineers in EE and AS tended to weaken with a more intense and frequent interaction with the design guideline; their goal achievement tended to go down with a longer dwell time. Further, we see that experts in GE performed significantly better than the novices there, not so in EE and AS. In addition, the results show that a more intense and frequent interaction with the design guideline tended to weaken the goal achievement rate of those experts in GE, not so in EE and AS. There, at the site in EE and AS, respectively, the experts’ goal achievement rate tended to increase when interacting more intensively and frequently with the design guideline.

We found, thus, that novices in EE and AS tended to benefit less from a more intense and frequent interaction with the design guideline, like experts in GE, but without achieving the same high level of goal achievement rate. In contrast, we found that experts in EE and AS tended to benefit more from interacting more intensively and frequently with the design guideline, like novices in GE. Therefore, interacting with the design guideline may impact design engineers’ performance in two different ways; either the performance tends to increase or to decrease, depending on the level of experience and the region the design engineers are from.

In addition, referring to RQ 1-1, we found that novices mainly adopted pattern 2, which also led them to a significantly higher goal achievement rate than in pattern 1 (at least in GE; only one novice design engineer from EE and AS adopted pattern 1), and therefore benefits them more. Thus, the way of how they strategically use the design guideline impacts their performance. Experts did not show a strong preference for a pattern. And since the pattern had no effect on the goal achievement rate in the case of the experts, both pattern 1 and pattern 2 may benefit them; but the way they strategically use the design guideline does not impact their performance. However, in fact, we found that pattern 1 is more an expert pattern (70% experts and 30% novices) and pattern 2 is more a novice pattern (37% experts and 67% novices).

Further, referring to RQ 1-2 and 1-3, the way the information is represented within the design guideline impacts design engineer’s attention, as information represented in terms of figures and tables, and thereof mainly information that shows how to dimension specific elements, caught the participants’ attention significantly more (except data obtained from the novices in AS did not show this significance) than information represented by written lines. Thus, design engineers are more likely to benefit from information that is represented in terms of figures and tables than in terms of text.

5 Discussion

This paper tackles the need to understand the design engineer and the task of engineering design more deeply, especially within the embodiment design phase of physical products. In doing so, the research reported here considers the dimensions of expertise, human behaviour and codified knowledge in terms of design guidelines and even goes one step further by including an international and global perspective. As such, a design study was developed in order to explore the benefit of design guidelines and their impact on multinational design engineers’ performance across three regions: Germany, Eastern Europe and Asia. The study was carried out with the same setup, the same materials and methods but with design engineers from different backgrounds, three times in total. The research’s major contributions and implications, as well as its limitations, are discussed in the following.

5.1 Contributions and implications

The work reported here provides a number of contributions and implications for academia, especially for the field of design research, and particularly engineering design research, industry and practitioners. By carrying out design research in times of internationalisation and globalisation, this work considers today’s common practice in the field of engineering design. The international and global setting this research implies, therefore, fits to industry’s needs but also re-shifts design research’s focus from managing and maintaining globally distributed teams, cf. Littler et al. (1995) and the reaffirmation by the multitude of literature arisen in the recent years, to the individual human being acting in this context.

This research identified three different patterns based on the strategies the participants pursued to approach the engineering design task to be accomplished in the study and, hence, contributes to a deeper understanding of how design engineers behave. Even further, by distinguishing between different levels of experience, as others do as well, cf. Ahmed et al. (2003), for instance, this work contributes in multiple ways to academia, industry and design practice. We were able to show that there are different behaviours depending on the level of experience, which allows, first, recommending researchers to consider this potential influence in their future research, second, sensitising industry for this potential factor and, third, making design engineers aware that their behaviour may affect their results, as found at the site in Germany.

By focusing on design engineers solely, the research reported in this paper implies two things: first, in contrast to Butenko et al. (2018), we focused on one specific target group who is, most likely, the addressee of design guidelines in an industrial context. We even extended the scope from one local engineering target group to multinational design engineers working for the same company at different sites in different regions but on similar products. Second, as some authors state that novice design engineers usually benefit more from design guidelines than experienced design engineers (Verein Deutscher Ingenieure 2004–01; Hoenow & Meißner 2016), although there is still little knowledge and a lack of evidence, the research here differentiates between experts and novices. In doing so, we found that, indeed, novices in Germany tended to achieve better designs here by interacting more intensively and frequently with the design guideline. However, those results have not been found in Eastern Europe and Asia since the novices’ goal achievement rate there tended to go down the longer they interacted with and focused on the design guideline. We were further able to show ‘how’ design engineers benefit and from ‘what’, on what kind of information do they focus and how this information is represented, which is a key contribution of this research. This also may allow approaching the issues of lacking specification for a uniform way of documentation (Butenko & Albers 2018) and the poor structure (Nowack 1997; Gouvinhas & Corbett 1999), as the research’s results show what exactly catches the participants’ attention more, which we could find through the application of mobile eye tracking as a research method. Yet, the findings are heavily based on the dwell time as the main metric.

Our research setting implies design engineers from different regions. The results of the studies here show that novices in GE tend to achieve better designs the more time they spent on the design guideline, like the experts in EE and AS do as well. However, the performance of novices in EE and AS tends to weaken with a more intense and frequent interaction with the design guideline here. Potentially, participants from GE might be more familiar with systematic and methodological approaches, as they are commonly taught at German institutions. One might argue that experts in EE and AS spent more time with the company, whose headquarters are in GE, than the novices there and therefore might be more used to using a design guideline. Thus, this might be a reason why they tend to benefit more from a longer and more frequent interaction with the design guideline, like the novices in GE do. Novices in EE and AS, in turn, tend to benefit less, as their performance tends to go down, and might first have to get used to using a design guideline before benefiting from the knowledge and information captured therein. However, such cultural influences, such as the educational background, were not investigated in depth; and since there were no significant correlations, the results here may indicate these differences between the groups, yet further research is required to thoroughly investigate the correlation between the goal achievement rate and the dwell time.

Some authors state that design guidelines are particularly useful in the conceptual stage of the design process (Edwards 1994; Nowack 1997) and that the usefulness tends to weaken during the embodiment design phase (Edwards 1994). The research reported here approaches exactly this challenging (Albers & Matthiesen 2002) and less clarified (Hoenow & Meißner 2016; Matthiesen 2016) embodiment design phase, focuses on design guidelines specifically developed to support the embodiment design and explores the benefit of such a design guideline, which was originally developed by industry, in this particular phase. We could find that 70% of the novices in Germany, 75% in Eastern Europe and 100% in Asia assessed the benefit-to-effort ratio for this design activity with high or very high, as well as 86% of the experts in Eastern Europe and 63% in Asia (except in Germany, there is a lower percentage of 28%). Thus, on the one hand, we show that the majority of the participants highly appreciated the design guideline in the embodiment design phase and, on the other hand, together with the findings that uncover the information and its representation the participants primarily focus on, we show the potential for further improvement, which is in total a key contribution of this research.

Based on the results, this research contributes to the further development and improvement of design guidelines in a human-centric and user-specific manner tailored exactly to the needs of multinational design engineers located at different sites. Especially the kind of information and its representation are one of the key findings that may heavily contribute to the development of more beneficial and user-centred types of design guidelines. This work may thus lay a proper foundation to formulate hypotheses on how to further increase the benefit of engineering design guidelines in the embodiment design phase of physical products. The findings may suggest, for instance, that reducing information to a certain extent, e.g. to those information the participants were mainly focusing on, and that representing the information mainly in terms of figures and tables might increase the design guideline’s benefit as well as improve the design engineer’s performance. When revising current design guidelines that way, one may argue that this not only clearly presents the most relevant information in a suitable manner but also sharpens the scope and refines the purpose of a design guideline. However, whether these revisions might increase the benefit of a design guideline and create a positive impact on design engineers’ performance, like it is intended, needs to be thoroughly assessed.

5.2 Limitations

The first limitation to mention here is that the participants had sole access to the presented design guideline. Although this research was carried out in an industrial environment, this might not be the case in the design engineer’s daily work and, thus, limits this research. However, further input, like embodiment design documentations of predecessor or reference products, which is according to Matthiesen (2019), besides experiences with similar products, the prime input to the embodiment design phase, as well as various other sources of information, would potentially lead to information-seeking activities in various sources, which was beyond the scope of this research.

The research reported in this paper is, furthermore, limited to single engineering design elements. As such, we used the example of screw joints to explore the benefit of design guidelines within our studies. There might be more comprehensive guidelines, such as design guidelines for the design of plastics parts in general, e.g. E.I. du Pont de Nemours and Company (2000), BASF Corporation (2007a), LANXESS Corporation (2007), etc. However, designing single engineering design elements, no matter what kind of material or production technique used and involved, may be part of the professional proficiency of design engineers, no matter whether locally or globally, and is one of the basic tasks in engineering design. Due to the fundamental nature of single engineering design elements in engineering design, as well as the need for more fundamental research in regard to design guidelines, cf. Edwards (1994), for instance, exploring the benefit of such comprehensive design guidelines was out of scope.

The work reported here was carried out in collaboration with one single company and is further limited by the number of participants and by the number as well as the regions of the sites included in this research. In total, 47 design engineers participated in the study, thereof 22 experts (7 in Germany, 7 in Eastern Europe and 8 in Asia) and 25 novices (10 in Germany, 8 in Eastern Europe and 7 in Asia). All the participants were employed by the company during the period of time each study in the respective region took place. Although this study involves design engineers with different backgrounds, there might be further regions that are of interest but not considered here. Butenko et al. (2018), for instance, considered participants from North America but without focusing on one specific target group. And even though our study focused on one specific target group and comprised design engineers with various backgrounds, cultural influences were not investigated in depth, which further limits our findings.

We also would like to draw attention to the fact that there were some language issues that potentially limit this research. First, the design guideline, originally written in German, had to be translated for the purpose of the study. The participants in Germany and Asia got the design guideline presented in their mother tongue. Solely, the participants in Eastern Europe had to deal with the design guideline as well as with the interview parts in English. However, the engineering department in Eastern Europe is headed by an expatriate which forces the local design engineers there to communicate in English on a daily basis so that they are quite used to using the English language and therefore are familiar with technical vocabulary as well. Second, as there were some interviews in Asia accompanied by a local translator, a colleague from the same department, some information may have been lost. However, in case there were any issues, the questions and answers, respectively, were repeated and translated again to ensure high quality of data.

6 Conclusion

Design guidelines are a means of sharing and reusing valuable design knowledge. A study was developed and presented in order to explore the benefit of design guidelines and their impact on design engineers’ performance, especially in the embodiment design phase of the design process. The study was carried out three times, each time in a different region; a total of 47 differently experienced design engineers from Germany, Eastern Europe and Asia participated in the studies, resulting in a comprehensive set of data. Based on the results, we conclude that the improvement and development of design guidelines that are intended to provide humans with knowledge and information during embodiment design is needed to suitably support design engineers. However, we propose to rethink current engineering design guidelines both content-wise and representation-wise, heading for a more user-centred approach that considers the design engineer’s point of view.