How do designers generate new ideas? Design heuristics across two disciplines

Abstract

Research supports the central role cognitive strategies can play in successful concept generation by individual designers. Design heuristics have been shown to facilitate the creation of new design concepts in the early, conceptual stage of the design process, as well as throughout the development of ideas. However, we know relatively little about their use in differing disciplines. This study examined evidence of design heuristic use in a protocol study with 12 mechanical engineers and 12 industrial designers who worked individually to develop multiple concepts. The open-ended design problem was for a novel product, and the designers’ sketches and comments were recorded as they worked on the problem for 25 min and in a retrospective interview. The results showed frequent use of design heuristics in both disciplines and a significant relationship to the rated creativity of the concepts. Though industrial designers used more heuristics in their concepts, there was a high degree of similarity in heuristic use. Some differences between design disciplines were observed in the choice of design heuristics, where industrial designers showed a greater emphasis on user experience, environmental contexts, and added features. These findings demonstrate the prevalence of design heuristics in individual concept generation and their effectiveness in generating creative concepts, across two design domains.

1. Introduction

How do designers create new concepts? When faced with the task of generating a new design for a novel or existing product, designers find a wide variety of ways to think of new ideas. The ability to take a problem and generate multiple, varied solutions that can lead to new, creative outcomes is often referred to as the concept generation or ideation stage of design (Simon 1969). Finke, Ward & Smith (1992) divided ideation strategies into generative (e.g., analogical transfer and association) (Lloyd & Scott 1994) and exploratory (e.g., context shifting and hypothesis testing). Perhaps the most ubiquitous method is called ‘brainstorming’, a group process that involves suspending evaluation and generating as many different ideas as possible (Osborn 1957). Variants of this method, including ‘brainwriting’ (Paulus & Yang 2000), focus on developing a large quantity of ideas.

The ideation stage promotes creativity when many different designs are generated for later evaluation (Christiaans & Dorst 1992). Fluency has long been used as a measure of creative ability (Clark & Mirels 1970; Torrance 1972), and generating more ideas will logically allow the selection of a better idea from the set. But at times, continuing to generate a diverse set of ideas may prove challenging. One reason is that designers can become ‘fixated’ (Jansson & Smith 1991; Purcell & Gero 1996), where their attention is focused on a single past example or on one new idea. As a result, other possible designs are never generated, limiting the diversity and therefore the quality of ideas generated, and opportunities for novelty and innovation are missed (Ullman, Dietterich & Stauffer 1988; Ball, Evans & Dennis 1994).

To assist designers in ideation, a variety of ideation tools exist (c.f. Fogler & Le Blanc 1995). These range from more complex approaches like morphological analysis (Pahl & Beitz 1996) to simple checklists (Osborn 1957). ‘Concept mapping’ helps by identifying links between concepts relevant to a design (Plotnick 1997). Synectics (Gordon 1961) provides a series of ‘idea triggers’ to help designers generate new concepts. The SCAMPER technique (Eberle 1995) guides with questions such as, ‘What else can this be used for?’ Another tool based in engineering, the theory of inventive problem solving (TRIZ) (Altshuller 1984), provides a system to modify solutions by solving contradictions or tradeoffs based on past product patents.

Designers naturally generate ideas also without tools (Purcell & Gero 1996); these natural approaches are developed based on designers’ experiences and preferences for problem solving (Kirton 2004). However, it can be difficult for designers to describe their own cognitive thought processes (Daly et al. 2010), which may occur largely unconsciously (Nisbett & Wilson 1977). For this reason, think-aloud protocols (Ericsson & Simon 1993) are used to study designers as they speak their thoughts while working through a design problem. Protocols have been shown to be effective in understanding designers’ thoughts without interfering with their natural thinking processes (Atman & Bursic 1998).

Many empirical studies have investigated the cognitive processes of individuals during the design process (Adams & Atman 1999; Christiaans & Dorst 1992; Dorst & Cross 2001; Stauffer & Ullman 1988; Dahl & Moreau 2002; Kavakli & Gero 2002; Okudan, Ogot & Shirwaiker 2006; Hernandez, Shah & Smith 2010; Lawson 1980). Important features about the cognitive processes within design have been discovered, such as the observation of unexpected discoveries (Suwa, Gero & Purcell 2000), ‘novel design discoveries’ (Akin & Lin 1995), and opportunism in design (Guindon 1990; Ball & Ormerod 1995).

This paper gives a cognitive account of generalized patterns observed in designers’ concepts. The outcomes of a think-aloud protocol study are presented to explore how design heuristics are used by designers from two disciplines of conceptual design: mechanical engineering and industrial design. We also investigated (1) how designers used design heuristics in generating multiple candidate designs, (2) how the use of heuristics impacted the variation among concepts and the creativity of concepts generated by designers, and (3) how the two groups of designers differed, if at all, in their use of design heuristics.

1.1. Design heuristics

‘Design heuristics’ are defined as cognitive ‘shortcuts’ that point toward useful design patterns (Daly et al. 2012c ; Yilmaz, Seifert & Gonzalez 2010; Yilmaz & Seifert 2011). ‘Heuristic’ commonly refers to strategies that use readily accessible information to control problem-solving processes (Pearl 1984), and has been shown to be advantageous in applied problems (Gilli, Maringer & Winker 2008; Goslar 1993). Cognitive heuristics help individuals arrive at a solution (Nisbett & Ross 1980) through readily accessible information in memory in order to control and direct problem-solving processes. Importantly, cognitive heuristics are not guaranteed to produce a successful solution, but can help to quickly identify possible solutions.

Behavioral research in domains like firefighting found that experts use cognitive heuristics automatically when facing new problems, and that heuristics can lead to fast, effective solutions (Klein 1998). Designers have also been shown to follow trains of thought to arrive at partial solutions with little cognitive cost (Guindon 1990). For example, a designer may display opportunism (Ball & Ormerod 1995) by combining the energy source for two different functions – reading and rocking – into a rocking chair that powers a reading light (see Figure 1). Once this concept has occurred to the designer, does she then look for other instances of the same type of opportunity? From this experience, the designer may learn that opportunities may come in the form of two functions that occur together, and so afford the possibility of merging their energy source (Yilmaz & Seifert 2010). This conclusion goes beyond observing cognitive processes at a general level, such as noting that an opportunity occurred; instead, the specific type of idea generated – ‘use the same energy source’ – may be useful to try again when functions co-occur.

Figure 1. Novague, a Prague design studio, has proposed a rocking-chair concept that converts rocking motion to power a reading light (from http://www.tuvie.com/novague-rocking-chair-generates-energy-to-light-a-led-lamp/ ).

Design heuristics are specific content patterns reflecting the cognitive strategies used to create new concepts (Daly et al. 2012c ; Yilmaz & Seifert 2011, 2010). In one study, an experienced industrial designer working on a universal access bathroom generated a concept where the sink, toilet, and shower share a single water source (see Figure 2). The design heuristic identified was ‘Use a common element for multiple functions’; specifically, its content suggests organizing functions around a common source. This heuristic can be applied in other design problems, such as a charging station for different devices (Figure 3 a & b).

Figure 2. A design concept (a) using a central water source for aligning sink, toilet, and shower components, and in contrast, a different heuristic is evident in the second concept (b) showing a railing system for aligning components. Credit: Allen Samuels, Industrial Designer.

Figure 3. A design similar to that in Figure 2 a involves a central base for diverse components in (a) the multiple charging station by Weiku (http://callpod.com/products/chargepod) and (b) a cooler for different beverages by Bryden Road (http://www.winevine-imports.com/noble-four-bottle-sealed-compartment-cooler/ ). These products also use a common energy source (electricity and ice) for components with differing functions.

Systematic comparisons of the over 200 concepts generated by this designer over a period of 2 years (Yilmaz & Seifert 2011), along with a content analysis of over 400 award-winning consumer products (Yilmaz & Seifert 2010), suggested a variety of cognitive strategies like this one. Design heuristics have been identified through a series of empirical studies that captured the content of the strategies appearing in varied design concepts (Daly et al. 2012c ; Yilmaz & Seifert 2011, 2009). In other studies, they have been demonstrated to serve as a useful design tool when provided to engineering students (Christian et al. 2012; Daly et al. 2011, 2012b , c ; Kramer et al. 2014) and professional design engineers (Yilmaz et al. 2013a ).

1.2. Design across disciplines

As design is a common activity across multiple disciplines (Cross 2004; Daly, Adams & Bodner 2012a ; Goel & Pirolli 1992; Zimring & Craig 2001), one question about design heuristics in ideation is how designers from different disciplines use them. The identification of design heuristics occurred in product design settings with engineers and industrial designers (Daly et al. 2012c ; Yilmaz & Seifert 2011, 2010), and in several studies design heuristics were provided as a tool to aid student designers in both engineering (Christian et al. 2012; Daly et al. 2012b ; Kramer et al. 2014) and industrial design (Yilmaz et al. 2012), showing the successful use of design heuristics in both groups (Yilmaz et al. 2013b , 2010a ). However, our work has not directly compared whether there are differences in how both groups use design heuristics as a natural cognitive strategy.

The utility observed for design heuristics as a tool for both industrial designers and mechanical engineers is perhaps not surprising. There is considerable overlap in the type of design work performed by engineers and industrial designers, and Cross (2004) discussed similarities in creative strategies across domains. Both are often called upon to create by ‘designing pieces of technology and initiating change in man-made things’ (Jones 1970). However, designers from engineering and industrial design may differ substantially in their design processes and outcomes, perhaps as a consequence of substantial differences in their education and training. Differences in their cognitive processes for generating new concepts might then be expected.

Engineering has been characterized as combining scientific discoveries with principles targeted at developing useful products for bettering human life (Pahl & Beitz 1996). Emphasis is often placed on improving function by solving technical problems. Because ‘real-world’ engineering and operations call for critical judgment and creativity (Felder et al. 2000), many engineering programs include opportunities for students to develop their creativity in the context of design (Court 1998; Daly, Mosysjowski & Seifert 2014). However, engineering students have reported creative thinking to be less natural for them than technical thinking (Court 1998). As a result, it can be challenging for engineers to generate diverse creative concepts (Christiaans & Venselaar 2005).

Industrial designers are trained through repeated experience in generating design concepts, with instruction through critique sessions led by instructors or professional designers (Barrett 2000; Uluoglu 2000). Industrial design intersects several disciplines, including engineering, ergonomics, business, aesthetics, and social, environmental, and cultural issues (Tovey 1997). For example, form selection in industrial design is driven by aesthetic, creative, and emotional values in addition to functional considerations (Gotzsch 1999). However, despite an emphasis on creative explorations, industrial designers have been shown to experience limitations when attempting to generate diverse concepts (Bruseberg & McDonagh-Philp 2002).

Do these differences in design training or practice influence the ways in which designers in the two fields generate ideas? In a study comparing industrial design and engineering novices, students were presented with design problems along with example solutions (Purcell & Gero 1996). The findings showed that engineering students were affected when initial example solutions were presented, but industrial design students were not. Industrial design students were better able to avoid fixation while creating their own design solutions.

On the other hand, Goel & Pirolli (1992) laid out a case for common, invariant features in design tasks across domains, including architecture, mechanical engineering, and instructional design. Industrial design and engineering students are challenged to move beyond technical aspects of the problem (Stouffer, Russell & Olivia 2004) and to develop creative outcomes through non-linear and flexible approaches (Pappas 2002). Both fields make use of the experiential learning approach (Tynjälä 1998), as project-based courses in engineering include design teaming (Dym 1994), external clients (Little & Cardenas 2001), and integrating areas like graphics into their designs (Ogot & Kremer 2004).

Within both domains, there is little specific pedagogy on how to generate concepts and limited assessment of successful concept generation (Daly et al. 2014; Dym et al. 2006). Pedagogy for enhancing design creativity is essential because most engineering and industrial design problems demand innovative approaches to the design of products, equipment, and systems. This demand arises from continual changes in the market, new technologies, new legislation, and sustainability concerns (Basadur & Finkbeiner 1985). The result of design activity is often expected to be original, adding value to the base of existing designs by solving technical problems in new ways. Studies of designers in both domains may lead to a more informed understanding of how ideation can be facilitated, and of building expertise in design (Cross 2003).

2. Aims

The goals of this research were to examine evidence of design heuristic use in concept generation through a controlled laboratory study of both engineering and industrial designers. We sought to identify (1) how designers generated multiple candidate designs, (2) how their cognitive processes led to variations in concepts and creative outcomes, and (3) how the two groups of designers differed, if at all, in their idea generation processes (i.e., examining both similarities and differences).

Specifically, the hypotheses tested in this study were as follows:

1. H1 Designers show evidence of design heuristic use in their concepts.

2. H2 Use of design heuristics is related to creativity and diversity in the designs generated.

3. H3 Design heuristic use differs among designers based on their field of training.

We hypothesized there would be evidence of heuristic strategies in both engineers’ and industrial designers’ concepts, and that the application of design heuristics during idea generation would support exploration of more potential designs. We expected participants in industrial design to generate concepts focused on aesthetics, emotion, and user engagement, and engineers to focus on function and technical product features.

3. Significance

The importance of the study lies in the discovery of the types of cognitive processes used in concept generation in design. The ability to generate more creative and diverse concepts may result in the selection of more innovative designs. By examining design performance in two disciplines (industrial design and mechanical engineering), it is possible to determine the commonalities in the use of cognitive heuristics across fields. If design heuristics are uncovered in the designs created by both engineering and industrial designers, this may lead to a better understanding of how design knowledge is utilized by experts. In addition, it would suggest the importance of training novice designers in both fields to make use of these cognitive strategies in creating concepts.

4. Method

It is often difficult for designers to describe their own design processes. Intuitive processes are often difficult to articulate and analyze, and occur largely unconsciously (Nisbett & Wilson 1977). For this reason, concurrent think-aloud protocols (Ericsson & Simon 1993) are used to identify the cognitive strategies occurring in this study. In this method, designers are given information about a design problem, and are asked to speak aloud to explain their choices and thoughts as they work through their design process. Protocols have been used to explore design in a variety of studies (e.g., Atman & Bursic 1998; Akin & Lin 1995; Christiaans & Dorst 1992; Suwa & Tversky 1997), and have been shown to be effective in understanding designers’ thoughts without interfering with their natural thinking processes (Atman & Bursic 1998).

In this study, participants were also asked to elaborate on their concepts in a retrospective interview at the end of the session. During the interview, participants were asked to define each concept in their drawn sketches, and discuss each concept. Retrospective interviews have been used in studies analyzing expert designers’ concept generation process (e.g., Prats et al. 2009). In this study, the retrospective interview assisted in understanding the concepts generated and the strategies consciously applied.

4.1. Participants

Twenty-four participants were recruited from professional conferences and at a Midwestern university, twelve from each design field. We balanced gender (six females and six males in both groups) and expertise between the two groups (six undergraduates, four graduates, and two practitioners); see Figure 4. The professional mechanical engineers and industrial designers each had over 5 years of full-time design experience in their fields.

Figure 4. Participating designers’ age, gender, and experience.

4.2. Materials

The main criterion in developing a design problem was to structure a novel task where designers would not be biased by existing solutions. Additionally, the problem needed to be an open-ended design problem with many potential solutions as well as one that would not require complex technical knowledge and could be addressed in a short design session. The problem selected was to design a ‘solar-powered cooking device that was inexpensive, portable, and suitable for family use.’

4.3. Procedure

Participants enrolled in the study individually and worked alone on the task. They were asked to generate multiple concepts for the design problem, and to talk aloud as they went through the idea generation task while writing notes and drawing sketches. They used an electronic pen that recorded both their voices and drawings simultaneously throughout the study session.

Participants were given 25 min for the concept generation task. This brief task occurred in a one-time session due to the practical limitations involved in sampling expert designers across fields. This was not intended to reflect a typical assignment for designers, who typically consider a problem over a much longer period of time, and often work as part of a design team. However, as a sample of the thinking that takes place when initially considering a new design problem, the session length was sufficient to allow the emergence of some evidence about the initial stage of design.

In pilot testing, participants sometimes expressed the need for added technical specifications for the design problem. Consequently, in the study, the experimenter provided additional information about transferring solar energy into thermal energy by using reflection or absorption, or by trapping heat. Providing this information encouraged participants to focus on conceptual solutions.

Throughout the session, the experimenter asked the participants to keep talking if they became silent at any point.

Following the design task, retrospective interviews were conducted for approximately 5 min (terminated by each participant). While reviewing their sketches in sequence, participants were asked to identify each concept, describe what they recalled about each concept, and if possible, suggest how they conceived of it. Finally, they provided demographic information and rated their own performance in the study.

4.4. Analysis

Verbal data from the experimental sessions were transcribed to supplement the visual sketching data. The goal of the analysis for H1 was to characterize the various decision patterns evident in participants’ performance on the task. Thus, the analysis included identifying each concept generated as a separate idea, categorizing characteristics of the solution concepts generated, and determining the number of concepts and specific design heuristic(s) evident in the concepts. These features were coded for each concept, between concepts, and over the experimental session.

Both Torrance (1972) and Shah, Vargas-Hernandez & Smith (2003) have decomposed the creativity of a concept into individual, measurable components that are based on coded concept features. Thus, in our analysis, major elements and key features of the concepts were identified, and categories were created. For H2, the results compiled include a comparison of total number, and creativity and diversity of the concepts in each designer’s protocol. We also employed both statistical comparison and in-depth qualitative analysis to understand the potential similarities and differences between the engineering and industrial design samples. Given the relatively small sample size, the statistical analyses are powered to detect large effects, but the conclusions are limited in their generalizability.

5. Results

The number of concepts generated by each individual was defined through information from participants and their sketches as they indicated the beginning and ending to a given concept during the debriefing session. Then, each concept was categorized according to the type of solution depicted. These included the means used to direct sunlight, methods of obtaining energy, methods of maintaining heat, methods of cooking, the environmental context, ways to make it portable, form selection, and user interfaces. For example, some solutions directed sunlight using mirrors, some maintained heat by creating a closed product with a clear lid (so sunlight could get in), and some included straps so that the product could be attached to the user. Other solutions used a magnifying glass to direct sunlight, an insulated box to maintain heat, or a foldable container for easy transport. We used this categorization of solution types to assess originality of concepts as well as diversity of an individual’s set of ideas. Major elements and key features of the concepts were identified in terms of functionality, form, and user interaction (see Table 1).

Table 1. Solution characteristics for the solar-powered cooker problem and the total number of concepts observed in each category from both participant groups

From these categories, it is clear that a variety of concepts were generated across the 24 designer protocols. The concepts included a diverse sample of materials, methods, and features for the solar cooker. As a novel product, designers appeared to make use of knowledge of related cookware and basic features of heat production from sunlight. The designers seem to be able to successfully generate concepts despite the relatively short amount of time provided. The concepts generated addressed a wide range of solutions for the solar cooker problem.

5.1. H1: Evidence of design heuristic use

In the majority of the cases, the participants did not explicitly articulate their cognitive processes as they worked; however, use of design heuristics was very evident in the protocols. For example, in one protocol, the designer remarked, ‘I’ll use both a magnifying glass and a mirror, since I’m not sure if the energy will be enough to cook the food.’ This was coded as an indication of the design heuristic: Using multiple components to achieve one function. The designers’ sketches also provided evidence of design heuristic use within the specifications of products, the contexts of product use in sketches, and in the relationship of concepts in a sequence. Thus, both verbal and visual (sketched) data were considered for evidence of heuristic use.

The number of design heuristics evident within each concept was coded. This began with a base set of design heuristics from previous studies (Yilmaz & Seifert 2010, 2011). The coders, one with a background in industrial design and the other in engineering design, also added new design heuristics to better describe the concepts apparent in the protocols. The coders reviewed the entire dataset independently and then resolved any disagreement through discussion. Initial inter-rater agreement was 80% across the protocols.

Figure 5. The frequency of concepts with a given number of design heuristics observed within each.

Of the 129 separate concepts generated, only 3 (2%) showed no evidence of design heuristic use. This high proportion of concepts where design heuristics are apparent suggests the approach successfully characterizes important aspects of the designers’ concepts. The average number of design heuristics per concept was 5.1, and ranged from 0 to 15. Figure 5 shows that the mode was four heuristics per concept (19 concepts), and 58 of the 129 concepts (45%) exhibited between three and five heuristics. Fifteen concepts (12%) exhibited nine or more heuristics per concept.

The mean number of design heuristics per concept for the engineers was 5.37 and the mean for the industrial designers was 4.82. This difference was not statistically significant either by two-sample $t$ test or a random effects model that took into account multiple concepts per participant. Participants not only used design heuristics in their concept generation, but most frequently, used multiple heuristics within each concept.

For both engineers and industrial designers, one of the most commonly applied design heuristics was Attach independent functional components, where several different parts or systems with distinct functions are combined in a single device. Using this heuristic, some designers in both groups appeared to define each function independently, assign a form to each, and add a connection between the parts to create a concept. For example, as shown in Figure 6 a, an engineering designer attached the handle to the pot and the light-focusing lens, and in Figure 6 b, an industrial designer attached a continuous mirror inside the pot.

Figure 6. Examples using the design heuristic: Attach components that have different functions.

The other most common design heuristics for both groups were as follows: Cover/Form Shell/Wrap surface for another use, Elevate or lower product base, and Repeat a component; all were used more than 50 times. The least frequently observed design heuristics were the following: Design user activities to unite as a community, Include users in customizing the product, Texturize surface, Expose/Uncover internal components, Substitute an existing component with a new design, Use a common base to hold multiple components, and Adjust functions to needs of differing demographics; each was used only once. These differences appear to arise from the applicability of each design heuristic to specific functions within the solar cooker design problem. Thus, the content of the problem seemed to impact heuristic use.

The protocols replicated evidence of most of the design heuristics previously identified (Yilmaz & Seifert 2010, 2011). In previous studies, over 70 different heuristics were observed (Daly et al. 2012c ), but only 53 heuristics were observed in this study. This is likely due to the smaller number of participants and single-problem context in this study. For example, in order to apply the heuristic Use packaging as a functional component within the product, an existing concept is required.

In sum, heuristics were identified 659 times in the 24 individual protocols. The total number of heuristics per concept ranged from 1 to 15 for industrial designers and 0 to 12 for engineers. Both groups of participants used multiple heuristics within a single concept (64 concepts for industrial designers and 61 for engineers). Concepts without any heuristics, or with only one heuristic, were either very simple solutions (i.e. a plate capturing sunlight), were vague and undefined, or were redrawings of existing products. Supporting Hypothesis 1, designers showed evidence of design heuristic use in their concepts.

5.2. H2: Relationship of design heuristic use to creativity and concept diversity

To examine the relationship between design heuristics and the creativity of concepts, we used the consensual assessment technique (CAT) (Amabile 1982). Two coders blind to the heuristic analysis (different from those who coded heuristics) with backgrounds in industrial design and mechanical engineering rated each concept from all participants on a scale from 1 (not creative) to 5 (very creative). The Pearson correlation between the two coders was $r=0.87$ (an acceptable level of reliability), so their creativity ratings were averaged. For each participant, the creativity score across all of their concepts was determined, along with the average number of heuristics used in each concept (Table 2). There was no significant difference between the creativity ratings of concepts by industrial and engineering designers.

Table 2. Average diversity and creativity ratings, concept counts, average number of concepts, design heuristic counts, and average number of design heuristics in concepts generated by individual designers

A simple count of the number of different concepts generated by each individual is a measure of fluency commonly used in tests of creativity (e.g., Torrance 1972). However, more concepts did not appear to be an indicator of more creative concepts in this study. An individual’s total number of concepts and their creativity ratings (averaged over their set of concepts) were not statistically related, $r=0.10$ , n.s. This may be due to the relatively small number of concepts generated (averaging less than 6) in this task paradigm; other paradigms with longer work time may result in generating tens of concepts, as in the Torrance (1972) tests.

Industrial designers generated more concepts on average than engineers, but the difference was not significant ( $t<1$ ). Similarly, the industrial designers used more heuristics per concept than the engineers, but this difference was also not significant ( $t=1.1$ , n.s.).

However, the Pearson correlation between the average creativity scores and the number of design heuristics coded for each concept was $r(127)=0.51$ , $p<0.001$ . (The Spearman correlation was also 0.51, so we report Pearson correlations throughout.) We also conducted a multilevel regression to address non-independence due to subjects contributing multiple concepts to the analysis. Using the number of heuristics as a predictor of creativity rating, the relationship remains highly significant, $p<0.0001$ . Correlations of similar magnitude are observed within each of the two fields (industrial designers: $r=0.51$ ; engineers: $r=0.54$ ). This indicates a substantial, positive relationship between the use of heuristics within a concept and the separately rated creativity of concepts. It was the presence of heuristics, and not simply the number of concepts generated, that was found to relate to higher creativity ratings.

The diversity of the concepts generated by each designer was analyzed as a set for each participant. The coders again used a five-point Likert scale ( $1=\text{not diverse}$ , $5=\text{very diverse}$ ) to rate the set generated by each participant. The Pearson correlation between the two raters on diversity was $r=0.66$ , which is a relatively low reliability; to improve consistency, we averaged the coder’s diversity ratings. We also applied relevant structural equation models to take into account the relatively low reliability of the diversity ratings. These models led to the same decisions with respect to statistical significance, so we report the analyses using the simpler averaged diversity rating scores. No statistically significant differences were observed between the diversity ratings of industrial designers compared to the engineering designers (see Table 2).

Considering designers’ concept sets, neither diversity nor creativity measures were strongly related to the number of concepts in the set ( $r=-0.10$ and 0.16, respectively, n.s.). Perhaps surprisingly, generating more concepts was not related to the higher creativity or diversity of ideas. However, creativity and diversity (when entered into a multiple regression as two additive predictors) were both significantly associated with the number of design heuristics used in the concept set (creativity: $beta=2.75$ , $t(21)=4.41$ , $p=0.0002$ and diversity: $beta=-1.15$ , $t(21)=-3.48$ , $p=0.002$ ).

Comparing these scores across participants demonstrated the same positive relationship between creativity ratings and the use of design heuristics seen for the concept level analysis, $r=0.55$ , $p<0.005$ . The opposite trend occurred between concept set diversity and the use of design heuristics ( $r=-0.38$ , $p<0.063$ , marginally significant). Using design heuristics was negatively related to concept set diversity ratings. This pattern may have occurred because individuals who used multiple heuristics in generating concepts may have developed an idea through subsequent concepts, resulting in a set that shared common features and so received lower diversity scores.

The correlation between average creativity scores and average diversity scores for each participant was 0.20, $p=0.35$ . This suggests diversity of concepts and their creativity reflect different qualities of the designs, as shown by the differences in their relationships to the use of design heuristics.

In sum, ratings of the diversity within a concept set were marginally related to fewer heuristics used (on average) within the set. Considering the concepts as a set and independently, perceived creativity was related to more frequent use of design heuristics in concept generation. When a concept included more heuristics, the subjective creativity rating tended to be higher. Thus, Hypothesis 2 is supported for creativity of concepts, but not for diversity of concept sets.

5.3. H3: Comparing design heuristic use across design disciplines

On the main measures of the study, there were many similarities between the outcomes of the design processes for engineering and industrial designers. They generated a similar number of concepts as a group, and on average, shared similar ratings of the creativity and diversity of their concepts. They also rated their own performance similarly (ID: $\text{M}=5.0,~\text{SD}=1.48$ ; ENG: $\text{M}=5.1,~\text{SD}=0.996$ ) along with their creativity in the study (ID: $\text{M}=5.0,~\text{SD}=1.54$ ; ENG: $\text{M}=4.5,~\text{SD}=1.2$ ); $t<1.0$ , n.s. However, industrial designers used 24% more design heuristics within their concepts. This suggests design heuristics may be somewhat more familiar to, or at least somewhat more likely to be used by, industrial designers.

Of the 53 design heuristics observed in this study, 8 showed significant differences in their frequency of use between the two groups of designers. Table 3 presents the 53 design heuristics observed in the concepts separately for engineering and industrial designers, and a statistical comparison of frequency between the two groups of designers. Fisher’s exact test (two-tailed) is used throughout, with a significance level of $p<0.05$ . Of course, if the null hypothesis is true and each of these 53 tests is independent, we would expect about two or three significant results; we observe eight. If we apply a stricter criterion to address the multiple comparison problem by raising the bar with a Type I error rate of 0.01, we observe four significant results (approximately one result is expected by chance under independence); if we apply an even stricter criterion of 0.001, which roughly corresponds to Bonferroni’s correction under independence, we observe two significant results. We point out that the results indicate mostly similarity between the two groups, with a few potentially important differences between engineers and industrial designers. These potential differences should be explored in future research.

Table 3. Design heuristics identified in the analysis of concepts generated by engineering and industrial designers

Some interesting differences in design heuristic use are evident in these findings. Industrial designers used more design heuristics than did engineering designers (365 vs. 294). This difference seems to be due to the number of criteria brought into the problem. For example, one industrial designer said: ‘What else…. I will have to think about kids maybe. When you are a child, you play with your friends and you know, mimic your mom’s cooking…’, and she continued by adding more criteria: ‘it’s like making it more portable and then very flexible, so you can roll it like a pad, and then you just spread it on the ground and then you can just use it’.

The engineering designers focused more on technical feasibility during ideation. For example, 10 of the engineers used insulation as a heating method, while the 12 industrial designers mostly used closure to maintain the heat without insulation. The engineers used a diverse range of cooking methods, such as the greenhouse effect, liquid, and pressure, while the industrial designers were more likely to use direct sunlight and a heated surface. Another engineering solution was to use multiple mirrors to collect sunlight, demonstrating concern about functional heat, while only one of the industrial designers included this feature. In other concepts, engineers generated solutions incorporating fluids like water or oil for cooking, while none of the industrial designers did. Also, industrial designers utilized existing products (microwave oven, grill) as part of their concepts more often than the engineers.

Engineers more often used the heuristic, Use multiple components to achieve one function (8 vs. 1), adding different components contributing to the same function to enhance capturing sunlight. These components were sometimes identical to each other with minor changes (size, color) to facilitate adjustability in use, or they were different but served similar functions. The reason for the difference may be that engineers were concerned about function, which may have led them to use multiple components. For example, in Figure 7 a, Engineer 10 attached an umbrella with solar panels and mirrors to a conventional outdoor grill to ensure that the required energy would be generated. In Figure 7 b, Industrial Designer 4 also attached solar panels and magnifying lenses; however, the goal was not only to capture more sunlight, but also to use these functional surfaces as the lid and food preparation surfaces. Instead of attaching multiple components to achieve one function as in the engineer’s concept, the industrial designer assigned additional functions to those components.

Figure 7. Examples using the design heuristic: Use multiple components to achieve one function.

Industrial designers used the design heuristic, Use the same component for multiple functions, more frequently than engineers (23 vs. 3), perhaps because they considered portability and compactness, which would lead them to select components that perform two or more discrete functions using a single part. As seen in Figure 8 a, the concept generated by Engineering Designer 5 incorporates the use of a reflective curved surface, which also functions as a stand, and the concept formed by Industrial Designer 1 (Figure 8 b) shows the reflective surface angled for gathering sunlight as well as doubling the cooking surface, collapsing the entire product into two surfaces that share the same dimensions. Industrial designers also used Use inner space for added component placement more frequently (12 vs. 3), again suggesting a portability concern where the inner part of the product or an existing volume is used to allow for the placement of another component.

Figure 8. Examples using the design heuristic: Use the same component for multiple functions.

Another factor that appeared to be considered more often by the industrial designers than engineers was the interaction between the user and the product. For implementing a better interaction, they applied heuristics such as Change the surface material at points of human contact (6 vs. 1), and Create the hierarchy of features (8 vs. 0) more often. Some designs used a different material on the product where the user touched the product for safety or comfort purposes, or applied a different color or pattern to communicate where to touch the product. For creating a hierarchy of features, they presented the user with functions in a set order to assist them while using the product. In this manner, the users were not allowed to access the second function without using the first one. In fact, industrial designers depicted users in multiple concepts, while no engineers did so. The other heuristics significantly more commonly used by industrial designers were as follows: Integrate or attach the product to an existing item (19 vs. 11), Add gradations or transitions to use (6 vs. 0), and Mirror shapes (7 vs. 1).

Another interesting difference was that industrial designers more often used environmental contexts, such as bringing outdoor or indoor features into the concept. This difference might relate to educational differences, as industrial designer education focuses on the context of use, including examining personas and storyboards. Industrial designers also considered more diverse forms (e.g., pipes, racks, and ropes) more often.

Industrial Designer 7 demonstrated an additive approach in heuristic use, showing how heuristics can be used to add increasing elaborations to a design concept to produce variations. Seven similar concepts were generated, but multiple heuristics appear in each, and multiple heuristics to transition between them (see Figure 9). This designer began by attaching two existing components to each other – a magnifying glass and a griddle – to create a surface with focused sunlight. In her second concept, she transformed the magnifying glass to a square magnifying glass attached to the griddle. In the following concept, she made the lens height adjustable, and, in the fourth concept, she added sides to it to maintain the heat more effectively. She then considered portability by adding a rigid handle, which was changed to a flexible handle in Concept 6. In addition to all of the features included in the previous versions of the concept, the final concept included an attachment that held utensils and a spout for draining fluids from the cooking surface.

Figure 9. Seven sequential concepts generated by Industrial Designer 7.

Table 4 displays the design heuristics within each concept from Industrial Designer 7. The total number of heuristics increased in each sequential concept while the changes already introduced were maintained. This suggests later concepts were more complex and included more features.

Table 4. Design heuristics observed in Industrial Designer 7’s concepts

In contrast to this systematic building on earlier concepts, the protocol from Engineering Designer 1 offered nine concepts that differed more, shown in Figure 10. For his first concept, he created a container that could be transported by users to a larger community gathering. The second concept was a large Fresnel lens, adjustable to the angle of the sun as well as to the best angle for cooking. For his next concept, he extended the previous one by segmenting his original lens into four separate lenses. The fourth concept was a spit cooker, which utilized a lens to focus on a line of heat rather than a point. The fifth concept was a double boiler, consisting of a system pumping hot water from a boiler into an outer pot. Concept 6 was a synthesis of previous concepts combining a double boiler with a Fresnel lens. The seventh concept was a lightweight reflective blanket with a drying rack. The eighth concept proposed a smoking chamber. The final concept was a three-stage boiler, which included a solar heater to warm up water to be utilized for steaming or boiling food.

Figure 10. Nine sequential concepts generated by Engineer 1.

To generate these diverse concepts, Engineer 1 spoke about a specific type of food, and then generated a concept for cooking that food. For example, he said ‘Other things to eat. We’ve got shish-kabobs, jerked meat, the dried herbs, the soups and thing; um, let’s see’. He also emphasized different constraints from the problem as he worked; in Concept 3, he focused on ‘maximizing the intensity of the sunlight’, while in Concept 7, he emphasized the constraints of being ‘inexpensive and portable’. A number of design heuristics were evident in the concepts, such as Adjust functions by moving the product’s parts in Concept 3 (where the lens angles could be altered) and Repeat as he added multiple lenses. However, his concept series had more differences between concepts than did Industrial Designer 7’s series. Considering types of food that might be cooked led to a more diverse concept set, with few commonalities across them.

In sum, the design heuristics evident in the protocols were largely similar across design disciplines. Despite presumed differences in training and experiences, both sets of designers used similar approaches to concept generation, and both made heavy use of design heuristics in creating their concepts. Of the 53 different design heuristics evident in the concepts, there were no differences in frequency of use for 85%. This suggests the utility of different design heuristics may depend more on the content of the design problem, the context of the designer’s work, or the instructions to generate multiple ideas. Even so, some interesting differences in the application of design heuristics occurred between groups. In particular, industrial designers focused less on technical questions, while as might be expected, this was emphasized by engineering designers. Industrial designers emphasized user experience, environmental contexts, and added features, resulting in concepts that utilized more design heuristics, but not more concepts. Hypothesis 3 was partially supported; while design heuristic use was similar in the two disciplines, important differences in their use were observed among designers in engineering compared to industrial design.

6. Discussion

This study investigated how designers use design heuristics in concept generation when working alone. The study compared design heuristic use by design practitioners and students from both mechanical engineering and industrial design working on the same problem under the same conditions. The first goal of the study was to determine whether the use of design heuristics would be observed within in a short design task with a novel product problem. Evidence of design heuristic use was ubiquitous, occurring in almost all of the concepts generated by both engineering and industrial designers. This strongly indicates the usefulness of design heuristics in the early stages of the design process during concept generation.

Second, the concepts generated were compared in the use of design heuristics and in the perceived creativity and diversity of the concept set. These results showed a strong relationship between the use of multiple design heuristics within a concept and its rated creativity, such that greater heuristic use predicted higher creativity scores. The use of multiple design heuristics, in addition to facilitating interesting variations on possible designs, resulted in higher creativity ratings compared to concepts where fewer heuristics were observed.

However, the diversity of concepts within a set showed a negative relationship, where more heuristic use was correlated with a less diverse set of concepts. This finding may be due to the sequence of concepts generated using design heuristics, where ideas based on new heuristics are added onto earlier concepts to produce new ones. The result of this cumulative process may be a concept high in creativity, and a set of concepts with higher overlap in features. As a result, less diversity is evident in the set of concepts. Example protocols with creative outcomes included sets with more diverse concepts and those with a repeated concept with added features.

A final question was how industrial and engineering designers compare in their use of cognitive heuristics during concept generation. The results showed a great deal of similarity in the number and types of design heuristics used by the two groups, and in how design heuristics were used. This suggests that design heuristics may be an effective means of ideation in both design disciplines. The main difference between the two groups was that industrial designers focused more on user interaction, environment, and added features, aspects addressed by design heuristics. Engineering designers focused on function as demonstrated by some differences in the choice of heuristics observed. These disciplinary differences could be tied to education and experience. Engineering designers’ solutions were more functionally diverse, detailed, and provided more technical information, along with meeting more of the problem criteria. Despite lacking technical knowledge for designing solar products, industrial designers considered contextual and user perspectives common in their training. In testimony to the success of both disciplines, there were no significant differences between the two groups in terms of the creativity and diversity of the solutions they generated.

The findings from this study build on a long tradition of protocol analyses with designers in order to uncover the cognitive processes involved (Adams & Atman 1999; Christiaans & Dorst 1992; Dorst & Cross 2001; Stauffer & Ullman 1988; Dahl & Moreau 2002; Kavakli & Gero 2002; Okudan et al. 2006; Hernandez et al. 2010). These prior studies have laid the groundwork for the present study by demonstrating the occurrence of moments of creativity within the cognitive processes of design, such as ‘unexpected discoveries’ (Suwa et al. 2000), ‘novel design discoveries’ (Akin & Lin 1995), and ‘opportunism’ (Guindon 1990; Ball & Ormerod 1995).

The present findings go beyond these results to identify more specific ‘rules of thumb’ based on the content of concepts. Design heuristics specify a way to apply a known design strategy within a new problem. These proposed cognitive heuristics are more specific than those of similar approaches such as SCAMPER (Eberle 1995); for example, ‘modify’ is a strategy proposed by SCAMPER, while design heuristics suggests specific ways in which to modify a design (Change the material at points of human contact; Change flexibility; Replace with recycled materials). Synectics (Gordon 1961) offers idea triggers like, ‘amplify a feature’, while design heuristics provide ways to change features (Scale size up or down; Divide a continuous surface; Mirror shapes). These heuristics are more specific to the content of designs.

Of course, the present study had a relatively small sample size, so we are not able to detect small effects. There may be more differences that could be observed with larger studies. Also, a larger sample could include more diversity within engineers and industrial designers, which could be good for purposes of generalization but could reduce power by increasing the error term in the statistical model. We are also concerned with the fact that we may have an inflated Type I error rate because of the number of tests we computed. We view the 8 significant differences out of the 53 tests as potential areas for future research to test more carefully.

In addition, the design heuristics described here are based on patterns empirically observed in the work of designers (Daly et al. 2012c ; Yilmaz & Seifert 2010, 2011). The heuristics are based on evidence of their use by practicing designers. As a result, it is likely they will be useful in other concepts because they are grounded in the experiences of real designers. Another well-known tool in engineering, TRIZ (Altshuller 1984), uses a similar approach to identify solutions based on analyses of past successful product patents. This approach of making use of past designs in discovering ways to generate new concepts has met with some success (Ilevbare, Probert & Phaal 2013). However, applying the TRIZ principles requires specified design elements (in order to identify and resolve functional conflicts), and so may be most helpful in later stages of conceptual design. In addition, learning to use TRIZ requires substantial effort and commitment (Ilevbare et al. 2013).

From the present results, it is clear that designers in both industrial design and engineering fields make use of design heuristics frequently, and that their use results in more creative concepts. Exposure to a variety of heuristics, and experience in applying them on many different problems, may lead to the development of expertise in design in both disciplines. These findings suggest students may benefit from explicit instruction about design heuristics. In several studies, instruction in design heuristics has been provided as a tool to assist in design. Each design heuristic was presented on a card with a description and examples of the heuristic in consumer products (see Figure 11). The results showed the design heuristics tool to be supportive in successful concept generation by students (Christian et al. 2012; Daly et al. 2012b ; Kramer et al. 2014) and expert designers (Yilmaz et al. 2013a ).

Figure 11. The design heuristic, Utilize opposite surface, display as a tool for designers. One side of the $4\times 6$ card displays the description and a graphic depiction, while the other offers two consumer products where the design heuristic is evident (from www.DesignHeuristics.com).

Design heuristics are successful in aiding the designer by introducing variation in candidate designs, leading to a more diverse set of concepts to consider. They also appear to be very helpful in moving past fixation to consider other, non-obvious possibilities. However, this study was limited to a short design task performed by individual designers in a paradigm that differs significantly from the work of professional designers. The design task was an isolated, one-time session, and it did not allow the designers to seek added information nor reflect design as practiced by a team. In addition, the study did not examine differences in the design cultures and experiences of the individuals within each group.

Other limitations from the small scale of the study include the use of a single test problem in a laboratory setting, and limited generalizability based on the small sample size. In addition, the assessment of designs and their qualities depended on expert raters using Amabile’s CAT (1982). On one measure, the reliability between raters was lower than desired, perhaps due to the small sample of participants in the study. Finally, including more designers with high levels of experience may be helpful in assessing the use of design heuristics based on the level of expertise. In future studies, it is important to observe the impact of design heuristics on the product design process over a longer period of time and on projects situated in natural contexts including groups.

Engineering and design students and practitioners develop diverse ideation approaches based on their design knowledge and practice. However, recognizing heuristics together with specific problems from experience is challenging, as designers may not formulate heuristics from their design experiences in a way that generalizes to other problems. When discovered, they may not share their heuristics with other designers or students. The present study provides a collection of heuristics observed in practice across many individual designers, offering a potential new tool for students and practitioners to use in creating unexpected concepts. Pedagogy for enhancing design creativity is essential because most engineering and industrial design problems demand innovative approaches in the design of products, equipment, and systems. In this study, design heuristics were observed to be associated with effective innovation in both engineering and industrial design domains.

As reported in this paper, design heuristics can help designers create multiple variations based on an initial idea and suggest further development of concepts in unusual ways. For many design and engineering students, simply having an arsenal of design heuristics as tools might lead to improvement in the variety and creativity of concepts generated. The findings from this study can be implemented in design practice and education. The results provide a collection of useful design strategies that can be used by practitioners, educators, and students to facilitate idea generation. Future research will continue to identify how design heuristics are used by both practitioners and students to enhance creativity across design and engineering disciplines, and test their efficacy in classroom and professional settings.