A qualitative study of collaborative stimulation in group design thinking

Abstract

It is often assumed in both education and industry that collaboration encourages creativity. This assumption is explored by investigating the influence of designers’ interactions on creativity-relevant thinking processes by extending creative cognition to the group design context. It is proposed that sharing design entities and questions stimulates creativity-relevant thinking processes through four types of collaborative stimulation. Specific patterns are hypothesized to exist between each type of collaborative stimulation and thinking processes. A case study was conducted to determine whether the hypothesized types and patterns of collaborative stimulation exist. The results were analyzed using a directed coding approach and collaborative retrospective protocol analysis, which enable capturing both internal thoughts and external interactions with minimal interference to collaboration. The results indicate that the identified types of collaborative stimulation are observable and that they have recognizable patterns with stimulated thinking processes. Stimulation occurring through design entity questioning had the strongest relationship with generative thinking processes. Although creativity-relevant generative processes are stimulated by collaborative activity, this does not necessarily mean that collaboration results in a more creative product. However, these patterns can be used in future work to develop methods and interventions for promoting group idea generation and improving group creativity.

1 Introduction

Modern engineering designers face two key issues: complexity and globalization. The increased complexity of current design projects means the increased need for collaboration among multiple engineers, teams, and companies. At the same time, globalization has made it necessary for companies to continuously innovate in order to stay competitive. Therefore engineers must produce designs that are creative. These two issues provide motivation to explore how collaboration, which is already required, can drive creativity.

Researchers in the areas of business innovation, professional teams, and design groups have proposed many solutions for making the collaborative process more effective. These solutions suggest organizational structures and policies to promote creativity (Woodman, Sawyer & Griffin 1993; Amabile et al. 1996; Wilde 2010) and collaboration methods for creativity (Osborn 1957; Gallupe et al. 1992; Warr & O’Neill 2005). Although many of these approaches do improve idea generation, some, like brainstorming, have questionable effects (Diehl & Stroebe 1987). There is an opportunity to discover new effective ways of collaborating by taking a different approach.

Research on creative cognition (Finke, Ward, & Smith 1996) in the area of engineering design has generated new insights regarding design methodologies by taking a deeper approach that considers the individual’s thinking processes, or design thinking (Shah et al. 2003; Jin & Benami 2010). Although design in collaborative settings has been explored to investigate issues such as communication (Dong 2005), value interaction (Le Dantec & Do 2009), and ethics (Lloyd 2009), most of the studies exploring creative thinking processes have focused on individual design situations instead of collaborative design.

Group design thinking draws from the combination of these two bodies of research, group creativity and creative cognition/design thinking. Researchers have investigated the influence of collaboration on creativity from a cognitive perspective (Goldschmidt 1995; Stempfle & Badke-Schaub 2002; Nijstad & Stroebe 2006). However, these works have not focused on how collaboration influences multiple specific thought processes. There are also studies that explore group cognition, which takes a cognitive approach to modeling group behaviors by treating a group as the unit of analysis (Hollan, Hutchins, & Kirsh 2000; Shalley & Perry-Smith 2008).

This research proposes an interactive cognition approach, which differs from the group cognition approach as it explicitly explores the influence of individuals’ interactions on their thought processes and views the group behavior as the emergent result of the interactions among the individuals. An interactive approach has been taken before to study negotiations (Jin & Geslin 2010), but in this paper, the influence of interactive cognition on thought processes important for generating creative ideas is explored.

A major challenge in interactive cognition is to experimentally identify thought processes and their interplay with collaborative interactions without greatly altering the design process. To overcome this challenge, a case study style experiment was designed using retrospective protocol analysis. This allowed designers to work together naturally, while also collecting individual protocols. Both the individual protocols and conversation transcripts were then analyzed to provide a more holistic view of collaborative interactions and thinking processes. A directed approach to data analysis was taken, first identifying design entities (i.e., functions, behaviors, and structures/forms) (Jin & Benami 2010), which lead to thought processes, which lead to identification of instances of collaborative stimulation. The results are then interpreted through a bottom-up, inductive approach. Although most of the time design studies pursue experiments that generate large amounts of quantitative data, sometimes case studies are more appropriate, especially when studying complex structures like thought processes or cognition (Gruber 1981).

Using collaborative retrospective protocol analysis, this research was able to start extending creative cognition to collaboration through an interactive cognition approach. This was done by investigating the collaborative stimulation effect that creativity-relevant thought processes are collaboratively stimulated through shared design entities. To explore collaborative stimulation, a collaborative thought stimulation (CTS) model was developed, as an extension of the creative cognition generate–stimulate–produce (GSP) model (Jin & Benami 2010). The proposal of collaborative stimulation raises two research questions, summarized as What types of collaborative stimulation exist? and How influential is collaborative stimulation? To identify collaborative stimulation in the design process, a design case study was conducted where students collaborated on a design problem. Analysis of the results focused on the types of collaborative stimulation and the patterns that occur between each stimulation type and thought processes. The findings from this work bring a better understanding of how collaborative stimulation works and patterns in which it influences thinking processes. This knowledge can be used in the future for the development of more effective ideation techniques and for training design groups.

The rest of the paper is organized as follows. In Section 2, the related work is reviewed and the gap in the literature identified. Section 3 presents an interactive cognition based model of CTS that identifies various types of collaborative stimulation. A case study style experiment is described in detail in Section 4 and the results and discussion of the case study are presented in Sections 5 and 6. Finally conclusions, limitations, and future work are described in Section 7.

2 Related work

Creative cognition and group creativity are the foundations upon which this research is built. Creative cognition is an approach that explains creativity as a cycle of convergent and divergent thought processes occurring within an individual. The creative cognition approach has been expanded to the field of design revealing new insights for methodologies. Group creativity studies investigate the influence of group properties and dynamics on creativity. Group creativity research includes not only work on teams, but also collections of individuals that are not teams. Combining these two bodies of research results in the study of group design thinking.

2.1 Creative cognition

The creative cognition approach was introduced by Finke et al. (1996). Their Geneplore model divides the creative cognitive processes into two categories, generative processes that create ideas and exploratory processes that evaluate them. Similar to the convergent and divergent stages identified in earlier creativity research (Guilford 1967), exploratory and generative processes occur in a cyclical manner, maturing an idea from abstract to concrete with each cycle. The Geneplore model has been expanded to engineering design from multiple perspectives. Benami & Jin (2002) identified applicable thinking processes for creativity in engineering (defined by some as design thinking). Memory retrieval, transformation, and association generate design entities (or concepts) and problem analysis and solution analysis explore (or evaluate) them (also see Jin & Benami 2010). Shah et al. (2003) developed an approach to align experiments occurring in creative cognition research with those occurring in design. Specifically, they focused on incubation, with the hope of developing a more complete design ideation model. Chusilp & Jin (2006) identified three iteration loops: problem redefinition, idea stimulation, and concept reuse occurring in engineering design and explored the types of cognitive processes that occurred in each loop. These efforts in applying creative cognition to design have led to new insights for methodologies. For example, it has been found that more ambiguous and less mature concepts tend to provide the best stimulation (Jin & Benami 2010), and a better understanding has been developed of key components of ideation methods, like incubation and proactive stimuli (Vargas-Hernandez, Shah & Smith 2010).

2.2 Group creativity

Many authors in group creativity, the second foundation of this work, have proposed models of group creativity, which can be grouped into two categories: aggregate models and process models. Aggregate models explore how each individual’s creativity contributes to the total creativity of the team (Taggar 2002; Pirola-Merlo & Mann 2004; Shalley & Perry-Smith 2008). Pirola-Merlo & Mann (2004) propose a model that explores how each individual’s creativity is influenced by the organization’s climate, which is then summed into team creativity, which is then aggregated over time for the team’s total creative output. Taggar (2002) explores how individual factors are related to individual creativity, and then how these and organizational aspects correlate with team creativity. Shalley & Perry-Smith (2008) describe the team as a cognitive entity, and then reflect how diverse networks improve creativity. Process models view team creativity as a set of interaction processes that lead to a creative product (Goldschmidt 1995; Stempfle & Badke-Schaub 2002; West 2002; Sonnenburg 2004). Sonnenburg (2004) proposes a series of stages the team goes through on the path to a creative solution. West’s (2002) model investigates how a number of processes influence creativity and innovation implementation, which he claims are two different skills. Stempfle & Badke-Schaub (2002) have analyzed the collaborative process for design by dividing activities into task work and team work, and identify types of thinking operations relevant to both. To investigate team design processes, Goldschmidt (1995) analyzed protocols of individual and team design processes by measuring design productivity in terms of link-index and critical design moves; the individual and team processes were found almost identical.

There has also been much work in group creativity focused on the development of methods to improve creativity. Methods and tools have included the 6–3–5 method (Rohrbach 1969); brainstorming (Osborn 1957); collaborative notebooks (Michalko 2001); and tablets and personal digital assistants (PDAs) to share information (Warr & O’Neill 2005). Perhaps, the most popular of these methods and most frequently researched is brainstorming. Although brainstorming research has shown the method has positive stimulating effects (Brown et al. 1998; Dugosh et al. 2000), it has also been found to reduce the quantity and quality of creative ideas due to social inhibition and procedural issues (Diehl & Stroebe 1987; Mullen, Johnson & Salas 1991). More effective methods and tools can be designed if they are based on creativity research, as can be observed in the creation of the C-sketch method (Shah et al. 2001), which was developed by considering group creativity research.

2.3 Gap in the literature

In examining the past work on creative cognition and group creativity, a gap can be observed. Creative cognition explores thinking processes of each designer (Finke et al. 1996; Chusilp & Jin 2006; Jin & Benami 2010), but does not explore the influence of collaborative interactions. Group creativity examines team interactions, but treats individuals as ‘black boxes’, not investigating individual thinking processes (West 2002; Pirola-Merlo & Mann 2004; Sarmiento & Stahl 2008). Although there has been a lot of research that comes close to bridging this gap, few have explicitly addressed creative cognition in the collaboration context.

3 Collaborative stimulation of thought processes

The aim of this research is to bridge the gap between creative cognition and group creativity by proposing an interactive cognition approach to studying collaborative design, which considers both the individual’s thought processes and how collaborative interactions influence them. More specifically, this research attempts to investigate the effect of collaborative stimulation on creativity-related thought processes by building and evaluating a model of CTS.

Collaborative stimulation is defined as collaboratively shared design entities (i.e., design-related information and knowledge generated by a designer during the design process) inspiring an individual’s thought processes. To investigate the collaborative stimulation effect on thought processes, two key research questions must be addressed:

Q1: What are the types of collaborative stimulation created by designers’ interactions and can they be observed?

Q2: Do the different types of collaborative stimulation have recognizable patterns influencing specific thought processes?

The first research question can be answered by extrapolating what the existing literature (both academic and those in industry) alludes to. As outlined in the rest of the section, from the existing work, industry experience, and pilot experiments we expect generative thought processes will result from four types of collaborative stimulation: prompting, seeding, correcting, and clarifying. The existence of these types of collaborative stimulation will then be explored in a collaborative design case study. Although some ideas can be hypothesized about the second research question, the best way to investigate it is through the case study. We expect the case study to reveal patterns between each collaborative stimulation type and thought processes.

The collaborative stimulation effect has been alluded to in different ways by research in brainstorming (Brown et al. 1998), group creativity models (West 2002), and memory retrieval (Nijstad & Stroebe 2006). However, little systematic research on collaborative stimulation has been carried out, and the specific types of collaborative stimulation have not been identified. Also, relationships have not been drawn between collaborative stimulation types and thought processes. By combining the depth of creative cognition research with collaborative aspects identified in group creativity research, powerful insights into group design thinking may be generated.

3.1 Collaborative thought stimulation model

In this research, a CTS model has been created to visualize the collaborative stimulation effect. It is based on the GSP model of creativity in conceptual design (Jin & Benami 2010; also see Benami & Jin 2002). The primary attribute of the GSP model is that design entities (e.g., design sketches, information, and ideas) created by a designer are found to be the most important source of stimuli driving a designer’s creative thinking process. As shown in Figure 1(a), design entities stimulate thought processes (both generative and exploratory), which produce design operations, which in turn generate new design entities. The cycle continues until pre-inventive design entities (undeveloped concepts) mature to knowledge entities (the completed design). The research behind the GSP model revealed how different types of design entities, such as function, behavior, and structure, invoke different cognitive processes, such as memory retrieval, association, and transformation, under different design contexts.

Figure 1. GSP model (a); CTS model (b).

The elements of the GSP model can occur internally or externally. Some, like thinking processes, occur entirely internally, while others, like design entities, are often external. Design operations can be internal or external, depending on the approach. However, despite containing internal and external elements, the GSP model only focused on the design process for a single individual. The CTS model extends the GSP model to collaboration by emphasizing that many of the design entities created by one designer are visible to, and therefore shared by, the designer’s collaborators. (A collaborator is defined as a designer who works with another designer to achieve a mutual design goal.) As shown in Figure 1(b), in the CTS model, each designer engages in the same processes as the GSP model (shown in gray), but external actions are visible to the other designer (shown in white). The CTS model also combines concepts from both the aggregate and process models of group creativity research, as external design entities are aggregated while they stimulate thinking processes, which leads to a cycle of generation of new design entities.

Following the GSP model, the CTS model has been developed based on the recognition that a designer’s externally shared design entities are the major source of collaborative stimuli for their collaborator’s thinking processes. The CTS model proposes that in a collaboration context, a designer’s thinking processes are no longer performed independently. Rather, they are interactive as one designer’s thinking can be heavily influenced by the actions of their collaborators. Collaborative stimulation, when thinking processes are stimulated by shared external design entities, enables groups to develop ideas that would not have occurred to them had they worked alone.

While collaborative stimulation influences both exploratory and generative thinking processes, the current focus is on the stimulation of generative thought processes, consisting of memory retrieval (remembering an idea from the past), association (drawing relationships between two design entities), and transformation (altering a design entity). These three have been identified as key thought processes in engineering design (Jin & Benami 2010) from the many generative thought processes investigated by Finke et al. (1996). Therefore, the types of collaborative stimulation that results in exploratory thought processes, or other thought processes, are not identified in this work. Generative thought processes were chosen as the focus for this study as they result in the creation of new ideas, rather than exploratory processes, which result in a convergent evaluative process.

Figure 2. A breakdown of collaborative stimulation categories.

3.2 Types of collaborative stimulation

Treating design entities as stimuli provides a basis to categorize different types of stimulations. In the GSP model, the categorization of stimuli was based on the types of design entities, i.e., function, behavior, and structure. In this research, however, since the focus is on interactions between designers, the categorization is based on how the external design entities are applied and reacted to interactively during design collaboration. The types of collaborative stimulation described below are the result of a broad literature review, industry observations, and repeated pilot experimental case studies (Sauder 2013).

Two interactions, design entity sharing and design entity questioning, have been identified that elicit reactions, which stimulate generative thought processes (vs. elicit generative thought process stimulation). Design entity sharing occurs when a design entity created by a collaborator is disclosed to the designer. Design entity questioning occurs when the designer is questioned (or assumes a question) by the collaborator about a design entity. Specific reactions to design entity interactions that inspire new thought processes are prompting, seeding, correcting, and clarifying (Figure 2), which are defined as types of collaborative stimulation. Prompting and seeding are reactions to design entity sharing, and correcting and clarifying are reactions to design entity questioning. Individual stimulation also still occurs in the collaborative setting (Figure 4). Following is the detailed discussion of the theoretical underpinnings from literature review for each type of collaborative stimulation.

Figure 3. Step-by-step experimental process and study diagram.

Figure 4. Skateboard locking arm and lock location.

3.2.1 Prompting

Research on how collaboration influences memory reveals that ideas from team members will often stimulate an individual’s memory (Nijstad & Stroebe 2006). Brainstorming research agrees, finding that collaboration has stimulating effects on memory retrieval (Brown et al. 1998; Dugosh et al. 2000). In fact, the reasoning behind brainstorming is that ideas will stimulate each other. The authors base prompting on these research concepts, which occur when a design entity developed by a collaborator reminds the designer of a memory. It is expected that prompting will stimulate the thinking process of memory retrieval most often, as it primarily reminds the designer of past experiences. However, occasionally an association is also expected to connect a remembered idea to the current design task.

3.2.2 Seeding

Building on another’s ideas is one of the key steps in collaborative design (Holsapple & Joshi 2002) and is often the basis of aggregation models of group creativity (Taggar 2002; Pirola-Merlo & Mann 2004). There are multiple creative design techniques that are based on this effect, including the 6–3–5 technique (Rohrbach 1969) and C-sketch (Shah et al. 2001). Although ideal models would have individuals continuously contribute to a collaborative idea, more frequently individuals build on only parts of other’s ideas and then combine them. So rather than building on each other’s ideas being a continuous phenomenon it occurs sporadically throughout the design process (Kvan 2000). From these observations, it is proposed that the collaborative stimulation of seeding occurs when a design entity developed by a collaborator is furthered by an individual. It is expected that seeding will stimulate the thinking process of transformation, as building on another’s idea requires making changes.

3.2.3 Correcting

Critiquing and then revising the design based on those critiques is a key component of the ontological design process (Holsapple & Joshi 2002). A model by Stempfle & Badke-Schaub (2002) proposes that the evaluation of ideas (a challenge or question) reveals a concept’s inadequacies and encourages the generation of new ideas. Correcting comes from these observations, and occurs when the designer is asked a question or challenged by a collaborator, and alters the design entity to resolve the raised issue. Correcting is expected to most significantly influence transformation, as the thought process is required to alter a design entity. A moderate influence on association and a minor influence on memory retrieval are also expected as both can be part of the alteration process, but are not required. Association is believed to be more influenced than memory retrieval, as connecting the transformed idea to existing concepts is thought to be more likely than remembering existing ideas to build the new concept around.

3.2.4 Clarifying

Elaboration is an important practice that supports creativity, opening opportunities for additional insights (Vyas & Nijholt 2009). The authors propose clarifying to be a form of elaboration, which occurs when the designer is challenged or asked a question by a collaborator (or senses they do not understand a concept) and attempts to clarify their idea by explaining it in a different way, which leads to the occurrence of generative thinking processes. Clarifying is expected to have a heavy influence on association, and a moderate influence on both transformation and memory retrieval because it will often come through the form of an analogy (involving all three processes). The heavy influence on association is expected as association is critical for drawing connections with existing concepts. Analogies have been observed to be used to explain concepts (Goldschmidt 2011). This can often be seen in education, where educators will explain challenging or new concepts by using analogies (Glynn & Takahashi 1998). The authors’ recent work has revealed some of these explanatory analogies can become generative analogies, creating new concepts (Sauder & Jin 2012). These generative analogies consist of the three generative thinking processes of memory retrieval, association, and transformation (Jin & Benami 2010).

3.2.5 Additional types of collaborative stimulation

Originally, there were several other types of collaborative stimulation identified, e.g., accommodating and collaborative completion. In collaborative completion, one designer would reach a road block in their thinking process but their collaborator is able to bridge the missing link from their past experience or from a previous design entity, which was generated earlier in the session. However, through pilot experiments this was discovered to be a very specific and specialized case of seeding and was then removed. Accommodating was defined as an effort to incorporate ideas that the designer viewed as valuable or come to a general agreement. The designer will accommodate another’s idea on some level into their own to find a mutual solution. Through pilot experiments and consideration of the protocols, accommodating was also found to be a specialized type of seeding.

Although there may be other types of collaborative stimulation resulting from shared design entities that were not identified in this work, within the scope of this study, we have identified the above mentioned four as the main types of collaborative stimulation that stimulates generative thought processes. The definitions of the four types of collaborative stimulation are purposefully broad, and additional subcategories could likely be developed if desired. But for the purpose of clarity and to realistically code the transcripts, it was effective to keep collaborative stimulation to four main types.

It is worth mentioning that the types of collaborative stimulation introduced above are limited by the cognitive stimulation model used in this work – i.e., only the design-entity-relevant collaborative stimulations were included. Other potential collaborative stimulations may result from collaborators’ gestures, volume of voice, and other personality-related stimuli. As the first step of this research, however, only the design entity induced collaborative stimulations were explored.

4 Method: collaborative thinking experiment case study

The CTS model and the various types of collaborative stimulation identified have led to the following two research questions, discussed earlier:

Q1: Does the collaborative stimulation of thinking processes exist, or can it be observed?

Q2: Do the different types of collaborative stimulation have recognizable interactive patterns with specific thinking processes?

To address the questions, an experimental methodology must be established that allows for the observation of individual thinking processes in the collaborative setting. Q1 can be evaluated by multiple individuals identifying the existence of collaborative stimulation at the same points in collaboration. Q2 can be dealt with by identifying the probability that specific thinking processes would be stimulated by specific types of collaborative stimulation. A typical approach would be to use protocol analysis (Cross, Christiaans & Dorst 1997), where subjects think aloud while they are working through a design process. Their verbalized thoughts are then transcribed, divided into episodes and segments, and then a coding scheme is applied to identify thought processes. Inter-coder reliability is then used, to verify the thought processes and occurrences of collaborative stimulation observed.

4.1 Past approaches to analyzing collaboration

Using dialog transcripts is the typical approach to analyze collaborative activity. Sometimes, actual protocol analysis is done, applying a coding scheme to a dialog transcript (e.g., Artzt & Armour-Thomas 1992; Stempfle & Badke-Schaub 2002) while other times the conversation is simply analyzed for social interactions. Brereton et al. (1997) investigated how collaborative interactions influence the design process by either focusing it on a specific concept or transitioning to a new idea. Cross & Clayburn Cross (1995) explore the aspects of roles and relationships, planning and acting, information gathering and sharing, problem analyzing and understanding, concept generating and adopting, and conflict avoiding and resolving, in collaborative design.

Protocol analysis has been applied to dialog in order to identify thinking interactions. Wiltschnig, Christensen & Ball (2013) examine how collaboration influences problem/solution co-evolution between designers and is linked to creative activities (e.g., analogizing). Stempfle & Badke-Schaub (2002) specifically apply protocol analysis to a team’s dialog transcript, identifying underlying thought operations. They state it is valid to use protocol analysis on conversation to observe thought operations because of the work by Goldschmidt (1995), which compares individual verbalizations to group dialogs. Goldschmidt (1995) states the intimate nature of sharing occurring in design team conversations is close to the internal speech individual verbalizations produce. Similarly Tang, Lee & Gero (2011) explore the influence of digital versus traditional sketching and collocation versus non-collocation on high- and low-level thinking activities by using protocol analysis on two collaborators’ dialog. Another approach involving a detailed examination of conversation is latent semantic analysis (LSA), which has been used in design to computationally compare the dialog of one team member to the dialog of another (Dong 2005). However, the use of LSA is focused to comparing mental models (Dong, Kleinsmann & Deken 2013).

4.2 Evaluating concurrent versus retrospective protocol analysis

Few of the past collaborative approaches obtain individual protocols over the length of the design process. Goldschmidt’s (1995) and Stempfle & Badke-Schaub’s (2002) approach of only analyzing the conversation transcript was not sufficient, as the CTS model explores both external (shared) and internal (private) thoughts. Therefore a modified protocol analysis approach is required.

Two different methods, concurrent and retrospective protocol analyses, were developed based on the current techniques to obtain individual protocols in the collaborative setting. Both were evaluated by the authors through two pilot experimental case studies (Sauder et al. 2012). Collaborative concurrent protocol analysis used a physical barrier between designers that allowed communication but prevented verbalized thoughts from being communicated. This was accomplished by having two designers work remotely using Skype, employing screen share and a push-to-talk feature. This allowed the designers to verbalize their thoughts continuously while not in conversation, but prevented the collaborator from hearing their verbalizations. Both the verbalized thoughts and the conversation were recorded through the computer’s microphone.

With the collaborative retrospective protocol analysis, designers were allowed to collaborate and converse in person as they normally would have while being videotaped, and then performed retrospective thinking aloud while watching the video independently immediately after completing the task. Retrospective protocols have been found to produce similar results to concurrent protocols (Gero & Tang 2001). Conducting thinking aloud after collaborating on the design problem allowed the designers to collaborate in a natural environment, and allowed for continuous verbalization of their thoughts.

The first pilot case study consisted of six participants and explored the collaborative concurrent think aloud method. Although the concurrent approach successfully obtained protocols, the interruption from the collaborator trying to talk made the protocol discontinuous. Furthermore, working via Skype and an electronic sketchpad made the collaborative design process complicated and inefficient. The second pilot case study explored the retrospective think aloud method and had seven participants. The video provided adequate cues to the designers so that they would not forget what they had been thinking. Subjects reported that they were able to remember 90 % or greater of their thoughts for design processes that lasted under 30 min. One of the challenges this method faced was that occasionally while designers were retrospectively thinking aloud, they would slip into describing the task they were doing, instead of describing their thoughts. To correct this, the experimenter reminded the designer to verbalize their thoughts, not just their actions.

The two pilot case studies demonstrated that the retrospective methodology provided better data and allowed natural design conditions. Therefore, it was used to evaluate the CTS model and explore the types of collaborative stimulation.

4.3 Experimentally exploring collaborative stimulation and the CTS model

4.3.1 Subjects

Subjects for this case study consisted of senior undergraduate students and master’s level graduate students in mechanical engineering at the University of Southern California (USC). There were a total of 10 subjects, 3 females and 7 males. All students were taking classes focused on engineering design, and had been assigned group projects in those classes. Therefore, they were familiar with participating in collaborative design and had been taught the basics of engineering design methodologies. All subjects gave consent when arriving at the study and were compensated by being entered in a drawing for an iPod. The study was reviewed and approved by the institutional review board.

4.3.2 Procedure

Each designer participated in the case study by first taking a Biographical Inventory of Creative Behaviors (BICB) survey (Silvia et al. 2012), then being trained to think aloud, working on the design problem, and finally retrospectively verbalizing their thoughts.

4.3.2.1 Biographical Inventory of Creative Behaviors

Before coming to the study, participants were given the BICB, to determine their individual creative experience (this inventory was reviewed with other creativity inventories by Silvia et al. (2012) and found to be both quick and effective). This inventory provides a list of diverse creative activities, and asks the participants to identify which they have recently participated in. The results of the BICB were used to organize the teams, putting individuals with a team member who had a similar BICB score. This was done to ensure students were well matched, and was useful in other research investigating the influence of creative experience on collaborative stimulation (Sauder & Jin 2013).

4.3.2.2 Think aloud training

When first arriving at the study, participants were individually given an overview of the process and training in verbalizing their thoughts (see step 1 in Figure 3). The training started with verbalizing a simple process, and continued to increase in difficulty until the subjects were verbalizing while performing a practice design problem. Students were allowed to move to the next step once they demonstrated they understood how to think aloud while designing.

4.3.2.3 Design problem

The designers were then brought together at a shared desk for the next step of the case study (see step 2 in Figure 3). Designers were provided with pencil, paper, and the design problem statement (given in the Appendix), which asked them to develop a device that would securely store skateboards, preventing students from stacking them up against the walls of the classroom. The designers were then videorecorded as they collaboratively worked through the design problem.

4.3.2.4 Retrospective thinking aloud

Immediately after the subjects completed the design problem, they were asked to retrospectively verbalize their thoughts from the design process (see step 3 in Figure 3). This was done while watching a video of the design problem, which provided verbal and visual cues. If the video moved too fast for the subject to provide a complete verbalization, they could pause the video and complete their thought. The retrospective verbalizations were recorded in an audio file for later transcription.

4.3.3 Measurements and data analysis

The data from each case study consisted of two audio files and a video file. The goal of this case study was to observe creativity-relevant generative thinking processes and the various types of collaborative stimulation. This required the use of a coding scheme to first identify the thinking processes and second, the collaborative stimulation. The coding scheme mirrored the model, translating the abstract experimental data such that it could be compared with the model (van Someren, Barnard & Sandberg 1994). The three main parts of the coding scheme consisted of the identification of design entities, thinking processes, and collaborative stimulation.

Given that this research was built on the previous GSP model (Jin & Benami 2010), a directed content analysis approach (Hsieh & Shannon 2005) was taken. Most of the coding scheme was chosen beforehand as directed by the previous research work on GSP. The only difference is that the stimulations generated by the external design entities of the collaborators were considered as collaborative stimulation. The analysis started with identifying design entities (ideas), which lead to thought processes and then to identification of instances of collaborative stimulation. The results were interpreted through a bottom-up, inductive process with the purpose of gaining insights into the existence and the pattern of collaborative stimulations.

4.3.3.1 Design entities

A design entity was identified as a potential or partial solution having a structure, function, and/or behavior. Structures consist of the physical shape of an object. A behavior consists of how an entity interacts with its environment. A function is the purpose an entity serves, related to the problem. Any time a structure, function, or behavior was mentioned, it was classified as a design entity. Initially, design entities started out as only a partial solution, but later developed into full solutions (Jin & Benami 2010). Sometimes, design entities were accompanied by sketches, which made them easier to identify.

4.3.3.2 Thinking processes

After the design entities were identified, all the thinking processes occurring in the transcript were identified. Thinking processes relevant to the CTS model in design consisted of the generative processes of memory retrieval, association, and transformation (Jin & Benami 2010). The identification of each thinking process is mentioned below:

• ∙ Memory Retrieval: when an experience or design entity that existed in the past is remembered.

• ∙ Association: when connections are drawn between two design entities.

• ∙ Transformation: when a design entity is altered or changed.

4.3.3.3 Collaborative stimulation processes

Next, the collaborative stimulation processes were identified by examining how thinking processes came about (reaction to an interaction or stimulated by an individual’s own idea), and if they could be attributed to an individual stimulation or collaborative stimulation. When a thinking process was determined to have been stimulated by an interaction, or collaborative stimulation occurred, the type of collaborative stimulation was identified from the list below:

• ∙ Prompting: An external design entity leads to a new memory retrieval. It can occur collaboratively or non-collaboratively.

• ∙ Seeding: A collaborator’s external design entity is internalized by the subject and modified.

• ∙ Correcting: A subject corrects their idea because of a partner’s question or challenge, to make it acceptable.

• ∙ Clarifying: A subject feels their collaborator does not understand an idea, so they further clarify it. The process of clarification leads to further development.

The coding scheme can be summarized as in Table 1. The coding scheme was checked by conducting inter-coder reliability on over 10 % of the data with a second coder. The inter-coder agreement was 0.86 for identifying collaborative stimulation, and 0.87 for identifying generative thinking processes. (In general, an inter-coder agreement of above 0.70 is considered acceptable (van Someren et al. 1994).)

4.4 Example analysis

Using collaborative retrospective protocol analysis provided the ability to identify both internal (private) and external (shared) thoughts. Consider the example dialog and verbalized transcript/coding below, discussing a wall-mounted skateboard rack.

Collaborative dialog transcript and coding

(1) You can just use like a padlock…(2) Well, who with a skateboard carry around a padlock? What if it was like ID card swipeable? Every USD student is going to have an ID card…

Coding: (1) S(padlock), (2) S(ID card, B(swipeable)) S(student, B(has, S(ID card)))

Table 1. Coding scheme for the CTS model

Individual retrospective protocol transcript and coding

One of the things that I was thinking about when we were talking about the locking mechanism is how, in convention center back in Chicago I saw…almost like people lockup their coats in individual lock boxes and they all had ID cards that were based, it was in a convention center located in a hotel and that’s part of where the ID card idea came from.

Coding: Pr(S(locking mechanism), MR(convention, S(people, B(lock, S(coats), S(lock boxes, S(ID cards, F(access S(lock boxes))))))))

Images of the way the skateboard is locked are shown in Figure 4. The locking mechanism (ID card or padlock) would be located at the front of the arm where the arrow points.

In the example given above, it can be observed how the individual verbalization brings additional information the collaborative dialog does not reveal. The dialog only discloses specific design entities being discussed. However, the retrospective verbalization reveals the thinking process (memory retrieval) creating the new entities, the design entity that stimulated the thinking process (locking mechanism/padlock), and the type of collaborative stimulation involved (prompting).

It can also be observed how this retrospective verbalization goes into detail about the memory that was retrieved. By using collaborative retrospective protocol analysis, internal thoughts can be observed that conversation analysis would ignore. Although subjects reported being able to remember 90 % or greater of their thoughts while using this method, it should be noted there is no certain way to determine how much information is missing. In addition to the fact that not all thoughts may be verbalized (Chiu & Shu 2010), retrospective protocol analysis raises the issue of memory accuracy. It is hard to quantify how large an issue memory recall may be, or if watching a video of their interactions with a collaborator ‘creates’ memories that may not have existed. However, this uncertainty is compensated by the gain that collaborative retrospective protocol analysis provides.

5 Results

The five cases of collaborative design lasted an average of 22 min, ranging from about 10 to 40 min. Ninety-seven collaborative stimulations occurred, which resulted in the stimulation of 161 generative thinking processes. In addition to the collaboratively stimulated thinking processes, individual stimulation resulted in an additional 158 generative thinking processes. Table 2 gives the number of occurrences of each type of collaborative stimulation, the collaboratively stimulated thinking processes, and individually stimulated thinking processes occurring in each group. Note that sometimes a single collaborative stimulation results in multiple thought processes being stimulated.

Table 2. Results by groups

Although 95 % of the collaborative stimulations fit within one of the four categories, there were five occurrences of collaborative stimulation that did not fit the definitions. This is reflected in the table, where the total collaborative stimulations are greater than the summation of each instance.

In examining each type of collaborative stimulation, it was possible to observe how often a specific type of thinking process was stimulated. Figure 5 displays the percentage of time specific types of stimulation resulted in each type of thought process (memory retrieval [MR], association [AS], transformation [TR]). The results are displayed for each group and the overall average pattern. Prompting (Figure 5 top) inspired memory retrieval all the time, but infrequently inspired association or transformation. Conversely, seeding (Figure 5 upper center) and correcting (Figure 5 lower center) resulted in the stimulation of transformation all of the time, but rarely association or memory retrieval. Clarifying (Figure 5 bottom) inspired all types of the generative thinking processes examined, with a greater influence on memory retrieval (whose stimulation resulted in 40 % more often than association or transformation through clarifying). As shown in Figure 5, there was no stimulation by clarifying on any thinking process for Group 4. The reason was that no clarifying was observed for the group rather than that clarifying did not stimulate any of the thinking processes.

Figure 5. Frequencies of thought process stimulation.

There was also found to be primary and secondary stimulation of thinking processes. A primary stimulation occurred if the collaborative stimulation directly resulted in a thinking process, whereas a secondary stimulation occurred when a generative thinking process was born out of the results of collaborative stimulation (i.e., the generative thinking process would not have been possible if the collaborative stimulation did not occur). The secondary stimulations were not included in the earlier data, as only 13 instances were observed to occur.

6 Analysis and discussion

The experimental results have shown that each of the collaborative stimulation types identified earlier was observed, answering the first question raised earlier about the types of collaborative stimulation. The existence of the collaborative stimulation types was reinforced by an acceptable inter-coder reliability agreement of 0.86, and although several instances of collaborative stimulation did not fit, 95 % could be categorized into the types of prompting, seeding, correcting, and clarifying.

The second question asks for interrelations or patterns between different types of collaborative stimulations and specific thinking processes. This is addressed by Figure 5 illustrating that each type of collaborative stimulation showed definite patterns of stimulating thinking processes.

Prompting mostly stimulated memory retrieval, as would be expected. These findings align with what others have suggested: ideas from one collaborator will stimulate the memory of another collaborator (Brown et al. 1998; Dugosh et al. 2000; Nijstad & Stroebe 2006). Seeding was strongly tied to transformation, as it occurred when collaborators would transform others’ ideas. This aligns with other research as aggregate models of group creativity predict that collaborators build on one another’s ideas, which establishes the total group creativity (Pirola-Merlo & Mann 2004; Taggar 2002). Similarly, correcting resulted in transformation a majority of the time, and occasionally memory retrieval and association.

Clarifying was particularly interesting as it had moderate to strong relationships with all three generative thought processes. However, if analogies consist of a memory retrieval, association, and transformation, why does clarifying stimulate memory retrieval more often? The reason may be that clarifying tends to stimulate analogies, but not all analogies are completed or become generative. This theory is supported by previous results when only some explanatory analogies became generative (Sauder & Jin 2012). One of the first stages in an analogy is memory retrieval, and association and transformation will not follow if the analogy remains incomplete (Jin & Benami 2010). Therefore, it would be expected that more memory retrievals occur than associations and transformations. It is interesting to note that no instances of clarifying occurred in Group 4, even though they had a number of other stimulations. This could imply the frequency of clarifying, and perhaps other types of stimulation are influenced by team social dynamics. In Group 4, one designer had a controlling personality whereas the other designer was a follower. This combination may have discouraged clarifying, as one designer assumed they were correct, and the other did not attempt to defend their ideas.

Another interesting type of group dynamics was the relation between BICB scores and stimulation. As the group times vary, it is obvious from Table 2 that BICB has little relation with the total number of stimulations. However, when exploring the stimulation frequency (stimulations per minute), each group’s average BICB score had a very strong positive correlation of $r(3)=0.93,p<0.025$ . A scatter plot summarizes the results (Figure 6), with the numbers similarly indicating the team number. This implies being involved in a diverse set of creative activities may either prime a designer for stimulation and possibly enable them to be a better collaborator.

Figure 6. Collaborative stimulation and BICB results.

However, from these observations, is there a certain type of collaborative stimulation that is more effective than the others? To begin answering this question, the observed patterns (from Figure 5) can be summarized with a relationship strength matrix between the various types of collaborative stimulation and generative thinking processes (Table 3). The strength of the relationship is defined by the percentage of time a specific type of generative thought process will result from a collaborative stimulation (66 % or higher being strong, 33–66 % being moderate, and 3 % or less being weak).

Table 3. Collaborative stimulation and thinking process relationship strength matrix

$\bullet \,\bullet \,\bullet$ Strong relationship; $\bullet \,\bullet$ Moderate relationship; $\bullet$ Weak relationship.

In comparing the results to current literature, much work pertains to the design entity sharing stimulations of prompting and seeding (e.g., Brown et al. 1998; Pirola-Merlo & Mann 2004). Although most work related to the types of collaborative stimulation resulting from design entity questioning is much scarce and more vague (focusing on different issues: e.g., analogies (Goldschmidt 2011) or ontological design (Holsapple & Joshi 2002)), there is work that notes how questions can generate new ideas (Eris 2003, 2004). Often questioning is discouraged in idea generation, and particularly in brainstorming (Osborn 1957). However, in examining Table 3, it is observed that design entity questioning stimulation inspires more thought processes than design entity sharing stimulation. In particular, the collaborative stimulation of clarifying is quite powerful as, unlike any of the other types of collaborative stimulation, moderate and strong relationships exist stimulating all three generative thinking processes. This aligns with the finding of Ozgur Eris (2004), who also found that questions play an important role in design ideation, but it is an area where there is a lack of research.

The importance of questioning was reinforced in a later follow-on study by the authors, which found that question-inspired collaborative stimulation resulted in higher novelty design entities. This was partially attributed to the strong relationships those types of collaborative stimulation have with the thought process of transformation, which is required to create new (or novel) design entities (Sauder, Lian & Jin 2013). These findings suggest that questioning can be more effective than the presentation of design entities in the collaborative setting. Also, question-inspired stimulation occurred much less frequently than design-entity-inspired stimulation.

A number of methods have been developed that encourage design-entity-inspired stimulation; for example, through sharing design notebooks (Michalko 2001), sharing sketches (Shah et al. 2001), or providing stimulus to individuals (Nijstad & Stroebe 2006). However, much less time has been spent on developing effective questioning methods. The findings of this work demonstrate the importance of developing effective questioning methodology and interventions to increase generative thinking process stimulation.

7 Conclusion

In this paper, group design thinking was investigated by examining the stimulation relations between the various types of collaborative stimulation among the designers and the generative thought processes of the designers. The investigation has resulted in the identification of various types of collaborative stimulation and elicitation of the patterns between the collaboration stimulation types and designers’ generative thought processes. The work is accomplished by extending creative cognition (Finke et al. 1996) and the GSP model (Jin & Benami 2010) to collaboration through the concept of collaborative stimulation. It has added value to group creativity research, by going deeper than past studies (e.g., Pirola-Merlo & Mann 2004; Sarmiento & Stahl 2008; Stempfle & Badke-Schaub 2002) in examining individual thinking processes. This has resulted in the finding that questioning-inspired collaborative stimulation is powerful, but has been underexplored. Additionally, collaborative retrospective protocol analysis was developed by this study to explore the influence of interaction on thought processes.

The study was able to investigate the influence of interactions on specific thinking processes because of collaborative retrospective protocol analysis. Collaborative retrospective protocol analysis modifies current protocol analysis methodologies to allow for the observation of processes that are external (shared) and of those that are internal (private). It is believed that the collaborative retrospective protocol analysis extends beyond evaluating the CTS model, and is a valuable research tool for others analyzing thinking processes in the collaborative setting. Beyond research implications, this work also contributes to education and practice. The identification of collaborative stimulation, mechanisms through which they work (design entity and question inspired), and the patterns between collaborative stimulation and thinking processes assists in explaining how current practices stimulate generative thinking processes (i.e., 6–3–5 method (Rohrbach 1969)) and provides the background information for the development of new practices. However, there are several limitations when considering this study. Only the interactions of dyads were explored; the case study had a total of 10 participants, which did not allow statistically significant data; and there may be an issue of memory accuracy. Also, although the stimulation of generative thought processes is required for creativity (Finke et al. 1996), this study does not claim collaboration results in more creative products, which is dependent on a number of other factors beyond thought processes. Future work should draw correlations between collaborative stimulation and creativity. Despite these limitations, the study has yielded valuable insights into interactive cognition by developing collaborative retrospective protocol analysis, identifying specific interactions, the types of collaborative stimulation elicited from each interaction, and patterns between the types of collaborative stimulation and thinking processes.

The development of new practices is an opportunity for future work. Specifically, this aids in examining how certain interventions influence the various types of collaborative stimulation. Group creativity research has found that social inhibitions and procedural issues tend to hurt creativity, specifically in the brainstorming setting (Diehl & Stroebe 1987; Mullen et al. 1991). This raises the question: Can collaborative stimulation, specifically in the high yield area of design entity questioning stimulation, be encouraged to overcome social inhibitions and cognitive interference? Our ongoing work explores both training and tool based intervention methods for entity and questioning based stimulations. Comparing the group thinking processes, i.e., patterns of stimulation and thinking processes, with individual processes is also part of our future work.