How to assess a company’s open innovation situation?

Abstract

Open Innovation (OI) supports companies in systematically collaborating with external partners, offering various advantages. However, companies still face several challenges when applying OI, e.g., identifying relevant OI partners, collaboration methods, and project risks. Often, insufficient planning is the reason for subsequent deficits in OI projects. The analysis of relevant context factors (‘situation’) is important, which affect and constrain OI. To date, a general approach for analyzing (open) innovation situations or guidelines for developing one do not exist. Usually researchers develop their own situation analysis, including extensive literature reviews and experiencing similar challenges. This publication sets the basis for successfully planning OI projects. It focuses on developing an analysis approach for OI situations and supports other researchers in developing their own analysis approaches. The resultant objectives of the publication are to: (1) provide a list of potential situation analysis criteria; (2) provide a guideline for developing a situation analysis; (3) provide initial indications of relevant OI-specific situation criteria. The criteria were derived from the literature and qualitatively evaluated by three industry partners to assess their usability. Although this work is exploratory, and the results are not automatically generalizable, it is an important contribution for ensuring the success of OI, and for analyzing enablers and barriers to knowledge transfer from academia to industry.

1 Introduction

When developing new products and services, companies continuously face dynamic changes and a variety of challenges, such as increasingly shorter innovation cycles, continuously changing customer needs, and new international competitors (Enkel, Perez-Freije & Gassmann 2005b ; Oehmen et al. 2010). A possible way to deal with these challenges are strategic and operative collaborations with external partners by combining different expertise and resources. A current approach from the area of external collaboration is Open Innovation.

Open Innovation (OI) describes the opening of a company’s innovation process towards its environment, and using internal as well as external knowledge to create new innovations (Chesbrough 2003; Braun 2012; Chesbrough & Bogers 2014). It allows the utilization of the expertise, creativity, and capabilities of external partners, such as suppliers, customers, research institutions, companies from different industries, or even competitors (Huizingh 2010). This offers companies a variety of potential benefits, such as better customer orientation, shorter time-to-market, and the exploitation of new markets (Enkel, Kausch & Gassmann 2005a ; Enkel et al. 2005b ; Braun 2012).

1.1 Challenges in Open Innovation and motivation for research

However, opening an innovation process in terms of OI also entails several challenges itself, such as knowledge drain, a reluctant attitude on the part of employees in the form of the ‘Not-Invented-Here syndrome’, and other barriers and risks (Enkel et al. 2005a ; Enkel 2009; van de Vrande et al. 2009; Lang et al. 2011; Lindemann & Trinczek 2011; Guertler, Holle & Lindemann 2014b ). An analysis of these challenges and risks reveals that the majority of them are related to the insufficient planning of the OI project, for example, with respect to the choice of OI partners (choosing non-beneficial ones or neglecting relevant ones) and the selection of OI collaboration methods (e.g., favoring ideation contests and neglecting supplier workshops). In particular, inexperienced OI teams from industry and academia often apply OI the first time using a trial-and-error approach, resulting in insufficient project outcomes. It is crucial to analyze specific context factors and boundary conditions, which set the framework and constraints for the OI project. Or as Huizingh (2010, 5) concretizes: ‘How to do it?’ and ‘When, how, with whom, with what purpose, and in what way should they [the company] cooperate with outside parties?’

However, this is not an exclusive issue for OI, but rather a general challenge for innovation projects. A central aspect of planning innovation projects is the analysis of the company-specific and project-specific boundary conditions and context factors (the ‘situation’). This leads to the following question: With regard to which criteria can such a situation be analyzed? To date, besides some exceptions such as (Gericke, Meißner & Paetzold 2013), no universal approaches exist, neither for OI nor for innovation projects in general. Usually each research group works on its own, i.e., the group defines its situation criteria intuitively or spends much effort on reviewing literature. When applying their situation analysis in practice, they most likely experience the same challenges and problems that we experienced in our research. In summary, besides a pool of potential situation criteria for reducing the effort needed to review the literature, there is also a lack of guidelines for defining situation criteria and developing a situation analysis.

This publication aims at closing this gap. We reviewed literature regarding existing approaches for analyzing (open) innovation situations as well as situation criteria which were indicated as relevant for (open) innovation projects. These criteria were clustered and evaluated by three industry partners with respect to their usability and measurability within an industry context. Both the list of potential criteria and the results of the usability evaluation can be utilized as a starting point by other researchers and can reduce the effort required for their own literature reviews as well as help avoid common mistakes. By using a matrix analysis, we also identified situation criteria that are especially relevant for selecting suitable OI collaboration methods. This serves as basis for the future identification of OI-specific key criteria. Due to the qualitative character of our evaluation, our work is limited in terms of generalizability. Nevertheless, it provides valuable indications for relevant situation criteria as well as enablers and barriers to introducing such a situation analysis to industry.

1.2 Aim of publication and contribution

This publication focuses on two goals:

• ∙ G1: Developing the basis for an efficient analysis of (open) innovation situations to allow the successful planning of OI projects;

• ∙ G2: Supporting researchers in developing a situation analysis.

In order to address these goals, we will first give a brief introduction to Open Innovation in general, Situative Open Innovation (SOI) in particular, and existing approaches for analyzing innovation situations in Section 2. Section 3 describes the four categories of situation criteria and the structure of the relevant criteria lists contained in the appendix A. Section 4 presents the results of the industry evaluation of the criteria’s usability, the results of the matrix-based OI-relevance analysis of situation criteria, and the derived key learnings for developing a situation analysis. After discussing the limitations of the current research in Section 5, Section 6 summarizes the presented research and provides a short overview of future research tasks.

Figure 1 illustrates the resultant focus of the publication within the overall research goal, i.e., the successful execution of OI projects. To this end, this publication sets the basis for the medium-term development of the SOI methodology. It supports OI teams from industry and academia in successfully planning OI projects. A matrix-based analysis identifies OI criteria, which are relevant for a heuristic approach to selecting suitable OI methods. Future observations of OI projects in industry will also provide indications of relevant OI-specific criteria for reflexive use.

Figure 1. Overall goal and focus of publication.

To ensure future utilization of the situation analysis, especially in industry, we evaluate the usability of our situation criteria with regard to comprehensibility and measurability. Besides improving the pertinent criteria, we also derive a guideline for developing a criteria-based situation analysis. This supports other researchers in defining their own situation criteria. As an additional support, this publication provides a large pool of potential situation criteria, extracted from the literature on innovation. This can be used as a basis and reduce the efforts needed for the researcher’s or team’s own literature review.

1.3 Research design

This publication focuses on the assessment and characterization of an OI situation by using criteria. To achieve the goals (G1, G2) of this publication, the resultant research questions are:

• ∙ (G1) How can an (open) innovation situation be assessed using criteria?

• ∙ (G1) Which situation criteria are especially relevant in the context of Open Innovation?

• ∙ (G2) What needs to be considered from an industry point of view when defining situation criteria?

• ∙ (G2) How can (future) researchers be supported when developing a situation analysis and defining situation criteria?

Through a review of the literature, existing approaches for analyzing innovation situations as well as potential criteria for describing an (open) innovation were identified. The applied search terms were combinations of situation, context, analysis, innovation, Open Innovation, design, product development, and planning. Using this approach, we collected and discursively selected all criteria which describe the context of an innovation or R&D project. Based on a categorization structure presented in Guertler et al. (2014a ), the situation criteria were clustered into four categories. To evaluate the comprehensibility and measurability of these criteria, interdisciplinary teams of three industry partners assessed them in independent workshops. This facilitated both improvement of the criteria as well as the derivation of general guidelines for defining situation criteria.

Subsequently, we mapped preselected situation criteria onto criteria from a method model of OI methods (Guertler et al. 2015) by using a Domain Mapping Matrix (DMM) (Danilovic & Browning 2004). This allowed the identification of situation criteria that are relevant for selecting suitable OI methods.

Principally analyzing dependencies between situation criteria by using a Design Structure Matrix (DSM) (Steward 1981) and deriving criteria clusters is possible. However, we did not apply it to our research, since we see the risk that criteria clusters might be too rough and might disguise the effects of single criteria. The clusters might have a different effect than single criteria, whose effects might be neutralized or overframed by the other criteria in the cluster. Therefore, we focus on reducing the amount of relevant criteria by analyzing their particular effects on OI methods as well as differentiating them into heuristic criteria and reflexive criteria.

2 State of the art

This section provides an overview of Open Innovation (OI) in order to describe the broader scope of this work in the existing body of knowledge: from the increasing emergence of collaborative innovation methods at the end of the 20th century and formal introduction of OI in 2003 up to today. A brief introduction into the methodology of SOI shows how the analysis of OI situations can be integrated into a recent OI approach. One of the reasons why SOI is a suitable framework is its explicit focus on an OI project and company-specific context to ensure the success of OI. Unlike other approaches, it emphasizes the context-specific planning phase prior to the application of OI methods, since failures of OI projects are often caused by insufficient planning. Finally, Section 2.3 presents existing approaches for defining and assessing innovation situations from the fields of OI, innovation management, product development, and design.

2.1 Open Innovation

Open Innovation (OI) was introduced by Chesbrough (2003) and describes the opening of a company’s innovation process to its environment to allow the purposeful exchange of knowledge (Chesbrough, Vanhaverbeke & West 2006; Dittrich & Duysters 2007; Gianiodis, Ellis & Secchi 2010; Braun 2012; Chesbrough & Bogers 2014). Internal and external knowledge are considered equal in value and usability (Chesbrough 2003). The basis for this knowledge exchange is the purposeful collaboration with external partners (West & Gallagher 2006), such as suppliers, customers, research institutes, companies from other industries or even competitors (Huizingh 2010). In this regard, OI is not a completely new phenomenon, since there are several examples of early precursor forms of OI, which date back long before 2003 (Trott & Hartmann 2009; Huizingh 2010). Thus, we consider OI to be a new methodical framework to structure existing approaches and methods of collaborative innovation, such as the Lead-User approach (von Hippel 1986) or distributed product development, e.g., Hameri & Nihtilä (1997). OI also supports the purposeful enhancement of this by adding new approaches, methods, and tools. In addition to the involvement of individuals and specific companies, OI also fosters the collaboration with unspecific groups of external partners (‘crowds’) from all over the world by using web technologies. In contrast to traditional market research, external partners participate actively and creatively in the innovation process (Hilgers & Piller 2009). The ongoing academic dispute between Chesbrough’s company-centric definition of ‘Open Innovation’ and the individual-centric ‘Open User Innovation’ (von Hippel 2005) is excluded at this point. Interested readers may refer to Chesbrough, Vanhaverbeke & West (2014).

Opening up an innovation process in terms of OI is not a binary decision. A company needs to carefully analyze and select suitable process phases or projects for OI. The suitability for OI depends on different influencing and context factors, e.g., phase of product lifecycle (PLC), competitive market situation, need for concealment, internal processes. To stress the situation-dependent differing levels of openness, Dahlander & Gann (2010, 699) introduced the term ‘permeability’. Considering the flow of knowledge (Gassmann & Enkel 2004), three types of OI can be distinguished, which are consistent with a differentiation based on the location of the innovation (Chesbrough & Crowther 2006; Chesbrough & Bogers 2014):

1. (1) outside-in/inbound OI: transferring external knowledge into the company, strengthening the internal innovation process, and enabling internal innovations,

2. (2) inside-out/outbound OI: giving internal knowledge to the company’s environment to allow external innovations and exploit new markets, and

3. (3) coupled OI: a combination of the above two types.

In addition to this, other categories of differentiation can be also found, such as pecuniary/non-pecuniary (Dahlander & Gann 2010) and the role of the company (Gianiodis et al. 2010).

2.2 Situative Open Innovation

In the context of this research, SOI is a methodology for planning OI projects (Guertler et al. 2014a ). The goal of SOI is to support teams from industry and academia with a methodology that can either be applied for autonomously planning OI projects or to allow a better evaluation of external service offers. The strategic decision for or against OI as well as the execution of the OI project itself are excluded. SOI consists of five phases, which are grouped into two areas, as illustrated in Figure 2. The outer ring represents the rough planning of the OI project, addressing the question: Who to involve, in which way and why? As illustrated by the arrows, SOI allows iterations, e.g., when new information is available or boundary conditions change, or when matching OI partners and OI methods. To ensure purposeful iterations, stage gates assess and control the planning progress (Cooper 2001). The inner area represents the detailed planning, including aspects such as acquisition of OI partners or specific starts and durations of OI methods.

Figure 2. Situative Open Innovation (Guertler et al. 2014a ).

The first phase of SOI 1 is the analysis of the characteristics of the OI project and of the company’s internal and external context factors, such as organizational aspects or dynamics of competitors. Their assessment is the focus of this publication. Based on SOI 1, the SOI 2 phase identifies existing stakeholders as well as new potential OI partners. Potential OI partners are assessed, ranked and selected with regard to their relevance to the OI project. Based on the OI situation and selected OI partners, suitable OI methods are derived in SOI 3 and adapted to the specific project situation. SOI 4 focuses on defining the performance assessment, controlling and risk management of the OI project. SOI 5 comprises the detailed planning of the rough project structure defined in the previous phases, e.g., the acquisition of OI partners or the specific start date of an OI method.

2.3 Situations in innovation management and product development

The basic element of SOI is the analysis of a company’s ‘situation’, i.e., context factors that influence and constrain the OI project. As already stated by Birkhofer, Jänsch & Kloberdanz (2005, 9f) and their ‘10 Commandments’ for product design, it is essential to ‘meet the design situation’. Different authors state the relevance of the situation-specific selection and adaption procedures for product development and innovation management in general (Birkhofer et al. 2002; Gericke et al. 2013; Lindemann 2009, 29; Ponn 2007, 43). However, besides its dynamic character, the term ‘situation’ itself is complex and a general definition is missing (Ponn 2007, 43), which makes the literature review challenging. In addition, while some sources explicitly name criteria and applicable values for assessing a situation (e.g., ‘form of governance’ (Bevis & Cole 2010, 6)), others only mention influencing factors of the innovation performance, which can be used to derive situation criteria (e.g., ‘centrality of R&D’ (Gassmann, Enkel & Chesbrough 2010, 213)). Other sources represent case studies, describing specific OI projects, and can also be used to derive situation criteria, e.g., von Hippel (1988). As a ‘mixed type’, some authors mention situation-based criteria, but no information about their assessment (e.g., ‘innovativeness’ (Hauschildt & Salomo 2005), or ‘company culture’ and ‘hierarchies’ (Ponn 2007, 55)). A partial exception is presented by Gericke et al. (2013). They analyze different definitions of the term ‘context’ and present a large collection of different context influencing factors, which might be relevant for the adaption of design methods. However, they neither provide detailed information about those influencing factors nor direct references to primary sources. Besides a lack of an OI focus, there is no analysis of the relevance of the factors in the context of adapting methods. Although the consideration of an industry perspective is stated as being crucial for a successful long-term application, it is missing from the literature so far.

Therefore, in the following, we will analyze different approaches for assessing situations in product development and OI. Based on this, we derive a definition of OI situations. At this point, we will focus on a general description of situation analysis, while single criteria and their references are presented in Section 4.

2.3.1 General definition of situation

Ponn (2007, 44), based on Brockhaus (1996, Band 20, 274), defines a situation as ‘a state or sum of all current circumstances and relationships’, referring to the Latin word ‘situs’, meaning position/condition. A closely related term is ‘context’, which is defined as ‘coherence, background, and periphery’, based on the Latin verb ‘contextere’, meaning to closely link (Brockhaus 1996, Band 12, 328). Both terms are often found in linguistics and philosophy.

2.3.2 Design situations

In the field of product development, the term ‘design situation’ is used (Ponn 2007, 44), based on Demers (2000, 3), Hutterer (2005, 29) and Zanker (1999, 4). A general definition can be found in Lindemann (2009, 336): ‘A situation is a point in the development process, which requires corresponding actions/decisions by the product designer; influenced by a multitude of factors (personal influencing factors, type of task and demanded results, external boundary conditions)’ (translated from German). In the English literature, Reymen (2001, 56), for example, defines a design situation ‘at a certain moment as (…) the combination of the state of the product being designed, the state of the design process, and the state of the design context at that moment. This means that it is the set of values of all properties describing the product (…) the design process, and (…) all factors influencing the product being designed and its design process’. Along with this, he stresses the dynamic character of a situation, which changes over time (Reymen 2001, 52f). In addition, some concrete criteria are mentioned, such as ‘budget’ and ‘maximal duration’ (Reymen 2001, 85). As already indicated by Lindemann (2009, 336), a design situation can be structured into different levels and perspectives, e.g., a strategic long-term level, a project-specific medium-term level, and an operative short-term level (Meißner et al. 2005, 73). Furthermore, special cases, such as critical situations, can be found (Badke-Schaub & Frankenberger 2004).

2.3.3 Descriptions of Open Innovation situations in the literature

An explicit definition of ‘Open Innovation situation’ is missing in the literature. Generally, only abstract statements about ‘situations’ can be found, such as: ‘A blanket approach (…) is unlikely to provide an optimal solution to these trade-offs, because each technology and market situation is different.’ (Fabrizio 2006, 158). Becker & Zirpoli (2007, 6) even call OI itself a situation: ‘Because the ‘open innovation’ situation is more complex (…)’.

Looking at the broader term ‘innovation situation’ does not reveal a distinctive definition either, but rather abstract statements about situations. However, although there is a lack of a definition, some publications still address the issue of assessing innovation situations. For instance, Sarkkinen & Kässi (2013, 4) define four categories, including specific values for assessing and characterizing the innovation situation of companies in rural regions of Finland: (1) innovation activities, (2) innovation types, (3) innovation goals, and (4) innovation barriers. Other researchers, e.g., Kline & Rosenberg (1986), focus on the innovation production process with their Chain-Linked Model of innovation, which was later expanded on by other researchers such as a Micaëlli et al. (2014). They focus on an innovation system, which they consider to be a ‘network of complementary components (actors, processes, institutions, etc.)’ in different geographical and juridical contexts (Micaëlli et al. 2014, 60).

Nevertheless, criteria for assessing and characterizing OI situations can be derived from (open) innovation literature, which can be differentiated into the following categories: The first category of literature addresses the implementation of OI projects, such as an abstract sequence plan for implementing OI in a company (Ili 2010a ) and implementing OI with a focus on the change process from closed to open innovation (Chiaroni, Chiesa & Frattini 2010, 2011). The second category consists of empirical studies analyzing different characteristics of OI, e.g., OI in South Korea (Abulrub & Lee 2012). Closely related, the third category comprises papers about empirical case studies, which analyze influencing factors of the OI performance, see Enkel (2009); Enkel et al. (2005a ), focusing on chances and risks of OI, and Lindemann & Trinczek (2011), who analyze industrial collaboration projects for best and worst practices as well as the underlying reasons for them. In the fourth category are papers focusing on specific OI methods and OI types, such as lead user (von Hippel 1988, 2005), crowdsourcing (Piller & Ihl 2009; Pirker et al. 2010; Sloane 2011), cross-industry OI (Enkel & Gassmann 2010; Echterhoff 2014), and inside-out (Inauen & Schenker-Wicki 2012). The fifth category comprises publications with a broader focus of innovation management aspects, such as inter-firm collaboration (Hagedoorn 2002; Hagedoorn & Cloodt 2003), absorptive capacity (Cohen & Levinthal 1990; Schmidt 2005), and the relationship between innovativeness and performance (Hauschildt & Salomo 2005). Since, e.g., Ertl (2010, 62) stresses the importance of the strategic goal and environment of an innovation project, publications are also included which address the company strategy (Porter 1985) and human resource management (Rastetter 2006).

2.3.4 Developing a definition of Open Innovation situations

To enable a sufficient assessment and characterization, a definition of an OI situation is required. This serves as a basis for determining suitable criteria and appropriate values. Based on the definition of a situation in general and a design situation in particular, our definition is as follows:

In terms of SOI, a company’s OI situation is a set of internal and external context factors, boundary conditions and characteristics of the OI project, which set the specific and dynamic constraints for an OI project and are assessed using criteria.

3 Determining an OI situation

For efficient future use, the criteria are clustered into four categories (Guertler et al. 2014a ). They combine common categories mentioned in the literature with findings from an industry study (Guertler et al. 2014b ) and consolidate the relevant findings. In the following, the categories and the structure of the criteria lists are explained. The lists themselves are presented in the appendix A to this publication, to allow utilization by other researchers.

A differentiation into company-internal and company-external influencing factors is proposed by several authors, e.g., Ertl (2010, 70f); Huizingh (2010, 4); Sarkkinen & Kässi (2013, 2). Based on the experience from a previous study (Guertler et al. 2014b ), we consider the company’s collaboration experience with externals as its own category, since it strongly influences the employees’ motivation for collaboration. Hilgers et al. (2011, 89) stress the importance of defining the goal of OI for its success. Thus, we also include the specific characteristics and goal of the OI project as an additional category. The OI goal directly affects the selection of suitable OI partners and OI methods. The collaboration experience influences the employees’ attitude towards external partners and collaboration methods, and therefore requires different incentive strategies.

The situation analysis can be conducted on different levels, see (Meißner et al. 2005, 73), which are comparable to the ‘levels of resolution’ from Hales & Gooch (2004): a long-term strategic level, a medium-term project-specific level, and a short-term operative level. In our approach, we slightly adapt those levels: While the company-internal and company-external criteria describe the general long-term situation of a company, the collaboration experience is dependent on the business area and OI project team, and needs to be updated more frequently in a medium-term perspective. The fourth category needs to be assessed independently for each OI project.

The four situation categories are illustrated in Figure 3. The criteria can be further differentiated with regard to their future application cases in the context of SOI: (1) reflexive utilization, which allows the discussion and documentation of (partly) implicit knowledge and ensures a homogeneous level of knowledge within the OI team; (2) heuristic use to assess and rank OI methods with regard to their applicability for the OI situation.

Figure 3. Four categories of an Open Innovation situation (based on: Guertler et al. 2014a ).

Figure 4. Structure of the evaluated criteria lists (see appendix A).

3.1 Criteria categories for describing an OI situation

The following section presents the structure of the evaluated situation criteria lists, grouped into the four categories derived from the literature as described above. The lists themselves can be found in the appendix A. The structure of the lists aims at supporting easy use by other research groups. Besides the criterion’s name, the tables also contain a criterion ID for a clear identification, a short description of the criterion, potential values based on the literature and evaluation, evaluation remarks, i.e., comments from the workshops and resultant activities, relevant references, and application cases within SOI.

Figure 4 illustrates the basic structure of the criteria lists. Criteria highlighted in dark gray and indicated by an ‘M’ in the application case (AC) column are part of the OI method selection tool. Light gray highlighting indicates criteria which were tested for the DMM situation method mapping, but showed a too weak correlation (for details, see Figure 6). Criteria for reflexive use are marked with an ‘R’. ‘N/A’ indicates criteria that were evaluated as being not applicable and were ‘canceled’ (omitted)/removed from the criteria set.

3.1.1 Company characteristics

Company characteristics assess the internal structure, state, and condition of a company at a strategic, organizational, and cultural level (Guertler et al. 2014a , 1544). They set the framework for innovation activities and projects within the company. For instance, the company strategy influences the OI project, since it states whether a company is primarily focused on finding a highly radical innovative solution or an incremental solution. Some criteria are closely linked to environmental or collaboration criteria, but are considered more relevant for the entire company.

The category is sub-structured into: (1) company details, such as age of the company (Barge-Gil 2010); (2) size of company and responsible business unit (Chiaroni et al. 2010); (3) the strategic orientation of the company (Porter 1985); (4) the R&D intensity, e.g., annual expenses for R&D (Laursen & Salter 2006); (5) the degree of internationalization (Gassmann 2006); (6) employees, such as their innovativeness (Hauschildt & Salomo 2005); and (7) the company’s management, such as durability of decisions (Verbeck 2001). The full list of criteria is shown Figure 7.

3.1.2 Company’s environment

This category assesses a company’s market environment on a strategic level (Guertler et al. 2014a , 1546). It defines the external boundary conditions and constraints for a company’s innovation activities. Besides general aspects, such as industry and market dynamics, the number and strength of customers, suppliers and competitors are also assessed. In particular, the competitive situation defines the need for the concealment and maximal openness of OI. In addition, global influencing factors such as laws, standards, and norms are considered.

The category is accordingly sub-structured into: (1) the company’s industry sector (Huizingh 2010); (2) market characteristics, e.g., market dynamics (Chiang & Hung 2010); (3) customer characteristics, such as their variety (Porter 1985); (4) external regulations, including compulsory certifications (Gassmann et al. 2010); (5) innovation cycle in industry (Bevis & Cole 2010); (6) general influencing factors, such as specific influence groups (Porter 1985); and (7) the competitive intensity (Porter 1985). The full list of criteria is shown Figure 8.

3.1.3 Collaboration experience

This category assesses the collaboration experience of the pertinent business unit (Guertler et al. 2014a , 1545). Considering the collaboration experience of employees is crucial for the success of an (open) innovation project: Positive experience can result in high intrinsic motivation. Negative experience with particular external partners, with OI, or with external collaboration in general requires specific incentive strategies to prevent operational barriers and opposition by employees (Stolzenberg & Heberle 2009, 4f).

The category is sub-structured into: (1) existing external partners, such as number of universities (Dahlander & Gann 2010); (2) the particular Open Innovation experience (Gassmann et al. 2010); (3) the general collaboration experience with externals, e.g., the type of collaboration (Dahlander & Gann 2010); (4) the general employee mindset (Bevis & Cole 2010); and (5) the existing infrastructure, such as web platforms (based on feedback by industry partners). The full list of criteria is shown Figure 9.

3.1.4 Topic/goal of Open Innovation project

This category characterizes the topic and goal of the OI project (Guertler et al. 2014a , 1545). The type of suitable external OI partners and OI methods differs depending on the intended type of innovation (product, process, etc.), the level of innovation (radical, incremental) (Boscherini et al. 2010, 1073), or the PLC phase and other aspects. While crowd involvement with an idea contest is useful for generating a large quantity of ideas in an early PLC phase, a cross-industry workshop might be a more suitable way to identify and adapt process concepts in the production phase. Other boundary factors and constraints include available resources, the project-specific need for concealment, and the strategic allocation in the company.

The category is sub-structured into: (1) the goal of the OI project (Ertl 2010); (2) the expected innovation, such as the type of innovation (Chiang & Hung 2010); (3) the modularity of the innovation object (Baldwin & von Hippel 2011); (4) the specific boundary conditions of the OI project, e.g., the available time (Reymen 2001); and (5) the organizational context of the OI project, such as the composition of the OI team (Enkel et al. 2005b ). The full list of criteria is shown Figure 10.

4 Evaluation of criteria for (open) innovation situation analysis

The following section presents the results of an initial evaluation of the usability of situation criteria and success factors for developing a situation analysis in an industrial context. By subsequently using a matrix-based approach, relevant criteria for selecting suitable OI methods were identified. While Section 4.1 addresses both research goals G1 and G2, Section 4.2 focuses on goal G1 and Section 4.3 on goal G2.

4.1 Industrial evaluation of usability and measurability

To evaluate the criteria’s usability in terms of comprehensibility and measurability, we evaluated them with three industry partners. The evaluation was conducted in three half-day workshops with small and medium enterprises (SMEs) that were not experienced in OI (in the focus of SOI); the SMEs are in the machine and plant engineering sectors: (1) a supplier of mechanical automotive parts, (2) a manufacturer of production machines and related services, and (3) a manufacturer of building technology products. Each workshop team consisted of three to four employees (specialists and managers) from different departments. In the case of the first company (1), the participants were: the director of corporate development, the manager of the relevant specialized department, an R&D engineer from that same department, and a R&D engineer from a central innovation department. For the second company (2), the OI team consisted of the manager and an engineer from the advance development department, an R&D engineer from the relevant specialized regarding specialty department and an expert from the purchasing department. The OI team from the third company (3) included two managers from the specialized department with differing areas of responsibilities and an expert from that same department, who had previously also worked in sales for a couple of years.

The teams had to fill in the questionnaire lists autonomously prior to the workshop and indicate problematic criteria. This allowed a comparison of expected criterion values with values assessed by the teams from the companies. The evaluation in general focused on practical significance, comprehensibility, and measurability of criteria. Based on this, criteria were modified, concretized, added or removed, and criterion values defined if possible. A cross-case comparison led to the main conclusions. The few cases of different opinions were discussed in a meeting with all participants involved in the situation analysis.

The underlying questions were:

• ∙ Which criteria are useful from a company’s point of view?

• ∙ Which criteria need to be improved in terms of comprehensibility and distinctiveness?

• ∙ Which criteria necessitate major effort in terms of data acquisition, or might be problematic due to other reasons, such as data privacy?

• ∙ Which criteria should be added to the existing list?

4.1.1 Criterion-specific feedback

In the following, we present the criteria-specific results from the evaluation, i.e., which criteria were modified due to which reasons (indicated by an ‘M’ in the criteria list), which were removed/canceled (‘C’), and which were added (‘N’ for new). Unchanged criteria are marked by a ‘U’.

Table 1 shows criteria that were modified as a result of the industry partners’ feedback as well as the underlying reasons.

Table 1. Modified situation criteria.

Criteria which were canceled/removed from the criterion list are depicted in Table 2.

Table 2. Canceled/removed situation criteria.

New criteria added during the workshops are listed in Table 3. The final tables of criteria are depicted in the appendix, Figures 7–10.

Table 3. New, added situation criteria.

4.1.2 General feedback

A central aspect was the total effort needed to assess the criteria. As an orientation, the companies stated half a day to one day as a temporal limitation for completing the situation criteria list. Besides the effort resulting from the multitude of criteria, the companies also mentioned the effort needed to assess the criteria, for which they did not have the pertinent data and needed to ask colleagues. They experienced it as discouraging when they wanted to answer the criteria questions, but were not able to do so. In this regard, they suggested categorizing criteria with respect to the group of stakeholders in a company who might have the pertinent data: e.g., controlling/marketing for company characteristics and environment, the responsible department for collaboration experience, and the OI project team for the OI goal.

For other criteria, the intended focus level within the company was not clear, i.e., whole enterprise, business units, project team, etc. Thus, the respective criterion’s value could vary greatly. In this regard, the companies suggested a differentiation into a company and business unit perspective, e.g., ‘number of employees in R&D’ (1.5.4 and 1.5.5). In addition, the temporal focus should be defined as well, e.g., number of current suppliers or within the last ten years.

For a potential future implementation of the criteria list as an electronic questionnaire, they stressed the importance of a progress bar or similar measure to provide an overview of the total and the remaining effort needed. Based on their experience with survey studies, a never-ending and intransparent questionnaire was very discouraging. In the worst case, this might lead to stopping the questionnaire, since the person did not know there was only one page remaining.

Another aspect was to clearly stress the benefit of each criterion for a later application. If the respondent understands the big picture and the project’s relevance for the situation analysis and each criterion, the intrinsic motivation to address it will increase, i.e., which aspect or decision of the OI project is influenced by a criterion.

In the workshops, a sufficient definition of criterion value scales proved to be essential. For the initial autonomous response phase by the companies, some criterion values had been left empty or only filled in with a rough description (e.g., ‘free values’) due to a lack of scales in the literature. The intent was to derive the scales based on the companies’ answers. This sometimes led to both unexpected answers that were not aligned with the intended meaning of the criterion as well as too much effort by the companies. For instance, the criterion ‘number of competitors’ (2.7.1) was only intended to get a rough impression of whether there was only one major, a few, or a multitude of relevant competitors. However, one company spent a lot of effort researching the number to be as precise as possible. In terms of the use of Likert scales (very low, low, medium, high, very high), it proved to be important to provide reference information. For example, each company rated the ‘need for concealment’ as high or very high (2.7.4). However, in one case it meant that no external stakeholders should know about developed product concepts, while in another case it meant that nobody should even know that the company was conducting an OI project in a specific field. By providing reference values, the resultant assessment is more precise.

4.2 Matrix-based identification of relevant criteria for selecting OI methods

To achieve our overall goal of successfully planning OI projects and our research goal G1 in particular, it is necessary to identify those criteria which are especially relevant for OI. By using a matrix-based approach, we identified criteria that are relevant for selecting suitable OI methods. We derived 27 situation criteria. The full process in the context of developing an OI method selection tool is described in Guertler et al. (2015). In the following, we focus on deriving the OI-relevant criteria.

By means of a workshop-based academic discussion, we pre-filtered the OI criteria list by criteria which provide only background information and do not support the selection of an OI method, such as ‘name of company’, ‘year of founding’, or ‘compulsory certifications’. The resulting 52 situation criteria were subsequently mapped onto OI method criteria by using a Domain Mapping Matrix (DMM) (Danilovic & Browning 2004). First, the mapping was conducted on a criteria level to identify relevant dependencies. In a second step, the identified dependencies were further detailed by mapping particular criteria specifications, as illustrated in Figure 5.

Figure 5. Two-step DMM mapping of OI situation and OI method criteria.

The criteria-level mapping was conducted independently by five members of our academic research team, using a four-step scale with zero (no link), one (weak link), two (medium), and three (strong link). Subsequently, the single DMMs were added up, resulting in mapping values ranging from zero to 15. To reduce the influence of subjectivity and singular perceptions, we analyzed a minimum mapping value. Assuming at least a medium-rated mapping value for each assessor, we derived ten as an initial minimum value. To validate this value, we analyzed all mappings with a value between eight and eleven with a detailed mapping on a criteria specification level, as shown on the right side in Figure 5. This revealed that a specification-level mapping is possible for mapping values of nine, but not below. Accordingly, we consider nine to be the minimum mapping value.

Figure 6 shows the pertinent criteria-level DMM. Matrix cells with a mapping value equal or higher than nine are highlighted in gray. Situation criteria are considered relevant for selecting OI methods if the respective matrix rows contain at least one mapping value equal to or higher than nine. An analysis of the excluded situation criteria revealed that (a) they are criteria which only indirectly affect the selection of OI methods, such as (1.6.1) ‘Innovativeness of employees’ and (4.5.2) ‘Strategic location of OI project in the company’, or (b) they address projects in the past, but not the current one, such as (3.3.2) ‘Type of external partners’ and (3.3.4) ‘Type of cooperation’.

Figure 6. DMM mapping of OI situation and OI method criteria (gray: relevant criteria and cells).

Other criteria were excluded as well, although they contained mapping values above the mapping limit. The main reasons were that they (a) are too fine-granular for the selection process, such as (4.1.3) ‘Secondary OI project goal’, or are quite similar to other criteria, such as (3.5.2) to (3.5.1), (4.2.2) to (4.2.1), (4.2.5) to (4.1.2), and (4.4.5) to (2.7.4).

The number of relevant mapping cells are not evenly distributed over the situation criteria. The majority contains only a low number, a minority contains a large number of mappings, which are especially criteria characterizing the OI project.

4.3 Key learnings and guidelines for developing a situation analysis

Summarizing the results from the industrial evaluation workshop and the DMM mapping, we derived the following key learnings, which contribute to our research goal G2. They provide guidance and support to research teams when developing a situation analysis:

• ∙ Is the assessment effort manageable? Optimally, there are no more than 30 criteria, and/or an analysis duration of one day maximum. Too many criteria can demotivate the industry team and lead to a boycott of the situation analysis.

• ∙ Who is able to assess the criteria? The industry partners stated that it could be very demotivating to receive a large number of criteria questions which they are not able to answer themselves, but need to identify colleagues who can. Therefore, it is helpful to state a potential department of the company that is likely to be able to provide the relevant information. In this case, the industry partner can directly forward the criteria to suitable departments and employees.

• ∙ Do you provide an overview of the analysis process? It is important to give an overview of the entire situation analysis process in the beginning as well as continuous updates about the current position in the process and the remaining steps. This lets the industry partners schedule the analysis sufficiently and maintain an overview of the progress.

• ∙ Do you provide a description of each criterion? Often the criteria names are not comprehensible in themselves and require a short description. We also recommended providing a short indication of why and where the criteria will be relevant in the subsequent steps of the methodology.

• ∙ Do you prevent ambiguity? Especially when working with interdisciplinary teams, terms and expressions can have varying meanings or interpretations, e.g., (2.5) ‘Innovation cycles in industry’ could be interpreted as times between two technologies as well as the time period for developing a new product.

• ∙ Are you aware of the dynamics of criteria properties? Generally, the situation criteria assess only one situation state of one particular project. For a subsequent project or even during the progression of a particular project, some criteria might change. To reduce the risk of misplanning, you should identify the most dynamic criteria as well as define a schedule for checking for changes in the properties.

• ∙ Do you use time-independent criteria? Dynamic changes of criteria properties pose a challenge in the situation analysis. Therefore, it is important to define the criteria themselves in a time-independent manner in order to minimize obsolescence, e.g., instead of assessing the age of a company, it is better to assess the year of founding.

• ∙ Do you use defined property scales for each criterion? To reduce effort and the risk of obtaining unusable data and demotivating the industry partners, distinctive property scales should be used. Often property ranges are sufficient, but without corresponding scales, companies might try to assess the criteria as precisely as possible and spend too much unnecessary time and effort.

• ∙ Do you provide references for Likert scales? Often scale properties such as ‘low’, ‘medium’, and ‘high’ are not absolutely defined. For example, the majority of companies might assess (2.7.4) ‘need for concealment’ as ‘high’, but in one case that could mean publishing product concepts are critical, and in another, it could even include mentioning the company’s name. Consequently, providing a reference example for each scale element is beneficial.

• ∙ Do you clarify the organizational and temporal focus level? Some criteria can be assessed on different levels and for different time periods. To avoid ambiguity, it is important to specify the focus level, e.g., OI experience on the level of the company, the business unit, the department, or the team. Define if you are assessing the present, the last 10 years, or other time periods.

• ∙ Do you consider the specific subjectivity of criteria? Some criteria have a high level of subjectivity such as (3.4.1) ‘Employees’ attitude towards external partners’. In those cases, an assessment by additional persons or substitution by other, more objective criteria might be beneficial.

• ∙ Are you aware of inherent risks in the context of individual-related data? Along with subjectivity and dynamics, individual-related data is critical. It is important to check if it is legal to even assess that data, how to ensure confidentiality, and how to prevent offending persons when seeing the assessments (e.g., innovativeness (1.7.1)).

• ∙ Are you aware of potential strategic risks of the criteria? Companies might refuse to assess criteria due to their high strategic relevance and resultant risks if information, e.g., profits or expenses for R&D, gets into wrong hands. To avoid unassessed criteria, you should check why you need those criteria. It might be possible to obtain the underlying information with other, less critical criteria.

5 Discussion and limitations

Besides the benefits this publication offers, it also has a few limitations which need to be addressed in future research. A central aspect is the exploratory character of our research. Since the qualitative evaluation is based on only three companies, our results cannot automatically be generalized. In view of this, future research should both further examine the usability and effects of a situation analysis by in-depth case studies as well as quantitatively evaluate the effect of different situation criteria. A section containing all or a subset of our identified situation criteria can easily be added to any future study, which allows the analysis of the criteria’s effects on different (performance) aspects of Open Innovation.

Despite the broad literature review, the completeness of the criteria list cannot be confirmed. A major reason is the inconsistent use of keywords by different authors. While some speak about ‘situation analysis’, some use ‘context analysis’, while others use ‘influencing factors’. Still, we think we covered the most relevant ones. Another challenge is the already large number of potential situation criteria which are not manageable for an industrial application and are not all relevant to OI. Thus, an analysis of the criteria’s OI-relevance is crucial. Within this publication, we consider their relevance regarding the selection of suitable OI methods – focusing on criteria with quantifiable specifications. However, qualitative criteria and criteria without a direct link to OI methods are important factors for a successful OI project as well. They define the strategic project’s constraints, for example. By discussing them, they also foster the explication of implicit knowledge and ensure a homogeneous knowledge level within the OI team. In our industry workshops, this was experienced as a benefit, especially for interdisciplinary teams or teams with new employees. The general OI-relevance as well as the relevance for specific OI aspects (such as IP or risks) of situation criteria needs to be evaluated in future research by studying the planning and execution of OI projects. To increase the reliability of the OI-relevance assessment, a retrospective analysis of completed OI projects seems promising. A further challenge is the measurability of criteria since remarks in the literature are often limited. Although we presented strategies for dealing with this, it is still an issue which needs to be addressed in the future. To this end, strategies and methods from survey design might be beneficial. They might also help to increase the reproducibility and repeatability of situation analyses. Our goal (G2) of providing a list of potential situation criteria as a starting point for other researches has only been partially achieved so far. To allow better accessibility and future enhancements, an online database of different situation criteria and their relevance for different innovation aspects might be useful.

The qualitative research presented here identified basic trends and principles of assessing an (open) innovation situation. To ensure future applicability in industry, the involvement of industry partners in the development of the situation analysis was crucial. Sufficient communication with them was an important success factor: On the one hand, to motivate our industry partners, we needed to show that all presented criteria were beneficial based on the literature. On the other hand, to develop a sufficient situation analysis, we needed to encourage them to provide critical feedback. In this regard, a different, preceding evaluation of an early version of the criteria list was valuable in terms of getting a first impression (Guertler et al. 2014a ). Although it was conducted with students working in industry companies, it revealed basic trends confirmed by the later evaluation, e.g., difficulties in assessing the annual expenses for R&D (1.5.1) or the number of patents (1.5.8). A central issue in the industry evaluation was the definition of suitable specification scales for the criteria. In some cases, we left the scales blank in order to allow free answers and specifications and derive the scales afterwards. However, this caused more effort than benefit: Usually rough specification ranges would have been sufficient, but some industry partners spent much effort to research specifications as precisely as possible, such as the number of relevant suppliers. Despite the originally expected academic benefit, we recommend providing as much orientation as possible in order to limit the effort for the industry partners.

6 Conclusion and outlook

As several studies have attested, sufficiently planning an Open Innovation (OI) project is essential for its success. A major aspect is the analysis and consideration of a project’s boundary conditions and constraints (‘innovation situation’). They affect the solution space of the project, the choice of suitable OI partners and OI methods, as well as the potential project’s risks and risk handling strategies. The literature on OI provides a variety of empirical studies analyzing OI projects and potential influencing factors of its performance. To date, however, there is a lack of synthesis in this area in terms of methodical support for planning an OI project. Existing planning support is usually quite abstract (e.g., Ili 2010b ) or focuses on specific OI methods, such as crowdsourcing (Piller & Ihl 2009).

During our research, we discovered that analyzing innovation situations is a general issue in academia and industry. So far, neither a holistic approach for analyzing an OI situation nor a general innovation situation exists. Usually, each research group develops its own approach, including time-consuming literature reviews or intuitive definitions of situation criteria as well as probably experiencing the same challenges and common mistakes in practical application. Therefore, before developing an approach to analyze OI situations, it was necessary to create a general basis for situation analyses.

This paper thus pursues two goals: (G2) supporting other research groups in developing their own situation analysis approaches by providing a basic list of situation criteria and relevant guidelines for an industrial application; (G1) identifying first indications of OI-relevant criteria, focusing on the relevance for selecting suitable OI methods. Therefore, in the first step, we identified existing situation analysis approaches as well as different situation criteria in the literature. In the next step, the criteria were consolidated and clustered. In workshops with three industry partners, we evaluated the criteria’s usability and derived guidelines for developing situation analysis approaches for industry. In the third step, we identified OI-relevant criteria for selecting suitable OI methods by using a DMM analysis.

The main contribution of this paper is the list of situation criteria, which can be used as a starting point by other research groups and limits the efforts needed to conduct a literature review. This paper also provides a guideline for developing a situation analysis, and therefore helps companies avoid common mistakes and challenges. The initial analysis of the criteria’s OI-relevance forms a basis for subsequent research, to allow the successful planning and execution of OI projects.

Based on this paper, future research should focus on the evaluation of the criteria’s OI-relevance. Since the validity of our evaluation is limited due to the small number of participating companies, our results need to be further evaluated in future research. By including a section with situation criteria in future studies, the criteria’s influence on different aspects of OI can be evaluated quantitatively, e.g., project risks that arise, suitable OI methods, or IP strategies. Here, our key learnings support structuring the relevant section and phrasing the criteria. Besides this, it is also important to flank quantitative studies with qualitative in-depth studies. Accompanying OI projects from the planning, execution, and processing of results up to the long-term use of the OI input allows a detailed evaluation of key situation criteria in each project phase. In this regard, both are relevant: situation criteria, which directly influence a decision, e.g., the selection of suitable OI methods, as well as qualitative criteria, which indirectly affect an OI project. Those, e.g., support the explication of implicit knowledge and ensure a homogeneous knowledge level within interdisciplinary teams. Those studies also offer the opportunity to perform more in-depth analysis of additional aspects such as reproducibility and repeatability of situation analyses.