A logical critique of the expert position in design research: beyond expert justification of design methods and towards empirical validation

Abstract

This paper gives a general and logical analysis of the expert position in design research by which methods for innovative design can be derived from expert design practices. It first gives a framework for characterising accounts of design by the way in which they define and relate general, descriptive and prescribed types of design practices. Second, it analyses with this framework the expert position’s conservatism of prescribing existing expert design practices to non-expert designers. Third, it argues that the expert status of expert designers does not provide sufficient justification for prescribing expert design practices to non-expert designers; it is shown that this justification needs support by empirical testing. Fourth, it discusses validation of design methods for presenting an approach to this testing. One consequence of the need to empirically test the expert position is that its prescription has to be formulated in more detail. Another consequence is that it undermines the expert position since expert design practices are not anymore certain sources for deriving design methods with. Yet it also opens the expert position to other sources for developing design methods for innovation, such as the practices of contemporary designers and the insights of design researchers.

1 Introduction: the expert position

In design research, there is a regularly held position that methods for innovative design can be derived from the design practices of expert designers. By this position the established design methods in engineering and product development are taken as leading to incremental improvements of existing products, and contrasted with the innovative practices of expert designers who by thinking ‘out of the box’ found dazzling solutions to arduous problems and created market disrupting ‘game changers’. Also non-expert designers, so is the claim, can deliver such imaginative innovations if only they take distance from existing products and practices, and follow the experts’ examples. This position is as interesting and plausible as it is problematic and incoherent. Expert designers indeed have created telling innovations in engineering and architecture, and are increasingly delivering innovative solutions in business and society. Furthermore, advancing that designers should follow the examples of experts seems self-evidently justified advice since expert designers are by definition experts in designing. Yet, assuming that non-expert designers have the capabilities to design in a manner similar to how experts do, seems problematic. And taking existing expert design practices as standards for taking distance from existing products and practices appears incoherent; the focus on expert designers seems rather to introduce a conservative backward-looking constraint on innovation since these experts typically have their backgrounds in previous decades in which current issues such as product-services systems, responsible innovation and circular economy were not of central importance.

In this paper, I give a general and critical analysis of the position that methods for innovative design can be derived from expert design practices. I call this position in design research the expert position, for giving it a name and for also signalling that it is a position and not a necessary basic assumption in design research; there are rival traditions in which design methods are formulated using other sources than expert design practices. The analysis is aimed at, firstly, determining to what extent the expert position is conservative by prescribing methods for innovation design derived from expert design practices, and at, secondly, arguing that the justification of prescribing these design methods needs support by empirical testing.

For giving this analysis I present in Section 2 a framework for characterising accounts of design by the way in which they relate general, descriptive and prescribed types of design practices. In Section 3, I use this framework for reviewing the conservatism of the expert position. Section 4 assesses the justification of that prescribing design methods leads to innovation, and shows that this justification needs amendment by empirical testing. In Section 5, I expand the analysis to the validation of design methods for presenting a possible approach to organising this empirical testing of the expert position ¹ .

The analysis of the expert position as given in this paper is a logical one by abstracting from many details and choices that individual proponents of this position advance in their specific accounts of design. A design method, for instance, is represented in a general way by a set of types of design practices that are prescribed by an account of design. And ‘expert designer’ and ‘expert design practice’ are in the analysis assumed to be (logically primitive) notions that get their precise meaning in the accounts of design in which the expert position is advanced. I abstract from how proponents of this position make such specifications, which may make the analysis equally abstract yet ensures its general validity. Hence, independent of how ‘design method’, ‘expert designer’ and ‘expert design practice’ are understood in an account that advances the expert position, the justification of the prescribed design methods need empirical support. This general result implies, as will be shown, that the expert position accounts of design have to be formulated in more detail to indeed empirically test them. Another consequence is that the plausibility of the expert position is undermined: the authority of experts does not make their design practices a preferred and certain basis for deriving design methods with; expert design practices are at most (good) inspirational sources for finding design methods for innovation. This categorisation opens the expert position to also explicitly embracing other (good) sources for this goal: proponents of the expert position can equally use the practices of contemporary designers and the insights of design researchers for proposing new design methods, as is already done in traditions in design research in which the expert position is not held.

Table 1. The GDP sets of an account of design

2 The GDP of accounts of design

The expert position leads to accounts of design in which some actual design practices – those by expert designers – are taken as favourable design practices from which design methods can be derived by which non-expert designers can also arrive at such favourable practices. For capturing this aspect of the expert position and for contrasting it to alternatives, I start by giving in this section a general framework for characterising accounts of design by the way in which they relate general, descriptive and prescribed types of design practices.

With an account of design I mean in this paper a general theory of or an elaborated perspective on designing in engineering, product design or elsewhere, as given in, for instance, Hubka & Eder (1988), Gero (1990), Suh (2001), McDonough & Braungart (2002), Cross (2006), Pahl et al. (2007), Brown (2009), Verganti (2009) and Dorst (2015) ² . A design method part of an account of design is, as said, represented by a set of prescribed types of design practices within the account, and examples of design methods are given in the mentioned literature. The expert position is held in, e.g., Cross (2006), Visser (2006), Brown (2009), Lawson & Dorst (2009), Verganti (2009) and Dorst (2015). Expert designers are singled out in this literature, for instance, by their years of working experience, through design projects that are acknowledged to have been innovative, or simply by ostension (see, e.g., Lawson & Dorst (2009) for a more explicit discussion). And the favourable expert design practices are determined by extensive empirical analyses of the design projects of these experts. In the next section, I return to the expert position; in this section, the analysis focusses on accounts of design in general.

Assume that an account of design in principle singles out three types of design practices: types of practices that by the account are accepted as types of general design practices; types that are taken as types of design practices that are actually carried out by designers; and types of practices that are by the account prescribed to designers as favourable design practices. An account can then be characterised by three sets (see Table 1): a G-set with types of general design practices; a D-set with types of actual design practices; and a P-set with types of prescribed favourable design practices. This characterisation is abstract by glossing over motivations, assumptions and choices that underlie specific accounts of design. Choices part of accounts to accept certain types of practices as design practices is not the topic of this paper, and neither is the way accounts describe these practices. These elements are accepted as part of the accounts; only the expert position assumption made in some accounts that expert design practices can be prescribed as favourable design practices is critically analysed, and in this analysis it is again abstracted from the details of how in accounts expert designers and their design practices are defined. This abstractness may make the framework sterile and logical, yet also lets it be general and independent of the details of individual accounts of design.

The G-, D- and P-sets of accounts of design can metaphorically be taken as the accounts’ ‘GDP’ since firstly these sets represent how accounts demarcate design and how they describe and prescribe design, and since secondly accounts can be categorised as conservative, progressive and futuristic by means of the relations that exist between the accounts’ G-, D- and P-sets. The second point will be developed in Section 2.2; here the focus is on what the three sets represent individually.

Let us start with the D-set. In an account some actual practices as displayed by designers (and possibly non-designers) are taken as design practices. These practices are analysed and described in terms of design concepts, defining types of design practices that may be called types of actual design practices since instances of these types have been actually carried out by designers. These types of design practices make up the D-set of an account and represent the descriptive part of the account, that is, how the account captures actual design. For instance, in the account of Gero (1990) actual practices acknowledged as design are analysed in terms of reasoning from functional descriptions of products via behavioural descriptions to structural descriptions of the products. Gero mentions as an example the designing of a window. The functions the design start with are, among others, the provision of daylight and access to a view, and the design ends with drawings of the window and notes about it. These types of reasoning then define the D-set of Gero’s account, representing how in this account actual design is described.

An account may secondly define a larger set of types of practices that demarcates what types of practices are in the account taken as types of design practices. This set of types of practices defines the G-set of an account, characterising what it acknowledges as general types of design practices. The G-set is the set of all types of design practices that are in principle possible by the account, and thus contains types of design practices that have already been carried out and may contain types of design practices that have not yet been carried or even will never be carried out. The types of actual design practices in D are thus all in this G-set, but types of design practices in the G-set need not be in D. For instance, Gero may generalise his description of actual design practices by assuming that even social institutions can be designed in the future rather than only material products. Gero’s G-set contains then types of reasoning that concern the functions, behaviour and structure of material products, and types of reasoning that concern the functions, ‘behaviour’ and ‘structure’ of institutions. But these latter types of reasoning are not in Gero’s (current) D-set since (by assumption) institutions are not yet actually designed using reasoning about their functions, ‘behaviour’ and ‘structure’.

Finally an account may take some types of design practices as good practices and present them, for instance as design methods, that is, as practices by which designers can improve their designing. These prescribed types of favourable design practices make up the third P-set, and represent the prescriptive part of the account. The Cradle-to-Cradle account of design (McDonough & Braungart 2002), for instance, prescribes types of design practices that lead to eco-friendly design solutions. And accounts of participatory design (e.g., Simonsen & Robertson 2012) prescribe types of design practices in which users can influence and guide the practices and their outcomes.

2.1 Constraints on the GDP of accounts of design

By systematically exploring the set-theoretical relations that can exist between the G-, D- and P-sets of accounts, one can categorise them as conservative, progressive and futuristic accounts of design. Before giving this categorisation I chart the constraints on these relations.

The D-set is a subset of the G-set, for it would be inconsistent if an account accepts actual practices as types of design practices while simultaneously not acknowledging them as types of design practices. So a first constraint on the GDP of an account is that $\text{D}\subseteq \text{G}$ .

In a similar way it follows that the P-set is a subset of G, leading to the constraint $\text{P}\subseteq \text{G}$ . This constraint can be strengthened if it is assumed that the prescription by an account is not trivial, that is, that an account does not take all possible types of design practices as favourable. This assumption means that designers are prescribed to carry out specific type of design practices and should avoid other types. Hence, there then exist types of design practices in G that are not in P, which means in turn that P cannot be equal to G, but is a proper subset of G, so $\text{P}\subset \text{G}$ . This is a second constraint.

If it is also assumed that the prescription by an account implies that designers should change their actual design practices, then a third constraint becomes available. Such prescription implies that some types of actual design practices are not favourable types. The D-set is then not a subset of P, leading to a constraint that it is not the case that $\text{D}\subseteq \text{P}$ : there are types of actually design practices in D that designers should abandon by the account since they are not types of design practices in P that the account prescribes ³ .

The first constraint $\text{D}\subseteq \text{G}$ holds necessarily: if an account takes a type of actual practices as design, it acknowledges it as a type of design practices. The second and third constraints depend however on the two assumptions described in the previous two paragraphs. In the ideal case that all actual design practices are of the types that are prescribed in an account, one has that $\text{D}\subseteq \text{P}$ . Moreover, in the trivial case that an account prescribes all types of design practices, one has $\text{P}\subseteq \text{G}$ . Imposing the second and third constraints on accounts of design thus expresses that the prescription by accounts is non-trivial, and that designing has not yet evolved to the ideal case in which all actual design practices are of the prescribed types of favourable practices.

Consider, for instance, an account in which it is observed that actual design practices sometimes fail to produce eco-friendly design solutions, and in which the prescribed types of design practices do lead to eco-friendly solutions. All types of observed actual design practices, including those that produce eco-friendly solutions and those that do not, are in the account taken as design practices, which is captured by the first constraint: $\text{D}\subseteq \text{G}$ . All prescribed types of design practices that produce eco-friendly design solutions, are also general types of design practices but some general types of design practice do not produce eco-friendly solutions, which is captured by the second constraint: $\text{P}\subset \text{G}$ . Finally, since designers are by this account sometimes actually producing non-eco-friendly design solutions, there is at least one type of actual design practices that is not a prescribed type of design practices, which is expressed by the third constraint: it is not the case that $\text{D}\subseteq \text{P}$ .

The three constraints on the GDP of accounts are summarised in Table 2.

Table 2. General GDP constraints

2.2 Conservative, progressive and futuristic accounts of design

Consider the G-, D- and P-sets of an account by means of Venn diagrams. Constraints 1 and 2 ( $\text{D}\subseteq \text{G}$ and $\text{P}\subset \text{G}$ ) imply that G is always represented by the biggest diagram. There are now ten possibilities for D and P lying within G, depicted in the Figures 1–4. Constraints 2 and 3 ( $\text{P}\subset \text{G}$ and the impossibility that $\text{D}\subseteq \text{P}$ ) rule out the four possibilities given in Figure 1: P should not be as big as G ( $\text{P}\subset \text{G}$ should not be violated); and D cannot be equal to P or lie within P (it is impossible that $\text{D}\subseteq \text{P}$ ). The ruled-out GDP possibilities represent accounts in which the D-set describing types of actual design practices, is in or is equal to the P-set of prescribed types of design practices, hence represent the ruled-out ideal case that all actual design is favourable design. Possibilities 2 and 4 represent moreover the trivial case that all possible design practices are favourable.

Figure 1. The ruled-out GDP possibilities 1, 2, 3 and 4; $\varnothing$ is the empty set, such that $\text{G}=\text{P}$ for possibilities 2 and 4.

Two GDP possibilities that are allowed are characterised by P lying within D, see Figure 2. These possibilities 5 and 6 represent accounts that may be called conservative, for in accounts in which P lies within D, prescribed types of favourable design practices are always types of actual design practices; there is thus not a type of favourable design practices that has not yet been realised in actual design. Accounts in which the expert position is adopted may be cases in point, since on such accounts types of actual design practices by experts are taken as types that are to be prescribed (but this categorisation is qualified in the next section).

Figure 2. The conservative GDP possibilities 5 and 6; $\varnothing$ is the empty set, such that $\text{G}=\text{D}$ for possibility 6.

Two further possibilities 7 and 8 allowed by the constraints are characterised by D and P partly overlapping, see Figure 3. Accounts that are instances of these possibilities may be called progressive, for in such accounts there are types of favourable design practices that are already adopted in actual design, and there are such practices that are not (yet) adopted. By such accounts there can thus be progress: there are new ways of designing available that are (also) favourable. The theory of axiomatic design (Suh 2001) may be an example of such a progressive account that prescribes types of design practices that have already been carried out and types of design practices that are new.

Figure 3. The progressive GDP possibilities 7 and 8; $\varnothing$ is the empty set, such that $\text{G}=\text{D}\cup \text{P}$ for possibility 8.

The final two allowed GDP possibilities 9 and 10 are characterised by D and P not overlapping, see Figure 4. In accounts falling under these possibilities all types of actual design practices are not types of favourable design practices that are prescribed. Yet types of favourable design practices exist and may in the future become actual. Let us call these accounts futuristic accounts. The introduction of the first design methods at the time when artisan production ruled, may be reconstructed as such futurism: existing artisan practices are discarded as unfavourable and new types of design practices that are considered to be favourable are made available.

Figure 4. The futuristic GDP possibilities 9 and 10; $\varnothing$ is the empty set, such that $\text{G}=\text{D}\cup \text{P}$ for possibility 10.

3 The expert position and its conservatism

The expert position seems by its very starting point conservative. It seems to take only some actual expert design practices as favourable and to rule out that design practices that have not yet been carried may be favourable as well. The expert position thus seems to lead to accounts that fall in the conservative GDP category as represented by the possibilities 5 and 6 (see Figure 2). As such the expert position appears to bring design methodology back to the craft tradition of guilds. For arriving at innovation non-expert designers are to adopt the types of practices that expert designers exercise; the option to realise innovations by (also) rethinking and changing the structure of these expert practices is not supported by the expert position. So, a new practice as developed by some avant-garde designers, say, collaborative practices in open design, cannot be favourable, and neither can unprecedented practices that design researchers formulate for, say, incorporating in design new technological tools as 3D printing. The expert position indeed introduces a ‘guild-like’ backward-looking perspective: experts that have proved themselves in previous periods are taken as revealing what design practices are favourable, also for dealing with contemporary and future challenges such as open design, 3D printing, product-services systems, sustainability, responsible innovation and circular economy. For instance, the Cradle-to-Cradle vision (McDonough & Braungart 2002) is on the expert position initially not defining types of design practices that are favourable and to be prescribed.

However, a more careful analysis of the expert position by means of the GDP categorisation provides a more subtle view. It can be argued that the expert position can be unpacked in two ways. There is a strict expert position in which the types of prescribed design practices correspond to only types that expert designers actually engaged in, and on this strict reading one indeed gets conservative accounts, as represented by the possibilities 5 and 6. But there is also a liberal expert position that allows for progressive accounts by also prescribing types of design practices that have not (yet) been actually carried out by expert designers, as represented by the possibilities 7 and 8 (see Figure 3). And both ways allow for adjusting the prescribed types of design practices, where the liberal expert position gives more room for this.

3.1 The strict and liberal expert positions

Consider accounts of design in which the expert position is adopted. In such accounts some designers are taken as experts and a number of their practices are singled out as favourable. Let these expert design practices be $\{e_{1},e_{2},\ldots ,e_{n}\}$ . The expert practices are then analysed as having certain shared characteristic factors, let us call them the x-factors. These $x$ -factors define types of expert design practices, and let X be the set that contains these types of practices ⁴ . The types of expert practices in X are finally prescribed in the accounts to all designers as the types of favourable design practices.

When approached within the GDP categorisation, the expert position leads to two general constraints. First, since on this position the prescribed types of design practices are the types of expert practices in X, it follows that X is equal to the P-set. Second, since the singled-out expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ are instances of the types of expert practices in X, and since these practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ have all been actually displayed, it follows that their types are also in the D-set, meaning that the overlap between X and D cannot be empty. Formally this means $\text{X}=\text{P}$ and $\text{X}\cap \text{D}\neq \varnothing$ . The latter constraint allows that the X-set contains types of design practices that are not in D and hence not (yet) actually carried out. That this possibility is possible on the expert position becomes clear below when discussing the liberal expert position.

In a strict reading of the expert position the X-set does not contain other types of design practices than those of which the singled-out expert practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ are instances. The $x$ -factors describing X then characterise only the types of practices as displayed by the expert designers, and no other types of practices that resemble $\{e_{1},e_{2},\ldots ,e_{n}\}$ . For instance, one carefully analyses what expert designers in product design or in industrial innovation do (e.g., Cross 2006; Verganti 2009) and then this precise description defines what other designers should do for mimicking the successes of the experts. With this precise description of the expert design practices, X will only contain types of design practices that have already been carried out. Hence, on the strict expert position X is a subset of D, or $\text{X}\subset \text{D}$ .

A more liberal reading of the expert position is also possible and amounts to allowing in X types of design practices that have not yet been exercised. The analysis of the expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ could still be as precise as in the strict expert position, and lead to a detailed description in terms of $x$ -factors of the types of design practices that have $\{e_{1},e_{2},\ldots ,e_{n}\}$ as instances. Yet, by generalising over this description by letting some $x$ -factors have multiple values, the X-set can contain more types of design practices than only those that have $\{e_{1},e_{2},\ldots ,e_{n}\}$ as instances. Say, the experts may all have been sketching with pencils to explore their initial ideas, but it may be judged on the liberal expert position that the use of in particular pencils is not relevant to the favourability of the singled-out practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ . The X-set then contains types of favourable design practices in which exploration is done by pencil or by other means, and expert designers need not yet have exercised the latter in reality. Or the expert practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ may all have been practices in product design, yet the prescription may also concern other application domains, such that X also contains types of practices in social and policy design (e.g., Dorst 2015). And this transposition of design thinking to other domains may again lead to allowing types of design practices that have never been done actually. Hence, on the liberal expert position X need not be a subset of D.

Table 3. The expert position constraints

In Table 3, the constraints for the expert position and its two readings are listed. With these constraints it follows that the suggestion of conservatism is indeed confirmed for accounts in which the expert position is adopted in its strict reading, but not for accounts based on the liberal reading. In terms of the GDP categorisation the strict expert position leads to accounts that are always conservative. The possibilities 7 to 10 are ruled out because when in the diagrams given in Figures 3 and 4 the P-set is equated with X, one obtains diagrams in which X is not a proper subset of D, which is not allowed by the strict expert position constraint $\text{X}\subset \text{D}$ . The conservative possibilities 5 and 6 as given in Figure 2 however give diagrams in which the expert position constraint is met, leading to two GDP options for the strict expert position (see Figure 5).

Figure 5. The two options for the strict expert position, through GDP possibilities 5 and 6.

Since these two options are instances of the possibilities 5 and 6, it follows that the strict expert position is indeed conservative: it leads to accounts in which the prescribed types of favourable design practices are always types of actual design practices.

For accounts in which the liberal expert position is adopted, the possibilities 9 and 10 are still ruled out due to the general constraint for the expert position ( $\text{X}\cap \text{D}\neq \varnothing$ ; see Table 3) that some of the prescribed types of design practices in X are types that have actually been done by designers (namely the types corresponding to the singled-out expert practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ ). But the other possibilities 5 to 8 are now available, leading to four options for the liberal expert position, as given in Figure 6.

Figure 6. Four options for the liberal expert position, through GDP possibilities 5, 6, 7 and 8.

With the options 1 and 2 the liberal reading of the expert position leads again to conservative accounts of design, yet with the options 3 and 4 one can also arrive at this reading in progressive accounts in which prescribed types of favourable design practices can be types of design practices that have no precedence yet.

3.2 Routes for adjusting the prescribed types of design practices on the expert position

The above analysis showed that on the liberal reading the expert position can lead to a progressive account of design that allows for prescribing types of design practices that have not yet been carried out by expert designers. More generally there seem to be two routes available for adjusting the types of design practices the expert position prescribes, and one of them is even open to the strict expert position. The first route is defined for the above described progressive accounts allowed by the liberal expert position. Given a fixed set of expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ and given a fixed $x$ -factor description of the types of practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ are instance of, one can still adjust the prescribed types of design practices in X by varying the generalisation over the $x$ -factor description. One can, say, generalise over the means used for sketching, over the application domain and over other characteristics of the expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ . This first route for adjusting the prescription may also be suitable to incorporate design practices using new design tools: the favoured expert practices are then expanded from practices in which such tools are not used to practices in which they are used. Yet more substantial adjustments are harder to be achieved. Design practices that are envisaged by the Cradle-to-Cradle approach are difficult to construe as generalisations of earlier expert practices, for instance due to the Cradle-to-Cradle approach to see product design as part of the design of artificial ecosystems.

The first route for adjusting the types of design practices the expert position prescribes draws on the possibility to changing the contents of the X-set by generalising the $x$ -factor description of this set. The second route draws on the possibility to change the set of practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ singled out as expert practices. This route is available for both the strict and liberal readings of the expert position. On both these readings one may at some point decide that new actual design practices $e_{n+1},e_{n+2},\ldots$ are also of expert quality, and possibly that some of the practices in $\{e_{1},e_{2},\ldots ,e_{n}\}$ should not be considered as of expert quality anymore. This decision would imply that the designers involved in the newly singled-out practices $e_{n+1},e_{n+2},\ldots$ are taken as experts (assuming these designers were not involved in the practices in $\{e_{1},e_{2},\ldots ,e_{n}\}$ ) and it would involve that the X-set is enriched and starts to contain also the type of design practices associated with the new practices $e_{n+1},e_{n+2},\ldots$  . This adjustment can already take place on the strict reading; on the liberal reading this adjustment can be further amplified by considering different generalisations of the $x$ -factor descriptions of the new practices $e_{n+1},e_{n+2},\ldots$  . By this second route the expert position could eventually endorse the Cradle-to-Cradle vision; as soon as design practices by McDonough, Braungart or other Cradle-to-Cradle designers are accepted as expert design, the Cradle-to-Cradle types of design practices become prescribed types. This second route is also followed when expert designers are on the expert position taken as innovators of their expert design practices. When a designer who has already contributed with a design practice $e_{j}$ to the set $\{e_{1},e_{2},\ldots ,e_{n}\}$ of expert practices, develops a design practice of a type that does not fit the X-set, then this new design practice is in principle not an expert design practice. Yet on the expert position this new design practice can be added to $\{e_{1},e_{2},\ldots ,e_{n}\}$ as a new expert practice $e_{n+1}$ , by which it becomes a design practice of a type that fits X.

The two routes towards prescribing new types of design practices are given in Table 4.

Table 4. Routes for adjusting prescribed types of design practices on the expert position

4 Justification of the expert position

After the articulation of the expert position we can turn to its justification. This justification seems self-evident and inbuilt, yet it can be argued that it still needs empirical support. The self-evidence of the expert position is due to its focus on experts that seems to make its claims close to tautological: taking expert designers as experts seems to directly imply that their design practices are favourable; and prescribing types of expert practices to other designers seems therefore immediately equivalent to prescribing types of favourable design practices. In short, expert design practices seem a certain basis for prescribing favourable practices, and the expert position is thus justified. However, this reasoning hinges on at least two assumptions that are not self-evident but rather in need of empirical support. The first is an induction assumption that generalises the relation between the structure of the singled-out expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ and their favourability to all design practices in the X-set of prescribed types of design practices. The second is an assumption about the capabilities of designers, namely that non-expert designers can carry out all these prescribed types of expert design practices.

If one of these assumptions, or both, is not tenable, the prescriptions by the expert position are not warranted: non-expert designers then need not necessarily reproduce the successes of expert designers by mimicking their expert practices. Furthermore, separating the assumptions is relevant if the expert position prescriptions were to fail. If these prescriptions fail and the induction assumption holds but the capability assumption does not, then the prescribed practices are indeed favourable and it makes sense to try to improve the capabilities of non-expert designers such that they eventually can carry out the expert practices. If however, the expert position prescriptions fail and the capability assumption holds but the induction assumption does not, then it is simply falsified that non-designers can realise innovations by means of the prescribed types of expert design practices.

4.1 Justifying the induction assumption that types of expert design practices are favourable

The induction assumption that generalises the relation between the structure of the singled-out expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ and their favourability can be made precise as an assumption that the practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ have a specific structure and that it is this structure that makes these practices favourable. Let the structure of these practices be captured by their descriptions in terms of the $x$ -factors, and let $f$ refer to the favourable properties for which the expert practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ are singled out. Then the induction assumption is that all design practices that meet the $x$ -factors will display the favourable properties $f$ . Say, drawing from Verganti’s (2009) analyses of expert design practices in firms like Alessi, the $x$ -factors are ‘collaborating with key interpreters outside the firm’, ‘open innovation-style sharing of knowledge’ and ‘restraints on return on investment-assessments’, and the $f$ properties include ‘creating innovative product families’. The induction assumption underlying Verganti’s account of design is then that all design practices that meet the mentioned $x$ -factors are ones that lead to innovative product families.

In work advancing the expert position this induction assumption is typically supported by analyses of the expert practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ in which they are shown to have the favourable properties $f$ and a structure captured by the $x$ -factors. Verganti, for instance, analyses successful design practices in firms such as Alessi that did lead to innovative product families and that consisted of collaborations with key interpreters in an open innovation fashion, etc. However, such case studies can hardly count as conclusive evidence to the induction assumption; they show that there were design practices that met the $x$ -factors and that had the favourable properties $f$ , yet do not rule out that there are many other cases of design practices that also meet the $x$ -factor description but do not have $f$ . For instance, it is consistent with Verganti’s analyses that there exist large numbers of firms that have engaged in design practices similar to those of Alessi, yet failed miserably in creating innovative success.

For evaluating the tenability of the induction assumption that all the design practices that satisfy the $x$ -factors will also be displaying the favourable properties $f$ , one can logically discern three cases. First the favourable properties $f$ can be themselves $x$ -factors. The assumption is then simply a logical truth and the expert position becomes a tautology: all instances of types of expert design practices are favourable because they are all characterised by the favourable properties $f$ . This tautology typically makes the prescription useless; in cases where it still contains useful help, the prescription can be made more precise. For instance, if the $x$ -factors are merely the $f$ properties, the prescription is nothing more than the command to create an innovative product; how that is done is left unspecified. If the $x$ -factors contain more properties than $f$ , these redundant $x$ -factors are typically unnecessary constraints; the command to create an innovative product is then accompanied with additional directions because experts traditionally worked in that manner. Only if the favourable properties $f$ concern the final design outcome of the design practice and the redundant $x$ -factors concern the process of the practice, one may envisage that these redundant $x$ -factors help guide designers to arrive at the favourable outcomes; yet in that case the actual prescription seems to be that designers should follow practices that meet these redundant $x$ -factors in order to obtain the favoured innovative products, which brings us to the second case.

Second, if the $f$ properties are different to the $x$ -factors, the induction assumption can be taken to mean that the $x$ -factors make that the design practices will have $f$ , meaning that the $x$ -factors are sufficient conditions for $f$ . In this case, additional work is needed to support the assumption. The empirical basis of the expert position is that there are a number of actual expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ that meet the $x$ -factors and that have $f$ . This basis is insufficient for concluding that all design practices that meet the $x$ -factors have the favourable properties $f$ , as illustrated with the above example of firms that unsuccessfully copied Alessi’s innovation practices. What is needed in this second case is either a theoretical explanation that design practices that satisfy the $x$ -factors will have the favourable properties $f$ as well, or experimental results that demonstrate that this relation holds. In accounts in which the expert position is maintained, explanations why design practices meeting the $x$ -factors have $f$ are typically given in a schematic fashion. For instance, Verganti (2009) explains that interaction with key interpreters opens up firms to new perspectives on existing products and technologies, and enables them to come up with innovative new uses thereof. Yet, by the lack of a sufficiently firm body of design laws and theories, such theoretical explanations may add plausibility to the induction assumption but do not prove its truth.

In the third case, the $x$ -factors do not imply $f$ but predict the presence of $f$ with some measure of certainty; there may, for instance, be stable correlations between the $x$ -factors and $f$ without there being also clear-cut logical relations. In his third case one obtains a probabilistic relation that a design practice meeting the $x$ -factors has with a specific probability $\text{p}(f)$ the favourable properties $f$ , and this probability may be small or large. Although now the induction assumption strictly speaking does not hold, the expert position still makes sense: the types of expert design practices in X can still be prescribed since they are with probability $\text{p}(f)$ favourable design practices. When, for instance, this probability is large, design practice meeting the $x$ -factors have often the favourable property $f$ , while occasional exceptions are now to be expected and not undermining the (probabilistic) induction assumption. Even when $\text{p}(f)$ is small the correlations between the $x$ -factors and $f$ may be of value; the successes of firms like Alessi can be so attractive that a probability $\text{p}(f)$ of 0,2 to repeat it with a design practice meeting the $x$ -factors is appealing enough to engage in these practices (think, e.g., of the gambles venture capitalists take). Yet, it should be noted that understanding the probability $\text{p}(f)$ in terms of quantitative numbers such as 0,2 currently does not make sense in design research; if at all, $\text{p}(f)$ can better be understood in qualitative terms such as ‘small’ and ‘large’.

The three cases are listed in Table 5. For the first the induction assumption is indeed self-evident since tautological. But in the second and third cases this self-evidence is absent. The assumption then becomes a hypothesis advancing an empirical claim. This hypothesis may be supported by argument and theory, but is in principle in need of empirical testing for corroborating it and, in the third case, for determining its precise content by (qualitatively) fixing the probability $\text{p}(f)$ .

Table 5. Three cases for the relation between the $x$ -factors and the favourable properties $f$

4.2 Justifying the capability assumption that non-experts can carry out expert practices

When the induction assumption holds, then design practices in X meeting the $x$ -factors can be prescribed to other designers as practices that have the favourable properties $f$ . Yet whether these other designers can carry out the prescribed practices depends on the tenability of the second capability assumption that non-expert designers have the capabilities – the skills; the knowledge; the talents – to do so. It may seem self-evident that the expert designers themselves can carry out the practices in X, since they created them in the first place. Yet it seems equally plausible that laypersons who are not trained as designers can have severe difficulty with following these expert practices since it typically takes years of education and training to become a designer. For the expert position to make sense the question is now whether the capability assumption holds for regular designers who are not experts, for the prescription of the expert position concerns these non-expert designers.

The literature in which the expert position is embraced does not provide an unambiguous answer to this question. Regularly it seems that the answer is assumed to be positive: non-expert designers are taken as in principle able to carry out the prescribed expert design practices. In work by, for instance, Cross (2006), Brown (2009), Verganti (2009) and Dorst (2015), expert practices are analysed and described with the claim that all designers can profit: for non-expert designers the favourable properties $f$ are then in reach by just mimicking the experts. Experiences with non-designer professionals strengthen this positive answer since specifically in the design thinking movement, which makes expert design practices available to these other professionals, positive outcomes are reported about letting them carry out the prescribed expert design practices (e.g., in the Design Schools ⁵ and the Designing Out Crime organisation ⁶ ). But one can also find in the literature reasons to doubt that non-expert designers have always the ability to mimic the experts. The expert practices may be complex and subtle, and effectively be limited to only other expert designers. The expert position then does not define favourable design practices non-designers can follow, but spells out the reasons why these designers cannot engage in these favourable practices. Work by Lawson & Dorst (2009), and recently Curry (2014), may be taken as representing this line: it analyses in detail expert design practices, not for determining what novices and regular designers should do in their design practices, but for determining what they should learn for becoming experts themselves.

Whether or not the capability assumption holds may also depend on the types of expert design practices in X, ruling out one general answer to the question of whether non-expert designers can carry them out. Possibly this assumption can for specific types of practices be supported by argument and theory, yet in general it seems that support for the assumption again needs to come from empirical testing: in principle it seems implausible that non-expert designers can engage in expert design practices, yet a reality check with non-expert designers may prove the opposite as it already did in a number of the mentioned cases.

5 Empirically validating the expert position

In the previous section, it was argued that the justification of the expert position needs empirical support, and that this justification depends on the induction and capability assumptions. Demonstrating the tenability of the second assumption requires empirical testing, and if it is not tautological, the same applies to the induction assumption. This conclusion makes the justification of the expert position an empirical affair and introduces a set of problems associated with the validation of design tools and methods in design research.

In the literature that reflects on design research there is a complaint that the discipline lacks generally accepted research methods for empirically validating its prescriptions, leading to a situation where design tools and methods are promoted and advanced without being systematically evaluated by experiment or against earlier established results (e.g., Blessing & Chakrabarti 2009; Birkhofer 2011; Wallace 2011). It seems sometimes even accepted that the validation of design methods cannot be achieved simply because real design projects are too contextual and too open-ended (e.g., Frey & Dym 2006; Reich 2010). Design tools and methods, so is the reasoning, are ultimately meant to improve design practices in industry. But experiments to empirically validate these claims are practically impossible to carry out. It would, for instance, require years-long experiments with sufficiently large numbers of industrial firms, where some have to stick strictly to their established design practices and where other firms have to abandon their regular practices and switch to the tools or methods to be validated. And it would mean keeping many socially complex contextual factors controlled, like international economic growth and the behaviour of competitor firms. Meeting these requirements in experiments is not feasible since economic situations will typically change and firms will typically respond to that. Hence, the conclusion of this reasoning is that full validation of design methods cannot be achieved. And a consequence of this is – so one could conjecture – that in design research validation is not seen as being part of proposing a design tool or method; one can at most make a proposed method plausible by analysing some successful cases and by giving some explanation of why the method leads to the successes (e.g., Alessi for Verganti’s method for innovative product families), and then simply continue with promoting and advancing it.

The expert position seemed to have escaped these problems, which may explain its acceptance in design research: expert design practices seemed a certain basis for deriving design methods with, meaning that the design methods the expert position yields do not need empirical validation. Yet, given the arguments given in the previous section also the justification of the expert position will fall prey to the problems related to validation. This conclusion need however not to be taken to imply that the expert position cannot be justified. The described problems with validation may be resolved, as is argued in the remainder of this paper. This argument is meant to show that there exists an approach for empirically testing the expert position, yet only when one is willing to require that formulations of the expert position become more precise and detailed. However, this argument is not aimed at establishing that all design tools and methods should be validated by this approach. In design research there is, as said, neither consensus about how to validate design tools and methods, nor about whether this validation is needed in the first place. Blessing & Chakrabarti (2009) call for a thorough validation in design research and propose relatively systematic research methods for it. Seepersad et al. (2006) argue for the validation of design tools and methods and develop a research method specifically geared to this task (as will be explained below). But, for instance, Koskinen et al. (2011) take distance from this more systematic approach to design research.

In the same literature complaining about the lack of validation in design research one can find proposals for research methods that could make this validation possible (e.g., Frey & Dym 2006; Frey & Li 2006; Seepersad et al. 2006; Blessing & Chakrabarti 2009). These research methods are at first sight procedures that experimentalists can carry out independently of the design methods and tools to be validated. It can however be argued that these research methods for validation require that the prescriptive claims about design tools and methods are sufficiently specified for making them susceptible to testing; these research methods make validation part of the formulation of design tools and methods. Consider, for instance, the already mentioned research method by Seepersad et al. (2006) ⁷ . In this method the validation of the prescriptions of a design method proceeds by four steps (making up four quadrants of what they call the validation square). First, the structural validity of a design method is to be established by showing that the method is logically rigorous, internally consistent, mathematically correct and applicable to its intended domain. Second, example design tasks should be defined for verifying its performance, as well as an argument why these example design tasks are appropriate for, e.g., representing the full application domain of the method. Third, it should be demonstrated that the method gives successful results when applied to the example tasks and that these results are due to using the method. Fourth, an argument should be given that the usefulness of the method for the example tasks may be generalised to the full application domain of the method. This validation square method avoids the practical impossibility of (full) validation of a design method by limiting the empirical testing of the design method from its full application domain to a set of example design tasks. Hence, validation need not include unrealistic years-long experiments with large numbers of firms operating in heavily controlled circumstances; rather the example tasks should be chosen cleverly such that experimentalists can actually carry them out. However, this advantage comes at a price for those who formulate design methods since the choice of the example design tasks is with the validation square method a task that is part of giving a design method.

Return again to the expert position. Adoption of the validation square method would imply not only that an X-set of types of expert design practices is specified on the expert position, but also that example design tasks are defined for which these design practices should definitely lead to the claimed favourable properties $f$ . These example tasks should be representative to the full application domain for which the expert design practices in X are advanced, as required by step 2 of the validation square method. And these example tasks should be suitable for actual experiments for testing that the prescribed practices in X lead to the properties $f$ , for making possible step 3 of the validation method. Hence, merely saying that the prescription leads to favourable properties when firms follow them for a decade will not do; the expert position should also give example tasks that can reasonably be carried out in experimental testing.

In the previous section, the justification of the expert position was shown to be dependent on the induction and capability assumptions, hence the validation square method could also be applied directly to these assumptions (assuming again that the induction assumption is not tautological). For the induction assumption the example design tasks that should be defined are ones that represent the application domain of the design practices in X and to which these design practices can be applied in experimental testing. Moreover, that testing requires spelling out criteria by which it can be determined that a designer has actual properly carried out an expert practice in X. This testing is then aimed at determining whether design practices in X when carried out for the example tasks, lead to the favourable properties $f$ , either with certainty (the implication Case 2 of Table 5) or with a certain (qualitative) probability (the correlation Case 3 of Table 5).

For the capability assumption example design tasks should be formulated – not necessarily the same as for validating the induction assumption – that again represent the application domain of the design practices in X and to which these practices can be applied in experiments. The testing is now aimed at determining whether non-expert designers can carry out the expert design practices in X when taking up the example tasks. It then also becomes necessary on the expert position to specify what minimum capacities these non-designers should have – e.g.: none; a specific education; or a number of years of professional experience – and, again, by what criteria it can be determined that a designer has successfully carried out the expert practices in X.

Enabling its empirical validation thus implies that the expert position is spelled out in more detail than giving only expert design practices $\{e_{1},e_{2},\ldots ,e_{n}\}$ , their $x$ -factors, their types X and the favourable properties $f$ these practices have. When the validation square method is adopted for giving this validation, more precise claims should be added: representative example design tasks should be given for the testing of, say, the induction and capability assumptions; it should be specified which capacities the non-expert designers to whom the design practices in X are prescribed should minimally have; and it should be made clear what it means that such non-expert designers carry out the practices in X successfully. Validating the expert position thus requires that this position is made precise (Vermaas 2013) and hence shift the ‘burden of proof’ to the proponents of this position just as falsification does in the case of validating scientific theories (Vermaas 2014). It is not anymore the experimentalist who is faced with surveying the full application domain of the expert position and with figuring out how to collect evidence about whether its predictions hold (and allowing the proponents the defence that the experimentalist cannot test the real expert position application in industry). It is rather the proponent of the expert position who has to specify how it can be realistically tested and what such testing should confirm (allowing the experimentalist to do these tests and, if necessary, invent further tests).

Table 6. Validation tests by the validation square method for the induction assumption for the cases 2 and 3 given in Table 5, and for validation the capability assumption

The introduction of the validation square method in the argument that proponents of the expert position should spell out their prescriptions in more detail and indicate how it can be experimentally tested, raises the question whether this validation method is validated itself. In the literature on the validation square method this validation is given by applying the validation square method to itself and showing that this application has a positive outcome (Pedersen et al. 2000; Seepersad et al. 2006). Moreover, this method is now regularly used in undergraduate and graduate design research at the School of Aerospace and Mechanical Engineering of the University of Oklahoma ⁸ . However, the argument that expert position proponents should spell out their prescriptions does not hinge on the acceptance of specifically the validation square method; this detailing is also needed when one, for instance, opts for validating these prescriptions by means of the DRM approach (Blessing & Chakrabarti 2009). Which validation methods – the validation square method; DRM; or other alternatives – will be accepted in the design research community, if at all, is at this point an open issue. Hence my argument may be taken as conditional: if in design research it becomes generally accepted that prescriptive claims of accounts of design will have to be validated, then proponents of the expert position have to detail their claims, using the validation square method as my case.

6 Conclusion

In this paper, I critically considered the expert position by which methods for innovative design can be derived from expert design practices. This position is regularly though not generally held in design research as there are also other traditions in design research in which design methods are formulated on the basis of other sources than expert design practices. I presented a framework by which accounts of design are characterised by the way in which they define and relate general, descriptive and prescribed types of design practices, and argued that the expert position leads to conservative and progressive accounts. The expert position can be unpacked in a strict and liberal manner: the strict expert position generates conservative accounts by prescribing only types of expert design practices that have actually been carried out; the liberal expert position may however generate progressive accounts by prescribing types of expert design practices that are new to designers. I also analysed the justification of the prescriptions by the expert position. The expert position appears to be self-evident since expert design practices seem a certain basis for prescribing favourable practices. It was however argued that the prescriptions of the expert position depend on two assumptions: the induction assumption that all design practices that share the same structure as expert design practices are equally favourable; and the capability assumption that non-expert designers have the capacity to successfully carry out expert design practices. These assumptions may be tenable yet need empirical testing for their justification. Finally I argued that empirical validation is possible in design research if the prescriptions of the expert position are spelled out in more detail, for instance, by defining example design tasks as defined in the validation square method on which these prescriptions can be realistically tested in experiments.

Expert designers have created powerful innovations, which warrant analysis of their designs and of their practices. Yet taking expert design practices as a standard for how other designers should innovate is unnecessarily restrictive and conservative for developing design tools and methods. It rules out taking inspiration from the practices of avant-garde designers and from the insights of design researchers who deal with the societal and economic issues that are currently emerging. The upshot of the analysis in this paper is that the prescriptions as generated by the expert position need empirical support just as any other proposed prescriptions; the authority of experts does not make their design practices a certain basis for design methods, or a preferred basis as compared to other sources.

7 Glossary