Physical prototyping rationale in design student projects: an analysis based on the concept of purposeful prototyping

Abstract

Prototyping constitutes a major theme of design education and an integral part of engineering design academic courses. Physical prototypes and the model building process, in particular, have been proved to boost students’ creativity and resourcefulness and assist in the better evaluation of concepts. However, students’ usage of prototypes has still not been explored in depth with the aim of being transformed into educational guidelines. This paper presents an investigation of students’ reasoning behind prototyping activities based on the concept of Purposeful Prototyping, developed in the authors’ previous work. This is performed by identifying instances of prototype use in students’ design projects and by discovering which types of prototyping purposes they apply and to what extent, as well as by studying the relationships between purposes, early design stages, academic performance and project planning. The analysis of the results shows that prototyping can support students’ learning objectives by acting as a project scheduling tool and highlights the contribution of early-stage prototyping in academic performance. It is also confirmed that students’ limited prototyping scope prevents them from gaining prototyping’s maximum benefits and that they require strategic guidelines tailored to their needs. A new, improved list of prototyping purposes is proposed based on the study’s results.

1. Introduction and background

Prototyping constitutes an integral and inseparable aspect of any product development process in both professional and academic practices. Prototypes can take numerous forms, encompass multiple characteristics and can be made of a wide range of materials and fabricating methods depending on the designer’s needs and the project’s stage or nature, as seen in Figure 1 (Pei et al. 2015). According to the taxonomy established by Michaelraj, Hannah & Summers (2008), the four main characteristics of a physical prototype are its size, its type, that is, a novel or a modified existing solution, its material and its fabrication method. Prototypes can aid designers by serving multiple types of roles and purposes during design projects, as it is known that setting a clear purpose to be achieved by a prototype leads to better idea communication and more informed decision-making (Houde & Hill 1997).

Figure 1. Classification of prototypes [adapted from Pei, Campbell & Evans (2015)].

In engineering design university courses, students have to build a wide range of different prototypes during their projects to support their learning objectives. However, although prototyping is regarded as a major topic of design education, it is commonly viewed as an activity which is poorly understood and implemented by students in terms of the design process (Lande & Leifer 2009; Lauff, Kotys-Schwartz & Rentschler 2017). Therefore, this study has been primarily motivated by the importance of establishing the prototypes’ roles and purposes before initiating their creation, as well as understanding prototyping’s significance in engineering design academic courses and educational curriculums.

1.1. Definitions of prototypes

According to the English Oxford Dictionary, a prototype is ‘A first or preliminary version of a device or vehicle from which other forms are developed’. In engineering design, a prototype can be defined as ‘an approximation of the product along one or more dimensions of interest’ (Ulrich & Eppinger 2012), or ‘an artefact that approximates a feature (or multiple features) of a product, service or system’ (Otto & Wood 2001). Others state that any type of design representation that embodies aspects of functionality and look and feel of the final product can be considered as a prototype, including sketches and rough models (Gurjar 2015). Contrariwise, it is believed that the main difference between a prototype and a design concept is the fact that the prototype can be tested (Camburn et al. 2017). Relevant research has also defined prototypes by using their roles as a basis, namely as ‘representations of information’ (Ullman 2002) and ‘physical or digital embodiment of critical elements of the intended design, and an iterative tool to enhance communication, enable learning and inform decision-making at any point in the design process’ (Lauff, Kotys-Schwartz & Rentschler 2018).

1.2. Prototype roles in engineering design

Prototypes can increase the value of the overall design process by having a range of different roles, as summarised in Figure 2, according to the associated literature and design textbooks. One of the main attributes of prototypes is learning, which encompasses the delivery of various types of information to all stakeholders either by reinforcing their already gained knowledge or by discovering unknown and unexplored information (Lauff et al. 2018). Prototypes can answer questions related to product functionality and users’ preferences (Ulrich & Eppinger 2012), and they lead to unveiling unknown or unpredicted phenomena and detecting potential problems (Otto & Wood 2001; Hess & Summers 2013; Jensen, Elverum & Steinert 2017). They can also be used as both internal and external communicational tools, namely within the design team or between other stakeholders, respectively, which transfer information either in relation to functionality, project understanding or aesthetical features (Stowe 2008; Lande & Leifer 2009; Elverum & Welo 2014). Such information is particularly important, as it helps in avoiding misunderstandings and facilitates the acquisition of feedback which leads to the product’s refinement (Otto & Wood 2001; Camburn et al. 2017; Lauff et al. 2018, Deininger et al. 2019). Apart from enabling feedback and explanation of concepts, prototypes can also function as negotiating and persuading tools during design reviews and meetings (Lauff et al. 2020). In addition, prototypes can aid in the refinement of a design by being able to identify key performance features and reveal fabrication errors both while the process of building them as well as while interacting, testing and experimenting with them (Otto & Wood 2001; Camburn et al. 2017). As far as their advantages in integration purposes are concerned, by prototyping activities, designers can ensure the compatibility of parts and subsystems and confirm the product’s overall performance by building assembly prototypes (Ulrich & Eppinger 2012). In terms of planning and scheduling the design process, prototypes can be applied in order to set specific milestones by demonstrating that the product has reached a certain degree of the desired functionality, which leads to the informing of important decisions that should be taken (Otto & Wood 2001; Ulrich & Eppinger 2012). Prototypes as milestones can also create a sense of forward progress and enhance the psychological experience of a design team, can aid in planning the process in terms of time and cost and can facilitate the iterative refinement of prototypes (Thomke 2008; Lande & Leifer 2009; Gerber & Carroll 2012; Menold & Small 2017). The use of prototypes has also been proved very beneficial during the early design stages of exploring the design space and enhancing ideation, and has been linked to the processes of divergence and convergence, which are the expansion of multiple generated concepts and the selections of a smaller set of concepts, respectively (Hess & Summers 2013). Finally, prototypes are known to assist designers in uncovering and prioritising unknown and novel user requirements and therefore in reducing the uncertainty that exists in the fuzzy front end of a design project, a process that is commonly known as elicitation of requirements (Sutcliffe & Sawyer 2013; Jensen et al. 2017).

Figure 2. The seven roles of prototypes according to Petrakis, Wodehouse & Hird (2019).

1.3. Prototyping’s advantages for engineering design students

Since prototyping activities are considered a major part of design education, the advantages offered by prototypes to students are thoroughly discussed in the literature. In detail, through prototyping design, students can visualise their ideas and simulate features of the final design with the aim of providing answers to questions and minimising risk during their projects (Yang & Epstein 2005). Particular emphasis has also been given to prototyping’s cognitive benefits, namely the reasoning and understanding of a design problem during the early stages, as well as to the advantages offered by both learning by reflection and learning by thinking during prototyping exercises (Elsen, Häggman & Honda 2012; Böhmer et al. 2017). Prototyping can also support students’ problem-based and project-based learning, or in other words, ‘learning by making’, when it is facilitated by properly equipped modelmaking hubs known as ‘makerspaces’ or ‘fablabs’ (Jensen, Özkil & Mougaard 2016; Ramos & Wallace 2019). Similar findings regarding learning stimulation have been found by researchers Schaeffer & Palmgren (2017), who examined how different prototyping approaches reinforce ‘learning by doing’ and ‘learning by experiencing’. Since the overall design process is often considered as an open-ended design problem including high levels of uncertainty, prototypes contribute in reaching better design solutions by helping students to develop expertise of the product (Zemke 2012). Functioning as an enabler for students’ thoughts, prototypes also allow better communication among the project’s stakeholders, such as experts, teachers and potential users, and support students in tackling challenging problems (Berglund & Leifer 2013). Based on students’ reflections, prototyping activities can also be very effective in the planning of a design project as they offer the opportunity to work more efficiently by saving a lot of working time (Lande & Leifer 2009).

Physical prototypes, in particular, have been proved to be more beneficial in specific aspects of the design process comparing to other forms of representations, such as sketches or virtual models (Lemons et al. 2010). Through the use of physical models, students’ reasoning processes are supported and improved as they have the chance to demonstrate and explain physical phenomena while also understanding them better (Viswanathan & Linsey 2009, 2013). It is also known that physical models have the potential to be applied for the alleviation of design fixation, if used under certain conditions regarding time and cost (Viswanathan & Linsey 2013). They can also facilitate efficient communication by acting as a creative thinking prompt and by bridging the gap between the different backgrounds and interests of all stakeholders (Brandt 2007). The technical quality of end products, along with their functionality, is increased when a significant amount of time is spent on the fabrication of numerous physical prototypes during early stages of the design process (Elsen et al. 2012; Neeley et al. 2013). The actual process of model building, although still considered as ‘an often overlooked pedagogical tool that can enhance the engineering design process for students’, facilitates them in better evaluating and reflecting upon their concepts, aids in preventing potential early faults, increases their self-confidence in articulating and communicating their ideas and results in producing more useful and more functional models (Lemons et al. 2010; Menold et al. 2018).

1.4. Integration of prototyping into design education practices

In order to provide valuable insights in design education practices and deliver effective teaching of design courses by properly incorporate prototyping into them, the first steps are to explicitly understand the design process (Lemons et al. 2010) and then identify the most prominent professional prototyping practices and transform them into instructional frameworks (Lauff et al. 2017). As far as learning is concerned, Schaeffer & Palmgren (2017) suggest that prototyping exercises should be more explicitly associated to ‘learning by reflection’ and ‘learning by thinking’ phases of the learning cycle, in order to be more meaningful for students. Students also need clear guidance regarding the positive effects of prototypes, but most importantly how and when to operate them during the process, as for the successful delivery of a design, it is crucial to build the right type of prototype for the right purpose (Viswanathan & Linsey 2009; Böhmer et al. 2017). In addition, according to Houde & Hill (1997), an explicit purpose for each created prototype can aid designers in taking more informed decisions in relation to what types of prototypes they should build and therefore in communicating their ideas better. Moreover, it has been proved that following a prototyping strategy or an already established set of decisions which define prototyping variables can have a positive effect on the prototyping outcome by improving its design performance (Christie et al. 2012; Camburn et al. 2013). However, based on our observations during the delivery of several design classes, the activity of defining explicit prototyping purposes requires reflective thinking, which comes less naturally to students or inexperienced designers in general. Consequently, there is a clear lack of a prototyping strategy or tool that is adjusted to design education needs in order to foster students’ prototyping results and further develop their design skill set. Apart from the fact that such tools should comprehensively guide students through the design process, they should also be quite simple in order to be easily incorporated into already established design practices and to be embraced by both students and their educators (Lauff, Menold & Wood 2019). According to Menold, Jablokow & Simpson (2019), the effects of structured prototyping methods on design students have still not been explored in depth, resulting in the lack of information about students’ usage of prototyping tools. Berglund & Leifer (2013) also agree that prototyping-focused modules could clearly contribute to the enhancement of design engineering education practices; however, they state that even if there are some cases of structured approaches already applied in academic contexts, gaps and weaknesses still exist and have to be properly addressed.

As far as design educators are concerned, studies show that while they rate prototyping as imperative in engineering education, they do not appear particularly confident in incorporating prototyping activities into their courses (Jensen et al. 2016). Thus, this raises the need of providing appropriate support to educators and making them more capable of teaching design in a practical way that is better suited to modern project-based design courses (Dym et al. 2005), in which prototyping activities constitute a major part. Particular reference to the educator’s role in the management of prototyping activities is also made by Mikkonen (2017), stating that the teacher should discreetly monitor and manage students’ work by providing suggestions when needed, keep track of deadlines and create a prototyping-positive atmosphere.

1.5. Previous studies in students’ prototyping activities

In the existing related literature, there are notable examples of efforts which capture, understand and analyse prototyping activities of engineering students, while examining a number of different factors. Perceptions and usages of prototypes between students and professionals have been compared through the application of a quantitative approach which is based on surveys and semi-structured interviews (Lauff et al. 2017). Their key findings confirm professionals’ wider prototyping scope compared to students, which means that the latter have to be further informed in order to apply prototypes to their maximum potential. Such findings are in line with other studies which show that novice designers approach design tasks in a less efficient way, as they are not always fully aware of the type and amount of information they need to know, they demonstrate high levels of uncertainty in their decision-making and do not make use of established design strategies (Ahmed, Wallace & Blessing 2003; Ahmed & Wallace 2004). Deininger’s study (Deininger et al. 2017) also provides respected results regarding the conceptualisation of prototypes by inexperienced designers and results in the need for strategical guidelines and frameworks which are based on the lack of intentionality found in students’ prototypes. The relationships between students’ prototyping activities and their design outcome have also been explored, with a detailed focus given on the number of prototype parts and the time spent on the prototyping process (Yang & Epstein 2005). The results indicate that prototypes with fewer parts result in better design outcomes, the time spent on prototyping does not necessarily lead to higher project effectiveness and that spending more time on early-stage prototyping usually leads to better end products. Another study provides valuable input regarding students’ learning from prototyping activities. Based on the observation that professional design engineers usually require many prototype iterations, another study is based on the introduction of two prototyping cycles, with the aim of reducing risk in the design process (Zemke 2012). More recently, prototyping behaviours of undergraduate students in a middle-income country were also examined by presenting prototypes in different formats, in order to understand to which extent best practices are used beyond western designers (Deininger et al. 2019). It was found that virtual prototyping was preferred to physical models and that some of the most critical prototyping practices, such as communication and testing of concepts, were significantly utilised. However, the least used practices had to do with the early design process phases of engagement with stakeholders and problem definition. Other student-based studies reinforce the above statement by concluding that the application of structured methods and supporting tools raises students’ overall prototyping awareness and therefore results in the delivery of higher quality prototypes (Menold et al. 2016; Ramos & Wallace 2019). Along with the input of students’ self-reported behaviour, researchers particularly note that the employment of such frameworks and support platforms enables students to broaden their prototyping scope and extend their focus on multiple design qualities rather than only on technical aspects (Menold et al. 2019), as well as to recognise the prototyping methods available to them and increase their confidence during prototyping activities (Ramos & Wallace 2019).

2. Research aims and objectives

All the aforementioned studies of Section 1.5 provide noteworthy findings regarding students’ prototype usage; however, they focus on a range of different factors. In specific, the study conducted by Yang & Epstein (2005) examines time, number and design stage, while other studies draw attention to the usage of prototypes between novice and professional designers (Deininger et al. 2017; Lauff et al. 2017; Ahmed et al. 2003; Ahmed & Wallace 2004) or focus on prototyping behaviours according to background, experiences and cultural context (Deininger et al. 2019). Other research falls into the scope of evaluating structured methods and prototyping tools (Menold et al. 2016; Ramos & Wallace 2019; Menold et al. 2019); nevertheless, they do not explicitly study the purposes of prototyping. Therefore, with respect to this study, we have put the focus on the specific purpose of prototyping activities as the study’s key factor, building on our previous work that was established (Petrakis et al. 2019). In other words, the main goal of this study is to understand the reasoning behind students’ prototyping activities. We aim to realise what are the main reasons that prompt students to initiate the building of prototypes and examine the relationships between their used prototyping purposes and other factors such as early prototyping, academic performance and project management. This is achieved by initially identifying all instances of prototype use found in their delivered projects and then by performing a systematic analysis of the collected data.

As this case study is built on our developed concept of ‘Purposeful Prototyping’, another important reason for initiating it was to evaluate the practicality and the accuracy of the created list of prototyping roles and purposes and to realise how it can be applied in design projects. The findings of this study will assist us in defining any potential changes that need to be made and in properly modifying the created classification for it to be more adaptable by both design students and design educators, as well as more compliant with their project-based academic courses. In addition, although the majority of previous studies mentioned in the literature are controlling the collected information through the use of semi-structured interviews and questionnaires, we believe that an initial interpretation of students’ prototyping rationale, once they have already completed their projects and while being unaware of our developed list, can be proved very important as they have operated in a completely unrestricted manner.

In order to achieve a stronger understanding of our question, we have applied a mixed methods research through which we have qualitatively gathered the required data and then have analysed it using quantitative procedures (Creswell 2013). In particular, this research has been mostly directed by the following research questions:

1. (i) Which types of purposes, and to what extent, are used by student designers when they perform prototyping activities during their projects?

2. (ii) Does the usage of specific prototyping purposes contribute to the design projects’ quality and therefore to the students’ (academic) performance?

3. (iii) How are early design stage-related prototyping purposes applied by students and how do they affect the projects’ development?

4. (iv) What is the importance of prototypes and the prototyping process in the scheduling and planning of design projects and how are they used by students for these types of purposes?

3. Methodology

3.1. Scope of study

In this study, we have decided to focus solely on prototypes in the form of physical, tangible artefacts. These models facilitate understanding form, feel and function as well as physical testing, and are established as an important feature of a robust development process. To this end, we will consider every three-dimensional physical model found in students’ work, carrying either functional or aesthetical features of the final design, from rough mock-ups to detailed functional models, but excluding sketches and virtual or digital design representations. For example, we will be examining quick models created during early research phases or ideation exercises, all different types of prototypes formed during concept generation and evaluation as well as working models built in embodiment, detail design and final design stages.

3.2. Description of academic project-based module

This study was performed through the review of 1-year-long design projects undertaken individually by students taking part in three product development-based courses at a UK university. These projects are compulsory and were carried during the students’ final year of study. The module’s objective is the delivery of a product that represents a solution to a design or manufacturing problem. More specifically, each student is asked to address this problem and create a working prototype for their product; therefore, students have been encouraged by their instructors regarding the importance of prototyping and modelmaking during all the different stages of the design process, with a more specific focus given on early exploration stages, in addition to proof-of-concept modelling and physical testing. The themes of the projects have been suggested by the students themselves, and they retain full ownership of them. In some cases, students may also try to commercialise their delivered solutions or further develop them for entrepreneurial purposes, upon the completion of their studies.

The educational aim of this particular module is to help students gain experience in undertaking a major individual project, to prepare them to use design methodologies to design and prototype a product that meets specifications and user or market requirements as well as to effectively communicate their work using appropriate media and demonstrate proficient management of their projects. Apart from prototypes being key deliverables and strongly connected to the learning objectives of this module, they are also a major part of the marking criteria. Students are assessed according to their ability to investigate and discover using early models, as well as the quality of exploring, selecting and defining concepts through prototypes, including, of course, the quality and content of their final working models and prototypes.

The module is divided into two stages as seen in Figure 3, and at the end of each stage, there is a critique presentation where students present their work. The projects are also supervised on a regular basis by design lecturers who provide feedback and guidance. Students delivered their projects in the form of two files: a project report and a design portfolio, which are accompanied with created prototypes. The reports consist of 10,000 words (25 pages) in average and contain all the appropriate information about the projects, that is, technical information, calculations, design methodology and so on. Their text focuses on describing their rationale and decisions, and also includes their own reflections at the time of undertaking the required work. The design portfolios are an image-based documentation of the students’ design work, comprising of initial research, sketches, concepts, design methods and tools, photos of prototypes, digital models, renderings and technical drawings, and are 20 A3 landscape pages long in average. Providing that 133 projects were involved in this study, a total of 266 documents (133 reports and 133 design portfolios) were reviewed and analysed for the purpose of this study.

Figure 3. The suggested timescale for design projects.

Since all the studied projects were completed by either fourth- or fifth-year design students, it must be stated that they all have at least 4 years of engineering design experience in academic courses or through industrial work placements. During their time in the department, all students have received a foundational education in the design process and associated skills, including prototyping, and have been given the opportunity to apply them in various settings. Since these are open-ended design projects, they all have the freedom to approach them in their preferred way. For this module, students are also given an allowance of £50 for prototyping materials, and they all have access to the department’s facilities. In detail, they can use a digital design and manufacturing studio which accommodates different types of 3D printers, 3D scanners, laser cutters and CNC machinery, and they all have completed appropriate skills and safety training in order to be able to experiment and prototype with multiple materials, adhesives, hand tools, drills and sanders in a fully equipped modelmaking workshop.

As it was also mentioned earlier, 133 student projects were reviewed; however, our analysis for this study will be mainly focusing on 120, as 13 of the projects did not include a tangible prototype or a physical model, according to our used definition explained above. The total of 120 inspected projects have been undertaken by students who took part in three product development-based courses; 68 in product design engineering (PDE), 26 in sports engineering (SE) and 26 in product design and innovation (PDI) as seen in Table 1. Students from all three courses are given 40 credits for the successful delivery of this mandatory module. The projects are marked on a 0–100 scale, and all of them were anonymised prior to the initiation of the study according to confidentiality legislation. For the same reason, the projects’ marks were rounded up to the nearest five before being given access to and before the data analysis; for example, a mark of 62/100 was rounded up to 60/100, whereas a mark of 63/100 was rounded up to 65/100.

Table 1. Demographics of students and course description

3.3. Data collection

This research work can be considered as ‘historical’ and did not overlap with the students’ project progression, as the acquirement and assessment of the delivered projects was completed 12 months after they had been submitted and graded. The delivered technical reports and design portfolios were acquired in electronic files, and they were reviewed according to a developed assessment sheet that was used for each one of the participating students. The whole process of investigating the projects was performed thoroughly and comprised of an in-depth review, which lasted approximately 45 minutes for each one. The results were then transferred digitally and analysed using Microsoft Excel software. The collection and analysis of the data was performed by one of the authors, who was not involved in the module (teaching, delivery or assessment).

In order to validate the accuracy of the collected data, an inter-rater reliability test was conducted on a sample of the examined projects using another rater of comparable experience in design and prototyping (Creswell 2013). The reviewed sample consisted of 12 student projects, representing 10% of the total. For the purpose of determining the consistency of agreement between the two raters, Cohen’s kappa coefficient (κ) was applied (Cohen 1960). This method was chosen because it goes beyond the simple percentage of agreement calculation, as the factor of κ, which can range from −1 to +1, also considers the possibility of the agreement which occurs by chance between different raters. In assessing the results, the strength of agreement between researchers can be considered as substantial with an achieved kappa value of κ = 0.64 (Landis & Koch 1977).

The first part of the process was to check if any of the projects include any types of physical models or prototype use (Figure 4) during Stage 1 of the module, and in case they did, details of each prototype instance were noted accordingly. Students’ key quotes in relation to their prototyping usage were also gathered and archived by doing a comprehensive review of both their technical reports and design portfolios. This was done for the purpose of clarifying and better understanding students’ rationale behind their actions, considering that these quotes represent accurate reflections by the students themselves at the time of performing the required design and prototyping work.

Figure 4. A sports engineering student’s prototyping process and final outcome.

The concept of purposeful prototyping

The most extensive part of this study focuses on defining the purpose of each prototype documented in the students’ work. This factor is based on the concept of ‘Purposeful Prototyping’ and a list of 23 prototype purposes (Table 3) derived in the authors’ previous work (Petrakis et al. 2019). According to the developed definition, Purposeful Prototyping can be regarded as ‘the initiation of a prototyping activity through which the prototyping outcome will enable the achievement of specific design objectives’. The difference between the terms ‘role’ and ‘purpose’ is based on the definitions of ‘expected function’ and ‘intended use’, respectively. Consequently, for the Purposeful Prototyping concept, specific focus is given to the establishment of the prototype’s purpose, prior to initiating the prototyping process. The 23 developed prototyping purposes that are used arise from prototypes’ seven roles, as described above, and they can be also regarded as ‘subroles’ or ‘functions’ according to the more specific areas of each role where they are supportive to designers’ needs. For example, as far as their role in learning is concerned, prototypes can serve the purpose of answering questions regarding the product’s functionality and the users’ preferences.

The student projects were each assessed depending on which of the purposes are identified during the prototyping activities found in both the technical reports and the design portfolios as well as the level by which prototypes were used for each purpose, that is, explicitly or moderately used as explained by the codes used in Table 2. By explicit usage, we mean that the purpose is clearly used as also indicated by the student either through his/her quotes in text or by the way he/she presents the overall prototyping activity along with our personal assessment, whereas moderate use means that the prototyping purpose is used less evidently, to a lower degree. Also, we should clarify that one single physical prototype can serve more than one purpose at a time.

Table 2. Coding scheme used for assessing purpose usage

Table 3. List of 23 prototyping purposes and corresponding roles

4. Findings and results

4.1. Projects without prototypes

In 13 of the projects, there are not any physical prototypes found according to our used definition. We tried to identify the main reasons of this occurrence by doing a comprehensive review of the students’ reports and portfolios, as seen by the examples of their quotes and reflections on prototype usage in Table 4. All 13 students that did not create a physical model initially take account of the prototyping process in their management plan or Gantt charts, and all of them include prototypes, either rough, functional or aesthetic, in their list of deliverables, which shows their awareness of prototyping as an integral stage of the design process. Six of them also clearly refer to the purposes of creating a prototype, and two of them even base their methodology on prototyping-driven design processes. However, the fact that they did not actually achieved to prototype indicates that this awareness is only theoretical and that they must learn how to manage their prototyping activities properly in order to practically integrate them into the design process according to their needs. As far as poor project management is concerned, the main reason of failing to prototype is related to time constraints as students who were initially planning to prototype during Stage 1 had to move this work into Stage 2, ending up without being able to create a prototype at all. In five cases, the students were not able to build a physical model due to the nature of the project, for example, the design of a mobile application and coding-related projects, or because of facing technical-related issues, for example, large expense of materials and complexity of parts. At this point, it is important to mention that four of the students clearly regretted not creating a physical model and therefore tended to focus more on simulation testing, calculations and digital models to improve the overall quality of their project. As far as the academic performance of this student group is concerned, these 13 students achieved an average mark of 60.9/100, compared to the overall average of 66.5/100 among the 133 projects examined. This result is unsurprising as prototypes are considered as key deliverables of this module; however, it also indicates to some degree the contribution of a physical prototype in the design projects’ overall quality.

Table 4. Examples of students’ reflection quotes

4.2. Usage of prototypes in Stage 1 of the module

During Stage 1, only 15 out of 120 students (12.5%) engaged in prototyping activities and they belong to all three courses, 11 in PDE, 2 in SE and 2 in PDI. Their initial prototypes include modifications on existing products and rough mock-ups made of simple materials such as cardboard, foam, play-doh or paper.

Usage of prototyping purposes in Stage 1

As shown in Figure 5, the most used purpose (10 out of 15) was the prototypes’ character as a means of visual aid (8), either for initiating discussion through a focus group or for personal understanding of forms, shapes and volumes. Many (9 out of 15) also applied the use of prototypes in order to enhance their ideation (19) and therefore develop a wider range of potential design concepts. These purposes belong to communication and exploration roles, which is expected considering the need for feedback and ideation during the early design stages. Regarding the ones that were less used or not used at all, it is quite ordinary that they mostly have to do with refinement, validation and compatibility or integration of parts. Moreover, it is worth mentioning that students generally did not build models for comparing them with competitor products (20) (1 out of 15), for uncovering unknown factors related to performance (3) (2 out of 15) or for observing users interacting with early physical models (22) (4 out of 15), all purposes highly associated with the early stages of the design process. Finally, students do not seem to apply prototypes as milestones for planning their process (17), a result that may affect the overall project’s management, as it is discussed in the previous section regarding students who did not deliver a prototype.

Figure 5. Prototype usage during Stage 1.

In general, we can claim that students who prototyped during Stage 1 of the module seem to demonstrate a broader knowledge in relation to prototyping activities, their used purposes and the way in which they can apply them during the design process, considering that they also explicitly refer to them in their texts. This can also be seen in Table 5, which comprises of examples of students’ quotes found in their projects in relation to the purposes of their created physical models during Stage 1.

Table 5. Examples of students’ quotes regarding prototype usage during Stage 1

4.3. Usage of prototypes in Stage 2 of the module

Total usage of prototyping purposes

The majority of the students completed their prototyping activities during Stage 2 of the projects. As seen in Table 6, the most popular purpose (84.2%) for building a prototype among students is getting feedback in relation to functional and aesthetic features of their product (7) and falls into the role category of communication. In the case of these projects, feedback is acquired by either potential users, experts or collaborating companies, fellow colleagues in the form of focus groups or by their instructors during critique presentations and regular meetings. Furthermore, the majority of the students (82.5%) need prototypes for validating specifications and defining improvement margins (11) and a significant percentage of them (71.7%) for finding answers to questions regarding the product’s functionality (1). Many also appear to apply prototypes in order to communicate and explain their concepts’ both aesthetical look and feel (6) as well as functionality aspects (5), by 69.2% and 65%, respectively. Other highly used purposes include gathering of data through testing and experiments (12) and answering questions considering users’ preferences (2). Apart from these seven purposes, all of the remaining ones are applied by less than 50% of the students.

Table 6. Total usage of prototyping purposes (Stage 2)

As far as the least used purposes are concerned, the two elicitation purposes of uncovering unknowns by observing users engaging with prototypes (22) in addition to prioritising requirements and identifying target groups (23) are both quite poorly used with percentages of 17.5% and 9.2%, respectively. In relation to exploration, the purpose of building prototypes in order to compare them to competitor products (20) was only used by 6.7%. Conversely, using prototypes to enhance ideation (19) is quite a common practice, taught in design education; therefore, we must state that a higher percentage than 26.7% was expected. Similarly, prototype comparison for informing concept selection (21) was only used in less than half of the projects (45%). As far as demonstration is regarded, 15% of the students create prototypes and use them in order to reach a certain degree of functionality and therefore establish progress (18) and only 1.7% of them treat their prototypes as an opportunity to set milestones in order to plan their process (17). This specific result justifies to a great extent the unsatisfactory project management seen in a lot of projects, as also explained earlier in the case of students not managing to deliver a physical model on time, even if they had included it as one of their key deliverables. Finally, the very low percentages (14.2% and 11.7%) of revealing potential errors during the prototype’s fabrication (13) and providing insights to manufacturing and tooling issues (4) agree with the reviewed literature and validates the fact that the actual model-building process is commonly neglected, despite the value and knowledge that it can add to a design project.

Explicit and moderate usage of prototyping purposes

Tables 7 and 8 show the percentages of explicit and moderate usage of prototyping purposes, respectively, as purposes are sorted from most used to least used. The most significant conclusions can be drawn from purposes’ moderate usage (Table 8). The purpose of gathering experimental data through testing (12) is the highest used with 31.7%, followed by communicating the product’s aesthetics (6) by 25%; both purposes which are significantly used in overall. However, the third most used purpose in moderate usage is about identifying key performance features (10) by 23.3%, although it is not one of the most used purposes in overall. Another noteworthy observation has to do with the least moderately used purposes and particularly the purpose of getting feedback (7). With 6.7% as opposed to 77.5% when explicitly used, it appears that it is the only purpose with such a great variance in its type of usage. This is quite natural as since it is one of the most common purposes of creating a prototype, students are highly aware of it and build physical models while also clearly referring on this clear purpose.

Table 7. Explicit usage of prototyping purposes

Table 8. Moderate usage of prototyping purposes

Moreover, it is also important to discuss the case in which specific purposes are more used moderately than explicitly. These six purposes are identifying key features (10), revealing unknown information (3), providing insights to business-related concerns (4), establishing forward progress by confirming functionality and the two elicitation-related purposes (22) and (23). One possible explanation of this result might have to do with the ambiguous nature of such purposes as at least four of them involve the uncovering of unknown factors.

Prototyping purposes leading to high academic performance

In order to draw valuable conclusions regarding the contribution of prototyping purpose to students’ academic performance, we categorised the student projects into three groups according to their success, as seen in Table 9.

Table 9. Student projects’ marking groups

An initial observation is that the majority of the highly used purposes, in overall, are also the ones who are being used by the highest marked projects, whereas the usage of them in the least effective of the examined projects is significantly lower. More specifically, all Group 1 and Group 2 projects demonstrate high usage of purposes (1), (5), (6), (11) and (12); in contrast, the usage of these purposes in Group 3 is considerably lower, as seen in Figure 7. The graph was created based on the 10 purposes used most by students of Group 1 (highest marking group), in order to compare their usage percentages to the other two marking groups. As it can be seen, only the purpose of getting feedback (7) is highly used by all of the three grade ranges. Similarly, validating specifications (11) is also significantly used in all three groups, with 96%, 86% and 64%, respectively.

On the other hand, there are purposes which are underutilised by Group 3 in comparison to Groups 1 and 2, a result which gives an initial indication about the importance of them in project effectiveness. In more detail, answering functionality questions (1) is used by 52% in Group 3 as opposed to 79% and 80% in Group 1 and Group 2, respectively. Likewise, answering user requirements questions (2) is used by 48% versus 68% and 71%. Moreover, there is a noteworthy difference in the usage of testing and gathering experimental data (12), by 75%, 78% and 39% in Groups 1, 2 and 3, respectively.

In order to enrich our results, a linear graph was created in order to show the increase or decrease of the usage of each of these 10 examined purposes from the highest marked projects to the lowest marked ones. As shown in Figure 8, the purposes with continuous decrease of their usage from Group 1 to Group 3 are (5), (10), (11), (16) and (21). However, besides their continuous decrease, purpose (11) is still significantly used by Group 3 and purpose (16) does not show very high usages in all three groups. On the other hand, purpose (12), despite its low decrease between Groups 1 and 2 from 75% to 78%, it is considerably underutilised in Group 3. Therefore, based on the above observations from Figures 7 and 8, we can claim that the four purposes which appear to contribute more to the successfulness of a design project, when applied by students, are:

1. (i) explaining a concept through demonstrating a prototype’s functionality (5);

2. (ii) using a prototype to identify key performance features (10);

3. (iii) gathering data by subjecting prototypes to a series of experimentation and testing (12);

4. (iv) informing concept selection by creating and comparing multiple prototypes (21).

Usage of prototyping purposes related to the early stages of the design process

The next step is to identify how students approach early-stage prototyping and explore its contribution to project effectiveness. This is done by analysing their usage of the prototyping purposes which are mostly related to the preliminary stages of the design process (Table 10), initially against the three aforementioned marking groups and then in relation to the three participating design courses. At this point, it must be clarified that this part of the study is still related to Stage 2 prototypes, considering that the majority of the students prototyped during this stage (since only 15 out of 120 students prototyped in Stage 1). However, since Stages 1 and 2 are only referring to the projects’ timescale, Stage 2 includes the early stages of the design process for many students.

Table 10. Prototyping purposes related to the early design stages

For this particular part of the analysis, we do not include purposes of answering questions to either user requirements (2) or functionality (1) as we believe they are more related to the validation of already known features, as opposed to exploration and elicitation purposes which have to do with revealing unknowns. As it can be seen in Figure 9 and the direction of each line, all of the usages of the selected purposes, apart from revealing unknown performance factors (3) and observing users’ requirements (22), seem to decrease according to the decrease of the project marks, showing that they strongly influence project effectiveness.

As it is seen in Figure 10, the most used early-stage purposes are the identification of key performance features (10) and comparing prototypes for concept selection (21) in average. PDI students care less about identifying key features compared to PDE and SE students; however, this can be justified by the less technical, but more creative, nature of their projects. This fact also somehow explains their higher usage in enhancing ideation through creating prototypes (19). However, compared to PDE and SE, PDI students tend to not use prototypes in order to elicit requirements through stimulating user interaction with physical models. As far as purpose (21) is concerned, it is seen that, even though being highly used compared to the rest of early-stage purposes, only about half of the students of each course apply prototyping to compare their concepts. An explanation for this maybe has to do with students hesitating to physically build their concepts due to the longer time needed as opposed to sketching or other representation methods, which leads them to apply other selection methods such as dot-sticking and weighting matrices among the most common.

5. Discussion

In this section, the already presented findings and results of this study will be discussed according to the research questions mentioned in Section 2:

1. (i) Which types of purposes, and to what extent, are used by student designers when they perform prototyping activities during their projects?

   • In overall, students demonstrated a narrow scope during their prototyping activities, as far as their used purposes are concerned. The results showed that their most used purposes arise from prototyping’s roles in learning, communication and refinement, namely the most common roles found and discussed in literature (Table 6). Such results also agree with previous research (Deininger et al. 2017; Lauff et al. 2017; Menold et al. 2019) and build on the existing literature by verifying students’ limited thinking and narrow scope when undertaking prototyping activities and, most importantly for this study, when establishing the specific purposes of their prototypes. Although students of all design courses have been taught several design methods which include prototyping in all design stages, it is shown that they tend to stick with the traditional roles of prototypes, such as feedback, testing and validation, which can be explained by their lack of experience and the relatively low number of projects they have completed so far. Consequently, it is vital to try and transform students’ perceptions of prototypes through educational guidelines and enable them to maximise the value that prototypes can add to their design projects, when used appropriately. Moreover, it is crucial to highlight the importance and the benefits gained from the actual prototyping and modelmaking process, apart from the final outcome, particularly for students and inexperienced designers, as it also confirmed in previous research (Menold et al. 2018).

1. (ii) Does the usage of specific prototyping purposes contribute to the design projects’ quality and therefore to the students’ (academic) performance?

   • In order to draw valuable conclusions from purposes’ diverse usage percentages between projects with different marks, we had to compare how purpose usage is increased or decreased from the highest-marked to lowest-marked projects. Despite the fact that every single prototyping purpose can increase the quality of a project when used correctly and at the right design process stage, according to the results from Figures 7 and 8 along with appropriate observations and interpretations of them, we have justified the selection of four prototyping purposes which seem to contribute to academic performance to a higher extent. These have to do with the prototypes’ ability to explain functionality and identify key performance features, their role in experimentation and testing as well as their importance in evaluation and selection of concepts. However, we believe that these results cannot be generalised as the usage of such prototyping purposes might have been strongly affected by the objectives and structure of this specific university module and the guidelines given to students; nevertheless, they can still be proved important in the case of replicating this study in the same educational setting and among the same three courses and be used for comparison of past and future results.

1. (iii) How are early design stage-related prototyping purposes applied by students and how do they affect the project’s development?

   • A general misconception related to design prototyping is that it serves more the final design stages of validation; however, based on our results from both Stages 1 and 2, we can claim that prototyping during the preliminary stages can lead to better design outcomes and therefore higher academic performance. In detail, such purposes are related to the identification of key performance features through prototyping, the comparison of prototypes during concept selection and the enhancement of ideation by creating a wide range of prototypes. The usage of these prototyping purposes is gradually reduced along with the decrease in the project marks, a fact that verifies their importance in the successfulness of the design projects (Figure 9). Still, it must be pointed out that early-stage prototyping is shown to be more effective when approached in a more systematic and structured way, as shown by the results in Stage 1, where students clearly used and referred to their prototyping purposes as well as planned, to a certain degree, their projects based on prototypes and set up clear goals to be achieved. Nevertheless, the overall low usage of early design stage purposes (Table 6 and Figure 10) indicates that students from all three courses do not take full advantage of early prototyping’s benefits, especially in relation to exploration and elicitation purposes and revealing unknowns, and clearly agree with similar studies’ results (Deininger et al. 2019) regarding problem definition. Moreover, the lower than expected usage (26.7%) of enhancing ideation through prototypes (Table 6) raises the need of finding ways to assist students to boost their creativity by incorporating the teaching of it through more systematic and procedural ways.

1. (iv) What is the importance of prototypes and the prototyping process in the scheduling and planning of design projects and how are they used by students for these purposes?

   • The results of this study confirmed the importance of prototypes acting as scheduling and planning tools during the design process, as it has also been mentioned in several cases of the literature (Otto & Wood 2001; Thomke 2008; Lande & Leifer 2009; Gerber & Carroll 2012; Ulrich & Eppinger 2012; Menold & Small 2017). This is initially validated as seen in the case of students not managing to build and deliver any physical models during their projects due to time constraints and poor project management, although they include prototyping activities in their time management plans and consider prototypes as final deliverables (Table 4). These results show that students might be are aware of prototypes’ benefits; however, they do not have the ability and the experience to manage their prototyping activities and integrate them into their design processes. Such findings are also strongly related to the very low usage percentages of demonstration (or scheduling and planning)-related purposes found in both Stages 1 and 2 of the projects (Figure 6 and Table 6), in which students do not appear to apply prototypes as milestones for planning their process in terms of time or to use them to establish a certain degree of forward progress. On the other hand, students who managed to deliver prototypes during Stage 1 of the project seemed to be more aware about prototyping’s planning benefits by starting early their prototyping activities and clearly referring to their purposes (Figure 5 and Table 5). Therefore, we believe that there is a major need to address this issue by making students more conscious of the advantages provided by prototyping in efficient project scheduling either as milestones or decision-making points. This facilitates the enforcement of deadlines and provides support during important decisions which define the project’s next step.

     

     Figure 6. Explicit and moderate usage of prototyping purposes.

     

     Figure 7. Prototyping purpose usage against three marking groups.

     

     Figure 8. Linear graph of prototyping purpose usage against three marking groups.

     

     Figure 9. Early-stage-related purpose usage by three marking groups.

     

     Figure 10. Early-stage-related purpose usage by three design courses.

Based on the total of this study’s results, we must state that there is a clear need of developing a more systematic approach for the planning and the completion of prototyping activities, as far as student design projects and academic practices are concerned. Apart from the general results, such important discoveries also arise from the results of prototyping purposes’ moderate usage (Table 8), or in other words, when the usage of a purpose is less evident. Such results indicate that there is a need to teach them more explicitly to students, since they show their willing to incorporate them when performing prototyping activities, but do not manage to do it in a distinct way. This is also connected to the result of prototyping purposes used mostly moderately than explicitly which include identifying key features, revealing new information and the two purposes of eliciting requirements (Figure 6). Expectedly, all these purposes are related to each other due to their ambiguous nature and the uncovering of unknowns. Therefore, it is essential to develop systematic procedures which are easily understandable and applicable by students when undertaking design projects. As far as such guidelines are concerned, they have to address the total of prototyping purposes, from the most common to the less used ones. In this way, we will also have the opportunity to achieve a better balance between the usage of prototyping’s convergent and divergent types of purposes among all different kinds of engineering design courses of either technical-focused or creative-focused nature.

6. New proposed list of prototyping purposes

Based on the results and the above discussion of the key outcomes of this design student case study, a new list of purposes is developed, as seen in Table 11. This is done through a number of modifications on our previous developed list according to both students’ usage of prototype purposes as well as on the quotes collected from their reports and design portfolios. Examples of such alterations include:

1. (i) sorting of roles and purposes so that they are more adjusted to the stages and timescale of the design process;

   Table 11. New proposed list of prototyping purposes and corresponding roles

   

2. (ii) rephrasing of purposes in order to be familiar with students’ practices;

3. (iii) introducing subpurposes, which address the precise areas of each purpose, when needed, and can provide insights to the particular type of prototype to be built for each subpurpose;

4. (iv) adding two new purposes of engaging with prototypes to define the design problem (1) and assessing feasibility through a proof of concept (9) as we consider them very important in the particular case of student projects. In particular, both purposes (1) and (9) are based on the students’ own quotes and reflections found in their work, in relation to their justification of creating a prototype, while (1) also stems from examples of exploratory modelmaking exercises, for example, empathic modelling, form fitting and existing product modifications used by students in order to better comprehend the design problem assigned to them.

We believe that this new proposed list can find application in the teaching of project-based university design modules and enable both students and educators to develop their confidence and skill set when engaging into prototyping activities. It can be also considered as a step towards the structuring of the prototyping process according to the different uses of prototypes at different design stages. Moreover, the adjustment made to the ordering of this list so that it follows the stages of the design process enables it to guide students through the overall process and help them with some of prototyping’s critical phases, based on our results, such as early design stages and scheduling and planning. As far as the latter is concerned, this list could also provide support to educators when monitoring students’ projects and managing their prototyping activities based on deadlines. This list also has the potential to be used as marking criteria when assessing and grading students’ prototyping activities and, of course, teachers can also use these guidelines for preparing their lectures or for documenting and categorising prototyping case studies. In overall, educators could base their teaching practices on this list of purposes in order to broaden students’ prototyping scope, make them aware of prototyping’s uncommon benefits and encourage them to try and experiment with all of prototyping’s diverse roles and purposes.

We strongly encourage the usage of this prototype purpose classification for further research by the application of it in additional student projects as well as industrial practices. In this way, the feedback collected from all stakeholders can potentially act as the basis for the continuous development and validation of it into a complete educational prototyping framework.

7. Limitations and future work

Apart from the exclusion of any kind of virtual and digital prototyping from this study, there are also a number of other limitations that do not allow the generalization of the results, as far as prototype use in academic practices is concerned. Still, this paper presents the first conducted study during this research project and all the limitations mentioned are being considered as parts of future studies:

1. (i) The most important limitation has to do with this study focusing solely on projects completed individually. The involvement of student team projects would result in more valuable results regarding communicational purposes, both internally and externally, and especially in relation to prototypes’ functions as visual aids and persuasion tools. Moreover, this would provide important insights in relation to students’ diverse perceptions of the same prototypes and models within the same design team.

2. (ii) Also, the diverse nature and topic of each project, along with the specific nature of each design course do not particularly aid in drawing generalised conclusions. In addition to this, as it is also referred earlier, the projects were not marked based solely on physical models and the prototyping activities. Consequently, the participation of students in the same prototyping exercise, in which the delivered prototypes along with the modelmaking process would be appropriately assessed and marked, could provide more generalisable results for the design education landscape.

3. (iii) Another limitation has to do with the lack of students’ active involvement in this study, through their own reflections on prototype use, along with significant inputs from their course instructors, through questionnaires and interviews before, during and after the completion of their projects. This would facilitate the gathering of data through a controlled experiment and would provide additional validated results.

Regarding future work, apart from tackling the limitations mentioned previously, it would be valuable to compare students’ use of prototypes to professionals according to their used purposes and roles, in order to understand industrial practices and transform them into an educational framework of instructions and guidelines. In addition, the next step could undoubtedly be to assess their prototyping activities after they have been aware about the Purposeful Prototyping concept and the developed list of prototyping purposes and given the opportunity to apply them during their design projects. This would also provide the chance to practically evaluate the new list in an academic design environment as well as to gather students’ and lecturers’ feedback and reflections on its usage. Moreover, the development of a more detailed list which will be based on further delineation of the purposes of the prototype from the end artefact is a potentially important direction for future work in order to facilitate appropriate selection of prototyping means in relation to these purposes.

8. Conclusion

This paper presents an investigation of physical prototyping instances found in individual projects of engineering design students from three different design courses. The analysis of the results is performed with a specific focus given on the explicit purpose of prototyping activities, according to the concept of Purposeful Prototyping and a created list of prototyping purposes. This study provides valuable insights in relation to students’ prototyping usage and the analysis of the results highlights the significance of early prototyping, both as far as early project planning and time management as well as the actual early design process stages are concerned. It is also confirmed that students do not maximise the benefits of prototyping and require more explicit guidelines and encouragement, a fact that provides important visions for future design education curriculums. These conclusions could lead to the development of a new prototyping-driven framework functioning as the basis of a structured prototyping strategy tailored to engineering design students, with a potential to be applied in academic practices and improve the quality of design education. The paper concludes with a proposal of a new list of 25 prototyping purposes which was developed based on the study’s results and is tailored to design students’ academic projects.