Digitally gamified co-creation: enhancing community engagement in urban design through a participant-centric framework

Abstract

Urban co-creation is an approach to urban design that actively involves stakeholders and end-users in the design process. As designers increasingly use digital tools to manage design information, stakeholders and residents may find it difficult to participate, resulting in a lack of engagement. The emergence of metaverse technologies offers a crucial opportunity to employ user-friendly and collaborative tools, enabling more effective participation. In the study presented in this article, a custom-designed digital game with virtual reality environment was used to facilitate a series of co-creation workshops. The study focused on changes in participants’ experience by comparing baseline and endline survey results against the design outputs. It employed a holistic framework considering four dimensions: game design, participatory experience, learning outcomes and co-creation results. The findings indicate that the digitally gamified approach helped enhance participation and knowledge sharing, and even though game design ratings varied, the use of video games motivated engagement, particularly in an intergenerational context. The co-creation workshop design documented in this article offers new methods to enhance community engagement in urban design. Especially during digital transformation, it opens renewed discussions on balancing traditional output-driven approaches with more participant-centric methods and design objectives.

1. Introduction

Urban design is differentiated from urban planning by its focus, scope and skills required. As opposed to planning land use and infrastructures from a top-down macroscopic perspective, urban designers design city features through processes that require creativity, cultural sensitivity and a deep understanding of end-user needs (Carmona 2021). Due to the focus on user experiences of space, a co-creation approach can help to develop more responsive environments by actively involving stakeholders and end-users in the design process (Bødker, Kyng & Grønbæk 1995, 2000).

Co-creation methods can contribute to public spaces that better support community needs, community building and placemaking – spaces that are meaningful in people’s everyday activities and social interactions (Corcoran, Marshall & Walsh 2017). Public spaces are potentially fertile but challenging ground for co-creation (UN-Habitat 2016) – its publicness presumes the participation of the public, which may refer to ‘every single individual’ and ‘the entire population as a whole’ (HKPSI 2011). The diversity of co-creation participants may bring together people who are young and old, educated and illiterate, expressive or reserved. Therefore, tools and methods of communication become the key to translate individual needs, knowledge and everyday wisdom into a shared expression in a spatial language.

Sandbox games are increasingly used as participatory systems in urban design, in which citizens co-create with designers to build communities and their spaces (Sanchez 2021). For instance, UN-Habitat researchers (2021, p. 10) utilised Minecraft in ‘breaking down the digital divide and the skills gap, thus facilitating a more equal dialogue between experts and non-experts’. The rise of virtual reality (VR) environments further enhances social and spatial experience of gaming, with more accurate depth estimations and sense of presence (Pallavicini, Pepe & Minissi 2019). VR environments are simulated 3D spaces, providing users the opportunity to explore and interact within a virtual setting that resembles an experience of reality (Liang, Shaw & Green 1991). It can help participants understand design implications three-dimensionally, better grasp site quality through street views and user perspectives, engaging them in a more immersive and empathic manner.

However, the application of which represents challenges in designing complementary engagement methods and tools that can accommodate the specificity of participants needs. Furthermore, as game-based approaches can be powerful mechanisms to engage communities in important design decision-making, a diagnostic evaluation on participants’ experiences and participation in these processes is required (Devisch, Poplin & Sofronie 2016; Thiel 2016; Muehlhaus et al. 2022; Kavouras et al. 2023). How can engagement design in co-creation workshops be enhanced with more participant-centric approaches?

Along these lines, our study organised a series of co-creation workshops that iteratively feedback and refine engagement methods and tools through understanding participants’ experiences in the process. Seven workshops were conducted, inviting residents, designers and social workers to collectively reflect on and co-create an open space in a public housing estate in Hong Kong, with the objective to support placemaking. The feedback from each workshop was used to articulate the subsequent one, and changes in participants’ experience were observed following an evaluation framework synthesised from related studies. The results indicated how balancing tangible outputs with intangible outcomes can help drive a more participant-centric process.

A digital sandbox game tailored with the web-based VR platform Mozilla Hubs was used to facilitate co-creation (Figure 1). A preceding article detailing the game development process titled ‘Digital common(s): the role of digital gamification in participatory design for the planning of high-density housing estates’ (Ng et al. 2023).

Figure 1. Through participants’ experience and design documentation, this study evaluates seven public space co-creation workshops facilitated by a digital sandbox game that was tailored using a web-based VR platform.

This article first reviewed literature in digital gamification, participant experience and urban co-creation to abstract variables used for evaluation. Then, workshop design, gamification process and data collection methods are detailed. Next, results of design documentation and questionnaire surveys are analysed into two types of findings: output and outcomes. Finally, the article concludes with a discussion on means to enhance engagement design and the role of designers in this process.

2. Literature review

2.1. Citizen participation: Outcomes vs. outputs

Citizen participation in the public sector is a nuanced process encompassing various factors, including the temporal and stage-dependent nature of participation, such as co-commissioning, co-design, co-delivery and co-assessment (Nabatchi, Sancino & Sicilia 2017). While there is a latent willingness among citizens to be involved, self-interest and constrained perspectives in individuals are limiting factors for effective collaboration (Bovaird & Loeffler 2013). Especially within co-creation, the distribution of different capacities and skills among stakeholders requires the balancing of participative democracy and professional expertise, avoiding reinforcing existing inequalities (Steen, Brandsen & Verschuere 2018).

While co-creation can generate large amounts of data for designers and researchers to learn from, there is a neglected emphasis on how much participants can and have taken away from the reciprocal process, especially in an increasingly technological decision-making environment. During digital transformation, Namioka & Schuler (1990) pointed out how we often end up with systems that are difficult for users to master, poorly suited for their tasks and perceived by them as threatening. On the other hand, workshop sessions can be long and recursive, the complexity and scale of the design problem involving numerous parties towards technically and output-oriented design decisions (Costa et al. 2019). These can contribute to individual experiences as a participant being overlooked, leading to decreased engagement.

To capture the collective experience and iteratively enhance engagement methods, Tarr, Gonzalez-Polledo & Cornish (2018) recommended to document and study the experiential dimensions of co-creation workshops. Workshops can be designed in various ways as a means to promote genuine participation, with the common feature of arranging activities within a limited duration for participants with a shared agenda; the groups are often kept small to allow everyone’s attention and the opportunity to be heard (Ørngreen & Levinsen 2017). Calling for a more engaged understanding of design workshops, Rosner et al. (2016) considered how social alignments may be animated or resisted through the activities designed, by observing collaborative processes and their relationships to the workshop outcomes.

Scholars have distinguished different types of results that can be generated from an activity: outputs, outcomes and impacts. Output is the direct product of an activity, such as the co-created public space designs (Belcher & Halliwell 2021). Whereas outcome is a term often used in social science to denote what comes out of a social process, for instance, the participatory experience (Bhattacherjee 2012). On the other hand, impact differs from the previous two terms by its broader and longer term effects, for instance, how the workshops have affected the overall change in participants’ experiences (Harding 2014). By studying co-creation processes through the lens of participants, this study hopes to contribute to enhancing community engagement in the design field, specifically balancing output-driven approaches with outcome-driven objectives.

With a participant-centric focus, literature was reviewed to look for outcome variables that can help evaluate co-creation processes, with the following guiding questions:

• • To enhance community engagement, what are the key elements of gamification to consider?

• • What are the indicators of participants’ experience and the extent of knowledge gained?

• • What factors contribute to successful digital urban co-creation?

Literature was collected based on keywords – digital gamification and serious games, participants’ experience and crowd learning, urban co-creation and digitalisation. Around 50 articles were selected based on their relevance to the guiding questions. Variables or key concepts that were present in most of the literature were then used to structure the evaluation framework.

2.2. Digital gamification and serious games

Gamification is an emerging approach to foster engagement, motivation, civic learning and enjoyment in citizen participation (Hassan & Hamari 2020). Games are systems in which players engage in artificial, rule-bounded activities and interactions to achieve goals; it is a participatory form of recreation that is often socially significant, adding playfulness to establish rapport (Frissen, Mul & Raessens 2015). Gamification is the application of gaming elements in non-gaming contexts so as to enhance the experience, motivation and further behavioural outcomes (Cechella, Montezano & Mello 2018).

Playing games results in a basic expertise of the game, which can be seen as serious and powerful learning tools, as participants are spontaneously drawn into the game; digital gaming helps participants to be fully immersed in the playful experience, learning rules through interaction, rather than passive instructions (Adams 2014). Positive learning effects have been well documented in empirical research, both with respect to children and adult players; for instance, de Andrade, Poplin & Sousa de Sena (2020) demonstrated how children were fast learners with lucid abilities of urban design, which can motivate other social actors. This highlights gaming as a participatory form of learning that can be applied to facilitate co-creation projects, in which participants co-learn design knowledge and digital skills.

As playing games is an important part of social and mental development, elements that players find interesting or useful are crucial to promote effective learning (Ravyse et al. 2016). Scholars have diverging views over what drives game design. According to Schell (2019), crucial game elements are aesthetics, mechanics, technology and story. The MDA (mechanics, dynamics, aesthetics) framework introduced by Hunicke, LeBlanc & Zubek (2004) included sub-factors of sensation, fantasy, narrative, fellowship, discovery, expression and submission. Building on this, Dormans (2012) saw games as the engineering of emergence governed by mission, space, rules, representation and realism. Also, games can integrate motivational factors for learning, like leader boards, badges and rewards (da Rocha Seixas, Gomes & de Melo Filho 2016).

In the realm of serious games, learning through participation is seen as interactive practices for players to acquire knowledge on the social, political and economic realities of a community, allowing a deep and sustained understanding of real-world issues (Stevens 2007). Serious digital games can assist stakeholders to be more expressive via joyful activities like storytelling, walking, moving, sketching and drawing in an online collaborative environment (Poplin 2012). This new model of learning may change the future of public education. However, enhancing participants’ experience while balancing learning objectives requires more than tools engineering but careful community engagement design.

2.3. Participants experience and crowd learning

In community engagement, participatory experience can assist in the building of communities that serve as the primary platform for articulating aspirations, desires and solidarities, assuming roles and sharing responsibilities, while implementing tangible and practical actions (Adell et al. 2015). To enable and promote a transformative experience for participants, it is crucial to mobilise community knowledge, building capacities of integrated problem-solving and critical thinking (Tremblay & Jayme 2015; Hakio & Mattelmäki 2019). In co-creation, design thinking methods can help build these key competencies by making design knowledge accessible – gain insights into user needs, challenge assumptions, redefine problems, develop innovative solutions through prototyping and testing (Davis, Docherty & Dowling 2016).

Learning from user experience (UX) research, learning enhancement and knowledge acquisition can be beneficial to facilitate a ‘creative turn’ in participatory experience (de Bruin & Jelinčić 2016). Such crowd learning entails the collective sharing of experiences between individuals as a method of acquiring and developing new information and skills as a group. Kalisz (2016) demonstrated how crowd learning is essential for leveraging the knowledge, potential and expertise of individuals to effectively respond to challenges or tackle pressing issues. As such, crowd learning should precede cooperation for more informed participation.

To assess the quality of learning, methods have long been developed in the field of education to effectively arrange resources (Chang & Lehman 2002; Woo 2014; Lee & Hao 2015). For instance, ARCS (attention, relevance, confidence and satisfaction) is a descriptive model for diagnosing problems of learning motivation (Huang et al. 2004; Driscoll 2005). Building on this, Keller (2010) suggested a macro model of ARCS to measure learners’ performance (effort, abilities, knowledge and skills) and satisfaction (equity, consequences and cognitive evaluation).

As a form of active learning, co-creation activities possess motivational factors including collaboration, fulfilment, fun and playfulness (Minoi et al. 2019). In urban co-creation, Franz, Tausz & Thiel (2015) suggested that the activity design should help participants to understand value in community building, community commitment, diverse perspectives of stakeholders and social innovation. On the other hand, participants should be exposed to new digital techniques and local resources, engage in mutual cultural exchanges and learn to prototype through an in-depth understanding of local context and challenges (Wang, Bryan-Kinns & Ji 2016). Especially in spatial design, co-creation activities should help participants to understand the complexity of design problems and the implications of their collective decisions (Marušić & Erjavec 2020). To sustain longer-term impact, co-creation should aim to build participants’ capacity and confidence to engage and contribute, and for members to appreciate themselves as a collective (Bradley & Mahmoud 2024).

To bring about a transformative and personalised experience, equity, reflexivity, reciprocity and mutuality are crucial (Pérez Jolles et al. 2022). Participation should bring about criticality and collectivity via self-reflective inquiry, data sharing, collection and action, empowering participants through the consideration of lived experiences (Baum, MacDougall & Smith 2006).

In a study comparing participatory and conventional methods in design teaching, Shanthi Priya et al. (2020) found that community engagement approaches can improve students’ understanding of sites, concepts, design parameters, project needs and confidence in design decisions. The study demystified the initial stages of design for students through direct engagement with end-users, which enhanced students’ design performance over time. In a classroom setting, Phillips & Napan (2016) suggested that the learning space should be structured as a place for dialogic processes, guided by principles of mutual regard, so as to cultivate participatory experience that can be leveraged when engaging local communities. These learning models can be adopted in co-creation to increase engagement, while promoting knowledge and skill sharing.

However, Ali & Liem (2015) problematized the relationship between participants’ experience and co-creation outputs, as there is a lack of specific methods and tools that effectively translate experiences into tangible design objects, such as forms, textures and colours. Also, studies have questioned the rhetoric of ‘participation is always good’ by measuring participants’ disposition change, participation quality, commitment level, thinking modes and future engagement potential; from angles of social justice, participation does not equate to inclusion and is subject to individual experiences of the process – whether participants are being encouraged to be creative and foster changes (Ferreira, Azevedo & Menezes 2012; Leino & Puumala 2020).

2.4. Urban co-creation and digitalisation

In urban co-creation, inclusivity is a key principle that ensures all voices are heard and considered in a collective decision-making process, prioritising consensus-building amongst varying needs to address complex issues, during which trade-offs can be a difficult task, nonetheless, necessary (Löffler 2021). It is also important to create public and social values among participants as intangible outcomes, including network building and a change in mind-set (Haug & Mergel 2021). However, participants’ experience is a subjective quality criteria and can be influenced by their satisfaction with the co-created output, the perception of relationships built and whether they feel themselves as part of the majority with salience being agreed with and valued (Brandsen et al. 2018).

To promote cooperation, compromises can be an important factor of concern, which may facilitate satisfactory progress and advancement (Costa 2021). However, it is imperative to persist in the pursuit of consensus through compelling arguments based on pertinent values and comprehensive knowledge of relevant facts (Brewer 2018). Consensus generated from means of tolerance, civility, reciprocal obligations and mutual concessions is the highest-valued outcome of any cooperation (van Parijs 2012).

One study looked at cooperative urban design as a form of citizen science and debunked some of the negative assumptions, which include the supposed lack of participants’ creativity, macroscopic understanding, high-level processing and digitally mediated communication abilities. In fact, researchers have found that digitalisation can bring more responsive feedback, pleasure and joy that could encourage and educate citizens to get involved in community decisions (Poplin 2014). For instance, digital games can offer three-dimensional affective experiences, visualising words, meanings and signs, which enable participants to share opinions, stimulate imagination and express creativity (Ermi & Mäyrä 2005).

Furthermore, immersive environments can help to build spatial consensus among stakeholders – take the case of Quick Urban Analysis Kit (QUA-kit) as an example, a digital tool for participatory urban analysis with an accessible interface and geometrical play-pieces to crowdsource citizen’s ‘image of the city’ (Gordon & Schirra 2011). When utilising these virtual environments, Silcock et al. (2021) stressed on aspects of interaction, facilitation and communication to help participants understand 2D and 3D information, so as to participate in the design process proactively. All in all, digitalisation can create new affordances in co-creation through design empowerment, design learning, design conversations, design consultation and cooperative design research (Pak & Verbeke 2014).

2.5. Key variables in measuring co-creation processes

Table 1 summarised key variables that were present in most of the literature. These variables were grouped under four key themes – game design, participatory experience, learning outcome and co-creation results. Amongst which, the first four under each category were used to structure the evaluation framework.

Table 1. Indicators to measure the qualities of a co-creation process.

3. Evaluation framework – The four dimensions

The literature review discussed the potentials for digitally gamified co-creation in bringing more creative ideas and original citizen perspectives to urban design, while facilitating collective decisions and encouraging cooperation in an immersive and engaging manner. In view of such potentials, this study focuses on enhancing community engagement with a digitally gamified approach. In theory, it can help relieve knowledge and skills gaps through ‘play’ as a form of active learning. In practice, it may accelerate the situation and negatively influence the experience of participants who are not familiar with this form of media.

Such challenge emphasises the importance of evaluating co-creation not merely by the quality of the outputs (the proposed design solutions) but also through participant-centric outcomes (their experiences with the process). Based on Table 1, the recurring outcome variables were used to structure the evaluation framework with four dimensions and sixteen levels (Figure 2).

Figure 2. The four dimensions and sixteen levels of the evaluation framework.

First, game design looks at whether the design of the game can help motivate engagement, facilitate expression and provide a spatial experience. Also, the game mechanics should be simple and user-friendly. Second, participatory experience can be evaluated by looking at whether the co-creation process can promote critical reflection on existing circumstances, creative thinking, aspirations to continue the effort and empowerment to help others. Third, learning outcomes focus on the knowledge that participants have acquired, specifically on the site context, local communities and design complexity, with increased confidence. Fourth, it is important to evaluate whether participants are happy with the co-creation results, whether the outputs can better suit their needs and are built on common consensus that can help them feel included as part of the larger community. In our study, these criteria were used to inform the research methods.

Finally, all variables were being reviewed and translated into Cantonese for questionnaire design. In the process, variables that might lack clarity and tangibility, especially in Chinese, are highlighted for adjustment.

First, empowerment in participatory experience. Relevant studies have shown that empowerment in social innovation can be a fragile and blurry term, also, it is not enough to consider empowerment without sustainable efforts and ecosystem building (Hochgerner 2018; Ohnishi et al. 2024). Thus, the framework used ‘sustainability’ to represent if participants are willing and able to help others in the future.

Secondly, design as a learning outcome. The term design has been reconsidered as it is a broad concept that encompasses the entire process from creating solutions to problem-solving during implementation. However, our co-creation focus involves combining and integrating various ideas, insights and information gathered during problematization and ideation phases to form coherent concepts or solutions. As such, the term ‘synthesis’ had been used.

4. Methodology

The research method employed participatory approaches by involving different stakeholders and end-users to co-create a local open space in a public housing estate of Hong Kong – Jat Min Chuen (JMC), Shatin. This site had been chosen for its generalizable character, as one of the projects built during the 1970s–1980s to provide decent living environments for over 1.8 million citizens (Hong Kong Heritage Museum 2004). These estates are facing the ‘double ageing’ problem – both the residents and facilities are growing old – and the government is gradually retrofitting the public spaces to better suit the needs of a changing demographic (Tsang, Chan & van Ameijde 2022).

Using a tailored digital sandbox video game with VR environments, participants worked in teams to propose new public space design for the central plaza of JMC (Figure 3). For the purpose of this study – assessing participant-centric outcomes – the evaluation method utilised baseline and endline questionnaires to compare participants’ changes in experiences and any observable patterns between different stakeholder groups. The surveys were designed based on the evaluation framework (Figure 2).

Figure 3. Site plan of Jat Min Chuen public housing estate. The co-creation process targeted the central plaza, an open space surrounded by three residential tower blocks (HKHS n.d.).

4.1. Workshop design

The goal of the workshops was to understand issues surrounding local public spaces and ways in which we can design for placemaking. A series of seven co-creation workshops took place, to which participants reflected on and co-created the central plaza using the tailored VR game. Based on recent co-creation efforts of local organisations in Hong Kong, such as JCDISI (2023), HKCSS (2023) and Commchest (2023), it has been observed that one of the most common workshop design is the standalone practice-based approach with approximately 2–3 h of duration, following the design-thinking method – a consistent structure of empathise, define, ideate, prototype and test:

• • Introduction (20 min) – PowerPoint presentation to introduce concepts of placemaking and community building, the estate’s history and facilities and guiding questions based on previous research by a local organisation (JCDISI 2021):

  • ○ How to implement spatial integration to promote more community exchanges?

  • ○ What is age-neutral design to you?

  • ○ What kind of intergenerational facilities would your community need?

  • ○ How to make use of unique spatial characteristics of the local context?

  • ○ Is it possible to utilise available technology for innovative community space?

  • ○ How to achieve user-centric management in the estate?

• • Empathy (20 min) – Persona exercises and participants’ journey maps were conducted.

• • Define (20 min) – Participants worked in teams to conduct spatial analysis. Photographs of the plaza, playground, entrance and elevated walkway in JMC were provided to identify potentials and limitations of existing designs.

• • Warm-up VR activity (20 min) – Participants competed in teams to navigate the VR environment to get used to the controls.

• • Ideation (20 min) – Participants identified design goals, community activities and facilities.

• • Prototype (60 min) – Participants visualised their ideas in the digital sandbox game. Team members collaborated to divide tasks, including:

  • ○ Sketch a plan and accordingly.

  • ○ Choose from the provided 3D asset kit and place them in the VR environment.

  • ○ If their desired asset cannot be found, participants can:

    • ▪ search for 3D assets online or

    • ▪ draw the item, take a photo and place it in VR as a proxy.

• • Review using VR headsets (throughout the prototyping process) – Participants were invited to review their design from street views using VR headsets anytime; however, due to health and safety concerns, they can only be immersed in the headset 5 min at a time. Each team would share one headset and they would take turns, while the other team members made changes to the design via the computer.

• • Presentation (20 min) – Each team presented their designs using the VR environments, engaging other teams in discussions and receiving comments.

Considering the convenience of older adult participants, the workshops were largely organised within the estate in a local NGO centre – H.K.F.Y.G. Jockey Club Jat Min Youth S.P.O.T. (HKFYG) – or the closest university in the same district (CUHK). Due to venue availability constraints, two of the workshops had to take place in the Construction Industry Council (CiC) and Central Ferry Pier (CFP), respectively. Both digital and physical tools were used in most workshops, except the first – the initial test only used physical tools, including worksheets, maps, sticky notes and hand drawings (Table 2).

Table 2. Organisational details of each workshop event.

4.2. Participant backgrounds

Participants were recruited through the aforementioned NGO and university, involving end-users of JMC public spaces, local residents, social workers who provide public services in JMC, secondary school students studying in the area and university design students. A total of 75 participants were invited (Figure 4), of which ~40% were male and ~60% were female. The age distribution is made up of ~20% youth and ~20% older adults. Participant’s computer skills were assessed using an e-questionnaire before the workshop, with a self-reported rating on a Likert scale of 1–5 – ‘I have experience with VR or video game’:

Figure 4. Distribution of participants’ gender, age and role.

Each workshop had an equal mix of stakeholder and age groups, evenly distributed to work in teams of 4–5. Based on their background and previous experiences, each team would have at least a member of designer, resident, social worker and a person with sufficient computer skills.

4.3. Gamification using VR environments

The co-creation workshops were facilitated using the tailored digital sandbox game (Figure 5). Sandbox games are a genre of video game that offers an expansive 3D world for players to freely explore and interact (UN-Habitat 2021). In our game, participants enjoyed a considerable level of autonomy to mould the space according to their preferences through adding 3D assets. There were no predetermined game rules in terms of what the participants can or cannot do; however, they were guided by the set of placemaking questions (section 4.1).

Figure 5. Screenshots of the designed digital sandbox game with 3D assets of public space furniture.

The VR environment was designed with a digital twin of the central plaza, published using a web-based VR platform (Mozilla Hubs), which enabled multiplayer online collaboration. The 3D scene was 1:1, and the site was approximately 500 m². It was customised to have pastel tones to create a more abstract and interactive atmosphere. A set of 3D assets with public space furniture were designed for participants to engage with (Ng et al. 2024).

The digital game underwent enhancements after each phase (α versions) with the aim of enhancing the design output, while minor issues from participants’ feedback were addressed following each workshop (β versions). The former involved internal testing that primarily focused on assessing the game’s functionality and usability, generally involving design students and conducted at the university in a controlled and straightforward environment. The latter were tested in the targeted context with real-world settings, placing greater emphasis on reliability and overall user experience, involving end-users and other stakeholders (Figure 6).

Figure 6. Photo documentations of the co-creation workshops.

Game version 2α was the first version used for testing and includes modelling the site environment, preparing the triggers, 3D asset kit and media frames. After the first test, it was realised that the 3D scene got too heavy when participants added more assets simultaneously. During 2β, minor troubleshooting took place to optimise each asset under 500 kb. During phase 2, it was observed that participants easily got lost in the VR environment when the sky was blocked by other elements. Thus, version 2.5α turned all of the media frames invisible to maintain a clear line-of-sight around the scene. In 3α, the 3D asset kit had been expanded according to the collected community preferences from previous workshops. Subsequently, 3β optimised each of the assets to streamline connectivity.

4.4. Survey design and analysis

The evaluation aimed to generate insights based on changes in participants’ experience on the four dimensions. The baseline and endline questionnaires were designed to be symmetrical (Appendix A); a Likert scale of 1–5 was used (Figure 7). All questionnaires were electronic and the response rate averaged >85% (Appendix B). When analysing the data, the median was used to represent the score of each level. Then, the mean was calculated to average the score of each dimension. Finally, differences between baseline and endline results were calculated for each dimension and each level of each workshop (Appendix C).

Figure 7. A sample of an endline question.

5. Results – Workshop (WS) survey

For game design, the feedback was diverse; it had both highest and lowest scoring of all levels (Figure 8). The highest rating was from WS2.2 and WS3.3 (out of 5, average scored 4.25 and 4.5, respectively; Appendix C). Most participants thought the VR environment helped with a better understanding of the site and space (‘space’ scored 5 and 4, respectively). The lowest scoring was from WS2.1 and WS3.1; all participants had never been to the site and both workshops used alpha versions. Most of them reflected that it was difficult to find direction in the virtual world and they easily got lost.

Figure 8. Baseline and endline questionnaire results (no bar = no data).

For participatory experience, the feedback was highly positive, and none of the levels scored below 4. The highest rating was from WS1 and WS3.3 (average score 4.5 and 4.75, respectively). The former thought the activities have enriched their imagination, enabled them to understand problems from different perspectives and express their visions for urban development (‘creativity’ scored 4.5). The latter strongly agreed that the activities have inspired them to think outside-the-box. After the workshop, they felt that the public facilities in the estate should be improved, and they would be able to help others in similar co-creation processes.

For learning outcomes, feedback was positive, and most workshops scored above 4, except for WS1 (scored 3.81). The highest rating was from WS2.2 (scored 4.5), followed by WS3.3 (scored 4.33); these involved a high percentage of local residents, and the average number of participants were older. All participants highly agreed that they have learned things about their neighbours and neighbourhood that they didn’t know (‘community’ received full scores twice). The lowest performance was ‘context’ in WS1 (scored 3.5); most participants slightly agreed that they learned more about old districts. Nonetheless, all other levels in this workshop scored 4 or above, including ‘space’, ‘dynamics’, ‘creativity’, ‘design’ and ‘community’. As the initial test used only physical tools, WS1 replaced the introductory presentations with a series of short animated videos, interlude by Family Feud style interactive question games that shared survey results of residents’ opinions on public spaces.

For co-creation results, feedback was diverse; it had the highest and lowest scoring of all levels. The lowest scoring was WS2.1 (averaged 3.25), underperformed in ‘needs’ (scored 2.5) – participants slightly disagreed that the co-creation can better meet their daily needs. This was the first test of the digital game; the control was particularly challenging and all teams struggled to finish. Participants did not have enough time to complete the design and discuss in-depth during the collaborative process. Lessons learnt in this workshop helped inform the subsequent ones; all other workshops scored 4 or above in ‘satisfaction’ and ‘needs’. The highest was from WS2.2 (scored 4.5), which did particularly well also in ‘inclusivity’. All participants were happy with the co-created solutions, which can better meet their daily needs. Also, they found that most of the others wanted similar things as them, helping them feel inclusive as part of the larger community.

6. Findings

6.1. The outcomes – Participants experiences

The workshops exhibited favourable outcomes in participants’ experiences, on a scale of 1–5; all dimensions scored 3/+, 66.7% scored 4 or above, only two levels (‘mechanics’ and ‘needs’) occasionally scored below 3 (Appendix C), with the exception of WS2.1 – the first workshop that deployed the VR game (Figure 9). Overall, the scoring trend line is positive over the seven workshops, indicating consistent improvements and the effectiveness of interventions between phases. Despite initial score drops with each version upgrade, beta testing led to score improvements after troubleshooting.

Figure 9. Changes in participants’ rating and digital game version of each workshop.

Digital gaming motivates engagement even if participants find it difficult to control and navigate. The lowest performance was ‘game design’, achieving only 5% increase (Table 3). Although ‘mechanics’ and ‘space’ experienced ~10–15% decrease, ‘dynamics’ performed well (+45.7%) and achieved full scores in several workshops. It showed that the game process was interesting and increased participants’ engagement with others despite the technical challenges, but there is a need to improve skill-sharing of gaming controls and the design of VR environments.

Table 3. Changes in participants’ responses on the four dimensions.

Enclosed spaces in a VR environment are suitable for more experienced participants; also, drag-and-drop is easier to engage with than grab-and-place. From the endline survey, participants of WS2.2 and WS3.3 were quite happy about the overall game design but had diverging views over how easy it was to orient themselves in VR environments. Both workshops had high percentages of local residents who are familiar with the site; however, participants of the former thought it was difficult to find direction in the virtual world. They were working with game version 2β, to which lines-of-sight were sometimes blocked by media frames. Whereas participants of the latter used 2.5α, with the three residential buildings and the sky clearly visible all around the scene, which helped them to orient themselves. On the other hand, participants of the former often had to grab 3D assets from one location and place them in another, whereas participants of the latter were being instructed to always drag-and-drop assets into the scene, minimising the need to navigate.

Disparities in digital competencies did not significantly impact participants’ experiences. Although WS2.1 had the highest self-reported baseline computer skills (median = 4.5/5), they also reported the least satisfactory workshop experience. Conversely, WS.2 and WS3.2 both showed strong computer skills but received differing workshop experience ratings. In contrast, WS3.3, despite reporting the lowest computer skill proficiency, received the highest overall workshop rating.

Intergenerational interaction, particularly those involving a higher percentage of older adults, can strongly enhance the overall co-creation experience. The highest overall rating was WS2.2 and WS3.3 (~90%), both had more older-adult residents than other workshops. These participants generally considered themselves quite extroverted, enjoyed helping others and engaging the community. The lowest was WS2.1 and WS3.1 (~70%); all participants were design students who were knowledgeable in urban design, but little to no experience in community engagement. They were not recruited on a voluntary basis – the workshop was conducted as part of their course activity.

Co-creation is inherently a learning process, regardless of workshop quality. All dimensions achieved positive changes, with ‘learning outcome’ having the biggest and most significant ~30% increase – even though the quality of workshops varied, all participants agreed that there was much to take away from the process. This demonstrates how co-creation is first and foremost a form of learning; therefore, it should be designed with active learning objectives and methods.

All in all, most participants were satisfied with the workshop experience. When being asked the question ‘the activity is interesting and attractive, if there are similar events in the future I will participate again and recommend friends to participate’, 100% of participants of WS2.3, WS3.2 and WS3.3 agreed, followed by WS1 and WS3.1 (81% and 87%, respectively).

The participatory experience scoring readily increases as more stakeholder groups are included. The endline survey showed that participatory experience performed the best (averaged 4.31 for all workshops). However, when examining potential trends among various stakeholder groups, we delve into the recent three workshops, each progressively incorporating more stakeholders:

• • WS3.1 involved solely designers.

• • WS3.2 integrated youth residents and social workers.

• • WS3.3 encompassed a diverse mix of all groups.

Visualising the data with evaluative compasses, participants’ ratings for each level were clearly depicted, where a larger area indicated a higher rating (Figure 10). A pattern has emerged – as the number of stakeholder groups increased, the overall ratings for ‘reflection’, ‘creativity’, ‘sustainability’, ‘community’ and ‘satisfaction’ also rose. Despite all workshops utilising the same game version 3, the last workshop stood out with ‘dynamics’ and ‘expressiveness’ achieving top scores – participants found the gameplay more fun and engaging, facilitating self-expression and consideration of their needs.

Figure 10. Participants’ evaluation of workshops on a Likert scale of 1–5, 0 = no data.

6.2. The outputs – Public space design

Considering WS3.2 and WS3.3 employed identical game versions but varied in participant composition (with/without older adults), with the latter achieving higher participants scoring, we shift our focus to their co-creation outputs – the public space designs. They comprised 3D scenes and hand-drawn sketches, annotated with group presentation contents (Figures 11 and 12). The designs were similar in their spatial program planning, which highlighted potential common needs of local communities – both designs featured zones for exercise, playgrounds, commercial activities and performances, with a central grass lawn as the key element. However, they varied in the community activities proposed and their organisation logics in response to the guiding questions (Section 4.2).

Figure 11. Public space co-creation outputs of WS3.2.

Figure 12. Public space co-creation outputs of WS3.3.

Spatial integration to promote community exchanges – Participants of 3.2 incorporated two community support spaces that are trendy among the younger generation, including a book-sharing club and a community expression area, both intended to be spaces for proactive interactions. Whereas 3.3 had a water feature of a massage pool, where residents, especially older adults, can relax and chat, while engaging in passive exercises. On the other hand, while both designs proposed commercial areas, the focuses were different. Participants of 3.2 designed a weekend market to build up the community economy, with local stalls to enhance the quality of life for residents, whereas 3.3 proposed to have shops for daily grocery and affordable goods for the convenience of local communities, particularly those with lower physical mobility. The former focused on entertainment and economic development and the latter highlights inclusivity and accessibility.

Age-neutral design and intergenerational facilities – It can be observed that 3.2 designed spaces for different age groups, centring on recreational activities that are largely periodic, but 3.3 focused on practical uses to suit daily intergenerational needs. In 3.2, participants proposed areas for Tai Chi, elderly gym and Cantonese opera, whereas 3.3 proposed a pharmacy, supermarket, community canteen and so on. They also proposed to turn part of the grass lawn into a community farm, so older adults can have a reason to spend some time outdoors every day. Further, more emphasis was put on aligning the daily activities of children and older adults, including play and exercise equipment, showcasing a borderless design to promote spontaneous dialogue between neighbours.

Considering the unique spatial characteristics of local context – Both designs had emphasis on enriching community activities but demonstrated different logics towards how to utilise the spatial characters of the site. Take the side nearest to the main bus stop as an example, 3.2 put a performance stage to attract more foot traffic, whereas 3.3 put the commercial area to be benefitted from the existing human flow. Another example is the area next to the residential tower on the west, which is generally shaded from extreme sunlight in the summer, 3.2 put the booking-sharing space so people can spend time reading comfortably and quietly, whereas 3.3 situated the performance stage there to shield performers and audience from the weather.

Technology utilisation for innovative community space and user-centric management in the estate – Proposal for technology adoption is limited; however, 3.2 utilised audio-visual systems for movie nights and performances, whereas 3.3 had no specific mention of such. Interestingly, both designs showed distinct management styles. As mentioned, 3.2 dedicated a small area for community expression to collect residents’ voices to be considered by management, whereas 3.3 proposed a common stewardship system of the community farms through drawing lots. This demonstrates the various ideas towards what constitutes equity and fairness by participants of different social groups.

Comparing the co-creation outputs with the existing design of the local public space (Figure 13), it can be observed that the latter attempts to reserve a vegetation oasis for the central plaza to counteract the high-density context and provide a pleasant green feature for residents to look at, especially from the surrounding tall residential towers. However, residents are being restricted to access the grass lawn so as to protect the vegetation. The co-creation outputs demonstrated that, as much as residents would like to keep the grass lawn, they wish to have more community activities in the area, fully utilising empty spaces in the estate to diversify spatial programs.

Figure 13. The existing site’s large grass lawn restricts access for residents.

Overall, although WS3.3 had more stakeholder groups involved and achieved a higher overall rating in participants’ experience, both workshops resulted in unique public space designs that considered varying community needs (Table 4). It demonstrated how involving a different composition of participants may impact both the outcome and outputs of co-creation workshops. Organisers should carefully plan the composition to achieve different design objectives.

Table 4. A summary of how the design differs from WS3.2 and WS3.3.

7. Discussion

7.1. Enhancing community engagement

To enhance community engagement, several strategies can be summarised from the study, including output–outcome balance, participant-centric design and intergenerational engagement.

In balancing output-driven approach with outcome-driven objectives, results have shown that co-creation workshops, which underwent beta testing, consistently scored higher than those subjected to alpha testing. The different attributes to alpha upgrades were output-driven (focused on enhancing public space designs), while beta upgrades were outcome-driven (focused on enhancing participants’ experience). For example, to diversify outputs of design, the 3D asset kit in workshop 3α was expanded. However, this resulted in decreased outcome scores for game ‘dynamics’, ‘mechanics’ and ‘space’, which indicates the gaming experience became less engaging, controllable and easy to understand. Participants commented the increased assets made navigation and comprehension challenging. After consideration, all assets in workshop 3β were being simplified to retain the size of kit without overwhelming participants. This iterative cycle of feedback and adjustment demonstrated both difficulties and opportunities in balancing outputs and outcomes when designing co-creation methods and tools.

Responding to the goal of participant-centric engagement design, three lessons can be learnt from the process. First, breaking down complex design operations through step-by-step gameplay made design knowledge more accessible, which empowered participants to self-explore. Second, offering a lucid learning experience to bolster participants’ confidence and capacity contributes to the establishment of a shared knowledge foundation for consensus-building. Third, promoting engagement and rapport through play can assist participants in expressing themselves more freely, developing an affective and profound comprehension of the design problem and the community.

Additionally, findings have shown that co-creation workshops, which involve a diverse composition of stakeholder and age groups, can impact both the outcomes and outputs positively:

• • Involving a higher proportion of older participants can lead to more practical, realistic and localised designs with enhanced participant experiences. Concurrently,

• • younger participant groups tend to generate more outgoing and active designs, unleashing potentials beyond more pragmatic considerations.

Thus, intergenerational engagement design can augment co-creation processes as a community-building and placemaking activity. Workshops that are engaging for participants of all ages can encourage the exchange of perspectives. Although accommodating their diverse needs can be a challenge, understanding the strength and weakness of different age groups and designing activities that encourage them to help one another—rather than passively serving each other—can be effective‥

By implementing these strategies, co-creation projects can achieve a more balanced and inclusive approach to bolster the community engagement experience for participants.

7.2. The role of designers

The greatest challenge in data collection and analysis was the constant changes in each phase of the project, including new input of context, agenda and stakeholders as knowledge and experiences accumulate. These changes, such as new consensus with partners, available resources, unforeseen circumstances, software and game version updates, all contributed to complicating the analysis process.

While comparing apples and pears can be difficult, they can be measured according to vitamins and other sub-constituents. Thus, a holistic framework can be useful to comprehend all variables considered:

• • Establish the framework early on to design and evaluate co-creation workshops, informing all members of the organising team to ensure implementation is aligned.

• • The framework should include multiple dimensions, each denoting a core pillar of the engagement design (e.g., learning outcomes, participatory experience, etc.); it helps to assess and reveal patterns macroscopically.

• • Design multiple levels within each dimension (e.g., aspiration, reflection, creativity, etc.) to generate more in-depth insights through descriptive-comparative analysis without manipulating the independent variable, offering opportunities to micro targeting and fine-tuning the engagement design.

The framework structures the data and the analyst’s worldview, similar to how a grid on a map structures the urban fabric. To address data voids, the descriptive focus can be on changes in data trends rather than specific data points, similar to how contour lines inform the lack of resolution on a map. This raises fundamental questions on the role of the designer, not only as a coordinator and facilitator during cooperative activities but also as an analyst post-activity to reformat the data and subsequently reformat the clients and future processes.

7.3. Potential limitations and next steps

The lowest performance was observed in levels of game ‘mechanics’, learning of ‘context’ and design ‘consensus’. Most participants expressed a neutral stance towards the need for compromise in reaching a solution within the team. This suggests that participants were either indifferent towards compromises or that the process design failed to identify conflict boundaries and the need to consent. This might also be influenced by the experimental setup. The standalone workshop format referenced existing local initiatives. However, such practices may lead to a lack of realism for participants. This should be tested by involving multiple sessions to facilitate a thorough collective learning process in the coming developments.

Additionally, there were limitations in the convenient sampling method that caused instability in participant size. Firstly, due to the participatory nature of the study, a limited number of participants were involved in each workshop. Secondly, it was challenging to recruit participants aged 26–59. Finally, not all stakeholders were involved due to the complexity of workshop organisation, such as estate management and governmental officers.

In terms of evaluation, the research is limited by its exclusive focus on analysing questionnaires and co-creation outputs. Considering the observable behaviour of participants on multiple game iterations, behavioural studies and mapping should be included in the next phases of the study so as to understand how articulating specific mechanics can impact game dynamics and user experiences.

It is important to consider these limitations when interpreting the findings of the study, especially in how these may impact the comprehensiveness of the insights gathered. However, the findings were largely aligned with participants’ post-workshop discussions. For instance, participants emphasised the importance of team spirit, taking time to consider their own needs and finding ways to express those needs precisely and collectively. These were in line with dimensions of ‘participatory experience’ receiving the highest ratings. Additionally, older adults expressed interest to further engage in this socio-technological experience, despite not fully mastering the digital techniques. They hoped for more youth volunteers to interact and share computer skills with them in the future. However, a comprehensive qualitative analysis is needed to validate the alignments and discrepancies.

Moving forward, the next steps involve studying the co-creation process as a form of public service, specifically in intergenerational programming. The focus will be on generating qualitative insights using participatory and relational theories to examine engagement levels, social participation types and task-role distribution patterns in a cooperative process.

8. Conclusions

This study explored the relationship between participants’ experience and design outputs in the context of urban co-creation. The framework and findings present an opportunity for novel participant-oriented analysis, particularly in three aspects. First, digital transformation involving interaction between members of different backgrounds and generations. Second, assessing constantly evolving co-creation workshops according to new input between different project phases. Third, deploying digital sandbox games in public participation.

Digital transformation is a common challenge in public programming, especially in community services across intergenerational contexts involving youth, adults and older adults. The study found that involving all three generations improved the overall experience, especially in learning outcomes. Through intergenerational cooperation in learning and designing, participants felt more confident in the basics of the subject and required techniques, demonstrating higher success expectancy in learning. This approach could be beneficial for similar urban renewal initiatives in other locations, which address the needs of ageing residents by promoting empathy and a detailed understanding of the related design challenges and potential solutions.

The research also showed a method to evaluate co-creation and community engagement workshops as the project evolved, considering different stakeholder dynamics. The use of frameworks with multiple dimensions, each encompassing sub-levels, demonstrates the interconnectedness of different workshop process characteristics. For instance, the dimension of participatory experience was enhanced when more stakeholder groups were involved, which also improved dimensions of co-creation results in meeting daily user needs. However, the sub-level evaluation showed that additional measures should be designed to unleash participants’ imaginative potentials beyond pragmatic considerations. Through such comparative analysis, the framework structure provides a means for organisations to assess initiatives and contractors, especially those focusing on emerging technologies.

Finally, incorporating digital sandbox games can help stimulate and simulate multi-stakeholder behaviours and examine potential managerial decisions. The prospect of fully utilising these metaverse tools to enhance the sense of human-scale and immersive design techniques is immense but requires significant research to balance between user autonomy and system design constraints.

The participatory approach that this study aimed to demonstrate considers citizens not merely as passive data subjects but active contributors in the urban design process. In an increasingly technical and technological design landscape, participants’ experience is a critical arena in evaluating the success of human-centric design and co-creation schemes, balancing the focus on cooperation as a collective learning process and an output-driven project.