Research into practice: Digital multimodal composition in second language writing

Abstract

Digital multimodal composing (DMC) has been valued as an engaging pedagogy in language teaching and learning in recent decades. Although research on DMC is flourishing and evidences its benefits for students' development as second language (L2) users and writers, there are some missing links between research findings and classroom practices. In this article, we examine three kinds of relationships between research and practice with regard to DMC: areas in which research findings have not been well applied, areas in which research findings have been reasonably well applied, and areas in which research findings have been usefully applied. As recent research–practice frameworks in education research emphasize a collaborative relationship between researchers and practitioners, we argue that L2 writing researchers' and teacher educators' reflections and experiences are crucial to facilitate the dialogue between DMC research and practice in writing contexts. We suggest that DMC should be incorporated into L2 teacher education programs so that instructors are equipped with the necessary knowledge and competence to design, implement, and assess students' DMC productions.

1. Introduction

Multimodality, described as a “concomitant of human communication” (Lotherington & Ronda, 2012, p. 107), addresses the complex nature of human communication through the dynamic interaction of various modes of expression. These modes include textual, visual, audio, gestural, and spatial elements. With the advancement of digital technologies and resources in the twenty-first century, multimodality has become the norm in contemporary communication and has attracted significant attention from language educators and practitioners (Bateman, 2014; Belcher, 2017; Kress, 2010; Zhang & Yu, 2024a, 2024b). The prevalence of digital multimodal communication is growing in both everyday life and educational settings, and second language (L2) learners and writers are increasingly engaging with diverse digital multimodal texts such as video essays, infographics, and digital posters (Yi et al., 2020). Against this backdrop, digital multimodal composing (DMC) has been increasingly adopted and researched across various L2 writing instructional contexts (e.g., Hafner & Ho, 2020; Jiang, 2017; Kessler, 2024; Park, 2021).

DMC is a technology-mediated task in L2 writing that employs digital tools to create texts by integrating various semiotic modes, such as linguistic, auditory, visual, gestural, and spatial modes (New London Group, 1996; Smith et al., 2021). Unlike traditional writing activities, DMC tasks can manifest in diverse forms, including graphic novels, academic posters, video essays, illustrated books, pamphlets, PowerPoint presentations, video documentaries, and web pages (Hafner & Ho, 2020; Li & Pham, 2022; Zhang & Yu, 2023a). Empirical research has demonstrated that DMC has the potential to offer students a range of benefits, which include heightened genre and audience awareness, enhanced semiotic awareness, improved critical thinking skills, facilitated identity construction, and more nuanced emotional expression (Cimasko & Shin, 2017; Hafner, 2014; Hafner & Miller, 2011; Kim & Li, 2021; Kim & Vorobel, 2017; Yi & Hirvela, 2010).

However, scholars have indicated that teachers use research findings to a limited extent in education, including L2 education (Coburn & Penuel, 2016; Farley-Ripple et al., 2018; Joyce & Cartwright, 2020; Sato & Loewen, 2022). While a plethora of research has been conducted to enhance the DMC instructional designs and the affordances of DMC for L2 students, little is known about the extent to which, and how, these research findings have been transferred into classroom practice to improve the teaching and learning of L2 writing. Many L2 teachers still struggle to implement DMC in writing classrooms, and they find it challenging to incorporate DMC into traditional writing classes featuring exam culture and a teacher-centered teaching approach. As recent research–practice frameworks in education research emphasize a collaborative relationship between researchers and practitioners (Cornelissen et al., 2017; McIsaac & Riley, 2020; Sato & Loewen, 2022), L2 writing researchers' and teacher educators' reflections and experiences are crucial to bridge the gap between DMC research and practice in writing contexts.

This article aims to examine the relationship between research and practice to determine the extent to which DMC research findings are taken up by L2 writing instructors. Specifically, the purpose of this article is to discuss what current DMC research has succeeded in getting through to classroom teaching, what has not succeeded, and why. We will examine three kinds of relationships between research and practice with regard to DMC: areas in which research findings have not been well applied, areas in which research findings are reasonably well applied, and areas in which research findings are usefully applied. In doing so, we will primarily draw upon the literature on DMC from language education, applied linguistics, and L2 writing perspectives. To better engage with the literature and interpret the use of research in classroom practice, we will also draw upon our experiences as teacher educators and writing teachers in a Chinese context, where we work with English as a Foreign Language (EFL) teachers in both school and university contexts.

2. Research findings that have not been well applied

2.1 Integration of DMC into L2 instruction and exams

The burgeoning research has gauged the effect of L2 DMC projects on students' monomodal L2 writing performance. Studies have reported that students who participated in L2 DMC projects performed better than those who wrote traditional essays in terms of grammatical accuracy and complexity, lexical diversity, text length, communicative effectiveness, content, task requirement, and comprehensibility (Kim et al., 2022; Vandommele et al., 2017; Yang & Wu, 2012). Researchers have also compared L2 multimodal and monomodal writing and noted that although they shared common cognitive processes of consulting external sources and initiating revisions, DMC could elicit more autonomous writing (Tan, 2023b). L2 students have acknowledged that multimodal composing is a useful tool for learning English and the analysis of the discussion episodes indicates that they gave substantial attention to language features, such as grammar, vocabulary, and mechanics along with other semiotic modes (Kim & Kang, 2020). Furthermore, DMC could promote students' revision of writing and writing quality (Dzekoe, 2017), and project both disciplinary and individual identities (Tardy, 2005) via the use of both verbal and visual modes.

A substantial body of research has delved into L2 teachers' implementation of DMC projects in class, and the findings reveal that L2 teachers positively perceived the DMC projects, particularly in terms of engaging and motivating the L2 students in language learning (Jiang, 2017; Li, 2020; Rance-Roney, 2010; Ryan et al., 2010; Yi & Choi, 2015), and fostering the students' audience awareness (Hafner, 2014; Zhang & Yu, 2023b), creativity (Jiang & Luk, 2016; Zhang, 2023; Zhang & Yu, 2023a), and confidence (Hava, 2021). For instance, Hafner (2014) reported on a study about an L2 teacher's implementation of DMC projects in an English for Science course in a Hong Kong setting. The L2 students were tasked with designing multimodal scientific documentaries to explain scientific experiments. It was found that DMC supported the teaching objective of involving L2 students in scientific English communication between specialist and nonspecialist audiences. Jiang (2017) focused on L2 teachers' implementation of DMC projects, reporting that the teachers viewed DMC as a useful tool to arouse L2 students' interest in English learning and nurture their creativity in expressing ideas. In addition, Jiang and Luk (2016) probed into L2 teachers' implementation of DMC projects in an English course at a mainland Chinese university, revealing that the L2 teachers perceived that DMC was engaging, as the students could design multimodal ensembles for social and personal purposes. The use of digital media and multimodal resources was seen as making English learning more enjoyable and motivating. Zhang and Yu (2023a) scrutinized an L2 teacher's implementation of DMC projects in a college English course in mainland China. The findings showed that DMC aided L2 students in expressing creative ideas, reducing writing anxiety, and forming the habit of text revision.

Although research on L2 teachers' implementation of DMC projects in class bears important implications for DMC practices (e.g., Hafner, 2014; Jiang & Luk, 2016; Zhang & Yu, 2023a, 2023b), these research findings have not been well applied in L2 instruction and exams. Many teachers in certain L2 contexts have voiced skepticism, concerned that multimodal discourse might divert students' attention away from target language acquisition or academic writing development (Qu, 2017). For instance, in a recent study conducted by Zhang and Yu (2024b), the L2 university students in mainland China mistakenly perceived the DMC task as a video-making assignment and spent substantial time and energy designing fancy videos instead of polishing their L2 writing, which might have distracted them from developing their L2 writing skills. In a Malaysian context, L2 teachers revealed that low-proficiency students' digital stories contained inadequate language and narration, and was not effective in boosting their language acquisition (Thang et al., 2014). In a setting in the United States, L2 teachers worried that DMC might wean the students off academic language and literacy acquisition if the students were accustomed to using multimodality to read and write rather than using their L2 knowledge (Yi & Choi, 2015).

In addition, textbook- and exam-oriented L2 curricula position DMC as peripheral to instruction, thus hindering the implementation of DMC in L2 instruction and assessment. For instance, in a Chinese mainland context, while the L2 teachers in Jiang and Ren's (2020) study encouraged students to design DMC projects on curricular topics and to practice language exercises from the textbook, the students preferred to express their ideas freely without being constrained by the curriculum and the textbook. In another study conducted in mainland China, the L2 teachers articulated that there is a genre gap between DMC and writing tasks in exams, and DMC falls short of boosting the students' scores in exams (Zhang et al., 2024). In addition, the teacher in Zhang and Yu (2024b) revealed that while she encouraged her students to use the words and phrases in the textbook, the students rarely did so, because they preferred to make their language comprehensible rather than complex, tailoring it to the language proficiency of their classmates as the audience. This made the L2 teacher feel powerless in bridging the gap between DMC and the curriculum. In an elementary school in Australia (Mills & Exley, 2014), the teachers perceived the genres and text types of DMC and the evaluation of video production and visual design in DMC as being not aligned with the English curriculum, so they expected their students to work on the English skills by merely embedding some digital elements into their writing, and revised the evaluation criteria to emphasize linguistic and textual features over the use of new media and modes. While multiliteracies and DMC practices are covered in the 2020 English Language Syllabus in Singapore (Ministry of Education, Singapore, 2018), there is still a gulf between the curriculum and classroom practices due to the absence of DMC in national examinations (Heydon, 2013; Tan et al., 2010). In a K–12 class in the United States and a university class in mainland China, the L2 teachers in Yi and Choi (2015) and Zhang et al. (2024) noted that the rigid language curriculum and standardized testing scheme posed time constraints for DMC, as they were under great pressure to help students pass the language tests.

While L2 teachers are encouraged to implement innovative technology-mediated pedagogy in their classes, the majority of them lack the training of abilities and skills to implement DMC, thereby probably preventing them from effectively embedding DMC into the curriculum. For example, in mainland China, the teacher in Zhang and Yu's (2024b) autoethnographic study reported that she felt uneasy about her lack of proficiency in providing technological assistance to the students. Likewise, in a Malaysian English a Second Language (ESL) context (Thang et al., 2014), the teachers described launching DMC projects as a chaotic experience due to their lack of preparation and unfamiliarity with leveraging technology to design DMC texts themselves. In a Hong Kong setting, the L2 teachers in Hafner and Ho's (2020) study revealed that they struggled to assess students' DMC projects while simultaneously attending to different types of information, such as sound, fluency, accuracy, and so forth. In an American ESL context, Yi and Angay-Crowder's (2016) study found that teachers showed resistance to DMC partly due to their insufficient technological skills and the content knowledge needed to implement it. DePalma and Alexander (2018) reported that L2 teachers in the United States lack the training and expertise to create audio and video composition in their teacher training programs, thus not excelling at implementing DMC in their classes. In a Singapore L2 context, the L2 teachers in Lim et al. (2021) pronounced the need for stronger support in instructional approaches and resources for the teaching of multimodal literacy in class.

Therefore, there is a pressing need to build a closer connection between research and teaching that enables L2 teachers to implement DMC in their classes to materialize its potential benefits for L2 students' writing and overall L2 learning in the elementary, secondary, and tertiary contexts. L2 teachers need to be cognizant of the benefits of DMC for L2 learning and writing acquisition, and attempt DMC projects in their classes. Ryu and Boggs (2016) reiterated the importance of ongoing communication and discussion among different stakeholders in education, such as teachers, students, parents, and administrators, to highlight the relevance and benefits of DMC and drive L2 teachers to explore teaching DMC in class. Schools in L2 contexts need to broaden their L2 curriculum to include DMC as a writing task, given that multimodal compositions reflect authentic everyday writing practices and are conducive to cultivating L2 students' new literacies demanded by the digital era. For instance, curriculum designers in Australia and Singapore are embedding multimodal literacy into the English curriculum (ACARA, 2014; Ministry of Education in Singapore, 2018), which set up good examples for curriculum designers in other L2 contexts to follow.

Apart from that, L2 test designers are being called upon to include multimodality in tests to make the task more authentic and to better support L2 students' multimodal literacies (Yi & Choi, 2015). Lim et al. (2021) in Singapore argued for the need for assessment reforms to align with the curriculum that embeds multimodal literacy. Though it is difficult for test takers to orchestrate multimodality in performance tests, such as writing and speaking, it is possible to utilize multimodal materials as input in the integrated writing/speaking tasks or the individual listening and reading tasks (Yu & Clark, 2023). With respect to L2 teachers, they can also agentively embed multimodality into the assessment tasks. For example, the Korean L2 teachers in Ryu and Boggs (2016) provided multimodal input to the students to motivate them to write more.

L2 teacher training programs are recommended to integrate DMC to equip teachers with the skills needed to create DMC artifacts themselves and design DMC lessons to guide their teaching of DMC. To teach students how to create DMC projects, DePalma and Alexander (2018) introduced a pedagogical approach known as distributed collaboration (DC), designed to foster collaboration among specialists with diverse expertise, such as new media professionals, software experts, community members, local professionals, and student teams. This approach has the potential to provide the necessary support to L2 teachers to launch DMC projects in class. It is also essential to note that L2 teachers need to make bottom-up efforts to negotiate the tension between DMC and the print-based writing curriculum and testing system. Zuo and He's study (2024) sets up a good example: the instructor in the Chinese mainland context agentively used DMC to negotiate these tensions by utilizing DMC to manage these challenges by structuring the project akin to the test task and rating criteria in the standardized tests, and guiding the students to prepare for the tests while stirring their passion for digital tools.

2.2 DMC assessment rubrics and approaches

There is a growing body of research addressing what to assess in DMC projects. Some researchers have taken a product-oriented approach to DMC assessment by using the rubrics of DMC to assess the DMC end-products. Drawing on key concepts from multimodality and genre theories, such as the New Rhetoric/Rhetorical Genre Studies approach (e.g., Bateman, 2008; Miller, 1984), Jiang et al. (2022) designed a rubric for DMC projects, which contains base, layout, navigation , and rhetoric. Zhang and Yu (2023b) conceptualized L2 DMC competence as the utilization of multiple modes, genre awareness (with a subdimension of audience awareness), digital skills, creativity, delivery, cohesion of modes, identity expression, language use (with a subdimension of linguistic choices), and organization.

Other researchers have taken a process-oriented approach to DMC assessment. Hafner and Ho (2020) investigated a process-based model of DMC assessment in an English for Science course in Hong Kong. Hafner and Ho (2020) suggested assessing L2 DMC projects at four main stages: (1) pre-design, (2) design, (3) sharing, and (4) reflecting. Teachers, peers, and/or students themselves can review and assess (1) mind maps, outlines, and notes during the pre-design phase, (2) students' draft scripts, storyboards, video clips, and other semiotic modes at the design stage, (3) DMC artifacts at the sharing stage, and (4) conversations with students, students' presentations, and reports at the reflection stage.

While DMC assessment has garnered increasing attention in research, the lingering questions of what aspects to assess and how to evaluate DMC performance still remains in L2 classrooms (Jiang et al., 2022; Kessler, 2024; Liang & Lim, 2021; Zhang et al., 2024), given the fundamental difference between traditional print-based monomodal L2 writing and multimodal composing (Hafner & Ho, 2020; Sorapure, 2006). For instance, in a Chinese mainland context, the L2 teachers in Zhang et al. (2024) reported an imminent challenge they faced in assessing the students' DMC projects due to the confusion about the inapplicability of the universal evaluation criteria to specific DMC tasks and the weighting of different dimensions of the evaluation criteria. L2 teachers in a mainland Chinese setting, as reported in Jiang et al. (2022), were concerned that the rubric is too comprehensive and that writing is only one of the multiple modes in the assessment criteria, which might undermine the importance of writing as legitimate learning evidence of DMC projects. In a Hong Kong setting, L2 teachers in Hafner and Ho (2020) observed that the DMC rubric lacks concrete examples to differentiate outstanding samples from good ones, and misses pace and delivery, as it is not sufficiently detailed. In an American L2 setting in Yi and Choi (2015), L2 teachers confessed to significant challenges in assessing students' multimodality in DMC, especially when compared to the linguistic mode that is favored by standardized tests.

In addition, it is challenging to apply the assessment rubrics in the literature to a wide range of DMC genres. For instance, Zhang and Yu (2023b) recognized that more genres need to be analyzed to revise and refine their L2 student DMC competence model to meet the needs of different DMC projects. Further, in process-oriented DMC assessments, the existing models might not cater to large classroom sizes (Kessler, 2024), which is generally a norm in L2 English learning contexts and thus makes L2 teachers shy away from the assessment of the DMC production process.

To better help teachers appreciate the scholarship on DMC assessment, it is suggested that they assess their students' DMC performance in accordance with their purposes of implementing DMC projects, as well as the overall institutional and curriculum goals, to gauge whether the students master the knowledge and indicators of learning. It is unnecessary to cover every DMC assessment criterion, and they can selectively focus on the dimensions that are relevant to the courses. For instance, in a setting in the United States, based on her DMC assessment experience, Sorapure (2006) advised L2 teachers to accommodate the differences between DMC genres by focusing on how effectively the DMC project addresses specific audiences to achieve specific purposes. In a mainland Chinese classroom, L2 teachers in Jiang et al. (2022) noted that there was a felt need to include a clear purpose (e.g., to argue, to inform, etc.) in assessing the DMC project, without which the DMC projects might be muddled. In a Chinese mainland context, based on the purpose of helping students to prepare for the argumentative essay writing assignments in standardized language tests, the instructor in Zuo and He (2024) adopted the rubric of the argumentative writing task in Test for English Majors (TEM) 4 and 8, which includes content, complete in ideas, well organized in structures, correct in grammar, and appropriate in language use. In a Korean as a Foreign Language setting, considering that the students' Korean proficiency was low and memes are featured in textual brevity, Ryu et al. (2022) incorporated memes as a DMC project in the class. They designed their own rubric for the specific DMC context, assessing multimodality, cultural aspects, linguistic components, and task/functions of memes as wit and humor. In an American L2 context, Li (2020) constructed the DMC rubric, including content, technology, graphic design, language and mechanics, and creativity, which was incorporated in the DMC projects for student teachers to demonstrate the linguistic knowledge they had gained and the effect of such knowledge on their own teaching. The DMC rubric also helped to design instructional resources based on the language teaching principles and methods the students learned.

Moreover, L2 teachers can draw on the product- and process-based approaches in DMC assessment to more holistically gauge the students' learning evidence derived from DMC projects. In the Hong Kong context, Cheung (2023) serves as a good model for L2 teachers to follow by proposing a set of product- and process-oriented classroom DMC assessment rubrics for L2 students. As regards the product dimension, she adapted Jiang et al.'s (2022) genre-based model to assess students' DMC academic blogs. Base was concerned with the accuracy and diversity of lexical choices, the clarity and creativity of the visual modes, and the relevance of the base units. Layout centered on the hierarchical structures and typographical/visual configurations of the base units. Navigation focused on the extent to which students put forward their arguments and engage readers with multimodal resources. Rhetoric represented the extent to which students can convey meanings with effective rhetorical structures of multimodal elements. In light of the process dimension, Cheung (2023) assessed equality, that is, the extent of students' engagement in collective discussions, decision-making, and equitable contributions, and mutuality, that is, the degree of students' responsiveness to each other's ideas, management of task-related problems, and encouragement for each other in L2 DMC production. The process-oriented assessment criteria captured the social and collaborative nature of DMC. The formative feedback from the teacher, peers, and themselves, which they obtained in terms of the DMC products during the DMC project, added to their understanding of the DMC genre, thus improving their product-oriented performance. In a Chinese mainland context, the instructor in Zhang and Yu (2024a) utilized the DMC assessment rubric (including digital skills, genre awareness, language use, and cohesion of modes) for self- and peer-evaluation of the DMC products, which raised the L2 students' metacognitive awareness of DMC competence, improving the quality of their DMC works, and transferring common criteria to monomodal L2 writing.

2.3 Theories of DMC and DMC pedagogy

A myriad of theoretical orientations have been leveraged to inform DMC inquiries. The systematic functional linguistics (SFL) theory posits that language is regarded as a social semiotic system in meaning making (Kress, 2010), which is frequently consulted in DMC studies as the fundamental theoretical framework for multimodality (e.g., Shin et al., 2021; Smith et al., 2017; Oskoz & Elola, 2016; Yang, 2012). Sociocultural theory asserts that human cognitive development is inherently a mediated process through individuals' leveraging of artifacts in a collective scaffolding way (Lantolf et al., 2020; Vygotsky, 1978), which has been frequently drawn on to investigate how the DMC production is mediated and scaffolded through students' interaction with teachers, peers, and tools (Honeyford, 2014; Tardy, 2005; Yi & Hirvela, 2010). Multiliteracies theory holds that human communications are multimodal in nature (New London Group, 1996). DMC research has been largely informed by multiliteracies theory to examine design activity based on multimodal meanings (Hafner & Ho, 2020; Jiang & Luk, 2016; Kim & Belcher, 2020; Lee et al., 2019; Yang et al., 2020; Yeh, 2018). Multiliteracies theory involves a pedagogy theory of literacy, including situated practice, overt instruction, critical framing, and transformed practice (New London Group, 1996), which can be used to guide the DMC pedagogy (Angay-Crowder et al., 2013; Chandler-Olcott & Mahar, 2014).

While many relevant theories have been utilized to underpin DMC-oriented studies, there is a disconnect between the theories and the DMC practice: L2 teachers find it daunting to apply these theories in their DMC implementation (Kessler, 2024). One reason is that most of the theories are usually derived from applied linguistics and L2 acquisition, which are slightly general and abstract and not fully-fledged for accessible classroom applications to explore and account for all features of DMC. The present theoretical underpinnings of DMC are seldom informed by observations on DMC practices. Thus, there is a pressing need for L2 teachers and researchers to refine the current DMC-related theories or propose new understandings based on the classroom DMC practices. For instance, in Hong Kong, the L2 teachers in Hafner and Ho (2020) suggested adding pace and delivery, and separating organization and content to refine the assessment theory. Likewise, in the mainland China context in Jiang et al. (2022), the L2 teachers' practice-based perceptions contributed to a refined genre-based model of DMC. For instance, the L2 teachers advised adding purpose as the core with which the base, layout, navigation, and rhetoric layers can be associated and interconnected. In the layout layer, the teachers added video/image, transition, audio/voiceover, soundtrack/music, and written captions to summarize the common moves in students' DMC projects.

Besides, the existing theories pertinent to DMC mainly aim to understand its processes and outcomes; they seldom address how to implement DMC as a pedagogy. Although multiliteracies pedagogy is proposed (New London Group, 1996), it might fall short of providing hands-on guidance for L2 teachers to launch DMC. With the knowledge of relevant theories, L2 teachers would have a legitimate means to justify and explicate their DMC-related pedagogical tasks, activities, and exercises (Kessler, 2024). L2 researchers and teachers are advised to zoom in on DMC and DMC pedagogy based on the teaching practices of DMC. For instance, Liang and Lim (2021) set up a good example for L2 teachers in other contexts to follow. Informed by SFL and Design Thinking, Liang and Lim (2021) put forward a pedagogical framework of DMC, including the critical domain, that is, the teaching and learning of metalanguage describing key features of multimodal compositions; the creative domain, that is, encouraging students to use design thinking in DMC projects; and the technical domain, that is, supporting students with the technological-related knowledge and skills for DMC. The L2 teachers in Liang and Lim (2021) adopted the framework to implement the DMC project and suggested adjusting the time for the creative domain to allow for more time on content generation, which helps tailor the pedagogical framework to the classroom practice. Additionally, based on the teaching practice, Huang and Xia (2024) proposed a DMC-integrated EAP pedagogy framework to inspire L2 teachers to integrate DMC into English for Academic Purposes (EAP) courses, and the framework comprises model configuration, visual enticement, modal discernment, knowledge customization, inter-modal alignment, and intra-modal concordance. Moreover, teacher educators need to highlight the intertwined connection between theory, research, and teaching practice, and encourage L2 teachers to learn relevant theories such as multiliteracies and multimodality to guide their DMC implementation in classes and DMC research.

L2 teachers also need to familiarize themselves with L2 acquisition and teaching theories to guide their instruction practices, which is specified as research literacy for teachers in the English Syllabus in mainland China (2020). For example, in a Chinese mainland context, the teacher in Zhang and Yu (2023a) was a researcher in DMC and, informed by the multiliteracies and multimodality theories, she managed to bridge the gap between DMC research and practice by aligning the theories with the classroom reality and implementing DMC. The study verified that DMC, indeed, could improve L2 students' monomodal writing performance. Teachers are also encouraged to exercise their agency in adopting or adapting L2 acquisition theories when they explore the DMC projects in their teaching practices, such as interactionist approaches (e.g., Gass, 1997; Long, 1996), noticing hypothesis (Schmidt, 1990), and cognitive process of writing theory (Flower & Hayes, 1981).

3. Research findings that have been reasonably well applied

3.1 DMC and collaborative learning

Research on DMC has witnessed an increasing trend of situating it within collaborative learning contexts (e.g., Kim & Kang, 2020; Kim et al., 2022; Vandommele et al., 2017). It has been revealed that collaborative DMC is embedded with learning opportunities beyond multimodality by involving social interaction, knowledge negotiation, and peer scaffolding (Kim et al., 2022). This strand of research on collaborative DMC has identified students' learning gains across linguistic, social, and psychological domains. Tanrıkulu (2022) found that group-based digital storytelling could divert students' attention to the structure of writing and reduce writing errors. Besides, group-based DMC is beneficial regardless of the learning context. Vandommele et al. (2017) found that both in-school and out-of-school collaborative DMC could enhance L2 Dutch students' writing proficiency concerning syntactic complexity, lexical diversity, and text length.

The impact of collaborative DMC also moves beyond the textual level by engaging students in critical reflection of their lived experiences. For example, Cummins et al. (2015) argued that collaborative multimodal identity texts could provide a supportive space for English language learners to revisit their experiences, patterns of interactions, and emotions, which in turn facilitated collaborative power creation. Another strand of research compared the learning effectiveness of individual DMC activities and collaborative ones. Based on their study exploring Vietnamese students' production of infographics, Li and Pham (2022) found that group-based DMC products received higher scores than individual ones in organization/content, visual effects, and language, although the difference was not significantly obvious.

In collaborative DMC activities, one important dimension of learning is peer interaction. Patterns of interaction have been identified by researchers (e.g., Akoto & Li, 2023; Cho, 2017). For example, in their context of L2 French collaborative DMC writing, Akoto and Li (2023) employed the proxies of mutuality and equality, and identified three patterns of peer interaction: collectively contributing/mutually supportive, dominant/withdrawn/cooperative, and active/passive. They also argued that the dynamics within the group could influence the quality of the DMC production, that is, higher levels of collaboration and mutual support leading to higher scores for the writing products. The peer interaction in collaborative DMC is further reflected in the members' power negotiations regarding the integration and deployment of semiotic modes in their products (Cheung, 2022). Factors shaping group dynamics include individual goals, medium of communication, task interpretation, perceptions of each other's roles, and peer feedback (Cho, 2017). Research along this line highlights that DMC serves not only as a pedagogy for improving writing skills but also as a space for nurturing students into skilled writers and effective collaborators.

With its dual role in facilitating language acquisition and social interaction, collaborative learning becomes a popular way of integrating DMC into teaching practices. In many EFL contexts, writing teachers tend to use group-based DMC activities instead of individual assignments due to their belief in the value of social collaboration, or for pragmatic considerations; for example, using group work to alleviate the workload of DMC projects among students. Especially given the large class size in most L2 contexts, group DMC projects, from the teachers' side, can reduce their teaching and assessment workload. Collaborative DMC also enables them to highlight the potential benefits of enhancing students' capacities as global citizens in the twenty-first century, where a sense of shared responsibility and interconnectedness is required (Andreotti, 2010). For these reasons, the current scholarly emphasis on collaborative DMC is widely and reasonably reflected in the classroom and upheld by L2 teachers. However, it should be noted that collaboration might add additional dimensions of challenge, as most teachers are confused about whether to assess DMC production as individual or group products.

3.2 The use of technology in DMC tasks

The conception of DMC is largely developed against the backdrop of multiliteracies (New London Group, 1996) and multimodality (Kress, 2010), which move beyond linguistic codes by directing scholarly and pedagogical attention to meaning itself and the way it can be expressed in the digital era. What and how technologies can be integrated into and mobilized in meaning-making have been extensively studied. Jiang (2017) argued for the technological affordances of DMC concerning the action possibilities that emerged in the interaction between students and the technological environment. He found two major affordances of DMC technologies: (1) digital recording that enabled self-paced revision and broad-reaching exhibition; and (2) transmediation across modes where students engaged in meaning-making with multiple semiotic resources in a personally relevant way. Aside from their learning potential, DMC technologies also present difficulties. For students, it could be challenging to incorporate various semiotic modes like videos, voices, and pictures into their writing products (Jiang & Luk, 2016). They might be unfamiliar with the software design options, which limits their access to advanced functionalities in creating DMC products (Zhang & O'Halloran, 2019). Some students might have a negative view of technology because, rather than simply learning writing, they have to learn it in a way that is complicated by technology (Hava, 2021). Teachers can face similar technology-related issues in designing and implementing DMC pedagogies. In a researcher–teacher collaborative action research project, Jiang et al. (2022) found that teachers would experience the anxiety of being digital novices and that they would seek help from students who served as technology assistants. Likewise, Thang et al. (2014) found that some ESL teachers might be resistant and negative towards the integration of technologies in students' storytelling. Kessler and Marino (2023) warned that teachers should be cautious of the copyright issues in students' DMC products and guide students on how to reference online materials.

In their review of DMC research in post-secondary contexts, Zhang et al. (2021) identified five categories of technologies: video creating/editing (e.g., iMovie), digital storytelling (e.g., Storybird), video sharing (e.g., YouTube), digital composing (e.g., Google Docs), and social networking (e.g., Facebook). Besides, they found that much technological emphasis was placed on video production, with only 17% of their reviewed studies discussing digital writing tools. Kessler and Marino (2023) discussed three commonly used DMC activities—that is, posters/brochures, storyboards, and digital video production—each having a range of specific technological platforms to be leveraged. For example, to create a poster, students can use Canva (canva.com), which provides free access to graphic design. Some studies are dedicated to examining the effectiveness of specific technological tools in student learning, such as Visme (Pham & Li, 2023).

While technologies pose challenges for teachers to introduce and implement DMC, their benefits are generally acknowledged. Especially in the current situation where technology-facilitated learning is the norm in most Chinese schools (for a review, see Wang et al., 2018), teachers, despite these possible challenges, are encouraged to incorporate e-learning into their teaching practices. However, even though multimodal technologies are reasonably used in L2 writing classrooms, there is still a gap in teachers' ability to realize their full potential for learning, mostly due to teachers' lack of proficiency in these technologies. For example, while L2 teachers are active in launching DMC projects as technology-mediated L2 writing tasks in class, they struggle to render hands-on technological assistance to their students due to the lack of technological training in teacher training programs and their high workload (Zhang & Yu, 2024b). Teachers may feel anxious about being unable to offer technical assistance to students, but this also creates a space for teachers to co-learn with students about the digital tools (Zhang & Yu, 2024b).

4. Research findings that have been usefully applied in the classroom

4.1 Focus on ESL/EFL undergraduate students in language classes

Much of the scholarly attention on the implementation of DMC is devoted to tertiary contexts with university learners as the focal participants in ESL/EFL language classes, with a few situated in K–12 classrooms or post-graduate education (e.g., Rahimi & Yadollahi, 2017; Shin et al., 2021; Yang et al., 2020). The affordances of DMC on non-English-major undergraduate students have been largely well-researched. Compared with L2 teachers at the kindergarten, elementary, and secondary levels, university teachers in L2 contexts are more inclined to implement DMC projects in their language classes. Their purposes are to develop students' language proficiency, such as speaking and writing skills, as well as L2-related psychological aspects, such as motivation, learning autonomy, learning enjoyment, high-order thinking, and the like. University instructors also have more autonomy in designing their writing courses, thus creating a more favorable environment for scholarly activities and innovative pedagogies.

In many primary and secondary schools in the L2 context, teachers need to follow the institutional requirements strictly and thus lack the incentives to implement innovative pedagogy in their classes (Lee et al., 2018; Zhang et al., 2021). Educational policymakers and course designers still harbor the traditional view of literacy as monomodal reading and writing, rather than embracing the new literacy view of multiliteracies and multimodal reading and writing (Lim & Kessler, 2021; Tan & McWilliam, 2009). They may need to glean insights from the curricula in Australia and Singapore to keep abreast of the new literacy landscape and literacy instruction needs across levels of schooling (ACARA, 2014; Ministry of Education in Singapore, 2018). Only when DMC is part of the learning goals in the curriculum can teachers be empowered to launch it in their classrooms. For example, given that DMC is part of the curriculum, the L2 secondary teachers in Liang and Lim (2021) in Singapore implemented DMC in the instruction, and the DMC project supported the students' learning of expressing ideas with multimodal texts and stimulated the students' creativity. The L2 primary teachers in Mills and Exley (2014) implemented DMC in their classes to ask the students to design multimodal and digital texts, such as web pages, online comics, blogs, and so forth.

For post-graduate ESL/EFL students, English courses are often designed for academic writing purposes, which is largely monolingual (O'Halloran et al., 2016). However, multimodality also matters to academic writing. For example, in the American L2 setting, the teachers in Tan (2023a) assigned a multimodal academic writing project to the students, who were asked to generate a video about a sustainability issue and solutions. The L2 students utilized multimodal citations to make attributions, provide examples, and establish links between sources. In a mainland China context, the L2 teachers in Huang and Xia (2024) embedded DMC into the EAP course to task students with the knowledge dissemination video project. The students created interpersonal (e.g., engaging the audience), ideational (e.g., explaining the disciplinary knowledge), and textual meanings in the DMC project (e.g., intermodal and intramodal coherences).

Teachers of other L2s are also encouraged to apply DMC research findings in their classroom (e.g. Zhang et al., 2021). Few teachers who teach an L2 other than English have attempted to implement DMC projects in their classes. As such, it is beneficial for teachers to attend to DMC and integrate it into their language curricula and test the research findings with evidence from their own teaching practices. For instance, in an American Foreign Language context, the Spanish writing teacher in Oskoz and Elola (2016) tasked the Spanish majors with designing DMC works, which helped the students to write with creativity and coherence. In a Korean as a Foreign Language context, Ryu et al. (2022) implemented DMC, that is, memes, among the students, and found that the students leveraged multimodality in memes to demonstrate their linguistic and cultural knowledge.

Furthermore, there is a particular need for L2 teachers to apply research findings about DMC implementation in content courses or content-and-language integrated learning courses (Li, 2020), such as Teaching English to Speakers of Other Languages (TESOL) courses, applied linguistics, L2 acquisition, general linguistics, phonetics, and literature courses. For instance, in a Hong Kong L2 context, the teacher in Ho (2024) implemented DMC projects in a content-based course, that is, a university writing course, in which students were expected to demonstrate a solid understanding of abstract concepts using DMC. The project adds to the L2 students' understanding of writing as multimodal design and bridges the “in-class” and “out-of-class” digital writing practices.

4.2 Application of digital videos as a major DMC genre

A proliferation of research has documented digital videos or digital storytelling as a DMC genre (e.g., Cimasko & Shin, 2017; Jiang & Hafner, 2024; Kessler, 2024; Li & Akoto, 2021; Zhang et al., 2021), which is also reflected in the teaching of DMC. Most L2 teachers focus on digital videos as a DMC genre when implementing DMC projects in the L2 context. They tend to guide their students to follow a series of steps in the project: textbook reading, brainstorming, script writing, filming, collecting and creating multimodal resources, editing and producing videos, and sharing and distributing the videos (Zhang et al., 2021).

However, the over-reliance on the digital video genre might lead to unwanted consequences. For example, students might dedicate a lot of time and energy to designing elaborate videos (Jiang & Ren, 2020; Zhang & Yu, 2024b), which lead to DMC diverting L2 students from the foundational language interaction needed for successful L2 acquisition (Lim & Kessler, 2021; Manchón, 2017; Qu, 2017). In addition, L2 students might also find it useful to film, design, or edit videos, and insert captions and voiceovers, thereby demanding more systematic teaching of digital knowledge and skills (Deng et al., 2023; Jiang & Ren, 2020; Zhang & Yu, 2024b). Given that L2 teachers lack professional training in digital skills, they are apt to leave technical problems for students to tackle. As a consequence, it is likely that L2 students would spend a substantial amount of time learning such techniques. One student revealed that their group spent approximately six hours removing a watermark from a video clip (they provided the source of the video at the end of their work to avoid plagiarism) to make it more aesthetic (Zhang, 2023). Moreover, while L2 teachers tend to choose digital videos as the DMC genre, when it comes to the assessment, only a fraction of scores is assigned to the digital aspects, which might adversely affect students' motivation (Mills & Exley, 2014).

Furthermore, many L2 students share the misconception that videos should be easily comprehensible and entertaining for the audience, leading them to prefer simple words and sentence structures in order to be audience-friendly. This could be detrimental to the students' L2 proficiency development, because their reliance on simplified words and sentence structures in DMC may hinder them from experimenting with and acquiring more complex linguistic forms. As a matter of fact, DMC genres can be academic, such as scientific documentaries (Hafner, 2014), academic posters (Tardy, 2005), video proposals (Tan, 2023a), video abstracts, and multimodal PowerPoint projects (Liu et al., 2024; Zhang & O'Halloran, 2019), in which L2 students are tasked with academic language use. L2 teachers need to introduce more DMC genres to students and guide students in better understanding their target audiences and adjusting their language use accordingly. For postgraduate English classes, DMC pedagogy is also relevant. L2 teachers may want to incorporate scientific documentaries, academic posters, video abstracts, and multimodal PowerPoint projects in their classes to help students develop academic multimodal literacy (Tan, 2023a).

For adolescent learners such as primary and secondary school students in the L2 context, L2 teachers might wish to assign relatively simple DMC projects, such as posters and digital stories. For instance, in Skinner and Hagood (2008), digital stories were adopted by the L2 elementary teacher in an American setting, which engaged the students in illustrating their interests. In an American first language context, digital texts writing was used by the teacher in Chandler-Olcott and Mahar (2014) to help the adolescents construct their desirable identities.

4.3 DMC task design: Collaborative project with multimodal output

The existing DMC research has centered on the collaborative task design of multimodal output (Jiang & Hafner, 2024; Kim & Kang, 2020; Kim et al., 2022; Li, 2021; Vandommele et al., 2017; Yang et al., 2020). The research on collaborative DMC projects has informed DMC teaching practices, and the majority of the teachers in the L2 context chose to implement collaborative projects that involve two or more students utilizing digital tools and multiple semiotic modes to jointly design DMC artifacts. While collaborative DMC projects have the potential to foster L2 students' collaborative skills and digital literacy skills, which are useful for them to address real-world tasks (Jiang & Luk, 2016), there are drawbacks to collaborative DMC projects—such as the existence of free-riders who contribute minimally to the project (Kessler & Bikowski, 2010; Li, 2021), and conflicts with the individually-based standardized testing scheme (Mills & Exley, 2014; Yi et al., 2017).

In the L2 context, most of the teachers chose to implement collaborative DMC projects. They revealed that, while DMC projects are time-consuming, collaborative DMC projects reduce students' workload and provide them with ample opportunities to scaffold each other (Hafner, 2015; Kim & Kang, 2020; Zhang et al., 2024). Nevertheless, some L2 teachers reported that several low-proficiency students complained that when dividing the tasks, only those with high English speaking and writing proficiency would receive the jobs of narrating and writing scripts to earn high scores for the DMC project (Arnold et al., 2012). All the low-proficiency students could do was edit videos, such as inserting captions and voiceovers, which demands a lengthy period of time to complete. That is to say, low-proficiency students might lose the opportunities for language learning in collaborative DMC projects. L2 teachers can also refer to research on individual DMC projects (e.g., Li, 2020; Li & Pham, 2022; Liu et al., 2024; Tan, 2023a; Yang, 2012) to enable every student to reap the benefits from DMC projects. In addition, DMC projects can involve complex sub-tasks (Jiang & Hafner, 2024; Kessler, 2024), such as pre-tasks, during-tasks, and post-tasks. The requirement to produce multimodal output in the DMC projects might account for the scarcity of DMC implementation in primary and secondary schools where students' learning priority is to master monolingual knowledge and skills to pass exams.

To narrow the gap between research and teaching, L2 teachers can simplify or adapt the DMC task designs to make them more manageable for their teaching practices. For example, instead of asking students to complete all the steps of the DMC project, they may consider tasking students with analyzing multimodal texts via writing critiques or providing oral comments. By critically analyzing the use of linguistic and non-linguistic resources in multimodal texts, students will learn how to make meaning by leveraging multimodality. For instance, the science teachers in Wanselin et al. (2022) asked the students to analyze multimodal texts to elucidate complex scientific concepts and processes. This practice can inspire L2 teachers to ask students to analyze DMC works to hone their multimodal literacy before making them design DMC works on their own.

L2 teachers can also provide input-based multimodal tasks. For example, the L2 teachers in Ryu and Boggs (2016) reported that the students were motivated to write more in the writing task with multimodal input than in that with monomodal input. L2 teachers might provide a series of pictures or some video clips, and then ask students to continue writing the stories based on their imagination and creativity. For example, in Hong and Shi (2016), the Chinese L2 teacher provided texts and pictures for students to continue writing stories and found that the task strengthened the interaction between the students and the multimodal input.

5. Conclusion

DMC has been valued as an engaging pedagogy in L2 writing in recent decades. It mobilizes and activates the semiotic resources available to learners in their meaning-making activities. In the current digital era, learning is a dynamic and mobile process where real-life communication is hybrid, multiple, and mixing, not a static, single-layered concept that accommodates social practices (Jones & Hafner, 2021). This further foregrounds the necessity of DMC, which reflects authentic communication processes. Although research on L2 DMC is flourishing and evinces its benefits for students' development as L2 users and writers, there are still some missing links between research findings and classroom practices. The teaching and assessment of L2 DMC represent a burgeoning issue. Teachers might be skeptical about the learning potential of such writing projects, as they find it hard to maintain an optimal balance between the technological elements and academic writing improvement. Therefore, DMC is largely treated as a teacher-initiated activity that is not legitimated in institutional curriculum. Besides, digital multimodal competences have not received enough attention from national policymakers, and DMC assessment is largely left to teachers' discretion, which is unfortunately unsystematic and improperly informed by scholarly efforts. Aside from the institutional barriers for the communication between DMC research and practices, the current theoretical discussion on DMC also brings challenges for L2 instructors. They might lack confidence in implementing DMC and in justifying its use on a theoretically sound basis.

As such, there is a great need to incorporate DMC into L2 teacher education so that instructors are equipped with the necessary knowledge and competence to design, implement, and assess students' DMC productions. Meanwhile, L2 teachers should view themselves as legitimate designers who iteratively engage in appropriation, implementation, and reflection. They should claim ownership of their pedagogical practices and acknowledge their own expertise, experience, and knowledge in teaching and assessing L2 DMC productions in their specific classroom contexts. A mentality of teacher-as-researcher is necessary as instructors can constantly collect and analyze data of student DMC performance and propose ways for improvement, thus building classroom-based DMC theories. The teacher–researcher approach could also be an effective response to institutional constraints. Teachers can collect evidence for their pedagogical choice of DMC in the form of, for example, scientific research reports, systematic recording of students' progress, and critical reflection of their teaching and assessment practices, thus building connections and trust with students, parents, colleagues, and school policymakers.