Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-10T02:38:58.920Z Has data issue: false hasContentIssue false
  • English
  • Français

Managing multimodal data in virtual world research for language learning

Published online by Cambridge University Press:  24 January 2018

Cristina Palomeque  and
Joan-Tomàs Pujolà
Cristina Palomeque
Affiliation:
University of Barcelona, Spain (email: [email protected])
Joan-Tomàs Pujolà
Affiliation:
University of Barcelona, Spain (email: [email protected])
Get access Rights & Permissions [Opens in a new window]

Abstract

The study of multimodality in communication has attracted the attention of researchers studying online multimodal environments such as virtual worlds. Specifically, 3D virtual worlds have especially attracted the interest of educators and academics due to the multiplicity of verbal channels, which are often comprised of text and voice channels, as well as their 3D graphical interface, allowing for the study of non-verbal modes. This study offers a multilayered transcription method called the Multi-Modal MUVE Method or 3M Method (Palomeque, 2016; Pujolà & Palomeque, 2010) to account for the different modes present in the 3D virtual world of Second Life. This method works at two levels: the macro and the micro level. The macro level is a bird’s-eye view representation of the whole session as it fits into one page. This enables the researcher to grasp the essence of the class and to identify interesting sequences for analysis. The micro level consists of three transcripts to account for the different communication modes as well as the interface activity that occurs in the virtual world of Second Life. This paper will review the challenges when dealing with multimodal analysis in virtual worlds and how the multimodal data were analyzed and interpreted by using a multilayered multimodal method of analysis (3M transcription). Examples will be provided in the study to show how different modes of communication were used by participants in the virtual world of Second Life to create meaning or to avoid communication breakdowns.

Keywords

multimodalitymultimodal transcriptionvirtual worldsSecond Life
Type
Regular papers
Information
ReCALL , Volume 30 , Special Issue 2: Interactions for language learning in and around virtual worlds , May 2018 , pp. 177 - 195
Check for updates
Copyright
Copyright © European Association for Computer Assisted Language Learning 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, T. L. (2009) Online instructor immediacy and instructor-student relationships in second life. In Wankel, C. & Kingsley, J. (eds.), Higher education in virtual worlds: Teaching and learning in Second Life. Warrington: Emerald Group Publishing, 101–114.Google Scholar
Atkins, C. and Caukill, M. (2009) Serious fun and serious learning: The challenge of Second Life. In Molka-Danielsen, J. & Deutschmann, M. (eds.), Learning and teaching in the virtual world of Second Life. Trondheim: Tapir Academic Press, 79–89.Google Scholar
Baldry, A. (2000) Introduction. In Baldry, A. (ed.), Multimodality and multimediality in the distance learning age: Papers in English linguistics. Campobasso: Palladino Editore, 11–39.Google Scholar
Baldry, A. and Thibault, P. J. (2001) Towards multimodal corpora. In Aston, G. & Burnard, L. (eds.), Corpora in the description and teaching of English. Bologna: CLUEB, 87–102.Google Scholar
Baldry, A. P. (2004) Phase and transition, type and instance: Patterns in media texts as seen through a multimodal concordancer. In O’Halloran, K. L. (ed.), Multimodal discourse analysis: Systemic functional perspectives. London: Continuum, 83–108.Google Scholar
Bearne, E. (2009) Multimodality, literacy and texts: Developing a discourse. Journal of Early Childhood Literacy, 9(2): 156–187. https://doi.org/10.1177/1468798409105585 Google Scholar
Bezemer, J. and Jewitt, C. (2010) Multimodal analysis: Key issues. In Litosseliti, L. (ed.), Research methods in linguistics. London: Continuum, 180–197.Google Scholar
Book, B. (2004) Moving beyond the game: Social virtual worlds. Proceedings of State of Play 2 Conference. New York: New York Law School, 6–8.Google Scholar
Cambra Giné, M. (2003) Une approche ethnographique de la classe de langue. Paris: Didier.Google Scholar
Cunningham, U., Beers Fägersten, K. and Holmsten, E. (2010) “Can you hear me, Hanoi?” Compensatory mechanisms in synchronous net-based English language learning. The International Review of Research in Open and Distance Learning, 11(1): 161–177. https://doi.org/10.19173/irrodl.v11i1.774 Google Scholar
Deutschmann, M. and Panichi, L. (2009) Instructional design, teacher practice and learner autonomy. In Molka-Danielsen, J. & Deutschmann, M. (eds.), Learning and teaching in the virtual world of Second Life. Trondheim: Tapir Academic Press, 27–44.Google Scholar
Domingo, M. (2011) Analyzing layering in textual design: A multimodal approach for examining cultural, linguistic, and social migrations in digital video. International Journal of Social Research Methodology, 14(3): 219–230. https://doi.org/10.1080/13645579.2011.563619 CrossRefGoogle Scholar
Downey, S. (2014) History of the (virtual) worlds. The Journal of Technology Studies, 40(2): 54–66. https://doi.org/10.21061/jots.v40i2.a.1 Google Scholar
Friedman, D., Steed, A. and Slater, M. (2007) Spatial social behavior in Second Life. In Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K. & Pelé, D. (eds.), Intelligent virtual agents: 7th International Conference, IVA 2007, Paris, France, September 2007 Proceedings. Berlin Heidelberg: Springer, 252–263. https://doi.org/10.1007/978-3-540-74997-4_23 Google Scholar
Hampel, R. and Hauck, M. (2006) Computer-mediated language learning: Making meaning in multimodal virtual learning spaces. The JALT CALL Journal, 2(2): 3–18.CrossRefGoogle Scholar
Hampel, R. and Stickler, U. (2012) The use of videoconferencing to support multimodal interaction in an online language classroom. ReCALL, 24(2): 116–137. doi: https://doi.org/10.1017/S095834401200002X.Google Scholar
Healey, D. (2016) Language learning and technology: Past, present and future. In Farr, F. & Murray, L. (eds.), The Routledge handbook of language learning and technology. Abingdon: Routledge, 9–22.Google Scholar
Herring, S. C. (2002) Computer-mediated communication on the internet. Annual Review of Information Science and Technology, 36(1): 109–168. https://doi.org/10.1002/aris.1440360104 Google Scholar
Herring, S. C. (2004) Computer-mediated discourse analysis: An approach to researching online behavior. In Barab, S. A., Kling, R. & Gray, J. H. (eds.), Designing for virtual communication in the service of learning. Cambridge: Cambridge University Press, 338–376. https://doi.org/10.1017/CBO9780511805080.016 Google Scholar
Herring, S. C. (2007) A faceted classification scheme for computer-mediated discourse. Language@Internet, 4: 1–37.Google Scholar
Herring, S. C. (2015) New frontiers in interactive multimodal communication. In A. Georgopoulou & T. Spilloti (eds.), The Routledge handbook of language and digital communication. London: Routledge, 398–402.Google Scholar
Hubbard, P. (2009) General introduction. In Hubbard, P. (ed.), Computer assisted language learning: Critical concepts in linguistics. London: Routledge, 1–20.Google Scholar
Kress, G. (1998) Visual and verbal modes of representation in electronically mediated communication: the potentials of new forms of text. In I. Snyder & M. Joyce (Eds.), Page To Screen: taking literacy into the electronic era (pp. 53–79). London: Routledge.Google Scholar
Kress, G. and van Leeuwen, T. (2001) Multimodal discourse: The modes and media of contemporary communication. London: Arnold.Google Scholar
Lamy, M.-N. (2006) Multimodality in online language learning environment: Looking for a methodology. In Baldry, A. and Montagna, E. (eds.), Third International Conference on Multimodality, Pavia: Palladino, 237–254. https://telearn.archives-ouvertes.fr/hal-00197411 Google Scholar
Lee, L. (2002) Synchronous online exchanges: A study of modification devices on non-native discourse. System, 30(3): 275–288. https://doi.org/10.1016/S0346-251X(02)00015-5 CrossRefGoogle Scholar
Naper, I. (2011) Conversation in a multimodal 3D virtual environment. Language@Internet, 8(7). http://urn:nbn:de:0009-7-32225 Google Scholar
Norris, S. (2004) Analyzing multimodal interaction. A methodological framework. New York: Routledge.Google Scholar
Nunan, D. (1989) Designing tasks for the communicative classroom. Cambridge: Cambridge University Press.Google Scholar
O’Halloran, K. (2004) Visual semiosis in film. In O’Halloran, K. (ed.), Multimodal discourse analysis: Systemic functional perspective. London: Continuum, 109–130.Google Scholar
O’Halloran, K. (2011) Multimodal discourse analysis. In K. Hayland & B. Paltridge (eds.), Companion to discourse analysis. London & New York: Continuum, 120–137.Google Scholar
Örnberg, T. (2005) Multimodality in a three-dimensional voice chat. In Allwood, J., Dorriots, B. & Nicholson, S. (eds.), Proceedings of the 3rd Conference on Multimodal Communication, Papers in Theoretical Linguistics. Göteborg: Göteborg University, 303–316.Google Scholar
Palomeque, C. E. (2016) Communication in a MUVE: An exploratory case study of teacher interactional devices in Second Life. University of Barcelona, unpublished PhD. https://www.educacion.gob.es/teseo/mostrarRef.do?ref=1246116# Google Scholar
Pellettieri, J. (2000) Negotiation in cyberspace: The role of chatting in the development of grammatical competence. In Warschauer, M. & Kern, R. (eds.), Network-based language teaching: Concepts and practice. Cambridge: Cambridge University Press, 59–86. https://doi.org/10.1017/CBO9781139524735.006 Google Scholar
Peterson, M. (2008) An investigation of learner interaction in a MOO-based virtual environment. University of Edinburgh, unpublished PhD. https://core.ac.uk/download/pdf/279166.pdf Google Scholar
Pujolà, J.-T. (2002) CALLing for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL, 14(2): 235–262. https://doi.org/10.1017/S0958344002000423 Google Scholar
Pujolà, J.-T. and Palomeque, C. (2010) Developing a multimodal transcription to account for interaction in 3D virtual worlds: The 3M method. In Bueno Alonso, J. L. (ed.), Analizar Datos. Describir Variación. Vigo: University of Vigo, Servizo de Publicacións, 134–145.Google Scholar
Salmon, G. (2009) The future for (second) life and learning. British Journal of Educational Technology, 40(3): 526–538. https://doi.org/10.1111/j.1467-8535.2009.00967.x CrossRefGoogle Scholar
Sweeney, P., Palomeque, C., González, D., Speck, C., Canfield, D. W, Guerrero, S. and MacKichan, P. (2011) Task design for language learning in an embodied environment. In Vincenti, G. & Braman, J. (eds.), Teaching through multi-user virtual environments: Applying dynamic elements to the modern classroom. Hershey, PA: Information Science Reference, 259–282.Google Scholar
Tan, S., O’Halloran, K. L. and Wignell, P. (2016) Multimodal research: Addressing the complexity of multimodal environments and the challenges for CALL. ReCALL, 28(3): 253–273. https://doi.org/10.1017/S0958344016000124 Google Scholar
Tudini, V. (2003) Using native speakers in chat. Language Learning & Technology, 7(3): 141–159.Google Scholar
Wang, A. (2015) Facilitating participation: Teacher roles in a multiuser virtual learning environment. Language Learning & Technology, 19(2): 156–176.Google Scholar
Warburton, S. (2009) Second Life in higher education: Assessing the potential for and the barriers to deploying virtual worlds in learning and teaching. British Journal of Educational and Technology, 40(3): 414–426. https://doi.org/10.1111/j.1467-8535.2009.00952.x Google Scholar
Warschauer, M. (ed.) (1995) Telecollaboration in foreign language learning. Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.Google Scholar
Werry, C. C. (1996) Linguistic and interactional features of Internet relay chat. In S. C. Herring (ed.), Computer-mediated communication: Linguistic, social and cross-cultural perspectives. Amsterdam: John Benjamins 47–64. https://doi.org/10.1075/pbns.39.06wer Google Scholar
Wigham, C. R. (2012) The interplay between nonverbal and verbal interaction in synthetic worlds which supports verbal participation and production in a foreign language. Université Blaise Pascal, unpublished PhD. https://tel.archives-ouvertes.fr/tel-00762382 Google Scholar
Williamson, R. (2007) El diseño de un corpus multimodal. Estudios de Lingüística Aplicada, 25(46): 207–231.Google Scholar
Yee, N., Bailenson, J. N., Urbanek, M., Chang, F. and Merget, D. (2007) The unbearable likeness of being digital: The persistence of nonverbal social norms in online virtual environments. CyberPsychology & Behavior, 10(1): 115–121. https://doi.org/10.1089/cpb.2006.9984 Google Scholar