Hostname: page-component-586b7cd67f-gb8f7 Total loading time: 0 Render date: 2024-11-28T12:28:07.921Z Has data issue: false hasContentIssue false
  • English
  • Français

Using elicited imitation to measure global oral proficiency in SLA research: A close replication study

Published online by Cambridge University Press:  12 August 2020

Kevin McManus  and
Yingying Liu
Kevin McManus*
Affiliation:
Department of Applied Linguistics, The Pennsylvania State University, University Park, USA
Yingying Liu
Affiliation:
Department of Applied Linguistics, The Pennsylvania State University, University Park, USA
*
*Corresponding author: Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

We closely replicated Wu and Ortega (2013), who found that an elicited imitation test (EIT) reliably distinguished low-level from high-level language abilities among instructed second language (L2) learners of Mandarin Chinese. The original study sampled learners (1) from second-level courses to represent low-level language abilities and (2) from third-, fourth- and graduate-level courses to represent high-level language abilities. Results showed high-level learners outperformed low-level learners on the Mandarin EIT. Our close replication used Wu and Ortega's (2013) materials and procedures in order to understand (1) the extent to which this EIT can additionally distinguish between finer-grained language abilities and (2) the ways in which the broad grouping of language abilities in the high group may have contributed to the findings. Sixty-five instructed L2 learners from four instructional levels were assigned to one of three groups: Beginner (first-level courses), Low (second-level courses), High (third- and fourth-level courses). Consistent with the original study, our results showed clear between-group differences, indicating that the EIT can distinguish between both broad (beginner vs high) and finer-grained (beginner vs low, low vs high) language abilities. These results are discussed in light of the original study's findings with implications for proficiency assessment in second language acquisition (SLA) research.

Type
Replication Research
Information
Language Teaching , Volume 55 , Issue 1 , January 2022 , pp. 116 - 135
Check for updates
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Berman, R., & Slobin, D. (1994). Relating events in narrative: A crosslinguistic developmental study. Mahwah, NJ: Erlbaum.Google Scholar
Bowden, H. W. (2016). Assessing second-language oral proficiency for research: The Spanish elicited imitation task. Studies in Second Language Acquisition, 38(4), 647–675.CrossRefGoogle Scholar
Brewer, M. B., & Crano, W. D. (2000). Research design and issues of validity. In Reis, H. T., & Judd, C. M. (Eds.), Handbook of research methods in social and personality psychology (pp. 3–16). Cambridge, UK: Cambridge University Press.Google Scholar
Brown, A. V., Plonsky, L., & Teimouri, Y. (2018). The use of course grades as metrics in L2 Research: A systematic review. Foreign Language Annals, 51(4), 763–778.CrossRefGoogle Scholar
Byrnes, H. (2013). Editor's introduction. Modern Language Journal, 97(1), 105–108.CrossRefGoogle Scholar
Dewey, D. P., Bown, J., & Eggett, D. (2012). Japanese language proficiency, social networking, and language use during study abroad: Learners’ perspectives. The Canadian Modern Language Review, 68(2), 111–137.CrossRefGoogle Scholar
Di Silvio, F., Diao, W., & Donovan, A. (2016). The development of L2 fluency during study abroad: A cross-language study. Modern Language Journal, 100(3), 610–624.CrossRefGoogle Scholar
Du, H. (2013). The development of Chinese fluency during study abroad in China. Modern Language Journal, 97(1), 131–143.CrossRefGoogle Scholar
Dussias, P. E., Valdés Kroff, J. R., Guzzardo Tamargo, R. E., & Gerfen, C. (2013). When gender and looking go hand in hand. Studies in Second Language Acquisition, 35(2), 353–387.CrossRefGoogle Scholar
Ellis, N. C. (2015). Foreword. Language Learning, 65(S1), v–vi.CrossRefGoogle Scholar
Erlam, R. (2006). Elicited imitation as a measure of L2 implicit knowledge: An empirical validation study. Applied Linguistics, 27(3), 464–491.CrossRefGoogle Scholar
Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Los Angeles, CA: Sage Publications.Google Scholar
Gaillard, S., & Tremblay, A. (2016). Oral proficiency assessment in second language acquisition research: The elicited imitation task. Language Learning, 66(2), 419–447.CrossRefGoogle Scholar
Gass, S. (2018). SLA elicitation tasks. In Phakiti, A., De Costa, P., Plonsky, L., & Starfield, S. (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 313–337). London, UK: Palgrave Macmillan.CrossRefGoogle Scholar
Hudson, T., & Llosa, L. (2015). Design issues and inference in experimental L2 research. Language Learning, 65(S1), 76–96.CrossRefGoogle Scholar
Hulstijn, J. H. (2010). Measuring second language proficiency. In Blom, E. & Unsworth, S. (Eds.), Experimental methods in language acquisition research (pp. 185–200). Amsterdam, Netherlands: John Benjamins.CrossRefGoogle Scholar
Hulstijn, J. H. (2011). Language proficiency in native and nonnative speakers: An agenda for research and suggestions for second-language assessment. Language Assessment Quarterly, 8(3), 229–249.CrossRefGoogle Scholar
Hulstijn, J. H. (2012). The construct of language proficiency in the study of bilingualism from a cognitive perspective. Bilingualism: Language and Cognition, 15(2), 422–433.CrossRefGoogle Scholar
Hulstijn, J. H. (2015). Language proficiency in native and non-native speakers: Theory and research (Vol. 41). Amsterdam, Netherlands: John Benjamins.CrossRefGoogle Scholar
Iwashita, N. (2018). Grammar and language proficiency. In Liontas, J. I. (Ed.), The TESOL encyclopedia of English language teaching. Retrieved from https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118784235.eelt0069Google Scholar
Iwashita, N., & Ortega, L. (1998). Scoring guidelines for elicited imitation task L2 English. Retrieved from https://www.iris-database.org/iris/app/home/download?id=york:852670&ip=73.187.219.252&placeValuesBefore%20, https://www.iris-database.org/iris/api/resource/york:852670/asset/Google Scholar
Jessop, L., Suzuki, W., & Tomita, Y. (2007). Elicited imitation in second language acquisition research. The Canadian Modern Language Review, 64(1), 215–238.CrossRefGoogle Scholar
Kim, Y., Tracy-Ventura, N., & Jung, Y. (2016). A measure of proficiency or short-term memory? Validation of an elicited imitation test for SLA research. Modern Language Journal, 100, 655–673.CrossRefGoogle Scholar
Laufer, B., & Goldstein, Z. (2004). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399–436.CrossRefGoogle Scholar
Leclercq, P., & Edmonds, A. (2014). How to assess L2 proficiency? An overview of proficiency assessment research. In Leclercq, P., Edmonds, A., & Hilton, H. (Eds.), Measuring L2 proficiency: Perspectives from SLA (pp. 3–23). Bristol, UK, Buffalo, NY, and Toronto, Canada: Multilingual Matters.CrossRefGoogle Scholar
Li, S. (2010). Corrective feedback in perspective: The interface between feedback type, proficiency, the choice of target structure, and learners’ individual differences in working memory and language analytic ability. Ph.D. dissertation, Michigan State University.Google Scholar
Li, X. (2015). Ditransitive constructions in Gan Chinese: A case study of the Yichun dialect. In Nolan, B., Rawoens, G., & Diedrichsen, E. (Eds.), Causation, permission and transfer: Argument realization in get, take, put, give and let verbs (pp. 177–193). Philadelphia, PA: John Benjamins.Google Scholar
Mackey, A., & Gass, S. (2016). Second language research: Methodology and design. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018). Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, 68(2), 321–391.CrossRefGoogle Scholar
McManus, K., & Marsden, E. (2018). Online and offline effects of L1 practice in L2 grammar learning: A partial replication. Studies in Second Language Acquisition, 40(2), 459–475.CrossRefGoogle Scholar
Ng, H. T., & Low, J. K. (2004). Chinese part-of-speech tagging: One-at-a-time or all-at-once? word-based or character-based? Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing 9, 277–284.Google Scholar
Ortega, L., Iwashita, N., Norris, J. M., & Rabie, S. (2002). An investigation of elicited imitation tasks in crosslinguistic SLA research. Paper presented at the Second Language Research Forum, Toronto, Canada.Google Scholar
Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30(4), 590–601.CrossRefGoogle Scholar
Phakiti, A., De Costa, P., Plonsky, L., & Starfield, S. (2018). Applied linguistics research: Current issues, methods, and trends. In Phakiti, A., De Costa, P., Plonsky, L., & Starfield, S. (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 5–29). London, UK: Palgrave Macmillan.CrossRefGoogle Scholar
Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35, 655–687.CrossRefGoogle Scholar
Plonsky, L. (2015). Advancing quantitative methods in second language research. New York, NY: Routledge.CrossRefGoogle Scholar
Plonsky, L., & Ghanbar, H. (2018). Multiple regression in L2 research: A methodological synthesis and guide to interpreting R 2 values. The Modern Language Journal, 102, 713–731.CrossRefGoogle Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912.CrossRefGoogle Scholar
Porte, G., & McManus, K. (2019). Doing replication research in applied linguistics. New York, NY: Routledge.Google Scholar
Qian, Y., & Pan, W. (2002). Prosodic word: The lowest constituent in the Mandarin prosody processing. Proceedings of the ISCA International Conference on Speech Prosody, 2002, 591–594.Google Scholar
Rebuschat, P., & Mackey, A. (2013). Prompted production. In Chappelle, C. A. (Ed.), The encyclopedia of applied linguistics (Vol. 5). Oxford, UK: Wiley-Blackwell.Google Scholar
Slabakova, R. (2015). The effect of construction frequency and native transfer on second language knowledge of the syntax-discourse interface. Applied Psycholinguistics, 36(3), 671–699.CrossRefGoogle Scholar
Solon, M., Park, H. I., Henderson, C., & Dehghan-Chaleshtori, M. (2019). Revisiting the Spanish elicited imitation task: A tool for assessing advanced language learners? Studies in Second Language Acquisition, 41(5), 1027–1053.CrossRefGoogle Scholar
Thomas, M. (1994). Assessment of L2 proficiency in second language acquisition research. Language Learning, 44(2), 307–336.CrossRefGoogle Scholar
Thomas, M. (2006). Research synthesis and historiography: The case of assessment of second language proficiency. In Norris, J. M. & Ortega, L. (Eds.), Synthesizing research on language learning and teaching (pp. 279–298). Philadelphia, PA: John Benjamins.CrossRefGoogle Scholar
Tracy-Ventura, N., McManus, K., Norris, J. M., & Ortega, L. (2014). ‘Repeat as much as you can’: Elicited imitation as a measure of oral proficiency in L2 French. In Leclercq, P., Hilton, H., & Edmonds, A. (Eds.), Measuring L2 Proficiency: Perspectives from SLA (pp. 143–166). Bristol, UK: Multilingual Matters.CrossRefGoogle Scholar
Tremblay, A. (2011). Proficiency assessment standards in second language acquisition research: ‘Clozing’ the gap. Studies in Second Language Acquisition, 33(3), 339–372.CrossRefGoogle Scholar
Underhill, N. (1987). Testing spoken language. Cambridge, UK: Cambridge University Press.Google Scholar
Wu, S. L., & Ortega, L. (2013). Measuring global oral proficiency in SLA research: A new elicited imitation test of L2 Chinese. Foreign Language Annals, 46(4), 680–704.CrossRefGoogle Scholar
Yan, X., Maeda, Y., Lv, J., & Ginther, A. (2016). Elicited imitation as a measure of second language proficiency: A narrative review and meta-analysis. Language Testing, 33(4), 497–528.CrossRefGoogle Scholar
Yuan, F., & Li, S. (2019). Classroom research on Chinese as a second language. New York, NY/London, UK: Routledge.CrossRefGoogle Scholar
Zhou, Y. (2012). Willingness to communicate in learning Mandarin as a foreign and heritage language. Ph.D. dissertation, University of Hawai‘i at Mānoa, Honolulu.Google Scholar
Zhou, Y., & Wu, S. L. (2009). Can elicited imitation be used for the measurement of oral proficiency in L2 Chinese? A pilot study. Unpublished manuscript, University of Hawai‘i at Mānoa.Google Scholar