Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-26T16:14:54.296Z Has data issue: false hasContentIssue false
  • English
  • Français

Considerations in assessing pragmatic appropriateness in spoken language

Published online by Cambridge University Press:  02 July 2019

Andrew D. Cohen
Andrew D. Cohen*
Affiliation:
Professor Emeritus, University of Minnesota, USA
*
Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper is a revised version of a plenary prompted by the upsurge of interest in the role of pragmatics in teaching, learning, and assessment, and has as its purpose to take a fresh look at recent developments in the assessment of target-language (TL) pragmatics in spoken language. The first issue considered is the question of whether to attempt to assess pragmatics as it unfolds naturally in interactions, and if so, how to do it. Next, micro-level and macro-level factors in the assessment of TL pragmatics are considered. Third, a close look is given to the specific elements of TL pragmatics to assess. Fourth, there is attention to the matter of which instruments are most appropriate for collecting the desired data in the given context. Fifth, issues relating to data analysis are discussed. Finally, matters pertaining to the assessment of classroom instruction are looked at. Recommendations are given as to potentially viable directions for dealing with these issues both in terms of research studies and for assessment of classroom instruction.

Type
Plenary Speech
Information
Language Teaching , Volume 53 , Issue 2 , April 2020 , pp. 183 - 202
Copyright
Copyright © Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Paper presented as a plenary lecture at the International Conference on Exploring and Assessing Pragmatic Aspects of L1 and L2 Communication: From Needs Analysis through Monitoring to Feedback. Department of Linguistic and Literary Studies, University of Padua, Italy, 25–27 July 2018.

References

Alemi, M., Eslami Rasekh, Z., & Rezanejad, A. (2014). Iranian non-native English-speaking teachers’ rating criteria regarding the speech act of compliment: An investigation of teachers’ variables. The Journal of Teaching Language Skills, 6(3), 21–49.Google Scholar
Alemi, M., & Khanlarzadeh, N. (2015). Native EFL raters’ criteria in assessing the speech act of complaint: The case of American and British EFL teachers. Iranian Journal of Applied Language Studies, 7(2), 25–60.Google Scholar
Alemi, M., & Khanlarzadeh, N. (2017). Native and nonnative teachers’ pragmatic criteria for rating the request speech act: The case of American and Iranian EFL teachers. Applied Research on English Language, 6(1), 67–84.Google Scholar
Al-Gahtani, S., & Roever, C. (2012). Proficiency and sequential organization of L2 requests. Applied Linguistics, 33(1), 42–65.CrossRefGoogle Scholar
Al Masaeed, K., Waugh, L. A., & Burns, K. E. (2018). The development of interlanguage pragmatics in L2 Arabic: The production of apology strategies. System, 74, 98–108.CrossRefGoogle Scholar
Bardovi-Harlig, K. (2017). Acquisition of L2 pragmatics. In Loewen, S. & Sato, M. (Eds.), Routledge handbook of instructed SLA (pp. 224–245). New York: Routledge.Google Scholar
Barron, A. (2019). Norms and variation in L2 pragmatics. In Taguchi, N. (Ed.), Routledge handbook of SLA and pragmatics. (pp. 447–461). New York/Abingdon: Routledge.Google Scholar
Beebe, L. M., & Waring, H. Z. (2002). The pragmatics in the Interlanguage pragmatics research agenda: The case of tone. The Annual American Association for Applied Linguistics Conference, Salt Lake City, UT, 6–9 April.Google Scholar
Beebe, L. M., & Waring, H. Z. (2005). Pragmatic development in responding to rudeness. In Frodesen, J. & Holten, C. (Eds.), The power of context in language teaching and learning (pp. 67–79). Boston: Thomson/Heinle.Google Scholar
Bieswanger, M. (2015). Variational pragmatics and responding to thanks – revisited. Multilingua 34(4), 527–546.CrossRefGoogle Scholar
Bowles, M. A. (2010). The think-aloud controversy in second language research. Abingdon: Routledge.CrossRefGoogle Scholar
Brown, A. (1995). The effect of rater variables in the development of an occupation-specific language performance test. Language Testing, 12(1), 1–15.CrossRefGoogle Scholar
Brown, J. D. (2017). Forty years of doing second language testing, curriculum, and research: So what? Language Teaching, 50(2), 276–289.CrossRefGoogle Scholar
Caprario, M. (2018). Pragmatic competence in English-language class discussions. International conference on exploring and assessing pragmatic aspects of L1 and L2 communication. Department of Linguistic and Literary Studies, University of Padua, Italy, 25–27 July.Google Scholar
Cohen, A. D. (2005). Strategies for learning and performing L2 speech acts. Intercultural Pragmatics, 2(3), 275–301.CrossRefGoogle Scholar
Cohen, A. D. (2006). The coming of age of research on test-taking strategies. Language Assessment Quarterly, 3(4), 307–331.CrossRefGoogle Scholar
Cohen, A. D. (2013). Verbal report. In Chapelle, C. A. (Ed.), The encyclopedia of applied linguistics (5 pp). Oxford: Wiley-Blackwell. Retrieved from https://sites.google.com/a/umn.edu/andrewdcohen/publications/research-methodologyGoogle Scholar
Cohen, A. D. (2018). Learning pragmatics from native and nonnative language teachers. Bristol: Multilingual Matters.CrossRefGoogle Scholar
Cohen, A. D., & Olshtain, E. (1993). The production of speech acts by EFL learners. TESOL Quarterly, 27(1), 33–56.CrossRefGoogle Scholar
Cohen, A. D., Olshtain, E., & Rosenstein, D. S. (1986). Advanced EFL apologies: What remains to be learned? International Journal of the Sociology of Language, 62(6), 51–74.Google Scholar
Cohen, A. D., & Shively, R. L. (2002/2003). Measuring speech acts with multiple rejoinder DCTs. Language Testing Update, 32, 39–42.Google Scholar
Eslami, Z., & Mirzaei, A. (2014). Speech act data collection in a non-Western context: Oral and written DCTs in the Persian language. Iranian Journal of Language Testing, 4(1), 137–154.Google Scholar
Félix-Brasdefer, C. (2006). Teaching the negotiation of multi-turn speech acts: Using conversation-analytic tools to teach pragmatics in the FL classroom. In Bardovi-Harlig, K., Félix-Brasdefer, C., & Omar, A. S. (Eds.), Pragmatics & language learning (pp. 167–197). Honolulu, HI: National Foreign Language Resource Center, University of Hawai'i.Google Scholar
Félix-Brasdefer, J. C. (2018). Role plays. In Hoffmann, C. R. & Bublitz, W. (Eds.), Pragmatics of social media (pp. 303–329). Berlin/Boston: De Gruyter Mouton.Google Scholar
Hamp-Lyons, L., & Condon, W. (1993). Questioning assumptions about portfolio-based assessment. College Composition and Communication, 44(2), 176–190.CrossRefGoogle Scholar
Hudson, T., Detmer, E., & Brown, J. D. (1992). A framework for testing cross-cultural pragmatics (Technical Report #2). Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.Google Scholar
Hudson, T., Detmer, E., & Brown, J. D. (1995). Developing prototypic measures of cross-cultural pragmatics (Technical Report #7). Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawai'i at Manoa.Google Scholar
Hymes, D. (1972). Models of the interaction of language and social life. In Gumperz, J. & Hymes, D. (Eds.), Directions in sociolinguistics: The ethnography of communication (pp. 35–71). New York: Holt, Rhinehart, & Winston.Google Scholar
Isaacs, T. (2016). Assessing speaking. In Tsagari, D. & Banerjee, J. (Eds.), Handbook of second language assessment (pp. 131–146). Berlin: De Gruyter Mouton.Google Scholar
Ishihara, N. (2009). Teacher-based assessment for foreign language pragmatics. TESOL Quarterly, 43(3), 445–470.CrossRefGoogle Scholar
Ishihara, N. (2013). Teacher-based assessment of L2 Japanese pragmatics: Classroom applications. In Ross, S. J. & Kasper, G. (Eds.), Assessing second language pragmatics (pp. 124–148). Basingstoke: Palgrave Macmillan.CrossRefGoogle Scholar
Ishihara, N. (2014). Assessment of pragmatics in the classroom. In Ishihara, N. & Cohen, A. D. (Eds.), Teaching and learning pragmatics: Where language and culture meet (pp. 286–317). New York: Routledge.CrossRefGoogle Scholar
Ishihara, N., & Chiba, A. (2014). Teacher-based or interactional? Exploring assessments for children's pragmatic development. Iranian Journal of Language Testing, 4(1), 84–112.Google Scholar
Jones, L. (2016). Language and gender identities. In Pierce, S. (Ed.), Routledge handbook of language and identity (pp. 210–224). Abingdon: Routledge.Google Scholar
Kasper, G. (2004). Speech acts in (inter)action: Repeated questions. Intercultural Pragmatics, 1(1), 125–133.CrossRefGoogle Scholar
Kasper, G., & Youn, S. J. (2017). Transforming instruction to activity: Roleplay in language assessment. Applied Linguistics Review. doi:10.1515/applirev-2017-0020Google Scholar
Koh, W.-Y., Lee, S., & Lee, J. (2018). L2 pragmatic comprehension of aural sarcasm: Context, intonation, and literal meaning. The Annual American Association for Applied Linguistics Conference, Chicago, 24–27 March.Google Scholar
Krashen, S. D. (1983). The din in the head, input, and the language acquisition device. Foreign Language Annals, 16(1), 41–44.CrossRefGoogle Scholar
Poehner, M. E. (2008). Dynamic assessment: A Vygotskian approach to understanding and promoting L2 development. New York: Springer.CrossRefGoogle Scholar
Roever, C. (2018). Testing of L2 pragmatics and possibilities for measuring implicit knowledge (Unpublished manuscript). Department of Linguistics & Applied Linguistics, The University of Melbourne, Australia.Google Scholar
Roever, C., & Ikeda, N. (in press). Testing L2 pragmatic competence. In Schneider, G. & Ifantidou, E. (Eds.), Handbook of developmental and clinical pragmatics. Berlin: Mouton de Gruyter.Google Scholar
Roever, C., & Kasper, G. (2018). Speaking in turns and sequences: Interactional competence as a target construct in testing speaking. Language Testing, 35(3), 331–355.CrossRefGoogle Scholar
Schauer, G. (2009). Interlanguage pragmatic development: The study abroad context. New York: Continuum.Google Scholar
Schauer, G. A. (2017). ‘It's really insulting to say something like that to anyone’: An investigation of English and German native speakers’ impoliteness perceptions. In Kecskes, I. & Assimakopoulos, S. (Eds.), Current issues in intercultural pragmatics (pp. 207–227). Amsterdam: John Benjamins.CrossRefGoogle Scholar
Schneider, K. P. (2017). Pragmatic competence and pragmatic variation. In Giora, R. & Haugh, M. (Eds.), Doing pragmatics interculturally: Cognitive, philosophical, and sociopragmatic perspectives (pp. 315–333). Berlin: Walter de Gruyter.CrossRefGoogle Scholar
Shively, R. L. (2008). Politeness and social interaction in study abroad: Service encounters in L2 Spanish (Ph.D. dissertation). University of Minnesota, Minneapolis.Google Scholar
Shively, R. L. (2013). Learning to be funny in Spanish during study abroad: L2 humor development. Modern Language Journal, 97(4), 930–946.CrossRefGoogle Scholar
Shively, R. L. (2018). Naturalistic data in L2 pragmatics research: Challenges and opportunities. In Gudmestad, A. & Edmonds, A. (Eds.), Critical reflections on data in second language acquisition (pp. 192–218). Amsterdam: John Benjamins.Google Scholar
Taguchi, N. (2018). Data collection and analysis in developmental L2 pragmatics research: Discourse completion test, role-play, and naturalistic recording. In Gudmestad, A. & Edmonds, A. (Eds.), Critical reflections on data in second language acquisition (pp. 7–32). Amsterdam: John Benjamins.CrossRefGoogle Scholar
Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford: Oxford University Press.Google Scholar
Tajeddin, Z., & Alemi, M. (2013). Criteria and bias in native English teachers’ assessment of L2 pragmatic appropriacy: Content and FACETS analyses. Asia-Pacific Education Researcher, 23(3), 435–434.Google Scholar
Tajeddin, Z., & Alemi, M. (2014). Pragmatic rater training: Does it affect non-native L2 teachers’ rating accuracy and bias? Iranian Journal of Language Testing, 4(1), 66–83.Google Scholar
Tajeddin, Z., Alemi, M., & Pashmforoosh, R. (2017). Idealized native-speaker linguistic and pragmatic norms in English as an international language: Exploring the perceptions of nonnative English teachers. Language and Intercultural Communication, 18(3), 300–314.CrossRefGoogle Scholar
Tajeddin, Z., & Alizadeh, I. (2015). Monologic vs. dialogic assessment of speech act performance: Role of nonnative L2 teachers’ professional experience on their rating criteria. Journal of Research in Applied Linguistics, 6(1), 3–27.Google Scholar
Tajeddin, Z., & Dabbagh, A. (2015). Interlanguage pragmatic test tasks: Does a low-stakes test have washback to L2 teachers and learners? The Journal of Asia TEFL, 12(4), 129–158.Google Scholar
Usó-Juan, E., & Martínez-Flor, A. (2014). Reorienting the assessment of the conventional expressions of complaining and apologising: From single-response to interactive DCTs. Iranian Journal of Language Testing, 4(1), 113–136.Google Scholar
Yates, L. (2017). Learning how to speak: Pronunciation, pragmatics and practicalities in the classroom and beyond. Language Teaching, 50(2), 227–246.CrossRefGoogle Scholar
Youn, S. J. (2015). Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225.CrossRefGoogle Scholar
Youn, S. J. (2018a). Rater variability across examinees and rating criteria in paired speaking assessment. Papers in Language Testing and Assessment, 7, 32–60.Google Scholar
Youn, S. J. (2018b). Task design and validity evidence for assessment of L2 pragmatics in interaction. In Taguchi, N. & Kim, Y. (Eds.), Task-based approaches to teaching and assessing pragmatics (pp. 217–246). Amsterdam: John Benjamins.Google Scholar
Youn, S. J. (Under review with the TESOL Quarterly). Interactional features of L2 pragmatic interaction in role-play speaking assessment.Google Scholar
Youn, S. J. (2019). Managing proposal sequences in role-play assessment: Validity evidence of interactional competence across levels. Language Testing.Google Scholar
Youn, S. J. & Bi, N. Z. (2019). Investigating test-takers’ strategy use in task-based L2 pragmatic speaking assessment. Intercultural Pragmatics, 16, 185–218.CrossRefGoogle Scholar
Youn, S. J., & Bogorevich, V. (2019). Assessment in L2 pragmatics. In Taguchi, N. (Ed.), Routledge handbook of SLA and pragmatics (pp. 308–321). New York/Abingdon: Routledge.Google Scholar