Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-78c5997874-xbtfd Total loading time: 0 Render date: 2024-11-03T08:41:50.715Z Has data issue: false hasContentIssue false

8 - The Explanation Inference for a Test of Academic English Collocational Ability

from Part II - Investigating Score Interpretations

Published online by Cambridge University Press:  14 January 2021

By
Erik Voss
Edited by
Carol A. Chapelle  and
Erik Voss
Carol A. Chapelle
Affiliation:
Iowa State University
Erik Voss
Affiliation:
Teachers College, Columbia University
Get access

Summary

This chapter presents argument-based validation research to evaluate the interpretation of scores from an English collocational ability test. The argument-based validity framework guided the development of an interpretation/use argument that helped identify the types and amount of research needed to evaluate the plausibility of the claims about test score interpretation. Research presented in this chapter focuses on the explanation inference which is made when test users interpret the score as having substantive meaning about the construct assessed, specifically the construct of collocational ability in academic writing. The construct was defined by specifying the nature and scope of the construct following an interactionalist construct, which consists of three parts 1) the knowledge skills and abilities of a trait, 2) the types of contexts that delimit the scope of applicability of the trait, and 3) the metacognitive strategies to put the trait into use in those contexts. The target collocation was identified and defined based on applied linguistics theory and research, analysis of test-takers‘ responses to items on the test and statistical analysis of test scores. The relationship between collocational ability and other constructs of language ability were hypothesized in the nomological network to provide a basis for interpreting observed statistical relationships among sets of test scores reflecting those constructs. Evidence from screen capturing analysis, responses on a post-test survey and post-test interviews provided backing of strategy use. Data were collected in two phases of an embedded, sequential explanatory design to first obtain results from qualitative analysis of test takers' responses and then explain the results of the quantitative data with results from the supplementary qualitative data. Evidence collected in this study supported the construct of collocational ability underlying the explanation inference and demonstrated how argument-based validity can be used to lay a foundation for interpretation of test scores that is essential to score meaning.

Keywords

collocational ability testacademic languageexplanation inferenceconstruct definitionmetacognitive strategies
Type
Chapter
Information
Validity Argument in Language Testing
Case Studies of Validation Research
 , pp. 176 - 203
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alderson, J. C. (2000). Assessing reading. Cambridge: Cambridge University Press.Google Scholar
Alderson, C., & Kremmel, B. (2013). Re-examining the content validation of a grammar test: The (im)possibility of distinguishing vocabulary and structural knowledge. Language Testing, 30(4), 535–556.Google Scholar
Bachman, L., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press.Google Scholar
Bahns, J., & Eldaw, M. (1993). Should we teach EFL students collocations? System, 21(1), 101–114.Google Scholar
Biskup, D. (1992). L1 influence on learners’ renderings of English collocations: A Polish/German empirical study. In Arnaud, P. J. L. & Bejoint, H. (Eds.), Vocabulary and applied linguistics (pp. 85–93). London: Macmillan.Google Scholar
Chapelle, C. A. (1994). Are C-tests valid measures for L2 vocabulary research? Second Language Research, 10(2), 157–187.Google Scholar
Chapelle, C. A. (1998). Construct definition and validity inquiry in SLA research. In Bachman, L. F. & Cohen, A. D. (Eds.), Second language acquisition and language testing interfaces. Cambridge: Cambridge University Press.Google Scholar
Chapelle, C. A. (2020). Argument-based validation in testing and assessment. Sage.Google Scholar
Chapelle, C. A., Enright, M. K., & Jamieson, J. (2008). Building a validity argument for the Test of English as a Foreign Language. New York and London: Routledge.Google Scholar
Cheng, W., Greaves, C., Sinclair, J. McH., & Warren, M. (2009). Uncovering the extent of the phraseological tendency: Towards a systematic analysis of concgrams. Applied Linguistics, 30(2), 236–252.Google Scholar
Conrad, S. M., & Biber, D. (2005). The frequency and use of lexical bundles in conversation and academic prose. Lexicographica, 20, 56–71.CrossRefGoogle Scholar
Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34, 213–238.Google Scholar
Creswell, J. W., & Plano Clark, V. L., (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.Google Scholar
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.Google Scholar
Davies, M. (2008). The corpus of contemporary American English: 425 million words, 1990–present. Retrieved from http://corpus.byu.edu/coca/Google Scholar
Durrant, P. (2009). Investigating the viability of a collocation list for students of English for academic purposes. English for Specific Purposes, 28(3), 157–169.CrossRefGoogle Scholar
Ellis, N. C. (2006). SLA: The associative cognitive CREED. In VanPatten, B., Williams, J., & Williams, A. F. (Eds.), Theories in second language acquisition: An introduction. Mahwah, NJ: Erlbaum.Google Scholar
Howarth, P. (1996). Phraseology in English academic writing: Some implications for language learning and dictionary making. Tübingen, Germany: Niemeyer.Google Scholar
Howarth, P. (1998). Phraseology and second language proficiency. Applied Linguistics, 19(1), 24–44.Google Scholar
Hyland, K. (2012). Bundles in academic discourse. Annual Review of Applied Linguistics, 32, 150–169.Google Scholar
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.Google Scholar
Kane, M. T. (2006). Validation. In Brennen, R. (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: Greenwood.Google Scholar
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.Google Scholar
Karimi, N. (2011). C-test and vocabulary knowledge. Language Testing in Asia, 10(4), 7–38.Google Scholar
Laufer, B., & Nation, P. (1999). A vocabulary size test of controlled productive ability. Language Testing, 16(1), 33–51.Google Scholar
Lee, D. Y. W. (2001). Genres, registers, text types, domains and styles: Clarifying the concepts and navigating a path through the BNC jungle. Language Learning and Technology, 5(3), 37–72.Google Scholar
Leong, C. K., Ho, M. K., Chang, J., & Hau, K. T. (2013). Differential importance of language components in determining secondary school students’ Chinese reading literacy performance. Language Testing, 30(4), 419–439.CrossRefGoogle Scholar
Messick, S. (1989). Validity. In Linn, R. L. (Ed.), Educational measurement (pp. 13–104). New York: Macmillan.Google Scholar
Nesselhauf, N. (2003). The use of collocations by advanced learners of English and some implications for teaching. Applied Linguistics, 24, 223–242.Google Scholar
Qian, D. (1999). Assessing the roles of depth and breadth of vocabulary knowledge in reading comprehension. Canadian Modern Language Review, 56(2), 282–307.Google Scholar
Read, J., (2016). Some key issues in post-admission language assessment. In Read, J. (Ed.), Post-admission language assessment of university students. Switzerland: Springer.Google Scholar
Revier, R. L. (2009). Evaluating a new test of whole English collocations. In Gyllstad, J. & Barfield, A. (Eds.) Researching collocations in another language: Multiple interpretations (pp. 125–138). New York: Palgrave Macmillan.Google Scholar
Roche, R., Harrington, M., Sinha, Y., & Denman, C. (2016). Vocabulary recognition skill as a screening tool in English-as-a-Lingua-France university settings. In Read, J. (Ed.), Post-admission language assessment of university students. Switzerland: Springer.Google Scholar
Römer, U. (2017). Language assessment and the inseparability of lexis and grammar: Focus on the construct of speaking. Language Testing, 34(4), 477–492.Google Scholar
Shiotsu, T., & Weir, C. J. (2007). The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing, 24(1), 99–128.Google Scholar
Voss, E. (2012). A validity argument for score meaning of a computer-based ESL academic collocational ability test based on a corpus-driven approach to test design. Unpublished doctoral dissertation, Iowa State University, Ames, IA.Google Scholar
West, M. (1953). A general service list of English words. London: Longman, Green.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×