Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-78c5997874-v9fdk Total loading time: 0 Render date: 2024-11-02T23:53:08.643Z Has data issue: false hasContentIssue false

10 - Justifying the Interpretation and Use of an ESL Writing Final Examination

from Part III - Investigating Score Uses and Consequences

Published online by Cambridge University Press:  14 January 2021

By
Heesung Jun
Edited by
Carol A. Chapelle  and
Erik Voss
Carol A. Chapelle
Affiliation:
Iowa State University
Erik Voss
Affiliation:
Teachers College, Columbia University
Get access

Summary

A web-researching-to-write integrated writing test was developed for the final examination in an ESL academic writing course for international undergraduate students a university in the US. To support the score interpretations, uses of the results for awarding grades, and consequences, research was conducted to investigate the seven inferences that made up the interpretation/use argument: domain definition, evaluation, generalization, explanation, extrapolation, utilization, and consequence implication. To answer research questions motivated by the assumptions in the interpretation/use argument, a convergent mixed methods design was used for data collection and analysis. Data included students’ essay responses, screen recordings, post-test and follow-up test-taker questionnaire responses, post-test and follow-up test-taker interviews, instructor interviews, and documents. Results showed that all of the assumptions underlying the seven inferences in the validity argument were at least partially supported. Further research is suggested to produce additional backing for some inferences. Conclusions are drawn about the usefulness of a validity argument for prompting curriculum change. In addition, this study provides an example of a validity argument constructed for low-stakes classroom-based assessment.

Keywords

esl writing courseweb researching to writesource based writingacademic writinginternational undergraduate students
Type
Chapter
Information
Validity Argument in Language Testing
Case Studies of Validation Research
 , pp. 235 - 263
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Chan, S. (2017). Using keystroke logging to understand writers’ processes on a reading-into-writing test. Language Testing in Asia, 7(10), 1–27.CrossRefGoogle Scholar
Chapelle, C. A., Enright, M. K., & Jamieson, J. M. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign LanguageTM. New York: Routledge.Google Scholar
Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5–43.Google Scholar
Fulcher, G. (2010). Practical language testing. London: Hodder Education.Google Scholar
Gebril, A., & Plakans, L. (2009). Investigating source use, discourse features, and process in integrated writing tests. Spann Fellow Working Papers in Second or Foreign Language Assessment, 7, 47–84.Google Scholar
Jun, H. S. (2014). A validity argument for the use of scores from a web-search-permitted and web-source-based integrated writing test. Unpublished doctoral dissertation, Iowa State University, Ames, IA.Google Scholar
Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527–535.Google Scholar
Kane, M. T. (2006). Validation. In Brennan, R. L. (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: Greenwood Publishing.Google Scholar
Kane, M. T. (2012). Articulating a validity argument. In Fulcher, G. & Davidson, F. (Eds.), The Routledge handbook of language testing (pp. 34–47). New York: Routledge.Google Scholar
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.CrossRefGoogle Scholar
Ranalli, J., Feng, H.-S., & Chukharev-Hudilainen, E. (2018). Exploring the potential of process-tracing technologies to support assessment for learning of L2 writing. Assessing Writing, 36, 77–89.Google Scholar
Stapleton, P. (2005a). Evaluating web-sources: Internet literacy and L2 academic writing. ELT Journal, 59(2), 135–143.CrossRefGoogle Scholar
Stapleton, P. (2005b). Using the web as a research source: Implications for L2 academic writing. Modern Language Journal, 89(2), 177–189.Google Scholar
Weigle, S. C., & Jensen, L. (1997). Assessment issues for content-based instruction. In Snow, M. A. & Brinton, D. (Eds.), The content-based classroom: Perspectives on integrating language and content (pp. 201–212). White Plains, NY: Addison Wesley Longman.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×