Book contents
- Validity Argument in Language Testing
- The Cambridge Applied Linguistics Series
- Validity Argument in Language Testing
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Series Editor’s Preface
- 1 Introduction to Validity Argument in Language Testing and Assessment
- Part I Basic Concepts and Uses of Validity Argument in Language Testing and Assessment
- Part II Investigating Score Interpretations
- 4 Domain Definition Inference for a Virtual Interactive Aviation English Test (VIAET) for Military Air Traffic Controllers
- 5 Support for the Evaluation Inference
- 6 Generalization Inference for a Computer-Mediated Graphic-Prompt Writing Test for ESL Placement
- 7 The Telephone Standard Speaking Test
- 8 The Explanation Inference for a Test of Academic English Collocational Ability
- 9 Extrapolation of the Meaning of Grades on Writing Tasks in an ESL Writing Class to Success on Tasks in University Courses
- Part III Investigating Score Uses and Consequences
- Part IV Conclusion
- Index
- References
6 - Generalization Inference for a Computer-Mediated Graphic-Prompt Writing Test for ESL Placement
from Part II - Investigating Score Interpretations
Published online by Cambridge University Press: 14 January 2021
- Validity Argument in Language Testing
- The Cambridge Applied Linguistics Series
- Validity Argument in Language Testing
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Series Editor’s Preface
- 1 Introduction to Validity Argument in Language Testing and Assessment
- Part I Basic Concepts and Uses of Validity Argument in Language Testing and Assessment
- Part II Investigating Score Interpretations
- 4 Domain Definition Inference for a Virtual Interactive Aviation English Test (VIAET) for Military Air Traffic Controllers
- 5 Support for the Evaluation Inference
- 6 Generalization Inference for a Computer-Mediated Graphic-Prompt Writing Test for ESL Placement
- 7 The Telephone Standard Speaking Test
- 8 The Explanation Inference for a Test of Academic English Collocational Ability
- 9 Extrapolation of the Meaning of Grades on Writing Tasks in an ESL Writing Class to Success on Tasks in University Courses
- Part III Investigating Score Uses and Consequences
- Part IV Conclusion
- Index
- References
Summary
This argument-based validation research investigates the validity of score interpretations on a computer-based, graphic-prompt writing test, focusing on the generalization inference. The graphic-prompt writing test assesses examinees’ ability to incorporate visual graphic information into their writing,. Both analytic ratings on Graph Description, Content Development, Organization, and Grammar/Vocabulary (n = 2,424) and composite ratings (n = 606) on written test responses from 101 ESL students were analyzed using Generalizability (G) Theory and Multi-Faceted Rasch Measurement (MFRM). Findings indicated three of the four analytic scales and the composites yielded dependable scores. In addition, the results of the G-studies and MFRM analysis revealed the relative effects of the raters on the total score variance was not trivial for both composite and analytic scores and the three raters were not quite equivalent in their rating severity. Nevertheless, the findings support the generalization inference to a large extent. Thus, it can be claimed the graphic-prompt writing task scores were dependable enough to be used for the intended purposes, particularly with the two-rater and three-task test administration design.
Keywords
- Type
- Chapter
- Information
- Validity Argument in Language TestingCase Studies of Validation Research, pp. 120 - 153Publisher: Cambridge University PressPrint publication year: 2021