In Chapter 11, the author takes readers through the process of creating and using holistic and analytic rating scales for performance assessments. The author uses an example of an assessment that a team of language teachers created for measuring the abilities of young Saudi learners who were learning English as a second language in the United States. The author discusses multiple approaches to creating rating scales, including adapting existing scales, patterning scales after course standards, a theory-based approach where developers use language and assessment theory to create scales, and a performance-driven approach where developers use test taker language samples to identify distinguishing characteristics among test taker abilities. The author discusses three sample writing performances of the young Saudi learners and shows how the teachers used them to create the scales. The chapter concludes with a discussion of the process of creating and using computer scoring systems for performance assessments.

In Chapter 4, the author introduces the concept of validity. The chapter begins with an exploration of approaches to defining a construct. These approaches include using language theory, a language needs analysis, corpora, and curriculum objectives to help test developers determine what specific language ability they desire to measure. The chapter emphasizes the importance of alignment, which relates to how well the test content and test taker response processes match the construct’s content and the response processes that the test aims to measure. The author uses a detailed example of assessing children’s ability to communicate on a playground in a second language. The major point of the example is that the assessment should require children to use the same kinds of language they use when they communicate on the playground. This alignment helps ensure that the assessment measures the targeted language ability and will lead to positive washback on teaching and learning.