13 - Validation Research and Its Limits

Summary

This chapter underscores the implicit messages about argument-based validity expressed in the volume. It highlights characteristics of validation illuminated by how the validation research was designed, carried out, interpreted, and presented by each researcher. The chapters show that argument-based validity applies not only to large-scale or high-stakes testing, but is relevant to a range of contexts where assessments are trusted and is called on at varying stages of test development. Argument-based validity has sufficiently detailed concepts for guiding research about technology-assisted testing methods, and it provides terms for defining different types of constructs. Argument-based validity frames research goals that are well-suited to mixed-methods designs, as illustrated in the chapters of this volume. The chapter ends by clarifying the limits of argument-based validation research by reviewing the facts about validation: Validity is not a yes-no decision about a test; validity is not an objective, deterministically derived result; and validity is not the sole responsibility of the experts. Argument-based validity does not change these facts, but rather provides a detailed and logical means of working within these parameters despite the desire of many test users for tests that have been validated by experts and can be adopted uncritically.