Hostname: page-component-745bb68f8f-5r2nc Total loading time: 0 Render date: 2025-01-07T23:15:08.904Z Has data issue: false hasContentIssue false
  • English
  • Français

Uniform Test Assembly

Published online by Cambridge University Press:  01 January 2025

Dmitry I. Belov
Dmitry I. Belov*
Affiliation:
Law School Admission Council
*
Requests for reprints should be sent to Dmitry I. Belov, Psychometric Research, Law School Admission Council, 662 Penn Street, Newtown, PA 18940, USA. E-mail: [email protected]; [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

In educational practice, a test assembly problem is formulated as a system of inequalities induced by test specifications. Each solution to the system is a test, represented by a 0–1 vector, where each element corresponds to an item included (1) or not included (0) into the test. Therefore, the size of a 0–1 vector equals the number of items n in a given item pool. All solutions form a feasible set—a subset of 2n vertices of the unit cube in an n-dimensional vector space. Test assembly is uniform if each test from the feasible set has an equal probability of being assembled. This paper demonstrates several important applications of uniform test assembly for educational practice. Based on Slepian’s inequality, a binary program was analytically studied as a candidate for uniform test assembly. The results of this study establish a connection between combinatorial optimization and probability inequalities. They identify combinatorial properties of the feasible set that control the uniformity of the binary programming test assembly. Computer experiments illustrating the concepts of this paper are presented.

Keywords

combinatorial optimizationbinary programmingprobability inequalitiesSlepian’s inequalitytest assemblyitem pool analysis
Type
Theory and Methods
Information
Psychometrika , Volume 73 , Issue 1 , March 2008 , pp. 21 - 38
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Armstrong, R.D., Jones, D.H., & Kunce, C.S. (1998). IRT test assembly using network-flow programming. Applied Psychological Measurement, 22, 237–247.CrossRefGoogle Scholar
Belov, D.I. (2005). Inverse problem of item pool usability in computerized adaptive testing. Presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada, April.Google Scholar
Belov, D.I., & Armstrong, R.D. (2005). Monte Carlo test assembly for item pool analysis and extension. Applied Psychological Measurement, 29, 239–261.CrossRefGoogle Scholar
Belov, D.I., & Armstrong, R.D. (2005b). A Monte Carlo approach for evaluating and designing multi-stage adaptive tests. Presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada, April.Google Scholar
Belov, D.I., & Armstrong, R.D. (2006). A constraint programming approach to extract the maximum number of non-overlapping test forms. Computational Optimization and Applications, 33 2/3319–332.CrossRefGoogle Scholar
Belov, D.I., & Armstrong, R.D. (in press). A Monte Carlo approach to the design, assembly and evaluation of multi-stage adaptive tests. Applied Psychological Measurement.Google Scholar
Boekkooi-Timminga, E. (1990). The construction of parallel tests from IRT-based item banks. Journal of Educational Statistics, 15, 129–145.CrossRefGoogle Scholar
Garey, M.R., & Johnson, D.S. (1979). Computers and intractability: A guide to the theory of NP-completeness, New York: Freeman.Google Scholar
ILOG, Inc. (2003). CPLEX 9.0 [Computer program and manual], Mountain View: IL OS, Inc..Google Scholar
Lord, F.M. (1980). Applications of item response theory to practical testing problems, Hillsdale: Lawrence Erlbaum.Google Scholar
Luecht, R.M. (1998). Computer-assisted test assembly using optimization heuristics. Applied Psychological Measurement, 22, 224–236.CrossRefGoogle Scholar
Luecht, R.M. & Hirsch, T.M. (1992). Item selection using an average growth approximation of target information functions. Applied Psychological Measurement, 16, 41–51.CrossRefGoogle Scholar
Slepian, D. (1962). The one-sided barrier problem for Gaussian noise. Bell System Technical Journal, 41, 463–501.CrossRefGoogle Scholar
Theunissen, T.J.J.M. (1985). Binary programming and test design. Psychometrika, 50, 411–420.CrossRefGoogle Scholar
Tong, Y.L. (1980). Probability inequalities in multivariate distributions, New York: Academic Press.Google Scholar
Tong, Y.L. (1990). The multivariate normal distribution, New York: Springer.CrossRefGoogle Scholar
van der Linden, W.J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195–211.CrossRefGoogle Scholar
van der Linden, W.J. (2005). Linear models for optimal test design, New York: Springer.CrossRefGoogle Scholar
van der Linden, W.J. (2005b). Personal communication.Google Scholar
van der Linden, W.J., & Adema, J.J. (1998). Simultaneous assembly of multiple test forms. Journal of Educational Measurement, 35, 185–198.CrossRefGoogle Scholar
van der Linden, W.J., Ariel, A., & Veldkamp, B.P. (2006). Assembling a CAT item pool as a set of linear tests. Journal of Educational and Behavioral Statistics, 31(1), 81–99.CrossRefGoogle Scholar