Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-01-07T17:10:58.777Z Has data issue: false hasContentIssue false
  • English
  • Français

Multidimensional Adaptive Testing

Published online by Cambridge University Press:  01 January 2025

Daniel O. Segall
Daniel O. Segall*
Affiliation:
Navy Personnel Research and Development Center
*
Requests for reprints should be sent to Daniel O. Segall, Defense Manpower Data Center, DoD Center Monterey Bay, 400 Gigling Road, Seaside CA 93955-6771.
Get access Rights & Permissions [Opens in a new window]

Abstract

Maximum likelihood and Bayesian procedures for item selection and scoring of multidimensional adaptive tests are presented. A demonstration using simulated response data illustrates that multidimensional adaptive testing (MAT) can provide equal or higher reliabilities with about one-third fewer items than are required by one-dimensional adaptive testing (OAT). Furthermore, holding test-length constant across the MAT and OAT approaches, substantial improvements in reliability can be obtained from multidimensional assessment. A number of issues relating to the operational use of multidimensional adaptive testing are discussed.

Keywords

computerized adaptive testingmathematical modelsmultidimensional adaptive testingmultidimensional item response theoryreliabilitytests
Type
Original Paper
Information
Psychometrika , Volume 61 , Issue 2 , June 1996 , pp. 331 - 354
Copyright
Copyright © 1996 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The work reported in this paper was sponsored by the Office of Naval Research. The author wishes to thank the three anonymous reviewers for their useful comments on an earlier version of this manuscript. The opinions expressed in this article are those of the Author, are not official and do not necessarily reflect the views of the Navy Department.

References

Ackerman, T. A. (1989). Unidimensional IRT calibration of compensatory and non-compensatory multidimensional items. Applied Psychological Measurement, 13, 113–127.CrossRefGoogle Scholar
Ackerman, T. A. (1991). The use of unidimensional parameter estimates of multidimensional items in adaptive testing. Applied Psychological Measurement, 15, 13–24.CrossRefGoogle Scholar
Anderson, T. W. (1984). An introduction to multivariate statistical analysis 2nd ed.,, New York: John Wiley & Sons.Google Scholar
Ansley, T. N., Forsyth, R. A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37–48.CrossRefGoogle Scholar
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In Lord, F. M., Novick, M. R. (Eds.), Statistical theories of mental test scores (pp. 397–479). Reading, MA: Addison-Wesley.Google Scholar
Bloxom, B., & Vale, C. D. (1987, June). Multidimensional adaptive testing: An approximate procedure for updating. Paper presented at the meeting of the Psychometric Society, Montreal.Google Scholar
Bock, R. D., Gibbons, R., Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261–280.CrossRefGoogle Scholar
Bollen, K. A. (1989). Structural equations with latent variables, New York: John Wiley & Sons.CrossRefGoogle Scholar
Carlson, J. E. (1987). Multidimensional item response theory estimation: A computer program, Iowa City, IA: The American College Testing Program.Google Scholar
Drasgow, F., Levine, M. V., McLaughlin, M. E. (1991). Appropriateness measurement for some multidimensional test batteries. Applied Psychological Measurement, 15, 171–191.CrossRefGoogle Scholar
Drasgow, F., Parsons, C. K. (1983). Application of unidimensional item response theory models to multidimensional data. Applied Psychological Measurement, 7, 189–199.CrossRefGoogle Scholar
Folk, V. G., Green, B. F. (1989). Adaptive estimation when the unidimensionality assumption of IRT is violated. Applied Psychological Measurement, 13, 373–389.CrossRefGoogle Scholar
Fraser, C. (1988). NOHARM II. A Fortran program for fitting unidimensional and multidimensional normal ogive models of latent trait theory, Armidale, Australia: The University of New England, Center for Behavioral Studies.Google Scholar
Harrison, D. A. (1986). Robustness of IRT parameter estimation to violations of the unidimensionality assumption. Journal of Educational Statistics, 11, 91–115.CrossRefGoogle Scholar
Hattie, J. (1981). Decision criteria for determining unidimensionality, Canada: University of Toronto.Google Scholar
Hendrickson, A. E., White, P. O. (1964). PROMAX: A quick method for rotation to oblique simple structure. British Journal of Mathematical and Statistical Psychology, 17, 65–70.CrossRefGoogle Scholar
IMSL (1991). International Mathematical and Statistical Libraries (Stat/Library), User's Manual, Houston, TX: Author.Google Scholar
Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187–200.CrossRefGoogle Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems, Hillsdale, NJ: Erlbaum.Google Scholar
McDonald, R. P. (1985). Unidimensional and multidimensional models for item response theory. In Weiss, D. J. (Eds.), Proceedings of the 1982 Computerized Adaptive Testing Conference (pp. 127–148). Minneapolis: University of Minnesota, Department of Psychology, Psychometrics Methods Program.Google Scholar
McKinley, R. L. (1989). Confirmatory analysis of test structure using multidimensional item response theory (Report No. RR-89-31), Princeton, NJ: Educational Testing Service.Google Scholar
McKinley, R. L., Reckase, M. D. (1983). MAXLOG: A computer program for the estimation of the parameters of a multidimensional logistic model. Behavior Research Methods & Instrumentation, 15, 389–390.CrossRefGoogle Scholar
Miller, T., Reckase, M. D., Spray, J. A., Luecht, R., & Davey, T. (in press). Multidimensional Item Response Theory.Google Scholar
Mislevy, R. J. (1984). Estimating latent distributions. Psychometrika, 49, 359–381.CrossRefGoogle Scholar
Mislevy, R. J. (1989). PC-BILOG: Item analysis and test scoring with binary logistic models, Mooresville, IN: Scientific Software.Google Scholar
Moreno, K. E., Segall, D. O. (1993). CAT-ASVAB Precision. Proceedings of the 34th Annual Conference of the Military Testing Association, San Diego: Navy Personnel Research and Development Center.Google Scholar
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115–132.CrossRefGoogle Scholar
Owen, R. J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351–356.CrossRefGoogle Scholar
Prestwood, J. S., Vale, C. D., Massey, R. H., Welsh, J. R. (1985). Armed Services Vocational Aptitude Battery: Development of an adaptive item pool, San Antonio, TX: Air Force Human Resources Laboratory.Google Scholar
Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4, 207–230.CrossRefGoogle Scholar
Reckase, M. D., Ackerman, T. A., Carlson, J. E. (1988). Building a unidimensional test using multidimensional items. Journal of Educational Measurement, 25, 193–203.CrossRefGoogle Scholar
Searle, S. R. (1982). Matrix algebra useful for statistics, New York: John Wiley & Sons.Google Scholar
Segall, D. O., Moreno, K. E., Hetter, R. D. (1987). ACAP item pools: Analysis and recommendations, San Diego: Navy Personnel Research and Development Center.Google Scholar
Sympson, J. B., & Hetter, R. D. (1985, October). Controlling item exposure rates in computerized adaptive tests. Paper presented at the 27th Annual meeting of the Military Testing Association, San Diego, CA.Google Scholar
Tam, S. S. (1992). A comparison of methods for adaptive estimation of a multidimensional trait. Unpublished doctoral dissertation, Columbia University.Google Scholar
Wainer, H. W., Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., Thissen, D. (1990). Computerized adaptive testing: A primer, Hillsdale, NJ: Erlbaum.Google Scholar
Way, W. D., Ansley, T. N., Forsyth, R. A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional IRT estimation. Applied Psychological Measurement, 12, 239–252.CrossRefGoogle Scholar
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.CrossRefGoogle Scholar