Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-01-07T17:15:21.686Z Has data issue: false hasContentIssue false
  • English
  • Français

General Ability Measurement: An Application of Multidimensional Item Response Theory

Published online by Cambridge University Press:  01 January 2025

Daniel O. Segall
Daniel O. Segall*
Affiliation:
Defense Manpower Data Center
*
Requests for reprints should be sent to Daniel O. Segall, Defense Manpower Data Center, DoD Center Monterey Bay 400 Gigling Road, Seaside, CA 93955-6771. E-Mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Two new methods for improving the measurement precision of a general test factor are proposed and evaluated. One new method provides a multidimensional item response theory estimate obtained from conventional administrations of multiple-choice test items that span general and nuisance dimensions. The other method chooses items adaptively to maximize the precision of the general ability score. Both methods display substantial increases in precision over alternative item selection and scoring procedures. Results suggest that the use of these new testing methods may significantly enhance the prediction of learning and performance in instances where standardized tests are currently used.

Keywords

computerized adaptive testinggeneral cognitive abilitygeneral mental abilitymultidimensional adaptive testingmultidimensional item response theorypersonnel selectiontests
Type
Articles
Information
Psychometrika , Volume 66 , Issue 1 , March 2001 , pp. 79 - 97
Copyright
Copyright © 2001 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The author wishes to thank the three anonymous reviewers for their useful comments on an earlier version of this manuscript. The views expressed are those of the author and not necessarily those of the Department of Defense, or the United States government.

References

Albert, J.H. (1992). Bayesian estimation of normal ogive item response curves using Gibbs sampling. Journal of Educational Statistics, 17, 251–269.CrossRefGoogle Scholar
Anderson, T.W. (1984). An introduction to multivariate statistical analysis. New York: John Wiley & Sons.Google Scholar
Béguin, A.A., Glas, C.A.W. (1998). MCMC estimation of multidimensional IRT models. Twente, The Netherlands: University of Twente.Google Scholar
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In Lord, F. M., Novick, M. R. (Eds.), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
Bock, R.D., Gibbons, R., Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261–280.CrossRefGoogle Scholar
Cronbach, L.J. (1970). Essentials of psychological testing 3rd ed., Evanston, NY: Harper & Row.Google Scholar
de Boor, C. (1978). A practical guide to splines. New York: Springer-Verlag.CrossRefGoogle Scholar
Glas, C.A.W. (1999). Bayesian modification indices for IRT models. Twente, The Netherlands: University of Twente.Google Scholar
Green, B. F. (1983). The promise of tailored tests. In Wainer, H., Messick, S. (Eds.), Principals of modern psychological measurement: A festschrift for Frederic M. Lord (pp. 69–80). Hillsdale, NJ: Lawrence Erlbaun Associates.Google Scholar
Gulliksen, H. (1987). Theory of mental tests. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Harman, H.H. (1976). Modern factor analysis 3rd ed., Chicago: The University of Chicago Press.Google Scholar
Hattie, J. (1981). Decision criteria for determining unidimensionality. Canada: University of Toronto.Google Scholar
Holzinger, K.J. (1937). The bi-factor method. Psychometrika, 2, 41–54.CrossRefGoogle Scholar
Horst, P. (1966). Psychological measurement and prediction. Belmont, CA: Wadsworth Publishing.Google Scholar
Hulin, C.L., Lissak, R.I., Drasgow, F. (1982). Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study. Applied Psychological Measurement, 6, 249–260.CrossRefGoogle Scholar
Humphreys, L.G. (1981). The primary mental ability. In Friedman, M.P., Das, J.P., O'Connor, N. (Eds.), Intelligence and learning (pp. 87–102). New York: Plenum.CrossRefGoogle Scholar
Humphreys, L.G. (1985). General intelligence: An integration of factor, test, and simplex theory. In Wolman, B.B. (Eds.), Handbook of intelligence: Theories, measurements, and applications (pp. 201–224). New York: Wiley.Google Scholar
Humphreys, L.G. (1986). Describing the elephant. In Sternberg, R.J., Detterman, D.K. (Eds.), What is intelligence? (pp. 97–100). Norwood, NJ: Ablex.Google Scholar
Hunter, J.E., Schmidt, F.L., Judiesch, M.K. (1990). Individual differences in output variability as a function of job complexity. Journal of Applied Psychology, 75, 28–42.CrossRefGoogle Scholar
Jensen, A.R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.Google Scholar
Lohman, D.F. (1989). Human intelligence: An introduction to advances in theory and research. Review of Educational Research, 59, 333–373.CrossRefGoogle Scholar
Lord, F.M. (1952). A theory of test scores. Psychometrika Monograph No. 7.Google Scholar
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Lord, F.M., Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
McBride, J.R., Martin, J.T. (1983). Reliability and validity of adaptive ability tests in a military setting. In Weiss, D.J. (Eds.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 223–236). New York: Academic Press.Google Scholar
McDonald, R.P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Mislevy, R.J. (1994). Evidence and inference in educational assessment. Psychometrika, 59, 439–483.CrossRefGoogle Scholar
Palmer, P., Hartke, D.D., Ree, M.J., Welsh, J.R., Valentine, L.D. Jr. (1988). Armed Services Vocational Aptitude Battery (ASVAB): Alternative forms reliability (Forms 8, 9, 10 and 11). Brooks Air Force Base, TX: Air Force Human Resources Laboratory.Google Scholar
Reckase, M.D. (1997). A linear logistic multidimensional model for dichotomous item response data. In van der Linden, W.J., Hambleton, R. K. (Eds.), Handbook of modern item response theory (pp. 271–286). New York: Springer-Verlag.CrossRefGoogle Scholar
Ree, M.J., Earles, J.A. (1991). Predicting training success: Not much more thang. Personnel Psychology, 44, 321–332.CrossRefGoogle Scholar
Ree, M.J., Earles, J.A. (1991). The stability ofg across different methods of estimation. Intelligence, 15, 271–278.CrossRefGoogle Scholar
Ree, M.J., Earles, J.A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1, 86–89.CrossRefGoogle Scholar
Ree, M.J., Earles, J.A. (1994). Predicting job success: Not much more thang. Journal of Applied Psychology, 79, 518–524.CrossRefGoogle Scholar
Sands, W.A., Waters, B.K., McBride, J.R. (1997). Computerized adaptive testing: From inquiry to operation. Washington, DC: American Psychological Association.CrossRefGoogle Scholar
Schmidt, F.L., Hunter, J.E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.CrossRefGoogle Scholar
Segall, D.O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331–354.CrossRefGoogle Scholar
Segall, D.O. (1998). IFACT computer program Version 1.0: Full information confirmatory item factor analysis using Markov chain Monte Carlo estimation. Seaside, CA: Defense Manpower Data Center.Google Scholar
Segall, D.O. (2000). Principles of multidimensional adaptive testing. In van der Linden, W.J., Glas, C.A.W. (Eds.), Computerized adaptive testing: Theory and practice (pp. 53–73). Boston: Kluwer-Nijhoff.CrossRefGoogle Scholar
Spearman, C. (1904). General intelligence, objectively determined and measured. American Journal of Psychology, 15, 201–293.CrossRefGoogle Scholar
Swanson, L., Stocking, M.L. (1993). A model and heuristic for solving very large item selection problems. Applied Psychological Measurement, 17, 151–166.CrossRefGoogle Scholar
Thurstone, L.L. (1947). Multiple factor analysis. Chicago: The University of Chicago Press.Google Scholar
Urry, V.W. (1977). Tailored testing: A successful application of latent trait theory. Journal of Educational Measurement, 14, 181–196.CrossRefGoogle Scholar
van der Linden, W.J. (1996). Assembling tests for the measurement of multiple traits. Applied Psychological Measurement, 20, 373–388.CrossRefGoogle Scholar
van der Linden, W.J. (1999). Multidimensional adaptive testing with a minimum error-variance criterion. Journal of Educational and Behavioral Statistics, 24, 398–412.CrossRefGoogle Scholar
van der Linden, W.J., Hambleton, R.K. (1997). Handbook of modern item response theory. New York: Springer-Verlag.CrossRefGoogle Scholar
Wilks, S.S. (1938). Weighting systems for linear functions of correlated variables when there is no dependent variable. Psychometrika, 3, 23–40.CrossRefGoogle Scholar
Zimowski, M.F., Muraki, E., Mislevy, R.J., Bock, R.D. (1996). BILOG-MG: Multiple-group IRT analysis and test maintence for binary items. Chicago: Scientific Software.Google Scholar