Hostname: page-component-745bb68f8f-hvd4g Total loading time: 0 Render date: 2025-01-08T11:06:10.476Z Has data issue: false hasContentIssue false
  • English
  • Français

Adjusting for Information Inflation Due to Local Dependency in Moderately Large Item Clusters

Published online by Cambridge University Press:  02 January 2025

Edward Hak-sing IP
Edward Hak-sing IP*
Affiliation:
University of Southern California
*
Request for reprints should be directed to Edward Hak-sing Ip, Marshall School of Business, Information and Operations Management Department, University of Southern California, Los Angeles, CA 90089-1421.
Get access Rights & Permissions [Opens in a new window]

Abstract

When multiple items are clustered around a reading passage, the local independence assumption in item response theory is often violated. The amount of information contained in an item cluster is usually overestimated if violation of local independence is ignored and items are treated as locally independent when in fact they are not. In this article we provide a general method that adjusts for the inflation of information associated with a test containing item clusters. A computational scheme was presented for the evaluation of the factor of adjustment for clusters in the restrictive case of two items per cluster, and the general case of more than two items per cluster. The methodology was motivated by a study of the NAEP Reading Assessment. We present a simulated study along with an analysis of a NAEP data set.

Keywords

multivariate binary dataitem response theoryNAEPitem clusterconditional correlation function
Type
Original Paper
Information
Psychometrika , Volume 65 , Issue 1 , March 2000 , pp. 73 - 91
Copyright
Copyright © 2000 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The research was supported under the National Assessment of Educational Progress (Grant No. R999G30002) as administered by the Office of Educational Research and Improvement, U.S. Department of Education. This work was started when the author was at the Division of Statistics and Psychometrics at the Educational Testing Service. The author thanks Juliet Shaffer, Bob Mislevy, Eric Bradlow, three reviewers and an associate editor for their helpful comments on the paper.

References

Ackerman, T., & Spray, J. (1987). A general model for item dependency (Report No. RR-87-9). Iowa City, IA: ACTGoogle Scholar
Allen, N., Kline, D. L., & Zelenak, C. A. (1997). The NAEP 1994 Technical Report. Washington, D.C.: Department of EducationGoogle Scholar
Bahadur, R. (1961). A representation of the joint distribution of responses to n dichotomous items. In Solomon, H. (Eds.), Studies in item analysis and prediction (pp. 158–168). Palo Alto, CA: Stanford University PressGoogle Scholar
Becker, R. A., Chambers, J. M., & Wilks, A. R. (1988). The new S language. New York: Chapman & HallGoogle Scholar
Bishop, Y., Fienberg, S., & Holland, P. (1975). Discrete multivariate analysis. Boston, MA: MIT PressGoogle Scholar
Bradlow, E. (1996). Negative information and the three-parameter logistic model. Journal of Educational and Behavioral Statistics, 21, 179–185Google Scholar
Cox, D. (1972). The analysis of multivariate binary data. Applied Statistics, 21, 113–120CrossRefGoogle Scholar
Dale, R. (1986). Global cross-ratio models for bivariate, discrete ordered responses. Biometrics, 42, 909–917CrossRefGoogle ScholarPubMed
Diggle, P. J., Liang, K. L., & Zeger, S. L. (1994). Analysis of longitudinal data. New York: Oxford University PressGoogle Scholar
Efron, B., & Hinkley, D. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information. Biometrika, 65, 457–481CrossRefGoogle Scholar
Efron, B., & Tibshirani, R. (1993). An introduction to the bootstrap. New York: Chapman & HallCrossRefGoogle Scholar
Fahrmeir, L., & Tutz, G. (1994). Multivariate statistical modelling based on generalized linear models. New York: Springer-VerlagCrossRefGoogle Scholar
Fitzmaurice, G. M., Laird, N. M., & Rotnitsky, A. G. (1993). Regression model for discrete longitudinal responses (with discussion). Statistical Science, 8, 284–309Google Scholar
Greenacre, M. J. (1984). Theory and applications of correspondence analysis. New York: Academic PressGoogle Scholar
Holland, P., & Rosenbaum, P. (1986). Conditional association and unidimensionality in montone latent variable models. Annals of Statistics, 14, 1523–1543CrossRefGoogle Scholar
Jannarone, R. J. (1992). Local dependence: Objectively measurable or objectionably abominable?. In Wilson, M. (Eds.), Objective measurement: Theory into practice, Volume 2. Norwood, NJ: Ablex PublishingGoogle Scholar
Junker, B. W. (1991). Essential independence and likelihood-based ability estimation for polytomous items. Psychometrika, 56, 255–278CrossRefGoogle Scholar
Junker, B. W. (1993). Conditional association, essential independence and monotone unidimensional item response model. Annals of Statistics, 21, 1359–1378CrossRefGoogle Scholar
Liang, K. L., Zeger, S. L., & Qaqish, B. (1992). Multivariate regression analyses for categorical data (with discussion). Journal of the Royal Statistical Society, Series B, 54, 3–40CrossRefGoogle Scholar
Prentice, R. L. (1986). Binary regression using an extended beta-binomial distribution, with discussion of correlation induced by covariate measurement errors. Journal of American Statistical Association, 81, 321–327CrossRefGoogle Scholar
Prentice, R. L. (1988). Correlated binary regression with covariates specific to each binary observation. Biometrics, 44, 1033–1048CrossRefGoogle ScholarPubMed
Press, W., Teukolsky, S., Vetterling, W., & Flannery, B. (1992). Numerical recipes in C. Cambridge University Press.Google Scholar
Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237–247CrossRefGoogle Scholar
Stout, W. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 52, 589–617CrossRefGoogle Scholar
Stout, W. (1990). A new item response theory modeling approach with application to unidimensionality assessment and ability estimation. Psychometrika, 55, 293–325CrossRefGoogle Scholar
Stout, W., Habing, B., Douglas, J., Kim, H., Roussos, L., & Zhang, J. (1996). Conditional covariance based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331–354CrossRefGoogle Scholar
Stout, W. F., Nandakumar, R., Junker, B., Chang, H. H., & Steidinger, D. (1991). DIMTEST and TESTSIM [Computer program]. Champaign: University of Illinois, Department of StatisticsGoogle Scholar
Suppes, P., & Zanotti, M. (1981). When are probabilistic explanations possible?. Synthese, 48, 191–199CrossRefGoogle Scholar
Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example. Applied Measurement in Education, 8, 157–186CrossRefGoogle Scholar
Wainer, H., & Kiely, G.L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–201CrossRefGoogle Scholar
Wainer, H., Thissen, D. (1996). How reliable should a test be? What is the effect of local dependence on reliability?. Educational Measurement: Issues and Practice, 15, 22–29CrossRefGoogle Scholar
Woodruff, H., Ritter, G., Lowry, S., & Isenhour, T. (1975). Density estimations and the characterization of binary infrared spectra. Technometrics, 17, 455–462CrossRefGoogle Scholar
Wu, H. (1997). Some issues in item response theory. Urbana, IL: University of IllinoisGoogle Scholar
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145CrossRefGoogle Scholar
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187–213CrossRefGoogle Scholar
Zhao, L. P., & Prenctice, R. L. (1990). Correlated binary regression using a generalized quadratic model. Biometrika, 77, 642–648CrossRefGoogle Scholar
Zwick, R. (1987). Assessing the dimensionality of NAEP reading data. Journal of Educational Measurement, 24, 293–308CrossRefGoogle Scholar