Factor Pattern of Test Items and Tests as a Function of the Correlation Coefficient: Content, Difficulty, and Constant Error Factors

Abstract

A dilemma was created for factor analysts by Ferguson (Psychometrika, 1941, 6, 323–329) when he demonstrated that test items or sub-tests of varying difficulty will yield a correlation matrix of rank greater than 1, even though the material from which the items or sub-tests are drawn is homogeneous, although homogeneity of such material had been defined operationally by factor analysts as having a correlation matrix of rank 1. This dilemma has been resolved as a case of ambiguity, which lay in (1) failure to specify whether homogeneity was to apply to content, difficulty, or both, and (2) failure to state explicitly the kind of correlation to be used in obtaining the matrix. It is demonstrated that (1) if the material is homogeneous in both respects, the type of coefficient is immaterial, but (2) if content is homogeneous but difficulty is not, the homogeneity of the content can be demonstrated only by using the tetra chorie correlation coefficient in deriving the matrix; and that the use of the phi-coefficient (Pearsonian r) will disclose only the nonhomogeneity of the difficulty and lead to a series of constant error factors as contrasted with content factors. Since varying difficulty of items (and possibly of sub-tests) is desirable as well as practically unavoidable, it is recommended that all factor analysis problems be carried out with tetrachoric correlations. While no one would want to obtain the constant error factors by factor analysis (difficulty being more easily obtained by counting passes), their importance for test construction is pointed out.