Experiments are considered where each of a sample of subjects is assigned to one of C categories separately by each of a fixed or varying group of observers. Building on earlier publications, general procedures are proposed to analyze agreements and disagreements among observers. In the case of a varying group of observers, it is shown that it is not necessary to demand a constant number of observers per subject. In the case of a fixed group of observers, the problem of missing data is considered.

The procedures are illustrated within the context of two clinical diagnosis examples. In the first example it is investigated which categories are relatively hard to distinguish from one another; a new theorem is applied that shows a useful property of the statistic kappa. In the second example it is investigated if a subgroup of observers can be found with a significantly higher degree of interobserver agreement.

A new subroutine has been developed for calculating the terachoric correlation coefficient. Recent advances in computing inverse normal and bivariate normal distributions have been utilized. The iterative procedure is started with an approximation with an error less than ±.0135.

A mathematical definition of the theoretical relation between the examinee's actual responses to the test items and his “true ability” is selected. A maximum-likelihood solution is obtained for estimating the examinee's “true ability” from his responses to the items. The standard error of the maximum-likelihood estimate is obtained, its relation to the discriminating power of the test is pointed out, and some generalizations are drawn as to the optimum level of item difficulty. The Neyman-Pearson power function is applied to determine which of two psychological tests is the most powerful for the selection of “successful” examinees.

The recent history of multidimensional data analysis suggests two distinct traditions that have developed along quite different lines. In multidimensional scaling (MDS), the available data typically describe the relationships among a set of objects in terms of similarity/dissimilarity (or (pseudo-)distances). In multivariate analysis (MVA), data usually result from observation on a collection of variables over a common set of objects. This paper starts from a very general multidimensional scaling task, defined on distances between objects derived from one or more sets of multivariate data. Particular special cases of the general problem, following familiar notions from MVA, will be discussed that encompass a variety of analysis techniques, including the possible use of optimal variable transformation. Throughout, it will be noted how certain data analysis approaches are equivalent to familiar MVA solutions when particular problem specifications are combined with particular distance approximations.

Current computer programs for analyzing linear structural models will apparently handle only two types of constraints: fixed parameters, and equality of parameters. An important constraint not handled is inequality; this is particularly crucial for preventing negative variance estimates. In this paper, a method is described for imposing several kinds of inequality constraints in models, without the necessity for having computer programs which explicitly allow such constraints. The examples discussed include the prevention of Heywood cases, extension to inequalities of parameters to be greater than a specified value, and imposing ordered inequalities.

In introducing the LISREL model for systems of linear structural equations, Jöreskog and Sörbom proposed two goodness-of-fit indices, GFI and AGFI. Their asymptotic distributions and some statistical properties are discussed.

Relationships between the Rasch model and both the law of comparative judgment and additive conjoint measurement are discussed. The distance between the ability of Person a and the difficult of Item i is, in the Rasch model, the baseline value corresponding to the probability that a will respond correctly to i, where this probability is interpreted as the area under a logistic curve (which is substantially equivalent to the normal curve) and is thus an application of the law of comparative judgment. Under certain assumptions, the Rasch model is also a special case of additive conjoint measurement and, properly reinterpreted, may be usefully applied in contexts other than individual differences.

This paper demonstrates the feasibility of using a Newton-Raphson algorithm to solve the likelihood equations which arise in maximum likelihood factor analysis. The algorithm leads to clean easily identifiable convergence and provides a means of verifying that the solution obtained is at least a local maximum of the likelihood function. It is shown that a popular iteration algorithm is numerically unstable under conditions which are encountered in practice and that, as a result, inaccurate solutions have been presented in the literature. The key result is a computationally feasible formula for the second differential of a partially maximized form of the likelihood function. In addition to implementing the Newton-Raphson algorithm, this formula provides a means for estimating the asymptotic variances and covariances of the maximum likelihood estimators.

One of the major concerns of reliability theory has been the estimation of the reliability of a composite measure from the degree of agreement among its component parts. In the classical theory, formulas were developed under the assumption that the parts are strictly equivalent. It was later shown that the same formulas follow from various sets of weaker assumptions which require the composites to be strictly equivalent and require the parts to have a certain homogeneity of statistical properties, but not necessarily to be equivalent. An alternative model which has received increasing attention in recent years regards a given measure as a random sample from a universe of measures whose homogeneity or equivalence is not specified a priori, and a composite test as a random sample of items from a universe of not-necessarily-equivalent items. This too permits an internal-consistency estimate of reliability. Both the equivalent-composites model and the randomsampling model appear to be unduly restrictive and unrealistic; we propose here to develop the implications of a third model in which a test is considered to have been formed by stratified sampling of items.

An alpha-O coefficient of internal consistency is defined for an observed score composite. Maximizing alpha-O leads to a system of psychometric (vs. statistical) factor analysis in which successive factors describe dimensions of successively less internal-consistency. Factoring stops when alpha-O is zero or less. In contrast to Kaiser-Caffrey's alpha-C analysis, when the factored matrix is rank 1, alpha-O does not reach unity; it can approach unity only as the number of variables reach infinity. The relative usefulness and domains of generalization of alpha-C and alpha-O are compared. Basically, alpha-C analysis is concerned with the representativeness of factors while alpha-O analysis is concerned with the assessibility of factors. Consequently, either system of factoring can and should be summarized by both the alpha-C and alpha-O coefficients. Not surprisingly, alpha-O analysis is computationally analogous to Rao's canonical factor analysis.

This paper discusses the properties of distributions of test scores and advances the view that the properties of the distribution should depend on the function which the test is intended to perform. A theory of test discrimination is developed which defines discriminatory capacity in terms of the number of relations of difference established by the operation of administering a test of k items to a sample of n individuals. A simple proof is presented which indicates that maximum discrimination between individuals is achieved when tests are constructed to yield distributions of the rectangular form. A coefficient of test discrimination is developed. The problem of obtaining in practice distributions approximating to the rectangular form is briefly discussed.

Proximity data can be represented by an extended tree, which generalizes traditional trees by including marked segments that correspond to overlapping clusters. An extended tree is a graphical representation of the distinctive features model. A computer program (EXTREE) that constructs extended trees is described and applied to several sets of conceptual and perceptual proximity data.

Using the theory of pseudo maximum likelihood estimation the asymptotic covariance matrix of maximum likelihood estimates for mean and covariance structure models is given for the case where the variables are not multivariate normal. This asymptotic covariance matrix is consistently estimated without the computation of the empirical fourth order moment matrix. Using quasi-maximum likelihood theory a Hausman misspecification test is developed. This test is sensitive to misspecification caused by errors that are correlated with the independent variables. This misspecification cannot be detected by the test statistics currently used in covariance structure analysis.

Monte Carlo procedures were used to investigate the properties of a nonmetric multidimensional scaling algorithm when used to scale an incomplete matrix of dissimilarities. Various recommendations for users who wish to scale incomplete matrices are made: (a) recovery was found to be satisfactory provided that the “degrees of freedom” ratio exceeded 3.5, irrespective of error level; (b) cyclic designs were found to provide best recovery, although random patterns of deletion performed almost as well; and (c) strongly locally connected designs, specifically overlapping cliques, were generally inferior. These conclusions are based on 837 scaling solutions and are applicable to stimulus sets containing more than 30 objects.

A simple and very general algorithm for oblique rotation is identified. While motivated by the rotation problem in factor analysis, it may be used to minimize almost any function of a not necessarily square matrix whose columns are restricted to have unit length. The algorithm has two steps. The first is to compute the gradient of the rotation criterion and the second is to project this onto a manifold of matrices with unit length columns. For this reason it is called a gradient projection algorithm. Because the projection step is very simple, implementation of the algorithm involves little more than computing the gradient of the rotation criterion which for many applications is very simple. It is proven that the algorithm is strictly monotone, that is as long as it is not already at a stationary point, each step will decrease the value of the criterion. Examples from a variety of areas are used to demonstrate the algorithm, including oblimin rotation, target rotation, simplimax rotation, and rotation to similarity and simplicity simultaneously. While it may be, the algorithm is not intended for use as a standard algorithm for well established problems, but rather as a tool for investigating new methods where its generality and simplicity may save an investigator substantial effort.

A general statistical model for simultaneous analysis of data from several groups is described. The model is primarily designed to be used for the analysis of covariance. The model can handle any number of covariates and criterion variables, and any number of treatment groups. Treatment effects may be assessed when the treatment groups are not randomized. In addition, the model allows for measurement errors in the criterion variables as well as in the covariates. A wide variety of hypotheses concerning the parameters of the model can be tested by means of a large sample likelihood ratio test. In particular, the usual assumptions of ANCOVA may be tested.

Millsap and Meredith (1988) have developed a generalization of principal components analysis for the simultaneous analysis of a number of variables observed in several populations or on several occasions. The algorithm they provide has some disadvantages. The present paper offers two alternating least squares algorithms for their method, suitable for small and large data sets, respectively. Lower and upper bounds are given for the loss function to be minimized in the Millsap and Meredith method. These can serve to indicate whether or not a global optimum for the simultaneous components analysis problem has been attained.

A scale-invariant index of factorial simplicity is proposed as a summary statistic for principal components and factor analysis. The index ranges from zero to one, and attains its maximum when all variables are simple rather than factorially complex. A factor scale-free oblique transformation method is developed to maximize the index. In addition, a new orthogonal rotation procedure is developed. These factor transformation methods are implemented using rapidly convergent computer programs. Observed results indicate that the procedures produce meaningfully simple factor pattern solutions.

Four types of metric scales are distinguished: the absolute scale, the ratio scale, the difference scale and the interval scale. A general coefficient of association for two variables of the same metric scale type is developed. Some properties of this general coefficient are discussed. It is shown that the matrix containing these coefficients between any number of variables is Gramian. The general coefficient reduces to specific coefficients of association for each of the four metric scales. Two of these coefficients are well known, the product-moment correlation and Tucker's congruence coefficient. Applications of the new coefficients are discussed.

In forced-choice questionnaires, respondents have to make choices between two or more items presented at the same time. Several IRT models have been developed to link respondent choices to underlying psychological attributes, including the recent MUPP (Stark et al. in Appl Psychol Meas 29:184–203, 2005) and Thurstonian IRT (Brown and Maydeu-Olivares in Educ Psychol Meas 71:460–502, 2011) models. In the present article, a common framework is proposed that describes forced-choice models along three axes: (1) the forced-choice format used; (2) the measurement model for the relationships between items and psychological attributes they measure; and (3) the decision model for choice behavior. Using the framework, fundamental properties of forced-choice measurement of individual differences are considered. It is shown that the scale origin for the attributes is generally identified in questionnaires using either unidimensional or multidimensional comparisons. Both dominance and ideal point models can be used to provide accurate forced-choice measurement; and the rules governing accurate person score estimation with these models are remarkably similar.