We present an approach to quantifying errors in covariance structures in which adventitious error, identified as the process underlying the discrepancy between the population and the structured model, is explicitly modeled as a random effect with a distribution, and the dispersion parameter of this distribution to be estimated gives a measure of misspecification. Analytical properties of the resultant procedure are investigated and the measure of misspecification is found to be related to the root mean square error of approximation. An algorithm is developed for numerical implementation of the procedure. The consistency and asymptotic sampling distributions of the estimators are established under a new asymptotic paradigm and an assumption weaker than the standard Pitman drift assumption. Simulations validate the asymptotic sampling distributions and demonstrate the importance of accounting for the variations in the parameter estimates due to adventitious error. Two examples are also given as illustrations.

Higher-order approximations to the distributions of fit indexes for structural equation models under fixed alternative hypotheses are obtained in nonnormal samples as well as normal ones. The fit indexes include the normal-theory likelihood ratio chi-square statistic for a posited model, the corresponding statistic for the baseline model of uncorrelated observed variables, and various fit indexes as functions of these two statistics. The approximations are given by the Edgeworth expansions for the distributions of the fit indexes under arbitrary distributions. Numerical examples in normal and nonnormal samples with the asymptotic and simulated distributions of the fit indexes show the relative inappropriateness of the normal-theory approximation using noncentral chi-square distributions. A simulation for the confidence intervals of the fit indexes based on the normal-theory Studentized estimators under normality with a small sample size indicates an advantage for the approximation by the Cornish–Fisher expansion over those by the noncentral chi-square distribution and the asymptotic normality.

When a statistically significant mean difference is found, the magnitude of the difference is judged qualitatively using an effect size such as Cohen’s d. In contrast, in a structural equation model (SEM), the result of the statistical test of model fit is often disregarded if significant, and inferences are drawn using “close” models retained based on point estimates of sample statistics (goodness-of-fit indices). However, when a SEM cannot be retained using a test of exact fit, all substantive inferences drawn from it are suspect. It is therefore important to determine the size of the model misfit. Standardized residual covariances and residual correlations provide standardized effect sizes of the misfit of SEM models. They can be summarized using the Standardized Root Mean Squared Residual (SRMSR) and the Correlation Root Mean Squared Residual (CRMSR) which can be used as overall effect sizes of the misfit. Statistical theory is provided that allows the construction of confidence intervals and tests of close fit based on the SRMSR and CRMSR. It is hoped that the use of standardized effect sizes of misfit will help reconcile current practices in SEM and elsewhere in statistics.

In this rejoinder we discuss the following aspects of our approach to model discrepancy: the interpretations of the two populations and adventitious error, the choice of inverse Wishart distribution, the perceived danger of justifying a model with bad fit, the relationship among our new approach, Chen’s (J R Stat Soc Ser B, 41:235–248, 1979) approach and the existing RMSEA-based approach, and the Pitman drift assumption.