A Note on Likelihood Ratio Tests for Models with Latent Variables

Abstract

The likelihood ratio test (LRT) is widely used for comparing the relative fit of nested latent variable models. Following Wilks’ theorem, the LRT is conducted by comparing the LRT statistic with its asymptotic distribution under the restricted model, a χ² distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models under comparison. For models with latent variables such as factor analysis, structural equation models and random effects models, however, it is often found that the χ² approximation does not hold. In this note, we show how the regularity conditions of Wilks’ theorem may be violated using three examples of models with latent variables. In addition, a more general theory for LRT is given that provides the correct asymptotic theory for these LRTs. This general theory was first established in Chernoff (J R Stat Soc Ser B (Methodol) 45:404–413, 1954) and discussed in both van der Vaart (Asymptotic statistics, Cambridge, Cambridge University Press, 2000) and Drton (Ann Stat 37:979–1012, 2009), but it does not seem to have received enough attention. We illustrate this general theory with the three examples.

1. Introduction

1.1. Literature on Likelihood Ratio Test

The likelihood ratio test (LRT) is one of the most popular methods for comparing nested models. When comparing two nested models that satisfy certain regularity conditions, the p-value of an LRT is obtained by comparing the LRT statistic with a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models. This reference distribution is suggested by the asymptotic theory of LRT that is known as Wilks’ theorem (Wilks 1938).

However, for the statistical inference of models with latent variables (e.g., factor analysis, item factor analysis for categorical data, structural equation models, random effects models, finite mixture models), it is often found that the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} approximation suggested by Wilks’ theorem does not hold. There are various published studies showing that the LRT is not valid under certain violations/conditions (e.g., small sample size, wrong model under the alternative hypothesis, large number of items, non-normally distributed variables, unique variances equal to zero, lack of identifiability), leading to over-factoring and over-rejections; see, e.g., Hakstian et al. (1982), Liu and Shao (2003), Hayashi et al. (2007), Asparouhov and Muthén (2009), Wu and Estabrook (2016), Deng et al. (2018), Shi et al. (2018), Yang et al. (2018) and Auerswald and Moshagen (2019). There is also a significant amount of the literature on the effect of testing at the boundary of parameter space that arise when testing the significance of variance components in random effects models as well as in structural equation models (SEM) with linear or nonlinear constraints (see Stram and Lee 1994, 1995; Dominicus et al. 2006; Savalei and Kolenikov 2008; Davis-Stober 2009; Wu and Neale 2013; Du and Wang 2020).

Theoretical investigations have shown that certain regularity conditions of Wilks’ theorem are not always satisfied when comparing nested models with latent variables. Takane et al. (2003) and Hayashi et al. (2007) were among the ones who pointed out that models for which one needs to select dimensionality (e.g., principal component analysis, latent class, factor models) have points of irregularity in their parameter space that in some cases invalidate the use of LRT. Specifically, such issues arise in factor analysis when comparing models with different number of factors rather than comparing a factor model against the saturated model. The LRT for comparing a q-factor model against the saturated model does follow a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution under mild conditions. However, for nested models with different number of factors (q-factor model is the correct one against the one with $(q+k)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(q+k)$$\end{document} factors), the LRT is likely not ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} -distributed due to violation of one or more of the regularity conditions. This is in line with the two basic assumptions required by the asymptotic theory for factor analysis and SEM: the identifiability of the parameter vector and non-singularity of the information matrix (see Shapiro 1986 and references therein). More specifically, Hayashi et al. (2007) focus on exploratory factor analysis and on the problem that arises when the number of factors exceeds the true number of factors that might lead to rank deficiency and non-identifiability of model parameters. That corresponds to the violations of the two regularity conditions. Those findings go back to Geweke and Singleton (1980) and Amemiya and Anderson (1990). More specifically, Geweke and Singleton (1980) studied the behavior of the LRT in small samples and concluded that when the regularity conditions from Wilks’ theorem are not satisfied the asymptotic theory seems to be misleading in all sample sizes considered.

1.2. Our Contributions

The contribution of this note is twofold. First, we provide a discussion about situations under which Wilks’ theorem for LRT may fail. Via three examples, we provide a relatively more complete picture about this issue in models with latent variables. Second, we introduce a unified asymptotic theory for LRT that covers Wilks’ theorem as a special case and provides the correct asymptotic reference distribution for LRT when Wilks’ theorem fails. This unified theory does not seem to have received enough attention in psychometrics, even though it has been established in statistics for long (Chernoff 1954; van der Vaart 2000; Drton 2009). In this note, we provide a tutorial on this theory, by presenting the theorems in a more accessible way and providing illustrative examples.

1.3. Examples

To further illustrate the issue with the classical theory for LRT, we provide three examples. These examples suggest that the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} approximation can perform poorly and give p-values that can be either more conservative or more liberal.

Example 1

(Exploratory factor analysis) Consider a dimensionality test in exploratory factor analysis (EFA). For ease of exposition, we consider two hypothesis testing problems: (a) testing a one-factor model against a two-factor model and (b) testing a one-factor model against a saturated multivariate normal model with an unrestricted covariance matrix. Similar examples have been considered in Hayashi et al. (2007) where similar phenomena have been studied.

1(a). Suppose that we have J mean-centered continuous indicators, $X={(X_1,...,X_J)}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {X}}= (X_1, ..., X_J)^\top $$\end{document} , which follow a J-variate normal distribution $N(0,\Sigma )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N({\mathbf {0}}, {\varvec{\Sigma }})$$\end{document} . The one-factor model parameterizes $\Sigma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Sigma }}$$\end{document} as

$\begin{array}{c}\Sigma =a_1{a}_1^{\top }+\Delta ,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {\varvec{\Sigma }} = {\mathbf {a}}_1{\mathbf {a}}_1^\top + {\varvec{\Delta }}, \end{aligned}$$\end{document}

where $a_1={(a_{11},...,a_{J1})}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {a}}_1 = (a_{11}, ..., a_{J1})^\top $$\end{document} contains the loading parameters and $\Delta =diag({\delta }_1,...,{\delta }_J)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Delta }} = diag(\delta _1, ..., \delta _J)$$\end{document} is diagonal matrix with a diagonal entries ${\delta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _1$$\end{document} , ..., ${\delta }_J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta _J$$\end{document} . Here, $\Delta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Delta }}$$\end{document} is the covariance matrix for the unique factors. Similarly, the two-factor model parameterizes $\Sigma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Sigma }}$$\end{document} as

$\begin{array}{c}\Sigma =a_1{a}_1^{\top }+a_2{a}_2^{\top }+\Delta ,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {\varvec{\Sigma }} = {\mathbf {a}}_1{\mathbf {a}}_1^\top + {\mathbf {a}}_2{\mathbf {a}}_2^\top + {\varvec{\Delta }}, \end{aligned}$$\end{document}

where $a_2={(a_{12},...,a_{J2})}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {a}}_2 = (a_{12}, ..., a_{J2})^\top $$\end{document} contains the loading parameters for the second factor and we set $a_{12}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{12} = 0$$\end{document} to ensure model identifiability. Obviously, the one-factor model is nested within the two-factor model. The comparison between these two models is equivalent to test

$\begin{array}{c}{H}_0:a_2=0\text{versus}{H}_a:a_2\ne 0.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} H_0: {\mathbf {a}}_2 = {\mathbf {0}} \text{ versus } H_a: {\mathbf {a}}_2 \ne {\mathbf {0}}. \end{aligned}$$\end{document}

If Wilks’ theorem holds, then under $H_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_0$$\end{document} the LRT statistic should asymptotically follow a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with $J-1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J-1$$\end{document} degrees of freedom.

Table 1. Values of the true parameters for the simulations in Example 1.

Figure 1. a Results of Example 1(a). The black solid line shows the empirical CDF of the LRT statistic, based on 5000 independent simulations. The red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with 5 degrees of freedom as suggested by Wilks’ theorem. The blue dotted line shows the CDF of the reference distribution suggested by Theorem 2. b Results of Example 1(b). The black solid line shows the empirical CDF of the LRT statistic, and the red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with 9 degrees of freedom as suggested by Wilks’ theorem

We now provide a simulated example. Data are generated from a one-factor model, with $J=6$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J = 6$$\end{document} indicators and $N=5000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N=5000$$\end{document} observations. The true parameter values are given in Table 1. We generate 5000 independent datasets. For each dataset, we compute the LRT for comparing the one- and two-factor models. Results are presented in panel (a) of Fig. 1. The black solid line shows the empirical cumulative distribution function (CDF) of the LRT statistic, and the red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution suggested by Wilks’ Theorem. A substantial discrepancy can be observed between the two CDFs. Specifically, the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} CDF tends to stochastically dominate the empirical CDF, implying that p-values based on this ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution tend to be more liberal. In fact, if we reject $H_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$H_0$$\end{document} at 5% significance level based on these p-values, the actual type I error is 10.8%. These results suggest the failure of Wilks’ theorem in this example.

1(b). When testing the one-factor model against the saturated model, the LRT statistic is asymptotically ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} if Wilks’ theorem holds. The degrees of freedom of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution are $J(J+1)/2-2J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J(J+1)/2 - 2J$$\end{document} , where $J(J+1)/2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J(J+1)/2$$\end{document} is the number of free parameters in an unrestricted covariance matrix $\Sigma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Sigma }}$$\end{document} and 2J is the number of parameters in the one-factor model. In panel (b) of Fig. 1, the black solid line shows the empirical CDF of the LRT statistic based on 5000 independent simulations, and the red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with 9 degrees of freedom. As we can see, the two curves almost overlap with each other, suggesting that Wilks’ theorem holds here.

Example 2

(Exploratory item factor analysis) We further give an example of exploratory item factor analysis (IFA) for binary data, in which similar phenomena as those in Example 1 are observed. Again, we consider two hypothesis testing problems: (a) testing a one-factor model against a two-factor model and (b) testing a one-factor model against a saturated multinomial model for a binary random vector.

2(a). Suppose that we have a J-dimensional response vector, $X={(X_1,...,X_J)}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {X}}= (X_1, ..., X_J)^\top $$\end{document} , where all the entries are binary-valued, i.e., $X_j\in \{0,1\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_j \in \{0, 1\}$$\end{document} . It follows a categorical distribution, satisfying

$\begin{array}{c}P(X=x)={\pi }_x,x\in {\{0,1\}}^J,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P({\mathbf {X}}= {\mathbf {x}}) = \pi _{{\mathbf {x}}}, {\mathbf {x}}\in \{0,1\}^J, \end{aligned}$$\end{document}

where ${\pi }_x\ge 0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi _{\mathbf {x}}\ge 0$$\end{document} and ${\sum }_{x\in {\{0,1\}}^J}{\pi }_x=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sum _{{\mathbf {x}}\in \{0, 1\}^J} \pi _{{\mathbf {x}}} = 1$$\end{document} .

The exploratory two-factor IFA model parameterizes ${\pi }_x$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi _{\mathbf {x}}$$\end{document} by

$\begin{array}{c}{\pi }_x=\int \int \prod _{j=1}^J\frac{exp\left(x_j(d_j+a_{j1}{\xi }_1+a_{j2}{\xi }_2)\right)}{1+exp(d_j+a_{j1}{\xi }_1+a_{j2}{\xi }_2)}\varphi \left({\xi }_1\right)\varphi \left({\xi }_2\right)d{\xi }_{1}d{\xi }_2,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \pi _{{\mathbf {x}}} = \int \int \prod _{j=1}^J \frac{\exp (x_j(d_j + a_{j1}\xi _1 + a_{j2}\xi _2))}{1+\exp (d_j + a_{j1}\xi _1 + a_{j2}\xi _2)} \phi (\xi _1)\phi (\xi _2)d\xi _1d\xi _2, \end{aligned}$$\end{document}

where $\varphi (·)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\phi (\cdot )$$\end{document} is the probability density function of a standard normal distribution. This model is also known as a multidimensional two-parameter logistic (M2PL) model (Reckase 2009). Here, $a_{\mathrm{jk}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{jk}$$\end{document} s are known as the discrimination parameters and $d_j$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_j$$\end{document} s are known as the easiness parameters. We denote $a_1={(a_{11},...,a_{J1})}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {a}}_1 = (a_{11},...,a_{J1})^\top $$\end{document} and $a_2={(a_{12},...,a_{J2})}^{\top }.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {a}}_2 = (a_{12},...,a_{J2})^\top .$$\end{document} For model identifiability, we set $a_{12}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{12} = 0$$\end{document} . When $a_{j2}=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a_{j2} = 0$$\end{document} , $j=2,...,J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j=2, ..., J$$\end{document} , then the two-factor model degenerates to the one-factor model. Similar to Example 1(a), if Wilks’ theorem holds, the LRT statistic should asymptotically follow a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with $J-1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J-1$$\end{document} degrees of freedom.

Simulation results suggest the failure of this ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} approximation. In Fig. 2, we provide plots similar to those in Fig. 1, based on 5000 datasets simulated from a one-factor IFA model with sample size $N=5000$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N = 5000$$\end{document} and $J=6$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J = 6$$\end{document} . The true parameters of this IFA model are given in Table 2. The result is shown in panel (a) of Fig. 2, where a similar pattern is observed as that in panel (a) of Fig. 1 for Example 1(a).

Table 2. Values of the true parameters for the simulations in Example 2.

Figure 2. a Results of Example 2(a). The black solid line shows the empirical CDF of the LRT statistic, based on 5000 independent simulations. The red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with 5 degrees of freedom as suggested by Wilks’ theorem. The blue dotted line shows the CDF of the reference distribution suggested by Theorem 2. b Results of Example 2(b). The black solid line shows the empirical CDF of the LRT statistic, and the red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with 51 degrees of freedom as suggested by Wilks’ theorem

2(b). When testing the one-factor IFA model against the saturated model, the LRT statistic is asymptotically ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} if Wilks’ theorem holds, for which the degree of freedom is $2^J-1-2J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2^J-1 - 2J$$\end{document} . Here, $2^J-1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2^J-1$$\end{document} is the number of free parameters in the saturated model, and 2J is the number of parameters in the one-factor IFA model. The result is given in panel (b) of Fig. 2. Similar to Example 1(b), the empirical CDF and the CDF implied by Wilks’ theorem are very close to each other, suggesting that Wilks’ theorem holds here.

Example 3

(Random effects model) Our third example considers a random intercept model. Consider two-level data with individuals at level 1 nested within groups at level 2. Let $X_{\mathrm{ij}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$X_{ij}$$\end{document} be data from the jth individual from the ith group, where $i=1,...,N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i = 1, ..., N$$\end{document} and $j=1,...,J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j = 1, ..., J$$\end{document} . For simplicity, we assume all the groups have the same number of individuals. Assume the following random effects model,

$\begin{array}{c}{X}_{\mathrm{ij}}={\beta }_0+{\mu }_i+{ϵ}_{\mathrm{ij}},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} X_{ij} = \beta _0 + \mu _i + \epsilon _{ij}, \end{aligned}$$\end{document}

where ${\beta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _0$$\end{document} is the overall mean across all the groups, ${\mu }_i\sim N(0,{\sigma }_1^2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu _i \sim N(0, \sigma _1^2)$$\end{document} characterizes the difference between the mean for group i and the overall mean, and ${ϵ}_{\mathrm{ij}}\sim N(0,{\sigma }_2^2)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon _{ij} \sim N(0, \sigma _2^2)$$\end{document} is the individual level residual.

To test for between-group variability under this model is equivalent to test

$\begin{array}{c}{H}_0:{\sigma }_1^2=0\text{against}{H}_a:{\sigma }_1^2>0.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} H_0: \sigma _1^2 = 0 \text{ against } H_a: \sigma _1^2 > 0. \end{aligned}$$\end{document}

If Wilks’ theorem holds, then the LRT statistic should follow a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with one degree of freedom. We conduct a simulation study and show the results in Fig. 3. In this figure, the black solid line shows the empirical CDF of the LRT statistic, based on 5000 independent simulations from the null model with $N=200$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N = 200$$\end{document} , $J=20$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J= 20$$\end{document} , ${\beta }_0=0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _0 = 0$$\end{document} , and ${\sigma }_2^2=1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma ^2_2 = 1$$\end{document} . The red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with one degree of freedom. As we can see, the two CDFs are not close to each other, and the empirical CDF tends to stochastically dominate the theoretical CDF suggested by Wilks’ theorem. It suggests the failure of Wilks’ theorem in this example.

This kind of phenomenon has been observed when the null model lies on the boundary of the parameter space, due to which the regularity conditions of Wilks’ theorem do not hold. The LRT statistic has been shown to often follow a mixture of ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution asymptotically (e.g., Shapiro 1985; Self and Liang 1987), instead of a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution. As it will be shown in Sect. 2, such a mixture of ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution can be derived from a general theory for LRT.

Figure 3. The black solid line shows the empirical CDF of the LRT statistic, based on 5000 independent simulations. The red dashed line shows the CDF of the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with one degree of freedom as suggested by Wilks’ theorem. The blue dotted line shows the CDF of the mixture of ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution suggested by Theorem 2 (Color figure online)

We now explain why Wilks’ theorem does not hold in Examples 1(a), 2(a), and 3. We define some generic notations. Suppose that we have i.i.d. observations $X_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {X}}_1$$\end{document} , ..., $X_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {X}}_N$$\end{document} , from a parametric model $P_{\Theta }=\{P_{\theta }:\theta \in \Theta \subset R^k\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal {P}}_{\Theta } = \{P_{{\varvec{\theta }}}: {\varvec{\theta }} \in \Theta \subset {\mathbb {R}}^k\}$$\end{document} , where $X_i={(X_{i1},...,X_{\mathrm{iJ}})}^{\top }.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {X}}_i = (X_{i1}, ..., X_{iJ})^\top .$$\end{document} We assume that the distributions in $P_{\Theta }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal P_{\Theta }$$\end{document} are dominated by a common $\sigma$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma $$\end{document} -finite measure $\nu$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\nu $$\end{document} with respect to which they have probability density functions $p_{\theta }:R^J\to [0,\infty )$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{{\varvec{\theta }}}: {\mathbb {R}}^J \rightarrow [0,\infty )$$\end{document} . Let ${\Theta }_0\subset \Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0 \subset \Theta $$\end{document} be a submodel and we are interested in testing

$\begin{array}{c}{H}_0:\theta \in {\Theta }_0\text{versus}{H}_a:\theta \in \Theta \backslash {\Theta }_0.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} H_0: {\varvec{\theta }}\in \Theta _0 \text{ versus } H_a: {\varvec{\theta }}\in \Theta {\setminus } \Theta _0. \end{aligned}$$\end{document}

Let $p_{{\theta }^{\ast }}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{{\varvec{\theta }}^* }$$\end{document} be the true model for the observations, where ${\theta }^{\ast }\in {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* \in \Theta _0$$\end{document} .

The likelihood function is defined as

$\begin{array}{c}{l}_N\left(\theta \right)=\sum _{i=1}^{N}log{p}_{\theta }\left(X_i\right),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} l_N({\varvec{\theta }}) = \sum _{i=1}^N \log p_{{\varvec{\theta }}}({\mathbf {X}}_i), \end{aligned}$$\end{document}

and the LRT statistic is defined as

$\begin{array}{c}{\lambda }_N=2\left(\underset{\theta \in \Theta }{sup},l_N,\left(\theta \right),-,\underset{\theta \in {\Theta }_0}{sup},l_N,\left(\theta \right)\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \lambda _N = 2\left( \sup _{{\varvec{\theta }}\in \Theta } l_N({\varvec{\theta }}) - \sup _{{\varvec{\theta }}\in \Theta _0} l_N({\varvec{\theta }})\right) . \end{aligned}$$\end{document}

Under suitable regularity conditions, Wilks’ theorem suggests that the LRT statistic ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} is asymptotically ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} .

Wilks’ theorem for LRT requires several regularity conditions; see, e.g., Theorem 12.4.2, Lehmann and Romano (2006). Among these conditions, there are two conditions that the previous examples do not satisfy. First, it is required that ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} is an interior point of $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} . This condition is not satisfied for Example 3, when $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} is taken to be $\{({\beta }_0,{\sigma }_1^2,{\sigma }_2^2):{\beta }_0\in R,{\sigma }_1^2\in [0,\infty ),{\sigma }_2^2\in [0,\infty )\}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{(\beta _0, \sigma _1^2, \sigma _2^2): \beta _0 \in {\mathbb {R}}, \sigma _1^2 \in [0, \infty ), \sigma _2^2 \in [0, \infty )\}$$\end{document} , as the null model lies on the boundary of the parameter space. Second, it is required that the expected Fisher information matrix at ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} , $I\left({\theta }^{\ast }\right)=E_{{\theta }^{\ast }}\left[\nabla l_N\left({\theta }^{\ast }\right)\nabla l_N{\left({\theta }^{\ast }\right)}^{\top }\right]/N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I({\varvec{\theta }}^*) = E_{{\varvec{\theta }}^*}[\nabla l_{N}({\varvec{\theta }}^*)\nabla l_{N}({\varvec{\theta }}^*)^\top ]/N$$\end{document} is strictly positive definite. As we summarize in Lemma 1, this condition is not satisfied in Examples 1(a) and 2(a), when $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} is taken to be the parameter space of the corresponding two-factor model. However, interestingly, when comparing the one-factor model with the saturated model, the Fisher information matrix is strictly positive definite in Examples 1(b) and 2(b), for both simulated examples.

Lemma 1

1. (1) For the two-factor model given in Example 1(a), choose the parameter space to be

   $\begin{array}{c}\Theta =\left({({\delta }_1,...,{\delta }_J,a_{11},...,a_{J1},a_{22},...,a_{J2})}^{\top },\in ,R^{3J-1},:,{\delta }_j,>,0,,,,j,=,1,,,.,.,.,,,J\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta = \left\{ (\delta _1,...,\delta _J, a_{11},...,a_{J1},a_{22},...,a_{J2})^\top \in {\mathbb {R}}^{3J-1}:\delta _j >0, ~j=1,...,J \right\} . \end{aligned}$$\end{document}
   If the true parameters satisfy $a_{j2}^{\ast }=0,j=2,...,J,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a^*_{j2}=0, ~j=2,...,J,$$\end{document} then $I\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I({\varvec{\theta }}^*)$$\end{document} is non-invertible.

2. (2) For the two-factor IFA model given in Example 2(a), choose the parameter space to be $\Theta =R^{3J-1}.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ \Theta = {\mathbb {R}}^{3J-1}. $$\end{document} If the true parameters satisfy $a_{j2}^{\ast }=0,j=2,...,J,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a^*_{j2}=0, ~j=2,...,J,$$\end{document} then $I\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I({\varvec{\theta }}^*)$$\end{document} is non-invertible.

We remark on the consequences of having a non-invertible information matrix. The first consequence is computational. If the information matrix is non-invertible, then the likelihood function does not tend to be strongly convex near the MLE, resulting in slow convergence. In the context of Examples 1(a) and 2(a), it means that computing the MLE for the corresponding two-factor models may have convergence issue. When convergence issue occurs, the obtained LRT statistic is below its actual value, due to the log likelihood for the two-factor model not achieving the maximum. Consequently, the p-value tends to be larger than its actual value, and thus, the decision based on the p-value tends to be more conservative than the one without convergence issue. This convergence issue is observed when conducting simulations for these examples. To improve the convergence, we use multiple random starting points when computing MLEs. The second consequence is a poor asymptotic convergence rate for the MLE. That is, the convergence rate is typically much slower than the standard parametric rate $N^{-1/2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N^{-1/2}$$\end{document} , even though the MLE is still consistent; see Rotnitzky et al. (2000) for more theoretical results on this topic.

We further provide some remarks on the LRT in Examples 1(b) and 2(b) that use a LRT for comparing the fitted model with the saturated model. Although Wilks’ theorem holds asymptotically in example 2(b), the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} approximation may not always work well as in our simulated example. This is because, when the number of items becomes larger and the sample size is not large enough, the contingency table for all $2^J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2^J$$\end{document} response patterns may be sparse, and thus, the saturated model cannot be accurately estimated. In that case, it is better to use a limited information inference method (e.g., Maydeu-Olivares and Joe 2005, 2006) as a goodness-of-fit test statistic. Similar issues might also occur to Example 1(b).

2. General Theory for Likelihood Ratio Test

The previous discussions suggest that Wilks’ theorem does not hold for Examples 1(a), 2(a), and 3, due to the violation of regularity conditions. It is then natural to ask: What asymptotic distribution does ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} follow in these situations? Is there asymptotic theory characterizing such irregular situations? The answer to these questions is “yes.” In fact, a general theory characterizing these less regular situations has already been established in Chernoff (1954). In what follows, we provide a version of this general theory that is proved in van der Vaart (2000), Theorem 16.7. It is also given in Drton (2009), Theorem 2.6. Two problems will be considered, (1) comparing a submodel with the saturated model as in Examples 1(b) and 2(b), and (2) comparing two submodels as in Examples 1(a), 2(a), and 3.

2.1. Testing Submodel Against Saturated Model

We first introduce a few notations. We use $R_{\mathrm{pd}}^{J\times J}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^{J\times J}_{pd}$$\end{document} and $R_d^{J\times J}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^{J\times J}_d$$\end{document} to denote the spaces of $J\times J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J\times J$$\end{document} strictly positive definite matrices and diagonal matrices, respectively. In addition, we define a one-to-one mapping $\rho$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho $$\end{document} : $R_{\mathrm{pd}}^{J\times J}\mapsto R^{J(J+1)/2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathcal {\mathbb {R}}_{pd}^{J\times J} \mapsto {\mathbb {R}}^{J(J+1)/2}$$\end{document} that maps a positive definite matrix to a vector containing all its upper triangular entries (including the diagonal entries). That is, $\rho \left(\Sigma \right)={({\sigma }_{11},{\sigma }_{12}...,{\sigma }_{1J},{\sigma }_{22},...,{\sigma }_{2J},...,{\sigma }_{\mathrm{JJ}})}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho (\Sigma ) = (\sigma _{11}, \sigma _{12}..., \sigma _{1J}, \sigma _{22}, ..., \sigma _{2J}, ..., \sigma _{JJ})^\top $$\end{document} , for $\Sigma ={\left({\sigma }_{\mathrm{ij}}\right)}_{J\times J}\in R_{\mathrm{pd}}^{J\times J}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Sigma = (\sigma _{ij})_{J\times J} \in \mathcal {\mathbb {R}}_{pd}^{J\times J}$$\end{document} . We also define a one-to-one mapping $\mu$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu $$\end{document} : $R_d^{J\times J}\mapsto R^J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}_{d}^{J\times J} \mapsto {\mathbb {R}}^{J}$$\end{document} that maps a diagonal matrix to a vector containing all its diagonal entries.

We consider to compare a submodel versus the saturated model. Let ${\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0$$\end{document} and $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} be the parameter spaces of the submodel and the saturated model, respectively, satisfying ${\Theta }_0\subset \Theta \subset R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0 \subset \Theta \subset {\mathbb {R}}^k$$\end{document} . Also let ${\theta }^{\ast }\in {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* \in \Theta _0$$\end{document} be the true parameter vector. The asymptotic theory of the LRT for comparing ${\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0$$\end{document} versus $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} requires regularity conditions C1-C5.

1. C1. The true parameter vector ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} is in the interior of $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} .

2. C2. There exists a measurable map ${\dot{l}}_{\theta }:R^J\to R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\dot{l}_{{\varvec{\theta }}}:{\mathbb {R}}^J \rightarrow {\mathbb {R}}^k$$\end{document} such that

   (1) $\begin{array}{c}\underset{h\to 0}{lim}\frac{1}{{\Vert h\Vert }^2}{\int }_{R^J}{\left(\sqrt{p_{\theta +h}\left(x\right)},-,\sqrt{p_{\theta }\left(x\right)},-,\frac{1}{2},h^{\top },{\dot{l}}_{\theta },\left(x\right),\sqrt{p_{\theta }\left(x\right)}\right)}^{2}d\nu \left(x\right)=0,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \lim _{{\mathbf {h}}\rightarrow \mathbf{0}} \frac{1}{\Vert {\mathbf {h}}\Vert ^2} \int _{{\mathbb {R}}^J} \left( \sqrt{p_{{\varvec{\theta }}+{\mathbf {h}}}({\mathbf {x}})} - \sqrt{p_{{\varvec{\theta }}}({\mathbf {x}})} - \frac{1}{2}{\mathbf {h}}^\top \dot{l}_{{\varvec{\theta }}}({\mathbf {x}})\sqrt{p_{{\varvec{\theta }}}({\mathbf {x}})} \right) ^2 d\nu ({\mathbf {x}}) = 0, \end{aligned}$$\end{document}
   and the Fisher information matrix $I\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I({\varvec{\theta }}^*)$$\end{document} for $P_{\Theta }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal {P}}_{\Theta }$$\end{document} is invertible.

3. C3. There exists a neighborhood of ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} , $U_{{\theta }^{\ast }}\subset \Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$U_{{\varvec{\theta }}^*} \subset \Theta $$\end{document} , and a measurable function $\dot{l}:R^J\to R$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\dot{l}: {\mathbb {R}}^J \rightarrow {\mathbb {R}}$$\end{document} , square integrable as ${\int }_{R^J}\dot{l}{\left(x\right)}^{2}d{P}_{{\theta }^{\ast }}\left(x\right)<\infty ,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\int _{{\mathbb {R}}^J}\dot{l}({\mathbf {x}})^2 dP_{{\varvec{\theta }}^*}({\mathbf {x}}) < \infty ,$$\end{document} such that

   $\begin{array}{c}|log{p}_{{\theta }_1}\left(x\right)-log{p}_{{\theta }_2}\left(x\right)|\le \dot{l}\left(x\right)\Vert {\theta }_1-{\theta }_2\Vert ,\forall {\theta }_1,{\theta }_2\in U_{{\theta }^{\ast }}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \vert \log p_{{\varvec{\theta }}_1}({\mathbf {x}}) - \log p_{{\varvec{\theta }}_2}({\mathbf {x}}) \vert \le \dot{l}({\mathbf {x}}) \Vert {\varvec{\theta }}_1 - {\varvec{\theta }}_2 \Vert , \quad \forall {\varvec{\theta }}_1,{\varvec{\theta }}_2 \in U_{{\varvec{\theta }}^*}. \end{aligned}$$\end{document}

4. C4. The maximum likelihood estimators (MLEs)

   $\begin{array}{c}{\hat{\theta }}_{N,\Theta }=\underset{\theta \in \Theta }{\arg \max }{l}_N\left(\theta \right)\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {{\hat{{\varvec{\theta }}}}}_{N, \Theta } = \mathop {\mathrm{arg}\, \mathrm{max}}\limits _{{\varvec{\theta }}\in \Theta } l_N({\varvec{\theta }}) \end{aligned}$$\end{document}
   and
   $\begin{array}{c}{\hat{\theta }}_{N,{\Theta }_0}=\underset{\theta \in {\Theta }_0}{\arg \max }{l}_N\left(\theta \right)\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {{\hat{{\varvec{\theta }}}}}_{N, \Theta _0} = \mathop {\mathrm{arg}\, \mathrm{max}}\limits _{{\varvec{\theta }}\in \Theta _0} l_N({\varvec{\theta }}) \end{aligned}$$\end{document}
   are consistent under $P_{{\theta }^{\ast }}.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{{\varvec{\theta }}^*}.$$\end{document}

The asymptotic distribution of ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} depends on the local geometry of the parameter space ${\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0$$\end{document} at ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} . This is characterized by the tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} , to be defined below.

Definition 1

The tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} of the set ${\Theta }_0\subset R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0 \subset {\mathbb {R}}^k$$\end{document} at the point ${\theta }^{\ast }\in R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* \in {\mathbb {R}}^k$$\end{document} is the set of vectors in $R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^k$$\end{document} that are limits of sequences ${\alpha }_n({\theta }_n-{\theta }^{\ast }),$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _n({\varvec{\theta }}_n - {\varvec{\theta }}^*),$$\end{document} where ${\alpha }_n$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha _n$$\end{document} are positive reals and ${\theta }_n\in {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}_n \in \Theta _0$$\end{document} converge to ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} .

The following regularity is required for the tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} that is known as the Chernoff regularity.

1. C5. For every vector $\tau$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\tau }$$\end{document} in the tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} , there exist $ϵ>0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon >0$$\end{document} and a map $\alpha :[0,ϵ)\to {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\alpha }}:[0,\epsilon )\rightarrow \Theta _0$$\end{document} with $\alpha \left(0\right)={\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\alpha }}(0) = {\varvec{\theta }}^*$$\end{document} such that $\tau ={lim}_{t\to 0+}[\alpha \left(t\right)-\alpha \left(0\right)]/t.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\tau }= \lim _{t\rightarrow 0+}[{\varvec{\alpha }}(t) - {\varvec{\alpha }}(0)]/t.$$\end{document}

Under the above regularity conditions, Theorem 1 holds and explains the phenomena in Examples 1(b) and 2(b).

Theorem 1

Suppose that conditions C1-C5 are satisfied for comparing nested models ${\Theta }_0\subset \Theta \subset R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0 \subset \Theta \subset {\mathbb {R}}^k$$\end{document} , with ${\theta }^{\ast }\in {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* \in \Theta _0$$\end{document} being the true parameter vector. Then as N grows to infinity, the likelihood ratio statistic ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to the distribution of

(2) $\begin{array}{c}\underset{\tau \in T_{{\Theta }_0}\left({\theta }^{\ast }\right)}{min}{\Vert Z-I{\left({\theta }^{\ast }\right)}^{\frac{1}{2}}\tau \Vert }^2,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \min _{{\varvec{\tau }} \in T_{\Theta _0}({\varvec{\theta }}^*)} \Vert {\mathbf {Z}} - I({\varvec{\theta }}^*)^{\frac{1}{2}}{\varvec{\tau }} \Vert ^2, \end{aligned}$$\end{document}

where $Z={(Z_1,...,Z_k)}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {Z}} = (Z_1, ..., Z_k)^\top $$\end{document} is a random vector consisting of i.i.d. standard normal random variables.

Remark 1

We give some remarks on the regularity conditions. Conditions C1-C4 together ensure the asymptotic normality for $\sqrt{N}({\hat{\theta }}_{N,\Theta }-{\theta }^{\ast })$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sqrt{N}({\hat{{\varvec{\theta }}}}_{N,\Theta }-{\varvec{\theta }}^*)$$\end{document} . Condition C1 depends on both the true model and the saturated model. As will be shown below, this condition holds for the saturated models in Examples 1(b) and 2(b). Equation (1) in C2 is also known as the condition of “differentiable in quadratic mean” for $P_{\Theta }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal {P}}_{\Theta }$$\end{document} at ${\theta }^{\ast }.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*.$$\end{document} If the map $\theta \mapsto \sqrt{p_{\theta }\left(x\right)}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}\mapsto \sqrt{p_{{\varvec{\theta }}}({\mathbf {x}})}$$\end{document} is continuously differentiable for every $x,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {x}},$$\end{document} then C2 holds with ${\dot{l}}_{\theta }\left(x\right)=\frac{\partial }{\partial \theta }log{p}_{\theta }\left(x\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\dot{l}_{{\varvec{\theta }}}({\mathbf {x}}) = \frac{\partial }{\partial {\varvec{\theta }}}\log p_{{\varvec{\theta }}}({\mathbf {x}})$$\end{document} (Lemma 7.6, van der Vaart (2000)). Furthermore, C3 holds if $\dot{l}\left(x\right)={sup}_{\theta \in U_{{\theta }^{\ast }}}{\dot{l}}_{\theta }\left(x\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\dot{l}({\mathbf {x}}) = \sup _{{\varvec{\theta }}\in U_{{\varvec{\theta }}^*}}\dot{l}_{{\varvec{\theta }}}({\mathbf {x}})$$\end{document} is square integrable with respect to the measure $P_{{\theta }^{\ast }}.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{{\varvec{\theta }}^*}.$$\end{document} Specifically, if $\dot{l}\left(x\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\dot{l}({\mathbf {x}})$$\end{document} is a bounded function, then C3 holds. C4 holds for our examples by Theorem 10.1.6, Casella and Berger (2002). C5 requires certain regularity on the local geometry of $T_{{\Theta }_0}\left({\theta }^{\ast }\right),$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*),$$\end{document} which also holds for our examples below.

Remark 2

By Theorem 1, the asymptotic distribution for ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} depends on the tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right).$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*).$$\end{document} If $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} is a linear subspace of $R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^k$$\end{document} with dimension $k_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k_0$$\end{document} , then one can easily show that the asymptotic reference distribution of ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} is ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} with degrees of freedom $k-k_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k-k_0$$\end{document} . As we explain below, Theorem 1 directly applies to Examples 1(b) and 2(b). If $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} is a convex cone, then ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to a mixture of ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution (Shapiro 1985; Self and Liang 1987). That is, for any $x>0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$x > 0$$\end{document} , $P({\lambda }_N\le x)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P(\lambda _N \le x)$$\end{document} converges to ${\sum }_{i=0}^k{w}_{k}P({\xi }_i\le x)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sum _{i=0}^k w_k P(\xi _i \le x)$$\end{document} , as N goes to infinity, where ${\xi }_0\equiv 0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\xi _0 \equiv 0$$\end{document} and ${\xi }_i$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\xi _i$$\end{document} follows a ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with i degrees of freedom for $i>0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i > 0$$\end{document} . Moreover, the weights sum up to 1/2 for the components with even degrees of freedom, and so do the weights for the components with odd degrees of freedom (Shapiro 1985).

Example 4

(Exploratory factor analysis, revisited) Now we consider Example 1(b). As the saturated model is a J-variate normal distribution with an unrestricted covariance matrix, its parameter space can be chosen as

$\begin{array}{c}\Theta =\{\rho \left(\Sigma \right):\Sigma \in R_{\mathrm{pd}}^{J\times J}\}\subset R^{J(J+1)/2},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta = \{ \rho ({\varvec{\Sigma }}) : {\varvec{\Sigma }} \in {\mathbb {R}}_{pd}^{J\times J} \} \subset {\mathbb {R}}^{J(J+1)/2}, \end{aligned}$$\end{document}

and the parameter space for the restricted model is

$\begin{array}{c}{\Theta }_0=\left(\rho ,\left(\Sigma \right),:,\Sigma ,=,a_1,a_1^{\top },+,\Delta ,,,,a_1,\in ,R^J,,,\Delta ,\in ,R_{\mathrm{pd}}^{J\times J},\cap ,R_d^{J\times J}\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta _0 = \left\{ \rho ({\varvec{\Sigma }}): {\varvec{\Sigma }} = {\mathbf {a}}_1{\mathbf {a}}_1^\top + {\varvec{\Delta }},~ {\mathbf {a}}_1 \in {\mathbb {R}}^J, {\varvec{\Delta }} \in {\mathbb {R}}_{pd}^{J\times J} \cap {\mathbb {R}}_{d}^{J\times J} \right\} . \end{aligned}$$\end{document}

Suppose ${\theta }^{\ast }=\rho \left({\Sigma }^{\ast }\right)\in {\Theta }_0,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* = \rho ({\varvec{\Sigma }}^*) \in \Theta _0,$$\end{document} where ${\Sigma }^{\ast }=a_1^{\ast }{a_1^{\ast }}^{\top }+{\Delta }^{\ast }.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Sigma }}^* = {{\mathbf {a}}_1^*}{{\mathbf {a}}_1^*}^\top + {\varvec{\Delta }}^*.$$\end{document} It is easy to see that C1 holds with the current choice of $\Theta .$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta .$$\end{document} The tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} takes the form:

$\begin{array}{c}{T}_{{\Theta }_0}\left({\theta }^{\ast }\right)=\left(\rho ,\left(\Sigma \right),:,\Sigma ,=,a_1^{\ast },b_1^{\top },+,b_1,{a_1^{\ast }}^{\top },+,B,,,,b_1,\in ,R^J,,,B,\in ,R_d^{J\times J}\right),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} T_{\Theta _0}({\varvec{\theta }}^*) = \left\{ \rho ({\varvec{\Sigma }}): {\varvec{\Sigma }} = {{\mathbf {a}}_1^*}{{\mathbf {b}}}_1^\top + {{\mathbf {b}}}_1{{\mathbf {a}}_1^*}^\top + {\mathbf {B}}, ~ {{\mathbf {b}}}_1 \in {\mathbb {R}}^{J}, {\mathbf {B}} \in {\mathbb {R}}_{d}^{J\times J} \right\} , \end{aligned}$$\end{document}

which is a linear subspace of $R^{J(J+1)/2}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^{J(J+1)/2}$$\end{document} with dimension 2J, as long as $a_{j1}^{\ast }\ne 0,j=1,...,J.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$a^*_{j1} \ne 0, ~j=1,...,J.$$\end{document} By Theorem 1, ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution with degrees of freedom $J(J+1)/2-2J.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$J(J+1)/2 - 2J.$$\end{document}

Example 5

(Exploratory item factor analysis, revisited) Now we consider Example 2(b). As the saturated model is a $2^J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2^J$$\end{document} -dimensional categorical distribution, its parameter space can be chosen as

$\begin{array}{c}\Theta =\left(\theta ,=,{\left\{{\theta }_x\right\}}_{x\in {\Gamma }_J},:,{\theta }_x,\ge ,0,,,\sum _{x\in {\Gamma }_J},{\theta }_x,\le ,1\right)\subset R^{2^J-1},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta = \left\{ {\varvec{\theta }}= \{\theta _{{\mathbf {x}}}\}_{{\mathbf {x}}\in \Gamma _J}: \theta _{{\mathbf {x}}} \ge 0, \sum _{{\mathbf {x}}\in \Gamma _J}\theta _{{\mathbf {x}}} \le 1 \right\} \subset {\mathbb {R}}^{2^J - 1}, \end{aligned}$$\end{document}

where ${\Gamma }_J:={\{0,1\}}^J\backslash \left\{{(0,...,0)}^{\top }\right\}.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Gamma _J := \{0,1\}^J \backslash \{(0,...,0)^\top \}.$$\end{document} Then, the parameter space for the restricted model is

(3) $\begin{array}{c}\begin{array}{c}{\Theta }_0=\left(\theta ,\in ,\Theta ,:,{\theta }_x,=,\int ,\prod _{j=1}^J,\frac{exp\left(x_j(d_j+a_{j1}{\xi }_1)\right)}{1+exp(d_j+a_{j1}{\xi }_1)},\varphi ,\left({\xi }_1\right),d,{\xi }_1,,,,a_1,,,d,\in ,R^J\right).\end{array}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \begin{aligned} \Theta _0 = \left\{ {\varvec{\theta }}\in \Theta : \theta _{{\mathbf {x}}} = \int \prod _{j=1}^J \frac{\exp (x_{j}(d_j + a_{j1}\xi _1))}{1+\exp (d_j + a_{j1}\xi _1)} \phi (\xi _1)d\xi _1, ~ {\mathbf {a}}_1,{{\mathbf {d}}}\in {\mathbb {R}}^J \right\} . \end{aligned} \end{aligned}$$\end{document}

Let ${\theta }^{\ast }\in {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* \in \Theta _0$$\end{document} that corresponds to true item parameters $a_1^{\ast }={(a_{j1}^{\ast },...,a_{J1}^{\ast })}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {a}}_1^* = (a^*_{j1},...,a^*_{J1})^\top $$\end{document} and $d^{\ast }={(d_1^{\ast },...,d_J^{\ast })}^{\top }.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathbf {d}}}^* = (d^*_{1},...,d^*_{J})^\top .$$\end{document} By the form of ${\Theta }_0,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0,$$\end{document} ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} is an interior point of $\Theta .$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta .$$\end{document}

For any $x\in {\Gamma }_J,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {x}}\in \Gamma _J,$$\end{document} we define $f_x={(f_1\left(x\right),...,f_J\left(x\right))}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {f}}_{\mathbf {x}}= (f_{1}({\mathbf {x}}),...,f_{J}({\mathbf {x}}))^\top $$\end{document} and $g_x={(g_1\left(x\right),...,g_J\left(x\right))}^{\top },$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {g}}_{{\mathbf {x}}} = (g_{1}({\mathbf {x}}),...,g_{J}({\mathbf {x}}))^\top ,$$\end{document} where

$\begin{array}{c}{f}_l\left(x\right)=\int \prod _{j=1}^J\frac{exp\left(x_j(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)\right)}{1+exp(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)}\left(x_l,-,\frac{exp(d_l^{\ast }+a_{l1}^{\ast }{\xi }_1)}{1+exp(d_l^{\ast }+a_{l1}^{\ast }{\xi }_1)}\right)\varphi \left({\xi }_1\right)d{\xi }_1,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} f_{l}({\mathbf {x}}) = \int \prod _{j=1}^J \frac{\exp (x_{j}(d^*_{j} + a^*_{j1}\xi _1))}{1+\exp (d^*_{j} + a^*_{j1}\xi _1)} \left[ x_l - \frac{\exp (d^*_{l} + a^*_{l1}\xi _1)}{1+\exp (d^*_{l} + a^*_{l1}\xi _1)} \right] \phi (\xi _1) d\xi _1, \end{aligned}$$\end{document}

and

$\begin{array}{c}{g}_l\left(x\right)=\int \prod _{j=1}^J\frac{exp\left(x_j(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)\right)}{1+exp(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)}\left(x_l,-,\frac{exp(d_l^{\ast }+a_{l1}^{\ast }{\xi }_1)}{1+exp(d_l^{\ast }+a_{l1}^{\ast }{\xi }_1)}\right){\xi }_1\varphi \left({\xi }_1\right)d{\xi }_1,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} g_{l}({\mathbf {x}}) = \int \prod _{j=1}^J \frac{\exp (x_{j}(d^*_{j} + a^*_{j1}\xi _1))}{1+\exp (d^*_{j} + a^*_{j1}\xi _1)} \left[ x_l - \frac{\exp (d^*_{l} + a^*_{l1}\xi _1)}{1+\exp (d^*_{l} + a^*_{l1}\xi _1)} \right] \xi _1\phi (\xi _1) d\xi _1, \end{aligned}$$\end{document}

for $l=1,...,J.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l=1,...,J.$$\end{document} Then the tangent cone $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} has the form

$\begin{array}{c}{T}_{{\Theta }_0}\left({\theta }^{\ast }\right)=\left(\theta ,=,{\left\{{\theta }_x\right\}}_{x\in {\Gamma }_J},:,{\theta }_x,=,b_0^{\top },f_x,+,b_1^{\top },g_x,,,,b_0,,,b_1,\in ,R^J\right),\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} T_{\Theta _0}({\varvec{\theta }}^*) = \left\{ {\varvec{\theta }}= \{\theta _{{\mathbf {x}}}\}_{{\mathbf {x}}\in \Gamma _J}: \theta _{{\mathbf {x}}} = {{\mathbf {b}}}_0^\top {\mathbf {f}}_{{\mathbf {x}}} + {{\mathbf {b}}}_1^\top {\mathbf {g}}_{{\mathbf {x}}},~{{\mathbf {b}}}_0,{{\mathbf {b}}}_1\in {\mathbb {R}}^J \right\} , \end{aligned}$$\end{document}

which is a linear subspace of $R^{2^J-1}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbb {R}}^{2^J-1}$$\end{document} with dimension 2J. By Theorem 1, ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to the distribution of ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} with degrees of freedom $2^J-1-2J.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2^{J}-1 - 2J.$$\end{document}

2.2. Comparing Two Nested Submodels

Theorem 1 is not applicable to Example 3, because ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} is on the boundary of $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} if $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} is chosen to be $\{({\beta }_0,{\sigma }_1^2,{\sigma }_2^2):{\beta }_0\in R,{\sigma }_1^2\in [0,\infty ),{\sigma }_2^2\in [0,\infty )\},$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{(\beta _0, \sigma _1^2, \sigma _2^2): \beta _0 \in {\mathbb {R}}, \sigma _1^2 \in [0, \infty ), \sigma _2^2 \in [0, \infty )\},$$\end{document} and thus, C1 is violated. Theorem 1 is also not applicable to Examples 1(a) and 2(a), because the Fisher information matrix is not invertible when $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} is chosen to be the parameter space of the two-factor EFA and IFA models, respectively, in which case condition C2 is violated.

To derive the asymptotic theory for such problems, we view them as a problem of testing nested submodels under a saturated model for which ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} is an interior point of $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} and the information matrix is invertible. Consider testing

$\begin{array}{c}{H}_0:\theta \in {\Theta }_0\text{versus}{H}_a:\theta \in {\Theta }_1\backslash {\Theta }_0,\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} H_0: {\varvec{\theta }}\in \Theta _0 \text{ versus } H_a: {\varvec{\theta }}\in \Theta _1{\setminus } \Theta _0, \end{aligned}$$\end{document}

where ${\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0$$\end{document} and ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} are two nested submodels of a saturated model $\Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta $$\end{document} , satisfying ${\Theta }_0\subset {\Theta }_1\subset \Theta \subset R^k$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0 \subset \Theta _1 \subset \Theta \subset {\mathbb {R}}^{k}$$\end{document} . Under this formulation, Theorem 2 provides the asymptotic theory for the LRT statistic ${\lambda }_N=2\left({sup}_{\theta \in {\Theta }_1},l_N,\left(\theta \right),-,{sup}_{\theta \in {\Theta }_0},l_N,\left(\theta \right)\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N = 2\left( \sup _{{\varvec{\theta }}\in \Theta _1} l_N({\varvec{\theta }}) - \sup _{{\varvec{\theta }}\in \Theta _0} l_N({\varvec{\theta }})\right) $$\end{document} .

To obtain the asymptotic distribution of ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} , regularity conditions C1-C5 are still required for ${\Theta }_0\subset \Theta$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0 \subset \Theta $$\end{document} . Two additional conditions are needed for ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} , which are satisfied for Examples 6, 7 and 8.

1. C6. The MLE under ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} , ${\hat{\theta }}_{N,{\Theta }_1}=\underset{\theta \in {\Theta }_1}{\arg \max }{l}_N\left(\theta \right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\hat{{\varvec{\theta }}}}}_{N, \Theta _1} = \mathop {\mathrm{arg}\, \mathrm{max}}\limits _{{\varvec{\theta }}\in \Theta _1} l_N({\varvec{\theta }})$$\end{document} , is consistent under $P_{{\theta }^{\ast }}.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_{{\varvec{\theta }}^*}.$$\end{document}

2. C7. Let $T_{{\Theta }_1}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*)$$\end{document} be the tangent cone for ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} , defined the same as in Definition 1, but with ${\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0$$\end{document} replaced by ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} . $T_{{\Theta }_1}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*)$$\end{document} satisfies Chernoff regularity. That is, for every vector $\tau$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\tau }$$\end{document} in the tangent cone $T_{{\Theta }_1}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*)$$\end{document} there exist $ϵ>0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon >0$$\end{document} and a map $\alpha :[0,ϵ)\to {\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\alpha }}:[0,\epsilon )\rightarrow \Theta _1$$\end{document} with $\alpha \left(0\right)={\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\alpha }}(0) = {\varvec{\theta }}^*$$\end{document} such that $\tau ={lim}_{t\to 0+}[\alpha \left(t\right)-\alpha \left(0\right)]/t.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varvec{\tau }= \lim _{t\rightarrow 0+}[{\varvec{\alpha }}(t) - {\varvec{\alpha }}(0)]/t.$$\end{document}

Theorem 2

Let ${\theta }^{\ast }\in {\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* \in \Theta _0$$\end{document} be the true parameter vector. Suppose that conditions C1-C7 are satisfied. As N grows to infinity, the likelihood ratio statistic ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to the distribution of

(4) $\begin{array}{c}\begin{array}{c}\underset{\tau \in T_{{\Theta }_0}\left({\theta }^{\ast }\right)}{min}\Vert Z-I{\left({\theta }^{\ast }\right)}^{\frac{1}{2}}{\tau \Vert }^2-\underset{\tau \in T_{{\Theta }_1}\left({\theta }^{\ast }\right)}{min}{\Vert Z-I{\left({\theta }^{\ast }\right)}^{\frac{1}{2}}\tau \Vert }^2,\end{array}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \begin{aligned} \min _{{\varvec{\tau }} \in T_{\Theta _0}({\varvec{\theta }}^*)} \Vert {\mathbf {Z}} - I({\varvec{\theta }}^*)^{\frac{1}{2}}{\varvec{\tau }} \Vert ^2 - \min _{{\varvec{\tau }} \in T_{\Theta _1}({\varvec{\theta }}^*)} \Vert {\mathbf {Z}} - I({\varvec{\theta }}^*)^{\frac{1}{2}}{\varvec{\tau }} \Vert ^2, \end{aligned} \end{aligned}$$\end{document}

where $Z={(Z_1,...,Z_k)}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {Z}} = (Z_1, ..., Z_k)^\top $$\end{document} is a random vector consisting of i.i.d. standard normal random variables, and $I{\left({\theta }^{\ast }\right)}^{\frac{1}{2}}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I({\varvec{\theta }}^*)^{\frac{1}{2}}$$\end{document} satisfies $I{\left({\theta }^{\ast }\right)}^{\frac{1}{2}}{\left(I{\left({\theta }^{\ast }\right)}^{\frac{1}{2}}\right)}^{\top }=I\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$I({\varvec{\theta }}^*)^{\frac{1}{2}} (I({\varvec{\theta }}^*)^{\frac{1}{2}})^\top = I({\varvec{\theta }}^*)$$\end{document} that can be obtained by eigenvalue decomposition.

Example 6

(Random effects model, revisited) Now we consider Example 3. Let $1_n$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathbf {1}}}_n$$\end{document} denote a length-n vector whose entries are all 1 and $I_n$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathbf {I}}}_n$$\end{document} denote the $n\times n$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n\times n$$\end{document} identity matrix. As $X_i={(X_{i1},...,X_{\mathrm{iJ}})}^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {X}}_i = (X_{i1},...,X_{iJ})^\top $$\end{document} from the random effects model is multivariate normal with mean ${\beta }_0{1}_J$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\beta _0{{\mathbf {1}}}_J$$\end{document} and covariance matrix ${\sigma }_1^2{1}_J{1}_J^{\top }+{\sigma }_2^2{I}_J,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sigma _1^2{{\mathbf {1}}}_J{{\mathbf {1}}}_J^\top + \sigma _2^2{{\mathbf {I}}}_J,$$\end{document} the saturated parameter space can be taken as

$\begin{array}{c}\Theta =\{{(\rho {\left(\Sigma \right)}^{\top },{\beta }_0)}^{\top }:\Sigma \in R_{\mathrm{pd}}^{J\times J},{\beta }_0\in R\}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta = \{ (\rho ({\varvec{\Sigma }})^\top ,\beta _0)^\top : {\varvec{\Sigma }} \in {\mathbb {R}}^{J\times J}_{pd}, \beta _0\in {\mathbb {R}}\}. \end{aligned}$$\end{document}

The parameter space for restricted models are

$\begin{array}{c}{\Theta }_0=\{{(\rho {\left(\Sigma \right)}^{\top },{\beta }_0)}^{\top }:\Sigma ={\sigma }_2^2{I}_J,{\sigma }_2^2>0,{\beta }_0\in R\},\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta _0 = \{ (\rho ({\varvec{\Sigma }})^\top ,\beta _0)^\top :{\varvec{\Sigma }} = \sigma _2^2{{\mathbf {I}}}_J,~ \sigma _2^2 >0, \beta _0\in {\mathbb {R}}\}, \end{aligned}$$\end{document}

and

$\begin{array}{c}{\Theta }_1=\{{(\rho {\left(\Sigma \right)}^{\top },{\beta }_0)}^{\top }:\Sigma ={\sigma }_1^2{1}_J{1}_J^{\top }+{\sigma }_2^2{I}_J,{\sigma }_1^2\ge 0,{\sigma }_2^2>0,{\beta }_0\in R\}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta _1 = \{ (\rho ({\varvec{\Sigma }})^\top ,\beta _0)^\top :{\varvec{\Sigma }} = \sigma _1^2{{\mathbf {1}}}_J{{\mathbf {1}}}_J^\top + \sigma _2^2{{\mathbf {I}}}_J, \sigma _1^2\ge 0, \sigma _2^2>0, \beta _0\in {\mathbb {R}}\}. \end{aligned}$$\end{document}

Let ${\theta }^{\ast }=(\rho \left({\Sigma }^{\ast }\right),{\beta }_0^{\ast })\in {\Theta }_0,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^* = (\rho ({\varvec{\Sigma }}^*),\beta ^*_0) \in \Theta _0,$$\end{document} where ${\Sigma }^{\ast }={{\sigma }_2^{\ast }}^2{I}_J.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\Sigma }}^* = {\sigma _2^*}^2{{\mathbf {I}}}_J.$$\end{document} Then, C1 holds. The tangent cones for ${\Theta }_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _0$$\end{document} and ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} are

$\begin{array}{c}{T}_{{\Theta }_0}\left({\theta }^{\ast }\right)=\{{(\rho {\left(\Sigma \right)}^{\top },b_0)}^{\top }:\Sigma =b_2{I}_J,b_0,b_2\in R\}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} T_{\Theta _0}({\varvec{\theta }}^*) = \{ (\rho ({\varvec{\Sigma }})^\top ,b_0)^\top :{\varvec{\Sigma }} = b_2{{\mathbf {I}}}_J,~ b_0,b_2\in {\mathbb {R}}\} \end{aligned}$$\end{document}

and

$\begin{array}{c}{T}_{{\Theta }_1}\left({\theta }^{\ast }\right)=\{{(\rho {\left(\Sigma \right)}^{\top },b_0)}^{\top }:\Sigma =b_1{1}_J{1}_J^{\top }+b_2{I}_J,b_1\ge 0,b_0,b_2\in R\}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} T_{\Theta _1}({\varvec{\theta }}^*) = \{ (\rho ({\varvec{\Sigma }})^\top ,b_0)^\top :{\varvec{\Sigma }} = b_1{{\mathbf {1}}}_J{{\mathbf {1}}}_J^\top + b_2{{\mathbf {I}}}_J,~ b_1\ge 0, b_0,b_2\in {\mathbb {R}}\}. \end{aligned}$$\end{document}

By Theorem 2, ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to the distribution of (4).

In this example, the form of (4) can be simplified, thanks to the forms of $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} and $T_{{\Theta }_1}\left({\theta }^{\ast }\right).$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*).$$\end{document} We denote

$\begin{array}{c}{c}_0=(0,...,0,1),c_1={(\rho {\left(1_J{1}_J^{\top }\right)}^{\top },0)}^{\top },c_2={(\rho {\left(I_J\right)}^{\top },0)}^{\top }\in R^{J(J+1)/2+1}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {\mathbf {c}}_0 = (0,...,0,1), ~{\mathbf {c}}_1 = (\rho ({{\mathbf {1}}}_J{{\mathbf {1}}}_J^\top )^\top ,0)^\top ,~{\mathbf {c}}_2 = (\rho ({{\mathbf {I}}}_J)^\top ,0)^\top \in {\mathbb {R}}^{J(J+1)/2+1}. \end{aligned}$$\end{document}

It can be seen that $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} is a two-dimensional linear subspace spanned by $\{c_0,c_2\},$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{{\mathbf {c}}_0,{\mathbf {c}}_2\},$$\end{document} and $T_{{\Theta }_1}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*)$$\end{document} is a half three-dimensional linear subspace defined as $\{{\alpha }_0{c}_0+{\alpha }_1{c}_1+{\alpha }_2{c}_2:{\alpha }_1\ge 0,{\alpha }_0,{\alpha }_2\in R\}.$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\{\alpha _0{\mathbf {c}}_0+\alpha _1{\mathbf {c}}_1+\alpha _2{\mathbf {c}}_2: \alpha _1\ge 0,\alpha _0,\alpha _2\in {\mathbb {R}}\}.$$\end{document} Let $P_0$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {P}}_0$$\end{document} denote the projection onto $T_{{\Theta }_0}\left({\theta }^{\ast }\right).$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*).$$\end{document} Define

$\begin{array}{c}v=\frac{c_1-P_0{c}_1}{\Vert c_1-P_0{c}_1\Vert },\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {\mathbf {v}}= \frac{{\mathbf {c}}_1 - {\mathbf {P}}_0{\mathbf {c}}_1}{\Vert {\mathbf {c}}_1 - {\mathbf {P}}_0{\mathbf {c}}_1\Vert }, \end{aligned}$$\end{document}

and then, (4) has the form

(5) $\begin{array}{c}\Vert v^{\top }{Z\Vert }^2{1}_{\{v^{\top }Z\ge 0\}}.\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Vert {\mathbf {v}}^\top {\mathbf {Z}}\Vert ^21_{\{{\mathbf {v}}^\top {\mathbf {Z}}\ge 0\}}. \end{aligned}$$\end{document}

It is easy to see that $v^{\top }Z$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {v}}^\top {\mathbf {Z}}$$\end{document} follows standard normal distribution. Therefore, ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} converges to the distribution of $w^2{1}_{\{w\ge 0\}},$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w^21_{\{w\ge 0\}},$$\end{document} where w is a standard normal random variable. This is known as a mixture of ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution. The blue dotted line in Fig. 3 shows the CDF of this mixture ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} distribution. This CDF is very close to the empirical CDF of the LRT, confirming our asymptotic theory.

Example 7

(Exploratory factor analysis, revisited) Now we consider Example 1(a). Let $\Theta ,{\Theta }_0,{\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta ,\Theta _0,{\varvec{\theta }}^*$$\end{document} and $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} be the same as those in Example 4. In addition, we define

$\begin{array}{c}{\Theta }_1=\left(\rho ,\left(\Sigma \right),:,\Sigma ,=,a_1,a_1^{\top },+,a_2,a_2^{\top },+,\Delta ,,,,a_1,,,a_2,\in ,R^J,,,a_{12},=,0,,,\Delta ,\in ,R_{\mathrm{pd}}^{J\times J},\cap ,R_d^{J\times J}\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \Theta _1 = \left\{ \rho ({\varvec{\Sigma }}): {\varvec{\Sigma }} = {\mathbf {a}}_1{\mathbf {a}}_1^\top + {\mathbf {a}}_2{\mathbf {a}}_2^\top + {\varvec{\Delta }},~ {\mathbf {a}}_1,{\mathbf {a}}_2 \in {\mathbb {R}}^J, a_{12}=0, {\varvec{\Delta }} \in {\mathbb {R}}_{pd}^{J\times J} \cap {\mathbb {R}}_{d}^{J\times J} \right\} . \end{aligned}$$\end{document}

The tangent cone of ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} at ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} becomes

$\begin{array}{c}{T}_{{\Theta }_1}\left({\theta }^{\ast }\right)=\left(\rho ,\left(\Sigma \right),:,\Sigma ,=,a_1^{\ast },b_1^{\top },+,b_1,{a_1^{\ast }}^{\top },+,b_2,b_2^{\top },+,B,,,,b_1,,,b_2,\in ,R^J,,,b_{12},=,0,,,B,\in ,R_d^{J\times J}\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} T_{\Theta _1}({\varvec{\theta }}^*) = \left\{ \rho ({\varvec{\Sigma }}): {\varvec{\Sigma }} = {{\mathbf {a}}_1^*}{{\mathbf {b}}}_1^\top + {{\mathbf {b}}}_1{{\mathbf {a}}_1^*}^\top + {{\mathbf {b}}}_2{{\mathbf {b}}}_2^\top + {\mathbf {B}}, ~ {{\mathbf {b}}}_1,{{\mathbf {b}}}_2 \in {\mathbb {R}}^{J}, b_{12}=0, {\mathbf {B}} \in {\mathbb {R}}_{d}^{J\times J} \right\} . \end{aligned}$$\end{document}

Note that $T_{{\Theta }_1}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*)$$\end{document} is not a linear subspace, due to the $b_2{b}_2^{\top }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathbf {b}}}_2{{\mathbf {b}}}_2^\top $$\end{document} term. Therefore, by Theorem 2, the asymptotic distribution of ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} is not ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} . See the blue dotted line in Panel (a) of Fig. 1 for the CDF of this asymptotic distribution. This CDF almost overlaps with the empirical CDF of the LRT, suggesting that Theorem 2 holds here.

Example 8

(Exploratory item factor analysis, revisited) Now we consider Example 2(a). Let $\Theta ,{\Theta }_0,{\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta ,\Theta _0,{\varvec{\theta }}^*$$\end{document} and $T_{{\Theta }_0}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _0}({\varvec{\theta }}^*)$$\end{document} be the same as those in Example 5. Let

$\begin{array}{c}\begin{array}{c}{\Theta }_1=\left(\theta ,\in ,\Theta ,:,{\theta }_x,=,\int ,\int ,\prod _{j=1}^J,\frac{exp\left(x_j(d_j+a_{j1}{\xi }_1+a_{j2}{\xi }_2)\right)}{1+exp(d_j+a_{j1}{\xi }_1+a_{j2}{\xi }_2)},\varphi ,\left({\xi }_1\right),\varphi ,\left({\xi }_2\right),d,{\xi }_1,d,{\xi }_2,,,a_{12},=,0,,,x,\in ,{\Gamma }_J\right)\end{array}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \begin{aligned} \Theta _1 = \left\{ {\varvec{\theta }}\in \Theta : \theta _{{\mathbf {x}}} = \int \int \prod _{j=1}^J \frac{\exp (x_{j}(d_j + a_{j1}\xi _1+a_{j2}\xi _2))}{1+\exp (d_j + a_{j1}\xi _1+a_{j2}\xi _2)} \phi (\xi _1)\phi (\xi _2)d\xi _1d\xi _2, a_{12} = 0,{\mathbf {x}}\in \Gamma _J \right\} \end{aligned} \end{aligned}$$\end{document}

be the parameter space for the two-factor model. Recall $f_x$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {f}}_{{\mathbf {x}}}$$\end{document} and $g_x$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {g}}_{{\mathbf {x}}}$$\end{document} as defined in Example 5. For any $x\in {\Gamma }_J,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {x}}\in \Gamma _J,$$\end{document} we further define $H_x={\left(h_{\mathrm{rs}}\left(x\right)\right)}_{J\times J},$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathbf {H}}_{{\mathbf {x}}} = (h_{rs}({\mathbf {x}}))_{J\times J},$$\end{document} where

$\begin{array}{c}\begin{array}{cc}{h}_{\mathrm{rs}}\left(x\right)=& \int \prod _{j=1}^J\frac{exp\left(x_j(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)\right)}{1+exp(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)}\left(x_r,-,\frac{exp(d_r^{\ast }+a_{r1}^{\ast }{\xi }_1)}{1+exp(d_r^{\ast }+a_{r1}^{\ast }{\xi }_1)}\right)\\ & \times \left(x_s,-,\frac{exp(d_s^{\ast }+a_{s1}^{\ast }{\xi }_1)}{1+exp(d_s^{\ast }+a_{s1}^{\ast }{\xi }_1)}\right)\varphi \left({\xi }_1\right)d{\xi }_1\end{array}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \begin{aligned} h_{rs}({\mathbf {x}}) =&\int \prod _{j=1}^J \frac{\exp (x_{j}(d^*_{j} + a^*_{j1}\xi _1))}{1+\exp (d^*_{j} + a^*_{j1}\xi _1)} \left[ x_r - \frac{\exp (d^*_{r} + a^*_{r1}\xi _1)}{1+\exp (d^*_{r} + a^*_{r1}\xi _1)} \right] \\&\times \left[ x_s - \frac{\exp (d^*_{s} + a^*_{s1}\xi _1)}{1+\exp (d^*_{s} + a^*_{s1}\xi _1)} \right] \phi (\xi _1)d\xi _1 \end{aligned} \end{aligned}$$\end{document}

for $r\ne s,$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r\ne s,$$\end{document} and

$\begin{array}{c}\begin{array}{cc}{h}_{\mathrm{rr}}\left(x\right)=& \int \prod _{j=1}^J\frac{exp\left(x_j(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)\right)}{1+exp(d_j^{\ast }+a_{j1}^{\ast }{\xi }_1)}\left({\left(x_r,-,\frac{exp(d_r^{\ast }+a_{r1}^{\ast }{\xi }_1)}{1+exp(d_r^{\ast }+a_{r1}^{\ast }{\xi }_1)}\right)}^2\right)\\ & \left(-,\frac{exp(d_r^{\ast }+a_{r1}^{\ast }{\xi }_1)}{{(1+exp(d_r^{\ast }+a_{r1}^{\ast }{\xi }_1))}^2}\right)\varphi \left({\xi }_1\right)d{\xi }_1.\end{array}\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \begin{aligned} h_{rr}({\mathbf {x}}) =&\int \prod _{j=1}^J \frac{\exp (x_{j}(d^*_{j} + a^*_{j1}\xi _1))}{1+\exp (d^*_{j} + a^*_{j1}\xi _1)} \left\{ \left[ x_r - \frac{\exp (d^*_{r} + a^*_{r1}\xi _1)}{1+\exp (d^*_{r} + a^*_{r1}\xi _1)} \right] ^2 \right. \\&\left. - \frac{\exp (d_r^* + a_{r1}^*\xi _1)}{(1+\exp (d_r^* + a_{r1}^*\xi _1))^2} \right\} \phi (\xi _1)d\xi _1. \end{aligned} \end{aligned}$$\end{document}

Then, the tangent cone of ${\Theta }_1$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta _1$$\end{document} at ${\theta }^{\ast }$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\varvec{\theta }}^*$$\end{document} is

(6) $\begin{array}{c}{T}_{{\Theta }_1}\left({\theta }^{\ast }\right)=\left(\theta ,=,{\left\{{\theta }_x\right\}}_{x\in {\Gamma }_J},:,{\theta }_x,=,b_0^{\top },f_x,+,b_1^{\top },g_x,+,b_2^{\top },H_x,b_2,,,,b_0,,,b_1,,,b_2,\in ,R^J,,,b_{12},=,0\right).\end{array}$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} T_{\Theta _1}({\varvec{\theta }}^*) = \left\{ {\varvec{\theta }}=\{\theta _{{\mathbf {x}}}\}_{{\mathbf {x}}\in \Gamma _J}: \theta _{{\mathbf {x}}} = {{\mathbf {b}}}_0^\top {\mathbf {f}}_{{\mathbf {x}}} + {{\mathbf {b}}}_1^\top {\mathbf {g}}_{{\mathbf {x}}} + {{\mathbf {b}}}_2^\top {\mathbf {H}}_{{\mathbf {x}}}{{\mathbf {b}}}_2,~ {{\mathbf {b}}}_0,{{\mathbf {b}}}_1,{{\mathbf {b}}}_2 \in {\mathbb {R}}^J, b_{12} = 0 \right\} . \end{aligned}$$\end{document}

Similar to Example 7, $T_{{\Theta }_1}\left({\theta }^{\ast }\right)$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$T_{\Theta _1}({\varvec{\theta }}^*)$$\end{document} is not a linear subspace, and thus, ${\lambda }_N$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda _N$$\end{document} is not asymptotically ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} . In Panel (a) of Fig. 2, the asymptotic CDF suggested by Theorem 2 is shown as the blue dotted line. Similar to the previously examples, this CDF is very close to the empirical CDF of the LRT.

3. Discussion

In this note, we point out how the regularity conditions of Wilks’ theorem may be violated, using three examples of models with latent variables. In these cases, the asymptotic distribution of the LRT statistic is no longer ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} , and therefore, the test may no longer be valid. It seems that the regularity conditions of Wilks’ theorem, especially the requirement on a non-singular Fisher information matrix, have not received enough attention. As a result, the LRT is often misused. Although we focus on LRT, it is worth pointing out that other testing procedures, including the Wald and score tests, as well as limited information tests (e.g., tests based on bivariate information), require similar regularity conditions and thus may also be affected.

We present a general theory for LRT first established in Chernoff (1954) that is not widely known in psychometrics and related fields. As we illustrate by the three examples, this theory applies to irregular cases not covered by Wilks’ theorem. There are other examples for which this general theory is useful. For example, Examples 1(a) and 2(a) can be easily generalized to the comparison of factor models with different numbers of factors, under both confirmatory and exploratory settings. This theory can also be applied to model comparison in latent class analysis that also suffers from a non-invertible information matrix. To apply the theorem, the key is to choose a suitable parameter space and then characterize the tangent cone at the true model.

There are alternative inference methods for making statistical inference under such irregular situations. One method is to obtain a reference distribution for LRT via parametric bootstrap. Under the same regularity conditions as in Theorem 2, we believe that the parametric bootstrap is still consistent. The parametric bootstrap may even achieve better approximation accuracy for finite sample data than the asymptotic distributions given by Theorems 1 and 2. However, for complex latent variable models (e.g., IFA models with many factors), the parametric bootstrap may be computationally intensive, due to the high computational cost of repeatedly computing the marginal maximum likelihood estimators. On the other hand, Monte Carlo simulation of the asymptotic distribution in Theorem 2 is computationally much easier, even though there are still optimizations to be solved. Another method is the split likelihood ratio test recently proposed by Wasserman et al. (2020) that is computationally fast and does not suffer from singularity or boundary issues. By making use of a sample splitting trick, this split LRT is able to control the type I error at any pre-specified level. However, it may be quite conservative sometimes.

This paper focuses on the situations where the true model is exactly a singular or boundary point of the parameter space. However, the LRT can also be problematic when the true model is near a singular or boundary point. A recent article by Mitchell et al. (2019) provides a treatment of this problem, where a finite sample approximating distribution is derived for LRT.

Besides the singularity and boundary issues, the asymptotic distribution may be inaccurate when the dimension of the parameter space is relatively high comparing with the sample size. This problem has been intensively studied in statistics and a famous result is the Bartlett correction which provides a way to improve the ${\chi }^2$ \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\chi ^2$$\end{document} approximation (Bartlett 1937; Bickel and Ghosh 1990; Cordeiro 1983; Box 1949; Lawley 1956; Wald 1943). When the regularity conditions do not hold, the classical form of Bartlett correction may no longer be suitable. A general form of Bartlett correction remains to be developed, which is left for future investigation.