19

Could anyone please tell me what the regularity conditions are for the asymptotic distribution of Likelihood Ratio test?

Everywhere I look, it is written 'Under the regularity conditions' or 'under the probabilistic regularities'. What are the conditions exactly? That the first and second log-likelihood derivatives exist and Information matrix not be zero? Or something else entirely?

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
Kingstat
  • 373
  • 1
  • 2
  • 12

1 Answers1

23

The required regularity conditions are listed in most intermediate textbooks and are not different than those of the mle. The following ones concern the one parameter case yet their extension to the multiparameter one is straightforward.

Condition 1: The pdfs are distinct, i.e. $\theta \neq \theta ^{\prime} \Rightarrow f(x_i;\theta)\neq f(x_i;\theta ^{\prime}) $

Note that this condition essentially states that the parameter identifies the pdf.

Condition 2: The pdfs have common support for all $\theta$

What this implies is that the support does not depend on $\theta$

Condition 3:The point $\theta_0$, the real parameter that is, is an interior point in some set $\Omega$

The last one concerns the possibility that $\theta$ appears in the endpoints of an interval.

These three together guarantee that the likelihood is maximised at the true parameter $\theta_0$ and then that the mle $\hat{\theta}$ that solves the equation

$$\frac{\partial l(\theta)} {\partial \theta}=0$$

is consistent.

Condition 4: The pdf $f(x;\theta)$ is twice differentiable as a function of $\theta$

Condition 5: The integral $\int_{-\infty}^{\infty} f(x;\theta)\ \mathrm dx$ can be differentiated twice under the integral sign as a function of $\theta$

We need the last two to derive the Fisher Information which plays a central role in the theory of convergence of the mle.

For some authors these suffice but if we are to be thorough we additionally need a final condition that ensures the asymptotic normality of the mle.

Condition 6:The pdf $f(x;\theta)$ is three times differentiable as a function of $\theta$. Further for all $\theta \in \Omega$, there exist a constant $c$ and a function $M(x)$ such that

$$\left| \frac{\partial^3 log f(x;\theta)}{\partial \theta^3} \right| \leq M(x)$$

with $E_{\theta_0} \left[M(X)\right] <\infty$ for all $|\theta-\theta_0|<c$ and all $x$ in the support of $X$

Essentially the last condition allows us to conclude that the remainder of a second order Taylor expansion about $\theta_0$ is bounded in probability and thus poses no problem asymptotically.

Is that what you had in mind?

JohnK
  • 18,298
  • 10
  • 60
  • 103
  • Thanks. But you sure that the regularity conditions related to the proof that -2log(lambda) follows Chi square with df 1 are the same? – Kingstat Jun 07 '14 at 15:06
  • 1
    @Kingstat Yes. These conditions come from Hogg and Craig "Introduction to Mathematical Statistics" and ensure that under $H_0$,that is $\theta=\theta_0$, $-2\log\Lambda \to^D \chi^2 (1)$ – JohnK Jun 07 '14 at 15:08
  • could you please also tell me how for a N(θ,1) density, Rao's Score test is equivalent to the UMPU test? – Kingstat Jun 07 '14 at 15:14
  • @Kingstat What does UMPU stand for? – JohnK Jun 07 '14 at 15:20
  • Uniformly most powerful unbiased. – Kingstat Jun 07 '14 at 15:23
  • @Kingstat That I am afraid I do not know. If that is a homework question, I suggest you start another thread with the self study tag. – JohnK Jun 07 '14 at 15:26
  • I have not understood condition 2. *"The pdfs have common support for all $\theta$"*. By `common support` does it mean for all $\theta$s, the pdf is not zero-valued. Also is the statement talking about several pdf ? And finally , how does it imply *" the support of $X_i$ does not depend on $\theta$ "* ? – ABC Mar 27 '15 at 11:42
  • @ABC -- I think you might be confusing the parameter ($\theta$) and the variable itself ($x$). Condition 2 says that no matter the value of $\theta$, the same values of $x$ have non-zero probability. – rcorty Apr 08 '17 at 20:54
  • @JohnK, I have found your answer useful but am having some difficulty applying it to my situation described in the question ["Failing to obtain $\chi^2(1)$ asymptotic distribution under $H_0$ in a likelihood ratio test: example 2"](https://stats.stackexchange.com/questions/506807). I wonder if you might have something to contribute. – Richard Hardy Jan 28 '21 at 06:52