6

When we say the likelihood ratio statistic and the Wald statistic of a set of binomial distributions are asymptotically equivalent, do we mean that the sampling distributions of the two statistics will get close as we take larger samples of both statistics from a fixed set of binomial distribution?

Or does this mean that the two sampling distributions will converge as we increase the number of trials $n$ of the binomial distribution of interest?

I am slightly confused as to what $n$ (as in $n\rightarrow \infty$) stands for in this case.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
HDB
  • 249
  • 1
  • 6

1 Answers1

5

Either the number of binomial trials or the number of observations will do; usually we think of this result as applying more generally than binomial data and so think of the number of observations as the $n\to\infty$.

It's also important to note that the asymptotic equivalence is local. Suppose 0 is the null value of $\theta$. If you set $\theta=\theta_A=\neq 0$ and take $n$ observations with that value of $\theta$, $n\to\infty$, there is no guarantee that the the test statistics will approach each other. The standard result is that if you take have a sequence values $\theta_n=h/\sqrt{n}$ and take $n$ observations with $\theta=\theta_n$, then as $n\to\infty$ the score, Wald, and likelihood ratio tests will converge in probability to the same random variable.

Here's the picture: on a graph with the score (derivative of loglikelihood) on the $y$-axis and $\theta$ on the $x$-axis, the Wald chi-squared statistic is twice the area of the blue triangle. The score chi-squared statistic is twice the area of the red triangle, and the likelihood ratio chi-squared statistic is twice the grey area under the curve.

triangles of Wald,LRT, score

With $n\to\infty$ and $\theta_n=h/\sqrt{n}$, we're zooming in to this picture. The curve locally asymptotically approaches a straight line and the three coloured areas become the same enter image description here

But if you fix $\theta\neq 0$ and just increase $n$, the picture doesn't change. It still just looks like enter image description here and there's no asymptotic equivalence.

Thomas Lumley
  • 21,784
  • 1
  • 22
  • 73
  • sorry, just to clarify more, so if I artificially generate a large number of likelihood ratio and wald test statistics from a given set of binomial distribution while keeping the number of samples drawn from the binomial distributions to be small, will the distributions of the simulated likelihood ratio and wald test statistics in this case be very close to one another? (i.e. due to the asymptotic equivalence?) thank you, – HDB Oct 28 '20 at 03:33
  • No. Not in general. They won't be too far off unless the true parameter value is a long way from the null, but there won't be any asymptotic equivalence. – Thomas Lumley Oct 28 '20 at 04:11
  • 2
    Thomas's answer was great but also note that the result that is being discussed is not specific to the binomial. It holds whenever the Wald test and the LR test are applicable. – mlofton Oct 28 '20 at 04:20
  • 1
    This kind of concept should be explained with this kind of graphic more often. Does $h$ represent either of the three statistics, that is Wald, score and likelihood ratio? – Single Malt Oct 28 '20 at 09:20
  • what do you exactly mean by "number of observation"? by this, do you mean that the number of samples drawn from the binomial distribution in order to generate a likelihood ratio statistic or the wald test statistic? I am confused because I am not trying to test the specific parameter of the binomial distributions, what I am trying to do is to demonstrate the asymptotic equivalence of the likelihood ratio statistic and the wald test statistic from a set of binomial distribution with given parameters via permutation test. – HDB Oct 28 '20 at 15:06
  • The number of observations is independent of whether one uses the LR or the Wald test. You use the same number in both tests. I'm not clear exactly where your confusion but if you google for asymptotic equivalence of LR and Wald test, I bet that a lot of things will come up. They just won't be specific to the binomial because the asymptotic result is a general result, no matter what the underlying distribution is. – mlofton Nov 05 '20 at 14:03
  • related: https://stats.stackexchange.com/questions/449344/likelihood-ratio-wald-and-score-are-equivalent – Christoph Hanck Nov 05 '20 at 14:12
  • 1
    +1: could you provide more detail on when the picture "applies"? Does it also work for, say, nonlinear models? – Christoph Hanck Nov 05 '20 at 14:19
  • 1
    Yes, it does. Not for mixed models so much. – Thomas Lumley Nov 05 '20 at 21:27