6

$N_A$ and $N_B$ are variables of the counts of the number of events 'A' and events 'B' respectively. Those variables follow Poisson distributions with parameters $\lambda_A$ and $\lambda_B$.

In nature I made one observation of each variable; I observed $n_A$ events 'A' and $n_B$ events 'B'. From those, I am asking: Are $\lambda_A$ and $\lambda_B$ equal or different?

How can I make a hypothesis testing on the null that the rates $\lambda_A$ and $\lambda_B$ are the same?

I thought about using maximum likelihood methods. I could calculate the confidence intervals (CI) for both the parameters $\lambda_A$ and $\lambda_B$ and then see if the CI overlap (seems wrong to me). Or would it be appropriate to use MLEs for $\lambda_A$ and $\lambda_B$ to make a likelihood ratio test?

Remi.b
  • 4,572
  • 12
  • 34
  • 64
  • You might want to clarify this a bit, you first state that you know the parameters, and you don't have an estimate. Since you know the parameters you already know if they are the same or not, so why are you doing a test? Then you start talking about CI's for parameters, unless you are doing something Bayesian, your parameters are not stochastic. Remember that you do tests based on the distribution of your estimator, and these tests should reflect the chance that the estimator comes from some null distribution. – Jonathan Lisic Jun 03 '15 at 20:15
  • 1
    Reading your comment I drastically changed my post. Not to change its meaning but to simplify the question. In short: You observe two values 'n_A' and 'n_B' drawn from two Poisson distributions. You want to know how whether the parameter $\lambda_A$ and $\lambda_B$ of the Poisson distributions from which 'n_A' and 'n_B' are drawn are the same or not. Does it make sense? – Remi.b Jun 03 '15 at 20:40
  • 1
    Would you consider a Wald test based on the mles? http://en.wikipedia.org/wiki/Wald_test – JohnK Jun 03 '15 at 21:51
  • 1
    Indeed Wald test seems like a good solution. I have the same issue than with a likelihood ratio. Can I just chose whichever $\lambda$ I want to be in the numerator (lieklihood ratio) or to be \theta_0 (Wald test)? – Remi.b Jun 04 '15 at 00:42
  • 1
    @Remi.b What you have to put in the numerator is the difference between the mles, under the null hypothesis that the population parameters are equal. It's quite straightforward and it gives reliable results for even moderately large samples. – JohnK Jun 04 '15 at 01:08
  • I am a bit confused. So you'd use the statistic: $D=2 ln\left(\frac{MLE_{\lambda_A} - MLE_{\lambda_B}}{\text{denominator}}\right)$ which is $\chi^2$-distributed. What is the denominator then? – Remi.b Jun 04 '15 at 01:29
  • @Remi.b See my answer. – JohnK Jun 04 '15 at 01:36
  • The exact likelihood-ratio test conditional on the total of the two counts is simple in this case: see [significance of difference between two counts](http://stats.stackexchange.com/a/155318/17230) – Scortchi - Reinstate Monica Jun 04 '15 at 08:33

1 Answers1

8

As noted in the comments, the Wald statistic is simple, powerful and therefore a good choice for this problem. Now, for two Poisson populations, presumably independent, we wish to test the hypotheses that their parameters are equal, namely:

$$H_0: \lambda_1=\lambda_2\quad \text{vs} \quad H_1 :\lambda_1 \neq \lambda_2$$

The Wald statistic in this case is defined as

$$Z=\frac{\widehat{\lambda}_1-\widehat{\lambda}_2}{\sqrt{var({\widehat{\lambda}_1})+var({\widehat{\lambda}_2)}}}$$

and according to the theory of maximum likelihood it has an asymptotic standard normal distribution. The mle for the parameter $\lambda$ is of course the sample mean, so this is what should go in the numerator.

The denominator is a little more complicated. To see this, note that for the sample mean

$$var(\bar{X})=\frac{\sigma^2}{n}$$

but under the Poisson assumption, $\sigma^2=\mu$, right? So the question is, which estimator should we use for $\sigma^2$, the sample variance or the sample mean? The asymptotic distribution holds either way.

The answer is the sample mean, despite the fact that this might seem counter-intuitive. The reason is that the sample mean in a Poisson distribution is the UMVUE for the parameter $\lambda$ and therefore by using that instead of the sample variance, we gain precision.

We now have everything we need. The test takes the form:

$$Z=\frac{\widehat{\lambda}_1-\widehat{\lambda}_2}{\sqrt{\frac{{\widehat{\lambda}_1}}{n_1}+\frac{{\widehat{\lambda}_2}}{n_2}}}$$

Once you compute it, you can find the two-sided p-value from the Normal distribution or you can square it and look at the one-sided p-value of the $\chi^2 (1) $ distribution. This is often more convenient.

Hope this helps.

JohnK
  • 18,298
  • 10
  • 60
  • 103
  • That was a perfect, detailed right to the level I needed answer! +1 Thank you – Remi.b Jun 04 '15 at 02:07
  • (+1). Why do you think it is "more convenient" to use the squared statistic? – Alecos Papadopoulos Jun 04 '15 at 12:41
  • @AlecosPapadopoulos This is a personal preference, nothing scientific. – JohnK Jun 04 '15 at 12:42
  • 1
    @JohnK Certainly, but I was asking exactly about such a personal preference, it is interesting to share such points of view. For example, at that level one could argue in favor of the t-statistic, that it immediately provides also a symmetric confidence interval. – Alecos Papadopoulos Jun 04 '15 at 12:47
  • @AlecosPapadopoulos The confidence interval is certainly a good point that can be made. I was focusing on computing p-values which I find easier when there is a one-tailed test involved. Less likely to make a mistake this way. In general I find the definition of the two sided p-values outside symmetric distributions $$p=2 \min \left\{ P \left( X\leq x|H_0 \right), P\left( X\geq x |H_0 \right) \right\} $$ a little off-putting – JohnK Jun 04 '15 at 12:55
  • @JohnK In fact, a two-sided test with a non-symmetric distribution is an awkward issue, and I would not say "resolved" in any widely agreed way. This post is relevant, http://stats.stackexchange.com/q/140107/28746 – Alecos Papadopoulos Jun 04 '15 at 12:58
  • @AlecosPapadopoulos I find Glen's answer closer to my intuition. This is how I would prefer computing p-values. By the way, this is how it is done in Bayesian statistics. Form a Highest Posterior Density interval and look at the area left outside. I find this quite satisfactory. – JohnK Jun 04 '15 at 13:01
  • @JohnK Yes, it is an approach that has intuition and scientific merit. – Alecos Papadopoulos Jun 04 '15 at 13:08
  • @AlecosPapadopoulos I think it is the only definition compatible with how we have learnt to define the p-value: *the probability of getting a statistic at least as extreme*. But that's just me. – JohnK Jun 04 '15 at 13:10
  • @JohnK: Note HPD's aren't the only way to construct a credible interval, & are open to the objection that they're not parametrization-invariant. See [Quantile intervals vs. highest posterior density intervals](http://stats.stackexchange.com/q/24588/17230) & [What is the decision-theoretic justification for Bayesian credible interval procedures?](http://stats.stackexchange.com/q/24681/17230). (Of course the *whole* posterior density is parametrization invariant.) – Scortchi - Reinstate Monica Jun 05 '15 at 08:36
  • @Scortchi Thanks for the references. I am certain there is a lot more to it than what I know now. Given enough time, I'll get there. – JohnK Jun 05 '15 at 08:39
  • Can you clarify the difference between lambda1 and n1 (and 2)? If we observe x1 and x2 counts over the same period, are not both lambda1 and n1 equal to x2? So the denominator of the test reduces to sqrt(2)? – Max Ghenis Jul 09 '16 at 22:36
  • Maybe this is real dumb, but I'm expecting my rates \lambda1 and \lambda2 to be identical, which gives me a 0 for Z every time. – sameagol Sep 16 '20 at 03:05