3

The Neyman-Pearson lemma, the Karlin-Rubin theorem, and the other for UMPU tests for the exponential family, etc. They all define a most powerful Rejection Region (RR), in a certain class of tests.

I was wondering how do these RR translate into graphs? I'm interested not only in those tests that are most powerful in a sense, but also the usual tests we use, even if we don't do a power study of them. For example, when we use a Wald test of the form $n(\hat\phi-\phi)^TC^{-1}(\hat\phi-\phi)$ we get a quadratic. So, the RR we're interested in is the one defined just by a one-tailed test, i.e., an ellipsoid around the true value $\phi$, and whenever our estimates lie outside the ellipsoid, we reject the null.

It would be nice to gather as much examples of RR for different tests as possible.

An old man in the sea.
  • 5,070
  • 3
  • 23
  • 57
  • In multiple testing (with e.g. Bonferroni correction) you get a rectangle in two dimensions, a box in three dimensions, ... –  Jul 21 '16 at 14:10
  • @fcop please write up an answer It would be great to be able to structure this. ;) – An old man in the sea. Jul 21 '16 at 21:45
  • maybe you can help me with this @An old man in the sea ? http://stats.stackexchange.com/questions/229193/whats-wrong-with-multiple-testing-correction-compared-to-joint-tests –  Aug 10 '16 at 19:05
  • @fcop I'm not sure whether I'm that capable. I'll have a look. Meanwhile, I've put +1 on it. ;) – An old man in the sea. Aug 10 '16 at 20:08
  • I am sure that, at least, you understood because your question on the geometric shape of the acceptance region made me think and led to this question –  Aug 10 '16 at 21:20
  • @fcop But I'm not myself familiarised with multiple testing corrections... – An old man in the sea. Aug 10 '16 at 22:19

1 Answers1

1

Refering to the Wald test that you mention and (for ease of understanding) limiting it to two dimensions, your test looks something like $H_0: \mu_1 = \mu_{10} \& \mu_2 = \mu_{20}$ versus $H_0: \mu_1 \ne \mu_{10} | \mu_2 \ne \mu_{20}$.

You can approach this problem in two ways; either you can use ''joint tests'' or you can use ''multiple testing procedures''. Joint tests start from a multi-variate distribution for a test statistic (the Wald test you mention is an example) while multiple testing procedures ''adjust'' the p-value (or equivalently , adjust the signficinace level). The Bonferroni correction is an example of a multiple testing procedure. For more details see this link.

The Bonferroni procedure states that, if you perform two tests, then you should perform two independent tests, but in order not to inflate the type I error, you should do each test with half the significance level (or you should double the p-values). (This also holds for more than two tests).

Obviously, if I perform each of the two tests in a ''univariate way'' with half the significance level, then I find an interval in each of the two dimensions or a reactankle in the two-dimensional plane.

You can see the different shapes in the figures 2,3,4 in the link supra. Figure 4 has a ''combined shape'', for details see the link.