3

Using the inference of mean as an example, the null and alternative hypothesis could be

$$H_0: \mu \le 0 \Leftrightarrow H_1: \mu > 0$$

It is often argued that this makes the calculation of the $p$-value (or critical region) easier, but from my understanding the $p$-value is $P(\bar{X} > 0|H_0) \stackrel{\color{red}{?}}{=} \sup_{\theta \in H_0} P(\bar{X} > 0 | \theta )$. Since I'm taking its supremum, the $p$-value doesn't change regardless of whether the equality sign is included. The same result goes for the critical region. However, I'm not sure if my understanding is correct, and my doubt is marked with a red question mark above.

Anyway, I wonder if it's correct (I know that would be unconventional, though) to write

$$H_0: \mu < 0 \Leftrightarrow H_1: \mu \ge 0$$

If that's OK, what about its $p$-value and critical region? If that's not OK, I would be really surprised, because in that way we are changing the question ("Is $\mu$ less than or equal to zero?") to something different ("Is $\mu$ less than zero?")

nalzok
  • 1,385
  • 12
  • 24
  • 2
    I guess Ho and H1 have to be mutually exclusive, don't they? – Isabella Ghement Jun 28 '19 at 15:05
  • @IsabellaGhement Yeah, that's why I have modified $H_1$ as well as $H_0$ – nalzok Jun 28 '19 at 15:06
  • 3
    You implicitly assume something that is not generally true: namely, that the null hypothesis includes values of $\mu$ arbitrarily close to $0.$ What if $\mu$ is known to be integral, for instance? I realize this is somewhat artificial, but it might provide some insight. – whuber Jun 28 '19 at 15:33
  • @whuber That's true, but in most textbook examples such extra information is not taken into account at all. Is it important for many real-life applications, or there's only some rare cases? – nalzok Jun 28 '19 at 15:56
  • 1
    @IsabellaGhement that's my understanding as well, but apparently it's controversial... see the discussion in the comments of this post: [One sided test $H_0:\mu=0$ or $H_0:\mu\leq 0$?](https://stats.stackexchange.com/questions/385314/one-sided-test-h-0-mu-0-or-h-0-mu-leq-0/385319#385319) – Scholar Jun 28 '19 at 15:57
  • 2
    Consider a continuous, composite null containing no boundary (i.e. an open set) - for simplicity a "standard" one tailed test of a population mean. Where do you calculate the significance level at? (Hint: you actually end up evaluating it under the alternative...) – Glen_b Jun 29 '19 at 07:16
  • 1
    This is a case where use of standard Bayesian posterior probabilities cuts through a lot of the complexities; just compute $P(\mu \in S)$ for any set $S$. – Frank Harrell Jun 29 '19 at 11:26

2 Answers2

4

The principle of the hypothesis test requires the computation of the rejection probability assuming H0 (type I error probability). In fact, if the null hypothesis is not just one point, we are fine with the supremum, so that we can state that the type I error probability of the test is smaller or equal to that value. So far this would be in line with what you're writing and wouldn't require the equality case to be included.

However, if in fact the rejection probability for the "equal" is equal to the supremum over the rest of the null hypothesis (as is the case for most tests although one could imagine tests for which this doesn't hold), the test will not distinguish the equality case from the H0, i.e., you will not have a higher rejection probability for "equal" than for the H0 as a whole, and therefore it makes sense to include "equal" in the H0.

In the words of E.S. Pearson (if I remember correctly), tests for which parameter values in the H1 have a lower rejection probability than cases in the H0 are "worse than useless", and if they have the same rejection probability, they are just useless.

Why would you want to reject a hull hypothesis in favour of an alternative that has a rejection probability that isn't higher? The idea of tests is that you collect evidence in favour of the alternative from observing a rejection event that has a probability under the alternative that is higher than under the H0.

PS (added later): Technically, according to the general definition of hypothesis tests, you do not always have to include $\mu=0$ in the null hypothesis (note also that not all tests are Neyman-Pearson tests). I'm not giving you a reason why this isn't permitted, I'm giving you a reason why in most cases it doesn't make sense. As written above, I can even imagine to construct a problem with a weird model and parametrisation according to which the rejection probability at $\mu=0$ is in fact higher than the supremum over $\mu<0$, in which case I wouldn't object at all against testing $H_0: \mu<0$ against $H_1: \mu\ge 0$. However pretty much all the standard tests that you find in the literature are not of this kind. In all the standard cases in fact the test level is computed on the hypothesis border, i.e. (using the parametrisation referred to in the question), $\mu=0$, and then $\mu<0$ is attached based on the fact that the supremum over those parameter values is not bigger. So it would be illogical to then attach $\mu=0$, the basis of the level computation under H0, to the H1.

PPS, prompted by your comment: In my (hopefully not too unconventional) terminology the "rejection probability" is the probability, given $\mu$, to reject the H0. This depends on $\mu$. Particularly, if $\mu$ is in the H0, it is a type I error probability, whereas if $\mu$ is in the H1, it is the power. In a good test, you want the type I error probability to be low and the power to be high. A test in which the power is not higher than the type I error probability (or be it the supremum of these) doesn't do what it's supposed to do, it doesn't indicate the H1 in case of rejection. Or let's say it doesn't indicate the $\mu=0$ part of the H1 in case you chose to include $\mu=0$ in the H1. It's nonsense to say "this level $\alpha$-test indicates evidence in favour of $\mu\ge 0$" if the power in case of $\mu=0$ is no better than that $\alpha$.

Christian Hennig
  • 10,796
  • 8
  • 35
  • You said $H_0: \mu \le 0$ and $H_0: \mu < 0$ can have the same rejection region, and I totally agree, but I don't get the "reject a null hypothesis in favour of an alternative that has a rejection probability that isn't higher" part. Isn't the rejection probability always 5% when the significance level is 0.05? – nalzok Jun 28 '19 at 17:20
  • 1
    Lehmann (AFAIK): see [here](https://stats.stackexchange.com/a/393527/17230). But if the null hypothesis were $\mu < 0$, then the supremum of the rejection probability over the null wouldn't be equal to the rejection probability when $\mu=0$, would it? @nalzok: It's always 5% or less *under the null*. – Scortchi - Reinstate Monica Jun 28 '19 at 17:39
  • @nalzok: See added PS and PPS. – Christian Hennig Jun 28 '19 at 22:22
  • I get the problem that you describe. A test which rejects $H_0$ in favor of an equally likely or even worse likely $H_1$ is not so great. But is this solved by switching? From $$H_0: \mu < 0 \Leftrightarrow H_1: \mu \geq 0$$ to $$H_0: \mu \le 0 \Leftrightarrow H_1: \mu > 0$$ To me this seems more a problem that the hypothesis testing is not such a great method because it doesn't involve priors (and replaces this with some worse case scenario). In practice, when I observe some high value of a statistic which is very unlikely under $H_0: \mu < 0$ then I have little problems accepting $H_1$.... – Sextus Empiricus Feb 01 '20 at 10:29
  • ....even better, instead of accepting $H_1$ would be to not use such dichotomic hypothesis selection rule. In practice it is not about hypothesis testing anyway, but instead about optimizing some outcome (e.g. profit). – Sextus Empiricus Feb 01 '20 at 10:35
  • "Accepting the $H_1$" is not a standard way to interpret a hypothesis test. Anyway, I have no issues with the question addressed by hypothesis tests, namely whether the data is *compatible* with the $H_0$, and if not, in what direction it points, which doesn't commit me to "accept" anything as true. That's relevant enough in many situations. And if I don't have information that comes in a way nicely expressible as a prior, I'm happy about a method that doesn't require one. – Christian Hennig Feb 03 '20 at 00:14
0

In traditional hypothesis testing the null hypothesis always contains an $=$-sign, whether it is as $=, \le,$ or $\ge.$

The mull hypothesis determines the null distribution of the test statistic. Hence also the critical value used in testing at a particular level or, in computer programs, the P-value.

Example: In a simple binomial test whether a coin is fair, when suspected of bias towards Heads, might be $H_0: p = 1/2$ vs $H_0: p > 1/2.$

Suppose data consist of 100 coin tosses: We can reject $H_0$ at the 4.43% level, if the number $X$ of Heads matches or exceeds the 'critical value' $c= 59.$ (Because of the discreteness of the binomial distribution a nonrandomized test at exactly the 5% level is not available.)

In R, $P(X \ge 59\,|\,p=.5) = 1 - P(X \le 58\,|\,p = .5) = 0.0443.$ 

1 - pbinom(58, 100, .5)
[1] 0.04431304

enter image description here

Intuitively, it might seem that getting 55 Heads in 100 should arouse suspicion that the coin is biased for Heads. "Suspicion" maybe Yes, but this would not be a statistically significant result at the stated level of significance. By random variation a truly fair coin can show 55 or more Heads in 100 tosses with probability about 0.184.

1 - pbinom(54, 100, .5)
[1] 0.1841008

Also, if we observe $X = 62$ Heads then the P-value is $P(X \ge 62\,|\,p=.5) = 0.01,$ leading to rejection of $H_0$ because the P-value $0.0105 < 0.0443.$ The P-value is defined as the probability, under $H_0,$ of an outcome more extreme (in the direction of the alternative) than what was observed.

1 - pbinom(61, 100, .5)
[1] 0.01048937

If the hypotheses were formulated as $H_0: p \le 1/2$ vs. $H_0: p > 1/2,$ then the $=$-sign in $H_0$ would still govern these probability computations.

BruceET
  • 47,896
  • 2
  • 28
  • 76
  • 1
    I can understand what you are saying, but could you elaborate on how does it answer my question? – nalzok Jun 28 '19 at 17:09
  • You seem to propose a null hypothesis $H_0: \mu < 0,$ which does not contain an $=$-sign. That is not a correct null hypothesis. It provides no specific value of $\mu$ that can be used to determine a null distribution. // First rule of hypothesis testing: Put an $=$-sign in $H_0.$ – BruceET Jun 28 '19 at 17:23
  • I noticed this issue and thus asked if the $p$-value can be calculated with $P(\bar{X} > 0|H_0) = \sup_{\theta \in H_0} P(\bar{X} > 0 | \theta)$. If this equation is correct, then I can work on $\lim_{n \to \infty} P(\bar{X} > 0 | \mu \le -\frac{1}{n}) = \lim_{n \to \infty} P(\bar{X} > 0 | \mu = \frac{1}{n})$. This is calculatable because each value of $\mu$ is specific. – nalzok Jun 28 '19 at 17:28
  • @BruceET: "First rule of hypothesis testing: Put an =-sign in 0". Nope, see my answer, particularly the PS part. I'm with nalzok on this matter. You won't find such a rule in any sufficiently general treatment of hypothesis tests. – Christian Hennig Jun 29 '19 at 11:28
  • @Lewian. Saw it. Hard to recognize it as traditional hypothesis testing. Pondering whether there are practical situations where that framework is useful. Disagreements about hypothesis testing are hardly new to statistics. – BruceET Jun 29 '19 at 15:51