13

In hypothesis testing, alternative hypothesis doesn't have to be the opposite of null hypothesis. For example, for $H_0: \mu=0$, $H_a$ is allowed to be $\mu>1$, or $\mu=1$. My question: Why is this allowed? What if in reality, $\mu=-1$ or $\mu=2$, in which case if one applies, say, likelihood ratio test, one may (wrongly) conclude that $H_0$ is accepted, or $H_0$ is rejected and hence $H_a$ is accepted?

What about this proposal: $H_a$ should always be the opposite of $H_0$? That is, $H_a: H_0$ is not true. This way, we are effectively testing only a single hypothesis $H_0$, rejecting it if the p-value is below a predefined significance level, and not have to test two hypotheses at the same time that can be both wrong.

Lei Huang
  • 756
  • 6
  • 13
  • Related (particularly the accepted answer): https://stats.stackexchange.com/questions/232665/understanding-formulation-of-hypotheses-in-difference-between-two-sample-means/232669#232669 – JDL Sep 06 '18 at 13:14
  • 1
    The statement "$H_0$ is not true" is undefined *until you describe the space of all possible hypotheses.* – whuber Sep 06 '18 at 20:59

4 Answers4

25

What you've identified is one of the fundamental flaws with this approach to hypothesis testing: namely, that the statistical tests you are doing do not assess the validity of the statement you are actually interested in assessing the truth of.

In this form of hypothesis testing, $H_a$ is never accepted, you can only ever reject $H_0$. This is widely misunderstood and misrepresented by users of statistical testing.

Jack Aidley
  • 416
  • 3
  • 10
12

$H_{a}$ is, properly the complement of $H_{0}$ in the sample space of the distribution under the null hypothesis. One-sided tests, should therefore properly have $H_{0}: \mu \ge c$ (for some number $c$), with $H_{a}: \mu < c$ (or vice versa: $H_{0}: \mu \le c$, with $H_{a}: \mu > c$), for precisely the reason you allude to: if the null hypothesis in a one-sided test is specified as $H_{0}: \mu = 0$, then a one-sided alternative hypothesis cannot express the complement of $H_{0}$. I (and others) therefore disagree with those who use the confusing nomenclature you describe.

See my answer here for a similar question and issue.

Alexis
  • 26,219
  • 5
  • 78
  • 131
  • 6
    Thanks. What about $H_0: \mu=\mu_1$ and $H_a: \mu=\mu_2$ as in Neyman–Pearson lemma? – Lei Huang Sep 06 '18 at 05:08
  • 5
    No you can do a classic likelihood ratio test involving just two points. But as Jack so well stated before, hypothesis testing was never a good idea except in very special, narrow, circumstances. Bayesians would say "get me evidence that the unknown parameter is in any interval you specify". – Frank Harrell Sep 06 '18 at 11:50
  • 1
    @LeiHuang Darn it! I muff that spelling all the time! Thank you. I personally would not use the language "alternative hypothesis" to describe the two hypotheses in the Neyman–Pearson lemma, for more or less the reason I give in my answer. The NPL is also, to my mind, using a different logic than Wald-type test statistics, by asking which of these two values of a parameter the data provide more evidence of. – Alexis Sep 06 '18 at 15:36
  • 1
    @LeiHuang Also: the NPL type hypothesis in your first comment is a different form of hypothesis, than the one-sided hypothesis in your original question (and, indeed, from $H_{0}: \mu = \mu_{1}, H_{a}: \mu \ne \mu_{1}$). – Alexis Sep 06 '18 at 15:39
1

This points to one of the few serious problems with the conventional statistics through null hypothesis significance testing (NHST). A much more meaningful approach in this case is to totally abandon NHST, and adopt the Bayesian framework. If you have some prior information available, just incorporate it into your model through prior distribution. Unfortunately most statistics consumers are simply too indoctrinated, obsessed and entrenched with the old school of thinking. See more discussion here.

bluepole
  • 2,376
  • 3
  • 25
  • 34
1

Put properly, we don't actually test if an alternative hypothesis is true. It is often described that way, but as far as basic statistics goes, that is incorrect.

We actually test whether there is, or is not, enough evidence to accept some "new"/"novel"/"not-default" hypothesis H. We do this by

  1. Taking into account what we know (Bayesian style if appropriate);
  2. Choosing a test we think is applicable to the data and hypothesis we are probing, and
  3. Stipulating a point which will be deemed "significant".

The significance level

This last item, the "significamce level", is often a source of confusion. What we actually say is, "If the hypothesis is wrong, then how exceptional would our results be?" So, suppose we set a significance level of 0.1% (P=0.001), what we are saying is:

"If our hypothesis is wrong, we just got a 1 in 1000 result by pure chance. That's so unlikely that we conclude the hypothesis is probably correct."

So you can "draw the line" where you like - for some research such as particle physics, you'd want 2 separate (independent) experiments both with a significance level of 1 in some millions, before concluding the hypothesis is probably correct. For a rigged dice game, a 1 in 3 level might be enough to persuade you not to play that game :)

But either way it is crucial to pick the level beforehand, otherwise you're probably just make a self serving statement using 'whatever level you like".

Stilez
  • 289
  • 1
  • 4