4

I have a very simple question for Hypothesis testing. Is there a way to calculate the probability that the null hypothesis is false?

Since in the literature they're always talking about the significance level and the power of a test, these values are calculated assuming something about the null hypothesis (being true or false in each case).

But, in general terms, can we calculate the probability that a given null hypothesis is false, or true?

Silverfish
  • 20,678
  • 23
  • 92
  • 180
S. Cow
  • 273
  • 1
  • 2
  • 17
  • 1
    Well, maybe if you're Bayesian ... but people may not agree with your priors, on which such calculations would depend. – Glen_b Aug 25 '16 at 02:15
  • 1
    See http://stats.stackexchange.com/questions/166323/misunderstanding-a-p-value/166327#166327 –  Aug 25 '16 at 06:23
  • 1
    You can also check threads tagged [tag:tost] – Tim Aug 25 '16 at 07:34

1 Answers1

6

The term "null hypothesis" is usually used in a frequentist setting, where characteristics of the population, such as its mean, are regarded as fixed, not random. There, it makes no sense to talk about the probability of the null hypothesis.

In a Bayesian setting, these characteristics are regarded as random and we can talk about things like the probability of a population mean equalling 0. However, a typical Bayesian would give a prior probability of 0 to many common frequentist null hypotheses, such as the hypothesis that the mean of a normal distribution exactly equals a prespecified value.

Kodiologist
  • 19,063
  • 2
  • 36
  • 68
  • 1
    So i can never measure the probability that a Null Hypothesis stated by someone is false right? All i can do (from the frequentist point of view) is Reject Ho if my data is telling me that. Can you give me a recomendation for an introductory Book of Bayesian Inference? Thanks a lot! – S. Cow Aug 25 '16 at 11:15
  • 1
    Yes and yes. Try John K. Kruschke's *Doing Bayesian Data Analysis* for a gentle introduction or Gelman et al.'s *Bayesian Data Analysis* for a deeper book that assumes some familiarity with mathematical statistics. – Kodiologist Aug 25 '16 at 14:22