0

I understand that the confidence level denotes how reliable is my test and the confidence level is not the probability associated with the confidence interval. This is well explained on Wikipedia:https://en.wikipedia.org/wiki/Confidence_interval

Nevertheless, in Wikipedia it is also stated in the subsection "Misunderstandings" that: A 95% confidence level does not mean that for a given realized interval there is a 95% probability that the population parameter lies within the interval.

This is illogical to me. If the confidence interval denotes how reliable a test is, and if the test provides a given interval, I would deduce that the confidence interval is reliable with 95% of probability and hence the parameter is in the interval with 95% probability.

Why is my reasoning wrong?

1 Answers1

0

Suppose we can sample data from the population $\mathsf{Norm}(\mu, \sigma = 4).$ You decide to find a 95% confidence interval for the unknown $\mu$ by looking at a random sample of size $n = 16.$ from this population.

Then $\bar X \sim \mathsf{Norm}(\mu. \sigma = 4/\sqrt{16} = 1),$ so that $$P(-1.96 \le \bar X - \mu \le 1.96) = P(\bar X - 1.96 \le \mu \le \bar X + 1.96) = 0.95.$$

Most statisticians are completely happy with that much, as long as $\bar X$ remains unobserved.

However, traditional frequentist statisticians object to saying that the statement $\mu \in (\bar X -1.96, \bar X +1.96)$ has probability $0.95,$ on the grounds that that this statement is now "either true or not true," but we don't know which and we can't assign a straightforward probability because $\bar X$ is now an observed value, not a random variable. (Much as in quantum mechanics, the 'probability' seems to 'collapse' as soon as $\bar X$ is observed. Never mind that the 'true value' of $\mu$ never becomes available.)

Substitution of the word confidence for the word probability has become a standard way to meet that objection. Then each textbook author (and Wikipedia contributor) writes a version of the story that satisfies their own philosophical preference.

You join a long line of statistics students who have been puzzled by the statements of the various versions. If you are a 'peacemaker' you will latch onto one of the versions as your favorite and carefully parrot it word-for-word to future generations of statisticians, tactfully suppressing (perhaps even eventually forgetting) your initial objections. You will avoid using the word illogical, however well you think it fits.

Bayesian statisticians avoid this thicket, beginning with the foundational viewpoint that $\mu$ is not an unknown fixed constant, but instead a random variable with a prior distribution, but Bayesians have their own PR problems.

BruceET
  • 47,896
  • 2
  • 28
  • 76