8

From the section on Significance Testing in onlinestatbook:

Whenever an effect is significant, all values in the confidence interval will be on the same side of zero (either all positive or all negative).

Why? Can't a confidence interval be positioned both sides of the zero? Say, can't its range be [-12; +12]?

CopperKettle
  • 1,123
  • 12
  • 18

4 Answers4

9

Can't a confidence interval be positioned both sides of the zero? Say, can't its range be [-12; +12]?

Certainly it can.

Did you miss the stated condition "Whenever an effect is significant"?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • I did not fully understand the meaning of the phrase "whenever an effect is significant". – CopperKettle Mar 12 '17 at 10:59
  • Are you familiar with hypothesis tests? – Glen_b Mar 12 '17 at 11:00
  • I've read through sections 1-8 of "The Logic of Hypothesis Testing" on that site, but that was a week ago.. maybe I've forgotten something. – CopperKettle Mar 12 '17 at 11:02
  • 1
    FIrst sentence of 2nd paragraph here: http://onlinestatbook.com/2/logic_of_hypothesis_testing/significance.html (*When the null hypothesis...*) – Glen_b Mar 12 '17 at 11:02
  • Thank you! So in hypothesis testing a confidence interval cannot include zero, but in statistical tasks of other kinds it can. – CopperKettle Mar 12 '17 at 11:12
  • 7
    No; in hypothesis testing you reject or fail to reject a hypothesis. Confidence intervals and hypothesis tests are different but related. If you do both on the same data, then *if you reject the hypothesis test that some effect is zero* you will also discover that the confidence interval for that effect doesn't include zero. If you fail to reject, then the CI will include zero. [Actually this isn't quite *always* true; there are commonly used tests and CIs which can fail to obey this rule] – Glen_b Mar 12 '17 at 11:17
  • 5
    [by the way please don't interpret the brevity of my answer and comments as implied criticism in any sense; when you're first learning something -- let alone trying to teach it to yourself -- the details are hard to grasp the first time because you don't have all the concepts while you're learning them so you don't have all the ideas to give context to put new concepts into, and details are bound to get missed. When beginning to learn statistics, which is very concept dense and sometimes counter-intuitive, doubly so) – Glen_b Mar 12 '17 at 11:21
  • "Actually this isn't quite *always* true; there are commonly used tests and CIs which can fail to obey this rule" — We ought not to claim it's true, then. I've seen quite a few people who seem to believe that an estimate is significantly nonzero if and only if a CI excludes 0. – Kodiologist Mar 12 '17 at 16:44
  • @Kodiologist Specifically, large sample approximations made at the null (for the hypothesis test) and at the alternative (the CI being based around an estimate) are sometimes inconsistent. e.g. the usual normal-approximation based intervals and tests in one-sample tests of proportion is an obvious example. In these fairly simple cases, we could at least argue that they *ought* to be consistent (and some other test/CI combinations for the binomial proportion are) – Glen_b Mar 12 '17 at 22:52
  • I the book that is in question it says "If the 95% confidence interval contains zero (more precisely, the parameter value specified in the null hypothesis), then the effect will not be significant at the 0.05 level." So as I understand the main point is not that CI contains zero. The main point is that CI contains value that is in Nul hypothesis and thats why we faild to reject null hypothesis not simpli becouse we just see zero in CI. Am I right? – vasili111 Apr 23 '19 at 23:09
5

Often the quantity you are interested in has as its null value zero. So for example if you are estimating differences between means then the null value is zero - no difference. In such a case you would be interested in whether the confidence interval included zero. In some cases though the null value is not zero but perhaps unity. For instance if you are interested in the odds ratio then the null value is unity (since it is a ratio and when the quantities are the same their ratio is unity). In such cases you are interested in whether the interval includes unity and in fact the ratio cannot be negative.

mdewey
  • 16,541
  • 22
  • 30
  • 57
  • I the book that is in question it says "If the 95% confidence interval contains zero (more precisely, the parameter value specified in the null hypothesis), then the effect will not be significant at the 0.05 level." So as I understand the main point is not that CI contains zero. The main point is that CI contains value that is in Nul hypothesis and thats why we faild to reject null hypothesis not simpli becouse we just see zero in CI. Am I right? – vasili111 Apr 23 '19 at 23:11
4

The quote talks about testing whether a certain value is significantly different from zero at some level of significance.

If you had a confidence interval proposed by you, then the test would not reject the null hypothesis that "the tested value is zero". Because the interval [-12, 12] actually tells you that it might easily happen, that the true value is indeed zero.

ira
  • 399
  • 2
  • 14
  • Does it mean that the quote only speaks of the [Physicians' Reactions case study](http://onlinestatbook.com/2/case_studies/weight.html)? – CopperKettle Mar 12 '17 at 10:56
  • 2
    No, this is in general true about significance testing. The result of significance test and the confidence interval are related. If you have an estimate of some value (let's say a mean of a certain variable), and a confidence interval for that estimate, then if zero lies within that confidence interval, you can't be really sure that the mean of that variable is different from zero (because zero is within you confidence interval). – ira Mar 12 '17 at 11:17
  • I the book that is in question it says "If the 95% confidence interval contains zero (more precisely, the parameter value specified in the null hypothesis), then the effect will not be significant at the 0.05 level." So as I understand the main point is not that CI contains zero. The main point is that CI contains value that is in Nul hypothesis and thats why we faild to reject null hypothesis not simpli becouse we just see zero in CI. Am I right? – vasili111 Apr 23 '19 at 23:11
2

Allow me to help. Basically, the confidence interval of a parameter are the endpoints of an interval or range that the parameter can reasonably achieve. So, if 95% of the time a parameter is greater than -12 and less than 12, then it could also be zero. And, if it can be zero then it can be worthless. For example, if $A X$ can have $A=0$ then its contribution is not significant to a regression. If we had 95% CI's for $A$ of 12 to 24, then it is not likely to be worthless as it is significantly not zero. Mind you, not significantly different from zero is not necessarily an insignificant contribution, and if we had more data, it might become significantly different from zero. It is a fine point, perhaps, that not significant does not mean insignificant, and just because we do not obtain significance as a result proves nothing special.

However, that does not mean that because we have a not significant result that it is also meaningless. It does have a meaningful context, and indeed one in which the confidence intervals contribute more than a probability alone would. Suppose we have 95% confidence intervals for $A$ as above that only extend from -1 to 1. Then they are 12 times lesser in range than the -12 to 12 CIs we had before, above. What that implies is that we would then know that a result of -5 or +5 would be significantly non-zero so that we now have a greater certainty of where the values of $A$ are not-located, and a narrower range of where our uncertain values of $A$ reside.

Carl
  • 11,532
  • 7
  • 45
  • 102
  • I the book that is in question it says "If the 95% confidence interval contains zero (more precisely, the parameter value specified in the null hypothesis), then the effect will not be significant at the 0.05 level." So as I understand the main point is not that CI contains zero. The main point is that CI contains value that is in Nul hypothesis and thats why we faild to reject null hypothesis not simpli becouse we just see zero in CI. Am I right? – vasili111 Apr 23 '19 at 23:11
  • 3
    @vasili111 I think you are, if I am understanding you correctly. For example, we might want to see if a slope includes the identity function $y=x$. In that case, we would be looking for a confidence interval that includes 1, not 0, such that it very much depends on exactly what the null hypothesis is. – Carl Apr 24 '19 at 03:05
  • Yes, that is exactly what I mean. Thank you. – vasili111 Apr 24 '19 at 14:38