0

Say I conducted a sample and did the relevant processing to get a 95% confidence interval of a parameter to fall between a [10,14].

My friend did another sample, and got a x% confidence interval of a parameter to fall between [11,13].

Is x = 95, <95 or >95?

Edit: Made a mistake here. The friend was supposed to use the same data in the qns so answer should be x< 95% :D

Joel Tan
  • 49
  • 4
  • 1
    Re the edit: the answer is *still* not determined: your friend might have used a more powerful procedure than you did. – whuber Oct 23 '20 at 14:23
  • @whuber, It can also be a less powerful procedure. – Sextus Empiricus Oct 23 '20 at 15:10
  • @Sextus That seems unlikely when that procedure shrank the interval so much, but it is possible in principle. – whuber Oct 23 '20 at 15:30
  • @whuber so you are suggesting that [10,14] wasn't truly a 95% confidence interval after all (creating boundaries more wide than necessary)? – Sextus Empiricus Oct 23 '20 at 15:38
  • @Sextus Not at all: it's perfectly possible for a statistical problem to have many admissible confidence interval procedures as well as many more inadmissible ones. The sense of "creating boundaries more wide than necessary" suggests there exists a unique admissible procedure, but that's rarely the case. A standard example is an iid sample from a lognormal distribution with unknown geometric mean and sd. – whuber Oct 23 '20 at 15:41
  • @whuber I am not sure how you argue that the method which leads to [11,13] indicates more power than the method that leads to the [10,14] interval. If the one method gives larger intervals for *all* samples then the intervals do not have the same confidence level. If the one method gives larger intervals only for some samples (and it will compensate with smaller intervals for other samples) then we can not know which one is more powerful than the other just because the interval of one method is smaller than the other in one case (since for a different sample it will be an opposite situation). – Sextus Empiricus Oct 23 '20 at 16:09
  • @Sextus Are you perhaps implicitly assuming both methods are using the same model for the data?? – whuber Oct 23 '20 at 16:12
  • 1
    Yes, I was assuming both methods are using the same model for the distribution/probability of the data. – Sextus Empiricus Oct 23 '20 at 16:20

3 Answers3

6

The answer is underdetermined.

The latter interval is narrower, but that could be explained by smaller sample variance (say by chance) or by the use of a different quantile (i.e. different $\alpha$ and thus different confidence).

Demetri Pananos
  • 24,380
  • 1
  • 36
  • 94
4

Demetri Pananos argued that the question is underdetermined based on the idea that the sample might be different. However, even when the same sample is used (as clarified on your edit) then we can still not know whether the confidence interval [11,13] is a lower or higher % confidence interval than the confidence interval [10,14].

Example

Below is an example that is a bit far-fetched, and this answer is pedantic. But it does illustrate how confidence intervals should be interpreted.

Say we compute the mean of a sample from a normal distributed population that is parametized by the 4-th root $\theta^{1/4}$ of $\theta$. Let the sample distribution of the mean be:

$$\bar{x} \sim \mathcal{N(\mu_{\bar{x}} = \text{sign}(\theta)\text{abs}(\theta)^{1/4}\, ,\, \sigma_{\bar{x}} = 1})$$

this expression $\text{sign}(\theta)\text{abs}(\theta)^{1/4}$ is like $\theta^{1/4}$ but also has a solution for negative $\theta$.

The image below shows how we can create different 95% confidence intervals for this case (see for more about this: The basic logic of constructing a confidence interval). We sketch two cases.

  • In one case, the solid lines, boundaries are chosen by the upper and lower 2.5% test intervals.
  • In the other case, the broken line, the boundaries are chosen by the upper 5% for $\theta<0$ and the lower 5% for $\theta>0$

In both these cases, the confidence intervals will contain the true value 5% of the time, independent of the true parameter $\theta$.

If we would observe $\bar{X} = 0$, then we would have two different intervals approximately $[-7.320,7.320]$ and $[-14.757,14.757]$. That's a difference by a factor two, just like your case.

example

I agree that this answer is pedantic, and shows how the question is technically underdetermined. But in practice, you will not find the situation of the example above. In a 'normal' situation we the confidence intervals will not be constructed in these strange/extreme ways and when one confidence interval is smaller (for the same data) then it typically relates to a smaller % confidence. But, as this answer shows, it is not necessary.

Sextus Empiricus
  • 43,080
  • 1
  • 72
  • 161
  • This is truly educational. Thank you. I don't think any of my Profs can make a lesson this interesting. – Joel Tan Nov 05 '20 at 13:04
  • 1
    @JoelTan to be honest, I find the intuition that [11,13] represents a confidence interval with a smaller x% than [10,14] not so bad. In practice, this difference, a factor 2, is large and you would not encounter this unless the confidence level is different. So, my answer here is only to show the theoretical possibility that the confidence levels for these two different intervals can be the same. In practice, you will not get a situation with such a strong difference in intervals for the same confidence levels. *(and that is why my teachers often did not like me)* – Sextus Empiricus Nov 05 '20 at 13:10
1

https://besjournals.onlinelibrary.wiley.com/doi/10.1111/1365-2656.12382

please read this article, here you will see differences in different calculation methods for confidence interval,

of course different samples yield different results in same method

in the end you can't point any answer, samples alone can distort heavily answers for different methods (+numerical errors)

quester
  • 472
  • 3
  • 12