7

I am encountering a difficulty with the following task. Have I made a mistake, or is this an inherent flaw in the notion of confidence intervals? (Other such flaws exist.)

Consider a random sample $X_1,\ldots , X_n$ from a Uniform($\theta$, $\theta + a$) distribution, where $\theta$ is unknown and $a$ is known. We wish to determine a confidence interval for $\theta$.

The reader may verify the following details: The statistics $Y=\text{min}_i X_i$ and $Z=\text{max}_i X_i$ are jointly sufficient for $\theta$. For $\theta \le c_1 \le c_2 \le \theta + a$, $P\{c_1 \le Y \le Z \le c_2\} = [(c_2 - c_1)/a]^n$. For $0 < \gamma < 1$, set $d_1 =(1-\sqrt[n]\gamma)/2$ and $d_2 =(1+\sqrt[n]\gamma)/2$. Then $\gamma = P\{\theta + ad_1 \le Y \le Z \le \theta + ad_2\} = P\{Z -ad_2 \le \theta \le Y-ad_1\}$. Thus, $[Z -ad_2, Y -ad_1]$ is a $\gamma$ confidence interval for $\theta$.

Now here's the difficulty: If we observe $Z - Y > a\sqrt[n]\gamma$, then $Z -ad_2 > Y -ad_1$, so our formula yields a nonsensical answer. Have I made an error in my calculations? Or is this one of those problems with confidence intervals?

(Homework? I guess so - but a homework problem that I wrote for my students. Inspired by another problem in DeGroot & Schervish.)

[Also posted at math.stackexchange. I didn't know about this group, and I received no satisfactory answer there.]

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • 2
    **Closely related:** http://stats.stackexchange.com/questions/40769 and http://stats.stackexchange.com/questions/26313. Also of interest: http://stats.stackexchange.com/questions/3911, http://stats.stackexchange.com/questions/2272, and http://stats.stackexchange.com/questions/11609. – whuber Aug 03 '13 at 20:11
  • @whuber- actually, not very related. My question presupposes an understanding of confidence intervals. The gist: Is my example a demonstration of how the confidence interval methodology can fail? – Stephen Herschkorn Aug 03 '13 at 21:14
  • 2
    @Stephen - I think of CIs as inverted hypothesis tests: the $1-\alpha$ CI is the set of values that would not be rejected by an $\alpha$-level test. It's not obvious to me what your test is. If $\theta = \theta_1$ and we hypothesize $\theta = \theta_0$, what is the rejection probability when $\theta_0 = \theta_1$? Does it rise monotonically as $\theta_0$ moves away from $\theta_1$? – Ray Koopman Aug 04 '13 at 07:31
  • While one can often relate confidence intervals to hypothesis test, that is not how C.I.'s are defined. Generally, one comes up with some region $A(\theta)$ such that $P_\theta\{X \in A(\theta)\}=\gamma$ (where $X$ is a sufficient statistic) and rewrite the event as $\{\theta \in B(X)\}$ for some region $B(X)$. The random set $B(X)$ is the confidence region. – Stephen Herschkorn Aug 04 '13 at 08:49
  • I suppose if one did want to test $H_0: \theta = \theta_0$ *vs.* $H_1: \theta \neq \theta_0$, one could come up with a test that rejects $H_0$ if $Y$ is too high or $Z$ is too low. – Stephen Herschkorn Aug 04 '13 at 08:54
  • 1
    Stephen, when a CI is correctly applied, it cannot fail. *Ergo*, we are discussing questions of application and interpretation. It seems to me that several of the answers in the related threads address these very points. – whuber Aug 04 '13 at 15:12
  • 1
    @whuber: 1) Flaws with the confidence interval methodolgy, indeed with frequentist statistics, in general, have been identified before. (See, for example, DeGroot & Schervish's _Probability and Statistics_.) You might as well say "The earth revolves around the sun. Ergo, Galileo must be wrong." An apporiate response would be to identify the error in my analysis, if there is one. 2) If you find any relevant points in the articles you cite, please identify them. I looked at those before I even posted my questions and found none. – Stephen Herschkorn Aug 04 '13 at 20:20
  • @RayKoopman: Re: the hypothesis test Yes, the appropriate test would be to reject $H_0$ if $Y > \theta_0 + ad_2$ or $Z < \theta_0+ad_1$. If $Z - Y > a \sqrt[n]\gamma$, $H_0$ is rejected regardless of the value of $\theta_0$. – Stephen Herschkorn Aug 04 '13 at 20:27
  • 4
    I have not claimed that you are wrong, Stephen. However, I do not see what (technically) is the problem with a "nonsensical" answer, because it does not necessarily violate the defining properties of a confidence interval. That's why I maintain this question is one about the *understanding* and *interpretation* of confidence intervals. It's fine with me if you see no connection between this discussion and the ideas presented in the other threads, but I hope that other readers will find the possible connections to be interesting and constructive. – whuber Aug 05 '13 at 00:42
  • 2
    I do not see how a nonsensical answer is consistent with the statement that "a CI... cannot fail." – Stephen Herschkorn Aug 05 '13 at 00:48
  • 1
    It sounds like you are worried about notation rather than content. The answer you have obtained is not nonsensical: it represents an empty interval. That's a valid result for a CI procedure. – whuber Aug 05 '13 at 16:38
  • @whuber: If it is "valid" to say that a parameter lies in the empty set with 95% confidence, the eaxmple indeed reveals a major flaw with the frequentist methodology. Perhaps that is your answer to my original question. – Stephen Herschkorn Aug 05 '13 at 19:37
  • 6
    You seem to be using "confidence" in your comment in the sense of a Bayesian credibility rather than as defined in the statistical literature. As an extreme example of the distinction, a procedure that uses a spinner to return $\mathbb{R}$ with 95% probability and the empty set with 5% probability is a valid 95% confidence interval (albeit a terrible one, obviously, because it entirely ignores the data!). That shows why there is more to evaluating the quality of a CI procedure than checking its coverage, but it does not demonstrate any inherent flaw with CIs. – whuber Aug 05 '13 at 19:43
  • 1
    "There is more to evaluating the quality of a CI procedure than checking its coverage." Thank you for that. I think that is the answer I was seeking. As I still await details regarding Cox's answer below to see a better approach. As the *purpose* of confidence intervals are to provide interval estimates for a parameter, I still consider a method that allows empty intervals to be flawed. – Stephen Herschkorn Aug 05 '13 at 20:10
  • 1
    @Stephen: Cox's answer [indeed](https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-29/issue-2/Some-Problems-Connected-with-Statistical-Inference/10.1214/aoms/1177706618.full)! – Scortchi - Reinstate Monica Aug 30 '13 at 12:08
  • @Scortchi - Reinstate Monica: That Cox's answer link do not work! – kjetil b halvorsen Sep 08 '21 at 03:21
  • See some comments in my answer at https://stats.stackexchange.com/questions/540797/empirical-implications-of-unbiased-estimators/540831#540831, which are equally valid for confidence intervals. – kjetil b halvorsen Sep 08 '21 at 03:23
  • @kjetilbhalvorsen: Thanks! I fixed it – Scortchi - Reinstate Monica Sep 08 '21 at 14:20

1 Answers1

6

It's possible to construct silly confidence intervals that are technically valid, i.e. have the claimed coverage, but it's not obligatory. So I don't know why doing so should demonstrate a flaw in "confidence interval methodology" (whatever that is)—you could have constructed a more appropriate one for your purposes.

A more orthodox approach to this particular problem:

First, note that you can reëxpress the sufficient statistic $(Y,Z)$ to contain an obvious ancillary component: the sample range $Z-Y$. It's a very strong precision index: as it approaches $a$, $\theta$ is known almost exactly. So any inference should be conditional on its observed value $z-y$.

Second, the sample minimum $Y$ conditional on the observed value of the range is uniformly distributed between $\theta$ and $\theta+a-r$. Confidence intervals based on this distribution could be constructed & would be well behaved. Still, the likelihood for $\theta$ is flat between $z-a$ and $y$, so it wouldn't be possible to say for any confidence interval with less than 100% coverage that it contained values of $\theta$ less discrepant with the data than those outside it. Best to stick with the 100% interval.

[Response to comments:

(1) Your confidence interval does have some undesirable properties* but that's no reason to tar all confidence intervals with the same brush. You only took unconditional coverage into account when you derived it & therefore have no right to complain that unconditional coverage is all it gives you.

(2) You're right that $\left(Y-\left(1-\frac{1-\gamma} 2\right)(a-r),Y-\frac{1-\gamma} 2(a-r)\right)$ is a valid C.I., conditional on $r$, but why not use $(Y-\gamma(a-r),Y)$? The likelihood is $\frac{1}{a-r}$ for all values of $\theta$ between $y+r-a$ and $y$, & I don't see any generally compelling reason to honour an arbitrary 95% of those values with inclusion in my confidence interval. The 100% interval $(y+r-a,y)$ is preferable as it separates zero-likelihood values of $\theta$ from those with positive likelihood.

*At least for inference on $\theta$ from a single sample: there may be some applications—say in quality control, where consideration of coverage over repeated samples is more than a thought experiment—for which it could have some use.]

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
  • Thank you very much for answering. I don't quite follow your comment about discrepancy with the data. Could you please elaborate? And what would be the exact formula for this interval? – Stephen Herschkorn Aug 05 '13 at 19:42
  • As for the flaw, see my comment towards the bottom above: The possibility of an empty confidence interval surely provides evidence of a flaw in the frequentist methodology. It seems you have to fudge here to say to the client, "Well, the confidence interval is useless in this case, but we *know* it's between $Z-a$ and $Y$..." – Stephen Herschkorn Aug 05 '13 at 19:50
  • Oh, I see. Your approach yields $[Y-(1-\frac\gamma 2)(a-R),Y-\frac\gamma 2(a-R)]$, where $R$ is the range, as the CI. Much better. Thanks! – Stephen Herschkorn Aug 05 '13 at 21:01
  • Response to addition: Why "conditional on r"? The unconditional probability $P\{Y-(1-\frac{\gamma}2(a-R) \le \theta \le Y-\frac{\gamma}2(a-R)\} = \gamma$. Also, I am not saying this is the only $\gamma$ CI, just as the usual formulae for a 95% confidence interval for a mean, say, do not provide the only 95% confidence interval for that parameter. – Stephen Herschkorn Aug 06 '13 at 20:39
  • Why "conditional on $r$"? Just to emphasize that the CI's valid for each and every observed value of $r$. And I didn't think you were saying that only one CI was possible, but I wanted to explain why only the 100% one is good in this case. – Scortchi - Reinstate Monica Aug 06 '13 at 22:14