7

Problem 8.7 From Van der Vaart's Asymptotic Statistics:

Given a sample of size $n$ from the uniform distribution on $[0,\theta]$, the maximum $X_{(n)}$ of the observations is biased downwards. Because $\text{E}[\theta-X_{(n)}] = \text{E}[X_{(1)}]$, the bias can be removed by adding the minimum of the observations. Is $X_{(1)} + X_{(n)}$ a good estimator for $\theta$ from an asymptotic point of view?

MY ATTEMPT:

The chapter is on efficiency of estimators (e.g. convolution theorem, relative efficiency) so my first thought was to calculate the asymptotic variance of $X_{(1)} + X_{(n)}$. From previous results in the class I have $$\text{Pr}\left\{ nX_{(1)} < x\right\} \to 1 - e^{-x/\theta}\qquad \text{Pr}\left\{ -n(X_{(n)}-\theta) < x\right\} \to 1-e^{-x/\theta}$$ that is, the asymptotic marginal distributions of the minimum and the maximum are exponential distributions. However, to get the asymptotic distribution of the sum I would need the joint asymptotic distribution. I'm not sure how to proceed.

ATTEMPT 2

Based on the linked question, if $X_{(1)}$ and $X_{(n)}$ are asymptotically independent then the asymptotic variance of $X_{(1)} + X_{(n)}$ would simply be $$\theta^2+\theta^2 + 2(0) = 2\theta^2$$ which is larger than the variance of the MLE ($X_{(n)}$). But the MLE is biased. Also, I have not found a way to show that the minimum and maximum are asymptotically independent.

Related: with no satisfactory answer Asymptotic distribution of uniform order statistics

Also somewhat related to the famous german tank problem, but that is for discrete uniform distribution.

Community
  • 1
bdeonovic
  • 8,507
  • 1
  • 24
  • 49
  • You might compare the asymptotic variance of $X_{(1)}+X_{(n)}$ with that of $\frac{n+1}{n}X_n$ – Glen_b Dec 10 '14 at 05:25
  • Could you elaborate? By $X_n$ do you just mean the last observation or the maximum? – bdeonovic Dec 10 '14 at 05:32
  • A typo. I mean $X_{(n)}$. Your complaint about $X_{(n)}$ was that it was biased (my reaction would be 'so?'); I was suggesting an obvious simple modification of it. [Personally I don't see why MMSE would not be preferable to forcing oneself to consider only unbiased estimators and then looking for small variance, but that's not so relevant to your question.] --- this should probably carry a `self-study` tag, btw – Glen_b Dec 10 '14 at 05:34
  • whats MMSE? Is this problem basically illustrating another example of bias/variance tradeoff? – bdeonovic Dec 10 '14 at 05:45
  • "minimum mean square error". I don't know what your problem is trying to illustrate, but my comment about MMSE certainly relates to it. – Glen_b Dec 10 '14 at 06:25

1 Answers1

3

Since this is for self study, I've decided to make this intentionally terse. First, when $\theta = 1$, show that $$ (X_{(1)}, X_{(n)} - X_{(1)},1 - X_{(n)}) \sim \operatorname{Dirichlet}(1, n-1, 1). $$ More generally, you might show $$ (X_{(1)}, X_{(2)} - X_{(1)}, \ldots, X_{(n)} - X_{(n-1)}, 1 - X_{(n)}) \sim \operatorname{Dirichlet(1, 1, \ldots, 1, 1).} $$ Then, we have $$\hat \theta = -(1 - X_{(n)}) + X_{(1)} + 1.$$ Using the Dirichlet distribution given above, we have immediately that $$\operatorname{Var}(\hat \theta) = \frac{2}{(n+1)(n+2)}.$$ Hence, for arbitrary $\theta$, $$\operatorname{Var}(\hat \theta) = \frac{2\theta^2}{(n+1)(n+2)}$$

To assess whether this is "good" or not, compare it to the UMVUE $\tilde \theta = [(n+1)/n]X_{(n)}$.

guy
  • 7,737
  • 1
  • 26
  • 50
  • $\hat\theta$ is not $(1 - X_{(n)}) + X_{(1)} - 1$ but $X_{(n)}) + X_{(1)}$. – Xi'an Dec 10 '14 at 06:44
  • @Xi'an whoops, I forgot a minus sign; should be $\hat \theta = -(1 - X_{(n)}) + X_{(1)} + 1 = X_{(n)} + X_{(1)}$. I also initially had the Dirichlet distribution wrong. – guy Dec 10 '14 at 06:48
  • This looks like a really interesting approach! A few questions: what inspired the connection to the Dirichlet? Is that something that holds true only in this particular case or more generally? How do you make the jump from $\theta=1$ to arbitrary $\theta$? – bdeonovic Dec 10 '14 at 13:05
  • @Benjamin I knew the order statistics were (scaled) Beta random variables, 1 - Beta flips the parameters, and then guessed that the "spacings" had a joint Dirichlet distribution because it gave the Beta result. It turns out this is well-known though. I don't think it is more general, but I'm not sure. To jump from $\theta = 1$ to arbitrary, just multiply and divide all the order statistics by $\theta$ since $X_{(j)}/\theta$ is the order statistic of a uniform on $[0,1]$ – guy Dec 10 '14 at 18:12
  • Excellent, this was a very fun exercise thank you! – bdeonovic Dec 11 '14 at 03:54
  • @StubbornAtom Good catch, there are actually $n+1$ spacings rather than $n$ spacings, meaning that I was off by $1$ for the Dirichlet. – guy May 08 '20 at 03:24