How to show the rate of convergence of this maximum likelihood estimator is $n^{-1}$?

Question

Suppose I have data $\{X_i\}_{i=1}^{n}\sim \text{Uniform}[0,\theta_0]$. We know that the maximum likelihood estimator of $\theta_0$ is $\widehat{\theta}=\max\{X_1,...,X_n\}$. How to show that $\widehat{\theta}$ converges to $\theta_0$ at rate $n^{-1}$? Thanks!

Answer 1

When $(X_i)$ is a sequence of uniformly distributed variables $\mathcal{U}_{[0,\theta_0]}$ we can derive the distribution of the MLE $\hat \theta = \max X_i$.
We will show that $n( \theta_0 - \hat \theta)$ converge towards a non trivial (i.e. not $0$) distribution, which means that the rate of convergence of the MLE is $n^{-1}$.

First note that since $\max X_i \leq \theta_0$, $n( \theta_0 - \hat \theta) \geq 0$.

Then, for $t \geq 0$,

\begin{align*} \mathbb P\left( n(\theta_0 -\hat \theta) \leq t \right) &=1- \mathbb P\left( \hat \theta \leq \theta_0 - \frac{t}{n} \right) \\ &=1-\mathbb P\left( \max X_i \leq \theta_0 - \frac{t}{n} \right) \\ \end{align*}

Since $\max X_i \leq \theta_0 - \frac{t}{n} \iff \forall i, X_i \leq \theta_0 - \frac{t}{n}$ and since all $X_i$ are independent and identically distributed, \begin{align*} \mathbb P\left( \max X_i \leq \theta_0 - \frac{t}{n} \right) &=\prod_{i=1}^n\mathbb P\left( X_i \leq \theta_0 - \frac{t}{n} \right) \\ &=\mathbb P\left( X_1 \leq \theta_0 - \frac{t}{n} \right)^n \end{align*}

For $n$ large enough we have $\frac{t}{n} \leq \theta_0$ and then, \begin{align*} \mathbb P\left( X_1 \leq \theta_0 - \frac{t}{n} \right)^n &= \left(\frac{\theta_0 - \frac{t}{n} }{\theta_0} \right)^n \\ &=\left(1 - \frac{t}{\theta_0 n}\right)^n \end{align*}

As $n \to \infty$ the last line is equivalent to $\exp\left(-\frac{t}{\theta_0}\right).$

So for $t \geq 0$: $$ \mathbb P\left( n(\theta_0 -\hat \theta) \leq t \right) \xrightarrow[n \to \infty]{} 1- \exp\left(-\frac{t}{\theta_0}\right). $$

We can see that $1-\exp\left(-\frac{t}{\theta_0}\right)$ is the cumulative distribution function of an exponential random variable with rate $\frac{1}{\theta_0}$.

Thus, $n (\theta_0 - \hat \theta )$ is asymptotically exponentially distributed, and the rate of convergence of $\hat \theta$ is $n^{-1}$.

Here is a R code to illustrate this result (with $\theta_0 = 3$ and $n=50$):

theta0<-3
n<-50
s<-sapply(1:1e4,function(i){
      X<-runif(n,0,theta0)
      return(n*(theta0-max(X)))
})
P_emp<-function(t){
      return(mean(s<=t))
}
#P_emp is an empirical cumulative distribution of n*(theta0 - MLE)
#P_emp should be close to the CDF of an exponential distribution

x<-seq(0,20,length.out = 1000)
y1<-sapply(x,function(t) P_emp(t))

plot(x,y1,type='l',col="red")
lines(x,pexp(x,rate=1/theta0),type='l',col="black")
legend(x=0,y=1,legend = c("Empirical CDF","Exponential 
CDF"),lty=1,col=c("red","black"))