4

We have a iid sequence of random variables $X_1, X_2, \dots, X_n$, where $E(X_i) = \mu$ and $var(X_i) = \sigma^2$. The sample mean $\bar{X}$ converges to $\mu$ at rate $\sqrt{n}$ thanks to the LLN.

If we have a continuous function $f()$, the continuous mapping theorem assures that $f(\bar X)$ converges to $f(\mu)$.

My question is the following: at what rate does $f(\bar X)$ converge to $f(\mu)$?

Asymptotically I would say $\sqrt{n}$, given that $f()$ is continuous and hence locally linear. But can we have convergence rates very different from $\sqrt{n}$ in small samples?

Matteo Fasiolo
  • 3,134
  • 2
  • 20
  • 29

1 Answers1

2

Whenever $f$ is differentiable at $\mu$, the theory is nice: $f(\bar{X})$ will converge at rate $\sqrt{n}$. To get the exact rate of convergence (including constants) you can use the delta method (http://en.wikipedia.org/wiki/Delta_method).

If $f$ is not differentiable, then weird stuff can happen and $\sqrt{n}$ convergence is not guaranteed. In most applications, thankfully, we don't need to worry about this.

Stefan Wager
  • 2,233
  • 11
  • 7
  • Thanks Stefan. $f$ is differentiable in my case, so you are right certainly right about what happens when the sample size $n$ is large enough, thanks to the delta method. But my doubt is whether $E((f(\bar{X})-f(\mu))^2) = O(n^{-1})$ also for small $n$. I was doing some simulation that seem to suggest that in some cases it takes a while to the asymptotic rate to "kick-in" but this might be due to programming or numerical errors so I'm still in doubt. – Matteo Fasiolo Jun 05 '13 at 21:52
  • Well, the statement $g(n) = O(n^{-1})$ means that $ng(n)$ is bounded as $n \rightarrow \infty$. It doesn't really make sense to ask whether $g(n)$ is $O(n^{-1})$ for small $n$. That's one of the major downsides of using asymptotics: There are in general no guarantees that the results will hold in the range you're interested in. (That being said... In general, the delta-method asymptotics will kick in slower when $f''$ is large. To see this, you can look at the higher-order terms in the delta method expansion.) – Stefan Wager Jun 05 '13 at 22:03
  • "To see this, you can look at the higher-order terms in the delta method expansion" that's exactly what I'm going to do tomorrow morning (I don't want to have nightmares tonight :) ). I got interested in small samples because in my application it looks like the asymptotic rate kicks in really slowly, so that (having finite computational power) small sample behaviour becomes important. – Matteo Fasiolo Jun 05 '13 at 22:22