9

I need to make inference about a positive parameter $p$. To acomodate the positiveness I reparametrized $p=\exp(q)$. Using MLE routine I computed point estimate and s.e for $q$. The invariance property of the MLE directly gives me a point estimate for $p$, but I am not sure how to compute s.e for $p$. Thank in advance for any suggestion or reference.

Macro
  • 40,561
  • 8
  • 143
  • 148
Marcel
  • 91
  • 1
  • 3

2 Answers2

20

The Delta method is used for this purpose. Under some standard regularity assumptions, we know the MLE, $\hat{\theta}$ for $\theta$ is approximately (i.e. asymptotically) distributed as

$$ \hat{\theta} \sim N(\theta, \mathcal{I}^{-1}(\theta)) $$

where $\mathcal{I}^{-1}(\theta)$ is the inverse of the Fisher information for the entire sample, evaluated at $\theta$ and $N(\mu,\sigma^{2})$ denotes the normal distribution with mean $\mu$ and variance $\sigma^{2}$. The functional invariance of the MLE says that the MLE of $g(\theta)$, where $g$ is some known function, is $g(\hat{\theta})$ (as you pointed out) and has approximate distribution

$$ g(\hat{\theta}) \sim N( g(\theta), \mathcal{I}^{-1}(\theta) [g'(\theta)]^{2} ) $$

where you can plug in consistent estimators for the unknown quantities (i.e. plug in $\hat{\theta}$ where $\theta$ appears in the variance). I would assume the standard errors you have are based on the Fisher information (since you have MLEs). Denote that standard error by $s$. Then the standard error of $e^{\hat{\theta} }$, as in your example, is

$$ \sqrt{s^{2}e^{2 \hat{\theta}}} $$

I may be interpreting you backwards and in reality you have the variance of the MLE of $\theta$ and want the variance of the MLE of $\log(\theta)$ in which case the standard would be

$$ \sqrt{ s^{2}/\hat{\theta}^{2} } $$

Macro
  • 40,561
  • 8
  • 143
  • 148
  • 1
    Just a side note: there are also appropriate multivariate extensions whereby the derivatives are replaced by gradients, and the multiplications have to be matrix multiplications, so there's a bit more headache in figuring out where the transpose goes. – StasK Aug 18 '11 at 15:52
  • 1
    Thanks for pointing that out StasK. I believe in the multivariate case the asymptotic covariance of $g(\hat{\theta})$ is $\nabla g(\theta)' \mathcal{I}(\theta)^{-1} \nabla g(\theta)$ – Macro Aug 18 '11 at 15:55
  • (+1) I added a link to the regularity assumptions (and some other things) since it isn't clear whether these are satisfied in the OP's problem. I might have said that $\hat{\theta}$ is _asymptotically_ normal and not _approximately_ normal, since the convergence rates can be slow at times. – MånsT Jul 05 '12 at 07:08
  • Thank you @MånsT, I also did clarify that I meant asymptotically when I said approximately :) – Macro Jul 05 '12 at 11:48
6

Macro gave the correct answer on how to transform standard errors via the delta method. Though the OP specifically asked for the standard errors, I suspect that the objective is to produce confidence intervals for $p$. Besides computing estimated standard errors of $\hat{p}$ you can directly transform a confidence interval, $[q_1, q_2]$, in the $q$-parametrization to a confidence interval $[\exp(q_1), \exp(q_2)]$ in the $p$-parametrization. This is perfectly valid, and it may even be a better idea depending on how well the normal approximation used to justify a confidence interval based on standard errors works in the $q$-parametrization versus the $p$-parametrization. Moreover, the directly transformed confidence interval will fulfill the positivity constraint.

NRH
  • 16,580
  • 56
  • 68