Unbiased estimator for AR($p$) model

Question

Consider an AR($p$) model (assuming zero mean for simplicity):

$$ x_t = \varphi_1 x_{t-1} + \dotsc + \varphi_p x_{t-p} + \varepsilon_t $$

The OLS estimator (equivalent to the conditional maximum likelihood estimator) for $\mathbf{\varphi} := (\varphi_1,\dotsc,\varphi_p)$ is known to be biased, as noted in a recent thread.

(Curiously, I could neither find the bias mentioned in Hamilton "Time Series Analysis" nor in a few other time series textbooks. However, it can be found in various lecture notes and academic articles, e.g. this.)

I was not able to find out whether the exact maximum likelihood estimator of AR($p$) is biased or not; hence my first question.

Question 1: Is exact maximum likelihood estimator of AR($p$) model's autoregressive parameters $\varphi_1,\dotsc,\varphi_p$ biased? (Let us assume the AR($p$) process is stationary. Otherwise the estimator is not even consistent, since it is restricted in the stationary region; see, e.g., Hamilton "Time Series Analysis", p. 123.)

Also,

Question 2: Are there any reasonably simple unbiased estimators?

I'm quite sure the ML estimator in an AR(p) is biased (the existence of the stationarity boundary suggests that it will be biased) but I don't have a proof for you right now (most ML estimators are biased in any case, but we have a little more than that to go on here). [Personally I don't see unbiasedness as a particularly useful property to have, at least in general -- it's like the old joke about statisticians going duck hunting. *Ceteris paribus,* having it is better than not, of course, but in practice the *ceteris* are never *paribus*. It's an important concept though. ] — Glen_b, Nov 20 '15 at 00:08
I thought unbiasedness would be desirable when working in small samples, and I have just faced such an [instance](http://stats.stackexchange.com/questions/182608/null-distribution-of-f-statistic-in-small-samples-with-co-integrated-time-se). In my understanding, in that case unbiasedness was more desirable than, say, efficiency as long as efficiency could be quantified. — Richard Hardy, Nov 20 '15 at 06:50
Where bias may not be small (as in small samples), I'd really tend to look for something more like minimum mean square error. What's the point in caring that your estimate could be wrong on average, when in fact your alternative estimate could be much more wrong because it's got a high variance? e.g. if my bias at this sample size for this $\phi$ is 0.1 that might be worryingly large so you'd say "let's use an unbiased estimator"... but if the standard error is large enough that my estimate is typically even further from the correct value ... am I better off? ...ctd — Glen_b, Nov 20 '15 at 07:14
ctd. ... I don't think so (not for my usual purposes at least, and I've almost never seen a good argument for unbiasedness in a practical situation that something more like MMSE wouldn't be better for). I care about how wrong *this* estimate is -- how far I may be off the true value -- not how much the shift in average is if I'm in this situation a million more times. The main practical value in working out the bias tends to be seeing whether you can reduce it easily without much impacting the variance. — Glen_b, Nov 20 '15 at 07:18
[Here](https://www.nhh.no/Admin/Public/Download.aspx?file=Files%2FFiler%2Finstitutter%2Fsam%2FConferences%2FNordic+Econometrics+2013%2FKaufmann+kruse+(2013)+-+Bias-corrected+estimation+in+potentially+mildly+explosive+autoregressive+models+(2).pdf) is a link dealing with bias and bias-correction technique (albeit in the context of mildly explosive processes - but hopefully you may find other papers on the stationary case from there). — Christoph Hanck, Nov 20 '15 at 08:02
@ChristophHanck, thank you, this is quite interesting and contains relevant references. — Richard Hardy, Nov 20 '15 at 08:37
The duck hunting joke cited above can be found [here](https://mathandchristina.weebly.com/math-humor/three-statisticians-go-hunting). — Richard Hardy, Jun 25 '21 at 08:11

score 1 · Answer 1 · answered Oct 11 '19 at 06:53

This is of course not a rigorous answer to your question 1, but since you asked the question in general, evidence for a counterexample already indicates that the answer is no.

So here is a little simulation study using exact ML estimation from arima0 to argue that there is at least one case where there is bias:

reps <- 10000
n <- 30
true.ar1.coef <- 0.9

ar1.coefs <- rep(NA, reps)
for (i in 1:reps){
  y <- arima.sim(list(ar=true.ar1.coef), n)
  ar1.coefs[i] <- arima0(y, order=c(1,0,0), include.mean = F)$coef
}
mean(ar1.coefs) - true.ar1.coef

score -1 · Answer 2 · answered Jul 05 '17 at 01:35

-1

I happen to be reading the same book that you are reading and found the answer to both of your questions.

The biasness of the autoregression betas is mentioned in the book on page 215.

The book also mentions a way to correct the bias on page 223. The way to proceed is through a iterative two step approach.

Hope this helps.

answered Jul 05 '17 at 01:35

denizen of the north

848
1
8
15

Per the site's [guidelines](https://stats.stackexchange.com/help/how-to-answer), answers should not simply consist of references to material elsewhere. – Alexis Oct 09 '17 at 21:20

Unbiased estimator for AR($p$) model

2 Answers2

Linked