0

I understand ARMA is a linear combination of lagged data points and lagged errors, but I am unclear on its implementation once parameters have been identified. Now suppose I have an ARMA model and some data. Where do I get these error terms, especially for the first term.

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
CBBAM
  • 159
  • 5

1 Answers1

1

They are unobservable, just like errors in a regression model. You can estimate them, again just like in a regression model. Estimation of ARMA models is done via maximum likelihood, frequently via state-space representation and Kalman filtering. A good description of ARMA estimation is available Hamilton "Time Series Analysis" and other time series textbooks (and probably some threads on Cross Validated; search for estimation and ARIMA).

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
  • A starting point could be ["ARIMA estimation by hand](https://stats.stackexchange.com/questions/77663). – Richard Hardy Apr 27 '21 at 06:05
  • Thank you! I have been doing some reading since I posted the question and it seems for an ARMA (1,1) process I would arbitrarily start with $\epsilon_0 = 0$ and recursively compute the sequence of $\epsilon_i$ through the ARMA equation, but rewritten as: $\epsilon_t = X_t - \mu - \theta X_{t-1} - \phi \epsilon_{t-1}$. With some assumed distribution, I take the maximum likelihood of these $\epsilon_t$ using the distribution's PDF. With $\mu$ being the mean of the data, I use this process to find $\theta$ and $\phi$. Am I on the right track? – CBBAM Apr 27 '21 at 06:23
  • @CBBAM, yes, I think so. There are two caveats, though. First, this is conditional likelihood. In full likelihood, you would optimize for $\epsilon_0$, too. Second, naive optimization of the likelihood may be highly inefficient, take long time and fail to converge. (I think I tried it some years ago, and it did not go well.) Just look at the explicit expression of the likelihood in terms of the observed data and the parameters; the likelihood function is highly nonlinear w.r.t. the parameters, and that gets worse with sample size, if I remember correctly. – Richard Hardy Apr 27 '21 at 16:51
  • Thank you, how big of an affect does not optimizing $\epsilon_0$ have? – CBBAM Apr 27 '21 at 17:00
  • @CBBAM, I presume it is small when the sample size is medium to large. If you get around writing code for ARMA or ARMA-GARCH, it would be cool to see it. Consider posting it by answering one of your questions such as [this one](https://stats.stackexchange.com/questions/521335). I would appreciate if you pinged me then. Thank you. – Richard Hardy Apr 27 '21 at 17:00
  • Sure I definitely will, I have written code for it over the past 2 days and have got it running but so far the results are not ideal (my predicted $\sigma$'s increase without bound during optimization. I am trying to debug, and if done successfully, I will post it. – CBBAM Apr 27 '21 at 17:06