0

Let’s consider some common linear time series models for which OLS does not usually yield unbiased coefficient estimates. These include ARIMA and ARIMAX models, regression models with ARIMA errors, autoregressive distributed lag (ARDL) models, and dynamic models, i.e. including a lagged dependent variable on the right hand side, with or without lagged independent variables, where the error is autocorrelated even after including a lagged dependent variable.

Suppose you have estimated coefficients for such a model in an appropriate fashion. If you want to regularize this models to prevent over-fitting, say with LASSO or ridge regression, is it appropriate to treat the RHS variables as if they were all independent and estimate the tuning parameter via cross-validation in the standard way? Or do each of these models require an individual, specialized form of regularization?

andrewH
  • 2,587
  • 14
  • 27
  • I would guess that you'd need to do some kind of cross-validation that doesn't assume independence (first google search hit): https://otexts.com/fpp3/tscv.html – Ben Bolker Jul 04 '20 at 23:55
  • Slightly related: ["Regularization for ARIMA models"](https://stats.stackexchange.com/questions/152202). Also note that OLS cannot be used for ARIMA(X) models when the MA order is greater than zero, making some of your examples a little misleading. – Richard Hardy Jul 07 '20 at 12:53
  • What do you mean by *treat the RHS variables as if they were all independent and estimate the tuning parameter via cross-validation in the standard way?* LASSO, ridge and such do not assume the variables are all independent. Also, could you elaborate on *Or do each of these models require an individual, specialized form of regularization?* – Richard Hardy Jul 08 '20 at 19:11

0 Answers0