Order of ARMA models

Question

Why we usually do not exceed ARMA(5,5) models in practice? Is there any mathematical justification for this?

Answer 1

I can think of two and a half reasons.

1. The metaphysical reason: I understand why a process should depend on the previous realization, in an AR(1) model. Or even that $y_{t-2}$ may have an impact on $y_t$ above and beyond the influence from $y_{t-1}$ (which is already modeled by the first autoregressive parameter $\phi_1$), yielding an AR(2) model instead of AR(1). With higher and higher orders, I find it progressively harder to think of a decent justification for why there should still be an appreciable impact of the $p+1$st AR lag over and beyond what the previous $p$ lags already model.

   Of course, this does not apply to seasonal ARIMA, which you were not asking about.

2. The statistical reason: more parameters = more degrees of freedom = more model variance but lower bias = beyond some point, worse forecasts. The bias-variance tradeoff. Even if your data-generating process is AR(7), fitting a misspecified AR(5) model may yield better forecasts, in the sense of lower mean squared error. Fitting high AR orders requires large amounts of data.

Essentially the same points apply to moving average (MA) terms.

The half reason I can think of is that it is hard to think of real-life moving average processes at all, apart from misspecification.