7

I am trying to fit an ARIMA model and I have already evaluated a few variations which I finally selected ARIMA(1,1,3) model. The residuals seems to be uncorrelated and all the lags are significant. However, in this model and even in all the others I tried, the normality condition for residuals is always violated and they look like this when plotted against normal distribution. Residuals against normal distribution

Should I transform my data somehow? I have already used natural log and first differencing in order to make the data stationary or can I ignore the assumption when I have a lot of observations (1,5M) ?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
m3div0
  • 569
  • 1
  • 7
  • 11
  • Why do you expect that errors should be normally distributed, whether or not the other checks you mention look good? There is no particular reason they should be. – Christoph Hanck Mar 24 '15 at 13:41
  • 1
    I have read in some works, that the normality of the residuals is imortant so that the t-statistics of the AR and MA terms are valid. As I understood, if the residuals wont be normally distributed I may mistakenly exclude some significant term or include some insignificant one. But I am not sure and therefore I came here to ask :) – m3div0 Mar 24 '15 at 13:49
  • Under very strict conditions, normality of the errors is important to get that t-statistics are indeed t-distributed. These are not met here anyhow, so that you need to rely on asymptotics to get a distribution for your t-statistics. Now, if you rely on asymptotics, normality of the errors is not (that) important anymore. – Christoph Hanck Mar 24 '15 at 14:15
  • 1
    I found two conflicting opinions here on Cross Validated: [Aksakal says](http://stats.stackexchange.com/questions/96598/what-if-the-residual-series-from-arima-not-looks-normal/96602#96602) that MLE is not reliable if the normality assumption is violated. [Rob J. Hyndman says](http://stats.stackexchange.com/questions/79400/does-arima-require-normally-distributed-errors-or-normally-distributed-input-dat/79793#79793) that non-normality is not that big of a problem. I am sure Rob J. Hyndman is very experienced and authoritative, I just wonder what the explanation for his observation is. – Richard Hardy Mar 24 '15 at 17:16
  • 1
    @ChristophHanck, your statements about asymptotics make sense in a linear regression setting. However, ARIMA models may be more sensitive to non-normality since the normality assumption is used in the maximum likelihood estimation (but not in OLS) and the ARIMA likelihood is quite complicated. Could you give an explanation or a reference supporting your opinion? I share the doubts of m3d1v0... – Richard Hardy Mar 24 '15 at 17:18
  • @RichardHardy, I'd first say it sure is true that asymptotics are often better guide in finite samples when errors are normal than if they come from, say, some weird skewed distribution. Second, I meant to refer to that MLEs are often also quasi-MLEs in that they often consistently estimate parameters of interest (with also a large-sample normal approximation) even if the likelihood is misspecified in that a wrong error distribution is chosen. A reference would be [Fuller (1996)](http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471552399.html), Theorem 8.4.1 (p. 432). – Christoph Hanck Mar 25 '15 at 09:29
  • I understand that. My main worry is whether quasi-MLE works for ARIMA. As you just noted, *MLEs are* ***often*** *also quasi-MLEs*, so I wonder if ARIMA belongs to the case where the normal MLE is a quasi-MLE. Would only the true erorr distribution matter when answering this question (I suspect a skewed error distribution would be a killer), or does it also depend on the model (like ARIMA, multiple linear regression etc)? – Richard Hardy Mar 25 '15 at 09:39
  • You are right about arIma, and indeed Fuller's theorem is about stationary time series. At least, White's (1982 Ecma) paper does not cover this case directly, but then, his are not necessary conditions. In short, I do not know. My hunch would be that it should also work, given that coefficients are typically estimated superconsistently in nonstationary time series models, but that really is just a guess. – Christoph Hanck Mar 25 '15 at 09:43
  • Actually, I was referring to stationary ARMA as well (the integratedness was not my concern, although perhaps it should have been?). Would you say (or at least guess) that MLE should work fine as a quasi-MLE when applied on a stationary ARMA? – Richard Hardy Mar 25 '15 at 16:49
  • I got a short comment by Rob J. Hyndman where he says *MLE with a Gaussian likelihood is asymptotically equivalent to least squares estimation. Minimizing the sum of squared errors should work ok for almost all distributions* [here](http://stats.stackexchange.com/questions/79400/does-arima-require-normally-distributed-errors-or-normally-distributed-input-dat/79793?noredirect=1#comment274002_79793). So perhaps you were right all along. – Richard Hardy Mar 25 '15 at 16:49

3 Answers3

2

Your QQplots could indicate $t$-distributed error terms might fit better. You could try to fit an ARIMA-model with $t$-distributed innovation terms, and see if the fit is very different from the fit you have now. I have done such things with the bugs software, there are certainly other options.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
1

Before you do an ARIMA model you have to check if the data is be stationary and if any seasonality should be defined using autocorrelation (ACF) and partial correlation functions(PACF). The auto correlation should follow the 95% confidence bands. Stationary data is detected using a run sequence plot or auto correlation.

If it is not stationary you might have to detrend it. My guess is it was not stationary.

ccsv
  • 1,008
  • 9
  • 17
  • Yes, that is exactly what I did. First I took a log transform because I am using exchange rate data and I wanted to smoothen a bit the excesive pikes. Then I run a Dickey-Fuller test which suggested that my series is not stationary, hence I differenced it by 1 and then it looked much beter visually and it passed the Dickey-Fuller. I also tried to regres the differenced values on various trends(linear,quadratice...) but all options turned to be insignificant. Therefore I think my data are stationary. Moreover, when I was evaluating the ARIMAs, there was no autorroelation in the residuals. – m3div0 Mar 24 '15 at 13:43
  • 1
    The wording of the answer may be a bit confusing, so let me make a couple of remarks. (1) Statistically significant ACF or PACF values do not mean that the underlying series is nonstationary. (2) There are multiple different forms of nonstationarity, and different methods to detect them. (Augmented) Dickey-Fuller test mentioned in the comment above is one option. Interpreting autocorrelations is less straightforward. – Richard Hardy Mar 24 '15 at 16:56
  • @m3d1v0 One doesn't/shouldn't take logs to "smoothen a bit excessive peaks" that is the role of pulse Indicators.. This post speaks to the role of power transforms of which a log is one choice http://stats.stackexchange.com/questions/18844/when-and-why-to-take-the-log-of-a-distribution-of-numbers . You have to carefully select the appropriate remedy for the problem at at hand. Transformations are like drugs ... some are good for you and some are not. – IrishStat Sep 27 '15 at 22:44
1

If the residuals contain pulses or level shifts this can lead to "non-normality" . Try detecting Interventions and add them as necessary. Another way residuals can exhibit non-normality is if there is a deterministic change in error variance suggesting Weighted Least Squares OR if the model's parameters are not constant over time suggesting data segmentation..

IrishStat
  • 27,906
  • 5
  • 29
  • 55