1

I've been using ARIMA modelling to predict the number of orders a business receives. I have data for 3 years, and the time series shows a strong (uneven) upward trend, with increasing variance over time. I can't share my data due to confidentiality issues, but here's a mockup of what it looks like over 3 years:[![An example of what my data looks like][1]][1]

So, the business is made up of offices in different cities. I'm using vectorized autoregressions to predict the order volumes in each city, and then summing them up to get total predicted order volumes. So for City 1, the endogenous variables are the past order volumes in City 1 - I've put in AR(1) and AR(52) terms. The exogenous variables here would be AR(1) and AR(52) terms in Cities 2 to 10, as well as a few dummy variables for month, year, public holidays, etc. I'm not using the VAR package in R - I'm manually running linear regressions to get the predicted values for the test sets in each city, so I have more control over the outputs and method.

I have logged and differenced the data to make it stationary. When I ran the model, I got an overall MAPE of 23% on my predictions, as well as odd swings in predicted values. See below:

[![Actual vs. Predicted][2]][2]

I then tried running the model on actual orders (ie. no logging or differencing). My MAPE was 12%, and there were no swings in the data. I got similar results for every country I ran the model on - MAPEs almost halving when I didn't log and/or difference the data. If it helps, I ran a KPSS test on the residuals here, and they are stationary around a constant mean. See below:

[![Actual vs. Predicted][3]][3]

I have a few questions:

1) Should I implement my model without accounting for stationarity since my predictions and MAPEs are so much better?

2) Is there a better way to account for trend and seasonality, since my data isn't a great example of cyclical data with a constant trend?

Any advice would be useful!

Arimariver
  • 11
  • 2
  • 1
    Are you assessing your MAPEs on holdout data or in-sample (your plots suggest the latter)? In-sample fit is not a good predictor for out-of-sample predictive accuracy. – Stephan Kolassa Jun 18 '19 at 12:52
  • Also, are you [bias-correcting](https://otexts.com/fpp2/transformations.html) when you backtransform the predictions from a model built on logged data? (Which should not be your underlying problem, because [the MAPE actually prefers biased forecasts](https://stats.stackexchange.com/q/299712/1352)). – Stephan Kolassa Jun 18 '19 at 12:54
  • After accounting for Stephan's excellent comments, I'd also suggest trying without the log transform, if you haven't already. In general, log transforms are justified by multiplicative (percentage) change rather than additive, and that may well be justified in this case, but maybe not. – Wayne Jun 18 '19 at 13:05
  • Hi Stephen. I’m assessing my MAPEs on holdout data - I used a rolling algorithm to split the data into train and test sets each week and project n weeks ahead. These charts compare 10-week ahead predictions to the actual values. I’ll look up bias-correcting as I wasn’t aware of it - thank you! – Arimariver Jun 18 '19 at 13:23
  • Hi Wayne, I’ve tried removing the log transform but keeping the differencing - this improved my MAPEs slightly, but not as significantly as removing both the logging and differencing. Furthermore, this caused my residuals to fail the kpss test. – Arimariver Jun 18 '19 at 13:28
  • Have you considered using weekly dummies and intervention detection to deal with pulses (unusual values) ? – IrishStat Jun 18 '19 at 20:10

0 Answers0