5

I would like to use Fourier terms to model seasonality in an ARIMA model. The reason for using Fourier terms instead of a seasonal ARIMA model is that the frequency of the time series is very high (672) and that I want to model some special days as if they were different weekdays (e. g. I want to treat Easter Monday as if it was a Sunday). I first wanted to do that by using seasonal dummies but 671 seasonal dummies are probably to much. Thus, I want to use Fourier terms which I would adjust for the special days to get the correct regressors.

Now, I have two questions:

  1. Does anybody have a good reference for using fourier terms as regressors in ARIMA models? I only find online references like blogs (e. g. http://robjhyndman.com/hyndsight/dailydata/) but no paper or book I could cite.
  2. Does anybody have comments on whether this approach is useful or not?

Note: I have to use ARIMA models, so I do not need suggestions regarding alternative methods.

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
Lila
  • 83
  • 1
  • 6
  • If you have daily data then the frequency probably shouldn't be 672. 7 or 365 makes more sense. Post your data and also flag your holiday variables with a 1 on those days. – Tom Reilly Jun 02 '16 at 14:01
  • 1
    I do not have daily data but one data point every 15 minutes, thus I have 96 data points per day resulting in a frequency of 96 * 7 = 672 for weakly seasonality. I cannot use just 96 because weakdays are quite different. I am not allowed to make the data publicly available. I do not only have holidays but also other types of special days. The problem is that I do not have enough training data for using a variable for special days (e. g. in some training sets there is not even one special day). Thus, I would like to model the special days as if they were normal days but from a suitable weekday. – Lila Jun 02 '16 at 16:52

2 Answers2

6

  1. It appears to me that this approach is sufficiently intuitive that many people must have looked at it, but I can't locate a useful reference in my bib file, either. Searching for "Fourier ARIMA" or similar at the International Journal of Forecasting (IJF) does not yield anything very useful. Ludlow & Enders (2000, IJF) do combine ARIMA and Fourier terms, but not as regressors in the way you envisage.

    A similar search at Google Scholar turns up a couple thousand hits that you would need to refine. This older paper seems to use this approach (so it's been around for thirty years at least), but I'm not sure you want to cite it.

  2. I'd say this approach is eminently useful. Rob Hyndman seems to agree: Forecasting with long seasonal periods and Forecasting weekly data. I see that you have to use ARIMA models (why?), but note that he writes that TBATS performs comparably well. Rob's recent update to the forecast package is also relevant.

    (Don't disregard these because they are "just blog entries". Rob Hyndman is one of the forecasting gurus, highly active in the community, and the Chief Editor of the IJF. I'd trust anything he blogs more than much of what other people publish in journals.)

Community
  • 1
Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
  • Thank you for your answer! I also did the search in the IJF as well as in Google Scholar and did not find anything suitable. But the old paper you found might be a good starting point for me to search (maybe other papers with a similar approach cited it later). If I find a good reference I will post it here. Regarding your question why I have to use ARIMA: I am doing a comparison of different methods for a certain problem. So I do not only use ARIMA but I definitely want to include it in the comparison. – Lila Jun 02 '16 at 11:29
  • I will leave the question open for a few days in case somebody has a good reference at hand. If nobody comes up with a better reference I will accept your answer. – Lila Jun 02 '16 at 11:33
  • Leaving it open for a few days makes perfect sense. If you found my answer helpful, please consider [upvoting it](http://stats.stackexchange.com/help/why-vote). – Stephan Kolassa Jun 02 '16 at 14:33
  • I did upvote it but I do not have enough reputation points so my votes are not counted so far. Once I have enough points the upvote should become visible. – Lila Jun 02 '16 at 16:46
  • I accepted your answer as it was helpful and no one else seemed to have a better one. – Lila Jun 09 '16 at 17:55
2

Maybe you can look this https://medium.com/intive-developers/forecasting-time-series-with-multiple-seasonalities-using-tbats-in-python-398a00ac0e8a

This article compared tbats and SARIMAX with Fourier Terms .

The answer to your question :

SARIMAX with Fourier Terms One can apply a trick [4] to utilize exogenous variables in SARIMAX to model additional seasonalities with Fourier terms. We will keep modeling the weekly pattern with seasonal part of SARIMA. For the yearly seasonal pattern we will use the above-mentioned trick. I have compared multiple choices for the number of Fourier terms and 2 provides the most accurate forecasts. Therefore we shall use 2 Fourier terms as exogenous variables.

# prepare Fourier terms
exog = pd.DataFrame({'date': y.index})
exog = exog.set_index(pd.PeriodIndex(exog['date'], freq='D'))
exog['sin365'] = np.sin(2 * np.pi * exog.index.dayofyear / 365.25)
exog['cos365'] = np.cos(2 * np.pi * exog.index.dayofyear / 365.25)
exog['sin365_2'] = np.sin(4 * np.pi * exog.index.dayofyear / 365.25)
exog['cos365_2'] = np.cos(4 * np.pi * exog.index.dayofyear / 365.25)
exog = exog.drop(columns=['date'])
exog_to_train = exog.iloc[:(len(y)-365)]
exog_to_test = exog.iloc[(len(y)-365):]
# Fit model
arima_exog_model = auto_arima(y=y_to_train, exogenous=exog_to_train, seasonal=True, m=7)
# Forecast
y_arima_exog_forecast = arima_exog_model.predict(n_periods=365, exogenous=exog_to_test)
Mithril
  • 205
  • 1
  • 12