8

Say I have a time series, Gt, and a covariate Bt. I want to find the relationship between them by the ARMA model:

Gt = Zt + β0 + β1Bt

where the residual Zt follows some ARMA process.

The problem is: I know for sure that β0 and β1 varies with the time of the year. Yet I do not want to fit a separate model to each month because that introduces discontinuity into my time series, which means I cannot calculate the autocorrelation function of the final residuals.

So, is there a time series model (or family of models, I wonder) that allows the correlation coefficients of its covariates to change seasonally?

========================

Edit: Thank you for those who replied here. I decided to just use seasonal dummies, but got busy so failed to reply in time.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
eddieisnutty
  • 211
  • 1
  • 5
  • No this is not a dumb question, when you mean "change seasonality", do you mean seasonality changes over time and is not constant? if that is the case you need a model that handles stochastic seasonality, dummy coding will not work as it handles only deterministic seasonality. See my earlier [question](http://stats.stackexchange.com/questions/103193/stochastic-vs-deterministic-trend-seasonality-in-time-series-forecasting). Simply model $Z_t$ as ARIMA (p,d,q)(P,D,Q) this should do it. – forecaster Feb 17 '15 at 20:24

4 Answers4

6

Edit (The same idea was proposed by Stephan Kolassa a few minutes before I posted my answer. The answer below can still give you some relevant details.)

You could use seasonal dummies. For simplicity I illustrate this for a quarterly time series. Seasonal dummies are indicator variables for each season. The $i$-th seasonal dummy takes on the value 1 for those observations related to season $i$ and 0 otherwise. For a quarterly series the seasonal dummies, $SD$, are defined as follows:

\begin{eqnarray} SD = \left[ \begin{array}{cccc} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{array} \right] \quad SDB = \left[ \begin{array}{cccc} B_{1} & 0 & 0 & 0 \\ 0 & B_{2} & 0 & 0 \\ 0 & 0 & B_{3} & 0 \\ 0 & 0 & 0 & B_{4} \\ B_{5} & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots \\ B_{n-3} & 0 & 0 & 0 \\ 0 & B_{n-2} & 0 & 0 \\ 0 & 0 & B_{n-1} & 0 \\ 0 & 0 & 0 & B_{n} \\ \end{array} \right] \end{eqnarray}

You can multiply each column in $SD$ by your explanatory variable $B_t$ and get the matrix $SDB$ defined above.

Then, you can specify your model as follows:

$$ G_t = Z_t + \beta_{0,s} SD_t + \beta_{1,s} SDB_t \,, $$

where the index $s$ indicates the season. Observe that we now have four coefficients (12 in your monthly series) $\beta_{1,s}$, one for each column in $SDB$.

The same for the intercept $\beta_0$ except that we must remove one column in $SD$ in order to avoid perfect collinearity. In a monthly series you would include for example the first 11 seasonal intercepts in $SD$.

Fitting the model for example by maximum likelihood will give you one coefficient estimate for each season. You could also test whether $\beta_{0,s}$ are the same for all $s$ or similarly if $\beta_{1,s}$ are constant across seasons.

javlacalle
  • 11,184
  • 27
  • 53
  • 1
    +1. Although you don't want to fit using Ordinary Least Squares if you have ARMA errors. – Stephan Kolassa Feb 17 '15 at 08:38
  • 1
    @javlacalle +1, Can we simply use $Z_t$ as ARIMA(p,d,q)(P,D,Q) instead of seasonal dummies to capture seasonality? That way you also account for stochastic seasonality in addition to deterministic seasonality. While this does not address the OP question on seasonality as regression coefficients it might be worthwhile to highlight the difference. – forecaster Feb 17 '15 at 20:20
  • @forecaster It depends on where the seasonal pattern comes from (from the influence of $B_t$, the structure of $Z_t$ or both). If $Z_t$ follows a seasonal pattern we should do what you mention. However, I left the term $Z_t$ aside because I think the OP is interested in the coefficients $\beta_0$ and $\beta_1$ and possible changes across seasons, probably due to a suspicion based on knowledge about the data. – javlacalle Feb 17 '15 at 23:08
  • 1
    @forecaster I think the pursue of the OP is to measure the influence of $B_t$ on $G_t$ at different seasons. This could be captured by allowing seasonally-varying coefficients, $\beta_{s,1}$. If $\beta_1$ is constant for all seasons then we are not able to measure the effect of $B_t$ at each season and test if differences are significant. Moreover, if $\beta_1$ is fixed, observing seasonality in the residuals could mean that there is a seasonal effect not captured by a single coefficient $\beta_1$, rather than the need to extend the model for $Z_t$ by means of a seasonal ARIMA model. – javlacalle Feb 17 '15 at 23:10
  • You mention deleting a column of $SD$ for $\beta_0$. This is unclear to me. If you delete one column, then whenever that season comes up won't you have $G_t = \beta_1,s B_t + Z_t$? Doesn't this mean you set the intercept to zero for season $s$? – Frank Jun 23 '19 at 14:13
  • 1
    @Frank The intercept is set to zero for the season that is left out. The coefficients of the intercepts related to the remaining coefficients are interpreted as a change with respect to the average value of the deleted season (which is no necessarily zero, but the value determined by the coefficients and values of the remaining variables at that season). – javlacalle Jun 23 '19 at 18:54
  • @javlacalle Thank you for the reply! If you leave out that season, you get $G_t = \beta_{1,s}B_t + Z_t$, and if $B_t = 0$ than $G_t = Z_t$ and $E(G_t) = 0$. So I think that you are saying $0$ is fine because the results are interpreted as a change in value from the left out season. In that case, how would you predict the true $G_t$? Would it be the expectation of the left out season * $(1+G_t)$? – Frank Jun 24 '19 at 23:06
  • 1
    @Frank If 11 colums are used in $SDB$, then you would in principle include a constant $\alpha$ (a column of ones); otherwise, the residuals may not be zero on average: $G_t = \alpha + Z_t + \beta_{0,s} SD_t + \beta_{1,s} SDB_t$. At the 12-th season (the one left out), the expected value of $G_t$ is $\alpha+\beta_{1,12}SDB_t$. The coefficients $\beta_{0,s}$, $s=1,\dots,11$ are interpreted as changes with respect to the estimate of $\alpha$. – javlacalle Jun 25 '19 at 08:53
5

Certainly there is. Simply include monthly dummies in an interaction with $B_t$. Let $M_{tm}$ denote a dummy that is 1 if time $t$ corresponds to month $m$ and 0 otherwise. Then fit the following regression with ARMA errors:

$$ G_t = \beta M_{t\cdot} + \gamma B_tM_{t\cdot} + Z_t $$

where $Z_t$ is ARMA(p,q) and $\beta$ and $\gamma$ are parameter vectors of length 12.

You can do the actual fitting using R with the nlme package, using the gls() function and specifying a corARMA() correlation structure.

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
  • What if you don't have many data points and want to preserve parameters? Is there a way to subtract a season while keeping parameters to a minimum? – Frank Jun 22 '19 at 12:08
  • 1
    @Frank: if we have too little data to support a complex model, then I personally would look to regularization, like the lasso, the elastic net or Bayesian approaches. – Stephan Kolassa Jun 22 '19 at 21:04
  • Thank you for replying on such an old question. May I ask, should $\beta M_t$ and $\gamma B_t M_t$ each have 12 terms? Or should $\beta M_t$ have 11 terms? I learned about "the dummy variable trap," but I cant find a reference that clearly discusses this case. For example, would this model work? Or do I need to decrease the length of $\beta$ vector by 1? $Y_t = \beta M_t + \gamma B_t M_t + f(t) + Z_t$ – Frank Jun 23 '19 at 14:10
  • 1
    @Frank: yes, both should have 12 terms *since there is no intercept*. If you remove one term, say $\beta_1$, that means that the mean in month $1$ for $B_t=0$ should be zero, which will usually not make sense. Alternatively, you could include an intercept and a term for $B_t$ as a main effect (without an interaction with $M$), then leave one entry out of both $\beta$ and $\gamma$ - which would give you $1+1+11+11=24$ parameters, exactly as many as the model I propose. It's just a reparameterization. The model you propose in your comment works (assuming a deterministic $f$). – Stephan Kolassa Jun 25 '19 at 05:55
  • Thanks! In that case, are all of these models equivalent? $\\ Y_t = \sum_{i=1}^{12} SD_i * [\beta_i +\gamma_i x_i] + \epsilon_t = \\ Y_t = \beta_0 + \sum_{i=1}^{11} SD_i * \beta_i + \sum_{i=1}^{12} SD_i \gamma_i x_i + \epsilon_t = \\ Y_t = \beta_0 + \gamma_0 x_i + \sum_{i=1}^{11} SD_i * [\beta_i + \gamma_i x_i] + \epsilon_t \\ \text{ Where} SD_i \text{ is a indicator variable which is $1$ at season $i$,}\\ \text{ $\epsilon$ is the error, $\beta$ and $\gamma$ are parameters, }\\ \text{$x_i$ is an external regressor, and $Y_t$ is the dependent variable}$ – Frank Jun 25 '19 at 23:29
  • 1
    They should be, yes – Stephan Kolassa Jun 27 '19 at 08:53
4

If you don't want to discretise the seasonal effect, you could assume that the regression coefficients vary in a cyclic manner as a function of the time of year, i.e. $\beta_0(t) = w_0 + w_1\sin nt + w_2\cos nt$ and $\beta_1(t) = w_3 + w_4\sin nt + w_5\cos nt$, then if you substitute these into your linear model, you should get something of the form

$G_t = Z_t + w_o + w_1\sin nt + w_2\cos nt + w_3B_t + w_4B_t\sin nt + w_5B_t\cos nt$

You could fit this model by using OLS regression (or whatever method you are already using) with the additional covariates $\sin nt$, $\cos nt$, $B_t\sin nt$ and $B_t\cos nt$, where $n$ is whatever constant you need to represent a year ($2\pi/365$ for a daily time-series).

This wouldn't introduce any discontinuities in the model as the seasonality in the regression coefficients are smooth functions of time. I suspect if you added sine and cosine components representing harmonics of the annual cycle you could model deviations from simple sinusoidal variation in the regression coefficients (Fourier series type approach).

Caveat: Been a long day, so I may have made a stupid error somewhere.

Dikran Marsupial
  • 46,962
  • 5
  • 121
  • 178
  • (+1) A trigonometric approach is an interesting alternative. Another appeal of the trigonometric approach is that it may require fewer parameters. Your equation uses 6 parameters against 11+12=23 in the approach that I discussed in my answer. In practice we would probably need to include in addition to the fundamental seasonal frequency ($2\pi/12$ in a monthly series) some of its harmonics, which will require more parameters. But we may get a reasonable fit without including all the harmonics and hence the number of parameters to be estimated can be reduced. – javlacalle Feb 17 '15 at 23:12
  • A disadvantage that I see is that the interpretation is less straightforward in the context of a regression model. The interpretation of the 0-1 seasonal dummies can be made in terms of months rather than cycles of seasonal periodicity. We may conclude for example that the effect of temperature on sales of a certain product is the highest in August and has no major effect in March. In the trigonometric approach we would conclude for example that the effect of temperature on sales follows a cycle that is repeated every 6 months. The former interpretation may be more informative. – javlacalle Feb 17 '15 at 23:18
  • You could still do that with this approach, you could plot the variation in each $\beta_0$ and $\beta_1$ by a weighted sum of the sine and cosine components, and you could discretise that to see how sales vary by month. The original question suggested that discontinuities were not wanted, which implies a smooth variation. At the end of the day, the right approach depends on what it is you are trying to find out. – Dikran Marsupial Feb 18 '15 at 08:38
  • 1
    As far as I understood, the concern of the OP was with discontinuities in the residuals, fitting 12 regression models (one for each month) will lead to 12 series of residuals instead of one series of residuals where to carry out some diagnostics looking at their autocorrelations. Both the 0-1 dummies and the trigonometric dummies would be an appropriate way to deal with this issue. Which one is a more natural approach would depend, as you say, on the purpose of the analysis and the kind of information that is wanted. – javlacalle Feb 18 '15 at 09:02
  • Let's underline that the question is general and only the tag `econometrics` discloses the OP's interest in that side. For environmental time series data the trigonometric approach is often highly successful and natural, while conversely months have little or no meaning even if the data are reported in that way. – Nick Cox Apr 08 '17 at 00:12
2

Fit the mean and the harmonics of the seasonal cycle to the time series of x and y. These provide the intercept terms. Then, subtract them from x and y to create anomalies. Use these anomalies x' and y' to compute seasonally varying regression slope coefficients: Fit the array product between the x' and y' with the mean and leading harmonics to the seasonal cycle. Do the same for the variance of the x'. Then divide the seasonal cycle fit to the covariance by the seasonal cycle fit to the variance to provide continuously evolving slope coefficients. For details, see http://onlinelibrary.wiley.com/doi/10.1002/qj.3054/full

Paul Roundy
  • 21
  • 2