3

I'm just starting to look into exponential smoothing models. Is there a way to fit a linear regression with exponentially-smoothed errors, similar to the standard technique of fitting a regression with ARMA errors? Or is this generally considered a bad idea?

Of course, I can come up with heuristics, like fit with AR(1) errors, then exp-smooth the residuals (possibly on the differences, then look for a trend) but that feels very ad hoc.

Johann Hibschman
  • 358
  • 1
  • 8
  • Like you said very very very ad hoc "like fit with AR(1) errors, then exp-smooth the residuals (possibly on the differences, then look for a trend) but that feels very ad hoc." and very probably just wrong ! – IrishStat Jun 04 '12 at 22:18

3 Answers3

1

What you want to do is to investigate Transfer Function Models ( also known as Dynamic Regression ) which seamlessly integrates "regression" and "arima" . If you were to assume that the arima portion was a simple exponential smoothing model AND the regression component to be a simple contemporaneous relationship you would have your objective. This of course could be silly as you are assuming the model structure. In general you would want to identify the best of these components making sure that pulses, level shifts , seasonal pulses and local time trends were included when necessary. Additionally care should be taken to ensure that the model parameters and errror variance are "proven" to be constant over time. If you had some real data we could take this discussion further.

IrishStat
  • 27,906
  • 5
  • 29
  • 55
  • Thanks, that gives some food for thought. Do you have any good links describing transfer function models? My current idea is to put together a state-space model (West&Harrison- or Durbin&Koopman-style), plug in a diffuse prior, no state variance (except on the constant and trend), and turn the crank, but that method tends to depend a lot on the exact variances chosen. – Johann Hibschman Jun 04 '12 at 18:26
  • you might want to look at http://stats.stackexchange.com/questions/19532/what-transfer-function-should-i-use-to-model-a-gradual-permanent-step-positive-i and others that I have commented on . http://stats.stackexchange.com/questions/1459/how-to-identify-transfer-functions-in-a-time-series-regression-forecasting-model http://stats.stackexchange.com/questions/8738/what-is-the-term-for-a-time-series-regression-having-more-than-one-predictor – IrishStat Jun 04 '12 at 19:33
  • more: If you form the Transfer Function Model y(t)=W(B)*X(t)+[THETA(B)/PHI(B)]*a(t) the operator [THETA(B)/PHI(B)] is the "smoothing component". For examnple if PHI(B)=1.0 and THETA(B)=1-.5B this would imply a set of weights of .5,.25,.125,... . in this way you could provide the answer to optimizing the "weighted moving linear regression" rather than assuming it's form. From http://stats.stackexchange.com/questions/9931/exponentially-weighted-moving-linear-regression – IrishStat Jun 04 '12 at 20:18
  • upon reflection perhaps PHI(B) should be [1-B] or the y and x be the first order differences of the observed Y and X – IrishStat Jun 04 '12 at 20:36
1

If you're thinking mainly about an EW version of an MA model, I'm not sure why this would be necessary. In general MA models are more of a pain to estimate since they rely on MLE. Oftentimes, people would just throw in sufficient lags in an AR model to cover the MA structure. If the worry is about fitting a particular PACF structure, then it might be appropriate to estimate fractionally integrated models.

John
  • 2,117
  • 16
  • 24
  • What I want is a regression setting a baseline, then looking for a slow-moving error trend on top of that. That's hard to turn into a straight AR or MA model. GARCH seems like a solution in search of a problem here, as usual; I'm not concerned about getting the volatility right, just the baseline. – Johann Hibschman Jun 04 '12 at 18:33
  • I think ARIFMA (fractional integration) might suit your needs then. That type of model is meant to handle long memory in the conditional mean. No need for the snark, but you weren't particularly clear in your question. – John Jun 04 '12 at 19:09
  • Thanks, I'll look into ARIFMA. Not trying to be snarky, but I'm a bit of a GARCH-skeptic. Probably comes from mostly looking at monthly data, rather than daily or intraday, where improved vol modeling doesn't pay off. Mortgages, housing, and econometrics, oh my. – Johann Hibschman Jun 04 '12 at 19:35
  • In my experience, it depends on the application. I've used Garch with daily, but intraday gets tricky. – John Jun 04 '12 at 19:47
0

It is perfectly reasonable to model the residuals of a regression model as a stationary ARMA model. This is done with harmonic regression. Exponential smoothing is IMA(1,1). So it would imply nonstationary residuals from the regression.

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • The obvious problem with this is that it is a two step approach rather than a simultaneous estimation ala a Transfer Function , but that's just my opinion. – IrishStat Jun 04 '12 at 20:24
  • @IrishStat I would disagree if the residual model were stationary. However, since exponential smoothing is nonstationary and the mean drifts the trend in the residual might have better been handled in the regression piece. – Michael R. Chernick Jun 04 '12 at 20:33