7

Why differencing a normal, i.i.d. time series $X$, generates a negatively correlated series $Y$?

> Acf(rnorm(5000, 0, 40))
> Acf(diff(rnorm(5000, 0, 40)))

enter image description here

I stumbled on this problem while looking at a time-series that I wanted to predict using an ARIMA model. The original series appeared to be non-stationary by looking at the plot, so I decided to apply a difference at lag 1 and check the acf/pacf. This seemed to indicate an MA(1) model but obviously there is something I'm missing.

enter image description here

Gavin Simpson
  • 37,567
  • 5
  • 110
  • 153
Robert Kubrick
  • 4,078
  • 8
  • 38
  • 55
  • 1
    **Hint**: $Y_n = X_n - X_{n-1}$ so $Y_n$ has zero mean. So, what's $\mathbb E Y_n Y_{n-1}$? Initial conclusion? Now check the autocorrelation at further lags. Final conclusion? – cardinal May 11 '12 at 13:06
  • $EY_n|Y_{n-1}$ should be something like $-Y_{n-1}$, if $Y_n$ has 0 mean? But then so is $X_n$. It has 0 mean. I still don't see it. – Robert Kubrick May 11 '12 at 13:30
  • Also, more in general isn't this a common ARIMA identification issue that should be described in any time-series book? – Robert Kubrick May 11 '12 at 13:32
  • I was asking about $\mathbb E(Y_n Y_{n-1})$ not $\mathbb E(Y_n \mid Y_{n-1})$. Write out the terms and use the iid property. It should be clear. Yes, you should find such discussions in an applied time-series book, e.g., Box, Jenkens & Reinsel or R. Tsay. – cardinal May 11 '12 at 13:40
  • What is $E(Y_nY_{n-1})$? Both $X_n$ and $Y_n$ are iid, so whatever iid conclusion I come up about $Y$ is also valid for $X$. – Robert Kubrick May 11 '12 at 13:47
  • Actually I have to correct my previous comment: $Y_n$ is *not* iid, given the strong acf at lag 1. So my question at this point is why would I have to look into idd properties? – Robert Kubrick May 11 '12 at 13:54
  • 1
    Robert: $\mathbb E Y_n Y_{n-1} = \mathbb E(X_n - X_{n-1})(X_{n-1}-X_{n-2})$. Can you continue by using the stated properties of $X_n$? – cardinal May 11 '12 at 13:56
  • @RobertKubrick: Not sure I understand your second comment to cardinal, but if I do, this appears to me to be "over-differencing" which I've seen mentioned in various references. – Wayne May 11 '12 at 13:56
  • @Wayne I have Hyndman and Tsay 2004 and I don't call any mention to over-differencing. I might have missed it but it's such a trap that it should have its own mini-section in an introductory textbook imo. Which second comment were you referring to? – Robert Kubrick May 11 '12 at 14:10
  • 2
    @cardinal I appreciate your interest in my question, but this is not a classroom, or even a chat room. If you know the answer, please post an explanation and I will work from there to understand the properties. – Robert Kubrick May 11 '12 at 14:29
  • 2
    +1 - these comments are *not* meant for a discussion (however well-intentioned) designed to help the OP find the answer for themselves. CV isn't a site for teaching - if a simple hint isn't sufficient either leave it at that or write up an Answer. – Gavin Simpson May 11 '12 at 14:51

2 Answers2

7

If I understand your comments correctly, you've overdifferenced, which is talked about in various guides.

EDIT: Your original series of numbers (rnorm(5000, 0, 40)) has, by definition and design, no relationship between adjacent numbers or every 2nd number or every 3rd number. It's "random" (pseudo-random, but not distinguishable from truly random by us mere mortals). So the ACF you calculate is random garbage.

But differencing takes that series of numbers and creates a new series which is related in a particular, deterministic way: subtraction of adjacent values. Consider your initial random number series: $(n_1, n_2, n_3, ...)$, then difference it to get $(d_1, d_2, ...)$. Both $d_1$ and $d_2$ are calculated using $n_2$, so you've now introduced autocorrelation at lag 1.

Now look at what happens at that lag 1. $n_2$ is used to calculate $d_1$ and $d_2$, once subtracting from and once being subtracted from. [Begin I'm-way-in-over-my-head part.] In order for $d_1$ and $d_2$ to have the same sign, we'd need to have $n_1 < n_2$ and $n_2 < n_3$ (or vice versa), which is less likely than the alternatives, so we expect that the autocorrelation will be negative. [End I'm-way-in-over-my-head part, gasping for air.]

Wayne
  • 19,981
  • 4
  • 50
  • 99
  • Thanks, very clear explanation. By the way, I checked Box and Jennings 2008, there is no note about overdifferencing in this context. There are a few references to other papers/books in the appendix but I'm not sure it's even related to this case. – Robert Kubrick May 11 '12 at 21:53
  • I nice online guide that mentions overdifferencing is: http://people.duke.edu/~rnau/411arim2.htm – Wayne Dec 14 '13 at 18:56
3

It is a little late ..... but , review the Slutsky Effect where a linear (weighted ) combinations of i.i.d. values leads to a series with auto-correlative structure. This is why assuming any filter picket out of the blue can be dangerous. X11-ARIMA assumes a 16 period equally weighted average ( you can change 16 to another integer ) to smooth the series not knowing the impact of assumed filters. Long live analytics !

IrishStat
  • 27,906
  • 5
  • 29
  • 55