3

A stochastic process is second order weakly stationary if all random variables have same mean (first moment), and same variance (second moment?), and covariances that are time-invariant (second moment as well?). I expected this definition to contain a statement about the mean (first moment) and the variance (second moment). How does covariance enter? Is covariance a moment? When is variance the second moment and and when is it an incomplete description of a second moment? Please clarify.

ColorStatistics
  • 2,699
  • 1
  • 10
  • 26
  • 2
    You are looking at defective definitions of what is meant by weak stationarity (also called wide-sense stationarity in the literature). See my answer to [this question](https://stats.stackexchange.com/q/65353/6633) and an answer of mine on dsp.SE that is referenced therein. – Dilip Sarwate Nov 29 '18 at 20:16
  • Thank you, Dilip. I looked at the referenced response. What I am not clear about is the relationship between "wide-sense-stationarity",(which I understand to be what others call "covariance stationarity" or "weak stationarity") and "stationary to order 2". What else in addition to "stationary to order 2" is assumed to get to "wide-sense-stationarity"? My question above is meant to be about covariance stationarity. – ColorStatistics Nov 29 '18 at 20:42
  • 1
    See [this answer of mine](https://dsp.stackexchange.com/a/38744/235) also over on dsp.SE for an example of a weakly stationary process that is not even stationary to order 1 let alone order 2. Weak stationarity _does_ say something specific about pairs of random variables separated in time by a fixed amount of time $\tau$: **all such pairs** of random variables must have the same _covariance_. (Pairs separated by a different amount of time will typically have a different covariance). A special case for $\tau=0$, covariance = variance and so all the random variables have the same variance. – Dilip Sarwate Nov 29 '18 at 21:12
  • In short, your question "how does covariance enter?" makes no sense if you use the generally accepted definition of weak stationarity: the covariance is right there as part of the definition of weak stationarity. – Dilip Sarwate Nov 29 '18 at 21:14
  • Got it. You're saying a process is weakly stationary if all random variables have the same mean, variance of random variables is finite, and the autocovariance depends only on the dispersion (tau) and does not depend on time (t). This definition would then imply (because autocovariance at dispersion zero is the variance and it depends only on the dispersion - tau) that the variance of all random variables is constant. – ColorStatistics Nov 29 '18 at 22:11
  • Am I correct to then conclude that to define a weakly stationary process as one for which all random variables have same mean, and same finite variance, and all autocovariances are time-invariant and depend on dispersion tau only - is perfectly correct, albeit repetitive because "same variance" is implied by autocovariance not depending on time t? – ColorStatistics Nov 29 '18 at 22:30
  • 1
    Yes, you are correct. – Dilip Sarwate Nov 30 '18 at 12:30

1 Answers1

1

The covariances are part of the second-order moment: In time-series problems you are dealing with random vectors indexed by time. When you have a random vector $\mathbf{X} = (X_1,...,X_n)$ its first and second-order moments are the mean vector and covariance matrix:

$$\mathbb{E}(\mathbf{X}) = \begin{bmatrix} \mathbb{E}(X_1) \\ \mathbb{E}(X_2) \\ \vdots \\ \mathbb{E}(X_n) \end{bmatrix} \quad \quad \quad \mathbb{V}(\mathbf{X}) = \begin{bmatrix} \mathbb{V}(X_1) & \mathbb{C}(X_1, X_2) & \cdots & \mathbb{C}(X_1, X_n) \\ \mathbb{C}(X_2, X_1) & \mathbb{V}(X_2) & \cdots & \mathbb{C}(X_2, X_n) \\ \vdots & \vdots & \ddots & \vdots \\ \mathbb{C}(X_n, X_1) & \mathbb{C}(X_n, X_2) & \cdots & \mathbb{V}(X_n) \end{bmatrix}.$$

As you can see, the variance matrix is composed of the variances of the individual elements of the random vector and also their covariances. Moreover, the scalar variance values are just covariances of the elements with themselves. Second-order stationarity imposes a condition on these moments that they are affected by the time indices only through the time-lag between values.

Ben
  • 91,027
  • 3
  • 150
  • 376
  • If weak stationarity/second-order stationarity only imposes that the first moment E(X) and second moment V(X) do not depend on time t, then am I to conclude that weak stationarity says nothing about all the elements of E(X) being identical [i.e. E(X1)=E(X2)=...=E(Xn)], because couldn't all these elements not depend on time and yet be unequal. And similarly am I to conclude that weak stationarity says nothing about the diagonal elements of V(X), the variances, being identical [i.e. V(X1)=V(X2)=...=V(Xn)], because couldn't they not depend on time and yet be unequal? – ColorStatistics Nov 30 '18 at 19:12
  • 1
    @ColorStatistics: The index is what we mean by "time", so stationarity would mean that $\mathbb{E}(X_t)$ does not depend on $t$. That immediately gives you $\mathbb{E}(X_1) = \cdots = \mathbb{E}(X_n)$ and $\mathbb{V}(X_1) = \cdots = \mathbb{V}(X_n)$. The covariances may depend on time only through the time-lag (i.e., index lag) between elements. – Ben Nov 30 '18 at 22:23
  • Thank you for this clarification, Ben. It all makes sense now. – ColorStatistics Nov 30 '18 at 22:26