Stationarity and Ergodicity - links

Question

In time series analysis stationarity and ergodicity have different definitions and meanings:

https://en.wikipedia.org/wiki/Stationary_process

https://en.wikipedia.org/wiki/Ergodic_process

Essentially stationarity deals with the stability of an entire distribution (in a strict sense) or the first two moments (in a weak sense) given a temporal shift. However, ergodicity is need in order to give us the possibility of inferring population characteristics from just one finite sample. More precisely, ergodicity, for some moments, warrants that these sample moments converge to exact moments.

Is possible to write examples where stationarity hold but ergodicity not. In Hamilton – Time Series Analysis (1994, page 47) there is an example where the process is stationary (weakly and strictly) but not ergodic for the mean. So the sample mean is a biased estimator for the exact mean. Also from this example we can realize that ergodicity implies finite memory of the process.

However this example is given in order to underscore that this sentences (same page):

For many applications, stationarity and ergodicity turn out to amount to the same requirements.

This conflates the two concepts, rather than keeping them separate.

However, in my experience the stationarity condition is much more widely known and considered by practitioners than the ergodicity condition. In fact, several tests for stationarity are widely used, but I have never seen (a direct) test for ergodicity.

For example, in the widely used $AR(1)$ process

$$y_t = \theta_0 + \theta_1 y_{t} + \epsilon_t$$

the weak stationarity condition ($0<|\theta_1|<1$) implies ergodicity for the mean also. Is not rare to read that stationarity implies low persistency, see white noise vs random walk example. We can extend this rule on the general class of ARIMA models (see here: Why is ergodicity not a requirement for ARIMA models besides stationarity?). Therefore stationarity seems to deal with memory also.

Question: considering that ARIMA models represent the cornerstone of time series analysis, the simplification/conflation above seem me much more than an detail. Does there exist a relevant class of time series model where stationarity and ergodicity, in some form, are implied from clearly different conditions? Are there are some examples on real data? Are there graphs which could be useful for developing some intuition?

Answer 1

Ergodicity is a property defined for strictly stationary processes, i.e. an ergodic process is by definition strictly stationary.

Note The property being shown by the answer in Why is ergodicity not a requirement for ARIMA models besides stationarity? is mean-ergodicity, which is a much weaker property than ergodicity. There are examples of ARMA processes which are not ergodic.

Every strictly stationary process $x_t$, $t = 1, 2, \cdots$, admits canonical representation $x_t(\omega) = S^t(\omega)$ for some shift transformation $S$ defined on probability space $\Omega$. A strictly stationary process $x_t$, $t = 1, 2, \cdots$, is then ergodic if $S$ has no non-trivial invariant sets (up to measure zero). See a related discussion here.

(It's a result that a strictly stationary $x_t$, $t = 1, 2, \cdots$, is ergodic if and only if a strong LLN holds for $f(x_t)$, $t = 1, 2, \cdots$, for any $f \in L^1(\Omega)$, i.e. $ \lim_{n \rightarrow \infty} \frac{1}{n} \sum_{t=1}^n f(x_t) = E[f(x_1)] $ almost surely.)

Is [it] possible to write examples where stationarity hold but ergodicity not?

This is immediate from the definition. Take two strictly stationary processes $y_t$ and $z_t$ with different distributions. Define, for some $0<p<1$, $$ x_t= \begin{cases} y_t,& \mbox{with probability } p \\ z_t,& \mbox{with probability } 1- p \end{cases}. $$ Then $x_t$ is strictly stationary but not ergodic. In fact every strictly stationary nonergodic process admits such a decomposition (just restrict $S$ to a non-trivial invariant set).

...ergodicity imply finite memory of the process...

That's incorrect. (What is true is that non-ergodicity should imply long memory for any proposed definition of "long memory".)

The phenomenon of long memory, or infinite memory, was first observed in fractional Gaussian noise (FGN), which is a strictly stationary ergodic Gaussian process. In fact, the FGN satisfies the mixing property---which is a still stronger requirement than ergodicity.

Various attempts have been made to divide ergodic processes into short ant long memory. Given the FGN example, a candidate definition of "short memory" should imply the mixing property (and everything else that's not short memory would be long memory). This motivated the introduction of strong mixing-type properties---e.g. $\alpha$-mixing, $\phi$-mixing, etc.

For many applications, stationarity and ergodicity turn out to amount to the same requirements...

Stationarity and ergodicity are trivially not the same. Colloquially one might take them to be "the same" due to background/interest of audience, limitation of data/techniques, etc.

...considering that ARIMA models represent the cornerstone of time series...Exist a relevant class of time series model where stationarity and ergodicity, in some form, are implied from clearly different condition?

(We consider ARMA models because they are useful representations---with interpretable building blocks---of a (proper) sub-family of weakly stationary time series.)

A Gaussian stationary time series if ergodic if and only if its spectral measure is absolutely continuous with respect to the Lebesgue measure (the "only if" part is easy to see). For example, a AR(1) with Gaussian innovations is ergodic---in fact it is mixing.

You may find more general characterizations of ergodic/mixing properties for sub-families of weakly stationary series in the literature. They usually involve frequency-domain conditions.