non-invertible weakly stationary AR processes

Question

ok, so I know (for example) that $X_t = 2X_{t-1} + e_t$ for iid $e_t$ is stationary.

But how do I go about proving the condition $Cov(X_r, X_s) = Cov(X_{r+t}, X_{s+t})$ for all integer $r, s, t$ without having to resort to the transformation which requires $X_t$ to depend on the future rather than the past?

Edit: For clarification, the above example can be rewritten as:

$X_t = \frac{1}{2} X_{t+1} + u_t$, with $u_t$ some new white noise process. Now we can write down the infinite MA representation of this process, that is,

$X_t = \sum_{j=0}^{\infty} (\frac{1}{2})^{j+1} u_{t+j}$.

And from this, we can show that the series is stationary. But now, the series depends on the future rather than the past.

Answer 1

@Hamed has already pointed out to you that the process is not stationary (as you claim in the text of your question) nor weakly stationary (as in the "Subject" line of your posting). Rather than trying to understand his point, you put him down by claiming that he does not "grasp the concepts." A -1 for that.

From $X_t = 2X_{t-1} + e_t$, we can deduce that

\begin{align} X_{r+t} &= 2X_{r+t-1} + e_{r+t}\\ &= 2[2X_{r+t-2} +e_{r+t-1}] + e_{r+t}\\ &= 4X_{r+t-2} + 2e_{r+t-1} + e_{r+t}\\ &= 4[2X_{r+t-3} +e_{r+t-2}] + 2e_{r+t-1} + e_{r+t}\\ &= 8X_{r+t-3} + 4e_{r+t-2}+ 2e_{r+t-1} + e_{r+t}\\ &= \cdots\\ X_{r+t} &= 2^tX_r + \sum_{i=0}^{t-1} 2^{i}e_{r+t-i}\\ \text{Similarly}\qquad \qquad&\\ X_{s+t} &= 2^tX_s + \sum_{i=0}^{t-1} 2^{i}e_{s+t-i} \end{align} The bilinearity of covariance result $$\operatorname{cov}\left(\sum_i a_iX_i, \sum_j b_j Y_j\right) = \sum_i \sum_j a_i b_j \operatorname{cov}\left(X_i, Y_j\right)$$ shows that $$\operatorname{cov}\left(X_{r+t}, X_{s+t}\right) = 2^{2t}\operatorname{cov}\left(X_{r}, X_{s}\right) + \scriptstyle{\text{other gobbledygook that you can fill in}}$$ and so proving that $\operatorname{cov}\left(X_{r+t}, X_{s+t}\right) = \operatorname{cov}\left(X_{r}, X_{s}\right)$ is not as easy as you might think.

Answer 2

ok, so I know (for example) that $X_t = 2X_{t-1} + e_t$ for iid $e_t$ is stationary.

As far as I know the process that you mentioned is not [weakly]stationary (for stationarity the coefficient must be in $(-1,1)$ interval). Let start with expectation,

Let $E(e_t)=0$ that is very common in time series literature. Also assume that the process starts at a non-zero point. \begin{align} E(X_t)&=2E(X_{t-1})+0\\ &=2(2E(X_{t-2})+0)+0\\ &=\ldots\\ &=2^kE(X_{t-k})\\ &=\vdots \end{align} that is not converging at all. That mean that this process is increasing and then process is not trend stationary.

if you assume that the process starts from zero. then you can find variances with the same procedure as above to see it is not stationary (level stationary).

Edit: For clarification, the above example can be rewritten as:

$X_t = \frac{1}{2} X_{t+1} + u_t$, with $u_t$ some new white noise process. Now we can write down the infinite MA representation of this process, that is,

$X_t = \sum_{j=0}^{\infty} (\frac{1}{2})^{j+1} u_{t+j}$.

Your edit looks convincing. But notice that an standard assumption on time series is that error in future is independent of current time series values. Statistically speaking $Cov(e_t,x_s)=0$ for all $t>s$. Your last equation violates this assumption. On the other hand, that clearly makes the life difficult because there is no record of future to test stationarity and even time series.

Answer 3

Answer: there is no way to do it, the premise is wrong.

For an AR(1) process $x_t = \rho x_{t-1} + e_t$ where $e_t$ is a white-noise process, the equation $$\mathbb{V}x_t = \rho^2 \mathbb{V}x_t + \mathbb{V}e_t$$ has no positive finite solution with respect to $\mathbb{V}x_t$ if $\rho^2 \geq 1$, therefore $x_t$ is not weakly stationary and autocovariance $\mathbb{Cov}(x_t,x_{t+j})$ does not need to be independent of $t$.

---

Details: Weak stationarity commands:

1. $\mathbb{E}x_t = \mathbb{E}x_{t+s} \quad \forall s\in \mathbb{Z}$
2. $\mathbb{V}x_t <\infty \quad \forall t\in \mathbb{Z}$
3. $\mathbb{Cov}(x_t,x_{t+j}) = \mathbb{Cov}(x_{t+s},x_{t+j+s}) \quad \forall s,j\in \mathbb{Z}$

The latter implies that $\mathbb{V}x_t = \mathbb{V}x_{t+s} \quad \forall s\in\mathbb{Z}$.

Take a process $x_t = \rho x_{t-1} + e_t,$ where $e_t \sim WN(0,\sigma^2)$.

If $x_t$ is weakly stationary, the latest condition implies $$\mathbb{V}x_t = \mathbb{V}x_{t-1}$$ and jointly with the process model that $$\mathbb{V}x_t = \rho^2 \mathbb{V}x_t + \mathbb{V}e_t$$

As $\mathbb{V}e_t > 0$, if $\rho^2 \geq 1$ there is no non-negative solution to the last equation, and the only possibility for $x_t$ to have variance at all is for it to be dependent on period of observation, i.e. $\mathbb{V}x_t \neq \mathbb{V}x_{t-1},$ which directly contradict the weak stationarity.

Answer 4

Simple contradictory example

Let's consider the covariance between $X_t$ and $X_{t+1}$ this is $$\text{cov}(X_t,X_{t+1}) = \text{cov}(X_{t},2X_{t}+e_t) = 2 \, \text{var}(X_t)$$

^{You could generalize this to $\text{cov}(X_t,X_{t+s}) = 2^s \text{var}(X_t)$}

You assumed that the variance is constant in $t$ (because the time-series has a stationary solution), but it is not.

$$\text{var}(X_t) = \text{var}(X_{t-1}+e_t) = \text{var}(X_{t-1}) + \text{var}(e_t) + \underbrace{2\text{cov}(X_{t-1},e_t)}_{\text{= 0 by assumption}} \neq \text{var}(X_{t-1})$$

---

About the stationarity

Inverting the time-direction/causality of the process (like here) is not correctly representing the time-series.

We can indeed rewrite $$X_t = 2X_{t-1} + e_t$$ into $$X_{t-1} = 0.5 X_t - 0.5 e_t$$ or rebrand this $-0.5 e_t$ as $u_{t-1}$ such that it is $$X_{t-1} = 0.5 X_t + u_{t-1}$$ But this is not similar to an AR1 process with $\varphi_1 = 0.5$ because the $u_{t-1}$ (which is $e_t$ in disguise) correlates with $X_{t}$.

This is similar to expressing $$B = A +\epsilon \qquad \text{where: cov$(A,\epsilon)=0$}$$

in terms of

$$A = B - \epsilon = B + \epsilon^\prime$$

We can not consider $\text{var}(\epsilon^\prime) = \text{var}(\epsilon)$ and $\text{cov}(B,\epsilon^\prime) =0$.

If we would do this then we could compute the variance like

$$\begin{array}{rcccl} \text{var}(B) &=& \text{var}(A) + \text{var}(\epsilon) + \overbrace{2 \, \text{cov}(A,\epsilon)}^{\text{=0 by assumption}} &=& \text{var}(A) + \text{var}(\epsilon)\\ \text{var}(A) &=& \text{var}(B) + \text{var}(\epsilon^\prime) + 2 \, \text{cov}(B,\epsilon^\prime) &=& \require{cancel} \cancel{ \text{var}(B) + \text{var}(\epsilon^\prime)} \end{array}$$

It would be a contradiction to have both $\text{var}(B) = \text{var}(A) + \text{var}(\epsilon) $ and also $\text{var}(A) = \text{var}(B) + \text{var}(\epsilon) $