Writing AR(1) as a MA($\infty$) process

Question

The AR(1) process is

$$ X_t = \phi X_{t-1} + \varepsilon_t $$

if we use this formula recursively, we get $$ X_t = \phi(\phi X_{t-2} + \varepsilon_{t-1}) + \varepsilon_t = \phi^2X_{t-2} + \phi\varepsilon_{t-1} + \varepsilon_t = \cdots = \phi^k X_{t-k} + \sum_{j=0}^k \phi^j\varepsilon_{t-j} $$

If we let $k\to\infty$, we get $$ X_t = \lim_{k\to\infty}(\phi^k X_{t-k} + \sum_{j=0}^k \phi^j\varepsilon_{t-j}) = \lim_{k\to\infty}(\phi^k X_{t-k}) + \sum_{j=0}^\infty \phi^j\varepsilon_{t-j} $$ The duality between AR(1) and MA($\infty$) states that there is an equivalence between the two, and that we can write $X_t$ as

$$ X_t = \sum_{j=0}^\infty \phi^j\varepsilon_{t-j} $$

The difference between the two results is the term $\lim_{k\to\infty}(\phi^k X_{t-k})$, which should be zero, but how do I show this?

Assuming $|\phi| < 1$, we have that $\lim_{k\to\infty}\phi^k = 0$ of course, but I don't see why $\lim_{k\to\infty} X_{t-k} < \infty$? Does convergence asuume the law of large numbers, or is there another way to show equivalence?

---

I know there is a proof which inverts the lag operator $1-B$, but I didn't find any justification for why the operator can even be inverted, so I wanted an alternative proof, as the one above.

Answer 1

The usual sense in which convergence is understood in this case is in mean square:

$$ E[Y_t-(\epsilon_t+\phi\epsilon_{t-1}+\phi^2\epsilon_{t-2} +\ldots+\phi^j\epsilon_{t-j})]^2=\phi^{2(j+1)} E[Y_{t-j-1}]^2 $$ If $Y_t$ is stationary $$ E[Y_{t-j-1}]^2=\gamma_0+\mu^2 $$ Hence $$ \lim_{j\to\infty}E[Y_t-(\epsilon_t+\phi\epsilon_{t-1}+\phi^2\epsilon_{t-2} +\ldots+\phi^j\epsilon_{t-j})]^2=0 $$

Answer 2

You are right to be suspicious of this step, and in fact, without further assumptions to limit the size of $X_{-\infty}$ you cannot get the required form. Remember that the recursive equation for the AR model is insufficient to yield the joint distribution of the process. (You need to impose a distribution for the error process, and even then, you either need to impose stationarity or specify an initial distribution that leads to some non-stationary model.) If you only have this recursive equation, there is no reason that the time-series values could not explode out to large values as $t \rightarrow -\infty$.

For example, the deterministic non-stationary AR process $X_t = \phi^t$ satisfies the recursive equation you have specified (with zero errors), and in this case you have $\lim_{k \rightarrow \infty} X_{t-k} = \infty$. In this model, for any $\phi \neq 0$ you also have:

$$\phi^k X_{t-k} = \phi^k \phi^{t-k} = \phi^t \neq 0.$$

Since the errors are zero in this deterministic model, this gives you the limiting result:

$$X_t = \underbrace{\sum_{k=0}^\infty \phi^k \varepsilon_{t-k}}_{0} + \underbrace{\lim_{k \rightarrow \infty} \phi^k X_{t-k}}_{\phi^t}. \\[6pt]$$

Clearly, in this case, the limiting term is non-zero, and cannot be removed from the result. If you would like to be able to remove this last term, you need to add further assumptions to your model (e.g, stationarity).