Difference between autocorrelation and partial autocorrelation

Question

I have read some articles about partial autocorrelation of time series and I have to admit, that I do not really comprehend the difference to a normal autocorrelation. It is often stated that the partial autocorrelation between $y_t$ and $y_t-k$ is the correclation between $y_t$ and $y_t-k$ with the influence of the variables between $y_t$ and $y_t-k$ removed? I do not understand this. If we calculate the correlation between $y_t$ and $y_t-k$ then anyways the variables in between are not consindered at all if you use the correlation coefficient for doing that. The correlation coefficient considers two variables only as far as I know.

This really confuses me. I hope you can help me on that. I'd appreciate every comment and would be thankful for your help.

Update: Can anyone try to explain how one could calculate autocorrelation and partial autocorrelation for a time series. I understood how to do this with a sample but not with a time series (because you need three variables according to the example here https://en.wikipedia.org/wiki/Partial_correlation). Do you know any example where this is done?

Answer 1

For a while forget about time stamps. Consider three variables: $X, Y, Z$.

Let's say $Z$ has a direct influence on the variable $X$. You can think of $Z$ as some economic parameter in US which is influencing some other economic parameter $X$ of China.

Now it may be that a parameter $Y$ (some parameter in England) is also directly influenced by $Z$. But there is an independent relationship between $X$ and $Y$ as well. By independence here I mean that this relationship is independent from $Z$.

So you see when $Z$ changes, $X$ changes because of the direct relationship between $X$ and $Z$, and also because $Z$ changes $Y$ which in turn changes $X$. So $X$ changes because of two reasons.

Now read this with $Z=y_{t-h}, \ \ Y=y_{t-h+\tau}$ and $X=y_t$ (where $h>\tau$).

Autocorrelation between $X$ and $Z$ will take into account all changes in $X$ whether coming from $Z$ directly or through $Y$.

Partial autocorrelation removes the indirect impact of $Z$ on $X$ coming through $Y$.

How it is done? That is explained in the other answer given to your question.

Answer 2

The difference between (sample) ACF and PACF is easy to see from the linear regression perspective.

To get the sample ACF $\hat{\gamma}_h$ at lag $h$, you fit the linear regression model $$ y_t = \alpha + \beta y_{t-h} + u_t $$ and the resulting $\hat{\beta}$ is $\hat{\gamma}_h$. Because of (weak) stationarity, the estimate $\hat{\beta}$ is the sample correlation between $y_t$ and $y_{t-h}$. (There are some trivial differences between how sample moments are computed between time series and linear regression contexts, but they are negligible when sample size is large.)

To get the sample PACF $\hat{\rho}_h$ at lag $h$, you fit the linear regression model $$ y_t = \alpha + \, ? y_{t-1} + \cdots + \, ? y_{t-h + 1} + \beta y_{t-h} + u_t $$ and the resulting $\hat{\beta}$ is $\hat{\rho}_h$. So $\hat{\rho}_h$ is the "correlation between $y_t$ and $y_{t-h}$ after controlling for the intermediate elements."

The same discussion applies verbatim to the difference between population ACF and PACF. Just replace sample regressions by population regressions. For a stationary AR(p) process, you'll find the PACF to be zero for lags $h > p$. This is not surprising. The process is specified by a linear regression. $$ y_t = \phi_0 + \phi_1 y_{t-1} + \cdots \phi_p y_{t-p} + \epsilon_t $$
If you add an regressor (say $y_{t-p-1}$) on the right-hand side that is uncorrelated with the error term $\epsilon_t$, the resulting coefficient (the PACF at lag $p+1$ in this case) would be zero.