Showing that estimating $Y$ from $X$ via linear regression and estimating $X$ from $Y$ via linear regression are not invertible

Question

This is a follow up to the question Regression on population values?.

In the theory of test equating (see, e.g., Test Equating, Scaling, and Linking by Kolen and Brennan (2014)), suppose we have a group of examinees whom we randomly assign one of two exam forms, say form X and form Y.

Let $X$ be a random variable representing scores of form X, and similarly for $Y$. Letting $\sigma(X)$ denote the standard deviation of a random variable $X$, linear equating assumes that $$\dfrac{X - \mathbb{E}[X]}{\sigma(X)} = \dfrac{Y - \mathbb{E}[Y]}{\sigma(Y)}$$ with probability one. To interpret the above, suppose we have a particular score $X = x$ from form X; we would use the equation above to yield an equivalent score for form Y.

Solving for $Y$ in the above equation yields $$Y = \dfrac{\sigma(Y)}{\sigma(X)}\cdot X + \mathbb{E}[Y] - \dfrac{\sigma(Y)}{\sigma(X)}\cdot \mathbb{E}[X]\text{.}\tag{1}$$ This can be interpreted as a linear equation of $X$, with slope $\dfrac{\sigma(Y)}{\sigma(X)}$ and intercept $\mathbb{E}[Y] - \dfrac{\sigma(Y)}{\sigma(X)}\cdot \mathbb{E}[X]$.

As stated in the prior question, Kolen and Brennan mention:

The equation for linear equating... is deceptively like a linear regression equation. The difference is that, for linear regression, the $\sigma(Y)/\sigma(X)$ terms are multiplied by the correlation between $X$ and $Y$.

Explaining this has already been well covered by the linked question above.

Kolen and Brennan then go on to say:

However, a linear regression equation does not qualify as an equating function because the regression of X on Y is different from the regression of Y on X, unless the correlation coefficient is 1. For this reason, regression equations cannot, in general, be used as equating functions.

To be more precise about this, with the inclusion of the correlation coefficient from performing linear regression in $(1)$, based on the above discussion, we obtain $$\begin{align} Y &= \dfrac{\sigma(Y)}{\sigma(X)}\rho_{X, Y}\cdot X + \mathbb{E}[Y] - \dfrac{\sigma(Y)}{\sigma(X)}\rho_{X, Y}\cdot \mathbb{E}[X] \\ &= \mathbb{E}[Y] + \dfrac{\sigma(Y)}{\sigma(X)}\rho_{X, Y}\left(X - \mathbb{E}[X]\right)\text{.} \end{align}$$ Denote this function $r_Y(X)$ - i.e., the regression equator of $X$ to $Y$ based on a linear regression of $Y$ on $X$ - more specifically: $$r_Y(X) = \mathbb{E}[Y] + \dfrac{\sigma(Y)}{\sigma(X)}\rho_{X, Y}\left(X - \mathbb{E}[X]\right)\tag{2}$$ If we were to perform regression of $X$ on $Y$, the regression equator of $Y$ to $X$ would be $$r_X(Y) = \mathbb{E}[X] + \dfrac{\sigma(X)}{\sigma(Y)}\rho_{X, Y}\left(Y - \mathbb{E}[Y]\right)\tag{3}$$

What I am merely trying to show is the following:

• If $Y$ is the response variable and $X$ is the explanatory variable in a linear regression, given $X = x$, the corresponding value $y = y(x)$ due to this regression would result in a different value of $X$ if we were to find the corresponding value of $X$ when looking at the regression based on $X$ being the response variable and $Y$ being the explanatory variable using $Y = y$ (and vice versa).

In other words, my aim is to show that $r_Y(r_X(Y)) \neq Y$ and $r_X(r_Y(X)) \neq X$ for $\rho_{X, Y} \neq 1$.

However, I suspect something is wrong with my equations $(2)$ or $(3)$. If $X$ is the response variable and $Y$ is the explanatory variable, we obtain $$r_X(r_Y(X)) = \mathbb{E}[X] + \dfrac{\sigma(X)}{\sigma(r_Y(X))}\rho_{r_Y(X), X}\left(r_Y(X) - \mathbb{E}[r_Y(X)]\right)$$ One can easily see that $\mathbb{E}[r_Y(X)] = \mathbb{E}[Y]$. Through variance and standard deviation calculations, one can also see that $$\text{Var}(r_Y(X)) = \rho^2_{X, Y}\text{Var}(Y)$$ so that $$\sigma(r_Y(X)) = |\rho_{X, Y}|\text{Var}(Y)\text{.}$$ Now \begin{align} \text{Cov}(r_Y(X), X) &= \text{Cov}\left( \mathbb{E}[Y] + \dfrac{\sigma(Y)}{\sigma(X)}\rho_{X, Y}\left(X - \mathbb{E}[X]\right), X\right) \\ &= \dfrac{\sigma(Y)}{\sigma(X)}\rho_{X, Y}\text{Var}(X) \\ &= \rho_{X, Y}\sigma(Y)\sigma(X)\text{.} \end{align} Therefore, $$\rho_{r_Y(X), X} = \dfrac{\rho_{X, Y}\sigma(Y)\sigma(X)}{|\rho_{X, Y}|\sigma(Y)\sigma(X)} = \dfrac{\rho_{X, Y}}{|\rho_{X, Y}|}\text{.}$$ Hence, \begin{align} r_X(r_Y(X)) &= \mathbb{E}[X] + \dfrac{\rho_{X, Y}}{|\rho_{X, Y}|} \cdot \dfrac{\sigma(X)}{|\rho_{X, Y}|\sigma(Y)}\left[\mathbb{E}[Y] + \rho_{X, Y}\cdot \dfrac{\sigma(Y)}{\sigma(X)}(X - \mathbb{E}[X]) - \mathbb{E}[Y] \right] \\ &= \mathbb{E}[X] + \dfrac{\rho_{X, Y}^2}{|\rho_{X, Y}|^2}(X - \mathbb{E}[X]) \\ &= X\text{.} \end{align} Thus, I suspect something is wrong here. Could someone point out exactly what I did wrong?

Answer 1

Without loss of generality, assume $\mathbb{E}[X] = \mathbb{E}[Y] = 0$ and $\text{Var}(X) = \text{Var}(Y) = 1$.

Then $$r_{Y}(X) = \rho_{X, Y} \cdot X$$ and $$r_{X}(Y) = \rho_{X, Y} \cdot Y\text{.}$$ Hence, as long as $\rho_{X, Y} \neq 1$, $$r_{Y}(r_{X}(Y)) = r_{Y}(\rho_{X, Y} Y) = \rho_{r_{X}(Y), Y}\rho_{X, Y}Y $$ Now $$\rho_{r_X(Y), Y} = \rho_{\rho_{X, Y}Y, Y} = \dfrac{\rho_{X, Y}\cdot \text{Var}(Y)}{|\rho_{X, Y}|\cdot \text{Var}(Y)} = \pm 1$$ hence $$r_{Y}(r_{X}(Y)) = \pm \rho_{X, Y}Y \neq Y\text{.}$$

Thus, $r_Y \neq r_X^{-1}$.