Bivariate normal distribution , link $\Bbb E(Y\mid X=x)$ and $\Bbb E(X\mid Y=y)$

Question

Note: I edited this question on 1/1/2018 because of the comments on the original question. So some comments relate to the earlier version. It is closed as duplicaten but I dusagree with that

For a bi-variate normal distribution with mean $(\mu_X, \mu_Y)$, variances $\sigma_X^2, \sigma_Y^2$ and correlation $\rho$ it holds (see e.g. http://athenasc.com/Bivariate-Normal.pdf, https://math.stackexchange.com/questions/33993/bivariate-normal-conditional-variance) that

$$\Bbb E(Y\mid{X=x})=\mu_Y + \rho \frac{\sigma_Y}{\sigma_X}(x-\mu_X)$$

By symmetry I would say that (is this correct?)

$$\Bbb E(X\mid{Y=y})=\mu_X + \rho \frac{\sigma_X}{\sigma_Y}(y-\mu_Y)$$

Next question:

These are two lines in an $(x,y)$ plane.

The picture below shows and example of a sample from such a bivariate normal distribution. Conditioning on $X$ means that I 'intersect' along the red lines, conditioning on $Y$ is 'intersecting' along the blue lines.

The contours of constant density for a bi-variate normal distribution are (rotated) ellipses.

Note that the rotation angle of the prinicipal axis wrt to the $x$-axis depends on the correlation $\rho$ (so, as opposed to what @whuber says in his comments below) correlation is the relevant topic here. Because, as argued by @MichaelHardy, there are values for $\rho$ where these lines correspond to these principal axis.

My question is whether the two lines of the conditional means correspond to (one of) the principal axes of these ellipses and if not how this can be explained geometrically.

Note: this question is not answered by the answer of @DilipSarwate here (and which I fully agree with) : Effect of switching response and explanatory variable in simple linear regression because he is using OLS (so regression techniques). The above formula are a theoretical property of the bivariate (multivariate) normal distribitions namely that all conditional distributions of it are also normal (see http://athenasc.com/Bivariate-Normal.pdf), there is no need for referring to regression of OLS to show that.

Answer 1

Your argument from symmetry is correct.

They do not represent the same line.

That can be seen by looking at $y = mx+b$ and solving for $x,$ getting $x= \frac 1 m y - \frac b m,$ and seeing that the coefficient of $x$ in the first equation and that of $y$ in the second are each other's reciprocals. But $\rho\sigma_X/\sigma_Y$ and $\rho\sigma_Y/\sigma_X$ are not reciprocals of each other.

To see why you ought to expect two different lines, consider the case in which $\rho=0.$ Then $X$ and $Y$ are uncorrelated. Thus the estimated expected value of $Y$ given $X=x$ should not depend on $x,$ and so the line is $y = \mu_X,$ a horizontal line. But similarly you'd get the line $X=\mu_Y,$ a vertical line if the $y$-axis is vertical, to estimate the average value of $X$ given $Y=y.$ Clearly two different lines. Then consider what you should expect if $\rho = 0.01,$ etc.

This has a seemingly paradoxical result: If you find the estimated average $y$-value for a given $x$-value, and then find the estimated average $x$-value of that $y$-value, then you don't return to where you started, but instead get something closer to the average $x$-value. For example, suppose you want to estimate an athlete's performance next week given his performance today. If he performs unusually well or unusually badly today, then this says he will be closer to average performance next week than he is today. But if they're always moving toward the average, how is it that we don't see them all near the average after some time has passed? The answer is that although most of the ones whose performance is far from the average today are closer to average next week, there will be some others whose performance diverges from the average then.

Answer 2

Your argument from symmetry re the formulas for $E[Y\mid X=x]$ and $E[X\mid Y=y]$ is correct. For bivariate normal random variables, $E[Y\mid X=x]$ is a linear function of $x$ and $E[X\mid Y=y]$ is a linear function of $y$ with formulas as you have found them.

---

With regard to fitting straight lines to a bunch of data points -- as you have asked in the highlighted "Next question" in your query -- a detailed description of what happens with data points as you have shown in the figure and fitting straight lines to them, read the answers to Effect of switching response and explanatory variable in simple linear regression. It doesn't matter in the least whether the points came from a bivariate normal distribution or not: the straight line fit is the same in either case, and, as the answers to the referenced question show, the two straight lines are different unless all the data points lie on a straight line.