2

Suppose X, Y are i.i.d standard normal (mean 0, standard deviation 1) random variables, a, b, c are constant scalars.

$$Z = a X + b Y$$

How to express $E[X|Z=c]$ using $a,b,c$?

Tony
  • 130
  • 5
  • 1
    Hint: this is an ordinary least squares regression problem, so you can just apply the standard formula for the conditional expectation. – whuber Feb 28 '21 at 16:03

3 Answers3

1

Still searching for a quick solution, but anyway I solved the problem in brute force.

  1. Prove that $f_{X,Z}(x_0,c) = \frac{1}{b} f_X(x_0) f_Y(\frac{c - ax_0}{b})$, where $f_{X,Z}$ is PDF of joint distribution of $X$ and $Z$ (hint: use Leibniz integral rule and Fundamental theorem of calculus)

  2. Prove $E[X|Z=c]=\frac{ac}{a^2+b^2}$ (hint: use definition of conditional expectation and PDF of standard normal random variable)

Tony
  • 130
  • 5
  • As I indicated in a comment to your question, you don't need to do this: it's simple to apply the OLS formula. You can also look up the answer by searching [conditional normal distribution](https://stats.stackexchange.com/search?tab=votes&q=conditional%20normal). It is also possible to avoid Calculus altogether, as explained at https://stats.stackexchange.com/a/71303/919. – whuber Feb 28 '21 at 17:48
  • Yeah you are right, thanks @whuber! – Tony Feb 28 '21 at 17:52
0

You could express the relationship as $X = Z/a - bY/a$, which gives you that $X\mid Z=c$ is distributed as $c/a - bY/a$ which means it is $\mathcal{N}(c/a, b^2/a^2)$ therefore you get $$\mathrm{E}[X\mid Z=c] = c/a$$ The absence of $b$ here comes from the fact $Y\sim\mathcal{N}(0,1)$.

ArnoV
  • 1,170
  • 3
  • 8
  • 1
    With this logic, we will have $E[Y | Z =c] = c/b$ by symmetry, but then $c = E[Z|Z=c] = E[aX + bY|Z=c] = aE[X|Z=c]+bE[Y|Z=c] = 2c$, so this result can't be correct when $c \neq 0$ – Tony Feb 28 '21 at 15:01
  • Excellent point, I however do not see which step is incorrect... – ArnoV Feb 28 '21 at 15:16
  • I think we can't say $X = c/a - bY/a$, because that means $X$ and $Y$ are not independent. – Tony Feb 28 '21 at 15:27
  • 1
    Of course $X=Z/a - bY/a$ (at least when $a\ne 0$) because that's just applying a rule of arithmetic. The error lies in (implicitly) assuming $Y$ and $Z$ are independent. – whuber Feb 28 '21 at 16:05
0

a)$E(aX+bY|aX+bY)=aX+bY$, so

$$aE(X|aX+bY)+bE(Y|aX+bY)=aX+bY \tag{1}$$

b)Find $d$ such that $X-dY$ be independent(Multivariate_normal_distribution#Correlations_and_independence) of $aX+bY$ so

$E(X-dY|aX+bY)=E(X-dY)=0$ and hence

$$E(X|aX+bY)=dE(Y|aX+bY) \tag{2}$$

$d$ can be computed as follows: $$0=\text{cou}(X-dY, aX+bY)=a\text{Var}(X)-db \text{Var}(Y)=a-db$$ so $d=\frac{a}{b}$

Now use (1) and (2)

$$aE(X|aX+bY)+b(\frac{b}{a}E(X|aX+bY))=aX+bY$$ so

$$E(X|aX+bY)=(\frac{a}{a^2+b^2})(aX+bY)$$

Masoud
  • 1,189
  • 4
  • 19