2

I vaguely remember seeing somewhere that the conditional expectation $E(Y|X)$ can be interpreted as projection of random variable $Y$ onto random variable $X$. My question is:

  1. Is the aforementioned interpretation that "$E(Y|X)$ is the projection of random variable $Y$ onto random variable $X$" correct? If yes, in what sense it's a projection?

  2. If 1 is correct, how is this projection related to the projection onto subspace we see in basic linear algebra class? For example, define a subspace $V$ spanned by the column vectors of $A$ as $V=span\{A\}$, then the projection of a vector $x$ onto $V$ will be $A(A^{\top}A)^{-1}A^\top x$. What are the corresponding "$A$", "$V$" and "$x$" in the conditional expectation case?

T34driver
  • 1,608
  • 5
  • 11
  • 1
    The measure theoretic account of conditional expectations provides the simplest and most rigorous explanation. Would answers at this level of abstraction be acceptable? – whuber Sep 19 '20 at 16:29
  • 1
    @whuber Yes, that's good, and if it could be combined with some intuition, that would be perfect. – T34driver Sep 19 '20 at 16:51
  • 1
    https://stats.stackexchange.com/questions/71863/problem-with-proof-of-conditional-expectation-as-best-predictor/74440#74440 – Michael Sep 19 '20 at 20:42
  • 1
    "If 1 is correct..."---1 is not quite correct. $E(Y|X)$ is the projection of Y onto (informally) the "vector space of functions of $X$". "...the projection of random variable Y onto random variable X" is the projection of Y onto the vector space spanned by X, which is given by $\frac{cov}{var} \cdot X$ (assuming mean zero). In general, the former is projection onto an infinite dimensional space while the latter is 1-dimensional, not the same at all. The conditional mean contains much more information than linear projection. In the special bi-variate normal case, they coincide. – Michael Sep 19 '20 at 20:55
  • @Michael Thanks, Michael.This is very helpful. – T34driver Sep 20 '20 at 17:32

0 Answers0