Conditional expectation and variable decomposition

Question

Suppose that $X$ and $Y$ have an uknown joint distribution $f_{XY}$.

How can I formally demostrate that it always exists a unique decomposition of the form : $$ Y = E[Y|X] +\epsilon $$

without assuming any explicity form of $f_{XY}$?

Answer 1

This follows from the linearity of expectation and law of total expectation

$X$ and $Y$ have an unknown joint distribution $F_{XY}$, and the distribution of $Y\mid X$ is some unknown $F_{Y|X}$. Suppose the mean of $Y|X$ is $\mu(X)$.

Consider the random variable $\epsilon = Y - \mu(X)$. Then $\epsilon$ is a mean 0 random variable. To see this, \begin{align*} E(\epsilon)& = E(Y - \mu(X))\\ & = E(E(Y - \mu(X) \mid X))\\ & = E\left[ E(Y\mid X) - \mu(X) \right]\\ & = E(0)\\ & = 0\,. \end{align*}

Thus, we can always write

$$Y = \mu(X) + \epsilon = E(Y\mid X) + \epsilon \,.$$

---

Uniqueness:

Assume that there exists a $\delta(X)$ and $\eta$, such that

$$Y = \delta(x) + \eta. $$

Uniqueness holds under two constraints.

1. $\eta$ is independent of $X$
2. $E(\eta) = 0$

Taking expectation with respect to $F_{Y|X}$, \begin{align*} E(Y\mid X) & = E( \delta(X) \mid X) + E(\eta \mid X)\\ \Rightarrow \mu(X) & = \delta(X) + 0\,. \end{align*}

Thus, we obtain that $\delta(X) = \mu(X)$, which implies $\eta = \epsilon$.

Note that the two constraints are crucial.