3

Consider the random variables $Y,X$. I believe that we can always write $$ Y=E(Y|X)+\epsilon $$ with $E(\epsilon|X)=0$.

Question: Is the above true regardless whether $Y$ is a discrete or continuous random variable?

My thoughts: I believe that the answer is yes.

For example, let $Z$ be a discrete random variable and $Y\equiv \mathbb{1}\{Z=3\}$, where $\mathbb{1}\{\cdots\}$ is the indicator function taking value $1$ if the condition inside is satisfied and zero otherwise. We can write $$ \mathbb{1}\{Z=3\}= E(\mathbb{1}\{Z=3\}| X)+ \epsilon \quad E(\epsilon|X)=0 $$ that is $$ \mathbb{1}\{Z=3\}= Pr(Z=3|X)+ \epsilon \quad E(\epsilon|X)=0 $$

TEX
  • 266
  • 3
  • 19
  • 4
    Sure you can--but this decomposition is not always meaningful or mathematically helpful. For instance, let $X$ have a Gamma distribution and $Y$ have a Poisson$(X)$ distribution. The possible values $\epsilon$ can take on (all with positive probability) are the possible values of $Y-E(Y\mid X) = 0-X, 1-X, 2-X, \ldots.$ When $X$ is not integral (and it has zero chance of being an integer) this isn't a particularly nice set of values. Even worse, these sets differ for *all* $X,$ meaning there's not a lot of commonality among the $\epsilon.$ – whuber Feb 11 '21 at 18:59
  • Wait, $E[A|B]=0$ only if but not if $E[A]=0$ right? – BCLC Jan 17 '22 at 09:47

1 Answers1

5

Yes this always possible, as long as the expectations exist:

$$E(Y|X) = E(Y|X).$$ $$E(Y|X) + Y= E(Y|X) + Y.$$ $$Y= E(Y|X) + Y - E(Y|X).$$ $$Y= E(Y|X) + \epsilon.$$

We then have $E(\epsilon) = E(Y - E(Y|X)) = E(Y) - E(E(Y|X)) = E(Y) - E(Y) = 0$ by linearity of expectations and the law of iterated expectations.

Note that in general and especially when $Y$ is discrete, $E(Y|X)$ will be non-linear in $X$.

Julian Schuessler
  • 2,025
  • 11
  • 16
  • 1
    Thanks. Is $E(\epsilon|X)=0$ by construction? – TEX Feb 11 '21 at 16:59
  • Well, as I show it follows from linearity of expectations and the LIE, which follow from probability axioms and further definitions, so yes, in this sense (and I believe this is the "common sense"), it holds by constructing/defining $E(\epsilon|X)$ as $Y - E(Y|X)$. – Julian Schuessler Feb 11 '21 at 17:02
  • 1 - ah so this holds even if $Y$ is neither continuous nor discrete? 2 - Wait, $E[A|B]=0$ only if but not if $E[A]=0$ right? – BCLC Jan 17 '22 at 09:44
  • AFAIK there are no random variables that are something else than continuous or discrete. Your second statement is not true. Note that in my answer, by definition, $\epsilon = Y - E(Y|X)$, so that $E(\epsilon|X) = E(Y|X) - E(Y|X) = 0$. – Julian Schuessler Jan 18 '22 at 10:36