ANOVA Theorem Explanation

Question

I am trying to figure out why the following holds:

Given $y_{i}=E[y_{i}|X_{i}]+\epsilon_{i}$ that

$E[\epsilon^{2}_{i}] =E[E[\epsilon^{2}_{i}|X_{i}]] = E[V[y_{i}|X_{i}]]$

Specifically I am trying to understand why $E[\epsilon^{2}_{i}|X_{i}] = V[y_{i}|X_{i}]$?

Clearly, I need a refresher on conditional variance and the rules for expected values....

Answer 1

The first equality $E[\epsilon^{2}_{i}] =E[E[\epsilon^{2}_{i}|X_{i}]]$ is just the law of total expectation.

Recall that the variance of $X$ is the expected squared deviance of $X$ from its expected value: \begin{equation} \textrm{Var}(X) = \mathbb{E}\left( \left(X - \mathbb{E}(X)\right)^2 \right). \end{equation} The conditional variance is defined similarly, but now both expectations are conditional: \begin{equation} \textrm{Var}(X \mid Y) = \mathbb{E}\left[\left( X - \mathbb{E}(X \mid Y)\right)^2 \mid Y\right]. \end{equation}

The second equality $E[\epsilon^{2}_{i}|X_{i}] = V[y_{i}|X_{i}]$ is obtained by

1. Applying the aforementioned definition of conditional variance to $\textrm{Var}[y_i \mid X_i]$
2. Use the given part to express this in terms of $\epsilon_i$.