What is the difference between $E(X|Y)$ and $E(X|Y=y)$?

Question

Generally, What is difference between $E(X|Y)$ and $E(X|Y=y)$?

Former is function of $y$ and latter is function of $x$? It's so confusing..

Answer 1

Roughly speaking, the difference between $E(X \mid Y)$ and $E(X \mid Y = y)$ is that the former is a random variable, whereas the latter is (in some sense) a realization of $E(X \mid Y)$. For example, if $$(X, Y) \sim \mathcal N\left(\mathbf 0, \begin{pmatrix} 1 & \rho \\ \rho & 1 \end{pmatrix}\right)$$ then $E(X \mid Y)$ is the random variable $$ E(X \mid Y) = \rho Y. $$ Conversely, once $Y = y$ is observed, we would more likely be interested in the quantity $E(X \mid Y = y) = \rho y$ which is a scalar.

Maybe this seems like needless complication, but regarding $E(X \mid Y)$ as a random variable in its own right is what makes things like the tower-law $E(X) = E[E(X \mid Y)]$ make sense - the thing on the inside of the braces is random, so we can ask what its expectation is, whereas there is nothing random about $E(X \mid Y = y)$. In most cases we might hope to calculate $$ E(X \mid Y = y) = \int x f_{X\mid Y}(x \mid y) \ dx $$

and then get $E(X \mid Y)$ by "plugging in" the random variable $Y$ in place of $y$ in the resulting expression. As hinted in an earlier comment, there is a bit of subtlety that can creep in with regards to how these things are rigorously defined and linking them up in the appropriate way. This tends to happen with conditional probability, due to some technical issues with the underlying theory.

Answer 2

Suppose that $X$ and $Y$ are random variables.

Let $y_0$ be a fixed real number, say $y_0 = 1$. Then, $E[X\mid Y=y_0]= E[X\mid Y = 1]$ is a number: it is the conditional expected value of $X$ given that $Y$ has value $1$. Now, note for some other fixed real number $y_1$, say $y_1=1.5$, $E[X\mid Y = y_1] = E[X\mid Y = 1.5]$ would be the conditional expected value of $X$ given $Y = 1.5$ (a real number). There is no reason to suppose that $E[X\mid Y = 1.5]$ and $E[X\mid Y = 1]$ have the same value. Thus, we can also regard $E[X\mid Y=y]$ as being a real-valued function $g(y)$ that maps real numbers $y$ to real numbers $E[X\mid Y = y]$. Note that the statement in the OP's question that $E[X\mid Y = y]$ is a function of $x$ is incorrect: $E[X\mid Y = y]$ is a real-valued function of $y$.

On the other hand, $E[X\mid Y]$ is a random variable $Z$ which happens to be a function of the random variable $Y$. Now, whenever we write $Z = h(Y)$, what we mean is that whenever the random variable $Y$ happens to have value $y$, the random variable $Z$ has value $h(y)$. Whenever $Y$ takes on value $y$, the random variable $Z = E[X\mid Y]$ takes on value $E[X\mid Y = y] = g(y)$. Thus, $E[X\mid Y]$ is just another name for the random variable $Z = g(Y)$. Note that $E[X\mid Y]$ is a function of $Y$ (not $y$ as in the statement of the OP's question).

As a a simple illustrative example, suppose that $X$ and $Y$ are discrete random variables with joint distribution \begin{align} P(X=0,Y=0) &= 0.1,~~ P(X=0, Y=1) = 0.2,\\ P(X=1,Y=0) &= 0.3,~~ P(X=1,Y=1) = 0.4. \end{align} Note that $X$ and $Y$ are (dependent) Bernoulli random variables with parameters $0.7$ and $0.6$ respectively, and so $E[X] = 0.7$ and $E[Y] = 0.6$. Now, note that conditioned on $Y=0$, $X$ is a Bernoulli random variable with parameter $0.75$ while conditioned on $Y = 1$, $X$ is a Bernoulli random variable with parameter $\frac 23$. If you cannot see why this is so immediately, just work out the details: for example $$P(X=1\mid Y = 0) = \frac{P(X=1, Y=0)}{P(Y=0)} = \frac{0.3}{0.4} = \frac 34,\\ P(X=0\mid Y = 0) = \frac{P(X=0, Y=0)}{P(Y=0)} = \frac{0.1}{0.4} = \frac 14,$$ and similarly for $P(X=1\mid Y=1)$ and $P(X=0\mid Y = 1)$. Hence, we have that $$E[X\mid Y = 0] = \frac 34, \quad E[X \mid Y = 1] = \frac 23.$$ Thus, $E[X\mid Y = y] = g(y)$ where $g(y)$ is a real-valued function enjoying the properties: $$g(0) = \frac 34, \quad g(1) = \frac 23.$$

On the other hand, $E[X\mid Y] = g(Y)$ is a random variable that takes on values $\frac 34$ and $\frac 23$ with probabilities $0.4 = P(Y=0)$ and $0.6 = P(Y=1)$ respectively. Note that $E[X\mid Y]$ is a discrete random variable but is not a Bernoulli random variable.

As a final touch, note that $$E[Z] = E\left[E[X\mid Y]\right] = E[g(Y)] = 0.4\times \frac 34 + 0.6\times \frac 23 = 0.7 = E[X].$$ That is, the expected value of this function of $Y$, which we computed using only the marginal distribution of $Y$, happens to have the same numerical value as $E[X]$ !! This is an illustration of a more general result that many people believe is a LIE: $$E\left[E[X\mid Y]\right] = E[X].$$

Sorry, that's just a small joke. LIE is an acronym for Law of Iterated Expectation which is a perfectly valid result that everyone believes is the truth.

Answer 3

$E(X|Y)$ is the expectation of a random variable: the expectation of $X$ conditional on $Y$. $E(X|Y=y)$, on the other hand, is a particular value: the expected value of $X$ when $Y=y$.

Think of it this way: let $X$ represent the caloric intake and $Y$ represent height. $E(X|Y)$ is then the caloric intake, conditional on height - and in this case, $E(X|Y=y)$ represents our best guess at the caloric intake ($X$) when a person has a certain height $Y = y$, say, 180 centimeters.

Answer 4

$E(X|Y)$ is expected value of values of $X$ given values of $Y$ $E(X|Y=y)$ is expected value of $X$ given the value of $Y$ is $y$

Generally $P(X|Y)$ is probability of values $X$ given values $Y$, but you can get more precise and say $P(X=x|Y=y)$, i.e. probability of value $x$ from all $X$'s given the $y$'th value of $Y$'s. The difference is that in the first case it is about "values of" and in the second you consider a certain value.

You could find the diagram below helpful.