$X$ and $Y$ are two correlated random variables. I am trying to estimate $E(X\mid Y)$ given $E(X)$, $E(Y)$, $\rho(X,Y)$, $\sigma(X)$ and $\sigma(Y)$. Could someone point me how to go about it. What if $X$ and $Y$ are vectors of RVs with $\Sigma_X$ and $\Sigma_Y$ being their covariance matrices.
Asked
Active
Viewed 35 times
1
-
If you are trying to do some simulations and can sample both $X$ and $Y$, I believe you can still numerically compute $\mathbb E[X|Y]$. I don't think a general closed-form expression is possible though – StratosFair Feb 19 '22 at 20:10
-
It is incorrect to write $\Sigma_x$ rather than $\Sigma_x$ here. When one writes things like $f_X(x),$ the choice of capital versus lower case is as it is for important reasons. – Michael Hardy Feb 19 '22 at 21:38
-
The only way to succeed is to make a parametric assumption strong enough so that these five properties pin down the joint distribution. This is the case for the family of Binormal distributions, for instance, but there are many such families. – whuber Feb 19 '22 at 21:48
-
@whuber so implicitly assuming normal distribution my solution below is correct? – Gerry Feb 20 '22 at 23:46
-
Yes: your solution is known as *regression.* It is considered *the* basic result about ordinary regression: namely, the conditional distribution in the Binormal family is linear. For some insight about this see my post at https://stats.stackexchange.com/a/71303/919. It derives this result assuming only the circular symmetry of the standard Binormal distribution. – whuber Feb 21 '22 at 13:17
2 Answers
1
I do not believe this is possible without more information regarding the relationship between the two variables.
For example, let's say that if X is positive, Y is negative, and vice-versa, and independently, they are two normally-distributed variables with a mean of $0$ and variance of $1$. You sample both of them, and then if their signs are the same, then flip one of them, e.g., by always choosing Y.
The correlation of these two variables will be negative, about $-0.63$. However, the only information you get from knowing X or Y is the sign. The expected value in either case would be roughly $0.8$ or $-0.8$.
If you have two normally-distributed variables that can be described using a standard correlation matrix $\Sigma_{xy}$, then you will have a unique value of $E(X|Y)$ for every value of $Y$.
There is definitely a more robust way of demonstrating more general cases of how this problem needs more constraints, but this is probably the easiest to visualize.
Here's a plot showing how two normal, correlated distributions can have a vastly different joint PDF, even if you know the correlation, their expected values, and standard deviations.
# code to generate plot
X = rnorm(10000)
Y = rnorm(10000)
Y = -abs(Y) * sign(X)
plot(X,Y,main='Signs are opposite')
cor(X,Y)
XYc = MASS::mvrnorm(
10000,
mu=c(0,0),
Sigma=matrix(c(1,cor(X,Y),cor(X,Y),1), nrow=2)
)
par(mfcol=c(1,2))
plot(X,Y,main='Opposite-signed X and Y',pch='.')
plot(XYc, xlab='X',ylab='Y', main='Correlated X and Y',pch='.')
```
Max Candocia
- 373
- 1
- 12
-
what do you mean when you say X is positive and Y is negative but E[X] = E[Y] = 0? – Gerry Feb 19 '22 at 20:08
-
-
1See the plots I added if this makes it a bit easier to understand. Both of the plots have the same exact underlying statistics, but their joint distribution is different, which changes $E(X|Y)$ – Max Candocia Feb 19 '22 at 21:37
-
I am not sure if I fully follow you. Perhaps if you could look at my solution and point what did I miss. Wouldn't $\rho(X,Y)$ as I described one of the inputs in the question be same as $\Sigma_{XY}$ as you said will be sufficient to arrive at $E[X|Y]$? – Gerry Feb 21 '22 at 00:12
-
In your solution, you assumed that $\hat{X} = \hat{\beta}_0+\hat{\beta}_1Y$ (for $n=1$), which wasn't a condition stated in the original question. If we add the condition/constraints of your example, however, then you should have sufficient information to solve for the BLUE (best linear unbiased estimator). The information you have in your question is a bit stronger than required, as you have the "true" expected values/covariances, rather than sample estimates, but you can still plug those in and get the conditional expected value. – Max Candocia Feb 21 '22 at 15:04
0
Thinking a bit about it (for non-vector case) what's the problem with following approach: $\hat{\beta}$ in OLS $\hat{X} = \hat{\beta}_0 + \hat{\beta}_1 Y_1 + \cdots + \hat{\beta}_n Y_n $ is estimated as follows: $$ \hat{\beta}_i = \frac{ Cov(Y_i, X) }{\sigma^2(Y_i) }$$ $$ \hat{\beta}_0 = E[X]$$ Above OLS equation can be re-written as: $$E[X | Y_1, ..., Y_n] = E[X] + \Sigma_i\frac{ Cov(Y_i, X) }{\sigma^2(Y_i)}E[Y_i]$$ In case $n=1$ and say $Y_1 = Y$: $$E[X | Y] = E[X] + \frac{ Cov(Y, X) }{\sigma^2(Y)}E[Y]$$ $$E[X | Y] = E[X] + \frac{ \rho(Y, X) \sigma(X) \sigma(Y) }{\sigma^2(Y)}E[Y]$$ $$E[X | Y] = E[X] + \frac{ \rho(Y, X) \sigma(X) }{\sigma(Y)}E[Y]$$
Gerry
- 141
- 1
- 4
-
Actually now I am thinking how $\hat{\beta}_0$ from the expression of $\hat{\beta}_i$ above by setting $i=0$. That is how $\hat{\beta}_0 = \frac{ Cov(Y_0, X) }{\sigma^2(Y_0) } = E[X]$ ?where, $Y_0 = 1$. – Gerry Feb 20 '22 at 23:48
-
Minor style comment, but I've seen $k$ used more for enumerating variable indexes (here $Y_1$, $Y2$,..., $Y_k$) and $n$ used for row numbers/indexes. – Max Candocia Feb 21 '22 at 15:08