Can zero covariance and zero expectation imply zero conditional expectation?

Question

Let $x$ and $\epsilon$ are two random variables. If $$Cov(x, \epsilon)=0$$ and $$E[\epsilon]=0,$$ can that lead to $E[\epsilon|x]=0?$

Answer 1

This figure is a complete answer.

For those who would like a gloss on the figure though, notice that in this sample of 1,000 values of $(X,\varepsilon),$

• "$\operatorname{Cov}(X,\varepsilon)=0$." When $X$ and $\varepsilon$ are centered around zero, as they are here, the covariance is their average product. The figure uses color to indicate the individual products: greens and blues for very negative values and oranges for slightly positive values. On balance the many oranges cancel the few greens and blues, giving zero covariance.

• "$E[\varepsilon]=0$." On average the value of $\varepsilon$ is zero. This is clear from the symmetry: rotating the plot 180 degrees preserves its features (down to fairly fine detail) but negates the values of $\varepsilon.$ Thus the average must be close to zero (the only finite number equal to its own negative).

The conditional expectation $E[\varepsilon\mid X]$ is traced out by the thick black curve: at each value of $X$ it estimates the average height of the points lying above that value. Clearly this is not always zero. (All that matters in this example is that the curve is not constantly zero: the details of its shape are irrelevant.)

Even when $X$ and $\varepsilon$ have zero covariance and $\varepsilon$ has zero expectation, locally the values of $\varepsilon$ may fluctuate with $X,$ provided they average out to zero globally (on the whole).

---

Appendix

Below is the R code that generated the figure. The (long) third line is the heart of it: to a curvilinear function $y = \sin(\pi x/\sqrt{3})$ it adds uniformly distributed errors runif(n, -1/2, 1/2) and then -- to assure the resulting response will have no correlation with $x$ -- removes the effect of $x$ on these values (using the sequence of calls scale(residuals(lm(...)))).

(If you are uncomfortable with this pre-processing, then add the errors after removing the effect of $x:$

eps <- runif(n, -1/2, 1/2) + residuals(lm(sin(pi*x/sqrt(3)) ~ x))

The result, because the errors are truly independent of $x,$ will not have a perfectly zero correlation, but only because of chance variation in the simulation. This gives a better simulation but doesn't guarantee a good plot!)

#
# Create data.
#
n <- 1e3                     # Specify the size of the dataset
x <- scale(runif(n))         # Create explanatory variable values
eps <- scale(residuals(lm(runif(n, -1/2, 1/2) + sin(pi*x/sqrt(3)) ~ x)))
zapsmall(cor(cbind(x, eps))) # Confirm lack of correlation
#
# Create a data frame for plotting and plot it.
#
X <- data.frame(x=x, eps=eps, Product=x*eps)
library(ggplot2)
ggplot(X, aes(x, eps)) +
  geom_hline(yintercept=0) + geom_vline(xintercept=0) +        # Draw axes
  geom_point(aes(fill=Product), size=2, shape=21, alpha=1/4) + # Plot the points
  geom_smooth(color="Black", se=FALSE, size=1.1) +             # Plot the regression
  scale_fill_gradientn(colors=topo.colors(13)[1:12]) +         # Specify colors
  ylab(expression(epsilon))                                    # Label an axis

Answer 2

Let $x = \epsilon = 0$. Now check the relevant moments.

Answer 3

Let $x$ and $\epsilon$ are two random variables. If $$Cov(x,\epsilon)=0$$ and $$E[\epsilon]=0,$$ can that lead to $E[\epsilon|x]=0?$

No, those conditions are not enough.

Indeed it is possible that both hold but $Cov(x^2,\epsilon)\neq0$, therefore $E[\epsilon|x]\neq 0$

However if this stonger condition hold $Cov(f(x),\epsilon)=0$ fon any $f()$, then $E[\epsilon|x] = 0$ hold

Note: sometimes $E[\epsilon|x] = 0$ is assumed but only $E[x \epsilon] = 0$ is checked/considered. This prassi can work if we assume (often implicitly) that only linear relations are permitted.

Answer 4

The following is wrong. Here is an counter example:

Here are my thoughts. Could some expert help check for me? thanks.

$Cov(x, \epsilon)=0\space\implies E[x\epsilon]-E[x]E[\epsilon]=0 \space \stackrel{E[\epsilon]=0}\implies E[x\epsilon]=0$

so we can say $x$ and $\epsilon$ are orthogonal. From the geometric perspective, $E[\epsilon|x]$ is the projection of $\epsilon$ onto the x's plane. Since $\epsilon$ is orthogonal to that plane, so the projection is zero. That is $E[\epsilon|x]=0$