6

Let $x,y$ be two variables that are functionally related, so that $x$ determines $y$:

$$y = f(x)$$

for some function $f$. We consider a probability distribution on $x$, $p(x)$. The question is:

Is it possible, for some probability distribution $p(x)$, that the correlation between $x$ and $y$ vanishes? That is,

$$\mathrm{cov}(x,f(x)) = \langle x f(x) \rangle - \langle x\rangle \langle f(x) \rangle = \sum_x xf(x)p(x) - \left(\sum_x xp(x)\right) \left(\sum_x f(x)p(x)\right) = 0$$

In other words, given a function $f$, is it possible to select a probability distribution $p$, such that $\mathrm{cov}(x,f(x)) = 0$?

The answer is yes in at least one trivial case, when $f(x) = c$ is a constant number. This case is trivial in the sense that any $p$ will do. Are there more interesting examples? Or general theorems available?

becko
  • 3,298
  • 1
  • 19
  • 36
  • You write sums. Do you intend to restrict your question to discrete probability distribution or are answers for continuous distributions of interest, too? – Horst Grünbusch Jul 06 '17 at 15:16
  • @HorstGrünbusch I think sums are easier to reason about, but if you have an idea with continuous distributions feel free to contribute. – becko Jul 06 '17 at 15:52
  • 2
    of course this is possible: let $X$ be uniform distributed on $[-1,1]$ (or $N(0,\sigma^2)$, in fact any symmetric distribution around $0$ should work), and $Y=f(X)=X^2$. you then have $$cov(X,Y) = E(XY)-E(X)E(Y) = E(X^3)= 1/4(-1+1)(1^2+1^2) = 0.$$ **added:** The covariance (correlation) only tells you something about a linear relationship, there are of course other relationships which are not linear and hence imply a covariance of $0$. – EliKa Jul 06 '17 at 15:53
  • @EliKa Nice example. I just find it counter-intuitive, though. This is saying that even though variables $x,y$ are deterministically related, a dataset will show *zero* correlation between them. – becko Jul 06 '17 at 15:55
  • $X$ and $Y$ are functional related (this does not implay, that they are necessary deterministically related: imagine a third variable, which determines in fact determines X and hence Y. beside that, see my edit in the comment (horst grünbusch's answer is just the negation: if they are linear related, then it is impossible.) – EliKa Jul 06 '17 at 16:02

2 Answers2

9

There are three mutually exclusive possibilities for $f$ (apart from the trivial one where the domain of $f$ has just one element). To be fully general and avoid trivial complications, let's not worry about correlation, but focus on covariance instead: when covariance is zero, correlation is either zero or undefined. (Correlation becomes undefined when the variance of either marginal distribution is zero.)

  1. $f$ is not injective. Suppose there exist $x_1 \lt x_2$ for which $f(x_1)=f(x_2)$. Putting probabilities of $1/2$ on each of $x_1$ and $x_2$ gives zero covariance (and undefined correlation).

  2. $f$ is injective but not monotonic. Otherwise suppose there exist $x_1\lt x_2 \lt x_3$ for which the $y_i=f(x_i)$ are not in order. Since covariance does not change with translations, make the calculations easier by shifting the $x$ and $y$ coordinates so that $x_2=y_2=0$. The assumption amounts to $x_1\lt 0,$ $x_3\gt 0$, and $y_1$ and $y_3$ have the same sign. Define $C(p)$ to be the covariance achieved by putting probabilities of $p$ on $x_1$, $1/2-p$ on $x_3$, and $1/2$ on $0$. $C$ clearly is a quadratic function of $p$ and therefore is continuous.

    Compute $$C(0)=\frac{1}{4}x_3y_3,\ C(1/2)=\frac{1}{4}x_1y_1.$$ The assumptions imply $C(0)$ and $C(1/2)$ have opposite signs. The Intermediate Value Theorem implies there is some $p\in (0,1/2)$ for which $C(p)=0$: use this value of $p$ to achieve zero covariance. The correlation will be defined and zero.

  3. $f$ is monotonic. Otherwise $f$ is strictly monotonic (increasing or decreasing). From the characterization of covariance as an expected signed area, it is obvious that all covariances must be strictly positive or strictly negative when the probability is not an atom. The correlation will be defined.

whuber
  • 281,159
  • 54
  • 637
  • 1,101
1

The answer is no if $f$ is affine-linear. This holds by the linearity and translation invariance of the covariance.

Horst Grünbusch
  • 5,020
  • 17
  • 22
  • The general answer is NO then, because not for every function you can find such probability distribution, if there's at least one for which you can't – Aksakal Jul 06 '17 at 15:20
  • Please, define "affine-linear" function. – becko Jul 06 '17 at 15:53
  • Also, can you give more details about your reasoning? I am not convinced about your answer. Thanks. – becko Jul 06 '17 at 16:00
  • 1
    Becko, this answer is correct, implicitly assuming $f$ has nonzero slope--but it only points out a special case. "Affine-linear" means the graph of $f$ is a line. – whuber Jul 06 '17 at 16:09