23

Independence between random variables $X$ and $Y$ implies that $\text{Corr}\left(f(X),g(Y)\right)=0$ for arbitrary functions $f(\cdot)$ and $g(\cdot)$ (here is a related thread).

But is the following statement, or a similar one (perhaps more rigorously defined), correct?

If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
  • 3
    I would say all bounded continuous functions rather than all possible functions. For some functions the variance may be infinite, so correlations will not be defined. But for bounded continuous functions that problem doesn't arise and that's a big enough class of functions to get the result. – Michael Hardy Jan 09 '21 at 20:32

6 Answers6

34

Using indicator functions of measurable sets like$$f(x)=\mathbb I_A(x)\quad g(x)=\mathbb I_B(x)$$leads to$$\text{cov}(f(X),g(Y))=\mathbb P(X\in A,Y\in B)-\mathbb P(X\in A)\mathbb P(Y\in B)$$therefore implying independence. As shown in the following snapshot of A. Dembo's probability course, proving the result for indicator functions is enough.

enter image description here

This is due to this monotone class theorem:

enter image description here

Xi'an
  • 90,397
  • 9
  • 157
  • 575
7

@Xi'an gives probably the simplest set of functions $f,\,g$ that will work. Here's a more general argument:

It is sufficient to show that the characteristic function $E[\exp(itX+iSY)]$ factors into $E[\exp(itX)]E[\exp(iSY)]$, because characteristic functions determine distributions.

Therefore, it is sufficient to show zero correlation

  • when $f,\,g$ are of the form $f_t(x)=\exp(itx)$ and $f_s(y)=\exp(isy)$
  • so $\sin(tx)$ and $\cos(sy)$ are also sufficient
  • by the Weierstrass approximation theorem, the sines and cosines can be approximated by polynomials, which also suffice
  • more generally, by the Stone-Weierstrass theorem, any other set of continuous functions closed under addition and multiplication, containing the constants, and separating points will also do ['separates points' means for any $x_1$ and $x_2$ you can find $f$ so that $f(x_1)\neq f(x_2)$, and similarly for $y$ and $g$]
  • the construction of integrals from indicator functions shows you can also use constant functions as @Xi'an does
  • and, like, wavelets or whatever

It might occasionally be useful to note that you don't have to use the same set of functions for $f$ as for $g$. For example, you could use indicator functions for $f$ and polynomials for $g$ if that somehow made your life easier

Thomas Lumley
  • 21,784
  • 1
  • 22
  • 73
5

Any continuous random variable can be mapped into a uniform [0,1] random variable using the cumulative distribution function. If the variables are independent, then the joint distribution on the 1x1 square will be the product of the two uniform margins and so uniform too. For the variables to be dependent, the joint distribution is not equal to the product, and therefore not uniform. The 1x1 square has bumps and dips in it. We then apply a permutation of intervals/blocks along each axis to rearrange those bumps along the diagonal and the dips far away from it - like permuting the rows and columns of a matrix with the Cuthill-McKee algorithm. This makes the correlation non-zero. Thus, zero correlation for all functions of continuous random variables implies independence.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Nullius in Verba
  • 59
  • 1
0

If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.

In the ref that I have the opposite is affirmed. If $X$ and $Y$ are independent we have that:

$E[f(X)]E[g(Y)]-E[f(X)g(Y)]=0$ (then $corr[f(X),g(Y)]=0$)

for any $f()$ and $g()$.

In words, we have no chance to find dependencies. Indeed if exist, them must be revealed by some functional relations. See: Econometrics – Verbeek; 5th edition pag 463. But some distributions/moments/functions conditions seems me implicit.

To move in the opposite direction is permitted, so from $\text{Corr}\left(f(X),g(Y)\right)=0$ the independence is implied.

However can be useful to note that the condition $\text{Corr}\left(f(X),g(Y)\right)=0$ imply some restrictions on the distributions/functions/moments. In some cases, this condition can fail. For example if $X$ and $Y$ are independent Cauchy r.vs: $\text{Corr}\left(f(X),g(Y)\right)=0$ not hold, or at lest not for some $f()$ and $g()$. Then, the condition in argument and the independence are not completely equivalent.

markowitz
  • 3,964
  • 1
  • 13
  • 28
  • Just to clarify: the implication from independence to lack of correlation is not my focus; I am only asking about the case of implication from lack of correlation to independence. Thus I think only your last two paragraphs are really of interest, especially *If we want move in the opposite direction, I fear that with some peculiar distributions and/or transformations some problems can appear. But in general the idea hold*. I think Xi'an's answer addresses your fear, does it not? – Richard Hardy Jan 07 '21 at 15:52
  • 1
    I edited my answer spending more attention on some details. – markowitz Jan 07 '21 at 16:58
  • 1
    The example with Cauchy random variables is helpful. – Richard Hardy Jan 08 '21 at 18:28
0

Two variables being dependent means that there is some value(s) of one variable that make some value(s) of the other variable more likely (the general statement is that it changes the probability, but WLOG we can assume that it increases the probability). And if that is the cases, then clearly there is positive correlation between the first variable having the value(s) in question, and the second variable having the value(s) in question. This correlation can be reflected in correlation between functions by taking functions that have different outputs depending on whether the variables take on the value(s) in question.

As a practical matter, this isn't generally a good method of proving independence. Given any countable set of functions, it's possible to construct two dependent variables for which all those functions are uncorrelated. So you have to prove that an uncountable set of functions are uncorrelated, at which point it's probably easier to just prove independence directly.

Acccumulation
  • 3,688
  • 5
  • 11
  • 1
    This characterization looks like a great way to prove two variables are *not* independent, though. – whuber Jan 09 '21 at 18:03
  • Thank you for the answer. Could you perhaps digest it to a more direct answer to my original question? – Richard Hardy Jan 09 '21 at 23:17
  • Is that second comment true? Since $Q := \{(-\infty, q] : q \in \mathbb Q\}$ is a $\pi$-system that generates the Borel $\sigma$-algebra, if $\mathbf 1_{A}(X)$ is uncorrelated with $\mathbf 1_B(Y)$ for every $A,B \in Q$ then $X \perp Y$, and that's a countable collection of indicators – jld Jan 11 '21 at 14:59
-4

Correlation catches only the linear dependence between two variables.

A and B are dependent but uncorrelated if $A = B^2$ for example

Pure independence implies the stochastic independence, which is that the occurrence of one does not affect the occurrence of the other. Similarly, two random variables are independent if the realization of one does not affect the probability distribution of the other (copied from the wiki)

Emil Mirzayev
  • 137
  • 7
  • 3
    Thanks for the general definition. How does that answer my question? Can we establish independence between $X$ and $Y$ when we know that $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$? The functions are of course not limited to just linear ones. – Richard Hardy Jan 07 '21 at 08:54
  • No we can not. Even though the correlation is zero, variables might be statistically dependent – Emil Mirzayev Jan 07 '21 at 09:24
  • 3
    I think that if Xi'an's answer is correct, your comment must be wrong. Would you challenge his answer or concede? – Richard Hardy Jan 07 '21 at 09:50
  • 2
    @EmilMirzayev Please don't perpetuate this fallacy about $A,B$ being dependent but uncorrelated if $A=B^2$. We have $Cov(A,B) = E(B^3) - E(B^2)E(B)$ and in general this is not zero. Try an Exponential with parameter $1$, whose moments are $E(B^n) = n!$. – Alecos Papadopoulos Jan 08 '21 at 04:24
  • @AlecosPapadopoulos Well, if the domain is symmetrical (e.g. [-1.1]), then they will be uncorrelated. But following Xi'an's answer, we can take indicator functions for a nonsymmetric set, revealing nonzero correlation. – Acccumulation Jan 09 '21 at 09:10
  • @Acccumulation What we need is that $E(B)$ and $E(B^3)$ are both zero. A sufficient (but not necessary) condition for this is that the density is symmetric around zero. But it is the usual case brought forth as an example. – Alecos Papadopoulos Jan 09 '21 at 14:33