8

I faced a limiting distribution with zero covariance between two variables but their correlation is $1$. Is there such a distribution? How it can be explained?


You are right may I need give more detail. OK, X and Y are bivariate normal distribution with different variances and means (free of n) but corr=1-(1/n), now investigate the limiting distribution of Yn|Xn=x.

dsaxton
  • 11,397
  • 1
  • 23
  • 45
Behgol
  • 83
  • 4

4 Answers4

20

Since the covariance depends on the scale of $X$ and $Y$ and the correlation does not (rescaled to $[-1, -1]$) it is possible. For example, if the variance decreases towards zero:

If $X=Y$ and $\sigma_x^2$ is the variance of $X$, then $\lim_{\sigma_x^2 \to 0} \operatorname{cov}(X, Y) = 0$ and $\lim_{\sigma_x^2 \to 0} \operatorname{cor}(X, Y) = 1$.

Note 1: when $\sigma_x^2 = 0$ the correlation is strictly undefined because its denominator would be equal to 0.

Pieter
  • 1,847
  • 9
  • 23
  • You are right may be I should give more detail. OK X and Y arebivariant normal distribution with different variance and mean (free of n) but corr=1-(1/n), now investigate the limiting distribution of Yn|Xn=x. – Behgol Oct 16 '16 at 19:25
  • The wording "Since the covariance depends on the scale" implies that this is given in the question. However, that seems to be more than the question implies. It seems to me that you are postulating that this might be so, with stated conclusions. Do correct me if that is wrong. – Nick Cox Oct 17 '16 at 16:02
18

As far as I can see (perhaps outside of some special circumstances, but you don't mention any), it's not possible.

The correlation is the covariance divided by the product of the two standard deviations, so if the covariance is zero, the correlation is either zero (when both standard deviations are non-zero) or undefined (when at least one standard deviation is 0). It should not be 1 when the covariance is 0.

I expect you have either made some error in your analysis or your description is insufficiently clear to discern the situation correctly.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
5

Following a clarification by the OP, it appears that a) we assume that the two variables follow jointly a bivariate normal and b) our interest is in the conditional distribution, which is then

$$Y_n\mid X_n=x \ \sim\ \mathcal{N}\left(\mu_y+\frac{\sigma_y}{\sigma_x}\rho_n( x - \mu_x),\, (1-\rho_n^2)\sigma_y^2\right)$$

Then we see that as $n \to \infty$, we have $\rho_n \to 1$, and the variance of the conditional distribution goes to zero. Intuitively, if correlation goes to unity, "knowing $x$" is enough to "know $y$" also.

But nowhere in the above do we get that $\text{Cov}(Y_n, X_n)$ is zero. Even at the limit covariance will remain equal to $\text{Cov}(Y_n, X_n) \to \sigma_y \sigma_x$.

Note that the conditional covariance (and then also the conditional correlation) is always zero, because,

$$\text{Cov}(Y_n, X_n \mid X_n =x) = E(Y_nX_n\mid X_n =x) - E(Y\mid X_n =x) E(X\mid X_n =x)$$

$$=xE(Y_n\mid X_n =x) - xE(Y\mid X_n =x) =0$$

This happens because by examining $X_n = x$ we have turned one of the random variables into a constant, and constants do not co-vary with anything.

Alecos Papadopoulos
  • 52,923
  • 5
  • 131
  • 241
1

You are probably having difficulty because you are visualizing the data as being Gaussian.

It's possible that all data represent the same point (though it would be redundant) and that you have two variables with different names (aliases of each other) comprising the data. This would lead to zero covariance, and a correlation of 1 as fundamentally, covariance represents how spread out the data is across the feature space, while correlation represents how much one variable depends on another, or the degree of influence they have on each other. If the data is not spread out at all, then covariance must be zero.

NOTE However the best thing you can do with such a dataset is simply predict all points as having the same output, which is most likely going to give a high bias

  • 2
    There seem to be quite a few different things going on in this answer, and I have a hard time seeing the relationship. For example, how is paragraph 1 relevant? How is paragraph 3 relevant? Also, how do you arrive at zero covariance in paragraph 2? – Richard Hardy Oct 16 '16 at 09:50
  • Thanks @Richard Hardy for pointing it out. One of the other answers initially suggested a Gaussian solution. That's why para 1. In para 3, I'm just giving my view on what he can do with such a dataset. Fundamentally, covariance represents how spread out the data is across the feature space. If the data is not spread out at , then covariance must be zero. I've added this to the answer too – R. S. Nikhil Krishna Oct 16 '16 at 12:26