6

$$ r = \frac{{\rm Cov}(X,Y)}{ \sigma_{X} \sigma_{Y}} $$ I do not understand this equation at all. Where does it come from?

From my personal understanding ${\rm Cov}(X,Y)$ comes from that fact that $X$ and $Y$ are dependent random variables, that is, $E[XY]$ is not the same as $E[X]E[Y]$. Is this analogous to saying that $P(A \cap B) = P(A)P(B|A)$ if $A$ and $B$ are not independent? I'm just confused as to why we want the ratio of $E[XY]-E[X]E[Y]$ over the product of the standard deviations for $X$ and $Y$.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Person
  • 406
  • 2
  • 6
  • You may read [here](http://stats.stackexchange.com/a/30724/3277) that this formula reduces to the formula of the [cosine](http://stats.stackexchange.com/a/36158/3277) similarity, and _r_ [is the cosine](http://stats.stackexchange.com/a/22520/3277) for centered data. – ttnphns Oct 19 '13 at 07:25

1 Answers1

5

One nice thing you get from dividing by the product of standard deviations is that it guarantees that the correlation coefficient will be between -1 and +1.

If you want to determine if $X$ has a stronger linear relationship with $Y$ or with $Z$ comparing $cov(X,Y)$ with $cov(X,Z)$ directly is not informative, since the scale of each of the covariances depends on the variance of $Y$ an $Z$, which could be very different.

Dividing by $\sigma_X \sigma_Y$ normalizes the covariance, so you can compare $cor(X,Y)$ with $cor(X,Z)$ in meaningful way.

Max S.
  • 1,666
  • 8
  • 7
  • Good point, but how do we know that the Cov(X,Y) is less than or equal to the product of the standard deviations of X and Y? – Person Oct 18 '13 at 20:38
  • 2
    I've never actually proven myself, but some Googling brought up this page: http://www2.math.umd.edu/~ddarmon/teaching/stat400/correlation-proof.pdf – Max S. Oct 18 '13 at 20:51