11

Reading Wikipedia about canonical correlation analysis (CCA) for two random vectors $X$ and $Y$, I was wondering if principal component anslysis (PCA) is the same as CCA when $X=Y$?

amoeba
  • 93,463
  • 28
  • 275
  • 317
Tim
  • 1
  • 29
  • 102
  • 189
  • Please make it more clear: 1) `vectors X and Y` Is that two variables (columns of data) or two cases (rows); given that we're going to perform the analyses of variables. 2) `X and Y are the same` Did you want to say that X=Y or any other way round? – ttnphns Jun 15 '13 at 07:25
  • @ttnphns: 1) $X$ and $Y$ are two random vectors. They are two vectors of random variables, two sets of columns of data, not two cases (rows). 2) $X=Y$. – Tim Jun 15 '13 at 12:01
  • If each set consists of a single variable, there's one canonical correlation which is exactly the Pearson _r_ between them; and CCA becomes linear regression of X by Y and vice versa. Decomposition of that _r_ by means of PCA is a bit another story. PCA and CCA are different analyses. – ttnphns Jun 15 '13 at 15:18
  • Hi, @Tim, I am wondering if my answer was useful or if you maybe still have some further questions? If so, I would be happy to clarify. – amoeba Dec 11 '14 at 00:39
  • @amoeba: Yes, it is. I don't have further questions right now, and will read your reply later. Thanks for your reply .+1 – Tim Dec 11 '14 at 02:57
  • Came across this thread three years later... hope you had a chance to read the answer by now :-) – amoeba Feb 02 '18 at 22:01

1 Answers1

8

Let $\bf X$ be $n\times p_1$ and $\bf Y$ be $n \times p_2$ data matrices, representing two datasets with $n$ samples (i.e. observations of your random row vectors $X$ and $Y$) in each of them.

CCA looks for a linear combination of $p_1$ variables in $\bf X$ and a linear combination of $p_2$ variables in $\bf Y$ such that they are maximally correlated between each other; then it looks for the next pair, under a constraint of zero correlation with the first pair; etc.

In case $\mathbf{X} = \mathbf{Y}$ (and $p_1=p_2 = p$), any linear combination in one dataset will trivially have correlation $1$ with the same linear combination in another dataset. So all CCA pairs will have correlations $1$, and the order of pairs is arbitrary. The only remaining constraint is that linear combinations should be uncorrelated between each other. There is an infinite number of ways to choose $p$ uncorrelated linear combinations (note that the weights do not have to be orthogonal in the $p$-dimensional space) and any of them will produce a valid CCA solution. One such way is indeed given by PCA, as any two PCs have correlation zero.

So PCA solution will indeed be a valid CCA solution, but there is an infinite number of equivalently good CCA solutions in this case.

Mathematically, CCA looks for right ($\mathbf a$) and left ($\mathbf b$) singular vectors of $\mathbf C_{XX}^{-1/2} \mathbf C_{XY} \mathbf C_{YY}^{-1/2}$, which in this case is equal to $\mathbf I$, with any vector being an eigenvector. So $\mathbf a=\mathbf b$ can be arbitrary. CCA then obtains the linear combination weights as $\mathbf C_{XX}^{-1/2} \mathbf a$ and $\mathbf C_{YY}^{-1/2} \mathbf b$. In this case it boils down to taking an arbitrary basis and transforming it with $\mathbf C_{XX}^{-1/2}$, which will indeed produce uncorrelated directions.

amoeba
  • 93,463
  • 28
  • 275
  • 317