Why does varimax applied to PCA outcome fail to do anything at all?

Question

I am trying to perform PCA with varimax rotation on a set of data with 2 variables, in order to orthogonalise them (not for dimension reduction). I used prcomp() to perform PCA and it gave me results like this:

> print(variable.pca)
Standard deviations:
[1] 1.1104314 0.8420727

Rotation:
                  PC1        PC2
LogFreq    -0.6646101 -0.7471903
Simulation -0.7471903  0.6646101

> summary(variable.pca)
Importance of components:
                          PC1    PC2
Standard deviation     1.1104 0.8421
Proportion of Variance 0.6349 0.3651
Cumulative Proportion  0.6349 1.0000

Then I applied varimax() to the variable.pca$rotation. And here's the results:

$loadings

Loadings:
           PC1    PC2   
LogFreq    -0.665 -0.747
Simulation -0.747  0.665

               PC1 PC2
SS loadings    1.0 1.0
Proportion Var 0.5 0.5
Cumulative Var 0.5 1.0

$rotmat
             [,1]          [,2]
[1,] 1.000000e+00 -7.142042e-16
[2,] 7.142042e-16  1.000000e+00

This result is completely uninterpretable to me. I thought that, after varimax rotation, one variable should load heavily on one factor and nearly 0 on the other factor, whereas the other variable should be the reverse. Was there something I did wrong? Or should I not use prcomp() to perform orthogonalisation? I'm pretty new to using PCA for orthogonalisation.

Answer 1

General remarks:

1. One should not varimax-rotate eigenvectors of PCA but loadings of PCA (i.e. eigenvectors scaled up by respective standard deviations).

2. Also, it does not make sense to rotate all PCA loadings, one would usually choose a subset of leading PCs and rotate those.

If you, nevertheless, take all eigenvectors of PCA and varimax-rotate those (as you did here), you will get the original variables back; i.e. the identity matrix (possibly permuted). See Strange results of varimax rotation of principal component analysis in Stata: rotated components are all zeros and ones. However, it seems that for some strange varimax() in R does not rotate $2\times 2$ matrices correctly.

---

The non-rotation in the 'VARIMAX' - procedure seems to be a(nother) bug in the R-package.

Checking your result with my matrix-tool MatMate I get the rotation to the identity matrix by the following commands:

---

a) using my matrix-tool MatMate:

[17] labels = {"LogFreq","Simulation"} '   //' // the [number] is command numbering
                                           // for reference
[18] loadings= {{ -0.6646101, -0.7471903}, _
                { -0.7471903, 0.6646101 }}

 [loadings]      pc1         pc2
   LogFreq: -0.66461    -0.74719
Simulation: -0.74719     0.66461


[20] vm = rot(loadings,"varimax")

  [VARIMAX]      vm1         vm2
   LogFreq:  0.00000    -1.00000
Simulation: -1.00000     0.00000

Now using normalization of the columns to remove artifacts from decimal truncation of the loadings-values (which might have made small deviations from being eigenvectors )

[22] evectors = normsp(loadings)  // columnwise norming to sqsum = 1

[eigvectors]         ev1         ev2
    LogFreq:    -0.66461    -0.74719
 Simulation:    -0.74719     0.66461

// checking the difference of the ev's from the loadings' column-lengthes
[27] chk = (hypothenusesp(loadings)-1)*1e8
[28] disp = {"column_norms(loadings) - 1 = "}||format(chk,6.4)+"e-8"

column_norms(loadings) - 1 =    -3.5282e-8  -3.5282e-8

 // rotate the true eigenvector-type matrix to varimax    
[24] vm = rot(evectors,"varimax")

  [VARIMAX]      vm1         vm2
   LogFreq:  0.00000    -1.00000
Simulation:  -1.00000    0.00000

---

b) Because there was some concern, that SPSS as well as R might not perform the rotation correctly here also the protocol using SPSS (V21). The following way I got the correct (rotated to unit-matrix) components:

data list records=1 
    /1 ROWTYPE_ 1-8(A) FACTOR_ 10
       LogFreq 12-21 (F,8) Simulation 23-33 (F,8) .
begin data 
FACTOR   1 -0.6646101 -0.7471903 
FACTOR   2 -0.7471903  0.6646101  
END DATA.

FACTOR  
  /MATRIX   IN( FAC = * ) 
  /PRINT    extraction rotation
  /ROTATION varimax    .

Part of the result:

Rotierte Komponentenmatrix ^a
    Komponente  
              1           2
LogFreq      ,000       1,000
Simulation  1,000        ,000

Rotationsmethode: Varimax mit Kaiser-Normalisierung.        
^a Die Rotation ist in 3 Iterationen konvergiert.