Estimated distribution of eigenvalues for i.i.d. (uniform or normal) data

Question

Assuming I have a data set with $d$ dimensions (e.g. $d=20$) so that each dimension is i.i.d. $X_i \sim U[0;1]$ (alternatively, each dimension $X_i \sim \mathcal N[0;1]$) and independent of each other.

Now I draw a random object from this dataset and take the $k=3\cdot d$ nearest neighbors and compute PCA on this set. In contrast to what one might expect, the eigenvalues aren't all the same. In 20 dimensions uniform, a typical result looks like this:

0.11952316626613427, 0.1151758808663646, 0.11170020254046743, 0.1019390988585198,
0.0924502502204256, 0.08716272453538032, 0.0782945015348525, 0.06965903935713605, 
0.06346159593226684, 0.054527131148532824, 0.05346303562884964, 0.04348400728546128, 
0.042304834600062985, 0.03229641081461124, 0.031532033468325706, 0.0266801529298156, 
0.020332085835946957, 0.01825531821510237, 0.01483790669963606, 0.0068195084468626625

For normal distributed data, the results appear to be very similar, at least when rescaling them to a total sum of $1$ (the $\mathcal N[0;1]^d$ distribution clearly has a higher variance in the first place).

I wonder if there is any result that predicts this behavior? I'm looking for a test if the series of eigenvalues is somewhat regular, and how many of the eigenvalues are as expected and which ones significantly differ from the expected values.

For a given (small) sample size $k$, is there a result if a correlation coefficient for two variables is significant? Even i.i.d. variables will have a non-0 result occasionally for low $k$.

hmmm, could you print those results with fewer sig figs? I cannot parse them easily... — shabbychef, Jun 28 '12 at 22:19
Well, as you can see the magnitude is of interest. Naively, one would expect them to all have the same magnitude. — Has QUIT--Anony-Mousse, Jun 29 '12 at 06:17

score 7 · Answer 1 · answered Jul 05 '12 at 16:21

There is a large literature on the distribution of eigenvalues for random matrices (you can try googling random matrix theory). In particular, the Marcenko-Pastur distribution predicts the distribution of eigenvalues for the covariance matrix of $i.i.d.$ data with mean of zero and equal variance as the number of variables and observations goes to infinity. Closely related is Wigner's semicircle distribution.

Estimated distribution of eigenvalues for i.i.d. (uniform or normal) data

1 Answers1

Linked