2

I'm learning about PCA. In general, I understand the concept and the underlying math, but I'm confused about a few things that I'm hoping someone can explain.

Lets say that after performing a PCA analysis on some high dimensional data I find that the first three components capture 99% of the variance in my data. Now what? Is it possible to find which features comprise the first, second, and third components? It seems that a common approach is to plot the principle components. For example, see this page under 2.3.1. Dimensionality Reduction and visualization for a PCA plot of the iris data. What is the interpretation of this plot? I see what I presume are two components: one component in red one component a mixture of green and blue?

Is there a general "next step" to do after PCA. I understand the concept, but fail to see how it can be practically applied. Thanks for the help.

turtle
  • 517
  • 4
  • 10

2 Answers2

6

First, the interpretation of the plot from the page you linked to. Each point is a data point from iris data set, transformed by PCA to 2-dimensional representation. The colors refer to distinct classes of irises (setosa, versicolor, virginica), not to principal components.

As Michael said, each principal component is a linear function of the variables. To see the coefficients of this function you can check the coefficients_ attribute in the PCA object from the tutorial you linked to.

The general next step after getting the components (each one a different linear function) is to apply those functions to the data, and get new lower-dimensional representation of the data. If the data was 100-dimensional, and you chose 7 principal components which explain a satisfactory portion of variance, you can from now on work on 7-dimensional data instead, which is easier and faster.

If you want to see a mathematical proof of PCA which uses only linear algebra, and which I hope will give you a good understanding of what PCA does, and how it does it, I have written one lately here : Making sense of principal component analysis, eigenvectors & eigenvalues

Community
  • 1
sjm.majewski
  • 3,548
  • 1
  • 19
  • 27
1

Well yes. Look a the magnitude of the coefficients in the linear combination of the components. That tells you how each variable contributes to each of these first few principal components.

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • Ok, lets say I had three coefficients: `0.98`, `0.09`, and `0.01`. It looks like most of the variance is captured in the first coefficient. How do I know what variables this corresponds too? – turtle Aug 05 '12 at 16:46
  • 1
    Each principal component is a linear function of the variables. Identify the coefficients of the variables in each component. The most influential is not necessarily the ones with the largest coefficients because they have different scales but the mean times the coefficient will give you an idea. – Michael R. Chernick Aug 05 '12 at 17:10