0

I am studying a data set with one continuous response variable and 4 continuous predictor variables. The two predictors most correlated with the response also have a high (but not perfect) correlation with each other. A more parsimonious model might rely on 3 predictors, but I don't want to drop any of the variables.

I am considering two approaches. One is to do principal components analysis (PCA) on the space of predictors, and running a principal components regression (PCR), regressing the response on the first 3 principal components of the predictors' space.

The other alternative is to run a factor analysis (FA), attempting to discover three factors underlying the predictors' space that best explain the response; the method of FA would be PCA.

My question is whether these approaches are equivalent, or whether there is a fundamental conceptual difference in the goals of these methods. Thank you!

Hal M. Switkay
  • 161
  • 1
  • 3
  • "The method of FA would be PCA" means that such an FA = PCA. Obviously, there is no difference then. – amoeba Apr 27 '17 at 13:35
  • Just to be sure, can you clarify what you mean by "the method of FA would be PCA"? What software are you using and what "method" are you talking about *exactly*? – amoeba Apr 27 '17 at 13:51
  • If so, @amoeba, I am curious why no article on PCR mentions its equivalence to PCA-FA, and no article on FA mentions the equivalence of PCA-FA to PCR. I am distinguishing between a FA performed using PCA versus one performed using another method like MLE. I am using R. I use prcomp for PCA. I am considering using the pls package for PCR, versus the psych package with factanal to do FA. – Hal M. Switkay Apr 27 '17 at 13:52
  • Wait a second - when you say that the other alternative is to run FA via PCA, you mean that you are later going to regress the response on the factor scores, right? Or something else? – amoeba Apr 27 '17 at 13:54
  • You have it right, @amoeba. – Hal M. Switkay Apr 27 '17 at 13:55
  • `factanal` only does ML. So it's still not clear what exactly you mean by PCA-FA. – amoeba Apr 27 '17 at 13:55
  • My R help is showing something else, @amoeba. The psych package, function fa, does indeed permit different methods: fm = "pa" provides a principal axes solution, while fm = "ml" does a maximum likelihood solution. Other methods are available, according to the documentation I'm seeing. – Hal M. Switkay Apr 27 '17 at 13:59
  • Ah yes, `fa` allows that (not `factanal`). So do you mean `pa` method? That's "principal axis" method, it's not equivalent to PCA. See https://stats.stackexchange.com/questions/50745. – amoeba Apr 27 '17 at 14:00

0 Answers0