0

I have survey data measuring the BIG five personality test. In total, there are 60 variables measuring the five components in a five-point scale. The goal is to look at whether certain experiences in childhood affect these personalities in a regressinal framework. So I need to generate a variable for each personality dimension.

Instead of summing all questions to get a score for each personality dimension, I was told to use factor analysis. (Does anyone have reference for the rationale of prefering factor analysis in this context?)

I understand the intuition and math for the factor analysis and principal component analysis. But I don't know how to use these factors in an econometric framework after getting the factor loadings. And I am quite confused when people use PCA interchangeably with factor analysis. Are these two the same thing?

In the statistical package I use (STATA), the factor command has a few optimizing options. One of them is pf (principal factor), another is principal component factor. And they give substantially different results. How are these two methods different?

Yan Song
  • 137
  • 5
  • 2
    People do use PCA and factor analysis interchangeably but they are distinct. PCA is the 'grandfather' of all variants of linear component analysis as it was first invented by Pearson in 1901. Rotation of the components (factors, dimensions) came later and is most associated with factor analysis. The distinction is that PCA is deterministic, i.e., there is no error term which means that the solution is mathematically unique. Factor analysis, on the other hand, is a model with an error term and since it rotates the components its solutions are no longer mathematically unique. – Mike Hunter Mar 05 '18 at 12:28
  • 1
    Another difference is in the purpose of the methods. PCA is about dimension reduction, FA is about finding latent traits. – Peter Flom Mar 05 '18 at 12:32
  • See also the linked threads (column to the right) discussing PCA vs. FA and comments under ReneBt's answer. – Richard Hardy Mar 05 '18 at 12:35
  • this seems like several questions; they should be asked separately – Peter Flom Mar 10 '18 at 13:57

1 Answers1

1

I’ll answer you questions the other way around:

Are PCA and factor analysis the same thing?

According to some definitions PCA is a type of factor analysis (where the term is used to describe a general family of methods such as here http://www.statisticssolutions.com/factor-analysis-sem-factor-analysis/ which explicitly includes PCA as a factoring method and here with a definition that allows PCA http://www.statisticshowto.com/factor-analysis/). According to others a much more strict as specific definition lead them to conclude they are separate (thanks to @Richard Hardy for the useful links in the comments that explain this in some depth). Factor analysis basically a set of tools to uncover hidden patterns in your data, which is why some people broaden the definition to include methods such as PCA which has the same basic aim. PCA specifically seeks to maximise the covariance (or correlation when appropriately scaled) in as few orthogonal factors as possible. Other methods can impose different limits (non-zero, orthogonality in scores as well and many more).

How do I estimate the relationship between the extraversion dimension and problematic behavior?

PCA will not return pure relationships between specific variables and an issue you are interested in. There are many different tools that are designed to do this (by the sound of it discriminant analysis may be relevant if you have categorical (problematic behaviour and non-problematic behaviour). However, PCA will be a powerful tool to explore your data. What PCA factors will reveal is how the extraversion behaviour interacts with the other dimensions. It will reveal what variables it tends to coincide with, which ones it is unrelated to and which ones it is inversely related to. The first PCA will show you the largest source of variation and so the strongest relationships between the different variables. Each PC after that describes an ever decreasing proportion of the variation, revealing different sets of interactions. Typically in real-world datasets such as yours, the hope is that some of these natural groupings of interactions is associated with some independent variable (in your case problematic behaviour). You would examine eigenvalues and restrict your focus to those >1 (this means the factors that describe at least the same amount of information as the original variables, other constraints can be used if this still leaves too many PCs to go through). Carry out statistical tests on the scores (scores are basically the proportion of those PC factors in each sample) to see if any of the factors provide a significant difference between your groups. If none of the PCs naturally describe a difference then discriminant methods are worth looking at, but as they deliberately reduce the data to enhance contrast in your grouping variable one must take great care to validate the model to ensure it is not overfitting.

If you specifically want to know how one dimension interacts with your dependent variable then use the relevant univariate test. The advantage of PCA compared to sequential independent statistical tests on each variable is that it summarises how each interacts and gives a more complete picture. So you will understand which other dimensions need to coincide with extraversion to make subjects more likely to be problematic.

ReneBt
  • 2,863
  • 1
  • 8
  • 24
  • 1
    I would suggest reading https://stats.stackexchange.com/questions/95038/how-does-factor-analysis-explain-the-covariance-while-pca-explains-the-variance and https://stats.stackexchange.com/questions/1576/what-are-the-differences-between-factor-analysis-and-principal-component-analysi?noredirect=1&lq=1, especially the contributions by @ttnphns; there the relation between PCA and factor analysis is explored quite nicely. My impression is that PCA and FA have quite different foundations and the relationship between the two is more complicated than might seem on the first glance. – Richard Hardy Mar 05 '18 at 11:45
  • 1
    Perhaps the two are similar on the surface but quite different in their fundaments. The following excerpt from the second link above summarizes it quite well: *...PCA is used as the default extraction method in the SPSS Factor Analysis routines. This undoubtedly results in a lot of confusion about the distinction between the two. The bottom line is that these are two different models, conceptually.* – Richard Hardy Mar 05 '18 at 11:48
  • This is the type of strict definition I refer to, which does indeed exclude PCA. I will insert some references including this to show what I am meaning. Thanks for the useful links which illustrate why we need to be precise when we use a term such as factor analysis. – ReneBt Mar 05 '18 at 12:00
  • Thanks. My point was not to stress a formal distinction between PCA and FA but the fundamental one. The latter is perhaps more often overlooked. For example, I was teaching introductory FA based on PCA and then rotation last week. I felt it was very unfortunate I had to include PCA in the picture. I would rather have taught these two on completely separate occasions and then shown the connection between the two on a third occasion so as not to blur the fundamental differences between the two. But this is just my take. – Richard Hardy Mar 05 '18 at 12:06
  • "PCA specifically seeking to maximise the covariance in as few orthogonal factors as possible". It's tough to summarize anything well in half a sentence, but this deserves qualifiers. First, PCA yields as many components as variables. Whether just some of those deserve use or attention is a researcher's decision. Second, PCA often (in my experience usually) is based on a correlation matrix. – Nick Cox Mar 05 '18 at 12:10
  • I think it is right to provide people access to sources to help them understand what people mean. Its a problem in all walks of life, people using a term to mean something very specific and then others use it to mean something more general, meaning we aren't always talking about the same thing even when we use the same words. – ReneBt Mar 05 '18 at 12:11
  • I don't think it's too facetious to say that people who like factor analysis regard PCA as a limiting or even degenerate special case, while people who don't like factor analysis are more likely to insist that they are different beasts. (I think my position on this sometimes contentious detail aligns with that of @Richard Hardy.) – Nick Cox Mar 05 '18 at 12:12
  • @Nick Cox, it is true that many people use correlation matrices for PCA, but it is not essential This can achieved by scaling the data before PCA, which means that the first step of PCA (D'D) gives a correlation matrix, which is simply the covariance matrix of scaled data. or you can find the eigenvalue then use that to convert covariance to correlation. – ReneBt Mar 05 '18 at 12:15
  • Agreed and I did know that. If you want to edit your summary to say something like "(possibly standardized) covariance matrix" the small point would be resolved. At the same time I appreciate that you want to keep your explanation informal and don't purport to be complete or rigorous. (I'd say there was a case for regard PCA as just eigenvector/eigenvalue-based calculations with certain kinds of matrices (not just covariance or correlation matrices). – Nick Cox Mar 05 '18 at 12:19