1

I am trying to get into SEM and factor analysis. I understand a factor is a latent construct, say e.g. $intelligence$, user-defined by the (weighted) average of a set of indicators $x_1, x_2\dots x_n$. The $x_i$ load (i.e. correlate) on the factor differently and the loadings should be over $0.40$ at minimum. So far so good (presumed).

When I run a factor analysis in Stata with the factor command, in the output there are a number of factors being displayed instead of only one which I define. The literature I found tells then, the factor with the highest eigenvalue is the best one.

Why there are a number of factors being displayed since with my indicators $x_i$ I intend to define just one? All the literature I found just says e.g. "the first factor has the strongest eigenvector" or "only one factor has an eigenvector greater than $1$". I'm missing the step where factors or their sets respectively are derived from their sets of indicators. I'm confused now, what factors actually are since in a defined set of indicators there could be more than one.

I am quite sure my question is very obvious to anybody being familiar with this stuff. I'd appreciate any clarification though.

Here is an example of a Stata output that should look familiar to anybody.

. factor x1-x9, pcf
(obs=1,625)

Factor analysis/correlation                      Number of obs    =      1,625
    Method: principal-component factors          Retained factors =          1
    Rotation: (unrotated)                        Number of params =          9

    --------------------------------------------------------------------------
         Factor  |   Eigenvalue   Difference        Proportion   Cumulative
    -------------+------------------------------------------------------------
        Factor1  |      3.76124      2.80650            0.4179       0.4179
        Factor2  |      0.95473      0.10627            0.1061       0.5240
        Factor3  |      0.84847      0.10176            0.0943       0.6183
        Factor4  |      0.74671      0.05561            0.0830       0.7012
        Factor5  |      0.69110      0.07429            0.0768       0.7780
        Factor6  |      0.61681      0.07780            0.0685       0.8466
        Factor7  |      0.53900      0.09177            0.0599       0.9065
        Factor8  |      0.44723      0.05252            0.0497       0.9561
        Factor9  |      0.39471            .            0.0439       1.0000
    --------------------------------------------------------------------------
    LR test: independent vs. saturated:  chi2(36) = 3863.18 Prob>chi2 = 0.0000

Factor loadings (pattern matrix) and unique variances

    ---------------------------------------
        Variable |  Factor1 |   Uniqueness 
    -------------+----------+--------------
              x1 |   0.6243 |      0.6103  
              x2 |   0.5883 |      0.6539  
              x3 |   0.7222 |      0.4785  
              x4 |   0.7131 |      0.4915  
              x5 |   0.5818 |      0.6615  
              x6 |   0.6197 |      0.6160  
              x7 |   0.6085 |      0.6297  
              x8 |   0.5968 |      0.6439  
              x9 |   0.7392 |      0.4535  
    ---------------------------------------
jay.sf
  • 556
  • 3
  • 19
  • You actually did not define a "factor" when you use `factor` command in stata. It corresponds to what is sometimes called exploratory factor analysis (EFA). So basically, you *explore* the patterns/structure underlying observed variables. That is why it suggests several factors. In Confirmatory Factor Analysis (CFA, which is a form of SEM), you specify the factor and test a hypothesis on the factor structure. There are many questions on EFA and CFA on CV. I recommend checking them. – T.E.G. Mar 31 '18 at 17:50
  • The example is from a textbook introducing into SEM. SEM then uses CFA, I know. Since I used option `pcf` I actually used principal component analysis (PCA). So your point is, PCA recommends me factors, aha. I could not find any question that clarifies my problem more than your comment, why don't you consider to point me to one of these or put some effort in it and make an answer out of it, thanks. – jay.sf Mar 31 '18 at 19:12
  • PCA and factor analysis using principal-components factoring method (`pcf`) yield very similar results, but they are not the same. In fact, PCA and EFA are used for different tasks. Here are some links which contain detailed discussions: https://stats.stackexchange.com/questions/1576/what-are-the-differences-between-factor-analysis-and-principal-component-analysi , https://stats.stackexchange.com/questions/311020/why-would-one-do-cfa-instead-of-efa , https://stats.stackexchange.com/questions/123063/is-there-any-good-reason-to-use-pca-instead-of-efa-also-can-pca-be-a-substitut , – T.E.G. Apr 01 '18 at 01:25
  • Unfortunately, I am very busy nowadays. But I will return for an answer when I have time. – T.E.G. Apr 01 '18 at 01:27

1 Answers1

1

First, let me suggest that your question is not necessarily obvious, but it uncovers one of the extreme complications surrounding factor analysis (and I, too, struggled with this while I was learning these concepts).

In brief, there is a difference between a latent construct influencing how observed variables are related to each other and measuring a latent construct by taking weighted averages of those observe variables. I'll stick with the congeneric (single) factor analysis model in this answer.

Technically, factors are not measured by weighted averages of indicators variables...though based on any reading of most texts that introduce these concepts, that is a reasonable error to make. In truth, factors are estimated using weighted averages of indicators. The distinction may seem minor, but in truth, it is very important. And this distinction stems from the direction of the arrows between the latent factor and the manifest (observed) variables. The arrows point from the latent factor to the manifest variables. This means that the value of the latent variable (that which we cannot see) is directly influencing the variables that we can see (that which we measured).

So, if I have a set of variables that I believe are influenced by one single factor, then the observations you cite in the question are correct: there should be one large eigenvalue, that should be the only eigenvalue greater than one, the (standardized) factor loadings should be at least 0.4, etc. But, remember the direction of the arrows for this process. The latent variable exists first; the measured items exist second. But, if this is theoretically sound way to measure that which we cannot see, then going in reverse, taking a weighted average of these values should provide a reasonable estimate for that latent factor that I can't measure directly.

This brings us to the next issue of confirmatory factor analysis (starting with a model that dictates which items load on which factors) vs. exploratory factor analysis (which attempts to locate possible factors among the measured variables). If you don't know how many factors there may be that are influencing your observed responses, then it is useful to employ a technique that generates multiple factors. With that info, you can then use various criteria to decide how many factors are appropriate. If you believe it is only one factor, then this would employ criteria comparable to what you mentioned above. However, the output of most procedures (like the example you provide from Stata) provides information on all the other possible factors.

So, ¿what are those other factors? Well, here's where a bit of 9-dimensional geometry is needed. Well, maybe not...because we can possible reduce the cloud of data in 9-dimensional space to a smaller dimension. The image that helps here is to think about a cigar, a frisbee, and a rugby ball. If the cloud of points can essentially be contained in a cigar shaped cloud, then there is really only one dimension...or, there is only one factor. If the cloud of points is more frisbee shaped, then there are two dimensions (or two factors) that best describe the data. If the cloud of points is more contained in an oval shaped football, then there are three dimensions/factors. If you have a strong theoretical construct, then the single factor should be sufficient (i.e., the data can be described using one dimension—one factor).

Happy to clarify as needed.

Gregg H
  • 3,571
  • 6
  • 25