0

I'm interested in the interpretation of the solution to the factor analysis problem with a 2-norm equality constraint on the columns of the loadings matrix. I plan to decompose $\mathbf{X}_i \in \mathbb{R}^{d}$ as $\mathbf{E}\mathbf{F}_i + \boldsymbol{\epsilon}_i$, where $\mathbf{E} \in \mathbb{R}^{d \times p}$ is the loadings matrix, $\mathbf{F}_i \in \mathbb{R}^{p}$ are the factors, and $\boldsymbol{\epsilon}_i \in \mathbb{R}^d \sim \mathcal{N}(\mathbf{0}, \mathbf{B})$.

We certainly need some constraint on $\mathbf{E}$ or $\mathbf{F}_i$ to make the parameters identifiable. For example, PCA is simply a factor analysis problem in which the constraint $\mathbf{E}^T\mathbf{E} = \mathbf{I}$ must be satisfied. If I solve for $\mathbf{E}$ and $\mathbf{F}_i$ such that the each column of $\mathbf{E}$ has 2-norm equal to $1$, is there some special interpretation of the factors like there is in PCA (where each factor is independent)? Each column of $\mathbf{E}$ lies on the unit ball, but what implications does that have on the solution?

Vivek Subramanian
  • 2,613
  • 2
  • 19
  • 34
  • Factor analysis does not solve for (unambiguously estimates) factor _values_ F, it estimates only loadings. Factor _scores_ [can be computed afterwards](http://stats.stackexchange.com/q/126885/3277) with the help of the loadings but are only approximations. Unlike PCs, true Fs are not linear combinations of variables. – ttnphns May 15 '15 at 10:05
  • Also, it is not a good idea to call [eigenvectors "loadings"](http://stats.stackexchange.com/a/143949/3277). In PCA, loadings after extraction are the eigenvectors scaled up to the respective eigenvalues, so the inner product matrix of the loadings is diagonal, not identity. – ttnphns May 15 '15 at 10:11

0 Answers0