3

I have 20 items rated on 25 dimensions. Each item was rated on each dimension ~30 times. Each of ~40 participants rated 15 of the 20 items on all of the dimensions.

I had originally thought to do PCA on the item means (i.e. averaging across all ratings for that item), but realized that I will have fewer observations than dimensions, which I understand is not idea for PCA.

My question is, can I do PCA at the trial level (i.e. using each individual rating [e.g. participant #1 rating item #1 on dimension #1])? It seems that I should somehow account for the fact that the data are non-independent (i.e. nested within subjects and items). I've seen PCA with random effects mentioned on here, but haven't come across a definitive way to do this.

Is it okay to do PCA at the trial level? Are there additional steps I need to take?

mkt
  • 11,770
  • 9
  • 51
  • 125
Dave
  • 1,641
  • 2
  • 14
  • 27
  • What is your ultimate analytical goal? There may be other ways to achieve it. – mkt Apr 03 '18 at 11:14
  • I am just interested to know if there are underlying factors in the 25 dimensions measured. I had assumed that I needed to account for the fact that ratings came from the same people/items. – Dave Apr 03 '18 at 17:14

2 Answers2

1

I had originally thought to do PCA on the item means (i.e., averaging across all ratings for that item), but realized that I will have fewer observations than dimensions, which I understand is not idea for PCA.

That's not necessarily the case, for example see this question.

Also you could try models that explicitly handle missing data, like probabilistic matrix factorization. The general term you could google is low-rank matrix factorization in the context of recommender systems. For an implementation you can see H2O - see this example for slides on application for missing values.

If you're into Python you could also check this example (it uses H2O on Movielens dataset).

Jakub Bartczuk
  • 5,526
  • 1
  • 14
  • 36
1

You could perform a multilevel factor analysis (Muthén, 1991, 1994; see Reise et al., 2005 for an accessible tutorial, and see Kim et al., 2016, for an updated overview). From Reise et al., the process sounds pretty straightforward:

  1. Conduct a generic EFA, ignoring dependency (gives you a starting, albeit misinformed factor solution).
  2. Estimate ICCs for measurements within each item (helps you appraise to what extent the multilevel approach is needed)
  3. Conduct factor analysis on within-subject correlation matrix
  4. Conduct factor analysis on between-subject correlation matrix

Given Muthén's publishing on this approach, it's a virtual certainty that it can be implemented in Mplus. I don't recall this kind of multilevel functionality for factor analyses in SPSS, and am unsure about what options you might have for R (I know there are packages for specifying multilevel confirmatory models, but unsure about exploratory models).

References

Kim, E. S., Dedrick, R. F., Cao, C., & Ferron, J. M. (2016). Multilevel factor analysis: Reporting guidelines and a review of reporting practices. Multivariate Behavioral Research, 51(6), 881-898.

Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28(4), 338-354.

Muthén, B. O. (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22(3), 376-398.

Reise, S. P., Ventura, J., Nuechterlein, K. H., & Kim, K. H. (2005). An illustration of multilevel factor analysis. Journal of Personality Assessment, 84(2), 126-136.

jsakaluk
  • 5,006
  • 1
  • 20
  • 45
  • This looks like it might be perfect! It's clear that it can handle multiple nested random factors, but I'm unclear if it can handle multiple crossed random factors? – Dave Apr 03 '18 at 17:24