2

It's very common for social scientists to have binary or ordinal indicators (e.g., yes/no interview items or likert-rated questionnaire items). It's also very common for them to model their ordered/non-ordered categorical data with structural equation model (SEM)/factor analytic (FA) techniques using weighted least estimators (e.g., WLS, robust WLS or WLSMV).

Yet, as far as I can tell, there is no established method for comparing two SEM/FA models which are not nested. With maximum likelihood estimators, people usually use AIC and BIC but they are not available for WLS estimation. I've seen some use fit statistics such as CFI, TLI, and RMSEA, but have not managed to find an empirically grounded criteria indicating a substantial change in model fit (e.g., compared to the less restricted model, an increase in RMSEA of say .05 suggests that the model fit worsened meaningfully).

Can anyone recommend an empirically credible criteria for model comparison using WLS estimation for non-nested models? If there is no empirically tested criteria, perhaps share your experience of what you see in the field.

PyjamaNinja
  • 479
  • 3
  • 12

1 Answers1

1

My understanding of the field is that the establishment and verification of ubiquitous fit statistics or fit criteria for WLS estimation is still an open topic.  That is to say, there are no established statistics, nor are there rigorously empirical criteria that have been well-established (via simulation research) to assess model fit, in general, let alone for model comparison.  This is not to say that there are not criteria that have not been proposed, but they have not been universally accepted.  (Of course, if I have missed something in the literature...please share any useful references).

As the fit statistics are sparse or currently non-existent, the ability to compare nested and non-nested models also is not readily available.  However, there is one advantage to binary and ordinal predictors that can be exploited.  It is possible to compare the contingency tables of the observed counts to the predicted counts.  Thus, one could at least ask if different models predict equally well or better just based on chi-squared statistics.  Of course, this requires non-zero entries...and the larger the number of variables being predicted, the more likely this will be an issue.

Happy to elaborate on anything (if I can), and I hope this is helpful.  Again, if I am not as current with the literature as I should be, any and all corrections are welcome.

Gregg H
  • 3,571
  • 6
  • 25