2

I'm in the early stages of planning an analysis which would involve a large number of ordinal variables, and I'm trying to work out if I have enough observations in my sample. The ordinal variables are always predictors, rather than outcomes (i.e. they're either measures of the latent variable we're interested in predicting, or predictors of it).

I know that in the regression world there's a suggestion of 10-20 observations per predictor. There seems to be a similar 10-20 per variable idea with SEM. If I was dealing with ordinal predictors in a regression then I would be aiming for (k-1)*(10-20) observations per predictor (i.e. 10-20 observations for each parameter being estimated for that predictor). Would the same extension of the rule of thumb hold when using SEM, rather than a single regression?

EDIT: To provide a little more detail on the model that we have in mind, we have around 25-30 (ordinal and continuous) indicators loading on likely 3-5 latent variables, and an additional 14 exogenous measures (mostly continuous). It's going to be the relationship between these exogenous measures and the latent variables that we'll be interested in. In particular, whether the measures are significant predictors of the latent variable.

Ian_Fin
  • 1,129
  • 8
  • 18
  • 1
    Are you using Mplus? It has sample size calculation tools. – AdamO Dec 22 '15 at 18:37
  • The original plan had been to use lavaan for R. However, given the combination of ordinal and continuous variables, it looks like Mplus will have to be what we use. I'm working on this plan as part of a funding proposal, and that will include funding for Mplus, which my institution doesn't have any licenses for. Can't really apply for funding without knowing the sample size. Can't estimate sample size without funding (for Mplus)... – Ian_Fin Jan 05 '16 at 09:18
  • @Ian_Fin Could you please provide the source where I can found the formula you provided (k-1)*(10-20) for calculating sample size when doing ordinal regression. Thank you. – Quirik Mar 21 '17 at 18:12
  • @quirik Just to be clear, I meant an ordinal predictor in a linear regression, rather than an ordinal outcome. It's been a while since I wrote that, but I think my thinking was likely of treating the ordinal variable as a nominal one (if there were only a few levels), which would be represented with k-1 variables. The rule of thumb of 10-20 observations per variable is pretty common and it shouldn't be too hard to find a reference. I normally think of Harrell's Regression Modelling Strategies. – Ian_Fin Mar 21 '17 at 18:25
  • @Ian_Fin Sorry, my mistake. I am searching for any rule of thumb for an ordinal outcome with only one group (there are examples with treatment and control groups). – Quirik Mar 21 '17 at 18:30
  • @quirik Perhaps half of what is suggested with a treatment and control group? Might be worth asking a new question about that. – Ian_Fin Mar 21 '17 at 18:33

1 Answers1

4

First, a clarification: in a traditional SEM, indicators of a latent variable are specified as an outcome of a latent variable--not a cause (i.e., the causal arrow is in the direction of from latent variable to indicator, not the other way around).

I'm away from my office, and therefore SEM texts, but I've written about SEM sample size needs on a related (though not identical) post here. In a nutshell, the simulation research cited by Little (2013) suggests that these observations:variable ratio guidelines perform quite poorly. Sometimes you could get away with as few as 50-100 observations, and other times your needs will be many 100s.

One way to think of power in SEM is to strive for enough observations that you are confident that you have relatively precise estimates of variances and covariances for each of your observed variables. After all, SEM is a means of more parsimoniously representing these variances and covariances.

All else being equal, model complexity is probably your main concern. You never articulate your model (maybe edit your post to include?), but if you're just talking about modelling a few predictors of a latent variable or two, your sample size needs might be more modest. But the more latent variables you estimate, and the more indicators each has, etc., the larger your sample size needs with be. If you have a complex model, you might want to consider parceling indicators to simplify the measurement model (Little, Cunningham, Shahar, & Widaman, 2002).

AdamO's suggestion is a good one: if you know the model you want to evaluate, you could run a few Monte Carlo power simulations, varying the sample size to see approximately how many you need. This is pretty straightforward to do in MPlus, and the simsem package for R is a nice alternative, especially if you plan on doing your analyses with the lavaan package.

Two additional pieces of information would be helpful in tailoring this answer to your needs:

  1. What, specifically, are you concerned about having adequate power for? To test the significance of a particular model parameter (e.g., a latent correlation, or regression)? To test a particular model constraint (e.g., examining group-invariance)? To appropriately reject poor-fitting models? Your sample-size needs might vary a bit depending on which is your primary concern.
  2. What is/are the response scale(s) for your ordinal variables? Rhemtulla, Brosseau-Liard, & Savalei (2012) suggest that variables with four or fewer response options are bests modelled with categorical estimators, whereas anything more than four is good enough to consider "continu-ish", and estimate using robust maximum-likelihood estimators (see here for a related post and answer). However, for what it's worth, I tend to strive to have bigger sample sizes when using a categorical estimator (I don't know if simulation research has been done on this topic, or supports my tendency).

References

Little, T. D. (2013). Longitudinal structural equation modeling. New York, NY: Guilford Press.

Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9, 151-173.

Rhemtulla, M., Brosseau-Liard, P. E., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354-373.

Community
  • 1
jsakaluk
  • 5,006
  • 1
  • 20
  • 45
  • Thanks, first, for the clarification on the arrow of causality in SEM. – Ian_Fin Jan 05 '16 at 09:24
  • To tackle your questions, our interest will be predominantly in the relationships between variables, so it'll come down to the significance of the coefficients between the variables. We have a combination of continuous and ordinal variables. Unfortunately, the ordinal ones have different numbers of responses (ranging from 2-7). – Ian_Fin Jan 05 '16 at 09:53
  • In some cases I think they can be relatively sensibly converted to continuous variables. For example, scales like "Once a week", "Once a month", "Once a year" could be thought of as number of times a year, e.g. 52, 12, 1 (and then possibly transformed as needed). But this may not work in all cases. We have around 25-30 (ordinal and continuous) indicators loading on maybe 3-5 latent variables, and an additional 14 exogenous measures (mostly continuous). – Ian_Fin Jan 05 '16 at 09:54
  • The model you describe is one you should be able to program a power simulation for pretty easily. If your funding for Mplus doesn't come through, you should really give the lavaan/simsem packages a look to meet your needs. – jsakaluk Jan 05 '16 at 14:28
  • @jsakuluk Does lavaan allow for models which contain both categorical and continuous variables? I was under the impression that it didn't. – Ian_Fin Jan 05 '16 at 14:58
  • It certainly does. See this link from the lavaan site about using categorical exogenous and endogenous variables (alongside continuous ones): http://lavaan.ugent.be/tutorial/cat.html – jsakaluk Jan 05 '16 at 15:00