0

I've been trying to search in the literature to see whether it makes sense to adjust for the variables I used to create matched pairs. To give context, I have a population of schizophrenia patients and a population of "general health" patients. I conducted a 1:1 matching of schizophrenia patients and general health patients, matching on age (within 5 years), sex, and race. Now, I have a dataset of about 90,000 patients, with 45,000 matched pairs.

I want to run logistic regression models and want to quantify the effects of age, sex, and race on my outcome variables and therefore want to throw them into my generalized estimating equations (GEE) or my mixed effects models (with a random effect for each matched pair). However, I was wondering if this makes sense to do, since I've already matched on those demographic variables. What's interesting is that when I add in those variables, they are statistically significant. I understand that age can still be a confounding factor, since it's not an exact match, but sex and race were exact matches, yet I saw multiple instances where they were significant.

Is this because while the within-pair differences may not be significant, the between-pair differences are? Any insight on this would be greatly appreciated!

user122514
  • 191
  • 5

1 Answers1

1

I assume schizophrenia vs. general population is your "exposure" variable of interest and your outcome is something related to e.g., survival or cognitive decline. You matched on age, sex, and race so that differences in the outcome between schizophrenia patients and the general population would not be attributable to those variables. Typically, one is only interested in the effect of the exposure variable, and the other covariates are there to remove confounding; their coefficients in the outcome model generally should not be interpreted because they don't represent causal relationships (i.e., the only causal relationship that can be assessed is the relationship between the exposure and the outcome). Incorrectly interpreting the coefficients on non-exposure covariates in the outcome model is known as the Table 2 fallacy (Westreich, 2013).

I encourage you not to interpret those coefficients, but I'll explain the statistical matter of why they might be significant even after matching. Matching makes the distribution of the covariate unrelated to the exposure. This means if you were to regress the exposure on the covariates, you would likely get non-significant and small coefficients. Indeed, these coefficients can be used as a measure of balance; if they are large, then the exposure groups differ with respect to that variable and your groups are not balanced. However, matching doesn't change the relationship between the covariates and the outcome, which is what the coefficients in the outcome model represent.

One way to think about this is to imagine a twin study. If one twin has schizophrenia and the other doesn't, the fact that they are twins, which makes their ages identical, doesn't change the relationship between age and cognitive decline. Older pairs of twins will still have greater cognitive decline. Likewise, in your study, the fact that pairs of units have the same age doesn't change the relationship between age and cognitive decline.

Again, I urge you to be extremely cautious in interpreting the coefficients on those covariates in the outcome model. In the MatchIt documentation for estimating effects after matching, the code was written specifically to blind researchers to those coefficients because they are generally not informative and possibly misleading.

Noah
  • 20,638
  • 2
  • 20
  • 58
  • Hi @Noah. Thank you for providing such a detailed response. You are right that schizophrenia vs. general population is the exposure and the main variable of interest. While you suggest that I be cautious in interpreting the coefficients for the demographic variables, is it even statistically sound to adjust for them since I've already matched on them? I assume yes, since like you said, matching makes the distribution of the covariates unrelated to the exposure. I just want to make sure I'm understanding correctly. Thank you! – user122514 Jan 20 '21 at 04:41
  • Also, I understand that we should not establish a causal relationship between the demographic variables and the outcome. In my study, the outcome variable is whether the individual had been arrested in the past 7 years (binary outcome). Would it be safe to state something like this: the odds of being arrested is xxx times more likely amongst males compared to females, controlling for other variables in our model? – user122514 Jan 20 '21 at 04:45
  • It is statistically sound and in fact a very good idea to adjust for the same covariates in the outcome model that one used to match units. This generally increases the power to detect an effect (and with a logistic regression outcome model will make the effect larger and change its interpretation to a conditional rather than marginal effect). – Noah Jan 20 '21 at 04:50
  • That would be a fine interpretation of the coefficient for gender, but it's the "controlling for other variables" part that gets tricky. If men are both more likely to get schizophrenia and be arrested, the coefficient on gender in a model that controls for schizophrenia represents a direct effect of gender not through schizophrenia, whereas the coefficient on schizophrenia represents a total effect. You just really need to be careful with the interpretations of the coefficients; there is no single "effect" of a variable. It depends on the causal structure of the variables in the model. – Noah Jan 20 '21 at 04:55
  • Thanks so much for the reassurance. Based on what I've seen on the web, when people have a matched cohort, they often use conditional logistic regression. However, for my analyses, I plan to model my outcome using a binomial mixed-effects model, with a random effect for each matched pair to account for the within-pair correlation. Does this seem like a feasible model? Sometimes I get stuck on the idea that once I have a matched cohort and have a binary outcome, I HAVE to use conditional logistic models, but GEEs and mixed models can account for the within-pair correlation, but I'm not sure. – user122514 Jan 20 '21 at 05:00
  • Sorry if this is a silly question, but can you clarify on what you mean when you say, "the coefficient on gender in a model that controls for schizophrenia represents a direct effect of gender not through schizophrenia, whereas the coefficient on schizophrenia represents a total effect"? I'm not sure if I understood you correctly. Again, thank you so much! – user122514 Jan 20 '21 at 05:05
  • Conditional logistic regression estimates a conditional effect, mixed models estimate a conditional effect, and GEEs estimate marginal effects. Conditional and marginal effects have different interpretations and are different parameters. This is kind of a big statistical concept so I can't explain it deeply in a comment. A covariate-adjusted logistic regression with a cluster-robust SE is the best model generally. Regarding the direct/total effect, please read the Westreich article I cited in my answer. Also read the `MatchIt` vignette I linked (which I wrote). – Noah Jan 20 '21 at 08:01