3

This is a really simple, newbie question. I am really confused about the notion of matching and when it can be used instead of a multiple regression?

Assume I have listed all the confounding variables (X), and my outcome (Y) and treatment assignment (A) are binary.

Can I reach causal inference only by running the following logistic regression: Y ~ A + X and focus on "A" coefficients, SE, p-value? Doesn't it provide a better power than "matching" that'd result in losing a bunch of data?

Any clarification would be really helpful?

EDIT: assume A is not randomly assigned (analysis is observational)

aghd
  • 249
  • 2
  • 8

1 Answers1

5

Your question rightly acknowledges that throwing away cases can lose useful information and power. It doesn't, however, acknowledge the danger in using regression as the alternative: what if your regression model is incorrect?

Are you sure that the log-odds of outcome are linearly related to treatment and to the covariate values as they are entered into your logistic regression model? Might some continuous predictors like age need to modeled with logs/polynomials/splines instead of just with linear terms? Might the effects of treatment depend on some of those covariate values? Even if you account for that last possibility with treatment-covariate interaction terms, how do you know that you accounted for it properly with the linear interaction terms you included?

A perfectly matched set of treatment and control cases would get around those potential problems with regression.* That leads to the next practical problem: exact matching is seldom possible, so you have to use some approximation. There are several approaches to inexact matching; see this page for some discussion. Matching based on propensity scores, the probability of being in a treatment group give the covariate values for a case, is one frequently used method.

You can also combine matching with regression. You could include covariates in a regression model of matched cases; some argue that you should do this in any event, as noted on this page. You can go even further to potentially include all cases: weighting cases according to their treatment/control propensity scores (inversely) in your regression model. This page nicely outlines matching versus weighting; this page goes into more details.

Both regression and matching have strengths and weaknesses. You need not think of them necessarily as alternatives; combining them intelligently can sometimes work better than either alone.

*Even a data set perfectly matched on the known covariates can't rule out the problem posed by unknown covariates that might affect outcome directly or change the effect of treatment on outcome. That's why randomized trials, which in principle average out those unknown effects, can be so important.

EdM
  • 57,766
  • 7
  • 66
  • 187
  • this was very informative. need to read it a few times. qq: if we use gradient boosted trees, doe we relax some strong assumptions of linear regression? – aghd Oct 17 '19 at 21:30
  • @aghd there's still the question of how well the boosted-tree analysis is modeling the outcome. And you might not get a single, simple coefficient expressing the effect of treatment. Note that boosted trees are used by the R [twang package] (https://CRAN.R-project.org/package=twang) for calculating propensity scores. So again, the issue is whether the outcome model or the propensity/ matching model is handling the situation better; often, combining the 2 in some way can be superior to either alone. – EdM Oct 17 '19 at 21:56
  • sure. it makes sense. thanks a lot. – aghd Oct 17 '19 at 22:17
  • @aghd You mentioned using gradient boosting. Statisticians have looked into using boosting & other machine learning models like neural networks and Lasso regressions for causal inference. But you run into a few problems: many ML models don't output coefficient estimates or standard errors for the "A" coefficient, and if they do (like with the Lasso) they can be heavily biased due to shrinkage. There are ways around this (this is a hot topic of research lately) - see this paper for an introduction: https://arxiv.org/abs/1903.00402 – RobertF Jun 11 '20 at 13:37