4

Gary King explained why he thought propensity score matching should not be used for matching. See Paper and the Video Lecture.

After all these years, have the academics/practitioners reached consensus on the following question:

should propensity score matching be avoided in favor of other matching methods?

Is there any counter-arguments given by other researchers to Gary King's explanation?

Pik-Mai Hui
  • 389
  • 7
  • Thank you for suggesting edits. I would like to ask future editors to be careful when they apply negation on the texts, because it shifts the focus of the question and apparently has misled readers. – Pik-Mai Hui Dec 10 '21 at 08:58
  • @Noah It does. And I most appreciate your fantastic answer there. – Pik-Mai Hui Dec 10 '21 at 17:35
  • 1
    Glad it helps. Several of my [most highly voted](https://stats.stackexchange.com/search?tab=votes&q=user%3a116195%20%5bmatching%5d) answers are on this very topic, so feel free to take a look. – Noah Dec 10 '21 at 17:48

1 Answers1

1

If you perform propensity matching and there is:

  • a large imbalance in sample sizes between groups, you will be asked: What objective criteria did you use during propensity matching to overcome sample size differences? (perhaps resampling, Monte Carlo, etc.) Then, provide the justification for employing resampling methods.
  • a notable difference in the severity of disease of patients in the various groups, it will be difficult to find covariate patterns from baseline medical histories (grade, stage, severity, duration of disease) which would allow matching to be made. This happens because there was no randomization of subjects to different treatments (exposure) -- as the data were likely from an administrative data set spawned from patients treated in a clinic/hospital who had widely varying severity of disease. Thus, there is wide variation in the treatment. Again, patients weren't randomized to different treatments --> low-risk patients received treatment for low-severity disease, high-risk patients received treatment for high-severity disease.

The above shortcomings surrounding severity are also in disagreement with the rule of thumb: "A clinical trial for a low-risk treatment will fail if high-risk patients are enrolled."

Clinicians often look to data analysts and statisticians to "salvage" the bias in their study because no randomization was performed. Clinical trials are expensive, and the majority of clinicians only treat patients, and are not grant-funded researchers who carry out appropriately designed trials. (only a small fraction of treated patients in an academic medical center are clinical trial members). The result is that a large "Excel" is constructed by residents/fellows (other attending clinicians) and the Excel-based data set "is the study." So what the statistician hears is: "that's a valuable data set that many people don't have," or, "it includes more than a thousand patients," or "people put a lot of effort into collection of that data." So the value that is ascribed to the data by the clinicians may be overstated.

  • Can you elaborate on how this ties to the preference for propensity score matching over other matching methods? I fail to see why propensity score matching is the better option for badly constructed clinical trials. – Pik-Mai Hui Dec 10 '21 at 08:23
  • I just noticed someone edited the question and made the bolded part opposite of the original question, probably by mistake. I've edited the question again to hopefully clarify it. It will be great if you can edit your answer accordingly as well. – Pik-Mai Hui Dec 10 '21 at 08:34
  • Other matching methods won't be able to overcome the shortcoming listed in the answer regarding propensity matching. –  Dec 10 '21 at 15:53
  • I am sorry that I still fail to see why other matching methods won't be able to overcome the shortcomings listed. For example, why would Coarsened Exact Matching not be able to solve these problems? – Pik-Mai Hui Dec 10 '21 at 17:22