7

I want to analyze some propensity score matched data. In the literature McNemar test is usually used, since the data is "paired". However matching is not pairing in the common sense.

Would it be more correct to use Fisher exact test? What opinions are there on using paired tests for matched data?

Viktor
  • 853
  • 1
  • 8
  • 22
  • A related question with technical details: https://stats.stackexchange.com/questions/147559/fisher-exact-test-on-paired-data – Viktor Jun 13 '18 at 03:30

1 Answers1

8

This is definitely an ongoing debate in the literature, but at this point the evidence points to using paired analysis to compute standard errors and p-values. Although the goal of matching is to arrive at two samples that mimic a randomized control trial, not a paired-randomized control trial, matching does still induce a covariance between the outcomes within each matched set, which needs to be taken account of in inference. P. C. Austin has written a great deal about this (e.g., Austin & Small, 2014). Zubizarreta, Paredes, & Rosenbaum (2014) showed that after matching (i.e., discarding unmatched units), pairing (i.e., creating matched pairs) can reduce the sensitivity of the eventual estimate to unmeasured confounding and reduce standard errors, which could only be realized if paired analyses were used on the sample.

Noah
  • 20,638
  • 2
  • 20
  • 58
  • "...the goal of matching is to arrive at two samples that mimic a randomized control trial" That is not true. The goal of matching is to remove confounding by eliminating any association between the matching factors and the treatment *option*. RCTs do this with randomization. The distribution of confounders in a randomized sample is representative of trial participants. With matching distribution of those confounding factors, disproportionately resembles that of people who opt for the rarer treatment. – AdamO Jun 12 '18 at 18:43
  • 1
    That's true, but it's possible to match for the ATE, which is what is estimated in an RCT. I was just corroborating OP's intuition that PS matching is not obviously the same thing as pairing, in the same way an RCT is not the same as a paired RCT. – Noah Jun 12 '18 at 18:50
  • Alright, seems like we agree. In short: matching/randomization is the same insofar as it's meant to eliminate confounding, but they lead to different distributions of factors in the resulting sample. The designs are valid to estimate ATEs when there's no undetected interactions with treatment, otherwise the two approaches can produce different/conflicting results. – AdamO Jun 12 '18 at 19:09
  • Since matched data is not paired data and is not data from independent groups, may be some new tests may be developed specifically for matched data? – Viktor Jun 13 '18 at 00:21
  • 1
    Many have attempted this work. Abadie and Imbens' work comes to mind, but I don't know much about it. For now, the paired analysis is most appropriate. – Noah Jun 13 '18 at 04:16
  • Are you aware of any paper where McNemar's and Fisher's exact tests are compared for matched samples? Or are there only papers with general considerations regarding pairing after matching? – Viktor Jan 19 '19 at 12:26