2

I have recently started looking into the synthetic control method, and found it quite appealing. But it seems as some issues do not receive a formal definition (or at least I haven't found one):

  • Similarity of the donors: [1] claims that "it is important to restrict the donor pool to units with characteristics that are similar to the affected unit". But how can this similarity be quantified?  What degree of similarity is required relative to the validity of the outcome?
  • Time horizon: How long into the future can we be confident of the obtained synthetic control?
  • Time varying confounders and other effect modifiers: What in the model protects us from the presence of such factors? It is assumed that a good pre-treatment fit acts as a proxy for a good accounting of all observed and unobserved factors, but what about factors that show up only in the post-intervention period? How can we be robust against these?

I'm sure these issues are acknowledged and discussed in some works, but I'm having trouble finding the answers. 

[1] Abadie, A. (2020) Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects. Journal of Economic Literature. Forthcoming.

Alexis
  • 26,219
  • 5
  • 78
  • 131
Tom Beer
  • 21
  • 3
  • One appealing aspect of synthetic control method is that it does require parallel trends assumption as in Difference-in-difference. And the method is also suitable when you have small numbers of treated units. See the article here: https://jech.bmj.com/content/72/8/673 – user3571389 Jul 06 '20 at 15:57
  • 1
    If I understand correctly, the method does **not** require the parallel trends assumption. It is stated also in the attached BMJ article. – Tom Beer Jul 07 '20 at 18:07
  • Yes, you are right. It is an error – user3571389 Jul 07 '20 at 22:32
  • Regarding your third question, I would suggest a read of [this](https://arxiv.org/abs/1812.09970) paper, which shows that with some added structure the synthetic control estimator can be seen as an approximation of an interactive fixed effects model. Conceptually you can think of it as modeling the pre-treatment $\lambda_i c_t$, where $\lambda_i$ and $c_t$ are the individual and time factors. It is also assumed that these factors follow the same structure post-treatment. – FightMilk Jul 09 '20 at 12:36
  • Possible duplicate of https://stats.stackexchange.com/questions/387965/synthetic-control-and-unobserved-confounders?rq=1 – Julian Schuessler Aug 26 '20 at 09:55

1 Answers1

0

The innovation of the synthetic control method (Abadie et al. 2010) is that it enables estimating the factor model that allows unobserved common factors (lambda) to vary across unit (mu) with fairly reasonable assumptions (equation 1 in the paper).

Choosing a synthetic control in this manner is, of course, not feasible because μ1,...,μJ+1 are not observed. However, under fairly standard conditions (see Appendix B), the factor model in Equation (1) implies that a synthetic control can fit Z1 and a long set of preintervention outcomes, Y11,...,Y1T0, only as long as it fits Z1 and μ1, so Equation (4) holds approximately. (Abadie et al. 2010)

So the existence of the set of weights that meet the balancing conditions (i.e., similar pre-trends and explanatory variables, equation 2 in the paper) means that these factor loadings (mu) have been estimated, or the unobserved time-varying confounders have been controlled. You can measure the fit, or the existence of such weights, by observing the pre-event RMSPE.

  1. Similarity of donor: it is to avoid the interpolation bias, which could exist because of the simple linear model specifications used for the estimation. Though, I couldn't find any papers pointing to how it might affect the validity of SC estimators.

  2. Time horizon: I think it can't be theoretically driven. You could simulate with your own data to find at which length the estimated effects start to drift off.

  3. Time-varying confounder: If there is an event that is believed to change not only the post-event outcome but also the factor loadings estimated from the pre-event characteristics (COVID?), I guess you can't use the synthetic control method or any estimation method for that matter. We would need some leverages for predicting counterfactuals, but the nature of that event would disqualify most external sources.

Abadie, A., Diamond, A., and Hainmueller, J. 2010. “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program,” Journal of the American Statistical Association (105:490), pp. 493–505.

Neal
  • 3
  • 2