How do difference-in-difference designs account for temporal autocorrelation

Question

Although there are doubtless many techniques for studying the impact of a discrete intervention over time, I am interested in two which have achieved widespread adoption in the social sciences:

• Interrupted time series
• Difference-in-difference

The former applies all the principles of time series analysis, using ARIMA models to account for non-stationarity, autocorrelation, etc. The latter uses linear regression or variants, but allows for a control group.

If anything, difference-in-difference is more widely adopted, particularly in the economic literature (after Card, Angrist, etc.) but also in health-related fields and possibly education (where it seems to be called "comparative interrupted time series"). Yet in the presence of substantial autocorrelation (likely, in the types of datasets being used), it would seem to over-state the precision with which results are estimated.

My questions are thus:

1. How are difference-in-difference analyses valid if autocorrelation exists?
2. Is there a time-series technique that allows a control group to be utilized?

Answer 1

Your point about the standard errors in DiD is an important issue which has been mostly ignored or forgotten until a paper by Bertrand et al. (2004) "How Much Should We Trust Differences-In-Differences Estimates?" in the Quarterly Journal of Economics. In there they discuss several methods to overcome the autocorrelation problem which will answer your first question.

Specifically, they examine:

1. Parametric $\left(AR(1)\right)$ adaptations - These do not do particularly well, which they attribute to the under-estimation of the auto-correlation coefficient as a result of the short time series typical in DiD work, as well as an incorrectly-specified process.
2. Block bootstrap - This also performs poorly, which they attribute to the small number of blocks typical of DiD work [ Cameron, Gelbach, Miller would seem to be relevant here ].
3. What they call the "Empirical Variance-Covariance Matrix" - This avoids the SE inflation but also has low power and relies on the DGP being the same across all states.
4. An arbitrary V-C matrix of their own design - "This method, therefore, seems to work well, when the number of treated units is large enough."
5. Pre/Post (discarding all the extra year information) - They suggest doing this on the model residuals so as to allow controlling for some trend and covariate information, and suggest that it works well when the number of treated units is large.
6. Randomization inference - This is their main proposal, in section 4.6. "It removes the over-rejection problem and does so independently of sample size. Moreover, it appears to have power comparable to that of the other tests."

As concerns your second question I am a bit puzzled because once you have a control group you are essentially dealing with panel data and not a pure time-series anymore. In this sense DiD compares the intervention effect between the time-series data of a control and a treatment unit but maybe I misunderstood the question.

As a last suggestion: if you are interested in newer methods than DiD (but which are similar) have a look at the synthetic control method by Abadie and his co-authors. Especially if you have aggregate data this might be a particularly useful method for your work and it is something which is increasingly used in very recent economics papers.

Answer 2

Another paper that I hadn't thought about in this context until @Andy's comment is Cameron, Gelbach, Miller's "Robust Inference with Multi-Way Clustering".

Upon re-reading, it looks like they explicitly replicate the Bertrand (2004) strategy and handle it via their multi-way clustering (which is a generalization of their one-way clustering strategy from the much-more-famous previous CGM paper).