Counterfactuals in Econometric Modeling (Abortion-Crime Hypothesis Revisited)

Question

Donohue and Levitt (2019) recently published a working paper revisiting the abortion-crime link. My question is specific to equation (2) in their paper (see below):

$$ ln(CRIME_{st}) = \beta_{1}ABORT_{st} + X_{st}\Theta + \gamma_{s} + \lambda_{t} + \epsilon_{st} $$

The left-hand side is the logged per capita crime rate in state $s$ at time $t$. The variable $ABORT_{st}$ is a measure of the effective abortion rate in state $s$ and year $t$ for a given crime category. In their previous paper (see Donohue and Levitt [2001]), the panel is comprised of annual state-level observations from 1985-1997. In their latest paper, they rerun this model using data from 1985 to 2014.

Questions:

• In reference to the foregoing equation, what is the counterfactual? For instance, this specification is similar in style to the more 'general' difference-in-differences framework, with state and year fixed effects. However, the main independent variable is a continuous measure (I would assume it could be viewed as a measure of intensity or 'bite'). Because legalization happened everywhere, we don't have any states where the law did not take effect, or where, in theory, abortions were not occurring. I can't understand how this model can disentangle any other macro-social events occurring in tandem with legalization. Don't we need non-adopter states, or can we just run this model and hope to partial out the influence of any confounders?

• The equation was estimated using a two-step procedure. Is it appropriate to weight a panel regression by state population while also including state fixed effects? Would including a measure of state population (covariate) on the right-hand size offer any advantages over weighting (or pose problems)?

The 2019 working paper is included below:

https://bfi.uchicago.edu/wp-content/uploads/BFI_WP_201975.pdf

Their earlier paper is also available to the public:

http://pricetheory.uchicago.edu/levitt/Papers/DonohueLevittTheImpactOfLegalized2001.pdf

Answer 1

Regarding the first question:

The counterfactual in this case are other years (for the same state) in which the abortion level and the crime rate differ. The authors aim to mitigate the effect of other macro-social events by adjusting for state-level and time differences in abortion rates and crime rates. As you already indicated in your question, this is not guaranteed. If there are, for example, other dynamic (time-varying) effects, the analyses can be confounded and lead to false conclusions. This approach not a causal analyses (as a randomized experiment). For this reason, there has been some critique of that study. A possible confounder be, for example, the phase-out of lead from gaseoline as suggested by Reyes (2007) in the link above. To be fair, one should point out that this has been admitted by the authors in the new paper:

There are a number of reasons why Donohue and Levitt (2001) provoked such a strong academic response. [...] The identification of the estimates was derived neither from a randomized experiment nor even from a credibly exogenous natural experiment (with the possible exception of the 1973 Supreme Court decision in Roe v. Wade). Instead, Donohue and Levitt (2001) presented evidence from a collage of different sources of variation, each of which had its weaknesses.

Regarding the second question:

Yes, it is of course valid to weight a fixed-effects approach by population size. This will however give a different interpretation. Without weighting, you treat each state equally while weighting by population size will give some states such as California or Texas much more weight than others, such as Wyoming or Vermont. This is true regardless of whether you use fixed-effects or not. By including fixed-effects you just weight the (within-state) differences of abortion and crime rates between the years differently. Without fixed-effects, you give different weights to the absolute levels of these variables. If this is desirable depends very much on what you aimt at. If you want to get representative results for the entire population, you should use population weights (or sparsely populated states will be overrepresented). If you are interested in legislative or macroeconomic questions, you might not because you do not want California to have a much higher weight than Wyoming. Typically, if weights differ a lot (as it is the case for US states), you would show via a robustness check that the results are not driven by the inclusion of weights. Using population as a covariate, just adjust for different levels of crime rates (as well as the other covariates) between states of different population size. This will usually not make a big difference in a fixed-effects setting (because the fixed-effects will already adjust for average/long-term population differences), unless there is a lot of change in population size between states over time.

Why does it make a difference? Think of the Frisch–Waugh–Lovell theorem. You can partition the fixed-effects regression into different parts. A first stage could be just regressing crime rates on state and year fixed-effects. If there are, for example, strong differences in let's say California in a certain year. That year effect will be much more pronounced in the regression using population weights because of California's weight.