5

Using a large, cross-sectional survey of victims of violence, I am interested in testing the effect of alcohol intoxication (let's call it the 'treatment') on victims' subjective rating of the seriousness of the assault (outcome).

I have run regression analyses to explore factors associated with seriousness ratings and, as you might expect, the amount of injury done is the strongest predictor of rated seriousness (more on this later).

Victims who were drunk at the time of their assault had somewhat different characteristics from the other victims (e.g. more men, less educated...) and that that assaults on drunk victims resulted in more serious injury than assaults on sober victims, so I thought it might be a good idea to match victims with just the 'drunk at time of assault' as a distinguishing factor ('treatment') so that I can compare (drunk) apples with (sober) apples. I have run propensity score matching using 14 theoretically-relevant variables to identify a matched sample of sober and drunk victims.

My problem is that, because important confounding factors like injury happened after the 'treatment allocation' (drunk at time of assault - intervention group; sober at time of assault - control group), I couldn't logically include those in my matching model. So, when I compare the groups, the biasing effect of injury on the outcome still remains.

As far as I can see, I'm faced with three options but I'm probably missing something:

  1. Include the statistically relevant, but logically problematic, 'post-treatment' factors in the matching (they are logically problematic because they happened after the 'treatment' was delivered but are likely to affect the outcome).

  2. Control for the propensity score and the post-treatment factors in a regression of the outcome on victim intoxication using just the matched sample.

  3. Stop messing around with treatment evaluation on a cross-sectional data set and just stick with regression.

Thanks in advance

user38318
  • 53
  • 3
  • Thanks for contributing this well-done and entertaining post. I hope you get great answers. – rolando2 Feb 01 '14 at 01:38
  • Have you considered any type of path analysis or structural equation modeling? – rolando2 Feb 01 '14 at 01:42
  • Can you clarify what you mean by problematic post-treatment confounders? I thought injury and the like were outcome variables. Also remember that confounders are those variables which affect selection into the treatment. – Andy Feb 01 '14 at 10:18
  • Thanks for the comment @Andy. Hopefully this edit has made it clearer - please let me know if it hasn't. For clarity, the outcome/dv is 'subjective rating of crime seriousness', the treatment is 'being drunk at the time of being assaulted' (or not). Predictors/matching covariates included injury, gender, education and 11 other variables. – user38318 Feb 01 '14 at 19:34

1 Answers1

2
  1. From a theoretical point of view you already excluded this option and I think that is the correct way. Especially propensity score matching is sensitive to the matching covariates (see for example Smith and Todd, 2005).
  2. That's the most promising approach so I'll expand on this below.
  3. Depends what you want. It's not impossible to estimate treatment effects from cross sectional data but it's certainly not easy. Regression alone in this case will not get you far for this purpose though.

Now related to your second point, I would try the following:
Match control and treatment individuals in pairs based on the observed pre-treatment characteristics. Do the usual post-matching checks like assessment of the common support or covariate balancing (after the matching treated and control units should have very similar summary statistics on their matching variables). If the matches are far apart from each other you might consider caliper matching instead but given that you have a large sample it should be fine.

Once you have your matched pairs, take for each pair the difference between the treated and control unit for the remaining post-treatment variables which were not used in the matching. Regress your outcome on these differenced variables and the treatment variable. This way you can combine matching and regression. The alternative to differencing is to declare your data a panel where pairs are the individual component and the treated and control units are "time" (nobody says that time actually must be time, for example see Ichino and Maggi, 2000) with treated being period one and control being period two. Then you can again use the difference estimator or the within effects estimator.

In a panel setting this estimation strategy amounts to doing difference-in-differences for the matched cases which eliminates the unobserved fixed effects. This of course doesn't work in your case but just to give the intuition behind the procedure. So you can try the best you can given the data at hand but it will be difficult to argue away relevant unobserved effects if people do not believe your selection-on-observables story. Though that's a general problem of matching methods.
I hope this input was useful. Good luck for your work!

Andy
  • 18,070
  • 20
  • 77
  • 100