2

What are the various methods used for binary classification other than logistic regression?

What are the advantages of logistic reg. model in developing Propensity score w.r.t. other methods?

Actually I have been asked why? I backed it by saying its binary classification, so logit is perfect. Then I was asked why specifically logistic regression when there are various other binary classification methods?

Andy
  • 18,070
  • 20
  • 77
  • 100
user2035158
  • 31
  • 2
  • 2
    While you headed and tagged this question as related to propensity scores, your two questions look at odd to me: Binary classifiers are one thing, the use of [tag:propensity-scores] with binary outcome is another topic. Are you asking whether logistic regression is better than, say, neural networks in deriving PSs? – chl Oct 20 '13 at 22:24
  • Exactly, and I asked two questions....sorry for bad formatting. – user2035158 Oct 21 '13 at 14:12
  • @chl it was my understanding that propensity scores can be used for any type of outcome, binary, continuous, or rates, events, etc. I had assumed that the Andy was asking about binary classifiers for the probability of receiving "treatment", i.e. how best to estimate propensity scores. – AdamO Sep 05 '14 at 15:01

2 Answers2

4

Logistic regression is not necessarily the best approach to predict propensity scores, but it seems to be the most commonly used. There are a number of alternatives including probit regression, classification and regression trees (Westreich, Lessler and Funk 2011, or generalized boosted models (Ridgeway, McCaffrey and Morral 2004). Each method has advantages and disadvantages. For example, Ridgeway and McCaffrey (2007) suggest that logistic regression results in more extreme weights in inverse propensity score weighting compared to some other alternatives.

Ahu
  • 148
  • 7
  • 1
    You're right that it is not universally the best approach. It is certainly defensible with a bare minimum of assumptions, such as reasonable sample size and number of propensity factors. All the caveats of developing a binary classifier apply. Hence I would look at boosting (and nearest neighbor, and LARS, and LASSO) as very specialized applications to be considered when $p > n$. – AdamO Sep 05 '14 at 15:29
  • It is, I suppose, worth mentioning that some methods that can predict propensities of exactly 0 or 1, ie deterministic methods, are going to lose the theoretical guarantees that p-scores have. I don't suppose anyone was really thinking of them though. – conjugateprior Sep 05 '14 at 21:05
2

Development of propensity scores is simply a matter of predicting likelihood of receiving a pseudo-"treatment" as you would handle imbalanced randomization in a clinical trial. The matter of developing such scores, then, becomes a prediction problem. This question becomes the same, then, as using logistic regression versus other techniques for development of a binary prediction model.

Logistic regression is a maximum likelihood routine for a regular exponential family, that means that the MLEs have nice regularity properties, except in the case of singularity. It takes advantage of the fact that the variance of a binary event is related exactly to its mean, and uses that to get better estimation of the curve (rather than a nonlinear least squares regression using the logit curve estimated using least squares for 0, 1 observations). The logit curve is a smooth, well behaved, and appropriately bounded curve for predicting risk. The logit curve gives approximately very similar inference and predictions to other "S" shaped regression routines like probit regression or some arbitrary made up link function. Logistic regression parameters are easily estimated using default software.

AdamO
  • 52,330
  • 5
  • 104
  • 209