3

Let's say I run a linear regression model with a binary dependent variable. If I ran logistic regression on the same data would the results be comparable or exactly similar? By results I mean both the beta values and the value of dependent variable. If not why? Also what can I say about linear regression being a subset of Logistic regression or vice versa?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Mukul
  • 737
  • 1
  • 6
  • 8
  • The models for linear regression and binary logistic regression are both special cases of the generalized linear model (GLM): Linear regression with an identity link and Gaussian conditional distribution (with equal variance), logistic regression with a logit link and binomial conditional distribution. – caracal Nov 28 '12 at 13:05
  • I discussed the nature of logistic regression in a slightly different context here: [difference between logit and probit models](https://stats.stackexchange.com/a/30909/7290). That answer also briefly mentions the connection b/t linear regression and the generalized linear model. It may be helpful to read it. – gung - Reinstate Monica Nov 28 '12 at 15:30

1 Answers1

12

Neither method is a subset of the other. Suppose you have response variable $Y$ using covariates $X$. In linear regression, you assume that $E[Y|X]$ is linear in the parameters $\beta$, whereas in logistic regression you assume the log-odds,

$$ \log\Bigg(\frac{P(Y=1|X)}{P(Y=0|X)}\Bigg) $$

is linear in the parameters $\beta$. For each method you assume two different quantities are linear in $\beta$. The estimates of $\beta$ will likely be different and have different interpretations as well.

It may or may not make sense to use logistic regression for your data, but linear regression is almost certainly not a good way to analyze your (dichotomous) data.

caburke
  • 1,342
  • 10
  • 12
  • Agreed...Is there any other way to estimate the parameter other than MLE technique in logistic regression ? – Mukul Dec 03 '12 at 11:09
  • You can use gradient descent with a specific objective function. Andrew Ng has several online lectures available on this topic, how to set up the procedure, its assumptions, etc. – Sycorax Jun 04 '13 at 17:28