38

I want to know why logistic regression is called a linear model. It uses a sigmoid function, which is not linear. So why is logistic regression a linear model?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
user34790
  • 6,049
  • 6
  • 42
  • 64
  • 6
    The logit of $\pi$ (the log of the odds) is linear in the parameters, but people don't refer to *logistic* regression as linear as far as I know. Can you cite who has said this? – gung - Reinstate Monica Mar 03 '14 at 18:05
  • 2
    @gung-ReinstateMonica For example, in the Deep Learning book at page 169 (http://www.deeplearningbook.org/contents/mlp.html). In the book they note "Linear models, such as logistic regression and linear regression, are appealing....." I think they meant Generalized Linear Model for logistic regression. – YOUNG Nov 27 '19 at 06:45

2 Answers2

45

The logistic regression model is of the form $$ \mathrm{logit}(p_i) = \mathrm{ln}\left(\frac{p_i}{1-p_i}\right) = \beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i} + \cdots + \beta_p x_{p,i}. $$ It is called a generalized linear model not because the estimated probability of the response event is linear, but because the logit of the estimated probability response is a linear function of the predictors parameters.

More generally, the Generalized Linear Model is of the form $$ \mathrm{g}(\mu_i) = \beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i} + \cdots + \beta_p x_{p,i}, $$ where $\mu$ is the expected value of the response given the covariates.

Edit: Thank you whuber for the correction.

P Schnell
  • 2,123
  • 12
  • 11
  • 8
    If you were to write "generalized linear" instead of "linear" and *parameters* instead of *predictors,* this would be correct. (Many logistic regression models are *not* linear in the predictors. For instance, no logistic regression with an interaction term will be linear in the predictors.) – whuber Mar 03 '14 at 18:42
  • You are correct, thank you. I've updated my answer to reflect this. – P Schnell Mar 03 '14 at 18:55
  • what is Pi there? – aerin Sep 14 '17 at 06:18
  • @whuber Why would an interaction term in a logistic regression wreck the linearity in the parameters when that is not the case for a linear regression? (I assume you mean predicting log-odds. If you mean predicting the probability, then I am confused how logistic regression could be linear in the parameters at all.) – Dave Oct 09 '20 at 16:20
  • 2
    @Dave In my parenthetical comment I wasn't writing about linearity of the *parameters:* it was explicitly about the *predictors.* Nowhere in the comment did I claim linearity of logistic regression models in either parameters or predictors. – whuber Oct 09 '20 at 16:24
  • I want to understand why the expected value of target variable, represented by b0 + b1*x1 ... is equal to the logit(log of odds). I couldn't find any resources. Can you point me to paper or explain it. – lego king Jul 03 '21 at 14:45
9

Logistic regression uses the general linear equation $Y=b_0+∑(b_i X_i)+\epsilon$. In linear regression $Y$ is a continuous dependent variable, but in logistic regression it is regressing for the probability of a categorical outcome (for example 0 and 1).

The probability of $Y=1$ is: $$ P(Y=1) = {1 \over 1+e^{-(b_0+\sum{(b_iX_i)})}} $$

lennon310
  • 2,582
  • 2
  • 21
  • 30