Interpretation of coefficient in logistic regression

Question

Let's say I have the following model:

$$\ln\Big(\frac{\mathbb{P}(Y_i = 1 | X_i)}{\mathbb{P}(Y_i = 0 | X_i)}\Big) = \beta_0 + \beta_1 X_{1,i} + \beta_2 X_{2,i}$$

Let $\rho = \frac{\mathbb{P}(Y_i = 1 | X_i)}{\mathbb{P}(Y_i = 0 | X_i)}$.

Then: $$\frac{\partial \ln\rho}{\partial X_1} = \frac{\partial \ln\rho}{\partial \rho} \frac{\partial \rho}{\partial X_1}= \frac{\partial \rho}{\rho} \frac{1}{\partial X_1} = \beta_1$$ So if $\partial X_1 = 1$ then $\frac{\partial \rho}{\rho} = \beta_1$.

Similarly, let $\ln\rho^0$ increase to $\ln\rho^1$ after the unit increase in $X_1$. If this is so, then: $$\ln\rho^1 - \ln\rho^0 = \beta_1 \\ \implies \frac{\rho^1}{\rho^0} = e^{\beta_1}$$

---

Question

Shouldn't it always hold that $e^{\beta_1} - 1 = \beta_1$ when $\partial X_1 = 1$?

---

My estimated model returned $\hat{\beta_1} = 0.0326$ and:

• $0.0326 \neq \big[ e^{\hat{\beta_1}} - 1 = 0.0332 \big]$

How come these two effects are not the same?

Answer 1

Let $\ln(\rho)=\beta_0+\beta_1x_1+\beta_2x_2$. This is called the log-odds. The partial derivative of $\ln(\rho)$ with respect to $x_1$ is then

$$\frac{\partial}{\partial x_1}\ln(\rho)=\frac{1}{\rho}\cdot\frac{\partial}{\partial x_1}\rho=\frac{\beta_1}{\beta_0+\beta_1x_1+\beta_2x_2}.$$ As correctly noted in comments, there is no way of setting the value of the differential of $x_1$.

Should you wish to assess the effect of a unit increase in $x_1$, the way is using log of the odds ratio: Denote $x,x'$ two vectors where $x'_1=x_1+1$ and the rest of the components unchanged. The log-odds ratio is then:

$$\ln\left(\frac{\rho(x')}{\rho(x)}\right)=\beta_0+\beta_1x'_1+\beta_2x'_2-\beta_0-\beta_1x_1-\beta_2x_2=\beta_1(x_1'-x_1)=\beta_1.$$

Another explanation, a bit more abstract: in logistic regression, much like in linear regression, we compute the linear predictor $\theta_i=x_i^T\beta$. In order to obtain a probability, we put it into the sigmoid function:

$$P(y_i=1|x_i)=\pi_i=sigmoid(\theta_i)=\frac{1}{1+e^{-\theta_i}}=\frac{e^{\theta_i}}{1+e^{\theta_i}}$$

The inverse of the sigmoid function is called the logit function:

$$logit(\pi_i)=\log\left(\frac{\pi_i}{1-\pi_i}\right)=\log\left(\frac{\frac{e^{\theta_i}}{1+e^{\theta_i}}}{\frac{1}{1+e^{\theta_i}}}\right)=\log(e^{\theta_i})=\theta_i$$ If you look closely, $logit(\pi_i)$ is exactly the log-odds for $x_i$, so it all connects.