Understanding the connection between 2 ways to write a logistic regression model

Question

I'm currently enrolled in a regression models course through Coursera and am having trouble conceptually understanding logistic regression. I've attached a picture of "where" I'm having trouble.

Linear vs logistic regression

Linear

$$ RW_i = b_0 + b_1RS_i + e_i $$ or $$ E[RW_i|RS_i, b_0,b_1] = b_0+b_1RS $$

Logistic

$$ {\rm Pr}(RW_i|RS_i,b_0,b_1) = \frac{\exp(b_0+b_1RS)}{1+\exp(b_0+b_1RS)} $$ or $$ \log\bigg(\frac{{\rm Pr}(RW_i|RS_i,b_0,b_1)}{1-{\rm Pr}(RW_i|RS_i,b_0,b_1)}\bigg) = b_0+b_1RS $$

In this particular example, the lecture is about modeling Bernoulli outcomes. I understand that when modeling binary data in a linear fashion, the expected value of the outcome is the probability of said outcome (first two formulas in the picture). What I'm having trouble with is where the second two formulas came from and how they can be interchanged (in the lecture, he says if we "invert/work with" [the 3rd formula] it can be rewritten as the 4th formula) I'd like to know the connection between the last two equations and how they should be interpreted.

Answer 1

(@Gijs is right that this is just algebra, but perhaps it will help you to see it worked out.)

Consider that "$\exp(b_0+b_1RS)$" is an odds of some event, and that "${\rm Pr}(RW_i|RS_i,b_0,b_1)$" is the probability of the same event (cf., Interpretation of simple predictions to odds ratios in logistic regression). To make it easier to see what's going on, we can replace these complicated expressions with "$o$" and "$p$", respectively.

Now, recognize that:
$$ p=\frac{o}{1+o} \qquad\qquad o=\frac{p}{1-p} $$ If you substitute the full expressions into the formula on the left, you will get your third equation.

On the other hand, if we substitute the full expressions into the right formula, you would get this:
$$ \exp(b_0+b_1RS) = \bigg(\frac{{\rm Pr}(RW_i|RS_i,b_0,b_1)}{1-{\rm Pr}(RW_i|RS_i,b_0,b_1)}\bigg) $$ which isn't quite the same. To get to your fourth formula, take the log of both sides, and then just switch the right hand side and the left hand side:
\begin{align} \log(\exp(b_0+b_1RS)) &= \log\bigg(\frac{{\rm Pr}(RW_i|RS_i,b_0,b_1)}{1-{\rm Pr}(RW_i|RS_i,b_0,b_1)}\bigg) \\[8pt] \log\bigg(\frac{{\rm Pr}(RW_i|RS_i,b_0,b_1)}{1-{\rm Pr}(RW_i|RS_i,b_0,b_1)}\bigg) &= b_0+b_1RS \end{align}

Thus, all we need to establish is the fact that the "$o$"s in the right equation above are the same as the "$o$" in the left equation, and likewise for the "$p$"s between the two equations. What is potentially unintuitive is that you will need to take the reciprocal of both sides at a couple of points. It's a bit tedious to write out the algebra, but it doesn't take too many steps:
\begin{align} p &= \frac{o}{1+o} &\text{right formula}& \\[8pt] \frac 1 p &= \frac{1+o}{o} &\text{taking reciprocals}& \\[8pt] \frac 1 p &= \frac 1 o + \frac o o &\text{separate out }\frac o o& \\[8pt] \frac 1 p &= \frac 1 o + 1 &\frac o o = 1& \\[8pt] \frac 1 p - 1 &= \frac 1 o &\text{subtracting 1}& \\[8pt] \frac 1 p - \frac p p &= \frac 1 o &1 = \frac p p& \\[8pt] \frac{1-p}{p} &= \frac 1 o &\text{simplifying}& \\[8pt] \frac{p}{1-p} &= o &\text{taking reciprocals}& \\[8pt] o &= \frac{p}{1-p} &\text{left formula}& \end{align}

You might also find What is the difference between logistic and logit regression? helpful.

Answer 2

It's just algebra to show that if

$$ \frac{e^x}{1 + e^x} = a $$

then

$$ \log(\frac{a}{1 - a}) = x $$

and this explains the equivalence between the last two formulas.

The first of these formulas translates the outcome of the regression line ($x$ in that formula), which could be any real number, to a number between zero and one ($a$ in that formula). This is called a "link" function. This number between zero and one is used as the modelled probability.