1

It is well known that for a linear regression problem $y=X\beta+\epsilon$, we have a closed form solution $\hat{\beta}=(X^TX)^{-1}X^Ty$.

Then, it's natural to ask the similar question about logistic regression, which has also been discussed (e.g. here).

However, I don't understand it well that what the reason there to forbid us from just inverse the logistic transformation and use the closed form solution for linear regression again.

For example, for a logistic regression problem $<y, X>$, since we model it in such a way: $$ y = \dfrac{1}{1+e^{-X\beta}} $$ why can't we inverse that function to get $X\beta=\log\dfrac{y}{1-y}=y'$, and simply following the closed form to get $\hat{\beta}=(X^TX)^{-1}X^Ty'$?

I hope the reason is more than "$1-y$ will be zero for $y=1$" because I believe we can always figure out a way to work around these computation issues.

EDIT:

Thanks for @Chaconne's suggestion of this almost identical question.

  • 5
    Does this question help? https://stats.stackexchange.com/questions/122103/why-is-glm-different-than-an-lm-with-transformed-variable Basically a GLM specifies a model for a transformation of the mean and a distribution for the responses, rather than directly modeling the responses – jld Sep 20 '17 at 15:30
  • Thanks @Chaconne This is truly helpful. I edited the question with a follow-up. –  Sep 20 '17 at 18:57
  • related question [here](https://stats.stackexchange.com/questions/229014/matrix-notation-for-logistic-regression). and do not know if AdamO's answer to my question help. – Haitao Du Sep 20 '17 at 19:03

0 Answers0