1

Here is the reference: http://nlp.chonbuk.ac.kr/BML/slides_freda/lec7.pdf

We know that logistic regression is implemented by Iteratively Reweighted Least Squares (IRLS), we have the final iteration formula of parameter $\theta:$ $$\theta^{k+1} = \theta^{k} - (H^k)^{-1}g^k$$ $$\Rightarrow\theta^{k+1} = (X^TS^kX)X^TS^kz^k,$$ $$z^k \triangleq X\theta^k+(S^k)^{-1}(y - \mu^k).$$

Here I ignore the statement of notations. From formula: $(X^TS^kX)X^TS^kz^k$ we know that it is the solution of General Least Square with error covariance inverse $(S^k)^{-1}.$ Since $S^k$ is diagonal. Then it degenerates as a Reweighted Least Squares.

I think the conclusion is purely from the result formula, is there any intuitive explanation before we know the result formula?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
user6703592
  • 745
  • 3
  • 8
  • 1
    See https://stats.stackexchange.com/questions/236676/can-you-give-a-simple-intuitive-explanation-of-irls-method-to-find-the-mle-of-a/237384#237384 – kjetil b halvorsen Jul 08 '21 at 18:30

0 Answers0