While I understand why we need to iterate in the case where we are solving non-linear weighted least squares or iteratively reweighted least square. But I do not understand why there is a need to iterate as shown below in the case of a linear model:
$$ b^{(m+1)} = b^{(m)} + (X^T WX)^{-1}X^T W z_{(m)} $$ where $ y = Xb$ is a linear model, $m$ is the iteration number, $z_{(m)}$ is the residual. b has the dimensions $(n,1)$ , y is a $(p,1)$ vector, X is $(p,n)$ matrix and W is a $(p,p)$ diagonal matrix. I would imagine we should be able to solve the closed-form solution (second term in the equation) without any iterative approach. Can anybody shed some light here, please?
