0

Please excuse me if my wording is bad. I am trying to get a hold of the concepts. To minimize the loss function for real values, we can use SVD to solve the following:

$\arg \min_{A, B \in \mathcal{B}} \sum_{i,j} (A[i,] \times B[,j] - X[i,j])^2$

To minimize a logistic loss function, however, I am not sure how to handle it:

$\arg \min_{b} \sum_{i=1}^n \log(1+\exp(b^T a_i)) - x_i b^T a_i$

Since it is not the same form as above due to the exponential, we can't just use the solution analogously. I have tried taking the derivative and setting it to 0, but I am just stuck now. Any ideas?

EDIT: Please note that I am not necessarily looking for an analytical solution. A numerical one is fine. For instance, I have heard of alternating minimization, but how do I use it in this context?

FullStack
  • 121
  • 6
  • 1
    For the exact reasons you outline this can't be solved using an analytical expression. The workhorse for fitting models of this type is Newton's method. – Sycorax Nov 20 '16 at 17:26
  • @Firebug it looks similar to what Sycorax is suggesting. I have updated my question to specify that I am not looking for an analytical solution. – FullStack Nov 20 '16 at 17:48
  • @FullStack There's a discussion of how to fit logistic regression in *Elements of Statistical Learning* which is available for free from the authors. http://statweb.stanford.edu/~tibs/ElemStatLearn/printings/ESLII_print10.pdf – Sycorax Nov 20 '16 at 18:32

0 Answers0