3

I was looking at logistic regression and softmax regression.

Based on this formula,

$P(y=j|\mathbf {x} )={\frac {e^{\mathbf {x} ^{\mathsf {T}}\mathbf {w} _{j}}}{\sum _{k=1}^{K}e^{\mathbf {x} ^{\mathsf {T}}\mathbf {w} _{k}}}}$

How does $k=2$ make it logistic regression? I cannot seem to derive this to this

$\sigma (t)={\frac {e^{t}}{e^{t}+1}}={\frac {1}{1+e^{-t}}}$

Firebug
  • 15,262
  • 5
  • 60
  • 127
aceminer
  • 813
  • 1
  • 9
  • 20
  • I thought your question may be duplicated. Please check this one: http://stats.stackexchange.com/q/87248/103153 and this one: http://stats.stackexchange.com/a/254071/12359 – Lerner Zhang Jan 31 '17 at 13:41

1 Answers1

1

The soft-max function is a function with potentially a multi-dimensional input $x$ and potentially multiple outputs, indexed by $j$. The sigmoid function is in principle a 1-dimensional function with a single output. After setting $k=2$, we still need to reconcile those facts. One way to do that is the following way:

Suppose:
$x(t)=t $
$w=\binom{0}{1}$
Then indeed we have:
$P(y=1|x(t))=\frac{e^{x^T w_1}}{\sum_{k=1}^{2}{e^{x^T w_k}}}=\frac{e^{t\cdot 1}}{e^{t \cdot0}+e^{t \cdot1}}=\frac{e^{t}}{1+e^{t}}=\sigma(t)$

dimpol
  • 882
  • 4
  • 13
  • that 1/(1+exp(-t)) is sigmoid, how about the generalised logistic function: https://en.wikipedia.org/wiki/Logistic_function is it still a special case of softmax? – Dee Oct 16 '19 at 08:08