12

Does a function in the form $e^x/(1+e^x)$ have a standard name? E.g. $y = a + bx$ is a linear function.

amoeba
  • 93,463
  • 28
  • 275
  • 317
joelsand
  • 231
  • 2
  • 8
  • 7
    it's the standard logistic function (https://en.wikipedia.org/wiki/Logistic_function) – gazza89 Oct 11 '18 at 19:32
  • 6
    It is the "Sigmoid" function. More commonly expressed as the equivalent $1 / (1 + e^{- x}).$ – Bridgeburners Oct 11 '18 at 19:32
  • 10
    @Bridge "Sigmoid," as suggested by its ending "...oid," is a generic description of any function whose graph is roughly s-shaped. It therefore is a poor (albeit common) choice of term to use when you really do mean the logistic function. – whuber Oct 11 '18 at 20:01
  • 4
    @whuber indeed; it is annoying to nowadays have to ask what the person using "sigmoid" actually intends, whereas some time ago it was safe to assume it took its literal meaning. – Glen_b Oct 11 '18 at 21:34
  • 3
    @Bridgeburners For example the [inverse probit](https://en.wikipedia.org/wiki/Error_function#Inverse_functions) function and [complementary log-log](https://en.wikipedia.org/wiki/Generalized_linear_model#Complementary_log-log_(cloglog)) link functions are *also* sigmoid functions. – Alexis Oct 17 '18 at 15:42
  • 1
    Stephen Jay Gould's essay "Is the Cambrian explosion a sigmoid fraud?" springs to mind. – Nick Cox Nov 29 '18 at 18:44

1 Answers1

27

It does not have a standard name. In different areas of statistics, it has different names.

In the neural networks and deep learning community, it is called the sigmoid function. This is confusing for everyone else, because sigmoid is just a fancy way of saying "S-shaped" and this function is not unique among S-shaped functions; for example, $\tanh$ is also S-shaped and widely used in neural networks, yet it is not commonly termed "sigmoidal" in neural network literature.

In the GLM literature, this is called the logistic function (as in logistic regression).

If the logit function is $$\text{logit}(p)= \log\left(\frac{p}{1-p}\right)= \log(p)-\log(1-p)=x$$ for $p\in(0,1)$, then $$\text{logit}^{-1}(x)= \frac{\exp(x)}{1 + \exp(x)}= \frac{1}{1+\exp(-x)}= p$$ for $x\in\mathbb{R}$. This is the reason some people call $\text{logit}^{-1}$ the inverse logit or anti-logit function. (Thanks, Glen_b!)

Rarely, I've seen the name expit used; as far as I can tell, this is a back-formation from the word logit but never really caught on. (Thanks, CliffAB!)

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • 3
    _arc_ has a specific meaning for inverse trigonometric functions as the inverse of any trigonometric function is, or at least is equivalent to, an angle. It's a pity that this is misunderstood quite often for inverse hyperbolic functions where notations like arcsinh are sometimes encountered. There the inverse does have interpretation as an _area_ and indeed arsinh is better (or asinh). Careful pronunciation may be advisable depending on accent and audience. But _arc_ really can't carry a general meaning of inverse. – Nick Cox Nov 29 '18 at 17:07
  • 1
    I'd say expit is growing slowly for inverse of logit. But it's still rare in what I read. – Nick Cox Nov 29 '18 at 17:09
  • 1
    @NickCox Thanks for the helpful context about *arc*. Indeed, the Google ngram viewer appears to support your observation about the usage of "expit." https://books.google.com/ngrams/graph?content=expit%2Cinverse+logit&year_start=2000&year_end=2008&corpus=15&smoothing=3&share=&direct_url=t1%3B%2Cexpit%3B%2Cc0%3B.t1%3B%2Cinverse%20logit%3B%2Cc0 But for some reason the largest end year allowed is 2008. :-\ – Sycorax Nov 29 '18 at 17:11