Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [sigmoid-curve]

A sigmoid function is a mathematical function having a characteristic "S"-shaped curve or sigmoid curve. Often, sigmoid function refers to a special case of the logistic function. It is closely related to the logistic regression.

A sigmoid function looks like this:

$P(Z) = \frac{1}{ 1 + e^{-Z}}$

Other examples of similar shapes include the Gompertz curve (used in modeling systems that saturate at large values of x) and the ogee curve (used in the spillway of some dams). Sigmoid functions have domain of all real numbers, with return value monotonically increasing most often from 0 to 1 or alternatively from −1 to 1, depending on convention.

The sigmoid function is also used in logistic neurons in neural networks. It is useful becauase you can take the derivative for any "z".

The sigmoid cursve is also used for modelling growth of a population in biology.

91 questions
191
votes
7 answers

What are the advantages of ReLU over sigmoid function in deep neural networks?

The state of the art of non-linearity is to use rectified linear units (ReLU) instead of sigmoid function in deep neural network. What are the advantages? I know that training a network when ReLU is used would be faster, and it is more biological…
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
64
votes
5 answers

Why is tanh almost always better than sigmoid as an activation function?

In Andrew Ng's Neural Networks and Deep Learning course on Coursera he says that using $tanh$ is almost always preferable to using $sigmoid$. The reason he gives is that the outputs using $tanh$ centre around 0 rather than $sigmoid$'s 0.5, and this…
Tom Hale
  • 2,231
  • 3
  • 13
  • 31
27
votes
3 answers

Relu vs Sigmoid vs Softmax as hidden layer neurons

I was playing with a simple Neural Network with only one hidden layer, by Tensorflow, and then I tried different activations for the hidden layer: Relu Sigmoid Softmax (well, usually softmax is used in the last layer..) Relu gives the best train…
Bill Ancalagon the black
  • 1,704
  • 2
  • 14
  • 16
13
votes
2 answers

sklearn logistic regression converging to unexpected coefficient for a simple case

The case is as follows: Suppose that import numpy as np X = np.array([1, 1, 1]) y = np.array([1, 0, 1]) Then I perform a logistic regression with no intercept to check out the fitted coefficient: from sklearn.linear_model import…
Syllabear
  • 393
  • 1
  • 11
9
votes
3 answers

Logistic function with a slope but no asymptotes?

The logistic function has an output range 0 to 1, and asymptotic slope is zero on both sides. What is an alternative to a logistic function that doesn't flatten out completely at its ends? Whose asymptotic slopes are approaching zero but not zero,…
Aksakal
  • 55,939
  • 5
  • 90
  • 176
9
votes
0 answers

Expected value of softmax transformation of Gaussian random vector

Let $\mathbf w_1,\mathbf w_2,\ldots,\mathbf w_n \in \mathbb R^p$ and $\mathbf v \in \mathbb R^n$ be fixed vectors, and $\mathbf x \sim \mathcal N_p(\boldsymbol{\mu}, \mathbf{\Sigma})$ be an $p$-dimensional Gaussian random vector. Consider the…
dohmatob
  • 468
  • 2
  • 13
8
votes
4 answers

Is Wikipedia's page on the sigmoid function incorrect?

Is Wikipedia's page on the sigmoid function incorrect? It states that: A common example of a sigmoid function is the logistic function From my knowledge of machine learning, I thought that "the sigmoid function" is defined as the logistic…
stackoverflowuser2010
  • 3,190
  • 5
  • 27
  • 35
8
votes
1 answer

Is there any difference between a Logistic Regression for inference and for prediction?

When studying Linear Regression, I remember that Multicollinearity is something that impacts inference and not prediction, at least not always. Also, I noted that the assumptions tends to be neglected by those who are only interested in…
dsbr__0
  • 707
  • 2
  • 7
7
votes
2 answers

Do you do linear regression in logistic regression?

I just studied single & multiple linear regression 2 days ago now I'm reading about logistics regression and I want to implement it from scratch, I just want to know If my understanding about logistic regression is correct. You see I was actually…
himynameisjm
  • 71
  • 2
7
votes
3 answers

Predicting proportions from time with a discontinuity

Imagine having a dependent variable $Y$ that is a proportion (i.e., the proportion of observations made at the given time point that satisfy a condition, where each time point involves 50 to 250 observations) $Y$ is measured at a series of time…
Jeromy Anglim
  • 42,044
  • 23
  • 146
  • 250
7
votes
7 answers

Looking for function to fit sigmoid-like curve

I'm looking for a function to fit sigmoid-like curves, from experimental data points. The model (the function) doesn't matter, it doesn't have to be physically relevant, I just want to be able to calculate y from any x. And I don't want to…
Rififi
  • 315
  • 2
  • 3
  • 8
6
votes
1 answer

Finding the slope at different points in a sigmoid curve

This is my data. x <- c(0.5,3.0,22.2,46.0,77.3,97.0,98.9,100.0) plot(x, pch = 19) I want to fit a curve through these points and then calculate the slope at different points. Could anyone tell me how to do this in R EDIT Previously I had fitted a…
89_Simple
  • 751
  • 1
  • 9
  • 23
6
votes
3 answers

Why do we use the natural exponential in logistic regression?

I would like to intuitively understand the benefit of using the natural exponential in the sigmoid function used in logistic regression. Why should it have to be $e^x$ instead of, for example $2^x$?
John Subas
  • 71
  • 1
  • 5
6
votes
1 answer

How to analyse growth rate in R?

Within the framework of an experiment I followed to growth rate of bird nestlings. I measured them every day for weight and tarsus. I have a number of continuous and categorical explanatory variables, as well as brood ID that should be used as a…
user3162369
  • 126
  • 6
4
votes
3 answers

How does logistic growth rate coincide with the slope of the line in the exponential phase of the growth?

For a logistic function $$f(x) = \frac{L}{1 + e^{-k(x - x_0)}},$$ people call $k$ the logistic growth rate. Now, I have encountered this statement: In the log scale the logistic growth rate coincides with the slope of the line in the exponential…
Dave
  • 43
  • 2
1
2 3 4 5 6 7