8

Most references I find say that the activation function used in nnet is 'usually' a logistic function. But in the case that I would like to test the performance of the trained neural network from nnet, it is necessary to know the exact activation function used.

Andre Silva
  • 3,070
  • 5
  • 28
  • 55
user35549
  • 81
  • 1
  • 2

1 Answers1

10

This is the implemented function (extracted from the C-sources; filennet.c, lines 156-165):

static double
sigmoid(double sum)
{
    if (sum < -15.0)
    return (0.0);
    else if (sum > 15.0)
    return (1.0);
    else
    return (1.0 / (1.0 + exp(-sum)));
}
rcs
  • 542
  • 1
  • 6
  • 22
  • Wonder why there is -15/15 limit, it is because it is faster to check for this condition than calculate `exp(15)`? – Tomas Greif Feb 27 '15 at 21:32
  • `1 / (1 + exp(- (15)))` is approx 0.999999694 and `1 / (1 + exp(-(-15)))` is approx. 0.000000306 so they just choose to take the error. 15 seems pretty arbitrary, but fair. – Jon Nov 18 '16 at 03:02