Sigmoid

A sigmoid function is a mathematical function that maps an input to an output following an “S”-shaped curve. The sigmoid function commonly used in deep learning is the logistic function given by

$$\sigma(x) = \frac{1}{1 + e^{-x}}$$

This function maps any input to an output in the interval (0,1).

Its derivative can be computed as

$$\frac{d\sigma}{dx} = \frac{e^{-x}}{(1+e^{-x})^2}$$

which can be rewritten in terms of $\sigma$ as

$$\frac{d\sigma}{dx}=\sigma(x)(1-\sigma(x))$$

This function is useful in, for instance, binary classification problems where the output of the sigmoid can be seen as the probability that the input belongs to the class.