Activations
Activation functions are a central part of neural networks. They are the pieces that add nonlinearity to the model and enable the network to learn how to approximate any function. The choice of the activation functions used in the network depends on the type of layer and the kind of model the neural network tries to approximate.