Non-linear activation functions, such as the widely used sigmoid or rectified linear unit (ReLU), introduce non-linearity to the network, ...