The Swish activation function is a relatively new activation function that has gained popularity in deep learning neural networks. It was introduced by researchers at Google in 2017.
The Swish activation function is defined as follows:
f(x) = x * sigmoid(beta * x)