How to select the best activation function, optimizer and loss function for your deep learning problem

I have been studying Tensorflow for a few months now, and one thing that I have learned is that there are a number of parameters the model must be set to before it will work properly. To avoid confusion, I have decided to put together a piece on what I have learned about the various functions that the model and compiler must be set to so the algorithm can run optimally.

I have put together a table that outlines the different parameters to tune the model to in order to achieve optimum performance:-

Activation functions:-

Activation functions are mathematical functions applied to the output of a neural network layer or a neuron to introduce non-linearity into the network, allowing it to learn and approximate complex relationships in data. Here are some commonly used activation functions:

1. Sigmoid: The sigmoid function is defined as f(x) = 1 / (1 + e^(-x)). It squashes the input values between 0 and 1, which makes it suitable for binary classification problems. However, it tends to saturate and suffer from the vanishing gradient…