Tag: cross entropy

What is cross entropy loss in deep learning?

Posted on by InterviewBuddy

Cross Entropy loss, serving as a loss function, is heavily used in deep learning models. This is derived from information theory. To explain the cross entropy, let true probability distribution be p  computed model probability be q  Then cross entropy loss or error is given by H(p,q) as: Cross entropy measures how is predicted probability…

What are the commonly used activation functions ? When are they used.

Posted on by InterviewBuddy

The commonly used loss functions are Linear : g(x) = x. This is the simplest activation function. However it cannot model complex decision boundaries. A deep network with linear activations can be shown incapable of handling non-linear decision boundaries. Sigmoid :   This is a common activation function in the last layer of the neural network…