Why is logistic regression a linear classifier?

There are two parts to this question –

  1. Why is it a classifier when it does regression ?

Logistic regression is used to estimate the probability that an instance belongs to a particular class. If the estimated probability for an instance is greater than 0.5, then the model predicts that the instance belongs to that particular class, usually called positive class labelled as 1, or else it predicts that it belongs to the negative class labelled as 0). This makes it a binary classifier.

  1. Why is it a linear classifier when it uses a nonlinear sigmoid function ?

Direct output of logistic regression is not a probability number(between 0 and 1). Instead it can output any real value like a regression function. Sigmoid function is used to convert the real value to a real number between 0 and 1 and hence a valid probability number. Now sigmoid function is nonlinear function but logistic regression was trained with a linear cost function. In other words, logistic regression is linear in parameters like any other linear classifier. But the nonlinear sigmoid function is applied on the output of logistic regression function. And this is done in order to convert this output from any real number to a real number between 0 and 1. This makes it easier for us to define the threshold(decision boundary) and hence, distinguish between two classes.


Leave a Reply

Your email address will not be published. Required fields are marked *