What is the difference between parametric and nonparametric models ?

One of the obvious answers to this question is parametric models have parameters while nonparametric models do not. But there is a precise explanation to this statement. Parametric models have predetermined number of parameters before the training starts. This in turn limits the degree of freedom for such models. Limited degree of freedom reduces the risk of overfitting. Example of parametric models are Logistic Regression, Naive Bayes etc.

On the other hand, it is not true that nonparametric models do not have parameters. But instead, they have a lot of parameters. The catch is that in nonparametric models, number of parameters are not determined prior to training. This often can makes the model more adaptable to the training data which leads to overfitting. The solution to this is to restrict the degree of freedom for nonparametric models by regularization techniques. Example of nonparametric model is SVMs, K-NN etc.

Leave a Reply

Your email address will not be published. Required fields are marked *