What is the bias variance trade-off ?

Bias-variance tradeoff is the tradeoff between training error and the test error. In other words, making the training error the lowest possible, may lead to high generalization error on the test set.The bias-variance tradeoff is a core concept in supervised learning.

We want to design models that best fit the training data capturing all the subtleties in the training data, at the same time generalize well to unseen test data. The bias-variance tradeoff says we cannot do both well simultaneously.

→ If we fit the training data very well, we might end up overfitting to the training data. This might cause high variance in predictions when we try the model on various versions of test data.

→ If we avoid overfitting training data by making the model simple, say, by using a regularizer, we might end up underfitting the training data. We end up with a biased predictor, but it might work well on unseen test data – the variability in predictions across different test data is low (low variance).

Ideally we want low bias (works best it can on training data), low variance (generalizes well to test data). But we need to pick a tradeoff point.

Here is more detailed article on how does bias-variance gets introduced.

Leave a Reply

Your email address will not be published. Required fields are marked *