- Less amount of data
- Simple Model like a linear classifier
- Complex Model like a classifier of high degree polynomial
- All of the above
Answer – (1), (3)
Overfitting generally happens if the model tries to fit everything because it is too complex or there is too less amount of data. When your model performs well on the training data but not on test data, it is highly likely that it is overfitting and not generalizing well. This could happen due to noise in the data or less amount of data. But if you’re sure that you’ve enough data and have eliminated the noise from the data like outliers, model should be made simpler in order to avoid overfitting. Process of making a model simpler in order to avoid overfitting is called Regularization. In practice, it’s good to start with simple models than jumping directly to complex models like deep neural nets.