What is dev set in machine learning? What are its requirements ?

Dev set or development set in machine learning is nothing but the validation set. Read here for more explanation. Dev set has following requirements:

  1. Dev set and test set needs to have same distribution. It is okay to have training and dev set slightly different distributions. However, dev set and test set must satisfy the same distribution.
  2. Few years back, the split of training, dev set and test set used to be 60-20-20 or 70-20-10. However with increasing amount of data and increase in deep learning models, training data needs more proportion. For eg, if there are 1 million samples in total, good thumb rule is to use 800-900K for training and rest can be split into dev set and test set.

Leave a Reply

Your email address will not be published. Required fields are marked *