Answer:
In a personal experiment where an M.L. student decides to not use a test set, only train-dev sets, the following is true:
c. He won't be able to measure the variance of the model.
When training a machine learning model, it is important to evaluate its performance on unseen data to understand how well it generalizes. The standard practice is to have three separate datasets: a training set, a development (dev) set, and a test set.
The training set is used to train the model, the dev set is used for hyperparameter tuning and model selection, and the test set is used to measure the final performance and generalization ability of the model.
If the M.L. student chooses to not use a test set and only relies on the train-dev sets, they will not have a separate dataset to measure the variance of the model. Variance refers to how sensitive the model is to the variations in the training data. Without a test set, it becomes challenging to assess if the model is overfitting to the training data or if it can generalize well to new, unseen data.
Therefore, option c is true: the student won't be able to measure the variance of the model if they don't use a test set.