Question

Notice that we had an overall accuracy greater than 96% in the training data, but the overall accuracy was lower in the test data. This can happen often if we overtrain. In fact, it could be the case that a single feature is not the best choice. For example, a combination of features might be optimal. Using a single feature and optimizing the cutoff as we did on our training data can lead to overfitting. Given that we know the test data, we can treat it like we did our training data to see if the same feature with a different cutoff will optimize our predictions. Which feature best optimizes our overall accuracy

Answer 1

Explanation:

Training data overall accuracy is >96%

Test data overall accuracy is <96%

A single feature is not the best choice but a combination of features might be optimal.

The feature that best optimizes your overall accuracy is the combination of both cut off scores. The mean of the two cut off scores will optimize your predictions of overall accuracy.