the validation set accuracy is somewhat higher than the test set accuracy

[Jessen-Li]

In chapter 6 Fine-tuning for classification, page 200

Training accuracy: 97.21%
Validation accuracy: 97.32%
Test accuracy: 95.67% 

It also said,

Typically, the validation set accuracy is somewhat higher than the test set
accuracy because the model development often involves tuning hyperparameters to
perform well on the validation set, which might not generalize as effectively to the test
set.

But in the def train_classifier_simple, there is no tuning hyper parameters to fit validation set, right?

[d-kleine]

Hi @Jessen-Li,

Unfortunately, @rasbt is currently recovering from an injury, he will reply to your question as soon as he’s feeling better. Therefore I’ll do my best to answer your question in the meantime.

But in the def train_classifier_simple, there is no tuning hyper parameters to fit validation set, right?

Yeah, I would say that you are right about that this is not implemented in the code. I suppose this is just a general statement explaining how val and test sets are used in ML workflows, not specific to the train_classifier_simple code.

I hope this answers your question! Please let me know if you have any further questions - I’m happy to help. 🙂

[rasbt]

Thanks for answering, and yes, that was meant as a general statement regarding

Typically, the validation set accuracy is somewhat higher than the test set
accuracy because the model development often involves tuning hyperparameters to
perform well on the validation set, which might not generalize as effectively to the test
set.

Regarding

But in the def train_classifier_simple, there is no tuning hyper parameters to fit validation set, right?

however, it's been some time since I ran these experiments, but I think I actually did try a handful of different hparam settings during the development process.