Why do we use only 10% of the training data for our experiments?

[ashikshafi08]

We use only 10% of the training data for two main reasons,

1. To speed up our experiments.

2. To understand how transfer learning works best with only a little amount of data.

In practice, you will be having a huge amount of data and it's obvious we want to run different kinds of experiments on our data to know what works and what doesn't. Imagine having GB's of data and training your model on it, this could take potentially hours for one epoch.

So it's wise to take 10% of our data and run series of experiments on our data and get a rough insight into what works and what doesn't. We use less data so that we will have a good amount of time for experimentation.

Coming to the second point, since we are using transfer learning with a little amount of data we can get good results because our model has already learned patterns on millions of images and 10% of our image data is enough for those models to draw patterns.