Data augmentation in EfficientNetB0 makes accuracy 2% worse?

[steveSchneider2]

In lecture 163 doing transfer learning, classification of 10 food types w/ and w/o data augmentation. with identical models (except for the data augmentation) the model w/o augmentation does 2% better accuracy. Daniel points that out. He does not discuss why that occurs. Does the explanation come later in the course?

I have been researching my own question on data augmentation. It seems to me that creating a layer like this: data_augmentation = Sequential([
preprocessing.RandomHeight(0.2),
preprocessing.RandomWidth(0.2),
preprocessing.RandomRotation(10) ], name='data_augmentation')
... can never make the image better...at best it doesn't get worse. Furthermore it would seem that the original image never makes it thru this layer w/o some kind of alteration. Therefore, data augmentation could only be useful, if you are ADDING such images to the original set of images. It looks to me like (in the course) we are REPLACING original images with often worse images...but, 1 for 1. So, if all that i have said is correct...then OF COURSE the accuracy drops. What might i be missing here?

[someshfengde]

data augmentation can have really good results on your model or maybe worsen your model
after you done data augmentation model get's some images which are modified from original data ( you are doing some operations on images while doing data augmentation ) but while model is predicting on the test data the images are not augmented they are provided as they are because of that you may experience some lower accuracy

in general augmentation helps model to generalize the data but some times it lowers accuracy as images its training on are really different from images it's predicting on

[steveSchneider2]

Would you mind 'augmenting' your answer? :) I had a second question buried in the original:

"Furthermore it would seem that the original image never makes it thru this layer w/o some kind of alteration. Therefore, data augmentation could only be useful, if you are ADDING such images to the original set of images. It looks to me like (in the course) we are REPLACING original images with often worse images...but, 1 for 1."

Phrased differently: When augmenting data... do BOTH original images AND augmented images get trained on?

[mrdbourke]

Usually when augmenting, only augmented images get trained on, not original images - this is default behaviour for augmentation layers in TensorFlow/Keras.

When an original image passes through an augmentation layer, a random augmentation is applied to that image and then that augmented image is used for training. The original image remains unchanged on disk but does not get used in training, only its augmented version does.

[steveSchneider2]

Thank you for your answer. Just to be sure i have this right... per the code below, If i have 100 images in the training set, the model will analyze 300 images, because each image will modified in 3 ways, and none of the original will 'make it thru to the model'.

Sequential([
preprocessing.RandomHeight(0.2),
preprocessing.RandomWidth(0.2),
preprocessing.RandomRotation(10) ]

[Keerthana0207]

I don't think the model will get to see 300 images if the training set contains 100 images. Rather the model gets to analyze slightly modified versions (in the above example height, width and rotation are modified) of the original 100 images.
preprocessing.RandomHeight(0.2) results in an output with height modified by a random value in the range[-20%, +20%]. That way if '0' is randomly selected from the range, height is not altered.
So, in the rare case if height, width and rotation are all modified by 0%, the model may get to see the original image.

[steveSchneider2]

As a follow on... if i wanted the originals to be seen by the model, I could add a 4th line...

Sequential([
preprocessing.RandomHeight(0.2),
preprocessing.RandomWidth(0.2),
preprocessing.RandomRotation(10),
preprocessing.RandomHeight(0.0) ] # notice i 'preprocess' by altering the height by ZERO

This way, the model sees three changed versions of each image PLUS the original.
It seems to me that the default OUGHT to always allow the original...but that's just me.