07_Milestone project predictions

[Natali-bali]

Hello, could you help me please. When i make predictions on all test data and then compare it to the labels, i don't get any coincidences (accuracy on test data is 75%)
This is my code below

pred_probs = model.predict(test_data)
y_pred = np.array(tf.argmax(pred_probs, axis = 1))
 
y_test = []
for img, label in test_data.unbatch():
    y_test.append(int(label))

Result

np.array(y_test[50:100])
array([ 45,  28,  23,  72,  30,  45,  94,  92,  46,  18,  99,  75,  40,
        16,  16,  60,  36,  83,  32,  48,  94,   0,  65,  56,  87,  37,
        29,  65,  64,  26,  57,  48,  79,  44,  43,  81,  47,  27,  46,
        22,  99,  70, 100,  90,  21,  59,  28,  12,  55,  32])
y_pred[50:100]
array([29, 80, 32, 78, 17,  1, 40, 40, 99,  1, 71, 77, 85, 44, 83, 53,         45,
       56, 23, 30, 72, 45,  8,  9, 49, 18, 39, 75, 79, 40, 34, 48, 79, 15,
       81, 43, 27, 47, 46, 22, 99, 70, 17, 91, 21, 59, 29, 81, 91, 53])

And if i do prediction on single image, i get pretty good result

#Predict 100 images
n=100
for img, label in test_data.unbatch():
  if n>=0:
    img = tf.image.resize(img, [224, 224])
    img = tf.cast(img, dtype = tf.float32) 
    img = tf.expand_dims(img, axis =0)
    pred_prob = model.predict(img)
    pred = np.array(tf.argmax(pred_prob, axis = 1))
    print(label, pred)
    n=-1
  else:
    break  

I guess the problem is in the way i unbatch test data. I had no problems, when data was preprocessed with image_dataset_from_directory
Thanks!

[gauravreddy08]

Hey @Natali-bali

I assume that your working on FoodVision Milestone Project, if that's the case, here's the reason you're facing that issue.

Initially, when we import the dataset from Tensorflow Datasets Module we shuffle it. So every time you try to access test_data variable it gets shuffled. Hence why, your models predictions and the ground-truth values don't match.

Solution

You can simply get rid of shuffling in test_data by setting shuffle=False (only for test_data), and it wont affect the model's accuracy.

[gauravreddy08]

Just test_data.shuffle = False would work !!

[MuhammadAwon]

@gaurxvreddy thanks for the response but I am still having the same issue. After I load in the data (train_data, test_data), ds_info = tfds.load(name="food101", split=["train", "validation"], shuffle_files=True, as_supervised=True, with_info=True), then I set the test_data.shuffle = False and train the model, but still getting different results when I compare the accuracy from model.evaluate(test_data) and accuracy_score(y_labels, pred_labels)

.

[MuhammadAwon]

Plus I can't able find out whether this .shuffle attribute is from TensorFlow or pure Python or fastai. That'll be helpful if I can find some docs.

[gauravreddy08]

Hey @MuhammadAwon do this instead,

Set shuffle_file False when importing the data :

(train_data, test_data), ds_info = tfds.load(name='food101',
                                             split=['train', 'validation'],
                                             shuffle_files=False,
                                             as_supervised=True,
                                             with_info=True)

But again shuffle it while your batching the data :

train_data = train_data.map(preprocess_img, tf.data.AUTOTUNE)
train_data = train_data.shuffle(buffer_size=1000).batch(32).prefetch(tf.data.AUTOTUNE)

Only set shuffle-->True for train_data because that's what you wanna do

[MuhammadAwon]

@gaurxvreddy thank You!! I went ahead with that.

[arghanath007]

Thanks man, this really solved my Accuracy score which so low like 0.0001 and was getting an error on the Confusion matrix as well. It was not plotting correctly.

[gauravreddy08]

Glad that I could help !!