There are a few issues related to the code that need to be addressed.

[jelymewang]

for i, channel in enumerate(diff):
    perturb[i] -= np.dot(perturb[i], channel) * channel

The meaning of this code is unclear, particularly regarding the choice of subtraction over addition and why the matrix product is multiplied by the channel afterwards

[jelymewang]

def get_diff(sample_1, sample_2):
sample_1 = sample_1.reshape(3, sample_1.shape[0], sample_1.shape[1])
sample_2 = sample_2.reshape(3, sample_2.shape[0], sample_2.shape[1])
sample_1 = np.resize(sample_1, (3, 271, 271))
sample_2 = np.resize(sample_2, (3, 271, 271))

diff = []
for i, channel in enumerate(sample_1):
    diff.append(np.linalg.norm((channel - sample_2[i]).astype(np.float32)))
return np.array(diff)

What is the reason for resizing to 271? I'm not entirely clear on the intention behind this.