no definition of 'val_files' in the data_processing.py

[despairisa]

I have used python to run the preprocessing file: data_processing.py, which is in the CUB folder. But it fails in a strange issue, below is the code and error message.

Code:

python3 data_processing.py -data_dir datasets/CUB_200_2011 -save_dir CUB_processed

Error:

Number of train images from official train test split: 5994
Traceback (most recent call last):
File "/home/chiayilai/Desktop/ConceptBottleneck-master/CUB/data_processing.py", line 85, in
train_data, val_data, test_data = extract_data(args.data_dir)
File "/home/chiayilai/Desktop/ConceptBottleneck-master/CUB/data_processing.py", line 64, in extract_data
if val_files is not None:
NameError: name 'val_files' is not defined

The strange thing is that I can’t find the definition of “val_files” in the data_processing.py. And I can’t find any keywords related to it in the readme file. I tried to add a new folder named “val_files”, but it was still no use. Then I altered the “val_files” to “val_data”. It successfully produced three .pkl files, but I don’t know if this way is correct or not.

[ellemcfarlane]

Hey, not an author but looks like they also had an undefined val_files var in the original commit (i.e. it isn't that they accidentally deleted something later on), so not sure what it's doing there 9f6ee6a#diff-2dd9fd989bcf4f6b4bdf75e647ebf91b6d0dc6ae23130e1b38fbfafddb067a5cR65

But they end up returning this anyway:

train_data = train_val_data[split :]
val_data = train_val_data[: split]

so you can probably just comment out all of these lines:

if val_files is not None:
    if img_path in val_files:
        val_data.append(metadata)
    else:
        train_data.append(metadata)

That's what I did anyway

(python36-env) bash-5.1$ python3 data_processing.py -save_dir full_data -data_dir CUB_200_2011
Number of train images from official train test split: 5994
Size of train set: 4796
Processing train set
Processing val set
Processing test set