KeyError: Caught KeyError in DataLoader worker process 0.

[ChenChao20180801]

Hi, when i running inferrnce code,i get the error below,need your help

D:\python\files\SwapNet-master>python inference.py --checkpoint checkpoints/deep_fashion --cloth_dir ./data/deep_fashion/ --texture_dir ./data/deep_fashion/ --body_dir ./data/deep_fashion/
model None
dataset None
=====OPTIONS======
config_file : None
comments :
verbose : False
display_winsize : 256
checkpoints_dir : ./checkpoints
load_epoch : latest
dataset : None
dataset_mode : image
cloth_representation : labels
body_representation : rgb
cloth_channels : 19
body_channels : 12
texture_channels : 3
pad : False
load_size : 128
crop_size : 128
crop_bounds : None
max_dataset_size : 50
batch_size : 8
shuffle_data : False
num_workers : 0
gpu_id : 0
no_confirm : False
interval : 1
warp_checkpoint : None
texture_checkpoint : None
checkpoint : checkpoints/deep_fashion
body_dir : ./data/deep_fashion/
cloth_dir : ./data/deep_fashion/
texture_dir : ./data/deep_fashion/
results_dir : results
skip_intermediates : False
dataroot : None
model : None
name :
is_train : False

The experiment directory 'results' already exists.
Here are its contents:
['args.json', 'texture', 'warp']

Existing data will be overwritten!
Are you sure you want to continue? (y/N): y

Set warp_checkpoint to checkpoints/deep_fashion\warp\latest_net_generator.pth
Set texture_checkpoint to checkpoints/deep_fashion\texture\latest_net_generator.pth
Running warp inference...
Rebuilding warp from checkpoints/deep_fashion\warp\latest_net_generator.pth
Not overriding: {'body_dir', 'cloth_dir', 'texture_dir', 'checkpoint'}
initialize network with kaiming
model [WarpModel] was created
loading the model generator from checkpoints/deep_fashion\warp\latest_net_generator.pth
---------- Networks initialized -------------
[Network generator] Total number of parameters : 137.584 M

Creating dataset warp... cloth dir ./data/deep_fashion/
Extensions: ['.npz']
body dir ./data/deep_fashion/
dataset [WarpDataset] was created
Warping cloth to match body segmentations in ./data/deep_fashion/...
0%| | 0/50 [00:00<?, ?img/s]D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
D:\python\files\SwapNet-master\env\lib\site-packages\torch\nn\functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
2%|█▋ | 1/50 [00:04<03:25, 4.20s/img 8%|██████▌ | 4/50 [00:04<02:15, 2.96 14%|███████████▍ | 7/50 [00:04<01:29, 20%|████████████████▏ | 10/50 [00:04<00 26%|█████████████████████ | 13/50 [00:0 32%|█████████████████████████▉ | 16/50 38%|██████████████████████████████▊ | 1 44%|███████████████████████████████████▋ 50%|████████████████████████████████████████▌ 56%|█████████████████████████████████████████████▎ 62%|██████████████████████████████████████████████████▏ 68%|███████████████████████████████████████████████████████ 74%|█████████████████████████████████████████████████████████ 80%|█████████████████████████████████████████████████████████ 86%|█████████████████████████████████████████████████████████ 92%|█████████████████████████████████████████████████████████ 98%|█████████████████████████████████████████████████████████ 100%|█████████████████████████████████████████████████████████ ████████████████████████| 50/50 [00:06<00:00, 7.49img/s]
Warp results stored in results\warp
Running texture inference...
Rebuilding texture from checkpoints/deep_fashion\texture\latest_net_generator.pth
Not overriding: {'body_dir', 'cloth_dir', 'texture_dir', 'checkpoint'}
initialize network with kaiming
model [TextureModel] was created
loading the model generator from checkpoints/deep_fashion\texture\latest_net_generator.pth
---------- Networks initialized -------------
[Network generator] Total number of parameters : 41.900 M

Creating dataset texture... dataset [TextureDataset] was created
Texturing cloth segmentations in results\warp...
0%| | 0/50 [00:00<?, ?img/s]
Traceback (most recent call last):
File "inference.py", line 222, in
_run_texture()
File "inference.py", line 184, in _run_texture
_run_test_loop(texture_model, texture_dataset, webpage)
File "inference.py", line 109, in run_test_loop
for i, data in enumerate(dataset):
File "D:\python\files\SwapNet-master\datasets_init.py", line 82, in iter
for i, data in enumerate(self.dataloader):
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data\dataloader.py", line 345, in next
data = self._next_data()
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data\dataloader.py", line 856, in _next_data
return self._process_data(data)
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data\dataloader.py", line 881, in _process_data
data.reraise()
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch_utils.py", line 394, in reraise
raise self.exc_type(msg)
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexes\base.py", line 2657, in get_loc
return self._engine.get_loc(key)
File "pandas_libs\index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas_libs\index.pyx", line 127, in pandas._libs.index.IndexEngine.get_loc
File "pandas_libs\index.pyx", line 147, in pandas._libs.index.IndexEngine._get_loc_duplicates
KeyError: './data/deep_fashion/body\MEN\Denim\id_00000080\01_1_front'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data_utils\worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\python\files\SwapNet-master\env\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\python\files\SwapNet-master\datasets\texture_dataset.py", line 118, in getitem
rois = np.rint(self.rois_df.loc[file_id].values * scale)
File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexing.py", line 1500, in getitem
return self._getitem_axis(maybe_callable, axis=axis)
File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexing.py", line 1913, in _getitem_axis
return self._get_label(key, axis=axis)
File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexing.py", line 141, in _get_label
return self.obj._xs(label, axis=axis)
File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\generic.py", line 3585, in xs
loc = self.index.get_loc(key)
File "D:\python\files\SwapNet-master\env\lib\site-packages\pandas\core\indexes\base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas_libs\index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas_libs\index.pyx", line 127, in pandas._libs.index.IndexEngine.get_loc
File "pandas_libs\index.pyx", line 147, in pandas._libs.index.IndexEngine._get_loc_duplicates
KeyError: './data/deep_fashion/body\MEN\Denim\id_00000080\01_1_front'

[HardiRathod]

Hi, I was facing the same problem. Are you running on Windows? Please check the forward and backward slashes in the command that you are running in windows. If it is still not resolved, let me know so I can detail you.

[AyeshaShafique]

Hi @ChenChao20180801 @HardiRathod @andrewjong im facing the same issue. Kindly help. Is quite urgent.

thanks

[Tomas1337]

Also having problem with this on windows.

[Tomas1337]

Just reporting my findings for now just in case someone knows how to solve this.

This problem occurs when enumerating on the self.dataloader on the texture stage. It has no problem on the warp stage.
Inspecting self.dataloader on Warp stage

special variables
function variables
batch_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202630D0648>
batch_size:1
dataset:<datasets.warp_dataset.WarpDataset object at 0x0000020257C15548>
drop_last:False
multiprocessing_context:None
num_workers:4
pin_memory:False
sampler:<torch.utils.data.sampler.RandomSampler object at 0x0000020257C154C8>
timeout:0
worker_init_fn:None
_DataLoader__initialized:True
_DataLoader__multiprocessing_context:None
_auto_collation:True
_dataset_kind:0
_index_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202630D0648>

while running the texture script:

special variables
function variables
batch_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202A5145A08>
batch_size:1
dataset:<datasets.texture_dataset.TextureDataset object at 0x00000202B58BD488>
drop_last:False
multiprocessing_context:None
num_workers:4
pin_memory:False
sampler:<torch.utils.data.sampler.RandomSampler object at 0x00000202B56D6DC8>
timeout:0
worker_init_fn:None
_DataLoader__initialized:True
_DataLoader__multiprocessing_context:None
_auto_collation:True
_dataset_kind:0
_index_sampler:<torch.utils.data.sampler.BatchSampler object at 0x00000202A5145A08>

[Tomas1337]

Hi, I was facing the same problem. Are you running on Windows? Please check the forward and backward slashes in the command that you are running in windows. If it is still not resolved, let me know so I can detail you.

@HardiRathod

Backslashes seem to be consisent after some modifications. I suspect that its the slashes somewhere when creating the dataset for the Texture portion. I just cant trace/debug it

[heyJatin]

same issue. Any solution yet?