error: TypeError: 'NoneType' object cannot be interpreted as an integer

[Guoli-w]

When I used bash dpo.sh via LORA on full with the hh_rlhf_harmless dataset in your script, I always encountered this error. Do you have any advice? Maybe it's because of incomplement code files? I hope you can help me. I really can't fix it.

Traceback (most recent call last):
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/media/volume/name1/home/exouser/SafetyNeuron/src/training/dpo.py", line 639, in
main()
File "/media/volume/name1/home/exouser/SafetyNeuron/src/training/dpo.py", line 635, in main
trainer.train()
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/transformers/trainer.py", line 1780, in train
return inner_training_loop(
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/transformers/trainer.py", line 2085, in _inner_training_loop
for step, inputs in enumerate(epoch_iterator):
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/accelerate/data_loader.py", line 452, in iter
current_batch = next(dataloader_iter)
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 630, in next
data = self._next_data()
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 674, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
return self.collate_fn(data)
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/trl/trainer/utils.py", line 338, in call
to_pad = [torch.LongTensor(ex[k]) for ex in features]
File "/home/exouser/miniconda3/envs/neru1/lib/python3.10/site-packages/trl/trainer/utils.py", line 338, in
to_pad = [torch.LongTensor(ex[k]) for ex in features]
TypeError: 'NoneType' object cannot be interpreted as an integer