Finetune Dense Passage Retrieval Model - 'str' object has no attribute 'items'

[Chuweiii]

Hey there, I am trying to finetune the dpr model, following the https://haystack.deepset.ai/tutorials/09_dpr_training here and using datasets - data/squad20. However, there is an error:

AttributeError                            Traceback (most recent call last)
[<ipython-input-6-1d070a39096a>](https://localhost:8080/#) in <cell line: 3>()
      1 # Start training our model and save it when it is finished
      2 
----> 3 retriever.train(
      4     data_dir=doc_dir,
      5     train_filename=train_filename,

5 frames
[/usr/local/lib/python3.9/dist-packages/haystack/nodes/retriever/dense.py](https://localhost:8080/#) in train(self, data_dir, train_filename, dev_filename, test_filename, max_samples, max_processes, multiprocessing_strategy, dev_split, batch_size, embed_title, num_hard_negatives, num_positives, n_epochs, evaluate_every, n_gpu, learning_rate, epsilon, weight_decay, num_warmup_steps, grad_acc_steps, use_amp, optimizer_name, optimizer_correct_bias, save_dir, query_encoder_save_dir, passage_encoder_save_dir, checkpoint_root_dir, checkpoint_every, checkpoints_to_keep, early_stopping)
    672             self.model.connect_heads_with_processor(self.processor.tasks, require_labels=True)
    673 
--> 674         data_silo = DataSilo(
    675             processor=self.processor,
    676             batch_size=batch_size,

[/usr/local/lib/python3.9/dist-packages/haystack/modeling/data_handler/data_silo.py](https://localhost:8080/#) in __init__(self, processor, batch_size, eval_batch_size, distributed, automatic_loading, max_multiprocessing_chunksize, max_processes, multiprocessing_strategy, caching, cache_path)
    102             # In most cases we want to load all data automatically, but in some cases we rather want to do this
    103             # later or load from dicts instead of file
--> 104             self._load_data()
    105 
    106     def _get_dataset(self, filename: Optional[Union[str, Path]], dicts: Optional[List[Dict]] = None):

[/usr/local/lib/python3.9/dist-packages/haystack/modeling/data_handler/data_silo.py](https://localhost:8080/#) in _load_data(self, train_dicts, dev_dicts, test_dicts)
    163             train_file = self.processor.data_dir / self.processor.train_filename
    164             logger.info("Loading train set from: %s ", train_file)
--> 165             self.data["train"], self.tensor_names = self._get_dataset(train_file)
    166         else:
    167             logger.info("No train set is being loaded")

[/usr/local/lib/python3.9/dist-packages/haystack/modeling/data_handler/data_silo.py](https://localhost:8080/#) in _get_dataset(self, filename, dicts)
    110         # loading dicts from file (default)
    111         if dicts is None:
--> 112             dicts = list(self.processor.file_to_dicts(filename))  # type: ignore
    113             # shuffle list of dicts here if we later want to have a random dev set splitted from train set
    114             if str(self.processor.train_filename) in str(filename):

[/usr/local/lib/python3.9/dist-packages/haystack/modeling/data_handler/processor.py](https://localhost:8080/#) in file_to_dicts(self, file)
   1065             ...]}
   1066         """
-> 1067         dicts = _read_dpr_json(
   1068             file,
   1069             max_samples=self.max_samples,

[/usr/local/lib/python3.9/dist-packages/haystack/modeling/data_handler/processor.py](https://localhost:8080/#) in _read_dpr_json(file, max_samples, proxies, num_hard_negatives, num_positives, shuffle_negatives, shuffle_positives)
   2211         sample = {}
   2212         passages = []
-> 2213         for key, val in dict.items():
   2214             if key in query_json_keys:
   2215                 sample["query"] = val

AttributeError: 'str' object has no attribute 'items'

I wonder why this occurs? cuz the datasets are downloaded and should have no problem with?

[Chuweiii]

the vars are:

doc_dir = "data/squad20"
train_filename = "train-v2.0.json"
dev_filename = "dev-v2.0.json"

query_model = "facebook/dpr-question_encoder-single-nq-base"
passage_model = "facebook/dpr-ctx_encoder-single-nq-base"

save_dir = "../saved_models/dpr"

[Chuweiii]

https://github.com/deepset-ai/haystack/blob/main/haystack/utils/squad_to_dpr.py