Fine-Tuning a Model

[Chuweiii]

Hi there, I would like to fine-tune my model using local data. However, when I followed the instruction Tutorial: Fine-Tuning a Model on Your Own Data, it shown file not found.

Similar, when I tried to use the datasets from huggingface, it also cannot be found. I wonder if this is the correct way to load data?

Moreover, after I ran the sample codes provided by this link and changed the save_dir to "Downloads", which worked just fine, I cannot find the trained model under "Downloads" dir and have no idea where it is. :((

data_dir = "C:/temp"
# data_dir = "GEM/squad_v2" # from huggingface
# data_dir = "PATH/TO_YOUR/TRAIN_DATA"
reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="Downloads")

Thank you so much!

[bilgeyucel]

Hi @Chuweiii, can you share the information about your OS and the error messages you get? Any info to reproduce this error is appreciated 🙏

[Chuweiii]

sureee! I use windows10 and the code is running via colab.

When I tried to use the dataset from huggingface, the code is:

reader = FARMReader(model_name_or_path="deepset/bert-base-cased-squad2", use_gpu=True)
data_dir = "GEM/squad_v2/squad_data"
# data_dir = "PATH/TO_YOUR/TRAIN_DATA"
reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="Downloads")

the error is:

INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
Downloading (…)lve/main/config.json: 100%
508/508 [00:00<00:00, 11.8kB/s]
INFO:haystack.modeling.model.language_model: * LOADING MODEL: 'deepset/bert-base-cased-squad2' (Bert)
Downloading (…)"pytorch_model.bin";: 100%
433M/433M [00:03<00:00, 163MB/s]
INFO:haystack.modeling.model.language_model:Auto-detected model language: english
INFO:haystack.modeling.model.language_model:Loaded 'deepset/bert-base-cased-squad2' (Bert model) from model hub.
Downloading (…)okenizer_config.json: 100%
152/152 [00:00<00:00, 6.99kB/s]
Downloading (…)solve/main/vocab.txt: 100%
213k/213k [00:00<00:00, 320kB/s]
Downloading (…)cial_tokens_map.json: 100%
112/112 [00:00<00:00, 7.07kB/s]
INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.data_handler.data_silo:
Loading data into the data silo ... 
              ______
               |o  |   !
   __          |:`_|---'-.
  |__|______.-/ _ \-----.|
 (o)(o)------'\ _ /     ( )
 
INFO:haystack.modeling.data_handler.data_silo:LOADING TRAIN DATA
INFO:haystack.modeling.data_handler.data_silo:==================
INFO:haystack.modeling.data_handler.data_silo:Loading train set from: GEM/squad_v2/squad_data/dev-v2.0.json 
INFO:haystack.modeling.data_handler.processor:Couldn't find GEM/squad_v2/squad_data/dev-v2.0.json locally. Trying to download ...
INFO:haystack.modeling.data_handler.processor:downloading and extracting file squad_data to dir /content/GEM/squad_v2
ERROR:haystack.modeling.data_handler.processor:Cannot download squad_data. Unknown data source.
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-7-884d9ba84792>](https://localhost:8080/#) in <module>
      2 data_dir = "GEM/squad_v2/squad_data"
      3 # data_dir = "PATH/TO_YOUR/TRAIN_DATA"
----> 4 reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="Downloads")

6 frames
[/usr/local/lib/python3.8/dist-packages/haystack/nodes/reader/farm.py](https://localhost:8080/#) in train(self, data_dir, train_filename, dev_filename, test_filename, use_gpu, devices, batch_size, n_epochs, learning_rate, max_seq_len, warmup_proportion, dev_split, evaluate_every, save_dir, num_processes, use_amp, checkpoint_root_dir, checkpoint_every, checkpoints_to_keep, caching, cache_path, grad_acc_steps, early_stopping, max_query_length)
    437         """
    438         send_event("FARMReader.train()")
--> 439         return self._training_procedure(
    440             data_dir=data_dir,
    441             train_filename=train_filename,

[/usr/local/lib/python3.8/dist-packages/haystack/nodes/reader/farm.py](https://localhost:8080/#) in _training_procedure(self, data_dir, train_filename, dev_filename, test_filename, use_gpu, devices, batch_size, n_epochs, learning_rate, max_seq_len, warmup_proportion, dev_split, evaluate_every, save_dir, num_processes, use_amp, checkpoint_root_dir, checkpoint_every, checkpoints_to_keep, teacher_model, teacher_batch_size, caching, cache_path, distillation_loss_weight, distillation_loss, temperature, tinybert, processor, grad_acc_steps, early_stopping, distributed, doc_stride, max_query_length)
    270             )
    271         else:  # caching would need too much memory for tinybert distillation so in that case we use the default data silo
--> 272             data_silo = DataSilo(
    273                 processor=processor,
    274                 batch_size=batch_size,

[/usr/local/lib/python3.8/dist-packages/haystack/modeling/data_handler/data_silo.py](https://localhost:8080/#) in __init__(self, processor, batch_size, eval_batch_size, distributed, automatic_loading, max_multiprocessing_chunksize, max_processes, multiprocessing_strategy, caching, cache_path)
    102             # In most cases we want to load all data automatically, but in some cases we rather want to do this
    103             # later or load from dicts instead of file
--> 104             self._load_data()
    105 
    106     def _get_dataset(self, filename: Optional[Union[str, Path]], dicts: Optional[List[Dict]] = None):

[/usr/local/lib/python3.8/dist-packages/haystack/modeling/data_handler/data_silo.py](https://localhost:8080/#) in _load_data(self, train_dicts, dev_dicts, test_dicts)
    163             train_file = self.processor.data_dir / self.processor.train_filename
    164             logger.info("Loading train set from: %s ", train_file)
--> 165             self.data["train"], self.tensor_names = self._get_dataset(train_file)
    166         else:
    167             logger.info("No train set is being loaded")

[/usr/local/lib/python3.8/dist-packages/haystack/modeling/data_handler/data_silo.py](https://localhost:8080/#) in _get_dataset(self, filename, dicts)
    110         # loading dicts from file (default)
    111         if dicts is None:
--> 112             dicts = list(self.processor.file_to_dicts(filename))  # type: ignore
    113             # shuffle list of dicts here if we later want to have a random dev set splitted from train set
    114             if str(self.processor.train_filename) in str(filename):

[/usr/local/lib/python3.8/dist-packages/haystack/modeling/data_handler/processor.py](https://localhost:8080/#) in file_to_dicts(self, file)
    492 
    493     def file_to_dicts(self, file: str) -> List[dict]:
--> 494         nested_dicts = _read_squad_file(filename=file)
    495         dicts = [y for x in nested_dicts for y in x["paragraphs"]]
    496         return dicts

[/usr/local/lib/python3.8/dist-packages/haystack/modeling/data_handler/processor.py](https://localhost:8080/#) in _read_squad_file(filename, proxies)
   2248         logger.info("Couldn't find %s locally. Trying to download ...", filename)
   2249         _download_extract_downstream_data(filename, proxies)
-> 2250     with open(filename, "r", encoding="utf-8") as reader:
   2251         input_data = json.load(reader)["data"]
   2252     return input_data

FileNotFoundError: [Errno 2] No such file or directory: 'GEM/squad_v2/squad_data/dev-v2.0.json'

When I tried to use a local dataset whose path is C:/temp/dev-v2.0.json:

reader = FARMReader(model_name_or_path="deepset/bert-base-cased-squad2", use_gpu=True)
data_dir = "C:/temp"
# data_dir = "PATH/TO_YOUR/TRAIN_DATA"
reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="Downloads")

error:

INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.model.language_model: * LOADING MODEL: 'deepset/bert-base-cased-squad2' (Bert)
INFO:haystack.modeling.model.language_model:Auto-detected model language: english
INFO:haystack.modeling.model.language_model:Loaded 'deepset/bert-base-cased-squad2' (Bert model) from model hub.
INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.utils:Using devices: CUDA:0 - Number of GPUs: 1
INFO:haystack.modeling.data_handler.data_silo:
Loading data into the data silo ... 
              ______
               |o  |   !
   __          |:`_|---'-.
  |__|______.-/ _ \-----.|
 (o)(o)------'\ _ /     ( )
 
INFO:haystack.modeling.data_handler.data_silo:LOADING TRAIN DATA
INFO:haystack.modeling.data_handler.data_silo:==================
INFO:haystack.modeling.data_handler.data_silo:Loading train set from: C:/temp/dev-v2.0.json 
INFO:haystack.modeling.data_handler.processor:Couldn't find C:/temp/dev-v2.0.json locally. Trying to download ...
INFO:haystack.modeling.data_handler.processor:downloading and extracting file temp to dir /content/C:
ERROR:haystack.modeling.data_handler.processor:Cannot download temp. Unknown data source.
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-8-fce7bbcec09a>](https://localhost:8080/#) in <module>
      2 data_dir = "C:/temp"
      3 # data_dir = "PATH/TO_YOUR/TRAIN_DATA"
----> 4 reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="Downloads")

6 frames
[/usr/local/lib/python3.8/dist-packages/haystack/modeling/data_handler/processor.py](https://localhost:8080/#) in _read_squad_file(filename, proxies)
   2248         logger.info("Couldn't find %s locally. Trying to download ...", filename)
   2249         _download_extract_downstream_data(filename, proxies)
-> 2250     with open(filename, "r", encoding="utf-8") as reader:
   2251         input_data = json.load(reader)["data"]
   2252     return input_data

FileNotFoundError: [Errno 2] No such file or directory: 'C:/temp/dev-v2.0.json'

when tried the code provided it worked, but had no idea where it is stored. It is not in the local /Downloads dir.

reader = FARMReader(model_name_or_path="deepset/bert-base-cased-squad2", use_gpu=True)
data_dir = "data/squad20"
# data_dir = "PATH/TO_YOUR/TRAIN_DATA"
reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="Downloads")

I'm quite confused about the dir and path in the function reader.train().

[bilgeyucel]

Hi @Chuweiii, colab has its own environment. That's why if you're running the tutorial on Colab, you need to upload your data to colab, and model files will also be saved in colab after training. As far as I know, there is no way to reach your local files through colab.
To upload files to colab, go to the "Files" tab on the left menu. There you'll see the current files and have the option to upload more files. This is how "Files" tab looks when I use the same code on the tutorial to upload training data and save the model.

If you don't want to upload your files to colab because of safety concerns, you need to download the tutorial and run the .ipynb locally. If this will be your way, I am happy to help you regarding setting the correct path for windows.

[Chuweiii]

I understood, thank you so much, you are really kind and sorry for replying so late! I will try to upload my files first. :))