unable to load google/flan-t5-xl

[Abishek104]

While loading google/flan-t5-xl model to promt node i got the error given below

prompt_node = PromptNode(model_name_or_path="google/flan-t5-xl", default_prompt_template=lfqa_prompt)==>code

Error

raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
ValueError: Could not load model google/flan-t5-xl with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForSeq2SeqLM'>, <class 'transformers.models.t5.modeling_t5.T5ForConditionalGeneration'>).

[anakin87]

Hello @Abishek104!
Please provide some details:

• which version of Haystack are you running?
• what is your environment? Google Colab/your infrastructure (Operating System...)?

[vblagoje]

Abishek, have you installed inference dependency? As we have simplified Haystack installation, to use HuggingFace models you now need to pip install farm-haystack'[inference]' If you haven't installed it - that should resolve your issue. Let us know

[Abishek104]

framework, model = infer_framework_load_model(

File "E:\june-2023\fine_tune_4\venv\lib\site-packages\transformers\pipelines\base.py", line 278, in infer_framework_load_model
raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
ValueError: Could not load model google/flan-t5-xl with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForSeq2SeqLM'>, <class 'transformers.models.t5.modeling_t5.T5ForConditionalGeneration'>).

[vblagoje]

Looks like a corrupted venv to me. I suggest starting fresh with a new environment, pip install farm-haystack'[inference]' and of you go. I just did that from scratch to make sure everything was ok, and it worked for me on the first try.

[Abishek104]

can you please give me the system configuration

[vblagoje]

Mine? I'm running OSX Apple silicon and conda.