KeyError: "Seq2SeqGenerator doesn't have input converter registered for google/t5-v1_1-base. Provide custom converter for google/t5-v1_1-base in Seq2SeqGenerator initialization"

[Chance-Obondo]

Coul someone kindly show me how to add in the "input_converter" as an initializayion in the Seq2SeqGenerator. i jave tried referring to the docs but it is not clear on my end. Here is the code I am using it in

`from haystack.utils.preprocessing import Callable
from haystack.nodes import Seq2SeqGenerator

generator = Seq2SeqGenerator(model_name_or_path="google/t5-v1_1-base", input_converter=)
query = "what vaccines are available in kenya?"

Retrieve related documents from retriever

retrieved_docs = retriever.retrieve(query=query)

Now generate answer from query and retrieved documents

answer = generator.predict(
query=query,
documents=retrieved_docs,
top_k=1
)`

I am trying to use on of the huggingface models as answer generator.

[anakin87]

Hello @Chance-Obondo!

Seq2SeqGenerator

• As explained in the API reference, you should provide an input_converter: a Callable to prepare model input for the underlying language model.
  For example, see

  haystack/haystack/nodes/answer_generator/transformers.py

  Line 487 in 3d3e9c9

  class _BartEli5Converter:

  
  (I recognize that the docs are not very clear about this point)

• google/t5-v1_1-base is not fine-tuned on downstream tasks, so you probably will get garbage out.
  You should try models that are fine-tuned on downstream tasks, such as valhalla/t5-base-squad (only an example, probably not the best model).

Example

from haystack import Document
from haystack.nodes import Seq2SeqGenerator
from haystack.nodes.answer_generator.transformers import _BartEli5Converter

# some texts about Queen Elizabeth
texts=[
    "Elizabeth II (Elizabeth Alexandra Mary; 21 April 1926 – 8 September 2022) was Queen of the United Kingdom and the other Commonwealth realms from 6 February 1952 until her death in 2022.",
    "She was queen regnant of 32 sovereign states during her life and served as monarch of 15 of them at the time of her death.",
    "Elizabeth met her future husband, Prince Philip of Greece and Denmark, in 1934 and again in 1937.",
    "Her reign of 70 years and 214 days is the longest of any British monarch and the longest recorded of any female head of state in history.",
    "Elizabeth's four children, along with her daughters-in-law and grandsons Prince William and Prince Harry, travelled to Balmoral. Her death was confirmed that evening at 18:30 BST",
    "From Elizabeth's birth onwards, the British Empire continued its transformation into the Commonwealth of Nations."
]

docs = [Document(text) for text in texts]

# as input_converter, I use _BartEli5Converter for a quick example but you should create a custom converter
generator = Seq2SeqGenerator(model_name_or_path="valhalla/t5-base-squad", input_converter=_BartEli5Converter())

generator.predict(
        query="When did Queen Elizabeth die?",
        documents=docs,
        top_k=1
     )

Output:
{'query': 'When did Queen Elizabeth die?', 'answers': [<Answer {'answer': '8 September 2022', 'type': 'generative', ...}>], ...}

PromptNode 💡

Nowadays, if you are interested in generative Question Answering, you should also try the recently introduced PromptNode:

• it supports more modern and powerful models (such as google/flan-t5-large)
• it is very flexible
• requires less setup effort.

[Chance-Obondo]

Just a quick question if maybe I wanted to try out various models such as bioGPT apart from google/flan-t5-large. Can I use them using PromptNode and if so how can I use them because the PromptNode docs specify other models to be used as google/flan t_5 models or the Open AI davinci model

[anakin87]

• In my opinion, BioGPT is not compatible with PromptNode.
  Currently, the PromptNode supports these models:
  

  haystack/haystack/nodes/prompt/providers.py

  Lines 119 to 120 in 2a360cd

  	def instruction_following_models() -> List[str]:
  	return ["flan", "mt0", "bloomz", "davinci", "opt-iml"]

• BioGPT is not even compatible with Seq2SeqGenerator, because it does not implement the seq2seq architecture (encoder+decoder).

I also tag @vblagoje, who is an expert on these topics.

[vblagoje]

Thank you @anakin87 We'll deprecate this class as everyone should move to PromptNode by now