How to use a PromptNode in a pipeline before a Retriever?

[koganei]

Hi, I'm trying to use a PromptNode in a pipeline before a Retriever, but the Retriever expects the 'query' variable to be a string. I can't see to get the output of a PromptNode to be a simple string. It comes by default as an array, and as soon as I try to use a Shaper like AnswerParser() it turns into a big object.

Can someone help me out with this?

[ZanSara]

Hey @koganei! Can you share the code you have so far and highlight the point where you're having issues? It's not clear to me why you would put a PromptNode before a Retriever, but in principle it should be possible.

Also, consider joining our Discord, the community usually helps a lot: https://discord.com/invite/VBpFzsgRVF

[koganei]

Sure, here is some code! My pipeline is used to do a document search in a chatbot format.

My pipeline looks like this:

        self.pipeline = Pipeline()
        self.pipeline.add_node(component=history_prompt_node,
                               name="QueryWithHistory", inputs=["Query"])
        self.pipeline.add_node(component=retriever,
                               name="Retriever", inputs=["QueryWithHistory"])
        self.pipeline.add_node(component=answer_prompt_node,
                               name="AnswerPromptNode", inputs=["Retriever"])

I basically use the "history_prompt_node" to create a single query from the latest message from the user taking into account the past conversation context. I want the prompt node to look at the conversation + the last message and create a single query that I can give to my retriever and my answering prompt node.

def create_history_prompt_node():
    from haystack.nodes import PromptNode, Shaper
    from haystack.nodes.prompt import PromptTemplate, AnswerParser

    return PromptNode(
        "gpt-3.5-turbo",
        api_key=os.environ["OPENAI_API_KEY"],
        max_length=700,
        top_k=3,
        default_prompt_template=PromptTemplate(
            name="prompt-enhancement-using-history",
            prompt_text="You are a chatbot for an Human Resources system in the middle of a conversation.\n"
            "---\n"
            "Here is the history of the conversation so far:\n"
            "{query}"
            "---\n"
            "Instructions: Please consider the history above and consider the last line of the conversation to be a question."
            "Rewrite the question so that the question does not require the history of the conversation."
            "Then rewrite the question so that it is more specific and the answer is more likely to be found in the documents."
            "Your response should only the last rewritten question and nothing else!\n"
            "For example, if the question is 'Am I eligible for a bonus?':\n"
            "What is the bonus eligibility criteria for an employee in Canada?"
        ),
        output_variable="query"
    )

When I try to run just that node, the output looks like this:

{
  "query": [
    {
      "answer": "What are the eligibility criteria for employee benefits in our company?\nWhat specific benefits are full-time employees eligible for in our company?",
      "type": "generative",
      "score": null,
      "context": null,
      "offsets_in_document": null,
      "offsets_in_context": null,
      "document_ids": null,
      "meta": {
        "prompt": "You are a chatbot for an Human Resources system in the middle of a conversation.\n---\nHere is the history of the conversation so far:\n\n        [\n  { \"role\": \"bot\", \"message\": \"Hi, what can I do for you?\" },\n  { \"role\": \"user\", \"message\": \"How do I know if I'm a full-time employee?\" },\n  {\n    \"role\": \"bot\",\n    \"message\": \"An employee is considered full-time if they're scheduled to work more than 40 hours a week\"\n  },\n  { \"role\": \"user\", \"message\": \"Does that mean I'm eligible for benefits?\" }\n]\n        ---\nInstructions: Please consider the history above and consider the last line of the conversation to be a question.Rewrite the question so that the question does not require the history of the conversation.Then rewrite the question so that it is more specific and the answer is more likely to be found in the documents.Your response should only the last rewritten question and nothing else!\nFor example, if the question is 'Am I eligible for a bonus?':\nWhat is the bonus eligibility criteria for an employee in Canada?"
      },
      "__pydantic_initialised__": true
    }
  ]
}

When I would really want it to just be:

{ "query": "What are the eligibility criteria for employee benefits in our company?\nWhat specific benefits are full-time employees eligible for in our company?" }

[koganei]

I ended up switching to an Agent model and it indirectly fixed my issue