There are issues with custom agents

[mc112611]

In addition to using the pre trained model of OpenAI for prompt tasks, if I want to customize my own agent, its access format is also through Python's requests library. The request format is standard JSON format, and the return value is also JSON format. What should I do.

[julian-risch]

Hi @mc112611 If I understand the issue correctly, you would like to use JSON formatted data in a prompt and the problem is caused by { and } symbols. These curly brackets are used in Haystack's PromptTemplates to indicate variables in the PromptTemplate. Have you tried putting the JSON formatted data in backticks ` in the prompt?

If you have a zero shot react template like the following:

"zero-shot-react": {
        "prompt": "You are a helpful and knowledgeable agent. To achieve your goal of answering complex questions "
        "correctly, you have access to the following tools:\n\n"
        "{tool_names_with_descriptions}\n\n"
        "To answer questions, you'll need to go through multiple steps involving step-by-step thinking and "
        "selecting appropriate tools and their inputs; tools will respond with observations. When you are ready "
        "for a final answer, respond with the `Final Answer:`\n\n"
        "Use the following format:\n\n"
        "Question: the question to be answered\n"
        "Thought: Reason if you have the final answer. If yes, answer the question. If not, find out the missing information needed to answer it.\n"
        "Tool: pick one of {tool_names} \n"
        "Tool Input: the input for the tool\n"
        "Observation: the tool will respond with the result\n"
        "...\n"
        "Final Answer: the final answer to the question, make it short (1-5 words)\n\n"
        "Thought, Tool, Tool Input, and Observation steps can be repeated multiple times, but sometimes we can find an answer in the first pass\n"
        "---\n\n"
        "Question: {query}\n"
        "Thought: Let's think step-by-step, I first need to {transcript}"
    },

you should be able to use curly brackets in the tool_names_with_descriptions for example. If that doesn't work, could you please give a minimal example of where the JSON format is causing problems? Is it in a custom Tool or in the Agent itself?

[mc112611]

If I need to send a request to a model to call, just like using chatgpt, I need to add the API_ KEY, as well as the sentences entered by the user, are accessed to the OpenAI server in JSON format as a request.

So, if I don't use LLM from other companies on the market. I have a model deployed in a pod on a cluster server. A library called "request" needs to be used in Python.

I want to call this LLM in haystack. I need to use the following command

import request
Request. post (http://'xxxxxxx ', JSON={data: {query: Please help me summarize the content of the article: xxxxxxxxxxx}})

So how can I use my own LLM in Haystack to perform prompt tasks and what is the return format.
Thank you very much for your attention