Using langchain.agent with HuggingFacePipeline does not work

[tancs711]

I am able to use the agent together with serpapi and OpenAI.
But I have a problem doing the same with a local model MTP-7b-chat with HuggingFacePipeline.
Please help.

CS

My codes

from langchain.agents import load_tools
from langchain.agents import initialize_agent
import json

query="Get Microsoft share price from the www.bloomberg.com and include the url where you got this information."
llm = HuggingFacePipeline(pipeline=generate_text)
toolkit = load_tools(["serpapi"], llm=llm, serpapi_api_key=SERPAPI_API_KEY)
agent = initialize_agent(toolkit, llm, agent="zero-shot-react-description", verbose=True, return_intermediate_steps=True)
response = agent({"input":query})

Verbose message

Entering new chain...
I should search for the information on the internet
Action: Search for the information on the internet
Action Input: www.bloomberg.com
Observation: Search for the information on the internet is not a valid tool, try another one.
Thought: I should use a search engine to find the information
Action: Search for the information on a search engine
Action Input: Microsoft share price
Observation: Search for the information on a search engine is not a valid tool, try another one.
Thought:
/home/ubuntu/.local/lib/python3.8/site-packages/transformers/pipelines/base.py:1081: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
warnings.warn(

[MinKhantYCC]

I am also trying to create an agent with the LLama2-7b-chat-hf model locally.
Here is how I create.

llm = HuggingFacePipeline(pipeline=pipe)
tools = load_tools(['wikipedia', 'llm-math'], llm=llm)
agent = initialize_agent(
    tools=tools,
    llm = llm,
   agent = AgentType.ZERO_SHOT_REACT_DESCRIPTION,
   verbose = True,
   handle_parsing_errors = True
)

agent.agent.llm_chain.prompt # get prompt and copy
prompt = ...  # reformat prompt with required tags like <s>[INST] {input} [/INST]
agent.agent.llm_chain.prompt = prompt # change the prompt template to be suitable to Llama 2
agent("What is the average life-span of a cat?")

If I tweak the prompt a little bit, the whole output changes. Sometimes, the agent called the wiki tool. Sometimes, it didn't call any tools and generated the Final Answer directly. Sometimes, it looped infinitely. I also don't know how to fix it.