ChatPromptTemplate will automatically add Human: prefix before the prompt word, anyway to custom?

[CharlinChen]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

streamer = TextIteratorStreamer(
        tokenizer=phi2_tk, skip_prompt=False, skip_special_tokens=True, timeout=300.0
    )
phi2_pipeline = pipeline(
        "text-generation",
        tokenizer=phi2_tk,
        model=phi,
        max_new_tokens=max_new_tokens,
        pad_token_id=phi2_tk.eos_token_id,
        eos_token_id=phi2_tk.eos_token_id,
        device_map="auto",
        streamer=streamer,
    )
hf_model = HuggingFacePipeline(pipeline=phi2_pipeline)
prompt = ChatPromptTemplate.from_template(
        "Instruct: {query}"
    )
chain = LLMChain(llm=hf_model, prompt=prompt, return_final_only=False)
query = f"5 tips to keep healthy"
chain.run(query)

Description

I used LLMChain to build my prompt words, but I found that it would prefix my prompt words with human
I checked the original text input to the model and it was
Human:Instruct: 5 tips to keep healthy
not
Instruct: 5 tips to keep healthy
The extra humans will affect the inference results of the model. Is there any way to not add human prefix or custom prefix?

System Info

langchain==0.1.8
langchain-community==0.0.21
langchain-core==0.1.25
linux
python3.11.7

[dosubot[bot]]

Hey @CharlinChen, great to see you back! Hope you're doing well.

Based on the context provided, it appears that the Human: prefix is being added to the prompts in the DEFAULT_TEMPLATE string within the libs/langchain/langchain/chains/conversation/prompt.py file. Specifically, it is part of the template string assigned to DEFAULT_TEMPLATE, where {history} is followed by Human: {input}, indicating that the prefix is added right before the user's input is inserted into the prompt.

To prevent LLMChain from automatically adding the Human: prefix to your prompt words or to customize the prefix in LangChain, you would need to modify how the prompt is loaded and handled within the LLMChain or related classes. However, based on the provided context from the libs/langchain/langchain/chains/loading.py file, there is no direct reference to automatically adding a Human: prefix to prompts within the LLMChain initialization or related functions.

The customization or prevention of automatic prefix addition likely occurs either in the prompt loading functions (load_prompt, load_prompt_from_config) or within the LLMChain class itself, but the implementation details for these functions and the class constructor are not provided in the context. These functions and the LLMChain class would be the first places to check for the logic that adds the Human: prefix.

If the prefix addition is happening within the prompt loading functions, you would need to modify these functions to either remove the prefix addition or to make the prefix customizable through the configuration. If it's within the LLMChain class, similar modifications would be required.

Without the implementation details of the prompt loading functions or the LLMChain class, it's not possible to provide a specific solution. You would need to review the implementations of these components in your codebase to identify where the prefix is being added and make the necessary adjustments.

Please note that there was a similar issue Can you make system_prefix customizable? which is now closed. You might find some useful information there.

I hope this helps! If you have further questions or need more clarification, feel free to ask.

Sources

• Can you make system_prefix customizable?
• libs/langchain/langchain/chains/loading.py
• libs/langchain/langchain/chains/conversation/prompt.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[maximeperrindev]

@CharlinChen It is because of internal ChatPromptTemplate default template. If you only want to pass your query to LLM, you want to use PromptTemplatefrom langchain_core.prompts.prompt

[CharlinChen]

thx i found the problem just now

[jamesev15]

I've tried this advice but I've got the "Human" at the beginning of the prompt. Idk why langchain does that? If I were using OpenAI instead of ChatOpenAI, the result would be just the "text" content without the word "Human" at the beginning.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate
from langchain.globals import set_debug

set_debug(True)
prompt = PromptTemplate(template="{text}", input_variables=["text"])
model = ChatOpenAI(model="gpt-3.5-turbo")
chain = prompt | model
chain.invoke({"text": "hello"})

...

{
  "prompts": [
    "Human: hello"
  ]
}

[Fabio-Monti-02]

@CharlinChen It is because of internal ChatPromptTemplate default template. If you only want to pass your query to LLM, you want to use PromptTemplatefrom langchain_core.prompts.prompt

I cant use Prompt Template class, Im using the agent component on langflow which relays on the function create_agent_runnable() which need a ChatPromptTemplate(), is there a solution to customize this prefix like in this example:

Here i need to add the special tags for llama like <|start_header_id|>system<|end_header_id|> which are need to work properly