How can I give a System Message to LLM?

[sam-siavoshian]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from flask import Flask, request, jsonify
import os
from PyPDF2 import PdfReader
from io import BytesIO
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain.text_splitter import CharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_openai import ChatOpenAI
import requests
from flask_cors import CORS
from dotenv import load_dotenv

app = Flask(__name__)
CORS(app)

# Set OpenAI API key
load_dotenv()
openai_api_key = os.getenv("OPENAI_API_KEY")
os.environ["OPENAI_API_KEY"] = openai_api_key

@app.route("/")
def index():
    return "App is running!", 200


@app.route('/process-pdf', methods=['POST'])
def process_pdf():
    model = ChatOpenAI(model="gpt-3.5-turbo")

    data = request.json
    pdf_url = data['pdf_url']
    pdf_response = requests.get(pdf_url)
    pdf_content = pdf_response.content
    reader = PdfReader(BytesIO(pdf_content))
    raw_text = ''.join([page.extract_text() for page in reader.pages if page.extract_text()])
    text_splitter = CharacterTextSplitter(separator="\n", chunk_size=1000, chunk_overlap=200, length_function=len)
    texts = text_splitter.split_text(raw_text)

    embeddings = OpenAIEmbeddings()

    docsearch = FAISS.from_texts(texts, embeddings)

    query = "Provide a concise summary of the document in a single paragraph, listing only the data that they collected from users. Format the list numerically as follows: 1) 2) 3) Do not include any additional information beyond what is directly related to data collection."
    docs = docsearch.similarity_search(query)

    prompt = ChatPromptTemplate.from_template("Provide a concise summary of the document in a single paragraph, listing only the data that they collected from users. Format the list numerically as follows: 1) 2) 3) Do not include any additional information beyond what is directly related to data collection. {topic}")
    output_parser = StrOutputParser()

    chain = prompt | model | output_parser

    document = docs

    answers = chain.invoke({"topic": document})

    print("Answers:", answers)

    print("App is running! (inside of process_pdf)")

    return jsonify({"answers": answers})

if __name__ == '__main__':
    port = int(os.environ.get("PORT", 8080))
    app.run(debug=False, host='0.0.0.0', port=port)

print("App is running! (outside of process_pdf)")

Description

Hey everyone, I have a problem with giving a system message to LLm using langchain, we can do this easily with OpenAi API like this:{"role": "system", "content": "You are a helpful assistant."},
But how can I apply it using LangChain?

System Info

.

[dosubot[bot]]

To integrate a system message into LangChain's workflow with the ChatOpenAI model, you can use SystemMessage and HumanMessage instances. Here's a concise way to do it:

1. Import SystemMessage and HumanMessage from langchain_core.messages.
2. Create a SystemMessage instance with the context or instructions you want to provide to the model, e.g., SystemMessage(content="You are a helpful assistant.").
3. Create a HumanMessage instance with the user's query or input.
4. Pass both messages to the ChatOpenAI instance in a list, ensuring the system message is placed before the human message.
5. The model will process these messages, considering the context provided by the system message for generating responses.

Here's a snippet showing how to implement this:

from langchain_core.messages import SystemMessage, HumanMessage
from langchain_openai import ChatOpenAI

# Initialize your ChatOpenAI model
model = ChatOpenAI(model="gpt-3.5-turbo")

# Create a system message
system_message = SystemMessage(content="You are a helpful assistant.")

# Create a human message (replace "Hello" with your actual query)
human_message = HumanMessage(content="Hello")

# Pass both messages to the model
response = model([system_message, human_message])

# Process the response
answers = response.content

This approach allows the ChatOpenAI model to generate responses that are aware of the context or instructions provided by the system message.

Sources

• How to use langchain with Open AI
• How to add a custom message/prompt template
• libs/community/tests/integration_tests/chat_models/test_openai.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[eyurtsev]

@dosu Please use .invoke instead of call

[eyurtsev]

Take a look at

https://python.langchain.com/docs/modules/model_io/prompts/quick_start/#chatprompttemplate