Error when packaging NVIDIA Embedding Model with RunnableWithMessageHistory

[ashish-kumar-hpe]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You're an assistant who's good at something. Here is some {context}",
        ),
        MessagesPlaceholder(variable_name="history"),
        ("human", "{question}"),
    ]
)

query_embedder  = NVIDIAEmbeddings(model="nvidia/nv-embedqa-e5-v5", base_url=base_url, api_key=api_key, verify="./security/chain.pem")

vector_store = Milvus(
        embedding_function=query_embedder,
        connection_args={"host": scalableai_constants.MILVUS_CONN_HOST, "port": scalableai_constants.MILVUS_CONN_PORT},
        collection_name=scalableai_constants.MILVUS_COLLECTION_NAME_1
)

retriever = vector_store.as_retriever()

reranker = NVIDIARerank(
        base_url=scalableai_constants.NVIDIA_RERANKER_MODEL_ENDPOINT,
        model=scalableai_constants.NVIDIA_RERANKER_MODEL,
        verify=scalableai_constants.SCALABLEAI_SECURE_SSL_KEY_CHAIN,
        api_key=scalableai_constants.NVIDIA_RERANKER_MODEL_TOKEN,
)

reranking_retriever = ContextualCompressionRetriever(base_compressor=reranker, base_retriever=retriever)

llm = ChatNVIDIA(
        base_url=scalableai_constants.NVIDIA_CHATBOT_MODEL_ENDPOINT,
        model=scalableai_constants.NVIDIA_CHATBOT_AI_MODEL,
        verify=scalableai_constants.SCALABLEAI_SECURE_SSL_KEY_CHAIN,
        api_key=scalableai_constants.NVIDIA_CHATBOT_MODEL_TOKEN
)

original_chain = (
        {"context": reranking_retriever, "question": RunnablePassthrough()}
        | prompt_template
        | llm
        | StrOutputParser()
)

chain_with_history = RunnableWithMessageHistory(
       original_chain,
       scalableai_persist_history.get_session_history,
       input_messages_key="question",
       history_messages_key="history",
)

config = {"configurable": {"session_id": "test_session_id"}}
result = chain_with_history.invoke({"question": "Can you tell me something about NVIDIA?"}, config)
print(result)

Description

Exception: [500] Internal Server Error
{'_content': b'Internal Server Error', '_content_consumed': True, '_next': None, 'status_code': 500, 'headers': {'content-length': '21', 'content-type': 'text/plain; charset=utf-8', 'date': 'Fri, 16 May 2025 11:14:15 GMT', 'server': 'istio-envoy', 'x-envoy-upstream-service-time': '19'}, 'raw': <urllib3.response.HTTPResponse object at 0x7fa90c220190>, 'url': 'https://pcai-embedding-deployment-1-predictor-admin-6c8b1978.aie.hpelab.local/v1/embeddings', 'encoding': 'utf-8', 'history': [], 'reason': 'Internal Server Error', 'cookies': <RequestsCookieJar[]>, 'elapsed': datetime.timedelta(microseconds=26621), 'request': <PreparedRequest [POST]>, 'connection': <requests.adapters.HTTPAdapter object at 0x7fa90c221b70>}

System Info

pip freeze|grep langchain
langchain==0.1.11
langchain-community==0.0.25
langchain-core==0.1.52
langchain-nvidia-ai-endpoints==0.1.4
langchain-text-splitters==0.0.2
langchain-weaviate==0.0.4

[ashish-kumar-hpe]

When i try to run this above example with just a simple llm then it works fine.

chain_with_history = RunnableWithMessageHistory(
llm,
scalableai_persist_history.get_session_history
)

But it throws error as above when i add complete chain.

[ashish-kumar-hpe]

I was able to resolve this issue with the help of example https://python.langchain.com/v0.1/docs/use_cases/question_answering/chat_history/#returning-sources