RunnableWithMessageHistory where input and output are Messages produces error

[sfitts]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from dotenv import load_dotenv
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.messages import HumanMessage
from langchain_core.runnables import RunnableParallel
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_openai import ChatOpenAI

store = {}

load_dotenv()


def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]


with_message_history = RunnableWithMessageHistory(
    ChatOpenAI(),
    get_session_history,
)

history = get_session_history("abc123")
print (history.messages)

result = with_message_history.invoke(
    HumanMessage(content="What did Simone de Beauvoir believe about free will"),
    config={"configurable": {"session_id": "abc123"}},
)
print (result)

print (history.messages)

Description

Based on the documentation found here https://python.langchain.com/docs/expression_language/how_to/message_history#messages-input-messages-output, I believe that the example code should correctly submit the prompt and add the prompt and subsequent result to the history. However, when the code is run I get:

[]
content="Simone de Beauvoir believed that individuals have free will and the ability to make choices that shape their lives. She argued that humans are not determined by outside forces or fate, but rather have the capacity to create their own meaning and purpose through their actions. De Beauvoir believed that individuals are responsible for their choices and must take ownership of their lives, even in the face of societal norms or constraints. She emphasized the importance of personal agency and autonomy in shaping one's destiny." response_metadata={'token_usage': {'completion_tokens': 95, 'prompt_tokens': 17, 'total_tokens': 112}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_3bc1b5746c', 'finish_reason': 'stop', 'logprobs': None}
[]
Error in RootListenersTracer.on_llm_end callback: KeyError('input')

Looking at the code in _exit_history (history.py, line 422) I can see that is expecting the inputs from run to be a dict with the key "input". However, at that point the key is "prompts" (presumably set by the OpenAI code, I haven't dug into that yet). The only way to change the key is it looking for would be to set input_messages_key, but doing that would require that the initial input is a dict as well, which I don't want.

This would seem to be either a documentation issue (though I don't see a way to make it work at all), a misunderstanding on my part (though I did get the other options given to work) or a bug in either RunnableWithHistory or ChatOpenAI. Just not sure which ;).

TL;DR; is there a way to make message history work when the input and output are a BaseMessage?

System Info

System Information

OS: Windows
OS Version: 10.0.19045
Python Version: 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]

Package Information

langchain_core: 0.1.33
langchain: 0.1.13
langchain_community: 0.0.29
langsmith: 0.1.31
langchain_openai: 0.1.1
langchain_text_splitters: 0.0.1
langserve: 0.0.51

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph

[dosubot[bot]]

Hey @sfitts, great to see you diving into another challenge! I remember our last interaction; let's see how we can tackle this one together.

Based on the code and error message you've provided, it seems like there's a mismatch between the expected key for the input messages in RunnableWithMessageHistory and the actual key in your input.

In the RunnableWithMessageHistory class, the input_messages_key parameter is used to specify the key in the input dictionary that contains the messages. By default, this key is set to "input". However, in your case, it seems like the key is "prompts".

To resolve this issue, you can specify the correct key when instantiating RunnableWithMessageHistory. Here's how you can do it:

with_message_history = RunnableWithMessageHistory(
    ChatOpenAI(),
    get_session_history,
    input_messages_key="prompts",
)

In this adjusted code, input_messages_key="prompts" tells RunnableWithMessageHistory to look for the input messages under the key "prompts" instead of "input".

However, please note that this solution assumes that "prompts" is the correct key for your input messages. If the key is different or if there are other issues in your code, you might need to adjust the solution accordingly.

For more information on how RunnableWithMessageHistory works, you can refer to the unit tests for RunnableWithMessageHistory in the LangChain repository. These tests provide examples of how RunnableWithMessageHistory can be used with different types of input.

I hope this helps! If you have any other questions or if something is not clear, please let me know.

Sources

• libs/core/tests/unit_tests/runnables/test_history.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[sfitts]

Unfortunately, that does not work (as I mentioned in my initial question). This is because input_messages_key doesn't just dictate where the output is found, it also dictates how the input is formatted. Using this forces the input the be a dict and what I want is for the input to be a Message.

Using the suggested change results in the following failure:

[]
Traceback (most recent call last):
  File "C:\Users\sfitts\GitHub\ag2rs\aimanager\src\main\python\convo_test3.py", line 29, in <module>
    result = with_message_history.invoke(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\base.py", line 4321, in invoke
    return self.bound.invoke(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\base.py", line 4321, in invoke
    return self.bound.invoke(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\base.py", line 2309, in invoke
    input = step.invoke(
            ^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\base.py", line 4321, in invoke
    return self.bound.invoke(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\passthrough.py", line 454, in invoke
    return self._call_with_config(self._invoke, input, config, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\base.py", line 1488, in _call_with_config
    context.run(
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\config.py", line 347, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\sfitts\.virtualenvs\aimanager\Lib\site-packages\langchain_core\runnables\passthrough.py", line 435, in _invoke
    assert isinstance(
AssertionError: The input to RunnablePassthrough.assign() must be a dict.

[sfitts]

The use of prompts as the key for the inputs comes from on_llm_start (line 254 of tracers/base.py). OTOH, when using RunnablePassthrough to wrap things, the Run object comes from on_chain_start and that uses the expected input key (after calling up to the chain itself). For now, I think we'll just build a custom version of RunnableWithHistory to handle this. This will also let us address the fact that _aenter_history calls a sync method to get the message history instead of using the async aget_messages method.

[maximeperrindev]

Hi @sfitts,

You should define the input key and then pass a dict in the invoke method

from dotenv import load_dotenv
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.messages import HumanMessage
from langchain_core.runnables import RunnableParallel
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_openai import ChatOpenAI

store = {}

load_dotenv()


def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]


with_message_history = RunnableWithMessageHistory(
    ChatOpenAI(),
    get_session_history,
    input_messages_key="question",
)

history = get_session_history("abc123")
print (history.messages)

result = with_message_history.invoke(
   {"question":"What did Simone de Beauvoir believe about free will"},
    config={"configurable": {"session_id": "abc123"}},
)
print (result)

print (history.messages)

[sfitts]

The documentation indicates that it should be possible to pass a Message and not a dict. Is this not actually possible? For my use case I don't want to pass a dict.

[karthikbharadhwajKB]

Hello @maximeperrindev, I tried your solution, but it's throwing this exception:
It says it only accepts PromptValue or list of BaseMessages but not a dictionary object.
ValueError: Invalid input type <class 'dict'>. Must be a PromptValue, str, or list of BaseMessages

[]
Unexpected exception formatting exception. Falling back to standard exception
Traceback (most recent call last):
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\IPython\core\interactiveshell.py", line 3553, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\Users\karthik.kolluri\AppData\Local\Temp\ipykernel_29972\3094225409.py", line 29, in
result = with_message_history.invoke(
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\langchain_core\runnables\base.py", line 4511, in invoke
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\langchain_core\runnables\base.py", line 4511, in invoke
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\langchain_core\runnables\base.py", line 2499, in invoke
config, callbacks=run_manager.get_child(f"seq:step:{i+1}")
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\langchain_core\runnables\base.py", line 4511, in invoke
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\langchain_core\language_models\chat_models.py", line 159, in invoke
def invoke(
File "c:\Users\karthik.kolluri\AppData\Local\anaconda3\envs\llm_apps\lib\site-packages\langchain_core\language_models\chat_models.py", line 142, in _convert_input
def OutputType(self) -> Any:
ValueError: Invalid input type <class 'dict'>. Must be a PromptValue, str, or list of BaseMessages.

[ArionIII]

Hello @karthikbharadhwajKB , I'm running on the exact same issue, don't know how to get over it.
Have you found any solution to this issue ?

Thanks !

[sfitts]

@ArionIII -- after not getting much in the way of traction here we ended up creating a wrapper class to fix the issue (turns out you have to adjust behavior for both inputs and outputs). Here is that class -- https://gist.github.com/sfitts/3f71efa772348f407a68dcae445de1fb. Hope it helps.

[hiromaily]

I faced same error ValueError: Invalid input type <class 'dict'>. Must be a PromptValue, str, or list of BaseMessages. But I don't know how to fix.