What is the request body for create_retrieval_chain(history_aware_retriever,question_answer_chain) when I deploy this chain using LangServe.

[forpr1093]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

contextualize_q_system_prompt = """Given a chat history and the latest user question \
which might reference context in the chat history, formulate a standalone question \
which can be understood without the chat history. Do NOT answer the question, \
just reformulate it if needed and otherwise return it as is."""

contextualize_q_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", contextualize_q_system_prompt),
        MessagesPlaceholder("chat_history"),
        ("human", "{input}"),
    ]
)
history_aware_retriever = create_history_aware_retriever(
    model, retriever, contextualize_q_prompt
)


qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "你是一个只会说中文的助手。在你回答问题时，请只通过{context}来回答。如果你不知道答案，就说不知道。\
            {context}"),
        MessagesPlaceholder(variable_name="chat_history"),
        ("human", "{input}"),
    ]
)

question_answer_chain = create_stuff_documents_chain(model, qa_prompt)

rag_chain = create_retrieval_chain(history_aware_retriever,question_answer_chain)

Description

I am using the example from https://python.langchain.com/v0.1/docs/use_cases/question_answering/chat_history/.
I am able to get response while using .invoke({input: input, chat_history: chat_history}).

However, when I tried to call it through http calls using the code below:

constructor(private http: HttpClient){}

getChatResposne(prompt: String, history: any[]){
	return this.http.post<any>(`${this.url}pdf/invoke`,{
		{input:{
			input: prompt,
			chat_history: history,
		},
	});
}

I received the errors below:

Traceback (most recent call last):
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 426, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\fastapi\applications.py", line 290, in __call__
    await super().__call__(scope, receive, send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\middleware\cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\middleware\cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in __call__
    raise e
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\fastapi\routing.py", line 241, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\fastapi\routing.py", line 167, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langserve\server.py", line 530, in invoke
    return await api_handler.invoke(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langserve\api_handler.py", line 833, in invoke
    output = await invoke_coro
             ^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 4439, in ainvoke
    return await self.bound.ainvoke(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 2430, in ainvoke
    input = await step.ainvoke(
            ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\passthrough.py", line 497, in ainvoke
    return await self._acall_with_config(self._ainvoke, input, config, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 1553, in _acall_with_config
    output: Output = await asyncio.create_task(coro, context=context)  # type: ignore
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\passthrough.py", line 484, in _ainvoke
    **await self.mapper.ainvoke(
      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\j\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 3066, in ainvoke
    results = await asyncio.gather(
              ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 4439, in ainvoke
    return await self.bound.ainvoke(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\branch.py", line 250, in ainvoke
    output = await runnable.ainvoke(
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 2430, in ainvoke
    input = await step.ainvoke(
            ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 3877, in ainvoke
    return await self._acall_with_config(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 1553, in _acall_with_config
    output: Output = await asyncio.create_task(coro, context=context)  # type: ignore
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 3824, in _ainvoke
    output = await acall_func_with_variable_args(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 3800, in f
    return await run_in_executor(config, func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jminiconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\config.py", line 513, in run_in_executor
    return await asyncio.get_running_loop().run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\concurrent\futures\thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Jminiconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\base.py", line 3794, in func
    return call_func_with_variable_args(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain_core\runnables\config.py", line 346, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\J\miniconda3\envs\langchain\Lib\site-packages\langchain\chains\history_aware_retriever.py", line 62, in <lambda>
    (lambda x: x["input"]) | retriever,
               ~^^^^^^^^^
KeyError: 'input'

I have tried to take a look on the swagger doc, but the request body seems to be incompleted.

Any help will be appreciated! Thank you!

System Info

aiohttp==3.9.4
aiosignal==1.3.1
altair==5.3.0
annotated-types==0.6.0
anyio==4.3.0
asgiref==3.8.1
attrs==23.2.0
backoff==2.2.1
bcrypt==4.1.2
beautifulsoup4==4.12.3
black==24.4.0
blinker==1.7.0
boto3==1.34.105
botocore==1.34.105
bs4==0.0.2
build==1.2.1
cachetools==5.3.3
certifi==2024.2.2
charset-normalizer==3.3.2
chroma-hnswlib==0.7.1
chromadb==0.4.3
click==8.1.7
colorama==0.4.6
coloredlogs==15.0.1
dataclasses-json==0.6.4
Deprecated==1.2.14
distro==1.9.0
dnspython==2.6.1
docarray==0.40.0
docstring_parser==0.16
fastapi==0.99.1
filelock==3.13.4
flatbuffers==24.3.25
frozenlist==1.4.1
fsspec==2024.3.1
gitdb==4.0.11
GitPython==3.1.43
google-ai-generativelanguage==0.6.3
google-api-core==2.18.0
google-api-python-client==2.129.0
google-auth==2.29.0
google-auth-httplib2==0.2.0
google-cloud-aiplatform==1.47.0
google-cloud-bigquery==3.20.1
google-cloud-core==2.4.1
google-cloud-resource-manager==1.12.3
google-cloud-storage==2.16.0
google-crc32c==1.5.0
google-generativeai==0.5.3
google-resumable-media==2.7.0
googleapis-common-protos==1.63.0
greenlet==3.0.3
grpc-google-iam-v1==0.13.0
grpcio==1.62.1
grpcio-status==1.62.1
h11==0.14.0
hnswlib==0.8.0
httpcore==1.0.5
httplib2==0.22.0
httptools==0.6.1
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.22.2
humanfriendly==10.0
idna==3.7
importlib-metadata==7.0.0
importlib_resources==6.4.0
Jinja2==3.1.3
jmespath==1.0.1
joblib==1.4.0
jsonpatch==1.33
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
kubernetes==29.0.0
langchain==0.2.1
langchain-chroma==0.1.1
langchain-cli==0.0.21
langchain-community==0.2.1
langchain-core==0.2.1
langchain-experimental==0.0.59
langchain-google-genai==1.0.5
langchain-google-vertexai==1.0.4
langchain-mongodb==0.1.3
langchain-openai==0.1.7
langchain-postgres==0.0.3
langchain-text-splitters==0.2.0
langchainhub==0.1.16
langgraph==0.0.32
langserve==0.2.1
langsmith==0.1.47
logger==1.4
markdown-it-py==3.0.0
MarkupSafe==2.1.5
marshmallow==3.21.1
mdurl==0.1.2
mmh3==4.1.0
monotonic==1.6
mpmath==1.3.0
multidict==6.0.5
mypy-extensions==1.0.0
networkx==3.3
numpy==1.26.4
oauthlib==3.2.2
onnxruntime==1.17.3
openai==1.25.1
opentelemetry-api==1.24.0
opentelemetry-exporter-otlp-proto-common==1.24.0
opentelemetry-exporter-otlp-proto-grpc==1.24.0
opentelemetry-instrumentation==0.45b0
opentelemetry-instrumentation-asgi==0.45b0
opentelemetry-instrumentation-fastapi==0.45b0
opentelemetry-proto==1.24.0
opentelemetry-sdk==1.24.0
opentelemetry-semantic-conventions==0.45b0
opentelemetry-util-http==0.45b0
orjson==3.10.0
overrides==7.7.0
packaging==23.2
pandas==2.2.2
pathspec==0.12.1
pgvector==0.2.5
pillow==10.3.0
platformdirs==4.2.0
posthog==3.5.0
proto-plus==1.23.0
protobuf==4.25.3
psycopg==3.1.18
psycopg-binary==3.1.18
psycopg-pool==3.2.1
pulsar-client==3.5.0
pyarrow==15.0.2
pyasn1==0.6.0
pyasn1_modules==0.4.0
pydantic==1.10.13
pydantic_core==2.18.1
pydeck==0.8.1b0
Pygments==2.17.2
pymongo==4.6.3
pyparsing==3.1.2
pypdf==4.2.0
PyPika==0.48.9
pyproject-toml==0.0.10
pyproject_hooks==1.0.0
pyreadline3==3.4.1
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
referencing==0.34.0
regex==2024.4.16
requests==2.31.0
requests-oauthlib==2.0.0
rich==13.7.1
rpds-py==0.18.0
rsa==4.9
s3transfer==0.10.1
safetensors==0.4.3
scikit-learn==1.4.2
scipy==1.13.0
sentence-transformers==2.6.1
shapely==2.0.3
shellingham==1.5.4
six==1.16.0
smmap==5.0.1
sniffio==1.3.1
soupsieve==2.5
SQLAlchemy==2.0.29
sse-starlette==1.8.2
starlette==0.27.0
streamlit==1.33.0
sympy==1.12
tenacity==8.2.3
threadpoolctl==3.4.0
tiktoken==0.7.0
tokenizers==0.15.2
toml==0.10.2
tomlkit==0.12.4
toolz==0.12.1
torch==2.2.2
tornado==6.4
tqdm==4.66.2
transformers==4.39.3
typer==0.9.4
types-protobuf==4.25.0.20240410
types-requests==2.31.0.20240406
typing-inspect==0.9.0
typing_extensions==4.11.0
tzdata==2024.1
uritemplate==4.1.1
urllib3==2.2.1
uvicorn==0.23.2
watchdog==4.0.0
watchfiles==0.21.0
websocket-client==1.7.0
websockets==12.0
wrapt==1.16.0
yarl==1.9.4
zipp==3.18.1

[dosubot[bot]]

Hey there, @forpr1093! I'm here to assist you with any bugs, questions, or contributions you might have. Let's work together to solve the issue you're facing.

The request body for create_retrieval_chain(history_aware_retriever, question_answer_chain) when deploying this chain using LangServe should be structured as follows:

{
    "input": "...",
    "chat_history": [...]
}

This is based on the example provided in the documentation:

chain.invoke({"input": "...", "chat_history": [...]})

The input field should contain the new question to be answered, and the chat_history field should contain the list of previous chat messages. Here is an example implementation:

from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

retriever = ...  # Your retriever
llm = ChatOpenAI()

system_prompt = (
    "Use the given context to answer the question. "
    "If you don't know the answer, say you don't know. "
    "Use three sentence maximum and keep the answer concise. "
    "Context: {context}"
)
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("human", "{input}"),
    ]
)
question_answer_chain = create_stuff_documents_chain(llm, prompt)
chain = create_retrieval_chain(retriever, question_answer_chain)

chain.invoke({"input": query})

When making an HTTP call, ensure that the request body matches the expected structure:

{
    "input": "Your question here",
    "chat_history": ["Previous message 1", "Previous message 2"]
}

This should resolve the KeyError: 'input' you encountered ^[1][2][3][4].

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[alice17]

I'm having the same problem. My code is exactly the same as the one from dosubot but I always get the same error message and the playground doesn't take any input.

[john-zermati]

Same problem here any idea ?