OutputParserException: Parsing LLM output produced both a final answer and a parse-able action

[yadav-shivani]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from database_llmdef import *
from langchain_community.agent_toolkits import create_sql_agent

db=database()# crate and read the database
llm=CustomLLM()# Defination of CustomLLM is given below. I have created customLLM class to invoke LLM through lambda function. Lambda function has uses bedrock.invoke_model function to call Antropic claude model version 2.1 and return json output back to my code. I cannot use BedrockLLM function from Langchain as it is required for me to call LLM through lambda ONLY.

agent_executor = create_sql_agent(llm, db=db, max_iterations=30, verbose=True, handle_parsing_errors=True)
question='MY question here?'
response=agent_executor.invoke({'input':question})
print(response)


def _call(
    self,
    prompt: str,
    stop: Optional[List[str]] = None,
    run_manager: Optional[CallbackManagerForLLMRun] = None,
    **kwargs: Any,
) -> str:
    """Run the LLM on the given input.

    Override this method to implement the LLM logic.

    Args:
        prompt: The prompt to generate from.
        stop: Stop words to use when generating. Model output is cut off at the
            first occurrence of any of the stop substrings.
            If stop tokens are not supported consider raising NotImplementedError.
        run_manager: Callback manager for the run.
        **kwargs: Arbitrary additional keyword arguments. These are usually passed
            to the model provider API call.

    Returns:
        The model output as a string. Actual completions SHOULD NOT include the prompt.
    """
    #response=lambda_handler({'prompt':prompt}, context=None)
            # Example usage
    function_name = "lambda_function"# Replace with your Lambda function name
    payload = {'prompt': prompt}  # The input data you want to send to the Lambda function

    # Invoke the Lambda function and get the response
    lambda_response = invoke_lambda(function_name, payload)

    
    return lambda_response

@property
def _identifying_params(self) -> Dict[str, Any]:
    """Return a dictionary of identifying parameters."""
    return {
        # The model name allows users to specify custom token counting
        # rules in LLM monitoring applications (e.g., in LangSmith users
        # can provide per token pricing for their model and monitor
        # costs for the given LLM.)
        "model_name": "CustomChatModel",
    }

@property
def _llm_type(self) -> str:
    """Get the type of language model used by this chat model. Used for logging purposes only."""
    return "custom"

Description

1. I am trying to use create_sql_agent function from langchain to do analysis over tabular data using.
2. My requirement is to call LLM through lambda function of aws, so cannot use BedrockLLM function of langchain(as per my understanding, don't know if there is a way to use BedrockLLM with llm replace to lambda function). To handle this case, I created a customLLM class, where _call function is replaced with lambda function call(customLLM defination with code)
3. After analyzing lambda logs it seems like agent calls llm once and does follow iterative approach like it do with inbuilt BedrockLLM function. I am getting output parser error.

System Info

System Info

System Information

OS: Linux
OS Version: #1 SMP Wed Aug 7 16:53:32 UTC 2024
Python Version: 3.10.16 (main, Jan 14 2025, 02:25:17) [GCC 12.2.0]

Package Information

langchain_core: 0.3.29
langchain: 0.3.14
langchain_community: 0.3.7
langsmith: 0.1.147
langchain_aws: 0.2.7
langchain_test: Installed. No version info available.
langchain_text_splitters: 0.3.5

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.11.11
async-timeout: 4.0.3
boto3: 1.34.131
dataclasses-json: 0.6.7
httpx: 0.28.1
httpx-sse: 0.4.0
jsonpatch: 1.33
langsmith-pyo3: Installed. No version info available.
numpy: 1.26.4
orjson: 3.10.14
packaging: 24.2
pydantic: 2.9.2
pydantic-settings: 2.7.1
PyYAML: 6.0.2
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.35
tenacity: 9.0.0
typing-extensions: 4.12.2

[yadav-shivani]

I modified inbuilt bedrockLLM class instead of using customLLM. Then it worked for me.