Passible Runnableconfig for AWS Lambda tool wrapper during runtime

[akash9771]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

I am trying to pass the sensitive information at runtime using Runnableconfig to aws lambda tool wrapper, its not getting reflected in my lambda function. do u have any example or notebook where u pass the runnableconfig to tools for any toolwrapper. i am going through below link but here its getting passed in the function defined in the notebook.

https://python.langchain.com/v0.2/docs/how_to/tool_runtime/
https://langchain-ai.github.io/langgraph/how-tos/pass-config-to-tools/

Description

I am trying to pass the sensitive information at runtime using Runnableconfig to aws lambda tool wrapper, its not getting reflected in my lambda function. do u have any example or notebook where u pass the runnableconfig to tools for any toolwrapper. i am going through below link but here its getting passed in the function defined in the notebook.

https://python.langchain.com/v0.2/docs/how_to/tool_runtime/
https://langchain-ai.github.io/langgraph/how-tos/pass-config-to-tools/

System Info

langgraph, langchain latest

[dosubot[bot]]

Hello @akash9771! I'm here to help you with any bugs, questions, or contributions. Let's work together to solve the issue you're facing.

To pass RunnableConfig to an AWS Lambda tool wrapper and ensure sensitive information is reflected in your Lambda function at runtime, you can follow this example:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableConfig
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda
from langchain_openai import ChatOpenAI
from langchain_community.callbacks import get_openai_callback
import json

def parse_or_fix(text: str, config: RunnableConfig):
    fixing_chain = (
        ChatPromptTemplate.from_template(
            "Fix the following text:\n\n```text\n{input}\n```\nError: {error}"
            " Don't narrate, just respond with the fixed data."
        )
        | ChatOpenAI()
        | StrOutputParser()
    )
    for _ in range(3):
        try:
            return json.loads(text)
        except Exception as e:
            text = fixing_chain.invoke({"input": text, "error": e}, config)
    return "Failed to parse"

with get_openai_callback() as cb:
    output = RunnableLambda(parse_or_fix).invoke(
        "{foo: bar}", {"tags": ["my-tag"], "callbacks": [cb]}
    )
    print(output)
    print(cb)

In this example, the RunnableConfig is used to pass tags and callbacks to the parse_or_fix function, which is then invoked by the RunnableLambda. This ensures that the configuration, including any sensitive information, is properly reflected in the Lambda function at runtime ^[1].

Additionally, you can use the LambdaWrapper class from the LangChain library to interact with AWS Lambda functions. Here is an example of how to set it up and use it:

from langchain_community.utilities.awslambda import LambdaWrapper

# Ensure you have boto3 installed and configured
# pip install boto3
# aws configure

lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

def lambda_handler(event, context):
    query = event.get("query", "")
    result = lambda_wrapper.run(query)
    return {
        "statusCode": 200,
        "body": json.dumps(result)
    }

# Example usage
query = "Your input query"
response = lambda_wrapper.run(query)
print(response)

In this example, the LambdaWrapper class is used to invoke an AWS Lambda function. The run method sends a query to the Lambda function and returns the result. This setup ensures that sensitive information can be securely passed and processed by the Lambda function at runtime ^[2].

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To bind multiple LambdaWrapper tools with a model in LangGraph and construct a graph, you can follow these steps:

1. Define your custom function and wrap it using RunnableLambda.
2. Use RunnableConfig to pass sensitive information.
3. Bind multiple LambdaWrapper tools with a model.

Here's an example:

from langchain_core.runnables import RunnableLambda, RunnableConfig
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from operator import itemgetter
import boto3
import json

# Define your custom function
def my_custom_function(input_data, config: RunnableConfig):
    # Use sensitive information from config
    sensitive_info = config.get("sensitive_info")
    # Your custom logic here
    return f"Processed {input_data} with {sensitive_info}"

# Wrap the custom function using RunnableLambda
lambda_wrapper = RunnableLambda(my_custom_function)

# Define the model and prompt
model = ChatOpenAI()
prompt = ChatPromptTemplate.from_template("Process this data: {data}")

# Bind multiple LambdaWrapper tools with a model
chain = {
    "data": itemgetter("input_data") | lambda_wrapper
} | prompt | model

# Pass sensitive information at runtime using RunnableConfig
config = RunnableConfig(sensitive_info="my_secret_key")

# Invoke the chain with input data and config
result = chain.invoke({"input_data": "example_data"}, config)
print(result)

In this example:

• my_custom_function is wrapped using RunnableLambda.
• RunnableConfig is used to pass sensitive information (sensitive_info).
• The chain is constructed by binding the lambda_wrapper with a model and prompt.
• The chain is invoked with input data and the configuration containing sensitive information.

Additionally, if you need to handle multiple arguments in your custom function, you can create a wrapper function that accepts a single input and unpacks it into multiple arguments, as shown below:

def _multiple_length_function(text1, text2):
    return len(text1) * len(text2)

def multiple_length_function(_dict):
    return _multiple_length_function(_dict["text1"], _dict["text2"])

This approach ensures that your custom functions can handle complex input structures while still being compatible with RunnableLambda.

For more detailed guidance on passing runtime values to tools and integrating with LangGraph, you can refer to the LangGraph how-to guide ^[1].

To use the LambdaWrapper class from LangChain for AWS Lambda integration, ensure you have the boto3 package installed and your AWS credentials configured. Here is an example of how to set up and use LambdaWrapper:

from langchain_community.utilities.awslambda import LambdaWrapper

# Configure AWS Lambda client
lambda_wrapper = LambdaWrapper(
    function_name="your_lambda_function_name"
)

# Example usage
response = lambda_wrapper.run("example_query")
print(response)

This setup allows you to invoke AWS Lambda functions and handle their responses within your LangGraph-based application ^[2].

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To ensure that the RunnableConfig values are properly passed and reflected in your AWS Lambda function when using LangGraph, you need to make sure that the configuration is correctly set up and accessed within your Lambda function. Here's how you can achieve this:

1. Define your Lambda function to accept and process the configuration values.
2. Ensure the RunnableConfig is passed correctly when invoking the Lambda function.

Here's an example of how you can set this up:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

import json

def lambda_handler(event, context):
    # Access the configuration values from the event
    client_id = event.get("client_id")
    x_vsl_client_id = event.get("x_vsl_client_id")
    bearer_token = event.get("bearer_token")
    correlation_id = event.get("correlation_id")
    
    # Your custom logic here
    response = {
        "client_id": client_id,
        "x_vsl_client_id": x_vsl_client_id,
        "bearer_token": bearer_token,
        "correlation_id": correlation_id,
        "message": "Configuration values received successfully"
    }
    
    return {
        "statusCode": 200,
        "body": json.dumps(response)
    }

Step 2: Pass RunnableConfig When Invoking the Lambda Function

Ensure that the RunnableConfig is correctly passed when invoking the Lambda function using LangGraph.

from langchain_core.runnables import RunnableLambda, RunnableConfig
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the model and prompt (if needed)
# model = ChatOpenAI()
# prompt = ChatPromptTemplate.from_template("Process this data: {data}")

# Bind the LambdaWrapper tool with a model (if needed)
# chain = {"data": itemgetter("input_data") | lambda_runnable} | prompt | model

# Pass sensitive information at runtime using RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the configuration
for output in lambda_runnable.stream(inputs, config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• RunnableLambda: The lambda_wrapper.run method is wrapped using RunnableLambda.
• RunnableConfig: The configuration values (client_id, x_vsl_client_id, bearer_token, correlation_id) are passed using RunnableConfig.
• Invocation: The lambda_runnable.stream method is used to invoke the Lambda function with the configuration values.

This setup ensures that the configuration values are properly passed and reflected in your AWS Lambda function at runtime.

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To ensure that the RunnableConfig values are properly passed and reflected in your AWS Lambda function, especially when using Pydantic models, you need to make sure that the configuration is correctly set up and accessed within your Lambda function. Here’s how you can achieve this:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

from pydantic import BaseModel, Field
from typing import List, Union
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

class RunnableConfig(BaseModel):
    client_id: str
    x_vsl_client_id: str
    bearer_token: str
    correlation_id: str

class InputPayload(BaseModel):
    user_query: str = Field(..., description="The query provided by the user.")
    config: RunnableConfig

class SuccessPayload(BaseModel):
    summary: List[str]

class OutputPayload(BaseModel):
    success: bool
    message: str
    detail: Union[SuccessPayload, str]

def lambda_handler(event: dict, context) -> dict:
    try:
        input_payload = InputPayload(**event)
        config = input_payload.config
        logger.info(f"Event: {input_payload}")
        logger.info(f"CONFIG IS: {config}")

        to_be_sent = [
            "Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline",
            "Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023"
        ]

        return OutputPayload(success=True, message="Pfizer summary retrieved successfully in details!", detail=SuccessPayload(summary=to_be_sent)).dict()
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return OutputPayload(success=False, message="Summary retrieval failed", detail=f"Error: {str(e)}").dict()

Step 2: Pass RunnableConfig When Invoking the Lambda Function

Ensure that the RunnableConfig is correctly passed when invoking the Lambda function using LangGraph.

from langchain_core.runnables import RunnableLambda, RunnableConfig
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the model and prompt (if needed)
# model = ChatOpenAI()
# prompt = ChatPromptTemplate.from_template("Process this data: {data}")

# Bind the LambdaWrapper tool with a model (if needed)
# chain = {"data": itemgetter("input_data") | lambda_runnable} | prompt | model

# Pass sensitive information at runtime using RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the configuration
for output in lambda_runnable.stream(inputs, config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• RunnableLambda: The lambda_wrapper.run method is wrapped using RunnableLambda.
• RunnableConfig: The configuration values (client_id, x_vsl_client_id, bearer_token, correlation_id) are passed using RunnableConfig.
• Invocation: The lambda_runnable.stream method is used to invoke the Lambda function with the configuration values.

This setup ensures that the configuration values are properly passed and reflected in your AWS Lambda function at runtime. Make sure that the RunnableConfig is correctly structured and passed as part of the event payload to the Lambda function.

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To ensure that the RunnableConfig values are passed as-is and not modified by the LLM, you can use the ensure_config function. This function ensures that the configuration is a dictionary with all keys present and does not modify the values provided.

Here is the relevant code snippet:

def ensure_config(config: Optional[RunnableConfig] = None) -> RunnableConfig:
    """Ensure that a config is a dict with all keys present.

    Args:
        config (Optional[RunnableConfig], optional): The config to ensure.
          Defaults to None.

    Returns:
        RunnableConfig: The ensured config.
    """
    empty = RunnableConfig(
        tags=[],
        metadata={},
        callbacks=None,
        recursion_limit=25,
    )
    if var_config := var_child_runnable_config.get():
        empty.update(
            cast(RunnableConfig, {k: v for k, v in var_config.items() if v is not None})
        )
    if config is not None:
        empty.update(
            cast(RunnableConfig, {k: v for k, v in config.items() if v is not None})
        )
    for key, value in empty.get("configurable", {}).items():
        if isinstance(value, (str, int, float, bool)) and key not in empty["metadata"]:
            empty["metadata"][key] = value
    return empty

This function ensures that the RunnableConfig values are passed as-is by updating the default configuration with the provided values without altering them ^[1].

Additionally, you can verify the configuration schema using the config_schema method, as shown in the test_configurable_fields function from the LangChain unit tests:

def test_configurable_fields() -> None:
    fake_llm = FakeListLLM(responses=["a"])  # str -> List[List[str]]

    assert fake_llm.invoke("...") == "a"

    fake_llm_configurable = fake_llm.configurable_fields(
        responses=ConfigurableField(
            id="llm_responses",
            name="LLM Responses",
            description="A list of fake responses for this LLM",
        )
    )

    assert fake_llm_configurable.invoke("...") == "a"

    assert fake_llm_configurable.config_schema().schema() == {
        "title": "RunnableConfigurableFieldsConfig",
        "type": "object",
        "properties": {"configurable": {"$ref": "#/definitions/Configurable"}},
        "definitions": {
            "Configurable": {
                "title": "Configurable",
                "type": "object",
                "properties": {
                    "llm_responses": {
                        "title": "LLM Responses",
                        "description": "A list of fake responses for this LLM",
                        "default": ["a"],
                        "type": "array",
                        "items": {"type": "string"},
                    }
                },
            }
        },
    }

This test ensures that the configuration schema is correctly defined and that the values are passed as-is ^[2].

Furthermore, the DynamicRunnable class in LangChain provides methods to handle configurations dynamically. The with_config method in this class uses ensure_config to merge and ensure the configuration:

def with_config(
    self,
    config: Optional[RunnableConfig] = None,
    **kwargs: Any,
) -> Runnable[Input, Output]:
    return self.__class__(
        **{**self.__dict__, "config": ensure_config(merge_configs(config, kwargs))}  # type: ignore[arg-type]
    )

This method ensures that the configuration values are passed as-is by merging the provided configuration with any additional keyword arguments and ensuring the final configuration using ensure_config ^[3].

By using these methods, you can ensure that the RunnableConfig values are passed as-is and not modified by the LLM.

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To integrate the ensure_config function with your LangGraph setup and ensure that the RunnableConfig values are passed as-is and not modified by the LLM, you can follow these steps:

1. Define your Lambda function to accept and process the configuration values.
2. Ensure the RunnableConfig is passed correctly when invoking the Lambda function.
3. Use the ensure_config function to validate and complete the configuration before passing it to the LLM.

Here's how you can achieve this:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

from pydantic import BaseModel, Field
from typing import List, Union
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

class RunnableConfig(BaseModel):
    client_id: str
    x_vsl_client_id: str
    bearer_token: str
    correlation_id: str

class InputPayload(BaseModel):
    user_query: str = Field(..., description="The query provided by the user.")
    config: RunnableConfig

class SuccessPayload(BaseModel):
    summary: List[str]

class OutputPayload(BaseModel):
    success: bool
    message: str
    detail: Union[SuccessPayload, str]

def lambda_handler(event: dict, context) -> dict:
    try:
        input_payload = InputPayload(**event)
        config = input_payload.config
        logger.info(f"Event: {input_payload}")
        logger.info(f"CONFIG IS: {config}")

        to_be_sent = [
            "Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline",
            "Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023"
        ]

        return OutputPayload(success=True, message="Pfizer summary retrieved successfully in details!", detail=SuccessPayload(summary=to_be_sent)).dict()
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return OutputPayload(success=False, message="Summary retrieval failed", detail=f"Error: {str(e)}").dict()

Step 2: Use ensure_config to Validate and Complete the Configuration

Ensure that the RunnableConfig is correctly set up and validated using the ensure_config function.

from langchain_core.runnables.config import ensure_config, RunnableConfig

# Example RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Ensure the config is complete and valid
ensured_config = ensure_config(config)

Step 3: Integrate with LangGraph

Integrate the ensure_config function with your LangGraph setup to ensure that the RunnableConfig values are passed as-is.

from langchain_core.runnables import RunnableLambda
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage
from typing import List, Literal
from langgraph.graph import StateGraph, MessagesState

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the tools setup function
def setup_tools(skills: List[str]) -> List[LambdaTool]:
    tools = []
    for skill_id in skills:
        skill_config = _fetch_skill_config(skill_id)
        input_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["input_schema"])) if skill_config.get("input_schema") else None
        output_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["output_schema"])) if skill_config.get("output_schema") else None
        lambda_tool = LambdaTool(
            function_name=skill_config['lambda_arn'],
            region=skill_config['region'],
            tool_name=skill_config['name'],
            tool_description=skill_config['description'],
            input_schema=input_schema,
            output_schema=output_schema,
        )
        tools.append(lambda_tool)
    return tools

# Setup tools
tools = setup_tools(skills=skill_list)
tool_node = ToolNode(tools)

# Define the model and bind tools
llm = IAS_ChatModel(**llm_config)
model = llm.bind_tools(tools)

# Define the state graph
def should_continue(state: MessagesState) -> Literal["tools", "__end__"]:
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "tools"
    return "__end__"

def call_model(state: MessagesState):
    messages = state["messages"]
    response = model.invoke(messages)
    return {"messages": [response]}

workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
workflow.add_edge("__start__", "agent")
workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("tools", "agent")

app = workflow.compile()

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the ensured configuration
for output in app.stream(inputs, ensured_config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• ensure_config: The ensure_config function is used to validate and complete the RunnableConfig before passing it to the LLM.
• LangGraph Integration: The RunnableConfig is ensured and passed to the LangGraph setup, ensuring that the configuration values are passed as-is and not modified by the LLM.

This setup ensures that the RunnableConfig values are properly passed and reflected in your AWS Lambda function at runtime without being modified by the LLM.

To continue talking to Dosu, mention @dosu.

[akash9771]

@dosu I have used ensured_config but still same issue, PFB LOGS. still LLM is populating

ugust 9, 2024 > 22:36:36 | INFO | utils.ias_openai_langchain:ias_openai_chat_completion_with_tools:308 | Calling chat completion endpoint with tools 
August 9, 2024 > 22:36:38 | INFO | utils.ias_openai_langchain:ias_openai_chat_completion_with_tools:312 | Received response from llm 
Output from node 'agent':
---
{'messages': [AIMessage(content='', additional_kwargs={'tool_calls': [{'function': {'arguments': '{\n  "user_query": "J&J revenue",\n  "config": {\n    "client_id": "my_client_id",\n    "x_vsl_client_id": "my_x_vsl_client_id",\n    "bearer_token": "my_bearer_token",\n    "correlation_id": "my_correlation_id"\n  }\n}', 'name': 'get_summary_api'}, 'id': 'call_uiZJqlskPAM4EDSkGOSAOhLw', 'type': 'function'}]}, response_metadata={'token_usage': 0, 'model_name': 'gpt-35-turbo-16k', 'finish_reason': 'stop'}, id='run-ce94485e-9b4c-4d19-9a2b-b41a23f14687-0', tool_calls=[{'name': 'get_summary_api', 'args': {'user_query': 'J&J revenue', 'config': {'client_id': 'my_client_id', 'x_vsl_client_id': 'my_x_vsl_client_id', 'bearer_token': 'my_bearer_token', 'correlation_id': 'my_correlation_id'}}, 'id': 'call_uiZJqlskPAM4EDSkGOSAOhLw', 'type': 'tool_call'}])]}

---

Running with state: {'user_query': 'J&J revenue', 'config': RunnableConfig(client_id='my_client_id', x_vsl_client_id='my_x_vsl_client_id', bearer_token='my_bearer_token', correlation_id='my_correlation_id')}
Validated input data: {'user_query': 'J&J revenue', 'config': {'client_id': 'my_client_id', 'x_vsl_client_id': 'my_x_vsl_client_id', 'bearer_token': 'my_bearer_token', 'correlation_id': 'my_correlation_id'}}

Tool Invocation ------------------------- get_summary_api
State: {'user_query': 'J&J revenue', 'config': {'client_id': 'my_client_id', 'x_vsl_client_id': 'my_x_vsl_client_id', 'bearer_token': 'my_bearer_token', 'correlation_id': 'my_correlation_id'}}
Input payload for get_summary_api
Basic payload: {'user_query': 'J&J revenue', 'config': {'client_id': 'my_client_id', 'x_vsl_client_id': 'my_x_vsl_client_id', 'bearer_token': 'my_bearer_token', 'correlation_id': 'my_correlation_id'}}
ITS COMING TO INPUT BRANCH {'user_query': 'J&J revenue', 'config': {'client_id': 'my_client_id', 'x_vsl_client_id': 'my_x_vsl_client_id', 'bearer_token': 'my_bearer_token', 'correlation_id': 'my_correlation_id'}}
Returning payload!!! 

FINAL PAYLOAD TO INVOKE LAMBDA======================================= {'user_query': 'J&J revenue', 'config': {'client_id': 'my_client_id', 'x_vsl_client_id': 'my_x_vsl_client_id', 'bearer_token': 'my_bearer_token', 'correlation_id': 'my_correlation_id'}}
Output payload for get_summary_api
Basic payload: {'success': True, 'message': 'Pfizer summary retrieved successfully in details!', 'detail': {'summary': ['Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline', 'Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023']}}
{'success': True, 'message': 'Pfizer summary retrieved successfully in details!', 'detail': {'summary': ['Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline', 'Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023']}}
Returning payload!!!

CODE IM USING

# Ensure the config is complete and valid
from langchain_core.runnables.config import RunnableConfig
from langchain_core.runnables.config import ensure_config, RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)
ensured_config = ensure_config(config)
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the configuration
for output in app.stream(inputs, ensured_config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

[dosubot[bot]]

To ensure that the RunnableConfig values are passed as-is and not modified by the LLM, you need to make sure that the configuration is directly passed to the Lambda function without being altered by the LLM. Here’s how you can achieve this:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

from pydantic import BaseModel, Field
from typing import List, Union
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

class RunnableConfig(BaseModel):
    client_id: str
    x_vsl_client_id: str
    bearer_token: str
    correlation_id: str

class InputPayload(BaseModel):
    user_query: str = Field(..., description="The query provided by the user.")
    config: RunnableConfig

class SuccessPayload(BaseModel):
    summary: List[str]

class OutputPayload(BaseModel):
    success: bool
    message: str
    detail: Union[SuccessPayload, str]

def lambda_handler(event: dict, context) -> dict:
    try:
        input_payload = InputPayload(**event)
        config = input_payload.config
        logger.info(f"Event: {input_payload}")
        logger.info(f"CONFIG IS: {config}")

        to_be_sent = [
            "Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline",
            "Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023"
        ]

        return OutputPayload(success=True, message="Pfizer summary retrieved successfully in details!", detail=SuccessPayload(summary=to_be_sent)).dict()
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return OutputPayload(success=False, message="Summary retrieval failed", detail=f"Error: {str(e)}").dict()

Step 2: Use ensure_config to Validate and Complete the Configuration

Ensure that the RunnableConfig is correctly set up and validated using the ensure_config function.

from langchain_core.runnables.config import ensure_config, RunnableConfig

# Example RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Ensure the config is complete and valid
ensured_config = ensure_config(config)

Step 3: Integrate with LangGraph

Integrate the ensure_config function with your LangGraph setup to ensure that the RunnableConfig values are passed as-is.

from langchain_core.runnables import RunnableLambda
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage
from typing import List, Literal
from langgraph.graph import StateGraph, MessagesState

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the tools setup function
def setup_tools(skills: List[str]) -> List[LambdaTool]:
    tools = []
    for skill_id in skills:
        skill_config = _fetch_skill_config(skill_id)
        input_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["input_schema"])) if skill_config.get("input_schema") else None
        output_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["output_schema"])) if skill_config.get("output_schema") else None
        lambda_tool = LambdaTool(
            function_name=skill_config['lambda_arn'],
            region=skill_config['region'],
            tool_name=skill_config['name'],
            tool_description=skill_config['description'],
            input_schema=input_schema,
            output_schema=output_schema,
        )
        tools.append(lambda_tool)
    return tools

# Setup tools
tools = setup_tools(skills=skill_list)
tool_node = ToolNode(tools)

# Define the model and bind tools
llm = IAS_ChatModel(**llm_config)
model = llm.bind_tools(tools)

# Define the state graph
def should_continue(state: MessagesState) -> Literal["tools", "__end__"]:
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "tools"
    return "__end__"

def call_model(state: MessagesState):
    messages = state["messages"]
    response = model.invoke(messages)
    return {"messages": [response]}

workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
workflow.add_edge("__start__", "agent")
workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("tools", "agent")

app = workflow.compile()

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the ensured configuration
for output in app.stream(inputs, ensured_config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• ensure_config: The ensure_config function is used to validate and complete the RunnableConfig before passing it to the LLM.
• LangGraph Integration: The RunnableConfig is ensured and passed to the LangGraph setup, ensuring that the configuration values are passed as-is and not modified by the LLM.

This setup ensures that the RunnableConfig values are properly passed and reflected in your AWS Lambda function at runtime without being modified by the LLM.

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To ensure that the RunnableConfig values are passed as-is and not modified by the LLM, you need to make sure that the configuration is directly passed to the Lambda function without being altered by the LLM. Here’s how you can achieve this:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

from pydantic import BaseModel, Field
from typing import List, Union
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

class RunnableConfig(BaseModel):
    client_id: str
    x_vsl_client_id: str
    bearer_token: str
    correlation_id: str

class InputPayload(BaseModel):
    user_query: str = Field(..., description="The query provided by the user.")
    config: RunnableConfig

class SuccessPayload(BaseModel):
    summary: List[str]

class OutputPayload(BaseModel):
    success: bool
    message: str
    detail: Union[SuccessPayload, str]

def lambda_handler(event: dict, context) -> dict:
    try:
        input_payload = InputPayload(**event)
        config = input_payload.config
        logger.info(f"Event: {input_payload}")
        logger.info(f"CONFIG IS: {config}")

        to_be_sent = [
            "Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline",
            "Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023"
        ]

        return OutputPayload(success=True, message="Pfizer summary retrieved successfully in details!", detail=SuccessPayload(summary=to_be_sent)).dict()
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return OutputPayload(success=False, message="Summary retrieval failed", detail=f"Error: {str(e)}").dict()

Step 2: Use ensure_config to Validate and Complete the Configuration

Ensure that the RunnableConfig is correctly set up and validated using the ensure_config function.

from langchain_core.runnables.config import ensure_config, RunnableConfig

# Example RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Ensure the config is complete and valid
ensured_config = ensure_config(config)

Step 3: Integrate with LangGraph

Integrate the ensure_config function with your LangGraph setup to ensure that the RunnableConfig values are passed as-is.

from langchain_core.runnables import RunnableLambda
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage
from typing import List, Literal
from langgraph.graph import StateGraph, MessagesState

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the tools setup function
def setup_tools(skills: List[str]) -> List[LambdaTool]:
    tools = []
    for skill_id in skills:
        skill_config = _fetch_skill_config(skill_id)
        input_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["input_schema"])) if skill_config.get("input_schema") else None
        output_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["output_schema"])) if skill_config.get("output_schema") else None
        lambda_tool = LambdaTool(
            function_name=skill_config['lambda_arn'],
            region=skill_config['region'],
            tool_name=skill_config['name'],
            tool_description=skill_config['description'],
            input_schema=input_schema,
            output_schema=output_schema,
        )
        tools.append(lambda_tool)
    return tools

# Setup tools
tools = setup_tools(skills=skill_list)
tool_node = ToolNode(tools)

# Define the model and bind tools
llm = IAS_ChatModel(**llm_config)
model = llm.bind_tools(tools)

# Define the state graph
def should_continue(state: MessagesState) -> Literal["tools", "__end__"]:
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "tools"
    return "__end__"

def call_model(state: MessagesState):
    messages = state["messages"]
    response = model.invoke(messages)
    return {"messages": [response]}

workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
workflow.add_edge("__start__", "agent")
workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("tools", "agent")

app = workflow.compile()

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the ensured configuration
for output in app.stream(inputs, ensured_config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• ensure_config: The ensure_config function is used to validate and complete the RunnableConfig before passing it to the LLM.
• LangGraph Integration: The RunnableConfig is ensured and passed to the LangGraph setup, ensuring that the configuration values are passed as-is and not modified by the LLM.

This setup ensures that the RunnableConfig values are properly passed and reflected in your AWS Lambda function at runtime without being modified by the LLM.

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To ensure that the RunnableConfig values are passed as-is and not modified by the LLM, you need to avoid passing the RunnableConfig directly to the LLM. Instead, use the RunnableConfig only for configuration purposes such as callbacks and tags, and keep the actual data separate. Here’s how you can achieve this:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

from pydantic import BaseModel, Field
from typing import List, Union
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

class RunnableConfig(BaseModel):
    client_id: str
    x_vsl_client_id: str
    bearer_token: str
    correlation_id: str

class InputPayload(BaseModel):
    user_query: str = Field(..., description="The query provided by the user.")
    config: RunnableConfig

class SuccessPayload(BaseModel):
    summary: List[str]

class OutputPayload(BaseModel):
    success: bool
    message: str
    detail: Union[SuccessPayload, str]

def lambda_handler(event: dict, context) -> dict:
    try:
        input_payload = InputPayload(**event)
        config = input_payload.config
        logger.info(f"Event: {input_payload}")
        logger.info(f"CONFIG IS: {config}")

        to_be_sent = [
            "Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline",
            "Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023"
        ]

        return OutputPayload(success=True, message="Pfizer summary retrieved successfully in details!", detail=SuccessPayload(summary=to_be_sent)).dict()
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return OutputPayload(success=False, message="Summary retrieval failed", detail=f"Error: {str(e)}").dict()

Step 2: Use ensure_config to Validate and Complete the Configuration

Ensure that the RunnableConfig is correctly set up and validated using the ensure_config function.

from langchain_core.runnables.config import ensure_config, RunnableConfig

# Example RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Ensure the config is complete and valid
ensured_config = ensure_config(config)

Step 3: Integrate with LangGraph

Integrate the ensure_config function with your LangGraph setup to ensure that the RunnableConfig values are passed as-is.

from langchain_core.runnables import RunnableLambda
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage
from typing import List, Literal
from langgraph.graph import StateGraph, MessagesState

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the tools setup function
def setup_tools(skills: List[str]) -> List[LambdaTool]:
    tools = []
    for skill_id in skills:
        skill_config = _fetch_skill_config(skill_id)
        input_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["input_schema"])) if skill_config.get("input_schema") else None
        output_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["output_schema"])) if skill_config.get("output_schema") else None
        lambda_tool = LambdaTool(
            function_name=skill_config['lambda_arn'],
            region=skill_config['region'],
            tool_name=skill_config['name'],
            tool_description=skill_config['description'],
            input_schema=input_schema,
            output_schema=output_schema,
        )
        tools.append(lambda_tool)
    return tools

# Setup tools
tools = setup_tools(skills=skill_list)
tool_node = ToolNode(tools)

# Define the model and bind tools
llm = IAS_ChatModel(**llm_config)
model = llm.bind_tools(tools)

# Define the state graph
def should_continue(state: MessagesState) -> Literal["tools", "__end__"]:
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "tools"
    return "__end__"

def call_model(state: MessagesState):
    messages = state["messages"]
    response = model.invoke(messages)
    return {"messages": [response]}

workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
workflow.add_edge("__start__", "agent")
workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("tools", "agent")

app = workflow.compile()

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the ensured configuration
for output in app.stream(inputs, ensured_config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• ensure_config: The ensure_config function is used to validate and complete the RunnableConfig before passing it to the LLM.
• LangGraph Integration: The RunnableConfig is ensured and passed to the LangGraph setup, ensuring that the configuration values are passed as-is and not modified by the LLM.

This setup ensures that the RunnableConfig values are properly passed and reflected in your AWS Lambda function at runtime without being modified by the LLM.

To continue talking to Dosu, mention @dosu.

[akash9771]

[dosubot[bot]]

To ensure that the RunnableConfig values are passed as-is and not modified by the LLM, you need to avoid passing the RunnableConfig directly to the LLM. Instead, use the RunnableConfig only for configuration purposes such as callbacks and tags, and keep the actual data separate. Here’s how you can achieve this:

Step 1: Define Your Lambda Function

Ensure your Lambda function is set up to accept and process the configuration values passed via RunnableConfig.

from pydantic import BaseModel, Field
from typing import List, Union
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

class RunnableConfig(BaseModel):
    client_id: str
    x_vsl_client_id: str
    bearer_token: str
    correlation_id: str

class InputPayload(BaseModel):
    user_query: str = Field(..., description="The query provided by the user.")
    config: RunnableConfig

class SuccessPayload(BaseModel):
    summary: List[str]

class OutputPayload(BaseModel):
    success: bool
    message: str
    detail: Union[SuccessPayload, str]

def lambda_handler(event: dict, context) -> dict:
    try:
        input_payload = InputPayload(**event)
        config = input_payload.config
        logger.info(f"Event: {input_payload}")
        logger.info(f"CONFIG IS: {config}")

        to_be_sent = [
            "Pfizer Full-year 2023 revenues were $58.5 billion, reflecting a 41 operational decline due primarily to the expected decline in COVID-19 revenues, while fourth-quarter revenues were $14.2 billion, reflecting a 42 operational decline",
            "Johnson & Johnson revenue for the twelve months ending June 30, 2024 was $86.576B, a 15.5 increase year-over-year. Johnson & Johnson annual revenue for 2023"
        ]

        return OutputPayload(success=True, message="Pfizer summary retrieved successfully in details!", detail=SuccessPayload(summary=to_be_sent)).dict()
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return OutputPayload(success=False, message="Summary retrieval failed", detail=f"Error: {str(e)}").dict()

Step 2: Use ensure_config to Validate and Complete the Configuration

Ensure that the RunnableConfig is correctly set up and validated using the ensure_config function.

from langchain_core.runnables.config import ensure_config, RunnableConfig

# Example RunnableConfig
config = RunnableConfig(
    client_id="dummy",
    x_vsl_client_id="dummy",
    bearer_token="dummy",
    correlation_id="something"
)

# Ensure the config is complete and valid
ensured_config = ensure_config(config)

Step 3: Integrate with LangGraph

Integrate the ensure_config function with your LangGraph setup to ensure that the RunnableConfig values are passed as-is.

from langchain_core.runnables import RunnableLambda
from langchain_community.utilities.awslambda import LambdaWrapper
from langgraph.prebuilt import ToolNode
from langchain.schema import HumanMessage
from typing import List, Literal
from langgraph.graph import StateGraph, MessagesState

# Define your LambdaWrapper
lambda_wrapper = LambdaWrapper(function_name="your_lambda_function_name")

# Wrap the Lambda function using RunnableLambda
lambda_runnable = RunnableLambda(lambda_wrapper.run)

# Define the tools setup function
def setup_tools(skills: List[str]) -> List[LambdaTool]:
    tools = []
    for skill_id in skills:
        skill_config = _fetch_skill_config(skill_id)
        input_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["input_schema"])) if skill_config.get("input_schema") else None
        output_schema = generate_pydantic_model_from_json_schema(json.dumps(skill_config["output_schema"])) if skill_config.get("output_schema") else None
        lambda_tool = LambdaTool(
            function_name=skill_config['lambda_arn'],
            region=skill_config['region'],
            tool_name=skill_config['name'],
            tool_description=skill_config['description'],
            input_schema=input_schema,
            output_schema=output_schema,
        )
        tools.append(lambda_tool)
    return tools

# Setup tools
tools = setup_tools(skills=skill_list)
tool_node = ToolNode(tools)

# Define the model and bind tools
llm = IAS_ChatModel(**llm_config)
model = llm.bind_tools(tools)

# Define the state graph
def should_continue(state: MessagesState) -> Literal["tools", "__end__"]:
    messages = state["messages"]
    last_message = messages[-1]
    if last_message.tool_calls:
        return "tools"
    return "__end__"

def call_model(state: MessagesState):
    messages = state["messages"]
    response = model.invoke(messages)
    return {"messages": [response]}

workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
workflow.add_edge("__start__", "agent")
workflow.add_conditional_edges("agent", should_continue)
workflow.add_edge("tools", "agent")

app = workflow.compile()

# Example input
inputs = {"messages": [HumanMessage(content="wht is j&j revenue use summary tool")]}

# Invoke the Lambda function with the ensured configuration
for output in app.stream(inputs, ensured_config):
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Explanation

• Lambda Function: The Lambda function (lambda_handler) is designed to accept configuration values from the event and process them.
• ensure_config: The ensure_config function is used to validate and complete the RunnableConfig before passing it to the LLM.
• LangGraph Integration: The RunnableConfig is ensured and passed to the LangGraph setup, ensuring that the configuration values are passed as-is and not modified by the LLM.

This setup ensures that the RunnableConfig values are properly passed and reflected in your AWS Lambda function at runtime without being modified by the LLM.

If this approach still doesn't work, it might be necessary to involve a maintainer to further investigate the issue. This is a tough nut to crack, could really use your help on this, @eyurtsev.

To continue talking to Dosu, mention @dosu.

[tyadav7]

@dosu have a similar kind of problem but i have used langserve and

add_route(RunnableLambda(my_func), path=“myfunc”)

def my_func(input:str, config: RunnableConfig):

how can i send runnable config in request body?

[akash9771]

Team can u please delete this issue. this contains some private variable in code

[akash9771]

@dosu Team can u please delete this issue. this contains some private variable in code

[dosubot[bot]]

A maintainer has already been looped in.

To continue talking to Dosu, mention @dosu.

[akash9771]

@dosu Team is it delete?, please delete this ASAP

[dosubot[bot]]

A maintainer has already been looped in.

To continue talking to Dosu, mention @dosu.

[tyadav7]

@dosu have a similar kind of problem but i have used langserve and

add_route(RunnableLambda(my_func), path=“myfunc”)

def my_func(input:str, config: RunnableConfig):

how can i send runnable config in request body?