create_pandas_dataframe_agent in Streamlit, extracting and displaying graphs

[ahadanjum]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

def extract_python_code(text):
    pattern = r'\s(.*?)'
    matches = re.findall(pattern, text, re.DOTALL)
    if not matches:
        return None
    else:
        return matches[0]

def extract_graph_code(output_code, queries, df):
    """
    Extracts and executes code snippets related to Plotly graphs from the given output code and queries.
    
    Parameters:
        - output_code (list): A list of code snippets extracted from the agent's output.
        - queries (list): A list of queries extracted from the agent's intermediate steps.
        - df (pd.DataFrame): The pandas DataFrame containing the data to be queried.

    Returns:
        None
    """ 
    plotly_pattern = re.compile(r'plotly\.(graph_objs|express).*\.show\(\)')
    fig_show_pattern = re.compile(r'fig\.show\(\)')
    plotly_graph_generated = False
    exec_context = {'df1': df1, 'df2': df2, 'df3': df3}
    
    exec_context.update({
        'pd': pd,
        'px': px,
        'sns': sns,
        'st': st
    })
    
    if queries is not None:
        for query in queries:
            if plotly_pattern.search(query) or fig_show_pattern.search(query):
                plotly_graph_generated = True
            execute_code(query, exec_context)
    if not plotly_graph_generated:
        if output_code is not None:
            for output in output_code:
                execute_code(output, exec_context)

def execute_code(code, context):
    """
    Executes the given code snippet within the specified context.

    Parameters:
        - code (str): The code snippet to be executed.
        - context (dict): The context in which the code will be executed.

    Returns:
        None

    Raises:
        Exception: If an error occurs during code execution.
    """
    try:
        print("Code executed: ", code)
        fig_show_pattern = re.compile(r'fig\.show\(\)')
        if fig_show_pattern.search(code):
                code = code.replace("fig.show()", "")
                code += """st.plotly_chart(fig, theme='streamlit', use_container_width=True)"""
        exec(code, context)
    except Exception as e:
        print(f"An error occurred during query execution: {str(e)}")
    return

def extract_queries(intermediate_steps):
    """
    Extracts queries from the intermediate steps of an agent.

    Parameters:
        intermediate_steps (list): A list of intermediate steps.

    Returns:
        list: A list of queries extracted from the intermediate steps.
    """
    queries = []
    for step in intermediate_steps:
        # Check if the step is a tuple and has at least one element
        if isinstance(step, tuple) and len(step) > 0:
            # Extract the first part of the tuple, which is assumed to be an AgentActionMessageLog object
            agent_action_log = step[0]
            # Ensure the item is an AgentActionMessageLog object by checking it has the 'tool_input' attribute
            if hasattr(agent_action_log, 'tool_input') and 'query' in agent_action_log.tool_input:
                # Append the 'query' from 'tool_input' to the list of queries
                queries.append(agent_action_log.tool_input['query'])
    return queries


if response["intermediate_steps"]:
                queries = extract_queries(response['intermediate_steps'])
        else:
            queries = None
        
        pattern = r'```python\s(.*?)```'
        code_snippets = re.findall(pattern, response["output"], re.DOTALL)
        
        if code_snippets:
                cleaned_snippets = [snippet.replace("python\n", "", 1) for snippet in code_snippets]
        else:
                cleaned_snippets = None
            
        extract_graph_code(output_code=cleaned_snippets, queries=queries, df=df)

Description

I have a create_pandas_dataframe_agent running on a streamlit application that reads 3 pandas dataframes and answers questions about the data. I want the agent to be able to display graphs in the chat as well. Currently when the agent is asked to produce a graph, it either opens it up in a new tab on my browser, displays the code in the output, or does both. From either of the three options, based on my code above, I extract the python code the agent has written and change the fig.show() line to be st.plotly_chart() and am able to get the graph in the chat window.

I want to know if there is a way for the agent to create these plotly graphs without it invoking and opening a new tab and without the code being returned in the chat itself. I haven't been able to find a solution online where this is possible. Within Jupyter Notebook, the agent can easily generate plotly graphs as it knows it is able to do so in that environment but in streamlit the agent can't. How can I get things to work like that with streamlit. If not streamlit, is there another frontend option such as Javascript or React that could work better?

System Info

System Information

OS: Windows
OS Version: 10.0.22621
Python Version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]

Package Information

langchain_core: 0.1.18
langchain: 0.1.5
langchain_community: 0.0.17
langchain_experimental: 0.0.50
langchain_openai: 0.0.5

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph
langserve

[dosubot[bot]]

🤖

Hey there, @ahadanjum! Nice to see you tackling new challenges. How's everything going on your end?

Based on your code and the context provided, it seems like you're already on the right track with using st.plotly_chart() to display Plotly graphs in Streamlit. However, to prevent the agent from opening a new tab or returning the code in the chat, you might need to modify the agent's behavior or the way it interacts with the Streamlit environment.

Currently, the agent might be using fig.show() to display the graphs, which opens a new tab in the browser. Instead, you can replace this with st.plotly_chart(fig), as you've done in your execute_code() function. This should display the graph directly in the Streamlit app without opening a new tab.

As for preventing the code from being returned in the chat, you might need to modify the agent's output handling. If the agent is currently returning the code as part of its output, you could add a step in your extract_graph_code() function to remove the code from the output before it's displayed in the chat. This could be done by filtering out the code snippets from the output_code list before it's passed to execute_code().

Regarding your question about other frontend options, both Javascript and React could potentially be used to create a more interactive chat interface. However, this would likely require a significant amount of additional development and might not be necessary if you're able to achieve your desired functionality with Streamlit.

As for how the agent generates Plotly graphs within a Jupyter Notebook environment, it's likely that the agent is able to do this because Jupyter Notebooks have built-in support for interactive outputs, including Plotly graphs. In a Jupyter Notebook, the agent can simply use fig.show() to display the graph directly in the notebook. This is different from Streamlit, which requires the use of st.plotly_chart() to display Plotly graphs.

I hope this helps! If you have any further questions or need more clarification, please let me know.

Sources

• libs/community/langchain_community/callbacks/streamlit/init.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ahadanjum]

@dosu-bot, the approach to remove/filter out the code before it is displayed is a smart approach. I do have a follow up question. Sometimes, when the agent responds with the code, it mentions how the code needs to be run in an interactive environment or in a notebook, for example. In my prompt template, I tell the agent to only return the code. How can I adjust it so that it returns the code so that I can grab it for making the streamlit plots, filter it out of the response, and make sure the response from the agent still flows in a cohesive manner?

[dosubot[bot]]

🤖

Based on the code you've shared and the context provided, it seems like you're trying to display a Plotly graph in a Streamlit application without opening a new tab or displaying the code in the chat. Here are some suggestions on how you can achieve this:

1. Modify the execute_code function: In your execute_code function, you're already replacing fig.show() with st.plotly_chart(fig, theme='streamlit', use_container_width=True). This is a good start. However, you might want to consider removing the print("Code executed: ", code) line to prevent the code from being displayed in the chat.

2. Filter out the code from the agent's response: You can modify the extract_graph_code function to filter out the code from the agent's response. One way to do this is by using regular expressions to match and remove the code snippets from the response. Here's an example of how you can do this:

def filter_code_from_response(response):
    pattern = r'```python\s(.*?)```'
    response = re.sub(pattern, '', response, flags=re.DOTALL)
    return response

You can call this function before returning the agent's response to ensure that the code is not included in the chat.

3. Handle the interactive environment issue: The agent might be mentioning that the code needs to be run in an interactive environment or in a notebook because it's trying to execute fig.show(), which opens a new tab. Since you're already replacing fig.show() with st.plotly_chart(), this issue should be resolved. However, if you're still encountering this issue, you might want to consider using a try-except block to catch and handle any exceptions related to this.

4. Ensure a cohesive flow in the agent's response: To ensure a cohesive flow in the agent's response, you can modify the extract_graph_code function to return the modified code snippets along with a flag indicating whether a graph was generated. You can then use this information to adjust the agent's response accordingly. Here's an example of how you can do this:

def extract_graph_code(output_code, queries, df):
    # Existing code...
    if not plotly_graph_generated:
        if output_code is not None:
            for output in output_code:
                execute_code(output, exec_context)
    return plotly_graph_generated, cleaned_snippets

You can then use the returned flag and code snippets to adjust the agent's response:

plotly_graph_generated, cleaned_snippets = extract_graph_code(output_code=cleaned_snippets, queries=queries, df=df)
if plotly_graph_generated:
    response["output"] = "Here's the graph you requested:\n\n" + "\n\n".join(cleaned_snippets)
else:
    response["output"] = filter_code_from_response(response["output"])

I hope this helps! If you have any further questions or run into any issues, feel free to ask.

Sources

• below code is not able to show grpahs at frontend , rather it returns code ,
• libs/community/langchain_community/callbacks/streamlit/init.py
• libs/langchain/langchain/chains/combine_documents/map_reduce.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot[bot]]

It looks like you've developed a neat solution by creating a custom tool to plot charts using Plotly within a Streamlit application. This approach is quite innovative and directly addresses the need to integrate interactive graphs into the chat interface seamlessly. Here are a few enhancements and considerations to further refine your implementation:

1. Adjust the Return Type: The plotChart function is currently annotated to return an int, but it seems like it doesn't return any value. If that's the case, changing the return type to None would more accurately reflect its behavior, ensuring clarity and correctness in your code.

2. Tool Invocation Clarity: Make sure it's clear within your agent's logic how and when the plotChart tool is invoked. The agent should call this tool specifically when it needs to plot a graph, ensuring that the tool's functionality is fully utilized whenever graph plotting is required.

3. Streamlit Callback Handler: Incorporating a StreamlitCallbackHandler could further streamline the process of displaying Plotly graphs within the Streamlit chat. This handler can help manage the inline rendering of graphs, potentially enhancing the interactivity and overall user experience of your chat interface.

4. Error Handling: Implementing robust error handling within your plotChart tool can help mitigate potential issues related to input data or the plotting process itself. This step is crucial for maintaining a smooth and error-free user experience, especially when dealing with diverse datasets or complex graph requirements.

5. Performance Optimization: For handling complex or large datasets, consider exploring Plotly's performance optimization options. Techniques such as using WebGL-based traces or converting graphs to static images for rendering can significantly improve the performance and responsiveness of your graphs within the Streamlit application.

By focusing on these areas, you can enhance the functionality, reliability, and user experience of your Streamlit application, making the integration of Plotly graphs within the chat interface even more seamless and effective.

Details

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[khankamranali]

I have solved this problem by writing a tool. Here is the code snippet.

from langchain.agents import tool
from plotly.graph_objects import Figure
from plotly.io import from_json
import json

@tool
def plotChart(data: str) -> int:
"""Plots json data using plotly Figure. Use it only for ploting charts and graphs."""
# Load JSON data
figure_dict = json.loads(data)

# Create Figure object from JSON data
fig = from_json(json.dumps(figure_dict))

st.plotly_chart(fig)
...

tools = [plotChart]
llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-1106")
agent = create_pandas_dataframe_agent(llm,
df,max_iterations=15, number_of_head_rows=5,
max_execution_time=30,
agent_type=AgentType.OPENAI_FUNCTIONS,
verbose=True, extra_tools=tools)