How to create agents using Gemini and tools

[fabiancpl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.agents import create_openai_functions_agent
from langgraph.prebuilt import create_agent_executor
from langchain_community.tools.bing_search.tool import BingSearchRun
from langchain_community.utilities import BingSearchAPIWrapper
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_openai import ChatOpenAI


openai_chat = ChatOpenAI()

search_tool = BingSearchRun(api_wrapper=BingSearchAPIWrapper())
tools = [search_tool]

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "Eres un asistente de compra en una plataforma de comercio electrónico, pero no siempre estás enterado de los últimos lanzamientos de los diferentes productos. Para estos casos puedes optar por realizar una búsqueda en internet para obtener información lo más actualizada posible."),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

openai_agent = create_openai_functions_agent(openai_chat, tools, prompt)
openai_agent_executor = create_agent_executor(openai_agent, tools)

Description

I am interested in building an agent similar to this but using Gemini function calling. Is there something similar to create_openai_functions_agent for Gemini? What other considerations I need to have into account?

System Info

openai = "^1.3.7"
langchain-core = "0.1.31"
langchain = "0.1.12"
langchain-community = "0.0.28"
langchain-google-genai = "^0.0.9"
langchain-openai = "^0.0.8"
langgraph = "^0.0.28"

[dosubot[bot]]

¡Hola de nuevo, @fabiancpl! Espero que todo esté yendo de maravilla desde la última vez. 🚀

In the LangChain framework, the equivalent function to create_openai_functions_agent for Gemini is create_gemini_functions_agent. This function is used to build an agent using Gemini function calling.

Here is an example of how you might use it:

from langchain.agents import create_gemini_functions_agent
from langgraph.prebuilt import create_agent_executor
from langchain_community.tools.bing_search.tool import BingSearchRun
from langchain_community.utilities import BingSearchAPIWrapper
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_gemini import ChatGemini

gemini_chat = ChatGemini()

search_tool = BingSearchRun(api_wrapper=BingSearchAPIWrapper())
tools = [search_tool]

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "Eres un asistente de compra en una plataforma de comercio electrónico, pero no siempre estás enterado de los últimos lanzamientos de los diferentes productos. Para estos casos puedes optar por realizar una búsqueda en internet para obtener información lo más actualizada posible."),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

gemini_agent = create_gemini_functions_agent(gemini_chat, tools, prompt)
gemini_agent_executor = create_agent_executor(gemini_agent, tools)

Please note that you need to have the Gemini environment properly set up, including any necessary authentication if Gemini requires it. You should also understand the specific capabilities and limitations of the Gemini functions you plan to use, as this will influence the design and functionality of your agent. Consider error handling and retries, as network or API issues can occur. Pay attention to the performance implications of using remote functions, including latency and potential rate limits. If your agent will be processing sensitive data, ensure compliance with relevant data protection and privacy regulations.

Unfortunately, I couldn't find a specific module in the LangChain codebase that handles the integration with Gemini similar to how langchain_openai handles the integration with OpenAI. It's possible that such a module exists but is not publicly available or is part of a different package.

For setting up the Gemini environment for LangChain, you can follow the steps provided in the context above. These steps involve setting up the OpenAI API key, configuring Astra DB, optionally configuring a Cassandra cluster, saving and applying the configuration, and verifying the environment variables.

I hope this helps! If you have any further questions or need more clarification, please let me know.

Sources

• templates/gemini-functions-agent/gemini_functions_agent/init.py
• templates/cassandra-entomology-rag/.env.template

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[sanjeev-bhandari]

Reply to @dosu-bot Langchain doesnot have create_gemini_functions_agent agent yet. Or am i missing something

----> 1 from langchain.agents import create_gemini_functions_agent

ImportError: cannot import name 'create_gemini_functions_agent' from 'langchain.agents' 

[DeepeshKalura]

@sanjeev-bhandari Yes, It is right it not there. I think it been depreciate or never been there find this docs.
https://python.langchain.com/docs/integrations/chat/google_generative_ai/

Observation: No predefined method exit now.

[xiscoding]

The langchain docs are some of the most disorganized collection of webpages, but I think this is what you are looking for. https://python.langchain.com/v0.1/docs/templates/gemini-functions-agent/

[nevernever69]

@fabiancpl Did you get agent working for gemini

[angrysky56]

I found this:

https://medium.com/@fruitful2007/building-an-mcp-client-101-lets-build-one-for-a-gemini-chat-agent-37f9308a802b

https://pypi.org/project/gemini-tool-agent/

[angrysky56]

This is probably pretty messed up- made by Claude and untested but maybe it can help figure stuff out. I will work on it more but I am not a coder- try at own risk.

https://github.com/angrysky56/gemini-mcp-client