How to create a custom Agent that reviews git commits?

[SAIL-Fang]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

template = '''Answer the following questions as best you can. You have access to the following tools:

            {tools}
            Use the following format:

            Question: the input question you must answer
            Thought: you should always think about what to do
            Action: Use one of the [{tool_names}] if needed, otherwise, analyze the git diff manually.
            Action Input: the input to the action
            Observation: the result of the action
            ... (this Thought/Action/Action Input/Observation can repeat five times)
            Thought: I now know the final answer
            Final Answer: the final answer to the original input question

            Begin!

            Question: {input}
            Thought:{agent_scratchpad}'''
            prompt = PromptTemplate.from_template(template)
            # search = TavilySearchResults(max_results=2)
            tools = [vector_store_tool]
            
            agent = create_react_agent(llm_chain, tools, prompt)
            agent_executor = AgentExecutor(agent=agent, tools=tools, handle_parsing_errors=True,  verbose = True)
            input_data = {
                "input": item_prompt,
            }
            result = agent_executor.invoke(input_data)

Description

I want to create a proxy that can review git commits (i.e. check their names), I have a code naming conventions, git diff section. How should I adjust the template to make the agent work better? I checked the documentation but couldn't find a hint on how to fine-tune the agent.

System Info

$ pip freeze |grep langchain
composio_langchain==0.5.8
langchain==0.2.14
langchain-anthropic==0.1.23
langchain-astradb==0.3.3
langchain-aws==0.1.16
langchain-chroma==0.1.2
langchain-cli==0.0.26
langchain-cohere==0.1.9
langchain-community==0.2.12
langchain-core==0.2.35
langchain-experimental==0.0.61
langchain-google-calendar-tools==0.0.1
langchain-google-community==1.0.7
langchain-google-genai==1.0.8
langchain-google-vertexai==1.0.7
langchain-groq==0.1.6
langchain-huggingface==0.0.3
langchain-milvus==0.1.4
langchain-mistralai==0.1.10
langchain-mongodb==0.1.8
langchain-nvidia-ai-endpoints==0.1.6
langchain-openai==0.1.22
langchain-pinecone==0.1.2
langchain-text-splitters==0.2.2
langchainhub==0.1.21

windows

[dosubot[bot]]

Hello, @SAIL-Fang!

To create a custom Agent that reviews git commits and checks their names using LangChain, you can follow these steps:

1. Define the tools: Create a tool that can interact with the git repository to fetch commit names.
2. Create the agent: Use the defined tools and a language model to create an agent.
3. Run the agent: Execute the agent to review git commits.

Here's a step-by-step guide with code snippets:

Step 1: Define the Tools

First, create a tool that can fetch commit names from a git repository. You can use the subprocess module to run git commands.

import subprocess
from langchain.tools import BaseTool

class GitCommitTool(BaseTool):
    def invoke(self, input):
        result = subprocess.run(["git", "log", "--pretty=format:%s"], capture_output=True, text=True)
        return result.stdout.split('\n')

git_commit_tool = GitCommitTool()

Step 2: Create the Agent

Next, create an agent that uses this tool to review commit names. You can use a language model like GPT-4 for this purpose.

from langchain_openai import ChatOpenAI
from langchain.agents import create_tool_calling_agent
from langchain import hub

# Initialize the language model
model = ChatOpenAI(model="gpt-4")

# Define the prompt for the agent
prompt = hub.pull("hwchase17/openai-functions-agent")

# Create the agent with the model, tools, and prompt
tools = [git_commit_tool]
agent = create_tool_calling_agent(model, tools, prompt)

Step 3: Run the Agent

Finally, run the agent to review the git commits.

from langchain.agents import AgentExecutor

# Combine the agent with the tools inside the AgentExecutor
agent_executor = AgentExecutor(agent=agent, tools=tools)

# Run the agent to review git commits
response = agent_executor.invoke({"input": "Review the git commit names"})
print(response)

This setup will allow the agent to fetch git commit names and review them using the language model.

Alternative: Using GitLab Toolkit

If you are working with a GitLab repository, you can use the GitLabToolkit from LangChain's community tools. This toolkit provides a more integrated way to interact with GitLab repositories.

Setup

1. Install the python-gitlab library

%pip install --upgrade --quiet python-gitlab langchain-community

2. Create a GitLab personal access token

Follow the instructions here to create a GitLab personal access token with the necessary permissions.

3. Set Environmental Variables

Set the following environmental variables:

import os

os.environ["GITLAB_URL"] = "https://gitlab.example.org"
os.environ["GITLAB_PERSONAL_ACCESS_TOKEN"] = "your_personal_access_token"
os.environ["GITLAB_REPOSITORY"] = "username/repo-name"
os.environ["GITLAB_BRANCH"] = "bot-branch-name"
os.environ["GITLAB_BASE_BRANCH"] = "main"
os.environ["OPENAI_API_KEY"] = "your_openai_api_key"

Example: Simple Agent

from langchain.agents import AgentType, initialize_agent
from langchain_community.agent_toolkits.gitlab.toolkit import GitLabToolkit
from langchain_community.utilities.gitlab import GitLabAPIWrapper
from langchain_openai import OpenAI

llm = OpenAI(temperature=0)
gitlab = GitLabAPIWrapper()
toolkit = GitLabToolkit.from_gitlab_api_wrapper(gitlab)
agent = initialize_agent(
    toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)

agent.run(
    "You have the software engineering capabilities of a Google Principle engineer. You are tasked with completing issues on a gitlab repository. Please look at the open issues and complete them by creating merge requests that solve the issues."
)

This example demonstrates how to set up an agent that can interact with a GitLab repository, fetch commit names, and perform other repository-related tasks ^[1][2].

To continue talking to Dosu, mention @dosu.

[jpchev]

does this work only with OpenAI? Which other llm can be used?

[SAIL-Fang]

@dosu
I kind of get what you mean, I need to wrap all the steps I want into a tool, and then let the agent call it automatically? For example, I have a custom code naming style spec that I need to wrap as a tool so that the agent can look at it at runtime and use it to inspect the git diff part?

[dosubot[bot]]

Yes, exactly! You can wrap your custom code naming style specification into a tool and then let the agent call it automatically. Here's how you can do it:

1. Define the Tool: Create a tool that checks the code naming style against your specification.
2. Create the Agent: Use the defined tool and a language model to create an agent.
3. Run the Agent: Execute the agent to review the git diff and check the naming style.

Step 1: Define the Tool

First, create a tool that checks the code naming style. You can use the subprocess module to fetch the git diff and then apply your naming style checks.

import subprocess
from langchain.tools import BaseTool

class CodeNamingStyleTool(BaseTool):
    def __init__(self, naming_spec):
        self.naming_spec = naming_spec

    def invoke(self, input):
        # Fetch the git diff
        result = subprocess.run(["git", "diff"], capture_output=True, text=True)
        git_diff = result.stdout

        # Apply naming style checks (this is a placeholder, replace with your actual logic)
        violations = self.check_naming_style(git_diff)
        return violations

    def check_naming_style(self, git_diff):
        # Implement your naming style checks here
        # For example, check if function names follow a specific pattern
        violations = []
        for line in git_diff.split('\n'):
            if "def " in line and not self.naming_spec in line:
                violations.append(f"Naming violation: {line}")
        return violations

naming_spec = "your_naming_spec_here"
code_naming_style_tool = CodeNamingStyleTool(naming_spec)

Step 2: Create the Agent

Next, create an agent that uses this tool to review the git diff and check the naming style.

from langchain_openai import ChatOpenAI
from langchain.agents import create_tool_calling_agent
from langchain import hub

# Initialize the language model
model = ChatOpenAI(model="gpt-4")

# Define the prompt for the agent
prompt = hub.pull("hwchase17/openai-functions-agent")

# Create the agent with the model, tools, and prompt
tools = [code_naming_style_tool]
agent = create_tool_calling_agent(model, tools, prompt)

Step 3: Run the Agent

Finally, run the agent to review the git diff and check the naming style.

from langchain.agents import AgentExecutor

# Combine the agent with the tools inside the AgentExecutor
agent_executor = AgentExecutor(agent=agent, tools=tools)

# Run the agent to review git diff and check naming style
response = agent_executor.invoke({"input": "Review the git diff for naming style violations"})
print(response)

This setup will allow the agent to fetch the git diff and review it against your custom code naming style specification at runtime.

To continue talking to Dosu, mention @dosu.

[SAIL-Fang]

@dosu
If the agent's job is to review git diff code according to code style guidelines, wouldn't it be better to create an agent using 'create_react_agent'?

[dosubot[bot]]

Yes, it would be better to create an agent using create_react_agent if the agent's job is to review git diff code according to code style guidelines. The create_react_agent method in LangChain is designed to handle complex state management and iterative processing, which is suitable for tasks like reviewing code diffs and ensuring they adhere to style guidelines.

Here's an example of how you can create a ReAct-style agent for this purpose:

from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

# Initialize the language model
model = ChatOpenAI(model="gpt-4")

# Define the tool for code review
@tool
def review_code_diff(diff: str) -> str:
    """Reviews git diff code according to code style guidelines."""
    # Implement the code review logic here
    return "Reviewed code diff"

tools = [review_code_diff]

# Create the ReAct-style agent
app = create_react_agent(model, tools)

# Example query for the agent
query = "Review the following git diff:\n<git_diff_code_here>"

# Invoke the agent
messages = app.invoke({"messages": [("human", query)]})
print({
    "input": query,
    "output": messages["messages"][-1].content,
})

This setup allows the agent to process the code diff and provide feedback according to the specified guidelines ^[1].

To continue talking to Dosu, mention @dosu.

[SAIL-Fang]

@dosu
How to make agent use LLM to review code? Do I need to encapsulate "LLM code review "as a tool?

[dosubot[bot]]

To make an agent use an LLM to review code, you need to encapsulate the "LLM code review" functionality as a tool. This can be done by converting a LangChain Runnable into a tool using the as_tool method. Here is an example of how you can achieve this:

1. Define the function for code review.
2. Convert the function into a Runnable.
3. Convert the Runnable into a tool using the as_tool method.
4. Use the tool in an agent.

Here is a sample implementation:

from langchain_core.runnables import RunnableLambda
from langchain_openai import ChatOpenAI

# Define the function for code review
def code_review(x: str) -> str:
    # Here you would integrate with an LLM to review the code
    # For demonstration, we will just return a placeholder string
    return "Reviewed code: " + x

# Convert the function into a Runnable
runnable = RunnableLambda(code_review)

# Convert the Runnable into a tool
code_review_tool = runnable.as_tool(
    name="CodeReviewTool",
    description="Tool to review code using an LLM."
)

# Instantiate a chat model that supports tool calling
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

# Create an agent and provide it the tool
from langgraph.prebuilt import create_react_agent

tools = [code_review_tool]
agent = create_react_agent(llm, tools)

# Use the agent to review code
for chunk in agent.stream({"messages": [("human", "Review the following code: print('Hello, world!')")]}):
    print(chunk)
    print("----")

This code demonstrates how to encapsulate the "LLM code review" as a tool and use it within an agent ^[1][2].

To continue talking to Dosu, mention @dosu.

[SAIL-Fang]

@dosu
If I need the agent to get the git diff section with git_commit tool, the source code with get_source_code tool, and then the custom naming style specification with code_naming_style tool, Finally, use LLM to review the git diff section, based on the source code and our custom naming conventions. What should I do?

[dosubot[bot]]

To create an agent that uses the git_commit tool to get the git diff section, the get_source_code tool to get the source code, and the code_naming_style tool to get the custom naming style specification, and then asks the LLM to review the git diff section based on the source code and custom naming conventions, you can follow the structure provided in the agent_executor function. Here is a modified version of the code to include the required tools and logic:

import os
import re
import subprocess  # nosec
import tempfile

from langchain.agents import AgentType, initialize_agent
from langchain.pydantic_v1 import BaseModel, Field, ValidationError, validator
from langchain_community.chat_models import ChatOpenAI
from langchain_core.language_models import BaseLLM
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import ConfigurableField, Runnable
from langchain_core.tools import Tool

# Define the tools
def git_commit():
    # Implement the logic to get the git diff section
    pass

def get_source_code():
    # Implement the logic to get the source code
    pass

def code_naming_style():
    # Implement the logic to get the custom naming style specification
    pass

def review_code(diff, source_code, naming_style):
    # Implement the logic to review the git diff section based on the source code and naming conventions
    pass

# Create tools from functions
git_commit_tool = Tool.from_function(
    git_commit,
    name="git_commit",
    description="Get the git diff section."
)

get_source_code_tool = Tool.from_function(
    get_source_code,
    name="get_source_code",
    description="Get the source code."
)

code_naming_style_tool = Tool.from_function(
    code_naming_style,
    name="code_naming_style",
    description="Get the custom naming style specification."
)

review_code_tool = Tool.from_function(
    review_code,
    name="review_code",
    description="Review the git diff section based on the source code and custom naming conventions."
)

# Define the prompt
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a world class Python coder who uses "
            "black, ruff, and *strict* mypy for all of your code. "
            "Provide complete, end-to-end Python code "
            "to meet the user's description/requirements. "
            "Always `check` your code. When you're done, "
            "you must ALWAYS use the `submit` tool.",
        ),
        (
            "human",
            ": {input}",
        ),
    ],
)

# Initialize the agent with the tools
tools = [git_commit_tool, get_source_code_tool, code_naming_style_tool, review_code_tool]

def get_agent_executor(
    llm: BaseLLM,
    agent_type: AgentType = AgentType.OPENAI_FUNCTIONS,
) -> Runnable:
    _agent_executor = initialize_agent(
        tools,
        llm,
        agent=agent_type,
        verbose=True,
        handle_parsing_errors=True,
        prompt=prompt,
    )
    return _agent_executor | (lambda output: output["output"])

class Instruction(BaseModel):
    __root__: str

agent_executor = (
    get_agent_executor(ChatOpenAI(model="gpt-4-1106-preview", temperature=0.0))
    .configurable_alternatives(
        ConfigurableField("model_name"),
        default_key="gpt4turbo",
        gpt4=get_agent_executor(ChatOpenAI(model="gpt-4", temperature=0.0)),
        gpt35t=get_agent_executor(
            ChatOpenAI(model="gpt-3.5-turbo", temperature=0.0),
        ),
    )
    .with_types(input_type=Instruction, output_type=str)
)

This code sets up an agent that uses the specified tools to get the git diff section, source code, and custom naming style specification, and then reviews the git diff section based on the source code and naming conventions ^[1].

To continue talking to Dosu, mention @dosu.

[SAIL-Fang]

@dosu
If I change the input to "Please review git diff against the source code and your custom naming style", do I still need review_code_tool?

[dosubot[bot]]

If you change the input to "Please review git diff against the source code and your custom naming style," you still need the review_code_tool tool. This tool encapsulates the logic for reviewing the git diff section based on the source code and custom naming conventions, which is essential for the agent to perform the review accurately.

Here's a concise summary of why you need the review_code_tool:

1. Encapsulation of Logic: The review_code_tool encapsulates the specific logic required to review the git diff against the source code and naming conventions. This keeps the agent's workflow modular and maintainable.
2. Tool Integration: The agent uses tools to perform specific tasks. By defining review_code_tool, you ensure that the agent can call this tool to perform the review as part of its workflow.
3. Separation of Concerns: Each tool handles a distinct part of the process (e.g., fetching git diff, fetching source code, checking naming conventions). The review_code_tool specifically handles the review logic, making the overall system more organized and easier to manage.

Therefore, even with the updated input, the review_code_tool remains necessary to ensure the agent can effectively review the git diff against the source code and custom naming style.

To continue talking to Dosu, mention @dosu.

[SAIL-Fang]

@dosu
What tools do I need if I want the Agent to review the named part of a git commit according to my custom code rules?

[dosubot[bot]]

To have the Agent review the named part of a git commit according to your custom code rules, you need the following tools:

1. check_code_tool: This tool checks the code using black, ruff, and mypy to ensure it conforms to the specified standards.
2. submit_code_tool: This tool is used to submit the code after it has been checked.

Here is the relevant code snippet that defines these tools:

check_code_tool = Tool.from_function(
    check_code,
    name="check-code",
    description="Always check your code before submitting it!",
)

submit_code_tool = Tool.from_function(
    strip_python_markdown_tags,
    name="submit-code",
    description="THIS TOOL is the most important. "
    "use it to submit your code to the user who requested it... "
    "but be sure to `check` it first!",
    return_direct=True,
)

tools = [check_code_tool, submit_code_tool]

These tools are then used to initialize the agent:

def get_agent_executor(
    llm: BaseLLM,
    agent_type: AgentType = AgentType.OPENAI_FUNCTIONS,
) -> Runnable:
    _agent_executor = initialize_agent(
        tools,
        llm,
        agent=agent_type,
        verbose=True,
        handle_parsing_errors=True,
        prompt=prompt,
    )
    return _agent_executor | (lambda output: output["output"])

Additionally, you can leverage the GitHub Toolkit from LangChain to interact with GitHub repositories. This toolkit includes tools for various GitHub operations such as creating pull requests, reading files, updating files, and more. Here is an example of how to instantiate and use the GitHub Toolkit:

from langchain_community.agent_toolkits.github.toolkit import GitHubToolkit
from langchain_community.utilities.github import GitHubAPIWrapper

github = GitHubAPIWrapper()
toolkit = GitHubToolkit.from_github_api_wrapper(github)

tools = toolkit.get_tools()
for tool in tools:
    print(tool.name)

By using these tools, the Agent can review and ensure the code conforms to your custom rules before submission, and also interact with GitHub for various operations ^[1][2].

To continue talking to Dosu, mention @dosu.

[SAIL-Fang]

@dosu
Is it possible to let the operation of the review also be performed by the LLM? Like asking LLM to name and review git diff commits based on my custom code style

[dosubot[bot]]

Yes, it is possible to let the operation of the review be performed by the LLM, including naming and reviewing git diff commits based on your custom code style. You can create a custom LLM class in LangChain to achieve this. Here is an example of how you might implement such a custom LLM:

from typing import Any, Dict, List, Optional
from langchain_core.callbacks.manager import CallbackManagerForLLMRun
from langchain_core.language_models.llms import LLM

class CustomReviewLLM(LLM):
    """A custom LLM for naming and reviewing git diff commits based on custom code style."""

    def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        """Run the LLM on the given input.

        Args:
            prompt: The prompt to generate from.
            stop: Stop words to use when generating. Model output is cut off at the
                first occurrence of any of the stop substrings.
            run_manager: Callback manager for the run.
            **kwargs: Arbitrary additional keyword arguments. These are usually passed
                to the model provider API call.

        Returns:
            The model output as a string. Actual completions SHOULD NOT include the prompt.
        """
        # Implement your custom logic here to name and review git diff commits
        # based on your custom code style.
        review = f"Review of commit: {prompt[:50]}..."  # Example logic
        return review

    @property
    def _identifying_params(self) -> Dict[str, Any]:
        """Return a dictionary of identifying parameters."""
        return {
            "model_name": "CustomReviewLLM",
        }

    @property
    def _llm_type(self) -> str:
        """Get the type of language model used by this chat model. Used for logging purposes only."""
        return "custom_review"

You can then use this custom LLM in your LangChain programs to name and review git diff commits based on your custom code style ^[1].

To continue talking to Dosu, mention @dosu.