The insert_edit_into_file tool does not seems to work for gemini-2.5-flash-preview-05-20

[Jufralice]

Hello,

The recent replacement of the @editor tool by the @insert_edit_into_file tool seems to be a great improvement.
Thanks a lot for the great work.

However, this new tool does not seems to work for some adapters like 'gemini-2.5-flash-preview-05-20'

The LLM will most of the time respond with a 'normal' llm response message (including his 'Patch' block inside the response message) instead of actually calling the tool using the function_call schema.

Or some other times (after I reply to the LLM that he must call the tool) it will eventually call the tool, but fail to use the Patch Format correctly (by missing the *** Begin Patch and *** End Patch surrounding lines).

I guess that we should be able to fix that issue by tweaking the system prompt of this tool.

I did some experiments without success so far:

• I tried to replace the current system prompt:

current system prompt

Before editing a file, ensure you have its content via the provided context or read_file tool.
Use the insert_edit_into_file tool to modify files.
NEVER show the code edits to the user - only call the tool. The system will apply and display the changes.
For each file, give a short description of what needs to be changed, then use the insert_edit_into_file tools. You can use the tool multiple times in a response, and you can keep writing text after using a tool.
The insert_edit_into_file tool is very smart and can understand how to apply your edits to the user's files, you just need to follow the patch format instructions carefully and to the letter.

Patch Format

*** Begin Patch
[PATCH]
*** End Patch

The [PATCH] is the series of diffs to be applied for each change in the file. Each diff should be in this format:

[3 lines of pre-context]
-[old code]
+[new code]
[3 lines of post-context]

The context blocks are 3 lines of existing code, immediately before and after the modified lines of code. Lines to be modified should be prefixed with a + or - sign. Unmodified lines used in context should begin with an empty space .

Multiple blocks of diffs should be separated by an empty line and @@[identifier] detailed below.

The immediately preceding and after context lines are enough to locate the lines to edit. DO NOT USE line numbers anywhere in the patch.

You can use @@[identifier] to define a larger context in case the immediately before and after context is not sufficient to locate the edits. Example:

@@Class BaseClass(models.Model):
[3 lines of pre-context]

• pass

• raise NotImplementedError()
  [3 lines of post-context]

You can also use multiple @@[identifiers] to provide the right context if a single @@ is not sufficient.

Example with multiple blocks of changes and @@ identifiers:

*** Begin Patch
@@Class BaseClass(models.Model):
@@ def search():

• pass
  

• raise NotImplementedError()
  

@@Class Subclass(BaseClass):
@@ def search():

• pass
  

• raise NotImplementedError()
  

*** End Patch

This format is a bit similar to the git diff format; the difference is that @@[identifiers] uses the unique line identifiers from the preceding code instead of line numbers. We don't use line numbers anywhere since the before and after context, and @@ identifiers are enough to locate the edits.
The system uses fuzzy matching and confidence scoring so focus on providing enough context to uniquely identify the location.

By a prompt following the 'CONTEXT, OBJECTIVE, RESPONSE' format like the other tools uses:

new system prompt

Insert Edit Into File Tool (insert_edit_into_file)

CONTEXT

• You have access to an 'insert_edit_into_file' tool running within CodeCompanion, in Neovim.
• You can use it to add, edit or delete code in a Neovim file.
• Before editing a file, ensure you have its content via the provided context or read_file tool.

OBJECTIVE

• To implement code changes in a Neovim file.
• Follow the tool's schema.

RESPONSE

• NEVER show the code edits to the user, only call the tool. The system will apply and display the changes.
• For each file, give a short description of what needs to be changed, then use the insert_edit_into_file tool.
• Use this tool once per file that needs to be modified, even if there are multiple changes for a file
• You can use the tool multiple times in a response if you need to apply changes to multiple files.
• You can keep writing text after using a tool
• The insert_edit_into_file tool is very smart and can understand how to apply your edits to the user's files, you just need to follow the Patch Format instructions (described below) carefully and to the letter.
• Ensure that the code you write is syntactically correct and valid and that indentations are correct.
• Do not return your Patch response in a 'normal' response message. You must call the insert_edit_into_file tool by using 'function_call' like for any other tool. Your Patch block should be put in the 'code' property of the 'insert_edit_into_file' function call like described in the tool's schema!

POINTS TO NOTE

• This tool can be used alongside other tools within CodeCompanion

Patch Format

*** Begin Patch
[PATCH]
*** End Patch

The [PATCH] block absolutely needs to be surrounded by *** Begin Patch and *** End Patch. Otherwise the system won't be able to apply your changes.

The [PATCH] is the series of diffs to be applied for each change in the file. Each diff should be in this format:

[3 lines of pre-context]
-[old code]
+[new code]
[3 lines of post-context]

The context blocks are 3 lines of existing code, immediately before and after the modified lines of code. Lines to be modified should be prefixed with a + or - sign. Unmodified lines used in context should begin with an empty space .

Multiple blocks of diffs should be separated by an empty line and @@[identifier] detailed below.

The immediately preceding and after context lines are enough to locate the lines to edit. DO NOT USE line numbers anywhere in the patch.

You can use @@[identifier] to define a larger context in case the immediately before and after context is not sufficient to locate the edits. Example:

@@Class BaseClass(models.Model):
[3 lines of pre-context]

• pass

• raise NotImplementedError()
  [3 lines of post-context]

You can also use multiple @@[identifiers] to provide the right context if a single @@ is not sufficient.

Example with multiple blocks of changes and @@ identifiers:

*** Begin Patch
@@Class BaseClass(models.Model):
@@ def search():

• pass
  

• raise NotImplementedError()
  

@@Class Subclass(BaseClass):
@@ def search():

• pass
  

• raise NotImplementedError()
  

*** End Patch

This format is a bit similar to the git diff format; the difference is that @@[identifiers] uses the unique line identifiers from the preceding code instead of line numbers. We don't use line numbers anywhere since the before and after context, and @@ identifiers are enough to locate the edits.
The system uses fuzzy matching and confidence scoring so focus on providing enough context to uniquely identify the location.

• In order to prevent the missing surrounding patch lines, I added the following part to the Patch Format section:

The [PATCH] block absolutely needs to be surrounded by *** Begin Patch and *** End Patch. Otherwise the system won't be able to apply your changes.

• In order to force the LLM to actually call the tool, I added the following part to the 'RESPONSE' section:

• Do not return your Patch response in a 'normal' response message. You must call the insert_edit_into_file tool by using 'function_call' like for any other tool. Your Patch block should be put in the 'code' property of the 'insert_edit_into_file' function call like described in the tool's schema!

Unfortunately, nothing seems to solve the issue and the LLM continues to reply with the PATCH response inside a normal response message instead of actually calling the tool.
The above addition mentioning the use of 'function_call' actually seems to make things worse since the gemini adapter then returns an internal error.
I tried to formulate this sentence in a few other ways but nothing seems to help.

If anybody find a way to fix that prompt for the gemini flash adapter, I'm more than interested.

Thanks a lot.

[ph1losof]

I also get this a lot +1

[Jufralice]

thank you @t3ntxcl3s . What model do you use? also gemini-2.5-flash-preview ?

[ph1losof]

Yes, specifically when I use it

[kxzk]

I also started having problems last night.

I was using copilot (sonnet-4) and openai (o3).

Some errors:

• Bad/incorrect dif
• #buffer uses the [CodeCompanion] buffer
• [CodeCompanion] buffer seems to be overwritten with diff
• No placement returned -> when running CodeCompanionChat

I'm guessing it's likely prompt issues [as noted above]. I'll try and tweak and report back if I get it working.

Also, ga doesn't seem to allow me to accept changes anymore.

[Jufralice]

Thanks @kxzk .

yes apparently you should now use gda or gdr to accept or reject changes (instead of ga and gr like before)
(and you have to be fast otherwise the tool will say that you did not respond in time :-) . I guess there is probably a config somewhere to change that behaviors, I'll check that tomorrow)

[kxzk]

Ah! I'm fine with gda I was just confused on how to accept and couldn't see it documented anywhere.

Thanks! @Jufralice

[Jufralice]

I've read the GPT-4.1 prompting guide as mentioned by @pratyushmittal in his original Refactor @files tool to use diffs for update PR and they seems to say that we should better use the tools's description instead of a separate system prompt.... They example with a python tool actually have the full Patch specifications inside the tool description.

I then tried and move most of the system prompt into the tool description, and that indeed seems to help a lot!

I'll test a bit more tomorrow but, so far, I didn't had the case anymore where the LLM put his Patch inside a 'normal' message instead of in a tool call. I only had one error due to a missing 'end' tag but It seems to be way less frequent.

here are my tool_description and the system_prompt

TOOL_DESCRIPTION

local TOOL_DESCRIPTION =
[[This function is used to insert new code or modify existing code in a file in the current working directory.
Use this tool once per file that needs to be modified, even if there are multiple changes for a file.
You can use the tool multiple times in a response if you need to apply changes to multiple files.
The system is very smart and can understand how to apply your edits to the user's files, you just need to follow the below Patch Format instructions carefully and to the letter. You should then pass your Patch as "code".

Patch Format instructions:
]] .. patch.FORMAT_PROMPT .. [[
The system uses fuzzy matching and confidence scoring so focus on providing enough context to uniquely identify the location.
]]

M.FORMAT_PROMPT = [[
*** Begin Patch
[PATCH]
*** End Patch

Where [PATCH] is the series of diffs to be applied for each change in the file. Each diff should be in this format:

[3 lines of pre-context]
-[old code]
+[new code]
[3 lines of post-context]

The context blocks are 3 lines of existing code, immediately before and after the modified lines of code. Lines to be modified should be prefixed with a + or - sign. Unmodified lines used in context should begin with an empty space .

Multiple blocks of diffs should be separated by an empty line and @@[identifier] detailed below.

The immediately preceding and after context lines are enough to locate the lines to edit. DO NOT USE line numbers anywhere in the patch.

You can use @@[identifier] to define a larger context in case the immediately before and after context is not sufficient to locate the edits. Example:

@@Class BaseClass(models.Model):
[3 lines of pre-context]

• pass

• raise NotImplementedError()
  [3 lines of post-context]

You can also use multiple @@[identifiers] to provide the right context if a single @@ is not sufficient.

An example of a message that you might pass as "code" to this function, in order to apply a patch with multiple blocks of changes and @@ identifiers, is shown below.

*** Begin Patch
@@Class BaseClass(models.Model):
@@ def search():

• pass
  

• raise NotImplementedError()
  

@@Class Subclass(BaseClass):
@@ def search():

• pass
  

• raise NotImplementedError()
  

*** End Patch

Make sure to always add the '*** Begin Patch' and '*** End Patch' lines respectively before and after your patch content otherwise the tool call will fail.
]]

PROMPT

local PROMPT = [[# Insert Edit Into File Tool (insert_edit_into_file)

CONTEXT

• You have access to an 'insert_edit_into_file' tool running within CodeCompanion, in Neovim.
• You can use it to add, edit or delete code in a Neovim file.
• Before editing a file, ensure you have its content via the provided context or read_file tool.

RESPONSE

• NEVER show the code edits to the user, only call the tool. The system will apply and display the changes.
• For each file, give a short description of what needs to be changed, then use the insert_edit_into_file tool.
• Carefully read the tool description, especially the 'Patch Format' instructions
• You can keep writing text after using a tool
  ]]

[Jufralice]

Should I maybe open a PR with my changes to these prompts to make it easier for other users to test them ?
(going to sleep right now, but I can do that tomorrow morning)

[archie-judd]

Thanks, I'd really appreciate a PR to test -- I am getting similar issues using copilot -- specifically that it doesn't add the *** Begin Patch and *** End Patch lines.

[olimorris]

@Jufralice really appreciate your diligence and hard work on this.

Some interesting points you raise. I've noticed that GitHub Copilot Chat also has the majority of tool detail in a non-system prompt too. Sharing that for reference:

Agent.json

If you share GPT-4.1 prompting guide with CodeCompanion (via the /fetch command), and ask it to compare our patch prompt, does it spot any differences? Could we be clearer in how we explain patching?

[Jufralice]

Hi @olimorris,

I tried to ask the LLM for suggestions on how to improve the prompts (passing it the prompting guide via the /fetch command).
It came with some suggestions, especially adding an example of how to actually call the tool (not only a Patch Format example but a tool call example with the 'explanation', 'filepath' and 'code' properties).
It didn't suggest to put the Patch Format into the tool's description instead of the system_prompt though.
I tested with his proposed changes and all my tool calls failed! It continues to pass the 'patch' in a normal response instead of actually calling the tool.

I then again moved most of the system_prompt (including the Patch Format) into the tool description and now... all my tests succeed! It always correctly call the tool.

I think that we can then indeed conclude that, at least for gemini-2.5-flash, the Patch Format needs to be in the tool's description.

I then tried to remove the proposed example of the tool call from the prompt to see if that was absolutely needed. Most of my tests still succeeded but I also got more failures.
Adding the example tool call in the tools's description apparently also cause confusion for the LLM. I seem to have the best results when I keep that example in the system prompt (maybe there is so size limitation for the tool's description field?)

I then noticed that the LLM nearly never asked for multiple code block changes in my buffer (it uses to call the tool for one change and say that he will do the rest in a next turn).
I then tweaked a bit more the prompt to emphasize that he can apply multiple changes in a given file in one tool call.
Now It does apply multiple changes correctly.
Updating the tool call example by an example where we do 2 code changes seems to also help for that.

The remaining issue that I from time to time have is that the LLM sometimes put line numbers in the @Identifier markers... But I suspect that it's because the buffer I shared with him does contains the line numbers.
Now that we don't use line numbers anymore for file and buffers updates, shouldn't we remove the inclusion of the line numbers when we share a buffer with the LLM?

Anyway, that tool works way better with these changes now!
It would be interesting to know if that also still work (and make sure that it doesn't make things worse) with other models...
I'll now open a pull request so that you can review these changes and maybe test them with your models

Thanks a lot.

[olimorris]

I've made a few comments regarding the PR - some things I don't understand. I'll also ask @pratyushmittal to jump in on that too as he is the foremost expert of this.

Now that we don't use line numbers anymore for file and buffers updates, shouldn't we remove the inclusion of the line numbers when we share a buffer with the LLM?

Not a fan of doing this and I'll push back by saying if an LLM is putting line numbers in places it shouldn't be then that is a performance defect of that LLM that CodeCompanion shouldn't be compensating for. I'm yet to experience that with any LLM in nearly 18 months of using the plugin. Moreover, I don't use the @insert_edit_into_file tool a whole lot (I like to review everything myself and choose what to carry over to a buffer). Having an LLM say, "Change lines 143 to 167" is super handy.

[pratyushmittal]

Hey, I will have a look at the PR and leave a review today.

These are my flying thoughts.

Around Line numbers
I will also recommend removing line numbers. It removes conflicting examples / instructions. The new models follow these instructions way too closely.

From the above tweet:

But then we hit a weird issue. The model kept botching citation formats - same mistake in every response. We started panicking, thinking something was wrong with the model.

Then someone actually checked our prompt examples.
And there it was: our citation examples were a complete mess. We were using inconsistent formatting and styling across the examples.

Sidenote: I will also recommend updating the diffs we provided for watched files. They currently contain the lines like these: @@ -18,7 +18,7 @@. We can replace these with @@. I will try to create a PR around this.

Around where to keep prompt instructions
The 4.1 guide recommends keeping keeping tools descriptions "thorought but relatively concise". It recommends keeping examples and long instructions in system prompt.

If your tool is particularly complicated and you'd like to provide examples of tool usage, we recommend that you create an # Examples section in your system prompt and place the examples there, rather than adding them into the "description' field, which should remain thorough but relatively concise.

The guide also recommends repeating instructions in case of long contexts (long contexts = long code files).

If you have long context in your prompt, ideally place your instructions at both the beginning and end of the provided context, as we found this to perform better than only above or below.

So no harm in repeating the instructions to use tools in user/system prompts.

Around not calling of function calls and LLMs replying in messages

• I frankly don't know why is this happening.
• Are we passing right tool names everywhere? I hope we are not using @files in our instructions anywhere instead of @insert_edit_into_file.
• We wrap our instructions inside <editFileInstructions>...</editFileInstructions> to pass instructions for insert_edit_into_file. Should we consider changing it to <insert_edit_into_file>...</insert_edit_into_file> or better # Instructions for using insert_edit_into_file tool?
• I will change my default model to gemini-2.5-flash-preview-05-20 to reproduce it. I will drop in a PR if I find any fixes.

[Jufralice]

Hi @pratyushmittal,
Thanks a lot for you great response and for taking the time to look into this. 👍

Sidenote: I will also recommend updating the diffs we provided for watched files. They currently contain the lines like these: @@ -18,7 +18,7 @@. We can replace these with @@. I will try to create a PR around this.

Thanks. I think that this will help a lot. This is indeed one of the issues I had where the proposed Patch contains identifiers like @@-18,7 +18,7...

We wrap our instructions inside ... to pass instructions for insert_edit_into_file. Should we consider changing it to <insert_edit_into_file>...</insert_edit_into_file> or better # Instructions for using insert_edit_into_file tool?

Totally agree, I also noticed that and though that that could be a reason why the LLM would not get that that prompt was linked to the tool call.

I will change my default model to gemini-2.5-flash-preview-05-20 to reproduce it. I will drop in a PR if I find any fixes.

Thanks you very much 🙏

[olimorris]

Let me provide some clarity on <editFileInstructions>. This is a pre-cursor to what will be a larger agent prompt in the future. There will be numerous tools that allow an LLM to edit a file. Line 3 of this prompt states Use the insert_edit_into_file tool to modify files so is being made explicit to the LLM.

@Jufralice I thought the crux of this issue is that Gemini is not understanding the patch format, rather than it not calling the tool?

[Jufralice]

Hi @olimorris ,
No, sorry if I was not clear. The bigger problem was that Gemini did not call the tool. It is true that he also sometimes forgot a 'begin patch' or 'end patch' but the real problem was that he used to respond with his patch response in a normal message instead of calling the tool .

Moving the Patch Format specifications into the tool description instead of in the system prompt does fix that problem of not calling the tool. It was like if he didn't understand that the system prompt was related to the insert_edit_into_file tool.

Apart from sometimes missing the begin or end statements, he does understand the patch format pretty well. And adding an example of how he should call the tool seems to help him to not forgot these begin or end.

[olimorris]

The recent changes in 17.1.1 plus a refactor to the system prompt have gemini-2.0-flash working on a basic file (and with gemini-2.5-flash-preview-05-20):

2025-06-16.23_35_23.-.WezTerm.mp4

[Jufralice]

Thanks a lot @olimorris .
I tested with your example file and it indeed work. But I still have a lot cases where the LLM does not call the tool (returning the proposed patch in a normal message). And it is still 100% solved by moving some parts of the system prompt into de tool description. I guess it occur most on bigger files. Maybe the LLM looses track of the relationship between the system prompt and the tool instructions when it get bigger buffer context in between? I can't share the file im testing with since it comes from a private project but I'll try to find some other example files from public repositories where I can reproduce the issues and I'll share them.

[GitMurf]

I tend to agree. From the OpenAI cookbook for newer models and tool calling suggestions they say:

We encourage developers to exclusively use the tools field to pass tools, rather than manually injecting tool descriptions into your prompt and writing a separate parser for tool calls, as some have reported doing in the past. This is the best way to minimize errors and ensure the model remains in distribution during tool-calling trajectories...

As well as...

...Developers should name tools clearly to indicate their purpose and add a clear, detailed description in the "description" field of the tool. Similarly, for each tool param, lean on good naming and descriptions to ensure appropriate usage.

[arnm]

@Jufralice @olimorris Great work on this! About to give 17.1.1 a test. I did want to mention that I frequently get similar issue, related to tool calling prompt engineering, for the @read_file tool where it will request lines past the last line of the file (see below) I see this constantly with gpt-4.1. Much less with more capable models but still an issue with all of them to some degree.

**Read File Tool**: end_line_number_base_zero (49) is beyond file length. File `requirements.txt` has 18 lines (0-17)

[Jufralice]

Hi @olimorris ,

You can reproduce the issues with this scenario (using a file from the codecompanion plugin):

• model: "gemini-2.5-flash-preview-05-20
• buffer: lua/codecompanion/strategies/chat/init.lua
• prompt:

call the @insert_edit_into_file tool to update the #buffer :
I want you to fix and finish the code of the functions

(I intentionally use a quite vague prompt in this example, to force the LLM to think more since that seems to help to reproduce these issues. I use a more specific prompt when I tested in my personal project but I can't share the file here.)

Example output:

I tested that prompt 15times from the master branch, 15times from this #1628 PR branch, and 15times from a merge of this #1628 PR and this #1663 PR
Here are my results:

From main branch

• Nbr success calls: 3
• Nbr missing 'start'-'end' markers errors: 6
• Nbr tool not called(patch returned in normal response message): 6

From the #1628 PR branch (where we moved some of the tool's system promt into the tool's description)

• Nbr success calls: 9
• Nbr missing 'start'-'end' markers errors: 6
• Nbr tool not called(patch returned in normal response message): 0

From the #1628 PR branch merged with the #1663 PR branch (that allow empty line before @@ focus)

• Nbr success calls: 13
• Nbr missing 'start'-'end' markers errors: 2 (this is the fallback error when the patch could not be applied when there was no begin-end markers)
• Nbr tool not called(patch returned in normal response message): 0

We can then see that in this scenario we have no more errors due to the tool not being called when we moved some parts of the system prompt into the tool's description.
And the #1663 PR seems to indeed help reduce the errors due to missing start and end markers.

Do you see similar results if you test with this scenario?
Thanks a lot.

[soulsam480]

adding to the conversation here, after struggling a lot with making diff from chat with gemini work, i found out it works beautifully when triggered inline. so the trick that I've been using is just asking inline question and then set buffer as context. not sure if this helps or known but noticed today.

[soulsam480]

sure

[soulsam480]

works perfectly. is there any plan of getting this in ?

[Jufralice]

Hi @soulsam480 ,
Not right now. See the latest message in my PR: #1628 (comment)
We'd like to avoid fixing an issue for a specific model when it already works fine for other models.
The plan is to go for a 'Aider' style like proposed by GitMurf below.
In the meantime I think we can use our own edit_file tool

[AlejandroGomezFrieiro]

If anyone needs a temporary solution, I actually have gotten very good results with the native Neovim tools with mcphub, in particular the write_file tool works great.

[soulsam480]

I'm still not able to figure out how to use mcp-hub with code companion. there's no clear doc on how to know tools are available or how does the @mcp thing works

[GitMurf]

I don't remember where I saw it but I remember the Aider developer mentioning that Gemini does better with full file edits versus something like gpt 4.1 doing better with diff patches. Point is, it does not surprise me Gemini models may be struggling with diff patches.

[GitMurf]

Ok here is the different edit strategies Aider does: https://aider.chat/docs/more/edit-formats.html

And it looks like here it shows all models and then the specific edit strategy that each does best. Looks like Gemini flash uses the "diff" strategy from the above link instead of the "udiff" which is more like a git patch format. https://aider.chat/docs/config/adv-model-settings.html

[olimorris]

Appreciate this. This has to be the direction moving forwards. I envisage this type of scenario only becoming more amplified in the future.

CodeCompanion will take this approach in the coming months.

[olimorris]

FYI, some useful links to how Copilot Chat implements patching:

• parser.ts
• parserApplyPatch.ts

They reference: https://cookbook.openai.com/examples/gpt4-1_prompting_guide#reference-implementation-apply_patchpy

[soulsam480]

does the gemini CLI write file tool help here https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/tools/write-file.ts ?