Code companion does not return the full LLM answer

[coolapso]

My apologies if this has been discussed before, but I my "search fu" may not be strong enough to find something relevant.

As stated in the title, sometimes codecompanion does not return the full answer from the LLM and cuts it somewhere halfway through.

This used to happen often in certain models with copilot. But then it stopped happening. Now it seems to be back again.

In this case now it happened using copilot with cloud-sonnet-4 ... the logs don't actually provide anything that seems useful to understand what's going on ...

I've seen this also happening with the anthropic adapter before ...

[DEBUG] 2025-07-14 22:34:57
Executing slash command: /buffer
[DEBUG] 2025-07-14 22:34:57
Calling slash command: strategies.chat.slash_commands.buffer
[DEBUG] 2025-07-14 22:34:57
Starting to watch buffer: 1
[DEBUG] 2025-07-14 22:34:59
Executing slash command: /file
[DEBUG] 2025-07-14 22:34:59
Calling slash command: strategies.chat.slash_commands.file
[INFO] 2025-07-14 22:35:28
Chat request started
[DEBUG] 2025-07-14 22:35:28
Authorizing GitHub Copilot token
[INFO] 2025-07-14 22:35:29
Request body file: /tmp/nvim.coolapso/mENuRr/1.json
[DEBUG] 2025-07-14 22:35:29
Request:
{ "-sSL", "-D", "/run/user/1000/plenary_curl_6c4d4de5.headers", "-X", "POST", "-H", "Content-Type: application/json", "-H", "Editor-Version: Neovim/0.11.2", "-H", "Copilot-Integration-Id: vscode-chat", "-H", "Authorization: Bearer ********", "-d", "@/tmp/nvim.coolapso/mENuRr/1.json", "--retry", "3", "--retry-delay", "1", "--keepalive-time", "60", "--connect-timeout", "10", "--tcp-nodelay", "--no-buffer", "https://api.business.githubcopilot.com/chat/completions" }
[INFO] 2025-07-14 22:35:36
Chat request finished

[coolapso]

Tried a few other models ... here's the results:

gemini-2.5-pro - Worked
o3-mini - Worked
gpt-4.1 - Broken
claude-3.7-sonnet-thought - Broken
claude-3.5-sonnet - Broken
claude-sonnet-4 - Broken
gpt-4o - Worked
gemini-2.0-flash-001 - Worked
claude-3.7-sonnet - Broken

Using, for example claude-sonnet-4 directly with anthropic adapter, works just fine. So guess there's something wrong on how codecompanion handles the llm answers via copilot?

[olimorris]

Are you triggering a content filter? That's what it seems to be saying for finish_reason

[coolapso]

What content filter could I possibly be hitting with such prompt and just the code companion defaults?

Successful response from GPT-4.1 has similar response, seems to have a bit more details tho ...

data: {"choices":[{"finish_reason":"content_filter","index":0,"content_filter_offsets":{"check_offset":3715,"start_offset":3863,"end_offset":3963},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}},"delta":{"content":",","annotations":{"TextCopyright":[{"id":0,"details":{},"citations":{}}]},"copilot_annotations":{"TextCopyright":[{"id":0,"details":{},"citations":{}}]}}}],"created":1752609443,"id":"chatcmpl-Btg4BCn73Il0ouuZe0Wx0B0q6Dajh","model":"gpt-4.1-2025-04-14","system_fingerprint":"fp_ffd5ba9d2e"}

[olimorris]

What content filter could I possibly be hitting with such prompt and just the code companion defaults?

No idea. I'm just asking the question...

It looks like CodeCompanion is doing its job as expected. There's nothing in the logs that suggest it's terminating the stream early. I'm using "https://api.individual.githubcopilot.com/chat/completions" as an endpoint whereas yours looks to be the enterprise one. Any guardrails setup that could be terminating things early?

[coolapso]

I took a quick look at the settings and I don't really have any guard rails setup other than to not allow usage of our stuff for training and to not give results that match public code,

[coolapso]

@olimorris could it be that some rate limit is being hit and the answer gets cut?