Skip to content

fix: handle GitHub Copilot gpt-4.1 model ID response bug #6106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

roomote[bot]
Copy link

@roomote roomote bot commented Jul 23, 2025

This PR fixes issue #6105 where GitHub Copilot's gpt-4.1 model returns only the model ID ("copilot/gpt-4.1") instead of generating actual AI responses.

Problem

When using the VS Code Language Model API with GitHub Copilot's gpt-4.1 model, users were seeing "copilot/gpt-4.1" as the response instead of actual AI-generated content. This appears to be a bug in either the VS Code Language Model API or the GitHub Copilot extension.

Solution

  • Added debug logging to track responses from VS Code LM API for Copilot models
  • Implemented detection logic to identify when the model ID is returned as chunk content
  • Skip chunks that contain only the model ID and continue processing valid content
  • Throw a descriptive error when the entire response consists only of the model ID
  • Added comprehensive test coverage for all edge cases

Testing

  • Added 3 new test cases specifically for this bug
  • All existing tests continue to pass
  • Verified that the workaround correctly handles:
    • Model ID followed by valid content (skips ID, returns content)
    • Only model ID returned (throws descriptive error)
    • Different variations of model ID patterns

Notes

This is a workaround for what appears to be an upstream bug in the VS Code Language Model API or GitHub Copilot extension. The fix ensures users get a clear error message instead of seeing the model ID as the response.

Fixes #6105


Important

Fixes GitHub Copilot gpt-4.1 model bug by skipping model ID responses and throwing errors for invalid content in vscode-lm.ts.

  • Behavior:
    • In vscode-lm.ts, added logic to detect and skip chunks containing only the model ID for GitHub Copilot's gpt-4.1 model.
    • Throws an error if the entire response consists only of the model ID.
  • Testing:
    • Added test cases in vscode-lm.spec.ts to verify handling of model ID responses, including variations in model ID patterns.
  • Logging:
    • Added debug logging in vscode-lm.ts to track responses and identify when model ID is returned as content.

This description was created by Ellipsis for 9c8b9e4. You can customize this summary. It will automatically update as commits are pushed.

- Added debug logging to track VS Code LM API responses for Copilot models
- Implemented detection for when model ID is returned instead of actual content
- Skip model ID chunks and continue processing valid content
- Throw descriptive error when only model ID is returned with no valid content
- Added comprehensive tests for the workaround

Fixes #6105
@roomote roomote bot requested review from mrubens, cte and jr as code owners July 23, 2025 11:52
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Jul 23, 2025
@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Jul 23, 2025
@beenycool
Copy link

il test the fixes tommorow

@daniel-lxs
Copy link
Collaborator

This could help debug but debug changes shouldn't be merged into main

@daniel-lxs daniel-lxs closed this Jul 24, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Jul 24, 2025
@github-project-automation github-project-automation bot moved this from Triage to Done in Roo Code Roadmap Jul 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Github Copilot gpt-4.1 not working
4 participants