Skip to content

Conversation

@roomote
Copy link

@roomote roomote bot commented Oct 25, 2025

Description

This PR fixes an issue where LM Studio was receiving images with data URL prefixes (data:image/png;base64,) when it expects raw base64 encoded data only.

Problem

When using the Web Browser Tool with vision-capable models in LM Studio, users were encountering the error:
[Server Error] 'url' field must be a base64 encoded image

Solution

  • Added an optional lmStudioFormat parameter to convertToOpenAiMessages function
  • When enabled, this parameter strips the data URL prefix from images
  • Updated the LM Studio handler to use this new parameter
  • Maintains backward compatibility - default behavior unchanged for other providers

Testing

  • Added comprehensive test coverage for both default and LM Studio formats
  • All existing tests pass
  • New test verifies the conditional behavior works correctly

Related Issue

Fixes #8827

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective
  • New and existing unit tests pass locally with my changes

Important

Adds lmStudioFormat parameter to handle base64 images without data URL prefix for LM Studio in convertToOpenAiMessages.

  • Behavior:
    • Adds lmStudioFormat parameter to convertToOpenAiMessages in openai-format.ts to strip data URL prefix from base64 images.
    • Updates LmStudioHandler in lm-studio.ts to use lmStudioFormat parameter.
    • Maintains backward compatibility; default behavior unchanged for other providers.
  • Testing:
    • Adds tests in openai-format.spec.ts for both default and LM Studio formats.
    • Verifies conditional behavior for image data formatting.

This description was created by Ellipsis for 629830d. You can customize this summary. It will automatically update as commits are pushed.

LM Studio expects raw base64 encoded image data without the data: URL prefix.
This fix adds an optional lmStudioFormat parameter to convertToOpenAiMessages
that strips the prefix when sending images to LM Studio.

Fixes #8827
@roomote roomote bot requested review from cte, jr and mrubens as code owners October 25, 2025 22:48
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Oct 25, 2025
@roomote
Copy link
Author

roomote bot commented Oct 25, 2025

Review Complete

No issues found. The implementation correctly handles LM Studio's requirement for raw base64 image data by:

  • Adding an optional lmStudioFormat parameter to convertToOpenAiMessages
  • Conditionally stripping the data URL prefix when enabled
  • Maintaining backward compatibility for all other providers
  • Including comprehensive test coverage for both formats

The fix is well-isolated and properly tested.

Follow Along on Roo Code Cloud

@dosubot dosubot bot added the bug Something isn't working label Oct 25, 2025
Copy link
Author

@roomote roomote bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found.

@hannesrudolph hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Oct 25, 2025
@MarkErik
Copy link

Now there is a different error in LM Studio: [Server Error] Invalid url

2025-10-25 19:10:07 [DEBUG]
Received request: POST to /v1/chat/completions with body {
"model": "qwen/qwen3-vl-30b",
"messages": [
{
"role": "system",
"content": "You are Roo, a highly skilled software engineer wi... ...user gives you instructions below to do otherwise."
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "\nCan you check if my website at https://google.com is displaying correctly?\n"
},
{
"type": "text",
"text": "<environment_details>\n# VSCode Visible Files\n\n\n# V... ...or involves multiple steps.\n</environment_details>"
}
]
},
{
"role": "assistant",
"content": "<browser_action>\nlaunch\nhttps://google.com\n</browser_action>"
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "[browser_action for 'launch'] Result:"
},
{
"type": "text",
"text": "The browser action has been executed. The console ... ...rowser before you can use the write_to_file tool.)"
},
{
"type": "image_url",
"image_url": {
"url": "UklGRt5aAABXRUJQVlA4WAoAAAAgAAAAgwMAVwIASUNDUMgBAA... ...p5IKVQI3kvAwpRJ8MxVMmXCBzK/ym+PVWjcCCVIAAAAAAAAAA="
}
},
{
"type": "text",
"text": "<environment_details>\n# VSCode Visible Files\n\n\n# V... ...or involves multiple steps.\n</environment_details>"
}
]
}
],
"temperature": 0,
"stream": true
}
2025-10-25 19:10:07 [ERROR]
[Server Error] Invalid url

@MarkErik
Copy link

I have tracked down the issue - LMStudio wants images in PNG format, not webP.

I'll submit a PR with a fix for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. size:M This PR changes 30-99 lines, ignoring generated files.

Projects

Status: Triage

Development

Successfully merging this pull request may close these issues.

[BUG] Error when sending image to LMStudio with Web Browser Tool

3 participants