Skip to content

fix: force UTF-8 encoding for stdio streams #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

iberianpig
Copy link
Contributor

@iberianpig iberianpig commented May 13, 2025

This change forces the stdin and stdout encodings to UTF-8, resolving platform-dependent issues—specifically, errors arising from handling multibyte characters (such as Japanese) under US-ASCII encoding.

Motivation and Context

On certain platforms the default encoding may be US-ASCII, which causes errors when processing multibyte characters like Japanese. By ensuring that the I/O streams always use UTF-8, it fix encoding-related issues that arise from varying default settings.

Related issues

How Has This Been Tested?

The changes were tested in an environment where the MCP server (using the Ruby SDK) was executed via llm-functions. The testing steps and results were as follows:
ref: https://github.com/sigoden/llm-functions/tree/main/mcp/bridge

  1. A server was configured using the following mcp.json settings:
{
  "mcpServers": {
    "example": {
      "command": "ruby",
      "args": [
        "/home/iberianpig/.ghq/github.com/modelcontextprotocol/ruby-sdk/mcp_demo.rb"
      ],
      "env": {
        "PATH": "/home/iberianpig/.rbenv/shims:/usr/bin"
      }
    }
  }
}

server:

#!/usr/bin/env ruby

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"
  gem "model_context_protocol", path: "/home/iberianpig/.ghq/github.com/modelcontextprotocol/ruby-sdk/"
  gem "debug"
end

require "model_context_protocol"
require "model_context_protocol/transports/stdio"

# Create a simple tool
class ExampleTool < ModelContextProtocol::Tool
  description "A simple example tool that echoes back its arguments"
  input_schema(
    properties: {
      message: { type: "string" },
    },
    required: ["message"]
  )

  class << self
    def call(message:, server_context:)
      ModelContextProtocol::Tool::Response.new([{
        type: "text",
        text: "Hello from example tool! Message: #{message}!!!!",
      }])
    end
  end
end

# Set up the server
server = ModelContextProtocol::Server.new(
  name: "example_server",
  tools: [ExampleTool],
)

# Create and start the transport
transport = ModelContextProtocol::Transports::StdioTransport.new(server)
transport.open
  1. The server was started with the following command:
$ DEBUG=mcp:* argc mcp start  
Start MCP Bridge server...  
Merge MCP tools into functions.json  
Build bin/example_example_tool
  1. The tool list was retrieved successfully:
$ curl http://localhost:8808/tools  
[{"name":"example_example_tool","description":"A simple example tool that echoes back its arguments","parameters":{"type":"object","properties":{"message":{"type":"string"}},"required":["message"]},"mcp":"example"}]
  1. Before the fix, sending a POST request with a Japanese message resulted in an error:
$ curl -X POST http://localhost:8808/tools/example_example_tool -H 'content-type: application/json' -d '{"message": "こんにちは!"}'
{"code":-32000,"name":"McpError"}$

The server log indicated the following error:

$ argc mcp logs -f
/home/iberianpig/.ghq/github.com/modelcontextprotocol/ruby-sdk/lib/model_context_protocol/transports/stdio.rb:20:in `strip': invalid byte sequence in US-ASCII (Encoding::CompatibilityError)
      from .../ruby-sdk/lib/model_context_protocol/transports/stdio.rb:20:in `open'
      from .../ruby-sdk/mcp_demo.rb:42:in `<main>'
  1. After the change, the same POST request returned a successful response:
$ curl -X POST http://localhost:8808/tools/example_example_tool -H 'content-type: application/json' -d '{"message": "こんにちは!"}'
{"code":0,"result":"Hello from example tool! Message: こんにちは!!!!!"}

Breaking Changes

None. This change does not introduce any breaking changes to the existing functionality of the MCP server or its tools.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

This change is similar to the modifications made in modelcontextprotocol/python-sdk#198.

This change forces $stdin/$stdout to use UTF-8 encoding,
similar to PR 198 in the python-sdk.
modelcontextprotocol/python-sdk#198

It addresses platform-dependent encoding issues.
@topherbullock topherbullock merged commit bdbf015 into modelcontextprotocol:main May 20, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants