Initial Mistral implementation #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

OriPekelman wants to merge 2 commits into crmne:main from OriPekelman:mistral

OriPekelman commented Mar 18, 2025

Tried to follow existing implementation as much as possible and hopefully integrated correctly.

OriPekelman mentioned this pull request

Include Mistral AI #7

Open

crmne added the new provider label

crmne linked an issue

that may be closed by this pull request

Include Mistral AI #7

Open

Owner

crmne commented Mar 23, 2025

Thank you @OriPekelman for your contribution!

Two new features to consider in your provider implementation:

Model Aliases: Please add entries for your provider in aliases.json:

"claude-3-5-sonnet": {
  "anthropic": "claude-3-5-sonnet-20241022",
  "your-provider": "your-provider-specific-id"
}

Provider Selection: Users will be able to specify your provider:

chat = RubyLLM.chat(model: 'claude-3-5-sonnet', provider: 'your-provider')

Docs: https://rubyllm.com/guides/models#using-model-aliases

Please ask me for a code review when you're ready.

Owner

crmne commented Mar 25, 2025

Added configuration requirements handling in 75f99a1

Each provider now specifies what configuration is required via a simple configuration_requirements method (you will need to implement this in your main provider file) that returns an array of config keys as symbols. The Provider module uses this to:

Determine if a provider is properly configured
Throw an error if you're trying to use that provider without configuration
Include ready-to-paste configuration code in the error message
Skip unconfigured providers during model refresh while preserving their models

Example of the new error messages:

RubyLLM::ConfigurationError: anthropic provider is not configured. Add this to your initialization:

RubyLLM.configure do |config|
  config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
end

Owner

crmne commented Apr 17, 2025

@OriPekelman is this still on your radar? I'd love to merge Mistral support soon. Whenever you are ready, could you resolve the conflicts and rerequest a review? Thanks!

Author

OriPekelman commented Apr 17, 2025

Yup, I'll try to get to it very soon.

Owner

crmne commented Apr 23, 2025

Hi @OriPekelman is Mistral OpenAI compatible?

OriPekelman added 2 commits

April 25, 2025 23:33


          Base mistral implementation

0df2751


          feat(mistral): Enhance Mistral provider with new models and API integ…

883ecfc

…ration

- Updated model aliases in aliases.json to include specific paths for Mistral models.
- Added `mistral_api_base` configuration option in configuration.rb.
- Introduced new models in models.json, including Mistral Tiny, Pixtral Large, and others.
- Refactored Mistral provider methods to support new API structure and error handling.
- Removed deprecated image generation methods from the Mistral provider.
- Updated tests to reflect changes in model handling and API responses.

OriPekelman force-pushed the mistral branch from 53be515 to 883ecfc Compare

April 27, 2025 21:12

Author

OriPekelman commented Apr 27, 2025

Hi @OriPekelman is Mistral OpenAI compatible?

Well, like most providers, it has some basic OpenAI compatability ... but only for chat completions - and even there you are going to have quite a bit of differences in tool calling and multi-turn behaviour.

I rebased the whole thing and cleaned it up quite a bit + added the configure.

There are a number of tests I had to skip (for example Mistral does not support custom dimensions for embedding, the tool calling in multi-turn conversations doesn't support adding tools mid conversation, only the last chunk in a stream would have token count).

Hopefully this is useful.

crmne requested changes

View reviewed changes

Owner

crmne left a comment

Good stuff! I left you some comments

lib/ruby_llm/providers/mistral/chat.rb

Comment on lines +8 to +10

+                      def completion_url
+                        "#{Mistral.api_base(RubyLLM.config)}/chat/completions"
+                      end

Owner

crmne Apr 28, 2025

you don't need to pass the api base here

lib/ruby_llm/providers/mistral/chat.rb

+                            Array(tools)
+                          end
+                        puts "\n[DEBUG] Available tools: #{tools_array&.map { |t| t.name.to_s }}" if ENV["DEBUG"]

Owner

crmne Apr 28, 2025

Use RubyLLM.logger.debug

lib/ruby_llm/providers/mistral/chat.rb

+                            "none"
+                          end
+                        puts "[DEBUG] Tool choice: #{effective_tool_choice.inspect}" if ENV["DEBUG"]

Owner

crmne Apr 28, 2025

same here

lib/ruby_llm/providers/mistral/chat.rb

+                          frequency_penalty: frequency_penalty,
+                        }.compact
+                        puts "[DEBUG] Full payload: #{payload.inspect}" if ENV["DEBUG"]

Owner

crmne Apr 28, 2025

same here

lib/ruby_llm/providers/mistral/chat.rb

+                            arguments: tool_call.arguments,
+                          },
+                        }
+                        puts "[DEBUG] Rendered tool call: #{tool_call_spec.inspect}" if ENV["DEBUG"]

Owner

crmne Apr 28, 2025

same here

spec/ruby_llm/chat_spec.rb

@@ @@ -31,6 +31,7 @@ @@
                     it "#{provider}/#{model} successfully uses the system prompt" do
                       skip 'System prompt can be flaky for Ollama models' if provider == :ollama
+                      skip 'Mistral API does not allow system messages after assistant messages' if provider == :mistral

Owner

crmne Apr 28, 2025

but in this case the chat should start with the system message, right?

spec/ruby_llm/chat_streaming_spec.rb

Comment on lines +29 to +30

		skip 'Mistral API only returns token count on the last chunk.' if provider == :mistral

Owner

crmne Apr 28, 2025

also the other APIs, so don't worry

spec/ruby_llm/chat_tools_spec.rb

@@ @@ -36,6 +36,7 @@ def execute @@
                     model = model_info[:model]
                     provider = model_info[:provider]
                     it "#{provider}/#{model} can use tools" do # rubocop:disable RSpec/MultipleExpectations
+                      skip 'Mistral does not reliably support tool usage' if provider == :mistral

Owner

crmne Apr 28, 2025

really? that would be super bad! in what way is it not reliable?

spec/ruby_llm/embeddings_spec.rb

@@ @@ -23,6 +27,7 @@ @@
                     end
                     it "#{provider}/#{model} can handle a single text with custom dimensions" do # rubocop:disable RSpec/MultipleExpectations
+                      skip "Mistral embed does not support custom dimensions" if model == "mistral-embed"

Owner

crmne Apr 28, 2025

you can do if provider == :mistral

spec/ruby_llm/embeddings_spec.rb

@@ @@ -38,6 +43,7 @@ @@
                     end
                     it "#{provider}/#{model} can handle multiple texts with custom dimensions" do # rubocop:disable RSpec/MultipleExpectations
+                      skip "Mistral embed does not support custom dimensions" if model == "mistral-embed"

Owner

crmne Apr 28, 2025

same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels