Skip to content

86: Add default limit for tools completions #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
e7f7a0f
feat: add limit to tools calls
rhys117 Apr 1, 2025
47c8742
docs: add docs for removing limit
rhys117 Apr 1, 2025
190e9de
docs: Fix docs
rhys117 Apr 1, 2025
fe837de
docs: improve doc - more accurate & organisation
rhys117 Apr 1, 2025
0f7ce26
bug: reset tools calls for each new 'ask' call
rhys117 Apr 1, 2025
6b807cf
bug: move error so tool result is still added
rhys117 Apr 2, 2025
df12db9
merge: main
rhys117 Apr 3, 2025
3af500d
chore: rework approach to rely on completions instead of tool calls
rhys117 Apr 4, 2025
39894a8
bug: fix attr_reader declaration to match number_of_tool_completions
rhys117 Apr 4, 2025
4718e2d
test: add better example
rhys117 Apr 7, 2025
7885c4b
docs: correct docs after changing naming/strategy
rhys117 Apr 7, 2025
585af65
merge: Merge branch 'main' into max-tool-calls
rhys117 Apr 7, 2025
df26988
docs: add default to docs
rhys117 Apr 7, 2025
19c0cd1
chore: rename error to match new naming
rhys117 Apr 7, 2025
3ad25f4
Merge branch 'main' into max-tool-calls
rhys117 Apr 24, 2025
d94d6dd
chore: reorder configuration accessor and add comment
rhys117 Apr 24, 2025
71dba53
style: cops - disable / fix
rhys117 Apr 24, 2025
6455f08
test: add spec for configured limit
rhys117 Apr 24, 2025
5a44bff
docs: adjust tools docs
rhys117 Apr 24, 2025
17b55cc
merge: main
rhys117 May 3, 2025
d09ce92
bug: ensure with_max_tool_completions available through acts_as helpers
rhys117 May 3, 2025
69af539
Merge branch 'main' into max-tool-calls
rhys117 Jun 11, 2025
018cc3b
deps: remove faker gem
rhys117 Jun 11, 2025
4cb6765
chore: use existing context instead of with_max_tool_completions
rhys117 Jun 11, 2025
6cb8ec3
test: adjust spec for context use
rhys117 Jun 11, 2025
4cdb924
chore: rename to max_tool_llm_calls
rhys117 Jun 12, 2025
f8fc8a5
docs: minor doc correction after rename
rhys117 Jun 12, 2025
731fcc6
chore: rename error class
rhys117 Jun 12, 2025
092ff2c
test: ensure spec cases for openrouter, openai, anthropic passing
rhys117 Jun 12, 2025
a0a1fa8
bug: fix +1 issue for llm tool lomts
rhys117 Jun 12, 2025
397438e
docs: improve tool docs for max tool llm calls
rhys117 Jun 12, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions docs/guides/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,46 @@ class DataAnalysis < RubyLLM::Tool
end
```

## Maximum Tools Call Limiting

When calling tools it is important to consider if you're response could trigger a loop. RubyLLM provides built-in protection against infinite tool call loops:

```ruby
# Set a maximum number of tool calls per conversation
chat = RubyLLM.chat(max_tool_calls: 5)

# The chat will stop after 5 tool calls and return an error
response = chat.ask "What's the weather in every major city?"
# => "I apologize, but I've reached the maximum number of tool calls (5) for this conversation.
# Please try breaking down your request into smaller, more focused questions."
```

### Global Configuration

You can set a default maximum tool calls limit for all chats through the global configuration:

```ruby
RubyLLM.configure do |config|
# Default is 25 calls per conversation
config.max_tool_calls = 10 # Set a more conservative limit
end
```

If you wish to remove this safe-guard you can set the max_tool_calls to `nil`.
```ruby
RubyLLM.configure do |config|
# Unlimited tools calls allowed
config.max_tool_calls = nil
end
```

This setting can still be overridden per-chat when needed:

```ruby
# Override the global setting for this specific chat
chat = RubyLLM.chat(max_tool_calls: 15)
```

## Security Considerations

When implementing tools that process user input (via the AI):
Expand Down
4 changes: 2 additions & 2 deletions lib/ruby_llm.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ module RubyLLM
class Error < StandardError; end

class << self
def chat(model: nil, provider: nil)
Chat.new(model: model, provider: provider)
def chat(model: nil, provider: nil, max_tool_calls: config.max_tool_calls)
Chat.new(model: model, provider: provider, max_tool_calls: max_tool_calls)
end

def embed(...)
Expand Down
16 changes: 14 additions & 2 deletions lib/ruby_llm/chat.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ module RubyLLM
class Chat
include Enumerable

attr_reader :model, :messages, :tools
attr_reader :model, :messages, :tools, :number_of_tool_calls

def initialize(model: nil, provider: nil)
def initialize(model: nil, provider: nil, max_tool_calls: nil) # rubocop:disable Metrics/MethodLength
model_id = model || RubyLLM.config.default_model
with_model(model_id, provider: provider)
@temperature = 0.7
Expand All @@ -23,6 +23,8 @@ def initialize(model: nil, provider: nil)
new_message: nil,
end_message: nil
}
@max_tool_calls = max_tool_calls
@number_of_tool_calls = 0
end

def ask(message = nil, with: {}, &block)
Expand Down Expand Up @@ -105,6 +107,10 @@ def handle_tool_calls(response, &)
end

def execute_tool(tool_call)
raise ToolCallsLimitReachedError, "Tool calls limit reached: #{@max_tool_calls}" if max_tool_calls_reached?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth discussing if this should be handled via the chat instead of raising an unhandled error.


@number_of_tool_calls += 1

tool = tools[tool_call.name.to_sym]
args = tool_call.arguments
tool.call(args)
Expand All @@ -117,5 +123,11 @@ def add_tool_result(tool_use_id, result)
tool_call_id: tool_use_id
)
end

def max_tool_calls_reached?
return false unless @max_tool_calls

@number_of_tool_calls >= @max_tool_calls
end
end
end
4 changes: 3 additions & 1 deletion lib/ruby_llm/configuration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,16 @@ class Configuration
:default_embedding_model,
:default_image_model,
:request_timeout,
:max_retries
:max_retries,
:max_tool_calls

def initialize
@request_timeout = 120
@max_retries = 3
@default_model = 'gpt-4o-mini'
@default_embedding_model = 'text-embedding-3-small'
@default_image_model = 'dall-e-3'
@max_tool_calls = 25
end
end
end
1 change: 1 addition & 0 deletions lib/ruby_llm/error.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ class ConfigurationError < StandardError; end
class InvalidRoleError < StandardError; end
class ModelNotFoundError < StandardError; end
class UnsupportedFunctionsError < StandardError; end
class ToolCallsLimitReachedError < StandardError; end

# Error classes for different HTTP status codes
class BadRequestError < Error; end
Expand Down
Loading