Skip to content

[FEATURE] Allow specifying of the max output tokens when making a completion #274

Closed
@thedumbtechguy

Description

@thedumbtechguy

Scope check

  • This is core LLM communication (not application logic)
  • This benefits most users (not just my use case)
  • This can't be solved in application code with current RubyLLM
  • I read the Contributing Guide

Due diligence

  • I searched existing issues
  • I checked the documentation

What problem does this solve?

When making completions, sometimes we want to constrain the output tokens. Similar to how we control temperature.
Right now, there is no way to do that.

Proposed solution

Add a new parameter similar to temperature.

Why this belongs in RubyLLM

This requires changes to all the underlying providers and the interface in order to work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions