Skip to content

Conversation

@laviphon
Copy link

@laviphon laviphon commented Feb 27, 2025

  • Added tool_format to the db, this is a boolean that changes whether or not functions are sent to the endpoint via tools or functions. True means it sends as tools, false means it sends as functions.
  • calls.py was only sending functions if the model was an official OAI model. I added a few checks and dropped the function check code out of the OAI model check as to hopefully not break compatibility.
  • I tested with functions on and off, tool_format on and off, made sure ::assist had no embedding errors, and made sure that function_calling still worked of course. I do not have access to an OAI endpoint so I'm not able to test that directly, but I do not believe that any changes made are breaking.
  • I tested changes using Ollama and LM Studio and was able to get function calling working very quickly and painlessly.

@laviphon laviphon closed this Feb 27, 2025
@laviphon
Copy link
Author

Accidentally pushed this to the wrong repo.

I will submit this again after testing.

@laviphon laviphon reopened this Feb 28, 2025
@laviphon laviphon changed the title this should allow custom endpoints to use functions right out of the box Allow for easier function configuration for custom endpoints. Feb 28, 2025
@laviphon
Copy link
Author

Made some more changes and tested everything. All seems to be in working order so I went and re-opened the PR.

@vertyco
Copy link
Owner

vertyco commented Apr 7, 2025

There have been multiple changes since this PR was made, please resolve any conflicts. Additionally the minor version number will need to be bumped for this pr since it is adding a new command

@vertyco vertyco requested a review from Copilot June 7, 2025 01:03
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new configuration option to control how functions are sent to custom endpoints by adding a tool_format flag in the DB configuration and refactoring function call logic accordingly.

  • Adds the "tool_format" field to the DB model for selecting the output format.
  • Refactors the logic in calls.py to conditionally send functions via tools.
  • Adds an admin command to toggle the tool_format option.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
assistant/common/models.py Added the new "tool_format" field to the DB model.
assistant/common/calls.py Updated function call logic to conditionally use tool formatting.
assistant/common/api.py Passes the DB model to the request_chat_completion_raw function.
assistant/commands/admin.py Introduces a command to toggle the tool_format setting via admin command.

Comment on lines +73 to +75
if model not in NO_DEVELOPER_ROLE or db.endpoint_override:

if model in SUPPORTS_TOOLS or db.tool_format:
Copy link

Copilot AI Jun 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before accessing properties on 'db', such as 'db.endpoint_override' and 'db.tool_format', consider adding a guard to ensure 'db' is not None to prevent potential attribute errors.

Suggested change
if model not in NO_DEVELOPER_ROLE or db.endpoint_override:
if model in SUPPORTS_TOOLS or db.tool_format:
if model not in NO_DEVELOPER_ROLE or (db is not None and db.endpoint_override):
if model in SUPPORTS_TOOLS or (db is not None and db.tool_format):

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants