Skip to content

litellm updates to support only messages llm kwarg #1082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
Oct 21, 2024

Conversation

dtam
Copy link
Contributor

@dtam dtam commented Sep 19, 2024

this pr:

  • drops support for prompt and instructions and msg_history guard( and Guard( and Guard.from( arguments in favor of the streamlined messages argument that is now defacto for openai and other models
  • adds reask_messages support, not supplying this and having reask enabled will fall back to a message composed of our default Prompt and Instruction prompts to keep legacy support
  • drops special injection of json: xml: string: into llm input which we previously inferred and added
  • drops supports for most of the legacy Callables in favor of litellm support except for hugging face pipelines and the arbitrary callable for local inference support
  • updates message validation to be per message so that fix filter and refrain now work properly
  • updates docs
  • updates cookbook/example notebooks

needs
guardrails-ai/interfaces#13
guardrails-ai/guardrails-api#78
guardrails-ai/guardrails-api-client#26

@dtam dtam changed the title [wip]reapply litellm updates to support only messages llm kwarg litellm updates to support only messages llm kwarg Sep 27, 2024
Comment on lines +130 to +135
reask_messages: list[Dict[str, str]] = [
{
"role": "user",
"content": REASK_PROMPT,
}
],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-initializing lists in _init functions can have odd behaviors sometimes.

def foo(my_list=[]):
  my_list.append(len(my_list))
  return my_list

>>> foo()
[0]
>>> foo()
[0, 1]
>>> foo()
[0, 1, 2]

If we're modifying and appending to reask_messages internally, we should instead default this to None and check inside the method if it's None.

@zsimjee zsimjee changed the base branch from main to 0.6.0-dev October 18, 2024 21:15
@zsimjee zsimjee merged commit 730e9eb into 0.6.0-dev Oct 21, 2024
12 checks passed
@zsimjee zsimjee deleted the feature/litellm_cleanup branch October 21, 2024 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants