Skip to content

Port HuggingChat prompt templates to transformers tokenizers #978

@nsarrazin

Description

@nsarrazin

We added the option to use the tokenizer field to determine the chat template. (PR)

However we couldn't port the ones used in prod because we need to support system prompts. One way we could handle it would be:

  1. Try to apply the chat template
  2. Catch any error being thrown, if one is thrown get rid of the system prompt, inject it directly in the first user message and try to apply the chat template again.
  3. If an error is thrown again well something is wrong with the chat template 😅

Haven't tested it yet with our prod config though.

Metadata

Metadata

Assignees

No one assigned

    Labels

    huggingchatFor issues related to HuggingChat specifically

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions