-
Notifications
You must be signed in to change notification settings - Fork 613
Closed
Labels
Milestone
Description
What behavior of the library made you think about the improvement?
With all models other than exllamav2
having a single integration implementation via outlines.generate.api.SequenceGeneratorAdapter
and outlines.processors
, we don't need outlines.integrations
.
outlines.integrations.transformers
hasRegexPrefixAllowedTokens
andJSONPrefixAllowedTokens
which are now handled byoutlines.processors
outlines.integrations.llamacpp
has logits processors which are redundant withoutlines.processors
outlines.integrations.vllm
has logits processors which are redundant withoutlines.processors
outlines.integrations.utils
hasadapt_tokenizer
, useful for FSM construction, andconvert_json_schema_to_str
, used for the three model integrations listed above
After the vLLM integration, we no longer need outlines.integrations
How would you like it to behave?
Get rid of outlines/integrations/
and tests/generate/test_integration_*.py
- Remove
outlines.integrations.transformers
,outlines.integrations.llamacpp
, andoutlines.integrations.vllm
- Remove
outlines.integrations.utils
, but movedef adapt_tokenizer(...)
tooutlines.models.tokenizer
rlouf