vLLM for SBERT

[david-koleckar]

Hi there,
would it be possible to add sbert model (eg. paraphrase-multilingual-mpnet-base-v2) to vLLM as described in adding a new model?
Thank you, David

[WoosukKwon]

Hi @david-koleckar, thanks for the proposal. For now, we are not planning to support SBERT model or other embedding/encoder models. vLLM is optimized for generative (auto-regressive) Transformers. Its core technologies like PagedAttention cannot be used for embedding/encoder models.