Using vLLM or TGI

[prashil1996]

Hi, so I have a troubling question where I'm confused to use either vLLM or TGI (https://github.com/huggingface/text-generation-inference).
As of today vLLM only supports only Decoder Models as mentioned here (https://docs.vllm.ai/en/latest/models/supported_models.html) and my use case requires me to also support certain encoder-decoder models for which I want to use TGI.

Question:

How can I distinguish that a model would work with vLLM or TGI before I am about to bring up the model itself? Is there a way that would tell me if the model would come up successfully?

Can I make use of any parameters (which would be available for all models) present in config.json file for the models to say that yes, this model would work with vLLM and not TGI, and vice-versa.

FYI: The models I'm looking to host are if HuggingFace format itself.

@wooyeonlee0 @youkaichao @simon-mo @tmm1 @zhouyuan

[simon-mo]

The architecture field in config.json should be what you want to cross check: https://docs.vllm.ai/en/latest/models/supported_models.html

[prashil1996]

Thanks