Inferencing on Fine Tuned Mistral 7B

[Manivarsh-adi]

Hello,

I have fine tuned mIstral 7B on a specific data and want to inference using vLLM module, is it possible?

[zspo]

Absolutely. Currently vllm supports Mistral models. As long as the model is fine-tuned on the Mistral base, it can be used directly.