How to use the openai api service deployed by VLLM to use outlines

[deku0818]

I have a qwen model that runs using the vllm openai api service,
Use models.openai in outlines
But I made a mistake

[rlouf]

vLLM supports structured generation in its OpenAI API (using Outlines!). Check the documentation here: https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html

[deku0818]

vLLM supports structured generation in its OpenAI API (using Outlines!). Check the documentation here: https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html

I specified parameters, but the same error occurred,do I need to change the model name?

[deku0818]

Is there anyone who can help me answer this question?

[lapp0]

Could you please provide your full run script? If you're getting that error, it appears that you're using outlines.models.openai for a non-openai model.