DLL load failed while importing onnxruntime_pybind11_state

[exdysa]

Describe the bug

try to run python webui.py --port 50000 --model_dir pretrained_models/CosyVoice-300M
from FunAudioLLM#1075 (comment)

got

(cosyvoice) PS D:\cosyvoice> python webui.py --port 50000 --model_dir pretrained_models/CosyVoice-300M

Traceback (most recent call last): File "D:\cosyvoice\webui.py", line 25, in from cosyvoice.cli.cosyvoice import CosyVoice, CosyVoice2 File "D:\cosyvoice\cosyvoice\cli\cosyvoice.py", 
ine 21, in from cosyvoice.cli.frontend import CosyVoiceFrontEnd File "D:\cosyvoice\cosyvoice\cli\frontend.py", line 17, in import onnxruntime File "C:\Users\bb\miniconda3\envs\cosyvoice\lib\site-packages\onnxruntime__init__.py",
line 61, in raise import_capi_exception File "C:\Users\bb\miniconda3\envs\cosyvoice\lib\site-packages\onnxruntime__init__.py",
line 24, in from onnxruntime.capi._pybind_state import ( File "C:\Users\bb\miniconda3\envs\cosyvoice\lib\site-packages\onnxruntime\capi_pybind_state.py",
line 32, in from .onnxruntime_pybind11_state import * # noqa
ImportError: DLL load failed while importing onnxruntime_pybind11_state: A dynamic link library (DLL) initialization routine failed.

looks like onnx is expecting something here. This is with an Intel Arc A770 using XPU. Presumably this means we need XPU provider.

initial research:

FunAudioLLM#893
https://github.com/eighteen-k-gold-malow/CosyVoice-XPU/commit/05bab70168b92976d70ffbe647e7b22762cfea86

which reveals.... directml too???
line 99 export_onnx
providers = ['DmlExecutionProvider']

interesting. though this is an immediate fail in webui, it ought to be presumed that it needs to be changed all the way down the pipe... this xpu fork seems to be very involved, will try to manage this tomorrow
https://github.com/eighteen-k-gold-malow/CosyVoice-XPU/commits/main/

may need to also add MPS providers????

[bli1348]

I knew the cosyvoice-xpu repo, the reason I chose yours is because his ver involve a lot of onxx conversion with directml. yours simply throw onxx to cpu, which is what origin did, slower but simpler. Didn't you have this code working with your intel GPU?

[bli1348]

i rollbacked to onnxruntime==1.15.1, no more dll error , but webui still didn't start server.
(cosyvoice) PS D:\cosyvoice> python webui.py --port 50000 --model_dir pretrained_models/CosyVoice-300M-Instruct failed to import ttsfrd, use WeTextProcessing instead
C:\Users\bb\miniconda3\envs\cosyvoice\lib\site-packages\diffusers\models\lora.py:393: FutureWarning: LoRACompatibleLinear is deprecated and will be removed in version 1.0.0. Use of LoRACompatibleLinear is deprecated. Please switch to PEFT backend by installing PEFT: pip install peft.
deprecate("LoRACompatibleLinear", "1.0.0", deprecation_message)
2025-04-27 15:52:16,478 INFO input frame rate=50
(cosyvoice) PS D:\cosyvoice>

[exdysa]

Didn't you have this code working with your intel GPU?

I don't have an intel GPU. I have admittedly never run CosyVoice. I always work with PyTorch and AI, and I do have MLX. I like to help people, and it is code that I know by heart. I do intend to use it in the future for my own software, when that is ready..

I can see what you mean about DML in the eighteen-k code.. I don't know the logic behind that, and they haven't answered my PR yet. I am glad you see the benefit of the plain method.

I will try to find where this stopped, if there is a more verbose diagnostic console log method it would help, this is sadly not too much to go on. I can look into the eighteen-k repo also for ideas on a bypass.

Did you try switching to peft backend, or maybe installing ttsfrd as indicated? I presume you have https://www.intel.com/content/www/us/en/developer/articles/tool/pytorch-prerequisites-for-intel-gpu/2-7.html#driver-installation already.

It also is documented that you should change input_frame_rate to 25 in the appropriate .yaml file ( these https://github.com/search?q=repo%3Aexdysa%2FCosyVoice+path%3Ayaml&type=code ) when you use 300m training. This could be another point of failure.

[bli1348]

update
I gave up when i tried to run cosyvoice inference in cli the system crashed. But with the eighteen-k-gold, it does work with xpu on webui. Maybe you can use his code as a reference to make xpu work on the latest cosyvoice

[exdysa]

Since the initial DLL issue is resolved I'll close the issue, but I'm focusing on everything else mentioned today