[Question]: Linking onnxruntime, shared library hell on MacOs

[rdcm]

Hi,

Right now I'm working on porting a Python service that uses silero-vad and sherpa-onnx to Rust. For model inference in the Rust version of the application, I’m using:

• https://github.com/pykeio/ort
• https://github.com/thewh1teagle/sherpa-rs

Example of how the libraries are included:

sherpa-rs = { version = "0.6.6", features = ["sys", "download-binaries" ] }
ort = { version = "=2.0.0-rc.9", features = ["ndarray", "cuda", "load-dynamic", "download-binaries"] }

At the moment, I’ve run into an issue that could be described as “shared library hell,” as mentioned in the documentation:

1. sherpa-rs copies its own version of libonnxruntime.dylib (version 1.17) into target/debug.

2. ort apparently detects that libonnxruntime.dylib is already loaded and doesn’t try to load its own.

3. At runtime, when I call the ort API, I get the error: "The requested API version [20] is not available, only API versions [1, 17] are supported in this build".

This brings me to a couple of questions:

1. Where does ort get libonnxruntime.dylib from, and why doesn’t it appear in target/debug even when I’m only using ort?
   My current hypothesis is that ort looks it up from somewhere in the cache:
   /Library/Caches/ort.pyke.io/dfbin/x86_64-apple-darwin/{hash}/onnxruntime/lib/libonnxruntime.a.

2. What is the recommended way to configure ort in my case?
   So far, it seems like I need to manually download a matching ONNX Runtime distribution and specify its path using ORT_DYLIB_PATH.

Thanks

[decahedron1]

1. Where does ort get libonnxruntime.dylib from, and why doesn’t it appear in target/debug even when I’m only using ort?

ort does not provide dylibs on macOS, it links to ONNX Runtime statically (in this case, though, it doesn't link at all since load-dynamic is also enabled). The libonnxruntime.dylib is coming from sherpa.

2. What is the recommended way to configure ort in my case?

I'd say just override sherpa's libonnxruntime.dylib with a newer version.

[rdcm]

I'd say just override sherpa's libonnxruntime.dylib with a newer version.

My current solution:

1. Upload latest bin files to libs folder from onnxruntime release page.
2. Copy libonnxruntime.1.21.0.dylib to target/debug via custom build.rs script
3. Enable load-dynamic feature
4. Set ORT_DYLIB_PATH to target/debug/libonnxruntime.1.21.0.dylib

Are there any other solutions to my problem? I would like ort to work with its own version of the library, and sherpa-rs with its own.

[decahedron1]

I don't know if sherpa uses dlopen like ort's load-dynamic does, or if it is using compile-time dynamic linking. If it is, then another solution would be to let ort use its static libraries by disabling load-dynamic, then extracting any logic using sherpa-rs to its own crate which is linked as a dylib (or the other way around, instead splitting out ort logic). That way, the sherpa code & ort code are in completely separate binaries, and the libraries they link to don't interfere.

I would like ort to work with its own version of the library, and sherpa-rs with its own.

Why?

[csukuangfj]

FYI: To build sherpa-onnx with a different version of onnxruntime, please have a look at
k2-fsa/sherpa-onnx#1719