Help Needed: Exporting kokoro.onnx to CoreML .mlmodel

[abdussamad0]

I’m working on deploying the kokoro.onnx model from the Sherpa-ONNX project in an iOS application and have encountered challenges converting it into a Core ML .mlmodel or .mlpackage.

My primary goal is to run the model using Metal on A15+ devices for improved performance and reduced thermal load. Unfortunately, using ONNX directly with onnxruntime and the coreml execution provider has not resulted in any GPU utilization. In practice, it still runs on the CPU, leading to higher inference times, significant heat buildup, and accelerated battery drain.

Here’s what I’ve attempted so far:
• coremltools.convert() with source="pytorch" and source="onnx" → both unsupported or fail with missing framework issues
• onnx_coreml.convert() fails due to deprecated coremltools.converters.nnssa dependencies
• A PyTorch traced model using dummy input tensors → fails Core ML type inference or triggers issues with unsupported ops

Could you please help with:
1. A working method to convert kokoro.onnx (non-quantized) into .mlmodel or .mlpackage format
2. The correct toolchain setup (Python version, coremltools version, any patches)
3. Any known limitations with exporting Kokoro models to Core ML
4. Whether Sherpa-ONNX will offer Core ML native support or Metal-tuned models in the future

Any guidance, best practices, or even a minimal working example would be greatly appreciated. Sherpa-ONNX is a fantastic project and I’m excited to bring its capabilities to iOS—but ideally in the most optimized, user-friendly, and battery-efficient way.

[csukuangfj]

It depends on onnxruntime. We use onnxruntime inside.

Do you happen to know if anyone has managed to run kokoro with coreml?

[abdussamad0]

Thanks! I’m not aware of anyone who’s run Kokoro with Core ML. My main goal is to get GPU acceleration on iOS via Metal—running with onnxruntime + coreml EP still uses CPU and causes high latency and heat.

[csukuangfj]

If you know anyone can use coreml with onnxruntime for kokoro tts, please let us know. We can port it to sherpa-onnx.

Otherwise, I am afraid we cannot help you.

[abdussamad0]

Thanks for the clarification! Totally understand—if I come across anyone who’s managed to use Core ML with ONNXRuntime for Kokoro TTS, I’ll definitely let you know.

Just to add: even when setting provider: "coreml" in ONNXRuntime, it still seems to run on the CPU, uses more RAM, and actually slows down the RTF. Would love any tips if there’s a way to convert Kokoro to a true .mlmodel or .mlpackage that fully takes advantage of Metal.

Really appreciate all the work on Sherpa-ONNX—it’s been awesome to work with!