Combination of Core ML and Metal

[LVCSRer]

Hello. Big thanks for community effort!

I'd like to know if my understanding is right.

With Core ML & Metal, we can have 4 combination. My understanding is

• No Core ML / No Metal : all operation on CPU
• No Core ML / Use Metal : all operation on GPU
• Use Core ML / No Metal : Encoding on ANE, others on CPU
• Use Core ML / Use Metal : Encoding on ANE, others on GPU

But not sure when I see below post. How can I control Core ML for ANE, CPU, and GPU?
#1270 (comment)

[ggerganov]

Yes, this is correct - there are 4 options as you listed them.

• No Core ML / No Metal: build with make -j, run with -ng
• No Core ML / Use Metal: build with make -j, run without -ng
• Use Core ML / No Metal: build with WHISPER_COREML=1 make -j, run with -ng
• Use Core ML / Use Metal: build with WHISPER_COREML=1 make -j, run without -ng

To control what hardware Core ML uses, you will need to edit the following source file:

https://github.com/ggerganov/whisper.cpp/blob/a3d0aa73d15fe2e076870cc7ba19cec7bbbab8fd/coreml/whisper-encoder.mm#L24-L30

Notice that Core ML can actually be configured to use the GPU instead of the ANE.

[LVCSRer]

Wow! Thanks for the answer!

[LVCSRer]

https://developer.apple.com/documentation/coreml/mlcomputeunits?changes=_8&language=objc