How to Analyze Operator Latency Breakdown in llama.cpp?

[Zijie-Tian]

I am currently looking to analyze the latency of operators in a model (e.g., a breakdown of operator time proportions). I know that test-backend-ops can be used to test individual operators, but I’d like to know if there is a tool in llama.cpp that can measure the latency proportions of all operators in a specific model. Or, would I need to implement such a tool myself?

[max-krasnyansky]

Take a look at #9659
Currently it supports only the CPU backend but will give you the info you're looking for.
I'm planning on updating it to the latest master and to add support for the OpenCL backend (and hopefully others one later).