RPC GPU Offload Tracing

[Allan-Luu]

Hello,

I currently have GPUs on a cluster that are receiving offloaded layers from my llama.cpp on a different host through RPC servers. I'm trying to see which process is offloading the LLM inference onto the GPUs and trace the GPU events triggered by llama.cpp.

Any advice on how to trace the GPU events triggered by RPC servers?

[rgerganov]

You can trace the calls to rpc-server by setting GGML_DEBUG to 1 here.

[Allan-Luu]

Is there an option for this to trace calls from local GPUs from host machine as well?