Maybe an interesting CUDA PR here.

[Nexesenex]

Title : Overlap CUDA graph building and processing to minimize GPU idle time and improve tokens per seconds performance.
#11867
Link : ggml-org/llama.cpp#11867
Author : @aendk
Use : a few % boost on Cuda PP and TG?

[ikawrakow]

Yes, I saw this PR. But to quote Diego's statement in the PR discussion

I still think that this change adds a significant amount of complexity, to code that is already too fragile and complex to reasonably maintain.

I fully agree with that. The back-end is really fragile, so performance gains must be way more than 2-3% to warrant a change such as that one.