Timing seems to have large impact on performance

[dirktheeng]

Hi all,

I am a compiler developer for the National Energy Technology Laboratory at the US Department of Energy. We are developing a new compiler that allows us to run codes with tensors as first-class citizens on multiple accelerator platforms. I just got a language to language compile working with cupynumeric/legate. I wanted to do some bench marking and followed the guide here (https://docs.nvidia.com/cupynumeric/25.07/user/howtos/benchmarking.html). However, when I did this, the code runs very slow. it seems that the timing calls introduce almost a second latency which seems unreasonable. I wanted to get some help/advice about how I have this running and why it may be so slow. I don't trust the timing results as you can visibly notice a significant slowdown in the execution. Is this a known problem with legate?

[dirktheeng]

I would actually be interested in talking to some members of your team about a larger collaboration. I sent an email to legate@nvidia.com so hopefully I get a reply soon. I like Legate and it seems very compatible with our compiler.

[Jacobfaib]

Hi @dirktheeng, sorry for the (very) late reply! Thanks for you interest in Legate, the compiler stuff sounds very cool, have you gotten responses to your emails yet? We'd love to hear more about your use-case. I'm cc'ing management and some product folks as well for vis @dongb @aterrel @manopapad

it seems that the timing calls introduce almost a second latency which seems unreasonable.

Indeed, this is pretty surprising. Are you able to share the code with us here? If so, could you attach:

1. The output of legate-issue.
2. Your benchmarked code (or some small enough reproducer of the problem).
3. Your exact runtime arguments/call command.

The timing mechanisms do add some overhead in Legate, but that overhead should be amortized and not nearly be on the order of seconds.