Control XLA tempfile location

[csparker247]

I've been running my Jax code on my university's computing cluster, and I recently had some jobs fail with the following error:

2023-08-30 13:50:27.422035: F external/xla/xla/service/gpu/nvptx_compiler.cc:524] ptxas returned an error during compilation of ptx to sass: 'PERMISSION_DENIED: /tmp/tempfile-gphnode002-833ff26f-166078-60427900f4488; Read-only file system' If the error message indicates that a file could not be written, please verify that sufficient filesystem space is provided.

While the node in question was having an issue, it highlights a potential problem for our cluster which is that our nodes are configured so that temp files should be placed in user-specific scratch location. Is there a way to control at runtime the temp directory for these XLA compilation calls?