Skip to content

Document exe_prefix configuration option #578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
manodeep opened this issue May 2, 2025 · 4 comments · May be fixed by #582
Open

Document exe_prefix configuration option #578

manodeep opened this issue May 2, 2025 · 4 comments · May be fixed by #582

Comments

@manodeep
Copy link

manodeep commented May 2, 2025

I would benefit if payu had a way to prepend an arbitrary string in front of the model exe. For example, when I am running the vtune profilers, my mpirun command looks like:

mpirun <mpi-params> <aps -r aps-output-dir> <model-exe>
mpirun <mpi-params> <vtune -r vtune-output-dir -- > <model-exe>

Now, from what I can tell, there is currently no way to specify the arbitrary string <aps -r aps-output-dir> to prepend the actual exe. Would it be possible to add such an "exe-loader" feature? I could also use it to run valgrind - that would help me debug the division by zero error affecting the oneAPI ESM1.6 PI config at the moment.

Such a feature could also be used for "LD_PRELOAD" and swap out dynamic libraries at runtime.
@jo-basevi
Copy link
Collaborator

I just realised there is pre-existing exe_prefix option, which is added just before the executable.

payu/payu/experiment.py

Lines 625 to 629 in 028805b

model_prog.append(model.exec_prefix)
# Use the full path to symlinked exec_name in work as some
# older MPI libraries complained executable was not in PATH
model_prog.append(os.path.join(model.work_path, model.exec_name))

It looks like this can be configured along with the exe in the submodel configuration in config.yaml:

submodels:
    - name: atmosphere
      model: um
      exe: um_hg3.exe
      exe_prefix: <aps -r aps-output-dir>

It currently isn't documented, but if it works then we should add it in!

@manodeep
Copy link
Author

manodeep commented May 7, 2025

I am cautiously optimistic that the exe_prefix works as (not-yet) advertised. Currently, a job with exe_prefix specified as aps -r <aps-output> is running, and I will know for sure once the job finishes.

@manodeep
Copy link
Author

manodeep commented May 7, 2025

It works 😀🎉

@manodeep manodeep closed this as completed May 7, 2025
@jo-basevi jo-basevi reopened this May 8, 2025
@jo-basevi jo-basevi changed the title Adding a new keyword for prepending strings to exes Document exe_prefix configuration option May 8, 2025
@jo-basevi
Copy link
Collaborator

Just re-opening this as a reminder to update the documentation to include exe_prefix

@jo-basevi jo-basevi linked a pull request Jun 6, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add exe_prefix option to documentation ACCESS-NRI/payu
2 participants
@manodeep @jo-basevi