Execute intermediate data wrapper in output recon file naming tests #578
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #577
Based on information in the hdf5 docs on file locks saying that the locks are not needed when running multiple processes working together (such as when running processes using MPI), I had noted that running the 6 offending tests in
test_save_intermediate_data
undermpirun -n 1
never produced the error, and chalked it off as some anomaly where those tests needed to be run under MPI to pass locally.However, since then, I noticed that those offending tests only ever create the hdf5 file for saving, and the
execute()
method on the wrapper object is never called.This test is parameterised to have 4 of the 6 failing cases:
httomo/tests/method_wrappers/test_save_intermediate.py
Lines 340 to 353 in c0036b9
and this test is parameterised with 2 of the 6 failing cases:
httomo/tests/method_wrappers/test_save_intermediate.py
Lines 401 to 414 in c0036b9
where the absence of
wrp.execute()
should be noted.The
execute()
method is what actually closes the hdf5 file:httomo/httomo/method_wrappers/save_intermediate.py
Lines 94 to 95 in c0036b9
This was confirmed to indeed be the issue, where simply running the
execute()
method on the intermediate data wrapper objects makes the hdf5 file locks not appear anymore when running tests locally.Checklist