Periodic anomalies occur when using MPI for optimization within the Simsopt framework.

[DMCXE]

Using an M1 MacBookpro with the Conda-managed vmecpp environment, I modified the QH optimization script from the examples in Simsopt-1.8.3.dev8+g4f263c82to perform a two-period QA optimization and added an iota target (which does not fundamentally alter the structure).
I set ngroups=4 and launched the optimization using: mpirun -n 4 python script.py
After the first successful run, which produced a good structure,

NUMBER OF JACOBIAN RESETS = 2
Final aspect ratio: 8.000046602537445
Quasisymmetry objective after optimization: 0.00016034825260331976
Total objective after optimization: 0.00016046050989974213
End of 2_Intermediate/QH_fixed_resolution.py
============================================

Then I modified the script (e.g., changing the iota target value). When running the script again immediately after modification, I encountered the following error: (Error message follows…)

Traceback (most recent call last):
  File "/Users/dmcxe/Downloads/vmecfree/simsopt_qh_fixed_resolution.py", line 27, in <module>
    vmec = Vmec(str(filename),keep_all_files=True)
  File "/Users/dmcxe/miniforge3/envs/vmecpp/lib/python3.10/site-packages/vmecpp/cpp/vmecpp/vmec/pybind11/simsopt_vmecpp.py", line 145, in __init__
    with simsopt_compat.ensure_vmecpp_input(Path(filename)) as vmecpp_filename:
  File "/Users/dmcxe/miniforge3/envs/vmecpp/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/Users/dmcxe/miniforge3/envs/vmecpp/lib/python3.10/site-packages/vmecpp/cpp/vmecpp/simsopt_compat/_vmec_indata_compat.py", line 45, in ensure_vmecpp_input
    os.remove(vmecpp_input_path)
FileNotFoundError: [Errno 2] No such file or directory: 'nfp4_QH_warm_start.json'

After that, I started writing this issue to report the above error, which took about four minutes.
By the time I ran the script again, it started the optimization normally.

Final aspect ratio: 8.000021857485418
Quasisymmetry objective after optimization: 7.619638433144436e-05
Total objective after optimization: 7.633131165844296e-05
End of 2_Intermediate/QH_fixed_resolution.py
============================================

This issue does not occur if the script is not launched using mpirun. Has this issue occurred in other tests, or is it due to a problem with the local system environment?

[eguiraud-pf]

Hi @DMCXE , thank you for reaching out! this is not a known issue. If I had to guess it's a race condition between the different processes which apparently try to remove the same file. I'm creating an issue to track this and we'll take a look as soon as possible.

[eguiraud-pf]

Likely, using a VMEC++ JSON input file instead of a classic VMEC input.* input file works around the issue.

You can do the conversion to JSON using the (currently private) function _util.indata_to_json.

EDIT: although I'm not sure whether SIMSOPT might complain about that...we'll try to get the underlying issue fixed as soon as possible.

[eguiraud-pf]

#119 should address this if my diagnosis is correct

[eguiraud-pf]

@DMCXE the patch is in. please try out VMEC++ with the latest main branch and let us know whether you still encounter similar problems!

[eguiraud-pf]

making this as answered, please let us know if you encounter further problems