Validation of work complete: profiling

[lmorabit]

We are aiming to have an end-to-end non-interactive pipeline that will work on a single 8-hour dataset to produce sub-arcsecond (and intermediate and 6 arcsec resolution) images, that works in the majority of cases (i.e., a standard, non-complicated field with a decent in-field delay calibrator). We also need to understand the compute resources this needs; running the workflow and profiling all the steps will be validation that the workflow is ready, and provide information on the compute resources required.

[JonnyPierce]

Looking into doing this with pyprofqueue (https://pypi.org/project/pyprofqueue/), which was previously being used by Marcus Keil to profile various LOFAR-VLBI processing steps

[JonnyPierce]

Currently debugging issues with installation of requirements for pyprofqueue (with the likwid package)