DD Calibration: final validation step

[jurjen93]

Add a final validation step to the DD-calibration after all selfcals have finished.

[lmorabit]

Using what @r-timmerman made in the hackathon to get noise for solutions - need to test this on large samples and see if thresholds work. then see if it can be merged with the DI validation so only use one script.

the aim at the moment is just to assess continue / stop (future work in a separate ticket can come back to make automated decisions on what to do, possibly involving loops)

[jurjen93]

New branch for implementation in pipeline: https://git.astron.nl/RD/VLBI-cwl/-/tree/ddcal_validation

[jurjen93]

Ran updated workflow successfully. Testing now the validation scripts on data from different observations.

[jurjen93]

Working on a related issue: The neural network for image validation needs internet, even though it is cashed locally -> sara-nl/cortExchange#2

[JonnyPierce]

Images for inspection made, with relevant image quality metrics shown on them. Current "accept" vs "reject" decision also shown based on preliminary thresholds determined by @jurjen93 from previous image inspection. Some examples below.

Images metrics so far determined for images from 10 selfcal cycles for 46 sources from @jurjen93's ELAIS-N1 LOFAR-VLBI processing.

Next step is to look at the outputs based on the current thresholds for "accept" vs "reject" to assess the quality of the current metrics, and alter those as necessary. @jwpetley has also provided several hundred example DD cal images from EDF-N to use for further testing.

[jurjen93]

New ddcal_validation workflow ran successfully from start to finish with cwltool.
We currently only have to focus on the validation python scripts.

[jurjen93]

Updated the validation script by adding similar metrics as @r-timmerman came up with during the hackathon in May2025:
https://git.astron.nl/RD/VLBI-cwl/-/blob/ddcal_validation/scripts/validate_lofar_sols.py
Script needs a DI and DD mode.

[lmorabit]

Updated the validation script by adding similar metrics as @r-timmerman came up with during the hackathon in May2025: https://git.astron.nl/RD/VLBI-cwl/-/blob/ddcal_validation/scripts/validate_lofar_sols.py Script needs a DI and DD mode.

The number of wraps need to be normalised like this, and use the freq axis instead of the time axis in the sum:

phase_freq_diff = ( np.diff(phase_sols, axis=axes.index('freq')) - np.pi ) % (np.pi2) - np.pi
freqsum = np.nansum(phase_freq_diff/(2np.pi), axis=axes.index('freq') )
abssum = np.abs(freqsum)
wrap_count = np.max(abssum)

otherwise it calculates way too many phase wraps!

[jurjen93]

Image validation scores already look good for ELAIS and Bootes. @JonnyPierce and I will do a final inspection tomorrow and tweak the validation thresholds if necessary and then run it on EDFN images.

[JonnyPierce]

Image validation script currently being run on EDF-N images, so will also be available to inspect tomorrow to help decide on final threshold tweaks

[jurjen93]

@JonnyPierce and I went through the image validation selection and we are happy with the results. So, we leave the thresholds as they are. And now fully focus on calibration solution validation.