Reproducibility of plots in JOSS paper

[gdalle]

To ensure complete reproducibility of the plots in the paper, can you please:

• specify the complete environment where the experiments are run, with a project and a manifest
• explain how the JSON files pca_data.json and epca_data.json are obtained
• create the random generator with StableRNGs.jl to fix it across Julia versions

[FlyingWorkshop]

working on this now

[FlyingWorkshop]

Done :)

Changes:

• made project/manifest in scripts folder
• the jsons are obtained from this script
• RNG uses StableRNG
• minor parameter tweaks to make graphs look nice w/ new seed

[FlyingWorkshop]

Let me know what you think @gdalle

[gdalle]

if you can also turn the jupyter notebook into a script + integrate the new plots into the paper, I'll test it again tomorrow and then we'll be able to close this

[FlyingWorkshop]

Here is a script. The new plot is in the paper too.

[gdalle]

The script doesn't seem to run because x is not defined on line 33. Moreover, you should probably use relative paths with e.g. @__DIR__ so that the saving location doesn't depend on where the REPL was launched.

[FlyingWorkshop]

That's so strange, it ran fine when I was running it (perhaps I accidentally ran it twice and didn't notice it failed the first time, so that x was defined in the REPL the second run). I've updated the script and path dependencies so that it should run now (I just removed the path prefixes).

The script should create several images (mirroring the original notebook), but only the last image combo.png matters for the paper.

[FlyingWorkshop]

@gdalle, I cleaned up the plotting script significantly. The whole pipeline now runs from a single file and there are no extraneous images/JSONs. This should hopefully be very easy to test.

[gdalle]

perfect, thanks!