User story for Checklist

[SoloSynth1]

As a leader in ML research, I want to come up with a set of guidelines to improve and safeguard the industrial practices used in Machine Learning systems, so that we can guarantee the quality and the output of the systems are fully tested and can be reproducible.

To do this, I expect the said guidelines would be presented in the form of a checklist so that other ML researchers can develop a comprehensive test suite to test their own research projects.

I expect this checklist to cover every aspect in the ML end-to-end pipeline, including:
(Not complete, feel free to add)

• Data Presence
• Data Quality
• Data Ingestion
• Model Fitting
• Model Evaluation
• Artifact Testing

Also, I expect this checklist would be written in a format that can be read by both humans and machines. (e.g. YAML/TOML/JSON format)

Good to Have:

• Use bib/common reference addition method in the checklist YAML #50

[JohnShiuMK]

What is Artifact Testing, @SoloSynth1?

[SoloSynth1]

What is Artifact Testing, @SoloSynth1?

It refers to testing any artifacts created from the ML pipeline e.g. models, plots that cannot be directly tested.