Criteria for evaluating ASDF implementations

[braingram]

We're starting to look at developing a set of criteria for evaluating ASDF implementations. This may involve general categories like:

• yaml header parsing (basic yaml, references)
• binary blocks (reading, writing, compression, external blocks)
• support for standard tags
• support for non-standard tags (and behavior for unknown tags)
• schema support and validation
• extension machinery (manifest file support)

One place to take inspiration is jsonschema test suite: https://github.com/json-schema-org/JSON-Schema-Test-Suite which provides a defined set of language independent tests.

We currently provide basic reference files: https://github.com/asdf-format/asdf-standard/tree/main/reference_files but little guidance on how to use these files or how they relate to the standard.

[braingram]

Commenting to raise awareness. There have been discussions of:

• evaluating the c++ implementation: https://github.com/eschnett/asdf-cxx
• a new julia implementation (to replace or perhaps add to https://github.com/eschnett/ASDF2.jl)
• possible new implementations in java, c and/or rust

Expanding the test suite and defining criteria will be helpful for any of the above work that might begin.