Body encoding in HTTP protocol compliance tests

[david-perez]

So far, all protocols supported by Smithy use text-based serialization formats. However, in protocols using a binary-based serialization format (like the upcoming Smithy RPC v2 CBOR protocol), the binary body field in HTTP protocol compliance tests needs to be encoded in a text-based format, since the protocol tests are written in regular .smithy files.

This makes it onerous for a human to maintain and review the tests. For example, the following @httpRequestTests test:

{
    id: "RpcV2CborDeserializesDenseSetMapAndSkipsNull",
    documentation: """
        Clients SHOULD tolerate seeing a null value in a dense map, and they SHOULD
        drop the null key-value pair.""",
    protocol: rpcv2Cbor,
    appliesTo: "client",
    code: 200,
    body: "v2xzcGFyc2VTZXRNYXC/YXif/2F5n2FhYWL/YXr2//8=",
    bodyMediaType: "application/cbor",
    headers: {
        "smithy-protocol": "rpc-v2-cbor",
        "Content-Type": "application/cbor"
    },
    params: {
        "denseSetMap": {
            "x": [],
            "y": ["a", "b"]
        }
    }
}

has a base64-encoded body field representing a CBOR map that in annotated hex looks like (I used https://crates.io/crates/cbor-diag/ to convert to this representation):

bf                             # map(*)
   6c                          #   text(12)
      7370617273655365744d6170 #     "sparseSetMap"
   bf                          #   map(*)
      61                       #     text(1)
         79                    #       "y"
      9f                       #     array(*)
         61                    #       text(1)
            61                 #         "a"
         61                    #       text(1)
            62                 #         "b"
         ff                    #       break
      61                       #     text(1)
         78                    #       "x"
      9f                       #     array(*)
         ff                    #       break
      ff                       #     break
   ff                          #   break

However, base64 is not a format amenable to human review. I suggest that the body field use a different canonical format, perhaps similar to the above. Such a format should be easy for a human to review but also easy to parse by a machine. The Smithy library could contain a parser for it, as well as a check that enforces that the binary body that the test aims to use is indeed correctly represented by the human-readable format.