Provide automatic validation of services (as they are added to the census?)

[awagner-mainz]

In the current list of available services, there are entries that do not comply to the spec (e.g. offering GET support for reconciliation only, not POST support). Maybe there could be way to automatically test submissions. Possibly even with results being mirrored in differentiated status/support values in the testbench list.

Maybe an automatic and a manual (with form values) test method?

[wetneb]

Yes I think it would be nice to have a tool to automatically test compliance with the spec. I think it is difficult to do so only given the endpoint of the service, but I suspect we could already go a long way with:

• the endpoint
• the name of an entity exposed by the endpoint
• the name of a property exposed by the endpoint (if any)
• the name of a type supported by the endpoint (if any)

With those example names provided, we would already be able to generate quite a few requests to the service:

• reconciliation queries with the exact name, or slight variations of it
• suggest queries with any of the example names, or prefixes of them
  Those two sorts of queries should return some identifiers, which can be in turn used for:
• data extension
• property proposals
• reconciliation queries with some properties
  and so on.

Just by checking the compliance of the responses with the JSON schemas we have, we should already be able to pick up quite a few issues.

Checking the access to the service (HTTPS, GET/POST, CORS headers…) would also be useful indeed.

[wetneb]

Given that the testbench is a web app without a backend, we will be a bit restricted in how we check the service access (CORS, HTTP(S)) since we are bound by the browser restrictions, but I do not think it should be a huge problem (the current test bench works well despite that).