Which parties carry what costs of text/turtle changes, and do those outweigh which benefits for whom?

[RubenVerborgh]

Summary

In rdfjs/N3.js#484, I learned that the specifications intend to redefine the set of valid documents under the text/turtle media type (and presumably others).

Such a change might not be possible/desired, or should at least be acknowledged as a breaking change, with a resulting cost/benefit analysis.

Definitions

• text/turtle as the media type defined by https://www.w3.org/TR/turtle/
• valid-turtle as the (infinite) set of valid Turtle 1.1 documents
• invalid-turtle as the (infinite) set of documents that are not in valid-turtle
• spec-compliant Turtle parser as a piece of software that:
  • for each document in valid-turtle, produces the corresponding set of triples
  • for each document in invalid-turtle, rejects it (possibly with details on the syntax error)

Note here that the above definition includes rejection; the 1.1 specification text does not, its test cases do.

Potential problems

1. Retroactively changing the definition of text/turtle breaks existing spec-compliant Turtle parsers, as they will incorrectly label valid text/turtle documents as invalid.
2. There is no way to distinguish Turtle 1.1 from Turtle 1.2.

• While 1 could be argued away as "1.1 parsers only break on 1.2 Turtle", it's a problem that the parser will not be able to tell you why it breaks. Does it break because it's invalid Turtle 1.1? Does it break because it's valid Turtle 1.2? Does it break because it's invalid Turtle 1.2, despite this document intending to be within the 1.1 subset? i.e., should or shouldn't it have worked with this particular text/turtle document and no other context?

3. Building on 2, neither new nor old parsers will be able to fully automatically validate Turtle documents, since they need to be told out of band whether to validate for 1.1 or 1.2.
4. Because of the closed-set nature of text/turtle in the Turtle 1.1 spec, any changes to that set (whether deletions or additions) would contradict the Turtle 1.1 spec itself / make it invalid.
5. The problem will happen again in RDF 1.3.
6. As a more specific instance of 3, there is no standards-based way for clients or servers to indicate they only support Turtle 1.1, nor to discover whether recipients support Turtle 1.1 or 1.2 (or 1.3), as Accept: text/turtle does not tell them. Nor does Content-Type: text/turtle tell them whether their parser can handle the contents, and we could be 20 gigabytes in until we notice it doesn't.

Analysis

Unlike formats like HTML, Turtle 1.1 does not contain provisions for upgrading. The specification assumes a closed set of valid documents. We find further evidence in a number of bad test cases (https://www.w3.org/2013/TurtleTests/), which explicitly consider more permissive parsers to be non-compliant.

There is a note in the spec (but only a note, and thus explicitly non-normative):

This specification does not define how Turtle parsers handle non-conforming input documents.

but this non-normative statement is contradicted by the bad test cases, which parsers need to reject in order to produce a compliant report.

Although the considered changes for 1.2 are presumably not in contradiction with those bad cases, the test suite was not designed to be exhaustive. Rather, the 1.1 specification considers text/turtle to be a closed set, and the test cases consider a handful of examples to verify the set is indeed closed.

In particular, no extension points where left open on purpose.
Therefore, the 1.1 spec is not only defining “Turtle 1.1”, but also strictly finalizing text/turtle.

(The IANA submission's reservation that "The W3C reserves change control over this specifications [sic]." does not change the above arguments.)

Potential solutions

A set of non-mutually exclusive solutions, which each cover part or all of the problem space:

1. Factual disagreements with the above.

2. The introduction of a new media type.

3. The introduction of a new profile on top of the existing text/turtle media type.

4. A change to the Turtle 1.1 spec that adds extension points or otherwise opens the set of text/turtle.

5. Syntactical support in Turtle 1.2 for extension and/or versioning.

[afs]

Thank you for the analysis.

We do have https://www.w3.org/TR/rdf12-turtle/#changes-12 and the matter will be in "RDF 1.2 New".

The WG is discussing levels of conformance.

[afs]

Related:
There are links to specific versions of documents for both RDF and SPARQL:

https://www.w3.org/TR/rdf12-turtle/ -- currently, the 1.2 working draft. This will be the REC when published. Title "RDF 1.2 Turtle".
https://www.w3.org/TR/rdf11-turtle/ -- The RDF 1.1 published standard. Title "RDF 1.1 Turtle".
https://www.w3.org/TR/rdf-turtle/ -- Tracks the latest publication. Currently, 1.1.
https://www.w3.org/TR/turtle/ -- old name, tracks "rdf-turtle".

[RubenVerborgh]

The WG is discussing levels of conformance.

Interesting; may I suggest a standards-based mechanism for agents to indicate this level? (A media type or profile comes to mind.)

Or would “classic conformance” de facto be the same as effectively only parsing the RDF 1.1 subset (in which case it would be equivalent to one of the points above)? However, this does not seem to be the case, with for instance base directions being added to literals (in which case “classic” might be a confusing/misleading term).

[afs]

[This is not a WG response]

Any approach for versioning can have costs on both the reader-side and the writer-side.

For example, anything in the HTTP header to make the data consumer's task easier puts a requirement on the data producer. In the same way that RDF 1.2 syntax can be a long way in the delivered stream for a reader, having an HTTP header with the information makes the writers life harder because it may need to see all the data first - no stream writing without recording the details of which version is in the data, which would also be a producer-side burden.

One way to publish data is using a web server support for mapping file extensions to Content-Type headers -- .htaccess (httpd) or types{} (nginx) etc. This also appears with data dumps in archives such as zip.

Today, a tool kit may need to "know" to look in the URL to get the content type if no realistic content-type is available.

Given the file extension situation, I think any solution will not help RDF that much. Software will want to handle the static/non-profile/file-extension/... cases anyway. Only a domain specific (i.e. consumer and producer) deployment can be sure the global rules are in-play.

There is a trade-off of whether the long term continued provision of a migration solution is a greater burden than the evolution itself. Such migration should never be withdrawn -- "The web is not versioned".

[RubenVerborgh]

Thanks, @afs. I want to leave space for others so will be brief, but quickly:

• Not explicitly indicating feature/version/… support also incurs costs.
• Your answer covers the case where such indications happen in the message; different arguments and trade-offs apply when they happen in the body. As a quick example, a first-line @version 1.2 or @features literal-direction would cause a desired fail-fast on 1.1 parsers, and assist 1.2 and future parsers.

[niklasl]

Maybe an optional version or feature declaration, to support fail-fast detection? With the implicit being "latest REC". It should perhaps be clearly stated that implementations are required to follow the evolution of the format; with the reciprocal requirement of evolving the format responsibly, aspiring to standardize once "sufficient" implementation coverage has been established. AFAIK, there is a requirement of multiple independent implementations; perhaps that number should be a function of the "cardinality of known deployments" and "how viable it is to upgrade them"? (I know it is a practical impossibility to quantify that on the web scale, but it goes to show awareness of the complexity underlying these judgement calls. And that we (W3C members) have a responsibility to care and cater for cooperative evolution to ensure web interop.)

I think this follows the conventions @afs referenced, which is a trade-off I'm cautiously in agreement with. Defining a new format (mime-type + suffix) is the only other viable option AFAICS; and while that caters for more overlap in deployments, it also induces a certain inertia and growing technical debt. (When is the previous format "sunset"? How is the data quality impacted during the overlap period? How do applications take the difference in expressivity into account?)

I see no practical way around some form of social contracts, as even content negotiation is not merely technical (q=0.9 ...). The most important contract is for publishers to avoid utilizing new features until their consumers have been notified and been able to upgrade; balanced with the need for precision in the domain of discourse among those who already have (we form a web after all).

[RubenVerborgh]

With the implicit being "latest REC"

There is a trade-off of whether the long term continued provision of a migration solution is a greater burden than the evolution itself.

"The web is not versioned".

The key difference being that—for example—HTTP, HTML, and CSS have explicit behaviors on how to deal with unsupported constructs. HTTP proxies have rules on how to deal with unknown headers, HTTP has version negotiation, HTML has rules for unknown tags and attributes, CSS has rules for unsupported properties and even syntax.

So the Web's ability to be non-versioned is baked into the design of those technologies. Conversely, RDF adopting the non-versioned philosophy does not equate doing nothing on the feature support/versioning front, but rather being very explicit about how non-versioning is to be made possible.

In summary, not doing anything put us on neither a versioned nor a non-versioned trajectory. They are not binary opposites, with the third option “incompatible with versioning and non-versioning” being the unfortunate default choice.

[lisp]

to take a concrete example as precedent, i do not recall that in the transition from sparql 1.0 to sparql 1.1 was the continued use of the same media type designators was problematic.

in what sense other than the concern about "late failure" for large documents should that matter for document media types?
the notion, that 1.2 documents would be marked may seem attractive, but to fail early would still require a change to import control flow.
that, where the inability to modified deployed 1.0 version resources is central to the problem.

[RubenVerborgh]

to take a concrete example as precedent, i do not recall that in the transition from sparql 1.0 to sparql 1.1 was the continued use of the same media type designators was problematic.

Apples and oranges.
SPARQL is not a data language, nor is it problem-free.
The context of a query language is very different, including:

• limited average and typical document length
• different consequence of failure, with immediate and specific feedback
  • failure is in fact is sometimes triggered deliberately for endpoint feature discovery
• absence of streaming parsing
• different reuse context: individual queries tend to be sent to specific endpoints

So the upgrade path of SPARQL is much more similar to that of SQL, with similar challenges and non-issues.
Not comparable to that of HTTP, HTML, CSS, RDF.

And quite a pain in practice: one typically needs to know out-of-band what precise SPARQL endpoint software an interface is running, which determines how well certain SPARQL 1.0 or 1.1 features are supported.

In contrast, at least today, text/turtle has been 100% unambiguous since the introduction of the spec.
If anything, let's not go the SPARQL route.

in what sense […] should that matter for document media types?

RDF is about enabling interoperability. Yes, on the semantic level, but not having interoperability on the syntactical level precludes that.

In the pre-1.1 days, “Turtle” had been around as a format for over a decade, and parsers were incompatible with each other. It was quite the nightmare, trying to exchange data or write parsers. There was no established (let alone standard) way of knowing what subset was supported by everyone. The Turtle standard solved this by bringing certainty about what is and isn't text/turtle.

The proposed re-definition of text/turtle without any explicit indication, sends us back on a path where parsers may or may not be compatible with a certain Turtle version, and they can't even tell us. We cannot ask servers or clients. We have to know what software they are running. Not exactly the automated interoperability goal.

other than the concern about "late failure" for large documents

One might not even know. One could've parsed a 1.2 document wrongly without ever knowing. One could've rejected or accepted a document based on the wrong assumption (because assumptions are all you have, in band). One doesn't know if downstream systems are compatible with 1.1 or 1.2, because they can't tell.

It's an absolute interoperability nightmare that systems don't even have the words to express what they do and do not support. In a context where we're advocating for semantic interoperability, failing at syntactic interoperability is a serious flaw from a technical and strategic perspective. It adds a serious degree of brittleness, the details of which only a small group of people understand, which carries a major risk of reflecting badly on RDF as a whole for not being a sustainable—let alone interoperable—technology. People will say that RDF doesn't work reliably across systems, and they will be right.

[lisp]

SPARQL is not a data language, ...

that may be, unless one is concerned with sparql processors.

[lisp]

RDF is about enabling interoperability. Yes, on the semantic level, but not having interoperability on the syntactical level precludes that.

we agree - vehemently.
as much as ambiguous recommendations is not the answer, neither is error signalling and handling.
would graph store protocol endpoint service descriptions provide sufficient information to the architectures which you envision, in order for them to more effectively control requests?

[Ostrzyciel]

2 cents from someone who did implement a non-standard RDF format that has an analogue of Ruben's proposed @version 1.2 or @features literal-direction – it sounds like a nice idea, but implementing a serializer that would reliably set such flags is a pain. You essentially need to predict the future: "will this document need 1.2 features or not?" This may seem like a trivial question if we are dealing with a small piece of metadata on the Web, but is completely impossible if we have something like a database dump or any other long stream of data.

I ended up making the serializer always claim that all features are used by default. Then, it's up to the user to tell the serializer that "this and that" feature won't be needed. This creates an obvious compatibility problem, because parsers will simply refuse to read these files, even though in practice the feature may not be used. I have not found a better solution to this problem. I think this is a sensible compromise for my ugly format, but I would be against this in W3C formats. More details here.

Overall, I think a sensible solution would be to embrace the mess and just live with the fact that RDF formats can evolve. I would also like to ask the WG to kindly consider producing some "best practices" for how to mark that an RDF file is 1.2, in a use case-specific manner. I like the suggestion from @lisp for adding some info in graph store protocol descriptions. I'm also curious if something like a non-mandatory HTTP header would be an option. Or maybe a comment at the start of the file (like a shebang in .sh files) – of course, entirely optional. (disclaimer: I did not think these ideas through, they may be VERY bad)

[HolgerKnublauch]

Intuitively to me it sounds like TTL documents that use any of the new features need a new media type and file ending.

[lisp]

I'm also curious if something like a non-mandatory HTTP header would be an option.

legacy software will not see them.
placing them such that the control flow of those components will have to be aware of them is not effective.
a service description, based on which a higher-level process can orchestrate operations would be much more effective.