Incompatibility with "Schema" based designs

[cyberphone]

This issue is related to #33

AFAIK all existing schema based designs presume that the type of an element stays the same regardless of its value. The variable type concept for numbers used by jsonb/jsonp breaks that.

Have you performed any interoperability tests with other JSON tools?

[bravehorsie]

@cyberphone I am not up on whats going on, which elements are we talking about? Can you please provide an example?

[cyberphone]

@bravehorsie If I understood it correctly, the jsonb/jsonp solution uses a variable serialization method of values in order to automatically cope with I-JSON requirements:
https://github.com/cyberphone/I-JSON-Number-System#java-json-b
This is incompatible with OAS as well as most other systems as well.

[bravehorsie]

@cyberphone OpenAPI and ECMA Script generally doesn't support numeric types longer than IEEE754. JSONB/JSONP on the other hand has to deal with numbers outside of that precision. Is there any other type defined by ECMA Script suitable to be used for BigDecimal/BigInteger besides String? Do you suggest to write these types always as String regardless of the fact if its value "fits" or is outside of ECMAScript's Number type / IEEE754?

[cyberphone]

@bravehorsie OpenAPI do support long:
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#dataTypes
Long always serializes as JSON Number and thus should be incompatible with a system using a variable serialization scheme. Your interpretation of I-JSON is correct but pretty illogical for declarative models. IMO, the only viable solution is either a global option like I-JSON or annotations that permit doing this with ease.

However, it may get worse depending how the following pans out:
https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-05#section-3.1
"An additional constraint is that parsed JSON String data MUST NOT be altered during subsequent serializations"

[rmannibucau]

I guess it is ok since openapi is about data of the application which can be enforced through an adapter if needed/desired, same applies to jsonschema. Note that this last one is particular in early stages (even if "old") and I don't think we should consider more than being compati-able (being compatible without enforcing anything for it) in the context of JSON-B otherwise we'll get into troubles (like breaking API).

[bravehorsie]

@cyberphone I see, it doesn't say anything about double precision, just that it is a 64bit number. That makes their long also not parsable by ES JSON.parse(). There is already an I-JSON option in the API, however it doesn't affect serializing numbers in any way. I-JSON states:

I-JSON
messages SHOULD NOT include numbers that express greater magnitude or
precision than an IEEE 754 double precision number provides, for
example, 1E400 or 3.141592653589793238462643383279.
...
For applications that require the exact interchange of numbers with
greater magnitude or precision, it is RECOMMENDED to encode them in
JSON string values.

https://tools.ietf.org/html/rfc7493#section-2.2
It doesn't say anything about type conversion / mapping, just that numbers (I understand that as number values) of greater precision should be written as strings.

@rmannibucau My first thought on adaptive notation problem was that it would be pretty awkward to have java POJOs being serialized with their long id values as strings no matter of value. On the other hand having an id in JSON either as a number or as a string depending on value is also not the best option.

[rmannibucau]

@bravehorsie while a round trip works I think it works even if not perfect (yes it is linked to javascript world historical habits). AFAIK JSON-B is friendly with that behavior and does its best here, no?

[cyberphone]

@bravehorsie @rmannibucau If you take the currently shipping version of Chrome and feed it with

JSON.stringify(BigInt("5"))

it returns "Uncaught TypeError: Do not know how to serialize a BigInt".

OTOH, your solution for dealing with BigInteger serialization differs (to the best of my knowledge) from every other implementation out there.

That is "we" (the industry) obviously have a problem.

I'm looking for workable (and hopefully) future proof changes to address this issue. That's all.

[rmannibucau]

Afaik it is common to use a number when possible and switch to string when not. It is what I saw and used since 6-7 years so not sure "we" have a problem. Some language libs didnt solve this issue but it is their problem and not main impls I think.

[cyberphone]

I'm talking about things that could be regarded as standard. You won't find a single standard out there utilizing JSON for information interchange which builds on the serialization scheme used by JSON-B/P tools. For in-house solutions you can do whatever works for you but that is not my interest.

Feel free ignoring (and closing) this issue but I'm pretty sure it will pop up some day again 😀

[rmannibucau]

@cyberphone point is jakartaa is not about defining json standards, it just follows them and ensure it works for users.

[cyberphone]

@rmannibucau That I fully understand. What I'm questioning is the interpretation of the standard. Lossy long variables doesn't sound very cool.

I can't imagine a JSON schema designer adding a BigIntger would ever consider "variable" as a proper serialization format unless he/she is working on an in-house design. That it "usually works" is great but sounds awfully "hackish" in my ears.

I leave it there.

[rmannibucau]

Hmm, dont mix two things, jsonschema users will put strong for big numbers to type it or alternatively - and it is jsonschema compatible - an anyOf with string and number so JSON-B by itself is compatible with that.

[cyberphone]

"Schema" was on a conceptual level like in IETF standards using JSON where the serialization of each element is described in detail.

[rmannibucau]

Ok, issue is then than bug numbers are out of the definition of these schema where only primitives, structures and arrays exist so they would fall into string category naturally and it is trivial to enforce it with an adapter ;)