Skip to content

[C++] parser: Add zero_on_float_to_int logic and improve json parsing. #8628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Dwordcito
Copy link

Motivation

This change addresses a fundamental issue when parsing JSON serialized using libraries like cJSON that represent all numbers as double.

Problem Statement

  1. Precision limitation: double values in JSON can only represent integers exactly up to 53 bits (2^53 - 1 = 9,007,199,254,740,991)
  2. Precision loss: When an integer field in FlatBuffers contains values exceeding this limit, cJSON converts them to double, causing precision loss ex "1.8446744073709552e+19"
  3. Parsing failure: The current parser rejects these float values for integer fields, causing the entire parsing operation to fail

Implemented Solution

Instead of failing the entire parsing operation, this change introduces the zero_on_float_to_int option that:

  • Detects when a float value is received for an integer field
  • Preserves the validity of the parsing event
  • Assigns a default value (0) to the problematic field
  • Emits a warning to inform about precision loss

Use Cases

This is particularly useful for systems that:

  • Process large volumes of JSON data with 64-bit integers
  • Require fault tolerance in parsing operations
  • Prefer partially valid data over complete parsing failure
  • Integrate with legacy systems using cJSON or similar libraries

Behavior

// With zero_on_float_to_int = true
"{ id: 1.8446744073709552e+19 }" → id field = 0 (with warning)

// With zero_on_float_to_int = false (default)
"{ id: 1.8446744073709552e+19 }" → parsing fails

Test Coverage

The change includes comprehensive test coverage in ZeroOnFloatToIntTest() to ensure the feature works correctly in both enabled and disabled states.

@github-actions github-actions bot added c++ codegen Involving generating code from schema labels Jun 27, 2025
@Dwordcito Dwordcito changed the title parser: Add zero_on_float_to_int logic and improve json parsing. [C++] parser: Add zero_on_float_to_int logic and improve json parsing. Jun 27, 2025
@Dwordcito
Copy link
Author

@dbaileychess I hope you're okay, we're using this change in our product, I think it would be good to have it in the library (it would help us a lot)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ codegen Involving generating code from schema
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant