Skip to content

Support VariableList longer than 2**31 on 32-bit architectures #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

color-typea
Copy link

  • Improves handling large VariableList for 32-bit architectures
    • loudly crash now vs. silently overflow (and produce wrong results) previously
  • Adds feature to enable capping typenum to usize conversion to usize::MAX
  • Tests + github actions

Why:

  • BeaconState.validators is defined as VariableList<Validator, typenum::U1099511627776> (2**40)
  • U1099511627776::to_usize() returns 0 on 32-bit architectures
  • ... such as riscv32 or wasm32 that are popular choices for Zero-Knowledge VMs (such as SP1)

How:

typenum_helpers.rs:

  • Conditional compilation on 64-bit vs. 32-bit architecture
  • By default, loudly panic if overflow happens on 32-bit
  • Added a feature to allow capping to usize::MAX instead - allows working with "virtually larger than usize" VariableLists that are guaranteed to have less than usize::MAX elements (i.e. only larger in typenum)

tree_hash.rs

  • Split on the pointer size
  • no changes for 64-bit arches - only wrapped into the module for more convenient access.
  • For 32-bit, cap the actual tree computation to 2**29, than "expand" (O(depth) time) to the target tree depth by hashing with zerohash of corresponding level.
    • Should've been 2**31, but there are some downstream operations in ethereum_hashing/tree_hash that cause overflows with larger values - so this is empirically determined maximum that still works
    • Note: ethereum_hashing::ZEROHASHES ends at depth 48, so won't work for larger lists on 32-bit architectures. This could be solved in this repo by constructing zerohashes independently, or there by expanding the ZEROHASHES to 64 levels.

variable_lists.rs mostly contains uses for to_usize helper + tests for the whole deal

* Improves handling large VariableList for 32-bit architectures
   - loudly crash vs. silently overflow (and produce wrong results)
* Adds feature to enable capping typenum to usize conversion to usize::MAX
* Tests + github actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant