WIP: add support to bigger ints (256 bit)

[bettio]

This PR introduces support to signed 256 bit integers.

Briefly:

• Add code for generic big integer handling (intn.c), limited to 256 bit for the sake of simplicity, but it might be expanded to generic big integers with some additional work
• Refactored arithmetic BIF helpers
• Refactored integer_to_binary/_to_list in order to reduce both code duplication and make it simpler adding big integers support
• Reimplemented binary_to_integer in to make it compliant with OTP (binaries such as <<"0xCAFE">> or <<" 42">> must be rejected).
• Replaced lltoa with more performant functions that do not rely on slow helpers, specially for base 10 and 16

Notes:

• Big integers implementation is quite decoupled from term.h, so functions in term.h handle them as opaques storing bytes
• In our implementation Value bor 0x80000000000000000000000000000000 will cause a sign flip, unlike OTP

Todos:

• Big integer literals, without them 0xCAFE1234CAFECAFE = calculate() cat not work)
• term_to_binary and binary_to_term
• binary_to_integer and list_to_integer

These changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).

SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later

[pguyot]

Is it bigint or big_int?

[bettio]

I'm using bigint now.

[pguyot]

I don't understand why this exists as a function. Nothing seems shared between each pair member.

[bettio]

All binary operations (add, mul, div, etc...) have the same repeated pattern, so I was factoring it.
Honestly I think I should factor in a macro also this part before that is going to be repeated:

 698     intn_digit_t tmp_buf1[INTN_INT64_LEN];
 699     intn_digit_t tmp_buf2[INTN_INT64_LEN];
 700
 701     intn_digit_t *bn1;
 702     size_t bn1_len;
 703     intn_digit_t *bn2;
 704     size_t bn2_len;

Anyway that function is required to handle all the combos such as: small-int * big-int, big-int * small-int and big-int * big-int.

[pguyot]

Unsupported

[bettio]

Fixed.

[pguyot]

Unsupported endianness

[bettio]

Fixed.

[pguyot]

    2559181265480533323615999200984578944503596644 = Square,

We also need support for literal decoding.

[bettio]

Indeed, we are getting there step by step.

[pguyot]

This isn't only about mul, the function could be split into several.

[bettio]

I'm still not sure about this comment, can you explain further?

[pguyot]

I am confused about the signature, returned value is [] or invalid_term, isn't it?
Couldn't we return just a bool?

[bettio]

ugly signature indeed, that function raises different kinds of errors for both integer_to_binary and integer_to_list

[pguyot]

Since maybe_fail_ret is not used, UNLIKELY could test the returned value of integer_to_buf directly.

[bettio]

It is actually used:

2254     if (UNLIKELY(term_is_invalid_term(maybe_fail_ret))) {
2255         return maybe_fail_ret;
2256     }

[pguyot]

Sorry for the confusion. What I mean is that we only test result being invalid term or not. When it's invalid term we return invalid term. In this respect we don't really use this value.

[bettio]

oh, right, got it.

[pguyot]

Do we have a compile-time test for this?

[bettio]

We need to test different combos indeed.

[pguyot]

Aren't mbedtls_mpi_* functions linked into AtomVM if SSL is enabled? By default I think they are 4096 bits. I would favor these (and I also have a branch where I vendor crypto functions as we discussed in Fosdem for wasm).

[bettio]

Aren't mbedtls_mpi_* functions linked into AtomVM if SSL is enabled? By default I think they are 4096 bits. I would favor these (and I also have a branch where I vendor crypto functions as we discussed in Fosdem for wasm).

Yes, they are, but there are is some additional complexity.

1. mbedtls mpi is focused on crypto operations, so it lacks all the bitwise operations we need. Also integer bytes are private, so we are not supposed to access them for our operations without serializing them back and forth.
2. all functions are supposed to work on data that is stored on the heap (and it has its own memory management), so we need to handle each number as a resource, unless we do the same serialization work back and forth as for 1.
3. if we vendor it, we should remove all the additional crypto functions we don't use (e.g mbedtls_mpi_exp_mod)
4. it cannot be vendored by just taking that source file or a single function, more work has to be done in order to take their mempool and other required bits / cleaning up their sources from their macros and defines.

So we can use it, but it is not that trivial, and I'm not sure it is worth it.

As an additional note, in our implementation the most complex part are div and mul algorithms, that have been copied from Hacker's delight, and based on Knuth work. They are pretty efficient and well tested.

[bettio]

Noteworthy additional changes:

• (NIFs: refactor binary_to_integer/1) bettio@5006d3d

The binary_to_integer function was not compliant with OTP behavior, it was accepting trailing spaces, and base prefixes (such as 0x).
This implementation has same behavior of OTP one, and also tries to optimize base 10 and base 16 cases.
It also avoids expensive int64_t multiplications on 32 hardware, as much as possible.

• (NIFs: refactor integer_to_binary/list) bettio@a68cf2f

It replaces our implementation, with an optimized one.
Having a specialized case for base 10 allows the compiler to replace division by 10 with reciprocal mul.