80-bit pseudo-denormals

[sards3]

It seems that 80-bit pseudo-denormals (that is, 80-bit extended floats with exponent of zero and significand greater than 0x8000000000000000) are not handled properly. For example, there is an unhandled overflow here:

berkeley-softfloat-3/source/s_addMagsExtF80.c

Line 84 in a0c6494

sigZ = sigA + sigB;

(i.e. if you add the smallest denormal w/ significand 0x0000000000000001 to the largest pseudo-denormal w/ significand 0xffffffffffffffff, SoftFloat gives zero as the answer.)

I don't know if handling pseudo-denormals is within the scope of Berkeley Softfloat. But since the 80-bit extended format is fairly specific to Intel x87 FPUs, it seems like pseudo-denormals should be handled just as x87 handles them.

[GregoryMorse]

Per https://en.wikipedia.org/wiki/Extended_precision#x86_extended-precision_format : "80387 and later properly interpret this value but will not generate it" - so technically it seems x87 is only supporting these for backward compatibility with the 8086 and 80286. There is some perspective that the pseudo-values are all undefined behavior. Though zeroing the high bit of pseudo-subnormals could be a better behavior, as interpret but not generate is more stable behavior.

[GregoryMorse]

Just to confirm against a correct value, this absolutely is a bug in the library as pseudo-subnormals despite not being generated should be properly interpreted:

./berkeley-testfloat-3/build/Linux-x86_64-GCC/testfloat_ver extF80_add
Testing extF80_add, precision 80, rounding near_even.
00000000000000000001 0000ffffffffffffffff 00028000000000000000 00    
Errors found in extF80_add, precision 80, rounding near_even:
+0000.0000000000000001  +0000.FFFFFFFFFFFFFFFF
        => +0002.8000000000000000 .....  expected +0000.0000000000000000 .....