Increase floating point precision

[An-anonymous-coder]

2^49/5 should return 112589990684262.4, but due to a floating point error it returns 112589990684262.40625 instead

[VoxelPrismatic]

only so much precision with 64 bits without implementing IEE 754 in 128 bits

[An-anonymous-coder]

I'm curious how other calculator apps work around this issue so well

[VoxelPrismatic]

actually, I wonder if this is using 32-bit floating points not 64, that's the same hex value

[VoxelPrismatic]

sorry to say, 64 bits won't help you either

[An-anonymous-coder]

128 bits seems to give enough precision for these calculations.

[VoxelPrismatic]

Yeah, but there is no native impl of float128, because, usually, with such large numbers, the precision after the decimal isn't that important.

[An-anonymous-coder]

How do other calculators solve this issue? Do they implement their own version of f128?

[VoxelPrismatic]

I think this one is using float32, most others use float64, or use something like py's decimal library which basically stores both sides of the decimal as ints, yielding near perfect precision.