Replies: 1 comment 3 replies
-
It looks like there were a couple issues with the examples I originally posted above. First, when I tried the 2nd example again today in a fresh repl, I realized it doesn't work as-is. After fixing it, I get the following error: >>> from decimal import Decimal
>>> from pint import Quantity, UnitRegistry
>>> ureg = UnitRegistry(non_int_type=Decimal)
>>> d2 = Quantity(Decimal("1.0"), ureg.inch)
>>> d2.to(ureg.thou).magnitude
Traceback (most recent call last):
[...]
TypeError: unsupported operand type(s) for ** or pow(): 'float' and 'decimal.Decimal'
>>> I think when I was originally trying this out I had accidentally typed >>> from decimal import Decimal
>>> from pint import UnitRegistry, Quantity
>>> ureg = UnitRegistry()
>>> d2 = Quantity(Decimal("1.0"), ureg.inch)
>>> d2.to(ureg.thou).magnitude
Decimal('1000.00') The second issue is that I was importing >>> from decimal import Decimal
>>> from pint import UnitRegistry
>>> ureg = UnitRegistry(non_int_type=Decimal)
>>> d2 = ureg.Quantity(Decimal("1.0"), ureg.inch)
>>> d2.to(ureg.thou).magnitude
Decimal('999.9999999999999999999999996') Couple questions:
|
Beta Was this translation helpful? Give feedback.
3 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I was surprised to find that converting
1 inch
tothou
using Decimal numbers produced a result with incorrect precision.However, if I use the Quantity() constructor, I get the correct precision I would expect using a Decimal number.
Can someone explain why these produce different results? What is the difference between creating the quantity with
Decimal("1") * ureg.inch
vs.Quantity(Decimal(1), ureg.inch)
?Beta Was this translation helpful? Give feedback.
All reactions