undefined | Better HN

0 pointsmasklinn6y ago0 comments

round(3) can only round to an integer. Python's round works to an arbitrary decimal position.

It would previously scale up, round (ceil/floor really) then scale down. That turned out to induce severe precision issues: https://bugs.python.org/issue1869

0 comments

f00zz6y ago

That's the first thing I'd try, multiply by 10^digits, round, divide by 10^digits. Thanks for the link.

j / k navigate · click thread line to collapse

0 pointsmasklinn6y ago0 comments

round(3) can only round to an integer. Python's round works to an arbitrary decimal position.

It would previously scale up, round (ceil/floor really) then scale down. That turned out to induce severe precision issues: https://bugs.python.org/issue1869

0 comments

f00zz6y ago

That's the first thing I'd try, multiply by 10^digits, round, divide by 10^digits. Thanks for the link.

j / k navigate · click thread line to collapse