[Tutor] round() yields in-expected results/commercial rounding
by other posts by this author
Dec 12 2003 5:51AM messages near this date
[Tutor] Python used in Data Acquisition systems?
|
Re: [Tutor] round() yields in-expected results/commercial rounding
I have the following problem:
> >> float(8.7)
8.6999999999999993
> >> float(870)/100
8.6999999999999993
> >> 8.7 * 100
869.99999999999989
> >> round(8.7,2)
8.6999999999999993
> >> math.modf(8.7)
(0.69999999999999929, 8.0)
> >> int(float(8.7*100))
869
especially anoying is:
> >> n = float(8.7*100) ; print n
870.0
> >> n
869.99999999999989
> >>
How can I ensure that 8.7 stays 8.7, so that
8.7 * 100 = 870 and not 869.9999...
The way I came up with is:
> >> f = float(8.7*100) ; i = int(f) ; i,f
(869, 869.99999999999989)
> >> if f>=(i+.5): i += 1 ; i
870
I am writing a commercial sales database. To save memory I use integers
internally, rather than floats.
This means that an amount of 8.70 is converted to 870 internally.
if it is converted to 869 I loose 1/10.
_______________________________________________
Tutor maillist - Tutor@[...].org
http://mail.python.org/mailman/listinfo/tutor
Thread:
Daniel Ehrenberg
Michael Janssen
|