Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Floating point calculus error...



Hi all...

I was doing some shell scripts that were supposed to add CPU usages from
the ps output... But I got puzzled when I got an answer with a 17th
decimal while adding numbers with only 1 decimal... Strange isn't it?

This behaviour can be observed by doing the following command:

zyk@Amok:/home/zyk> echo $[0.4+0.4]

The value returned is: 0.80000000000000004 which is obviously not the
correct answer... Okay, floating points numbers are inaccurate, but
still...

Patrick

PS: I don't have the time to read the mailing lists, so I don't know if
this problem has already been reported...

PS2: Please CC me on replies, I'm not subscribed to this mailing list...




Messages sorted by: Reverse Date, Date, Thread, Author