Zsh Mailing List Archive
Messages sorted by:
Reverse Date,
Date,
Thread,
Author
Floating point calculus error...
- X-seq: zsh-users 5284
- From: Patrick Aussems <zyk@xxxxxxxxxxxxxx>
- To: zsh-users@xxxxxxxxxxxxxx
- Subject: Floating point calculus error...
- Date: 25 Aug 2002 17:22:20 +0200
- Mailing-list: contact zsh-users-help@xxxxxxxxxx; run by ezmlm
Hi all...
I was doing some shell scripts that were supposed to add CPU usages from
the ps output... But I got puzzled when I got an answer with a 17th
decimal while adding numbers with only 1 decimal... Strange isn't it?
This behaviour can be observed by doing the following command:
zyk@Amok:/home/zyk> echo $[0.4+0.4]
The value returned is: 0.80000000000000004 which is obviously not the
correct answer... Okay, floating points numbers are inaccurate, but
still...
Patrick
PS: I don't have the time to read the mailing lists, so I don't know if
this problem has already been reported...
PS2: Please CC me on replies, I'm not subscribed to this mailing list...
Messages sorted by:
Reverse Date,
Date,
Thread,
Author