Zsh Mailing List Archive Messages sorted by: Reverse Date, Date, Thread, Author

Re: Floating point calculus error...

X-seq: zsh-workers 17579
From: dmeyer@xxxxxxxxxx
To: zsh-workers@xxxxxxxxxxxxxx
Subject: Re: Floating point calculus error...
Date: Sun, 25 Aug 2002 15:02:08 -0400
In-reply-to: <20020825180708.GE6783@xxxxxxxxxxxxxx>
Mailing-list: contact zsh-workers-help@xxxxxxxxxx; run by ezmlm
Organization: dmeyer.net
References: <1030288941.517.8.camel@Amok> <20020825155349.GD6783@xxxxxxxxxxxxxx> <87k7me2921.fsf@xxxxxxxxxx>
Reply-to: dmeyer@xxxxxxxxxx

In article <20020825180708.GE6783@xxxxxxxxxxxxxx> you write:
> On Sun, Aug 25, 2002 at 18:32:38 +0200, David Kågedal wrote:
> > Vincent Lefevre <vincent@xxxxxxxxxx> writes:
> > > $ echo $[0.4]
> > > 0.40000000000000002
> > >
> > > I think that it should give the minimum number of decimals in such a way
> > > that when converting back to binary, one should get the same result.
> > 
> > But it does.
> 
> No, this is not the *minimum* number of decimal digits. 0.4 would
> be sufficient in this case.

This is a bad idea, IMHO.  The number is internally represented as
0.40000000000000002, and that's what should be printed out.  If you
don't like all the digits, use printf.

-- 
Dave Meyer
dmeyer@xxxxxxxxxx

Messages sorted by: Reverse Date, Date, Thread, Author