Zsh Mailing List Archive
Messages sorted by:
Reverse Date,
Date,
Thread,
Author
zsh converts a floating-point number to string with too much precision
- X-seq: zsh-workers 45093
- From: Vincent Lefevre <vincent@xxxxxxxxxx>
- To: zsh-workers@xxxxxxx
- Subject: zsh converts a floating-point number to string with too much precision
- Date: Fri, 20 Dec 2019 02:37:11 +0100
- List-help: <mailto:zsh-workers-help@zsh.org>
- List-id: Zsh Workers List <zsh-workers.zsh.org>
- List-post: <mailto:zsh-workers@zsh.org>
- List-unsubscribe: <mailto:zsh-workers-unsubscribe@zsh.org>
- Mail-followup-to: zsh-workers@xxxxxxx
- Mailing-list: contact zsh-workers-help@xxxxxxx; run by ezmlm
With zsh 5.7.1, I get:
zira% echo $((1.1))
1.1000000000000001
because zsh seems to first select the precision independently
from the value, i.e. 17 to be able to convert the string back
to floating point, preserving the original value, then it
outputs the closest number in this precision.
Instead, zsh should select the minimum precision so that the
inverse conversion can give the original value, i.e. it should
output 1.1 here.
FYI, GNU MPFR has the same issue with mpfr_printf and %Re (with
an empty precision field), and I regard this as a bug:
https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00000.html
https://sympa.inria.fr/sympa/arc/mpfr/2019-12/msg00001.html
Note that Java does it right:
zira:~> cat tst.java
public class tst
{
public static void main(String[] args)
{
double x;
x = 0x1.1999999999999p+0;
System.out.println(x);
x = 0x1.199999999999ap+0;
System.out.println(x);
x = 0x1.199999999999bp+0;
System.out.println(x);
}
}
zira:~> javac tst.java
zira:~> java tst
1.0999999999999999
1.1
1.1000000000000003
--
Vincent Lefèvre <vincent@xxxxxxxxxx> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Messages sorted by:
Reverse Date,
Date,
Thread,
Author