Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

Floating-point arithmetic evaluation on Linux/x86 (traditional FPU)



The zshmisc man page says:

ARITHMETIC EVALUATION
  The  shell  can  perform  integer and floating point arithmetic, either
  using the builtin let, or via a substitution of the form $((...)).  For
  integers,  the  shell is usually compiled to use 8-byte precision where
  this is available, otherwise precision is 4 bytes.  This can be tested,
  for example, by giving the command `print - $(( 12345678901 ))'; if the
  number appears unchanged, the precision is at least 8 bytes.   Floating
  point arithmetic is always double precision.

But it is not true that zsh always uses double precision. It seems to
use the C double type, but this doesn't correspond to double precision
by default under Linux/x86 (when the traditional FPU is used, i.e. no
SSE2). For instance, I get:

vin:~> uname -a
Linux vin 2.6.22-1-686-bigmem #1 SMP Sun Jul 29 15:54:01 UTC 2007 i686 GNU/Linux
vin:~> echo $((9007199254740994.0+(1.0-1/65536.0)-9007199254740994.0))
2.

instead of "0." on other machines, due to the "double rounding"
effect. More information about this problem on:

  http://www.vinc17.org/research/extended.en.html

Either zsh should switch to double precision (assuming that modules
and so on won't change the precision back to the extended precision),
or the zsh documentation should be changed.

-- 
Vincent Lefèvre <vincent@xxxxxxxxxx> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)



Messages sorted by: Reverse Date, Date, Thread, Author