Zsh Mailing List Archive
Messages sorted by: Reverse Date, Date, Thread, Author

inconsistent long number truncation



From https://unix.stackexchange.com/a/474132

$ zsh -c 'echo $((12345678901234567890));echo $((12345678901234567890123))'

zsh:1: number truncated after 19 digits: 12345678901234567890
1234567890123456789

zsh:1: number truncated after 22 digits: 12345678901234567890123
-1363962815083169260

I think the:


          newcalc = calc * base + *s - '0';
          if (newcalc < calc)

Is an invalid way to check for numbers wrapping.

On GNU/Linux amd64
ulong 1234567890123456789*10 is
      12345678901234567890 and:

(gdb) p calc
$7 = 12345678901234567890
(gdb) p calc*10
$8 = 12776324570088369204
(gdb) p calc*10*10
$9 = 17082781258626382344
(gdb) p calc*10*10*10
$10 = 4807115922877858896

Why do we truncate in the first place? That's hardly more useful
than letting the numbers wrap.

-- 
Stephane



Messages sorted by: Reverse Date, Date, Thread, Author