Re: arithmetic anomaly

[Ray Andrews <rayandrews@xxxxxxxxxxx>]

On 2021-03-26 3:35 a.m., Roman Perepelitsa wrote:
> Maybe this will help: 
> https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html 
> <https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>


Good to know!

Seriously, I've often wondered what really goes on in a calculator. Ok, 
for now I know that zsh and probably everybody else will compute endless 
reams of garbage if you let it.  Here's my algorithm with the outer 
'for' loop increasing exponentially:

Ray's  Algorithm fractional:
Level: 1: win: 1 / 1
Level: 10: win: 6513215599 / 10000000000
Level: 100: win: 
6339676587267708385630188881009247729557115329795338127068495312693456362833186743509112446
7602830066897486524892436064552875964380756727929631216242621046676674030227216977589219271
483398068889452544 / 
9999999999999999697331222125103616594745032754550236264824175095034684843555407553419633840
4706251868027512415973882408182135734368278484639385041047239877871023591066789981811181813
306167128854888448
Level: 1000: win: inf / inf
Level: 10000: win: inf / inf
Level: 100000: win: inf / inf
Level: 1000000: win: inf / inf

.... I'm asking why the 'Level: 100' result isn't just aborted after 15 
digits given that all the rest is garbage?


> Understanding how floating point numbers work is very useful. It's 
> virtually mandatory when writing numerical computation code. None of 
> this is specific to zsh.

Amen!  Seriously, I had no idea this sort of thing was permitted. Thanks 
Roman.  For now I'll use ' %d' for output and at least I'll know when 
things turn to mud, there must be limits to precision and there must be 
overflow at some point.  (But of course we hafta use floats if there's a 
division anywhere, just don't use them for display.)