Hello,
did a simple test:
a=( a ); typeset -F SECONDS=0; repeat 1000000; do a+=( a ); done; echo $SECONDS
on HEAD and 5.2. It is 28s vs 947s (15.7 minutes). Just being glad at this result and sharing it. Ran callgrind on the 28s case (unoptimized HEAD), and arrlen is the only function reported:
110,005,200,361 ???:arrlen [/usr/local/bin/zsh-5.3.1-dev-0]
Badarrays branch is far away in commit history. Could not test it, but I guess time would be very low.
Was worried about realloc usage. The anxiety is irrational, because the function cannot be non-deterministic, random, otherwise the uptimes of all the systems running out there would be low. The function has its description and it has to work. Nevertheless I've tested 512 MB, no swap FreeBSD box. First I did:
b=${(r:1024*1024*135::_:)x}
This leaves free memory (checked with `vmstat fre`) at 143 MiB. Using 150 instead of 135 triggers kill of the zsh process. So 135 is a boundary for (r) flag. I then ran a+=( ... ) with a long string in a loop, on zsh-5.3.1-dev-0. It stopped with OOM kill at 383998 iteration. I then wanted to compare this to zsh-5.1 – however the (r) padding was leaving 271 MB free memory there (and this is a correct value (!), because 135 MB + 271 MB gives 406 MB, the value without test-zsh running). So I dropped the test, because saturating 270 MB with array append without optimizations could take more than a day. That said, why the extra memory on current HEAD?
Nevertheless realloc has been quite verified. I was checking with vmstat that it was operating at as low available memory as 2576 kB. Also, at each iteration number of elements in array was verified. So in general, it isn't like "realloc works in comfortable conditions, better use malloc if you're low at memory". Realloc just works.
Comparing BSD grep to GNU grep, BSD memmove to glibc memmove says to me that if there is a cumbersome pitfall somewhere out there, then it will be on BSD system. So this verification of realloc sanity gains more strength this way.
--
Sebastian Gniazdowski
psprint /at/ zdharma.org--------------------------------------------------------------------------------
Profile data file 'callgrind.out.29752' (creator: callgrind-3.12.0)
--------------------------------------------------------------------------------
I1 cache:
D1 cache:
LL cache:
Timerange: Basic block 0 - 30114473586
Trigger: Program termination
Profiled target: zsh-5.3.1-dev-0 ./callg_test.zsh (PID 29752, part 1)
Events recorded: Ir
Events shown: Ir
Event sort order: Ir
Thresholds: 99
Include dirs:
User annotated:
Auto-annotation: off
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
110,588,060,686 PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
110,005,200,361 ???:arrlen [/usr/local/bin/zsh-5.3.1-dev-0]
Attachment:
callgrind.out.29752
Description: Binary data