Re: Slowdown around 5.0.5-dev-0

[Sebastian Gniazdowski <sgniazdowski@xxxxxxxxx>]

On 18 October 2015 at 18:19, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> On Sat, Oct 17, 2015 at 2:12 AM, Sebastian Gniazdowski
> <sgniazdowski@xxxxxxxxx> wrote:
>> On 16 October 2015 at 02:35, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
>>> The array test only iterates 10000 times, the string test 50000 ?  Does
>>> that not account for the difference?
>>
>> Apparently not. I've modified the mem-test script so that it queries
>> usage of memory also during the tests. It can be seen that for strings
>> (and zsh 5.0.2) the memory is high from the beginning.
>
> I'm strongly inclined to think that's more likely an artifact of the
> testing method than of the test itself have you tried running the
> tests in the opposite order?

The test script is quite well written. Here is how a test is being run:

      "$current_zsh" -c "source ./$0 $$ \"$current_zsh\" $test" &

It starts new zsh process – the one in "$current_zsh" variable – and
feeds it with the same script file, giving in arguments pid of main
Zsh, name of tested Zsh and name of test function to run. This should
imply full separation of the tests. That's why order shouldn't matter.
However I did reverse the order to just check and results are the
same. The memory profile of three-patches version is thus:

# string_test 41, 164, 279, 388, 504, 602, 707, last: 11

You are quite right that the memory usage isn't "high from the
beginning". It starts from 41 MB, goes to 164, 279, etc. in quite
controlled way. Previous version of the script awaited 3 seconds
before doing first check, that's why it started from values of 100 MB,
and I rather misinterpreted this. How to interpret these still high
values is now open.

> You also haven't explained your
> reasoning for having the string test iterate so many more times than
> the array test.  Does it have to do with keeping total run time
> similar?

Yes that's the reason. I now do even more iterations to make the test
last. During this time I query memory size, obtaining a nice sequence
of memory usages. It nicely illustrates behavior of Zsh.

>> It can be also seen that the recent three patches increase memory
>> usage. A clean, free from the patches version of zsh
>> (5.1.1-dev-0-clean) keeps the memory always low.
>
> That's entirely to be expected -- all three patches increase speed by
> avoiding the work necessary to minimize memory footprint, until (as
> you noted in next message) the execution stack is unwound.

The script now computes averages. Average of 5.0.2 is 622.3, of
two-patches version 327.7, of the three patches (third is the new
heaps) 329.7. So it's better than without the patches, having in mind
also the quick drop after stack is unwound.

Tried a new test function:

function_test() {
    local count

    if [ -z "$1" ]; then
        repeat 10000; do function_test 100; done
        _finished_signal_wait
    else
        count="$1"
    fi

    if (( count -- > 0 )); then
        function_test "$count"
    fi
}

Its memory usage is low for all Zsh versions: ~2.5 MB, (e.g. 2.7, 2.9,
2.8, 2.6, 2.9, 2.8, 2.6, 2.9, 2.9, 2.8).

Best Regards,
Sebastian Gniazdowski