Re: Slowdown around 5.0.5-dev-0

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Oct 10, 12:54pm, Sebastian Gniazdowski wrote:
} Subject: Slowdown around 5.0.5-dev-0
}
} I was testing my scripts and saw that on 5.0.2 having 89k-elements
} long is much faster than on 5.1.1-dev-0. I ran git bisect and found
} the commit:
} 
} 2014-01-18 23f98c3 32285: restart the fheap search in freeheap if the
} current fheap arena is about to be discarded; fixes crash

If you look back through the zsh-workers articles, you'll find that
the performance was even worse prior to workers/29175; there was a
huge gain in performance from that patch, but it introduced a crash
bug, so we had to give back some of the performance in workers/32285
to prevent the crash from happening.

Read the large comment in freeheap() for a description of what is
going on there, note particularly the paragraph ending "performance
goes to hell."

I fear a signficant rewrite of the heap-allocation code would be needed
to both avoid the crash and improve the performance.  The current code
was designed with the expectation that zhalloc() would be a much more
common operation than freeheap(), but there are circumstances in which
the latter happens a lot.

It's possible that we could figure out a way to skip the scan that
updates all the h->used values in freeheap(), and pay the price on the
next zhalloc() instead.

I think (but am not 100% sure) that the pathological case happens when
the heap is already maximally filled, so a new arena is always added
to the end of the chain and then dropped again on each alloc/free.

On the other hand the shell really isn't intended to be manipulating
arrays of containing tens of thousands of elements.