Re: argv subscript range uses too many memory

[Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>]

On Sat, 10 Nov 2012 06:57:09 -0800
Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> Further discussion probably should be re-routed to zsh-workers.

I've done that.
 
> In a loop, the heap allocations are not popped until the loop is done,
> IIRC, so you'll end up with a large number of copies of the original
> array in the heap with slice results pointing into different parts of
> each copy.  Maybe there's a narrower scope in which a pushheap/popheap
> could be inserted.

What's puzzling me is that loops, including the "while" involved here,
execute freeheap() at the end of each iteration.  That should restore the pristine state of the loop when it was pushed each time around, shouldn't it?  Maybe I'm missing some subtlety of freeheap().

I'm a little bit worried there might be some pathology with permanent,
i.e. non-heap, allocation, in particular with zsh's own implementation
of malloc() (though I don't think anyone's compared between system and
zsh versions).  That would be something along the lines of: each time
around the loop we create an array that is just a little bit bigger than
the one we created last time, so doesn't fit in the memory we freed, so
we allocate yet another chunk of memory.  That's probably nonsense
because (i) if we allocated another chunk, the combination of the that
with the previous one would be order twice as large as the original, so
we shouldn't see pathological behaviour unless something was doubling
each time, and there's nothing obviously like that in this case (ii)
indeed we're looking at arrays getting smaller each time.  Still, I
can't quite get this out of my head.  That's probably because my head's
a funny shape inside.

-- 
Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/