Re: Slowdown around 5.0.5-dev-0

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Oct 14,  8:50am, Sebastian Gniazdowski wrote:
}
} With this patch the list is instant fast.

NEWHEAPS() is O(1) where pushheap() is O(N) where N is the number of
arenas already in the heap.  So it's not suprising that it's faster
in the specific case you're trying (lots of data in the heap for a
relatively long time) -- the question is would it slow down in the
more normal use case of a mostly-empty heap and a lot of function
calls, because the one operation done by NEWHEAPS() is heavier than
any one of the N operations in pushheap().

On the other hand for interactive use the difference in the small-memory
case is probably never going to be noticed.

} What's interesting is lower RES for the new patch. It's like with the
} new patch RES can easier go down after peak.

With pushheap() the code tries to maximally use available space in the
already allocated heap arenas; with NEWHEAPS() that available space is
ignored and we create all new arenas.  It's possible that allows (or
unintentionally forces) the system malloc() to clump new allocations
together better so that they can be released back to the system sooner.
A helpful side-effect if so, but I wouldn't count on it being true for
all malloc libraries.


On Oct 14,  2:27pm, Peter Stephenson wrote:
}
} I'd suggest a quick finger test with the completion system to see if
} anything is obviously worse --- if anything is going to exercise large
} function stacks with small-to-medium size functions, it's that.

Actually the completion system is the place it's most likely to entirely
break down -- completion already uses NEWHEAPS() to get its own heap, and
then frequently employs SWITCHHEAPS() to be able to pass values back on
the regular heap.  If we're also swapping the regular heap around with
NEWHEAPS() it's possible things might get very confused, or that an entire
list of heaps might be leaked.

However, I did just run the Test/Y* tests with valgrind and they all passed
and reported no leaks.  (I did get:

==32081== Warning: invalid file descriptor -1 in syscall close()
==32081== Warning: invalid file descriptor -1 in syscall close()

from both Y01 and Y02 but curiously not Y03.)