Re: memory usage with + / e glob qualifiers

[Mikael Magnusson <mikachu@xxxxxxxxx>]

On Fri, Oct 9, 2026 at 1:46 PM Stephane Chazelas <stephane@xxxxxxxxxxxx> wrote:
>
> Hi,
>
> I was surprised to find out that globs such as **/*(N+false)
> that expand to nothing end up using more memory than **/*
> according to time's %M
>
> It's even worse when combined with -:
>
> $ du --count-links --inodes --summarize
> 675285  .
>
> $ (TIMEFMT=%M; time zsh -c ': **/*(N+false)')
> 225176
> $ (TIMEFMT=%M; time zsh -c ': **/*(N-+true)')
> 370828
> $ (TIMEFMT=%M; time zsh -c ': **/*')
> 159624
>
> With another glob that expands to nothing without using the +
> qualifier:
>
> $ (TIMEFMT=%M; time zsh -c ': **/*(N./)')
> 48176
>
> Similar with the e qualifier:
>
> $ (TIMEFMT=%M; time zsh -c ': **/*(N-e[])')
> 233312
>
> What could explain that?

The eprogs allocated to run your shell code are allocated on 'the
heap' (eg, the pushheap/popheap zsh uses), but nothing pops this until
the whole glob is finished, so you may get quite a high peak memory
usage. However, the result from each eprog might also be allocated on
the heap, so you can't just push/pop it around each filename being
processed.

static int
qualsheval(char *name, UNUSED(struct stat *buf), UNUSED(off_t days), char *str)
{
    Eprog prog;

    if ((prog = parse_string(str, 0))) { <-- eprog allocated on the heap
        int ef = errflag, lv = lastval, ret;
        int cshglob = badcshglob;

        unsetparam("reply");
        unsetparam("REPLY");
        setsparam("REPLY", ztrdup(name));
        badcshglob = 0;

        execode(prog, 1, 0, "globqual");  <- we don't need the eprog
after this point

        if ((ret = lastval))
            badcshglob |= cshglob;
        /* Retain any user interrupt error status */
        errflag = ef | (errflag & ERRFLAG_INT);
        lastval = lv;

        if (!(inserts = getaparam("reply")) &&
            !(inserts = gethparam("reply"))) {
            char *tmp;

            if ((tmp = getsparam("reply")) || (tmp = getsparam("REPLY"))) {
                static char *tmparr[2];

                tmparr[0] = tmp; <- but this string is allocated on
the same heap and stored in an array until after the whole glob
finishes
                tmparr[1] = NULL;

                inserts = tmparr;
            }
        }

        return !ret;
    }
    return 0;
}

so if you want to reduce this peak memory usage, some surgery would be
required. My guess is that passing through a flag to parse_string to
do a permanent allocation so we can free prog after execode would be
the easier route, but I haven't looked into it very hard.

-- 
Mikael Magnusson