Re: Removing subshell from zargs (see "zargs with -P intermittently failing")

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Mon, May 30, 2022 at 9:34 AM Jun. T <takimoto-j@xxxxxxxxxxxxxxxxx> wrote:
>
> Sorry I missed this post. And I fear I will not be able to going into
> any detail at least for a few days.

No worries.

> (3) I tried the following dirty patch for debugging:
>
[...]
>
> When the problem occurs, I get:
>
> % zargs -n 1 -P 19 -- {1..40} -- f; jobs
> zsh: 0 0
> zsh: 1 17504    # thisjob = ignorejob

Aha.  Yes, this is what the original subshell + wait was intended to
prevent.  I think what's happened here is that "waid $j" was looking
for the exit of (what is here shown as) job 0, but job 1 exited first,
and zsh had to handle but disregard that exit in order to continue
waiting for the job whose status was wanted.  Doing a full "wait"
beforehand guarantees that we can return the status of any job without
having to handle intervening signals, and theoretically doesn't slow
us down because we eventually have to wait for all the jobs anyway.

> zsh: 8 2137
> [8]  + done       { "${call[@]}"; }
>
> maxjob = 8 here, but it should have been decremented to 1?
> The only place maxjob is decremented is in freejob(), I guess.

Yes, once a job is in (in this case) job table slot 8, new jobs have
to use 9 and up even if the jobs in slots 0 through 7 have already
exited.  Once the current maxjob slot is reclaimed, all slots before
it that point to exited jobs can be reclaimed too.

The question is why slot 8 wasn't reclaimed until "jobs" ran.  Likely
related in some way to the order in which the jobs exited.

Harmless, I guess, but potentially confusing enough that the subshell
wrapper should be left in place?  Given the behavior above, it would
also appear possible to fill up the parent shell job table by having
maxjobs get "stuck" at increasingly larger slots.

  Or else it's related to this:

    # These setopts are necessary for "wait" on multiple jobs to work.
    setopt nonotify nomonitor

Those options are local to the zargs function, of course, but perhaps
they should actually be inside the subshell?