Re: The big kre zsh bug report

[Robert Elz <kre@xxxxxxxxxxxxx>]

    Date:        Thu, 20 Dec 2018 22:25:12 +0000
    From:        Martijn Dekker <martijn@xxxxxxxx>
    Message-ID:  <18f684a8-2fec-4ebe-a63e-cf6688ae519f@xxxxxxxx>

  | Robert Elz a.k.a. kre (author of NetBSD sh)

definitely not author, but I am doing most of the current maintenance
(and a few upgrades).

  | All further quoted text below is from kre... (Note his test suite output 
  | doesn't quote the '-c' option argument, but the tests are executed as if 
  | it were quoted with single quotes.)

Yes, sorry about that, the tests are run by code that almost everyone would
like to replace, and which is mostly not maintained (but, aside from mostly
cosmetic issues like this, does work)...   That's where that output comes from.


  | Yes, I don't think the "${op}" being quoted has anything to do with it. 
  | It fails only at the second loop iteration, so with '-' as the operator.
  |
  | So it doesn't like this:
  |
  | $ zsh --emulate sh -c 'echo $(( $(echo 9) $(echo -) $(echo 2) ))'
  | zsh:1: bad math expression: operator expected at `2 '
  | $ zsh --emulate sh -c 'echo $(( 9 `echo -` 2 ))'
  | zsh:1: bad math expression: operator expected at `2 '

I thought I corrected this one, either later in the big message, or a 
subsequent one.   The issue there is the use of echo where the
first arg starts '-' which is definitely an undefined area (the AT&T
idiots who broke the original Bell Labs echo have a lot to answer
for!)    I am going to change my tests to use printf instead.  There
is no issue here with zsh.

  | $ zsh --emulate sh -c 'set -- a b c; echo ${#:-99}'
  | 2
  |
  | That should be 3, so this is a bug.

Dealing with # in parameter expansions is painful, as it has
so many possible meanings, which need to be figured out
by context (not all of which have defined results).

But where it can reasonably be ${#} (with perhaps some
extra modifiers) it really ought to be.

  | Confirmed. Modernish identifies this as a shell quirk (QRK_IFSFINAL) 
  | because the POSIX spec is quite ambiguous on this, so I was unable to 
  | confirm that it is actually a bug and not a legitimate behaviour variant.

It is not supposed to be ambiguous any more - it used to be once.

In XCU 2.6.5:

The shell shall treat each character of the IFS as a delimiter and use the delimiters as field
terminators to split the results of parameter expansion, command substitution, and arithmetic |
expansion into fields.
[...]
b. Each occurrence in the input of an IFS character that is not IFS white space, along
     with any adjacent IFS white space, shall delimit a field, as described previously.

The std (consistently) uses "delimit" to mean "terminate" not "separate" but 
some readers misunderstand, and assume it means "separate" (as in, occurs 
between)

The intent is that when something is being collected (a field here, a
token in the lexer, ...) and the text says the next char "delimits the xxx"
it means that whatever was being collected is finished, what follows
depends upon what appears (if anything) - the delimiting character
does not, by itself, create anything following.

This is particularly confusing with field splitting, as "IFS" is used, and
the 'S' in that is "separator" which is preisely what it is not.   But the
name is historic.


  | _____________________
  | > Next, zsh apparently does not implement the posix
  | > required -h option.   Nor do we, but we at least allow
  | > it to be set and cleared ....
  |
  | I'm not sure that allowing it to be set and then not doing what it 
  | promises is better than not allowing it to be set at all.

If anything actually differs when that nonsense (implemented as
specified, which I am not sure anything does) is turned on, it would
be something of a surprise, and most likely break everything.

It is just that posix says that the option is supposed to exist, so
some script might turn it on.   Why I could not guess...


  | This issue caused me headaches when implementing match() on modernish. I 
  | was given the impression that passing backslash-quoted characters 
  | through variables for use in 'case' patterns was one of those things 
  | that POSIX doesn't really specify one way or another. Of course I would 
  | much prefer the behaviour of bash, dash, and NetBSD sh.

The pattern matching part of posix is due a rewrite (there is an open issue
with some potential fixes) - the intent is that it should work as you describe,
but it is hard (but not impossible) to read the current text to mean that.

  | _____________________
  | Various other tests showed that zsh cannot handle file descriptors >9. 
  | Not really a bug as POSIX permits that limitation, but:
  |
  | > And again, this test fails the same way with --emulate bash
  | > and bash certainly permits fds > 9!
  |
  | So you could consider this a feature request.

The '9' limit was created when the per-process file descriptor limit
was 16...   The shell needs a few fds for itself, so ...   These days
programs have access to lots more fds, so should sh scripts.


kre