Re: [doc] "sh_word_split nothing to do with word splitting"?

[Stephane Chazelas <stephane.chazelas@xxxxxxxxx>]

2018-03-24 21:17:48 +0100, Martijn Dekker:
> Op 12-03-18 om 08:43 schreef Stephane Chazelas:
> > I'd rather zsh keep it that way as a statement of resistance
> > against silliness, at least until someone complains that his
> > POSIX script fails when run on zsh because its arithmetic
> > expansions are not split as expected.
> 
> The thing is, there are very good reasons for that silliness, and for
> not resisting it in sh emulation.
> 
> In a way, POSIX is primarily descriptive, not prescriptive. It intends
> to incorporate the historical practice that real-world shell
> implementations have copied into the standard, so that everyone can be
> compatible with it.
> 
> For the most part, that historical practice was established by AT&T
> ksh88. It's mostly obsolete now but very much lives on in all the shells
> that copy its behaviour.
[...]

There have been many cases where POSIX have specified accidents
of implementation or design bugs of original implementations and
the POSIX specification has been fixed later on when spotted
even in some cases forcing the original implementation to be
fixed.

IMO, here, it's clearly one of those. The solution here is not to
implement the bug but fix the specification.

> ksh88 implemented the original shell arithmetic expansion. It subjects
> those expansions to IFS field splitting.
> 
> Not emulating this in zsh's sh emulation mode will probably not cause
> many scripts to fail on zsh-as-sh. However, someone could use zsh to
> develop a POSIX script intended to be compatible with other shells.
> 
> And that someone might end up under the mistaken impression that it's
> pointless to quote arithmetic expansions. Cue potential mysterious
> breakage on non-zsh shells.

There are many "unspecified areas" in POSIX where it's the case,
if you rely on unspecified behaviour you can't expect anything
portably.

Here I would expect POSIX, when fixed would at first leave the
behaviour unspecified to allow both the bogus historical
behaviour and the zsh behaviour (which until not so long ago was
the behaviour of most other shells, it's quite sad to see that
many have broken their shell to meet POSIX compliance), maybe
with a "future direction" requiring the zsh behaviour.

In the mean time, we should document the difference so it's no
longer "mysterious" like in my  suggested patch.

> That's why I believe it's important that this is fixed in sh emulation.
> 
> > pdksh was not doing word/field splitting there and posh and
> > OpenBSD sh still don't.
> 
> Then pdksh, supposedly a ksh88 clone, failed to clone ksh88 in that
> aspect -- among many others.
[...]

pdksh has fixed a number of issues in ksh88 (itself broken on
many aspects as you've found out) as well. Another one shared
with zsh: that the ".*" glob does not include "." nor ".." (in
the case of pdksh actually inherited from the Forsyth shell).
That also breaks POSIX compliance, though last time I mentioned
it on the Austin Group mailing list, they would likely be
willing to change the spec to allow it (POSIX already leaves it
undefined if readdir() returns those IIRC).

Another example is brace expansion (also supported by zsh and bash)
which also breaks POSIX compliance which was not in ksh88, added
in pdksh (1992), and then later in ksh93.

-- 
Stephane