Re: Aliasing separators (Re: grammar triviality with '&&')

[Peter Stephenson <p.stephenson@xxxxxxxxxxx>]

OK, to state my basic position (though it's kind of moot --- as I said I
don't think anybody really needs the change)  1. tokenisation is part of
lexing  2. alias expansion comes between lexing and parsing  3. any
result of lexing is game for alias expansion, unless you make stricter
rules than zsh already has.  But this discussion isn't really going
anywhere.

On Thu, 5 Mar 2015 17:42:40 -0800
Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> torch% &&bar
> 
> I would argue that the "&&" is NOT "in command position" because in the
> normal lexical situation "command position" ENDS just to the left of any
> separator.  There's NOTHING in "command position" in that example.
> 
> Either "&&" is a separator token and should act like one, or it isn't
> and in that example the alias for "&&bar" should be looked up instead.

Well, that's not how the lexer actually works.  It's been told it's in
command position and it fetches the next token.  So whatever comes at
the start of the line *must* be in command position.  The parser can
throw an error if it thinks it shouldn't be, but that's after alias
expansion (so much is uncontroversial).

Actually, come to think of it, I think you mean the opposite: normally,
when you encounter a "&&", you're expecting the continuation of the
current command; here, it's the reverse --- you're expecting the start
of a command, and encounter something which only occurs after a command.
So you might argue that you "turn off" "&&" analysis in the same way
that you "turn on" "(" analysis at the same point --- and that example's
relevant because when "(" is at the start of a line we only take one
character at a time, i.e.

print ((foo))
((print foo); print bar)

are treated entirely differently.  (By the way, aliasing "(" here
therefore does exactly what you'd expect: in the second case it gets
replaced as a single token eacht time it occurs, in the first place not.
I don't know of a good use for this, which is a kind of motto for the
current discussion.)

But I don't really buy that; we know a ";" separator has to be detected
at this point whether there's a command there or not.  So there's not
really any sensible reason for not turning "&&" into a token.

Given that, in any case, no one is actually suggesting we change the
lexer to do something different with "&&" I don't think I see the
relevance anyway.  "&&" is a token and either expanded as an alias or
not, just as you get a parse error with ";;" because it's always treated
as a token whether we're in a case or not.

> } > Aliasing only of STRING tokens is exactly the right thing and this
> } > change is simply wrong. The doc only says "before parsing" as a
> } > shorthand instead of a long explaination about how the alias is
> } > replaced and then parsed all over again.
> } 
> } If you can produce an alternative patch describing the previous position
> } properly, go ahead.  I don't think anyone is actually screaming to use
> } this change.
> 
> I'll think about a documation patch if that's what you mean.

Yes, we certainly wouldn't need any code change in the other case.

pws