Re: Aliasing separators (Re: grammar triviality with '&&')

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Mar 6,  9:40am, Peter Stephenson wrote:
} Subject: Re: Aliasing separators (Re: grammar triviality with '&&')
}
} OK, to state my basic position (though it's kind of moot --- as I said I
} don't think anybody really needs the change)  1. tokenisation is part of
} lexing  2. alias expansion comes between lexing and parsing  3. any
} result of lexing is game for alias expansion, unless you make stricter
} rules than zsh already has.  But this discussion isn't really going
} anywhere.

Understood, but (prior to 34641) zsh *did* have stricter rules, and (in
terms of the lexer, not in terms of explaining to end users) the rule
was very simple:  only STRING tokens are subject to alias expansion.

In practice that means something like "only tokens that can be changed
by concatenating with another string using simple lexical pasting, may
be aliased."  But that isn't a very satisfying way to say it (and it's
not 100% true because of "{" being a reserved word).

As an aside, using zshlextext isn't really correct either if the real
intention is to allow aliasing of tokens.  Did you plan to allow the
aliasing of the NEWLIN token?  Because with 34641,

    alias $'\n'=...

does not work, but

    alias '\n'=...

actually does create an alias for hitting enter at a blank PS1 prompt.
The point being that for non-STRING tokens, zshlextext doesn't always
represent the actual input string.

The other minor point is that this slows down lexical analysis a lot.
Many more things are going through checkalias(), including in some
cases (as you pointed out) every individual character.

Finally it seems wrong that "setopt POSIXALIASES" disallows aliasing of
reserved words but (with 34641) still allows aliasing of other special
tokens.


} On Thu, 5 Mar 2015 17:42:40 -0800
} Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
} > torch% &&bar
} > 
} > I would argue that the "&&" is NOT "in command position" because in the
} > normal lexical situation "command position" ENDS just to the left of any
} > separator.  There's NOTHING in "command position" in that example.
} 
} Well, that's not how the lexer actually works.  It's been told it's in
} command position and it fetches the next token.  So whatever comes at
} the start of the line *must* be in command position.

This is curiously flipped around from the previous discussion; now you're
arguing from the strict lexer POV and I'm talking about what it ought to
mean to the end user.

The lexer can certainly be (and was, before, though it was not explicitly
stated) smart enough to know that any token that arrives at that point
with tokstr == NULL is not in fact something that could be a command and
therefore shouldn't be treated as one.

} Given that, in any case, no one is actually suggesting we change the
} lexer to do something different with "&&" I don't think I see the
} relevance anyway.  "&&" is a token and either expanded as an alias or
} not

It's relevant to "alias" vs. "alias -g".  If && at the start of the line
is not in command position, then it doesn't expand unless it has the
global-alias flag.

Incidentally here is a curious bug that is present both (long, long) before
and also after 34641:

torch% alias \{='print foo'
torch% { this is a test
foo this is a test

So far so good, now recall from history:

torch% { this is a test

Back up and delete the space after "{":

torch% {this is a test   
foothis is a test

Now recall from history again:

torch% { foothis is a test

That seems like an oopsie.  I'd actually rather the disambiguating space
after the brace was inserted when the alias expands instead of pasting
up "foothis".

But let's figure out whether to undo/redo all or part of 34641 first.

-- 
Barton E. Schaefer