Re: Command substitution parsing issues (not really Re: completion)

[Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>]

On Fri, 16 Jan 2015 18:57:32 -0800
Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> I still get:
> 
> schaefer<501> echo ÿÿÿÿÿÿÿÿ$(ls)
> Completing all expansions
> ÿÿÿÿÿÿÿÿ$(ls)
> Completing original
> $(ls)

I've narrowed this down (note no completion in the following repro,
hurrah):

% alias foo='echo poop'
% echo ${(e):-'$(foo)'}
ÿÿÿÿÿÿÿÿ$(foo)

This comes from the call from subst_parse_str() to parsestr() and
parsestrnoerr(), which make unwarranted and undocumented assumptions
about the way the lexer works, in particular that if it passes in a
variable s then any parsing happens within that string --- a pretty
stupid assumption.  The interface needs fixing up, which
ought to be easy enough just by ensuring parsestrnoerr() passes back a
reference to tokstr --- however, I see parsestr is called all over
the place so I'm not going to have time to look now.

We also need to ensure and document that the string parsed in comes from
the heap --- I think that's already a requirement, just nobody got
around to writing it down.  In fact parsestrnoerr() appears to be doing
all sorts of hair-raising things without documentation --- why set the
input string and then immediately after set the lexical buffer to the
original string?

This is nothing fundamentally to do with the rest of the change.  If
anyone else has time, go ahead --- this is killing off my free time at
the moment.

pws