Re: [PATCH] _arguments: Escape colons and backslashes in $opt_args unambiguously.

[Daniel Shahaf <d.s@xxxxxxxxxxxxxxxxxx>]

Bart Schaefer wrote on Wed, Sep 07, 2016 at 00:03:20 -0700:
> On Sep 4,  6:26pm, Daniel Shahaf wrote:
> }
> } Is it okay to refer to NEWS from the manual?
> 
> Are there any other examples of the manual referring directly to the
> behavior of previous versions?

Yes: CONTINUE_ON_ERROR in zshoptions(1).  (And the usage examples under
is-at-least, but that doesn't count.)

> It's not the kind of thing it's easy to search for,

I found that with:
.
    ag -C 3 -G 'yo$' '[345][.][0-9]' Doc/Zsh
.
which is equivalent to:
.
    grep -C 3 '[345][.][0-9]' Doc/Zsh/**/*yo
.
where the regexp is designed to catch references to zsh version numbers
between 3.0 and the present day.  'ag' is usually packaged as
"silver_searcher" in distros.

> but I can't think of any, so my inclination would be to drop your
> parenthetical Note:.

> } In the manual I wrote "See NEWS" even though the pointed-to text is in
> } README because (a) NEWS includes README by reference, (b) I doubted
> } readers would follow a reference to README in that context.
> 
> I doubt readers will follow a NEWS reference either.  If this is
> important enough to reference -- which I don't think it is -- then
> it's important enough to say it here, not cross-reference somewhere
> outside of the regular documentation tree, and certainly it shouldn't
> require going to NEWS, finding nothing, and then having to realize
> it might be in README.
> 

I'm happy to drop the note entirely, or to drop the reference to NEWS
and keep the sentence stating the behaviour difference.

> } +ambiguity: if the -x option took two arguments (as in
> } +    _arguments : -x:foo:${action}:bar:$action
> } +), it would be impossible to tell from $opt_args whether the command-line
> } +was '-x foo\:bar' or '-x foo\\ bar'.
> 
> Is this example correct?  Isn't the actual ambiguity between
> [[[ -x foo\:bar ]]] (one arg) and [[[ -x foo bar ]]] (two args)?

The example is correct, and the two cases you give are not ambiguous
with each other:

% echo $ZSH_PATCHLEVEL                                                                                                                                       
zsh-5.2-dev-1-335-g831a336
% _f() { local context state state_descr line; typeset -A opt_args; _arguments : -x:foo:-\>state:bar:-\>state2;  >/dev/tty { echo; typeset -p opt_args } }
% compdef _f f
% f -x foo\:bar<TAB><^C>
typeset -A opt_args=( -x 'foo\\:bar' )
% f -x foo\\ bar<TAB><^C>
typeset -A opt_args=( -x 'foo\\:bar' )
% f -x foo bar<TAB><^C>
typeset -A opt_args=( -x foo:bar )

What's happening here is that (a) every colon in the word gets preceded
by a backslash, (b) colons are inserted between words.  No other
transformation happens.  Consequently:

$BUFFER    colons to be escaped?  colons inserted?   result in ${opt_args[-x]}  which character(s) were inserted
=======    =====================  ================   =========================  ================================
foo\:bar   yes                    no                 foo\\:bar                  second backslash
foo\\ bar  no                     yes                foo\\:bar                  colon
foo bar    no                     yes                foo:bar                    colon

Again, that's before this patch.  With this patch the first two cases
are subjected to backslash doubling at the same time as colon escaping,
so they result in «foo\\\:bar» [the first and third slashes were added]
and «foo\\\\:bar» [ditto] respectively.

> If I'm wrong, what is it about your explanation that confused me?

I don't know.

Perhaps you assumed words were joined with backslash-escaped colons.

Perhaps you misparsed the phrase «was '-x foo\:bar' or '-x foo\\ bar'».
There are three distinct levels of quoting at play here: interactive
input quoting, ca_colonlist() quoting, and the English prose quoting,
where single quotes were used for the function tt() provides in yodl
documents.

How might the README text be improved?

> There's probably still an ambiguity between [[[ -x foo bar ]]] and
> [[[ -x foo -x bar ]]] ...

This depends on the spec of -x.

If -x takes two mandatory arguments, then the first case should parse as
as [opt_args=(-x foo:bar)] and the second one as [opt_args=(-x foo:-x)
with 'bar' as a positional argument].

However, if -x is repeatable and takes either one or two arguments — as in
«_arguments : '*-x:foo:->foo::bar:->bar'» — then I'd say the
command-line in the second case is ambiguous.

Good catch.

Cheers,

Daniel