[PATCH] _arguments: Escape colons and backslashes in $opt_args unambiguously.

[Daniel Shahaf <d.s@xxxxxxxxxxxxxxxxxx>]

---
tl;dr: the encoding of values into $opt_args is ambiguous.

For a longer description of the issue being fixed, see the README hunk
of the patch.  There's a README entry because this incompatibly affects
completion functions that use $opt_args[-x] where the argument to -x is
expected to contain backslashes (as in «foo -c 'hello\world'»).

I found this while looking into the _libvirt backslashed colons, see
39158 (I'm replying to it separately).

I verified this patch by hand, using:
.
    _f () {
        local context state state_descr line
        typeset -A opt_args
        _arguments : -c:foo:-\>state:bar:-\>state2
        typeset -p opt_args > /dev/tty
    }

Is it okay to refer to NEWS from the manual?

In the manual I wrote "See NEWS" even though the pointed-to text is in
README because (a) NEWS includes README by reference, (b) I doubted
readers would follow a reference to README in that context.

A good future improvement would be to teach _arguments to use NUL rather
than colon as the join character of $opt_args values: this would make it
a *lot* easier to split the value, and would relieve $opt_args users
that don't use the (somewhat obscure) option-with-multiple-arguments
syntax from having to manually unescape colons and backslashes.

Cheers,

Daniel

 Doc/Zsh/compsys.yo               |  6 ++++--
 README                           |  9 +++++++++
 Src/Zle/computil.c               | 12 +++++++++---
 3 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/Doc/Zsh/compsys.yo b/Doc/Zsh/compsys.yo
index 8c7ef0f..b2cc392 100644
--- a/Doc/Zsh/compsys.yo
+++ b/Doc/Zsh/compsys.yo
@@ -3948,8 +3948,10 @@ command line after the command name excluding all options and their
 arguments.  Options are stored in the associative array
 `tt(opt_args)' with option names as keys and their arguments as
 the values.  For options that have more than one argument these are
-given as one string, separated by colons.  All colons in the original
-arguments are preceded with backslashes.
+given as one string, separated by colons.  All colons and backslashes
+in the original arguments are preceded with backslashes.  (Note:
+Zsh 5.2 and older did not escape backslashes in the original string;
+see the tt(NEWS) file for details.)
 
 The parameter `tt(context)' is set when returning to the calling function
 to perform an action of the form `tt(->)var(string)'.  It is set to an
diff --git a/README b/README
index 9de5eb4..019294e 100644
--- a/README
+++ b/README
@@ -87,6 +87,15 @@ The "f" qualifier has for many years been the documented way of testing
 file modes; it allows the "and" test ("*(f+1)" is the documented
 equivalent of "*(1)") as well as many other forms.
 
+5) The completion helper function _arguments now escapes both backslashes
+and colons in the values of option arguments when populating the $opt_args
+associative array.  Previously, colons were escaped with a backslash but
+backslashes were not themselves escaped with a backslash, which lead to
+ambiguity: if the -x option took two arguments (as in
+    _arguments : -x:foo:${action}:bar:$action
+), it would be impossible to tell from $opt_args whether the command-line
+was '-x foo\:bar' or '-x foo\\ bar'.
+
 Incompatibilities between 5.0.8 and 5.2
 ---------------------------------------
 
diff --git a/Src/Zle/computil.c b/Src/Zle/computil.c
index ecfa2bc..1c90a54 100644
--- a/Src/Zle/computil.c
+++ b/Src/Zle/computil.c
@@ -2310,7 +2310,10 @@ ca_parse_line(Cadef d, int multi, int first)
     return 0;
 }
 
-/* Build a colon-list from a list. */
+/* Build a colon-list from a list.
+ *
+ * This is only used to populate values of $opt_args.
+ */
 
 static char *
 ca_colonlist(LinkList l)
@@ -2320,16 +2323,19 @@ ca_colonlist(LinkList l)
 	int len = 0;
 	char *p, *ret, *q;
 
+	/* Compute the length to be allocated. */
 	for (n = firstnode(l); n; incnode(n)) {
 	    len++;
 	    for (p = (char *) getdata(n); *p; p++)
-		len += (*p == ':' ? 2 : 1);
+		len += (*p == ':' || *p == '\\') ? 2 : 1;
 	}
 	ret = q = (char *) zalloc(len);
 
+	/* Join L into RET, joining with colons and escaping colons and
+	 * backslashes. */
 	for (n = firstnode(l); n;) {
 	    for (p = (char *) getdata(n); *p; p++) {
-		if (*p == ':')
+		if (*p == ':' || *p == '\\')
 		    *q++ = '\\';
 		*q++ = *p;
 	    }