Re: Quoting and ${(e)param} (was Re: destructive list-expand)

[Sven Wischnowsky <wischnow@xxxxxxxxxxxxxxxxxxxxxxx>]

I probably should have said some more.

Actually, the current state really isn't that far away from the right
thing. The (e) flag should make only the $-expansions from the value be
done on the result. Because of that parsing the string as in double
quotes is the right thing. Using parse_subst_string() or adding the
`parsestr()-as-if-not-in-quotes' Bart suggested would make glob pattern
be expanded, too.

There are only two problems: we have to handle the case when the
expansion stored in the parameter value (the one we have to expand after
(e) has done its work) needs to expand to more than one word. In that
case we have to selectively `de-quote' some of the string -- and that's
what the loop we're discussing does. It just de-quotes too many
Qstring's, namely those inside quoted nested expansions. That would be
fixed by the patch I sent (and that makes me like that patch quite a bit).

The other problem is the tokenization of pattern *inside* parameter
expansions only (it isn't a problem if the parameter we are using the
(e) flag on contains $(..) or $((..)) expansions -- both of them take
care of tokenization themselves).

As Bart correctly pointed out (I knew that and probably should have
explained it some more), there are two things playing together.
subst_parse_string() tokenizes as if in double quotes (which, as I said
above, I think is correct), which also means that the patterns inside
parameter expansions are not tokenized. Normally if one does `"${x$*}"',
paramsubst() will take of that because the `$' is turned into a Qstring
token, so paramsubst() knows that the pattern isn't tokenized and does
it itself. But if that string comes from subst_parse_string(), the
Qstring will be turned into a String token, but the pattern will not be
tokenized -- and hence paramsubst() will not tokenize it itself.

Completely tokenizing the string resulting from a (e)ed parameter
expansion isn't an option, because that would also tokenize patterns
outside of parameter expansions -- we get globbing which we don't want
to have there, that's the job of `${~x}'. At least, it would be quite a
serious change if we modified this to do, e.g., globbing if the `${(e)x}'
isn't in quotes and only the other expansions if one uses `"${(e)x}"'.

So, subst_parse_string() has to do the de-quoting to be able to take
into account the way the expansion with the (e) was quoted or not.

I think all this could be solved if we find a way to tell the
paramsubst() doing the expansion in the value of the (e)ed parameter
that it has to tokenize the pattern even if there is a String token and
not a Qstring token. But currently we don't have a way to do that, or at
least none I can think of. It's problematic, because the first
paramsubst() just passes the resulting words back to it's calling
function which then re-examines them, expanding the now tokenized
expansions.

So the easiest solution would be to just make paramsubst() always
tokenize the pattern strings, not only if it knows that the parameter
expansion is in quotes. I was fearing that this might result in some
quoting problems (having to double backslashes or some such), but it
seems to work, including handling of parameter expansions inside
patterns.

Just so that everyone interested can easily play with it, I'll add a
patch below containing both changes (just #if'ed out the test).

The tests at least still work for me...

Bye
  Sven

Index: Src/subst.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/subst.c,v
retrieving revision 1.17
diff -u -r1.17 subst.c
--- Src/subst.c	2001/04/28 17:38:01	1.17
+++ Src/subst.c	2001/05/17 09:03:40
@@ -720,9 +720,13 @@
 
     if (!(err ? parsestr(s) : parsestrnoerr(s))) {
 	if (!single) {
+            int qt = 0;
+
 	    for (; *s; s++)
-		if (*s == Qstring)
+		if (!qt && *s == Qstring)
 		    *s = String;
+                else if (*s == Dnull)
+                    qt = !qt;
 	}
 	return 0;
     }
@@ -1483,7 +1487,11 @@
 	case '#':
 	case Pound:
 	case '/':
+#if 0
 	    if (qt) {
+#else
+            {
+#endif
 		int one = noerrs, oef = errflag, haserr;
 
 		if (!quoteerr)

-- 
Sven Wischnowsky                         wischnow@xxxxxxxxxxxxxxxxxxxxxxx