Re: Bug with bash emulation regarding ':'

[Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>]

On Wed, 01 Feb 2012 08:29:29 -0800
Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> On Jan 31,  8:29pm, Bart Schaefer wrote:
> }
> } 2657               opts[SHWORDSPLIT] = 0;
> } 2658               multsub(&val, 0, NULL, &isarr, NULL);
> } 
> } The problem is that we want to suppress any further splitting in the
> } current shell, but when we enter the subshell for $(...) it should
> } revert to the original setting.
> 
> I can verify this by defining _test with "emulate sh -c ..." so that
> the SHWORDSPLIT option becomes sticky in that function.
> 
> I'm still not sure what to do about it.  The question goes a bit deeper
> in that there are several ways that shell code might get invoked during
> stringsubst() -- the (e::) glob flag, dynamic directory names, etc. --
> and presumably those should also all use the original splitting style,
> whereas simple nested expansions should not.

Realistically, I think we either have to bite the bullet and find
something ugly that works or forget about it entirely.

How about this for the original problem?  It's at least a simple change,
tied to one documented C variable.  It doesn't address the related areas
you mentioned, but I think those are less important, because they don't
affect Bourne/POSIX emulation and though in principle we should
certainly make all combinations of features behave consistently, in
practice I'm not keen on encouraging native zsh programmers to use
arbitrary option settings.  (Actually, I don't really want to think
about what 'a native zsh programmer' might mean...)

There's still a difference from bash, where the output includes newlines
(i.e. the literal function output), whereas zsh turns the newlines into
spaces.  We use IFS to split the value returned by the function, but
bash doesn't.  Defining an IFS without $'\n' gives us what bash does,
but that's not the difference since bash has $'\n' in the default IFS,
too.  So this difference is somewhere in word splitting for parameter
substitution, too.  I haven't looked into the details, but I would guess
that in bash

: ${...:=$(...)}

does what we would do with

: ${...:="$(...)"}

However, KSH_TYPESET doesn't change this (and as noted recently is not
on for bash anyway).  I would guess KSH_TYPESET *should* have that
effect, and am inclining to the view turning it on for SH mode would be
sensible.  So more investigation is needed.  Feel free.

Index: Src/exec.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/exec.c,v
retrieving revision 1.207
diff -p -u -r1.207 exec.c
--- Src/exec.c	16 Jan 2012 22:14:11 -0000	1.207
+++ Src/exec.c	5 Feb 2012 19:39:50 -0000
@@ -3724,6 +3724,19 @@ getoutput(char *cmd, int qt)
     redup(pipes[1], 1);
     entersubsh(ESUB_PGRP|ESUB_NOMONITOR);
     cmdpush(CS_CMDSUBST);
+    /*
+     * Special case if this is a command substitution called
+     * recursively from parameter substitution.  In that
+     * case we've tampered with SHWORDSPLIT in order to fix
+     * up parameter substitution arguments.  However, here
+     * we're back at command level, so we need to restore
+     * it to whatever it was before.  Then we'll pretend
+     * we never tampered with it.
+     */
+    if (real_shwordsplit != -1) {
+	opts[SHWORDSPLIT] = real_shwordsplit;
+	real_shwordsplit = -1;
+    }
     execode(prog, 0, 1, "cmdsubst");
     cmdpop();
     close(1);
Index: Src/subst.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/subst.c,v
retrieving revision 1.130
diff -p -u -r1.130 subst.c
--- Src/subst.c	21 Dec 2011 22:39:28 -0000	1.130
+++ Src/subst.c	5 Feb 2012 19:39:50 -0000
@@ -35,6 +35,22 @@
 /**/
 char nulstring[] = {Nularg, '\0'};
 
+/*
+ * The real value of the option SH_WORD_SPLIT.
+ * We need this because of the following hierarchy:
+ * - paramsubst alters SH_WORD_SPLIT
+ * - multsub() does some substitution with SH_WORD_SPLIT off
+ * - a nested $(...) executes commands.  This should reflect
+ *   the original value of SH_WORD_SPLIT.
+ *
+ * -1 indicates this isn't in use.
+ *
+ * Even so, yuk.  This will be done properly in the object-oriented rewrite
+ * of zsh currently scheduled for the year 3712, weather permitting.
+ */
+/**/
+int real_shwordsplit = -1;
+
 /* Do substitutions before fork. These are:
  *  - Process substitution: <(...), >(...), =(...)
  *  - Parameter substitution
@@ -213,7 +229,16 @@ stringsubst(LinkList list, LinkNode node
 		setdata(node, (void *) str3);
 		continue;
 	    } else {
+		/*
+		 * Save and restore the value of SHWORDSPLIT.
+		 * See notes on real_shwordsplit above.
+		 * The restore so that we don't use real_shwordsplit
+		 * outside code called from paramsubst().
+		 */
+		char old_shwordsplit;
+		old_shwordsplit = real_shwordsplit = opts[SHWORDSPLIT];
 		node = paramsubst(list, node, &str, qt, ssub);
+		real_shwordsplit = old_shwordsplit;
 		if (errflag || !node)
 		    return NULL;
 		str3 = (char *)getdata(node);
Index: Test/D04parameter.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/D04parameter.ztst,v
retrieving revision 1.60
diff -p -u -r1.60 D04parameter.ztst
--- Test/D04parameter.ztst	17 Aug 2011 19:00:10 -0000	1.60
+++ Test/D04parameter.ztst	5 Feb 2012 19:39:51 -0000
@@ -255,6 +255,20 @@
 >two
 >words
 
+  (setopt shwordsplit # ensure this doesn't get set in main shell...
+  test_splitting ()
+  {
+    array="one two three"
+    for e in $array; do
+      echo "'$e'"
+    done
+  }
+  test_split_var=
+  : ${test_split_var:=$(test_splitting)}
+  echo "_${test_split_var}_")
+0:SH_WORD_SPLIT inside $(...) inside ${...}
+>_'one' 'two' 'three'_
+
   print -l "${(f)$(print first line\\nsecond line\\nthird line)}"
 0:${(f)$(...)}
 >first line

-- 
Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/