Re: PATCH: nested ${(P)} (formerly SHWORDSPLIT and leading spaces)

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

[PWS mis-addressed his original message to "zsh-hackers" so I've included
it in its entirety below.]

On Nov 14, 10:31pm, Peter Stephenson wrote:
}
} On 14 Nov 2015 18:51, Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
} >
} >     1. there is an outer ${...} around the ${(P)...} expansion 
} >     2. the (P) is combined with one of (l), (r), or (j). 
} >
} > If there's only one level, ${(Pr.N.)...} et al. work as expected. 
}
} Yes, that's also deliberate. As there is no outer layer here, the code
} executed is identical to before (the only alternative would be yet a
} third type of behaviour; we can0x02BCt fake up a whole surrounding
} layer) . If the (P) is surrounded by a an outer layer, the effect
} mentioned before hsppens: the inner layer refers to the name, and
} the outer to the substituted value. As I already said, if we are
} separating the effects of layers to get a sensible effect with e. g.
} subscripts, there is no question of flags in the inner layer having an
} effect on the value. It is not just a minor side effect.
}
} pws


OK ... but this means we definitely need some updates to the "Rules"
section.  For example, rule #2 says that effect of "typeset -u"
should happen before the effects of (P) at rule #4.  This appears to
have been incorrect at least as far back as zsh 4.2 -- the name
deref'd by (P) is the raw value of the variable, though whether the
internal flags might change the raw value is different in 4.2 -- so
given:

torch% ARRAY=(foo bar)
torch% array=(one two)
torch% name=array
torch% typeset -u name
torch% print $name
ARRAY

In zsh-5.1.1:

% print ${(P)name} 
one two
% print ${${(P)name}}
one two

With zsh-5.1.1-160-gd5ba08a, nesting the (P) makes -u take effect:

torch% print ${(P)name}
one two
torch% print ${${(P)name}}
foo bar

Aside:  It's also worth remembering that the LRZul "internal flags"
have never worked for arrays; you can set them and they're remembered
but they have no effect on the individual array elements.

Subscripting #3 applies before (P) #4 as described, even when (P) is
within a nested substitution, but applies after #2 when there is no
(P) involved, so we somehow have to explain how the order of 2/3/4
changes when a nested (P) is introduced.

Here's a stab at it (and also a fix for _git, which appears to be the
only contributed function that relied on the former behavior).


diff --git a/Completion/Unix/Command/_git b/Completion/Unix/Command/_git
index 3dfd604..614185e 100644
--- a/Completion/Unix/Command/_git
+++ b/Completion/Unix/Command/_git
@@ -5244,7 +5244,7 @@ _git_commands () {
   for cmdtype in aliases $cmdtypes; do
     local -a ${cmdtype}_d
     (( $#disp )) && set -A ${cmdtype}_d \
-        ${${(Pr.COLUMNS-4.)cmdtype/(#s)(#m)[^:]##:/${(r.len.)MATCH[1,-2]} $sep }%% #}
+        ${${(r.COLUMNS-4.)${(P)cmdtype}/(#s)(#m)[^:]##:/${(r.len.)MATCH[1,-2]} $sep }%% #}
     alts+=( "${cmdtype//_/-}:${${cmdtype//_/ }%%(e|)s}:compadd ${(e)disp} -a ${cmdtype}_m" )
   done
 
diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index 4c373d1..6f08d7d 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -1380,9 +1380,13 @@ outermost.  The flags are not propagated up to enclosing
 substitutions; the nested substitution will return either a scalar or an
 array as determined by the flags, possibly adjusted for quoting.  All the
 following steps take place where applicable at all levels of substitution.
-Note that, unless the `tt((P))' flag is present, the flags and any subscripts
-apply directly to the value of the nested substitution; for example, the
-expansion tt(${${foo}}) behaves exactly the same as tt(${foo}).
+
+Note that, unless the `tt((P))' flag is present, the flags and any
+subscripts apply directly to the value of the nested substitution; for
+example, the expansion tt(${${foo}}) behaves exactly the same as
+tt(${foo}).  When the `tt((P))' flag is present in a nested substitution,
+the other substitution rules are applied to the value em(before) it is
+interpreted as a name, so tt(${${(P)foo}}) may differ from tt(${(P)foo}).
 
 At each nested level of substitution, the substituted words undergo all
 forms of single-word substitution (i.e. not filename generation), including
@@ -1400,6 +1404,12 @@ in particular the tt(L), tt(R), tt(Z), tt(u) and tt(l) flags for padding
 and capitalization, are applied directly to the parameter value.
 Note these flags are options to the command, e.g. `tt(typeset -Z)';
 they are not the same as the flags used within parameter substitutions.
+
+At the outermost level of substitution, the `tt((P))' flag ignores these
+transformations and uses the unmodified value of the parameter as the name
+to be replaced.  This is usually the desired behavior because padding may
+make the value syntactically illegal as a parameter name, but if
+capitalization changes are desired, use the tt(${${(P)foo}}) form.
 )
 item(tt(3.) em(Parameter subscripting))(
 If the value is a raw parameter reference with a subscript, such as
@@ -1413,8 +1423,10 @@ original array).  Any number of subscripts may appear.  Flags such as
 tt((k)) and tt((v)) which alter the result of subscripting are applied.
 )
 item(tt(4.) em(Parameter name replacement))(
-The effect of any tt((P)) flag, which treats the value so far as a
-parameter name and replaces it with the corresponding value, is applied.
+At the outermost level of nesting only, the effect of any tt((P)) flag,
+which treats the value so far as a parameter name and replaces it with the
+corresponding value, is applied.  This replacement occurs later if the
+tt((P)) flag appears in a nested substitution.
 )
 item(tt(5.) em(Double-quoted joining))(
 If the value after this process is an array, and the substitution
@@ -1534,6 +1546,11 @@ Strictly speaking, the removal happens later as the same happens with
 other forms of substitution; the point to note here is simply that
 it occurs after any of the above parameter operations.
 )
+item(tt(25.) em(Parameter name replacement))(
+If the `tt((P))' flag is present and this has not yet been done, the value
+so far is looked up as a parameter name.  Errors may occur if the value is
+neither a valid identifier nor an identifier plus subscript expression.
+)
 enditem()
 
 subsect(Examples)