Re: [PATCH 1/3]: Add named references

[Oliver Kiddle <opk@xxxxxxx>]

Bart Schaefer wrote:
> > One ksh feature which this doesn't cover is the special behaviour in ksh
> > that `typeset -n ref=$1` will use the calling function scope to resolve
> > the target.
>
> This should also work after patch 4/3.
>
> % typeset outside=OUTSIDE
> % () {
> typeset -n ref=$1
> print $ref
> } outside
> OUTSIDE
>
> Or do I misunderstand?

In ksh you could pass "ref" as a parameter and the nameref would provide
access to the outside variable despite the name clash.

> > The positional parameters are special-cased which I
> > find rather ugly.
>
> There's nothing explaining this in "man ksh93", can you point me
> somewhere?  Not that your description makes this sound particular
> attractive.

The manual does seem rather vague on this. I've long been aware of it so
the detail may have come to me via the old ast-ksh lists. But to
demonstrate:

% function f {
>   typeset out=99 var=$1
>   nameref ref1=$1
>   nameref ref2=$var
>   echo f: $ref1 $ref2
>   ref1=1
>   ref2=2
> }
% out=0
% f out
f: 0 99
% echo $out
1

So $1 as the target of a nameref is special. I don't especially like
this. Everywhere else in the shell, a variable reference like $1 is no
different from using the text it would expand to explicitly. I don't
think we need to worry about the level of ksh93 compatibility here
because our default dynamic scoping mean a ksh script will work if
variable names don't clash.

> > You could overload -u or -U for indicating Tcl-like
> > uplevel.
>
> It'd be easier to overload "-i N" for this since it already populates
> the "base" integer in Param.

Not with the right number, though. Tcl's uplevel gives a distance (up the
procedure calling stack) to move. 1 being the default. You can prefix
with # to get an absolute level but that doesn't seem especially useful.

So 1 would mean $((locallevel-1)) in absolute terms.
Note that Tcl's uplevel is like a precommand modifier in zsh terms so
is not limited to defining references.

However I still prefer the other solution I mentioned with \&.

> I'm still not entirely following what problem this solves.

It solves the problem of name clashes between a function and the calling
function.

Consider Completion/Base/Utility/_call_function,
It expects the name of a parameter in $1.
It does:
  local _name _ret
  _name="$1"
  ...
  _ret=...
  eval "${_name}=${_ret}"
The underscores are there largely to avoid name clashes. In ksh, this could
instead be:
  nameref ret="$1"
  ret=...
And it wouldn't matter if ret is also the variable name in the calling
function.

> > I think a better solution is to take some syntax that isn't a valid
> > variable name. e.g typeset -n ref=\&1 Repeated use of an initial & would
> > indicate where another level of scope should be ascended. Wrapper
> > functions then don't need their own nameref and can just pass, e.g.
> > \&var as the variable parameter.
>
> You're going to have to write out an example, sorry.

With this idea:
  typeset -n ret=\&1
would be like ksh's:
  typeset -n ret=$1
But you could also do
  typeset -n ret=\&var

The code would strip the initial \&, go up one local level and see what
$var is in that scope. If the result also starts with an &, it would
repeat the process going up a further level. So a wrapper function to,
e.g. _call_function could pass "&$1" directly as the first parameter
instead of needing a local nameref of it's own. You could optionally
allow "&&var" to go up two levels. It doesn't need to be & if you think
a different character is better, it just needs to not be a valid
variable name. Does that make any more sense than the previous
explanation?

> I'm also thinking about that.  The only use-case would be to prevent
> something from doing "typeset +n ref".

Also prevents typeset -n ref=othervar
It is just as useful or useless as -r with other variables.

> > > With respect to zsh/param/private parameters, it's presently
> > > prohibited to have a "public" reference to a private parameter.  It
> > > might be possible to change this if someone can think of a use case.
> >
> > To a user it would seem like a fairly arbitrary restriction.
>
> What purpose does it serve?  How does it help to have two names for
> the same thing, one of which is "hidden" and the other isn't?  The
> only analogy I can think of would be having a writable name for a
> read-only variable.

If two functions are developed independently, they shouldn't need
to worry about the possibility of using the same variable name for
different purposes. This is the main benefit of private in general.
An old function like _call_function (updated to use nameref but still
using local) could be used by someone writing a new function where they
pass a private variable as the first parameter. If they are treating
_call_function as a black box, how are they to know that private
variables are not valid for the passed $1. It'd be an arbitrary
limitation.

> The special behavior of namerefs in "for" loops and implementing
> "unset -n" might be good; the latter isn't mentioned until the "unset"
> builtin in the ksh93 doc so I missed it entirely.

Given that you implemented references to array members, being able to
iterate over array elements with for could be nice. Perhaps something
like: for ref in 'arr[*]'; do
maybe [@] to include empty elements.

On the subject references to array elements, it does seem both powerful and
dangerous that subscripts are evaluated on reference. The subscript
should perhaps be evaluated with the same local level as the array
itself. And it could be wise to limit what can be done as part of the
subscript evaluation to avoid a CVE similar to the last one.

> > I think I also concluded that it
> > wouldn't work for private variables because of their somewhat too clever
> > implementation.
>
> They could be a lot less clever if they could move into the main
> shell.  Most of the cleverness was necessary for making them modular.

I would favour moving them to the main shell. And then making extensive
use of them in completon etc. There are things that could move in the
opposite direction if we want to reduce bloat. Mail checking for
example.

Oliver