Re: [PATCH 1/3]: Add named references

[Oliver Kiddle <opk@xxxxxxx>]

Bart Schaefer wrote:
> My general feeling is to be ksh compatible where I don't think ksh is stupid.

Sounds reasonable.

> > It solves the problem of name clashes between a function and the calling
> > function.
>
> OK, so if
>   typeset ref
>   () { typeset -n ref=ref }
> works / is not a loop, then we don't need any other magic?

The following is similar:
  var=hello
  typeset -n ref
  () {
    typeset var=x
    ref=var
    echo $ref
  }
  typeset -p ref
  echo $ref

This creates a reference to a variable at a deeper local level. Ksh
prints "global reference cannot refer to local variable". With zsh's
dynamic scoping you could do the same thing with a reference that isn't
"global" but which has a scope that will outlast the variable. Making
var outlive the function is a feature you'd get with a reference
counting approach to namerefs. The best might be if ref returns to being
unset when the function returns but an error like ksh is fine too.

> I'm still not seeing that.  The writer of the caller of _call_function
> should know that it's a general rule that another function can't
> access a private variable.  Why would the writer pass a private name
> as an argument to any function, even a blackbox, if not expecting it
> to be referenced?  The rule for a private should be that you always
> pass its value.

I hadn't really thought about it that way, perhaps because ksh only
has private scoping and I'm used to writing in languages that only
have lexical scoping. Certainly if it is hard to implement, I have no
objection to this approach. It does have the big advantage of making the
whole uplevel question a non-issue. We do lose some orthogonality in
that you can use a private with builtins that take variable names like
read, compadd and printf (-v). Wrappers of those would have an added
limitation. (_approximate relies on a function wrapper of compadd so
privates can't be used with that)

When relying only on dynamic scoping, it would be good practice to
define all the namerefs to passed parameters as early as possible in a
function to reduce the risk of a name clash.

> Correct me if I still misunderstand, but I interpret the example above
> as implying that
>
>   f2() {
>     typeset -n ref=$1
>     # ... do stuff to $ref ...
>   }
>   f1() {
>     private var
>     f2 var
>   }
>
> should work just because the name is passed in a positional?  I would
> disagree with that on the grounds that you're making positionals into
> a special case, just a different kind of special case from ksh, that
> violates the intent of "private".

It isn't about the positionals being special but that it is useful to be
able to write a function that exposes an interface similar to read where
a variable can be named as a parameter. Ksh's making $1, $2.. special
on the rhs of typeset -n really is very ugly.

> The case I was asking about (and the only one I so far consider
> possibly viable) is
>
>   f1() {
>     private var
>     typeset -n ref=var
>     f2 ref
>   }

This puts the complexity into the caller so f2's interface is less
nice. And by needing ref as a local, f1 hasn't really benefitted from
var being able to be private. But if this is easy to implement then it
could be useful. If not, printing an error is fine too.

> Or perhaps you mean to be tagging on to your other suggestion and it would be
>   f2 \&var

My intention with that suggestion is that you'd only do that to refer to
$var from the scope of f1's caller. So in practice that'd sooner be
something like \&$3. For this, it'd be just `f2 var` and f2() would
declare `private -n mine=\&1`

> which feels like syntactic sugar for my example above, with the
> benefit of not needing to declare an extra parameter.  That's an
> interesting idea but I would not want this to work:

Again, the key difference is that the caller needed the extra parameter
where as using some sort of uplevel goes in the callee. You can't wrap
e.g. compadd transparently if the caller needs to change.

> With a hash that's just:
>
>   typeset -n ref
>   for ref in 'hash[(e)'${(k)^hash[(R)?*]}']'; do ...

"just"!?

> > And it could be wise to limit what can be done as part of the
> > subscript evaluation to avoid a CVE similar to the last one.
>
> validate_refname() is calling parse_subscript() ... would further
> examination of the accepted string be sufficient, or something more
> needed?  I think the greatest danger is of creating an infinite
> recursion, which can't really be caught in advance.

So if a function gets the target of a nameref from untrusted input the
function writer needs to know to validate it with something like

  [[ $var = [[:IDENT:]]## ]]

It might deprive us of many clever tricks but parse_subscript() could
gain a flag to disable anything questionable like command substitution
and math evaluation side-effects.

This should be an error perhaps:

  typeset -n ref=arr[1][2]

Currently it isn't possible to create a reference fo $!, $?, $@, $+, $#
and $$. If easy to add, there would be no harm in them.

And in 51388, Bart wrote:
> How about
>   can't change type via subscript reference
> ??

That sounds fine.

Oliver