Re: [PATCH 1/3]: Add named references

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Wed, Feb 8, 2023 at 4:47 PM Oliver Kiddle <opk@xxxxxxx> wrote:
>
> Bart Schaefer wrote:
> > > One ksh feature which this doesn't cover is the special behaviour in ksh
> > > that `typeset -n ref=$1` will use the calling function scope to resolve
> > > the target.
> >
>
> In ksh you could pass "ref" as a parameter and the nameref would provide
> access to the outside variable despite the name clash.

I think that will start working if I can fix the "invalid self
reference" mentioned in your next message.

> So [in ksh] $1 as the target of a nameref is special. [...] I don't
> think we need to worry about the level of ksh93 compatibility here
> because our default dynamic scoping mean a ksh script will work if
> variable names don't clash.

My general feeling is to be ksh compatible where I don't think ksh is stupid.

> Note that Tcl's uplevel is like a precommand modifier in zsh terms so
> is not limited to defining references.
>
> However I still prefer the other solution I mentioned with \&.

I have some half-formed thoughts about other ways to accomplish this
... I don't think the \& suggestion by itself gets around the problem
that the positionals are C-local to doshfunc().

> > I'm still not entirely following what problem this solves.
>
> It solves the problem of name clashes between a function and the calling
> function.

OK, so if
  typeset ref
  () { typeset -n ref=ref }
works / is not a loop, then we don't need any other magic?

> If two functions are developed independently, they shouldn't need
> to worry about the possibility of using the same variable name for
> different purposes. This is the main benefit of private in general.
> An old function like _call_function (updated to use nameref but still
> using local) could be used by someone writing a new function where they
> pass a private variable as the first parameter. If they are treating
> _call_function as a black box, how are they to know that private
> variables are not valid for the passed $1. It'd be an arbitrary
> limitation.

I'm still not seeing that.  The writer of the caller of _call_function
should know that it's a general rule that another function can't
access a private variable.  Why would the writer pass a private name
as an argument to any function, even a blackbox, if not expecting it
to be referenced?  The rule for a private should be that you always
pass its value.

Correct me if I still misunderstand, but I interpret the example above
as implying that

  f2() {
    typeset -n ref=$1
    # ... do stuff to $ref ...
  }
  f1() {
    private var
    f2 var
  }

should work just because the name is passed in a positional?  I would
disagree with that on the grounds that you're making positionals into
a special case, just a different kind of special case from ksh, that
violates the intent of "private".

The case I was asking about (and the only one I so far consider
possibly viable) is

  f1() {
    private var
    typeset -n ref=var
    f2 ref
  }

That is, the reference has to be declared in the scope where var is
"visible", otherwise you hit the privacy wall when "typeset -n" has to
up-level.

Or perhaps you mean to be tagging on to your other suggestion and it would be
  f2 \&var
which feels like syntactic sugar for my example above, with the
benefit of not needing to declare an extra parameter.  That's an
interesting idea but I would not want this to work:

  f2() {
    typeset -n upref=\&var
    # ... do things to var one level up ...
    typeset -n uptwo=\&\&var
    # ... do things to var two levels up ...
  }

I fear that puts us back to having something magic about positional
parameters, e.g., that unless the "&var" string appears in one of $1,
$2, ... then it's just an invalid name like any other.

Back again to my half-formed thoughts, I can't really elucidate yet.

> Given that you implemented references to array members, being able to
> iterate over array elements with for could be nice. Perhaps something
> like: for ref in 'arr[*]'; do

With a hash that's just:

  typeset -n ref
  for ref in 'hash[(e)'${(k)^hash[(R)?*]}']'; do ...

but since you have to "typeset -n ref" before the for-loop anyway, why not just

  typeset -n ref=hash
  for idx in ${(k)ref}; do ... $ref[$idx] ...

Ordinary arrays are harder because there's no "find all matching
elements" subscript flag.

> On the subject references to array elements, it does seem both powerful and
> dangerous that subscripts are evaluated on reference.

Hm ... I implemented that because ksh93 accepted the syntax, but I
never actually tried it -- in fact (Ubuntu's) ksh93 doesn't actually
expand it to anything sensible and the value of the nameref ends up
nonsense.

> The subscript
> should perhaps be evaluated with the same local level as the array
> itself.

That gets pretty weird, and you can already trivially bypass it with
${(P)...} for many cases.

> And it could be wise to limit what can be done as part of the
> subscript evaluation to avoid a CVE similar to the last one.

validate_refname() is calling parse_subscript() ... would further
examination of the accepted string be sufficient, or something more
needed?  I think the greatest danger is of creating an infinite
recursion, which can't really be caught in advance.