Re: Let's finish this new completion stuff

[Peter Stephenson <pws@xxxxxxxxxxxxxxxxx>]

Sven Wischnowsky wrote:
> - Using the positional parameters as we do now has some drawbacks. 

I haven't experienced any myself yet --- though I realise I'm danger of
forgetting to tack "$@" on the end of a subfunction --- but I've no
objection to using $words or whatever. (Anything beginning with `arg' is a
bit suspect because it's too like $argv which is exactly what it wouldn't
be; $words isn't very specific, but it's pretty memorable and I get to
quote Hamlet:

Polonius:  What do you read, my Lord?
Hamlet: $words, $words, $words.)

I would agree it's more logical to keep them separate from the function
call scope.

>   We
>   could make the completion widget store them in another array but
>   then the modifying tests (see below) would still change `argv'
>   unless we make the test get the name of the array to work upon as an 
>   argument.

Why is this a problem if the array name for the command-line arguments is
hard-wired?  And why would you not want to hard-wire it?  Do we finally
need to tackle the thorny question of localising special parameters?
(There's already similar stuff to handle `special=tempvalue builtin', it
seems a shame not to show it off.)

>   But then we could equally well let users use `read' to get 
>   the words (which I wouldn't like but that may be only my problem).

I don't like this either.  I'd prefer to keep the interface simple and use
parameters wherever possible.

>   Of course we could also make the completion code store the words in
>   an array and give them as positional parameters. The we would have
>   the problem mentioned by Bart that the value of CURRENT may be off
>   by one for those who use `ksharrays', unless we use the setting of
>   that option when setting CURRENT in the completion code.

I'd be happier to do one or the other, rather than this compromise.

> - I would like to remove the parameter `COMMAND' since this has
>   already become yet another way to describe the context (`COMMAND' is 
>   a bit of a misnomer currently). When using an array to report the
>   words on the line we could store it in this array as the first word
>   which should make non-automatic modification of this array
>   easier. With that we could get rid of the `command' or `argument'
>   CONTEXT, replacing both of them with one name.

In other words, you test [[ $CURRENT -eq 1 ]] and use $words[1] ?
That seems pretty logical.  But I would still want _normal able to handle
this separately, i.e. have something special it does when [[ $CURRENT -eq 1
]] just as it does now for [[ $CONTEXT == command ]].  By the way, is it
really necessary to have the `cmd=$(whence -p $COMMAND)' on every call to
command/argument completion?  I haven't thought about it, but maybe it can
be optimised away in a lot of cases.

> - We could also put the `COMMAND' into the `CONTEXT' parameter which I 
>   would like to replace with a associative array with keys:
>   - `command'   if this is used as the replacement for `COMMAND'.
>   - `redirect'  for the string of the redirection operator
>   - `parameter' for the `value' and `subscript' contexts where it will 
>                 contain the name of the parameter
>   - `type'      as the replacement for the old `CONTEXT'
>   - `quote'     if we want to do this, this could be set to `single',
>                 `double', or the empty string giving information about 
> 		the quotes the completion code thinks we are in
>   We may find other interesting things here and with an associative
>   array it would be easy to add new types of context information.

This sounds OK, but if $COMMAND is really just going to be $words[1] or
whatever it might be a bit confusing to put it here too --- for example, if
you are completing after `nice' so that $words gets narrowed to whatever's
after, does $context[command] refer to `nice' or to the next word, and when
you change the context (shift words; (( CURRENT-- )) ) is the user or the
shell responsible for changing it?  I think we should just stick to using
$words or whatever name anyone else suggests, if that's agreed upon.

> - The example code currently uses the return value of the shell
>   functions or a parameter (in the new version I just sent) to decide
>   whether more completion definitions should be used. This can not be
>   combined with `compcall' which currently gives no information about
>   things like that. I'm not too sure about this, but maybe we would
>   want to add a special parameter that will be used by `compcall' and
>   that can also (of course) be used in the shell code. (Someone else
>   should decide this since I don't care that much about `compctl' any
>   more.)

I'd have thought for anyone still using compctl it would be good enough
just to be able to test $NMATCHES.  If anybody wants anything more
sophisticated, they're going to be going over to new completion anyway.

> Some of the condition codes we currently have could easily be replaced
> by shell code so we should probably remove them. The question was/is
> with what we would replace them. There seems to be support for doing
> this with a builtin and indeed some of the examples have shown that we 
> sometimes only need the modification, not the result of the test.

I think more and more that `comptest' (or options of a more general command
e.g.  `compset') would be OK and that if you want to use [[ ... ]] the
parameters will be enough.  So [[ $[I]PREFIX = ... ]] wouldn't do any
modification, of course, but compset -iprefix ... would.  It is quite nice
being able to have complete lines like
  elif [[ -iprefix '+' || -iprefix '@' || -current -1 -draftfolder ]]; then
but, as Bart said, having tests with side-effects is just too much.

>   Here we could also add a built-in replacement for the `compsave' and 
>   `compreset' aliases from the example code. Let's give it two options 
>   to save and restore the state of all the special parameters. It
>   would be the user's responsibility to make those calls symmetrical.

I'm still vaguely of the opinion it would be quite nice to have long
Posix-style options for the comp* commands, as I suggested some time ago,
e.g. 

complist -mf

could also be

complist --external --files

or something for readability.  I mention it here since `compset --push' and
`compset --pop' are the obvious options in this case.

> - Options to get information from the calling completion code,
>   e.g. things like `is there already a valid list', `will menu-
>   completion be used' (which would also test for automenu) and the
>   like.

This goes nicely with associative arrays --- it's hard to get information
from builtins without forking and maybe this is a natural part of the
$context.  What I was murmuring about the other day was something like `if
there is a valid list, use it instead of generating a new one'; the latter
bit would presumably require some compset (I just invented that name but it
seems as good anything?) trickery --- or should it automatically use an
existing list of no new completions are generated?  (The next point, if
implemented, would make this pretty much irrelevant.)

> - Options to say what should be done with the matches genereated,
>   e.g.: `list them', `don't list them', `use menu-completion', `insert 
>   it into the line', etc.

In other words, internal control of what the widget is really doing: `I'm
fed up with being an expand-or-complete, I want to be a list-choices'.
Maybe this needs to integrate somehow with `zle ...' calls, but I can't
offhand think how.  (My first thought to do what I mentioned above was,
test if we just had a _complete-* as $LASTWIDGET and if so run `zle
list-choices' and return, but that doesn't work.)  Or maybe the ability to
set context[widget]=list-choices is good enough (hmm, altering your own
context is getting a bit post-modern, maybe the assoc array needs to be
called something else).

> making `compadd' do matching unless the `-U' option is used and
> remove the `-m' option. What keeps me from doing this already is that
> `-U' switches on menu-completion which is not what one wants if the
> function using `compadd' did the matching.

That seems sensible.

> - As Bart already said, the name `complist' is a misnomer (I was
>   thinking about `it adds a list of matches...'). Any suggestion for a 
>   better name from someone who knows more English synonyms or has more 
>   ideas than I would be welcome (we could use `compctl' for it, making 
>   it behave differently when called from a completion widget...).

compmatch? compflags? compaddflags [i.e. it's like compadd, but instead of
the matches themselves you specify flags for them]? compgen? complete
[don't ask what lete means, there's a clash with tcsh there, of course]?
compmake? compsel[ect]?

> - Since `compctl' is already used I'd also like to hear suggestions
>   for the control-builtin I proposed.

I suggested `compset' above, since it sets things; other possiblities are
things like `compopt[s]', but it does more than just change options.

> - I already spoke about the `problem' with completion after `~', `=',
>   and `$' when `complist' is used. Currently the code automatically
>   uses the builtin way to complete in such places. I'd like to change
>   this by making `complist' behave as if it didn't know better and
>   adding contexts reported to the user via the `context' parameter for 
>   these circumstances.

OK

>   The question is: should I add contexts for `~'
>   and `=' - almost doesn't look like it's worth it.

I presume it's easy enough to tell whether you're completing after a ~
anyway -- just look to see if there's a slash yet.  The advantage of
something like a context would be you can finally solve the problem compctl
-T was invented to solve in a modular way, i.e. making your own list of
usernames.  With a context you can just plug in your own comps[-tilde-] by
the autoload-scanning mechanism.  Thinks: this doesn't have to be a context
generated by the C code, there's nothing to stop _normal looking up
$comps[-tilde-] by itself if it thinks that's a good idea, [and maybe
completing users if there isn't one (it can even check $COMPSKIP) --- but
perhaps the best idea would be to provide a _tilde handler for -tilde-
which the user then modifies directly].

> - Bart suggested making the leading underscore special in the
>   `functions' builtin and `compctl -K ...'. I like this, but I'd like
>   to hear what others think about it before...

Sounds OK to me, but are we morally obliged to have functions generate all
functions including _* in ksh compatibility mode?  Seems a bit too trivial
to have its own option.  [[ -o posixbuiltins ]] && ...?  There's no such
problem with compctl -K, unless people are using a _ convention for that.

Another issue: ordering of pattern completions.  I think this can already
be done quite naturally.  `init' (should we turn the files into compinit
and compdump btw?) stores the patterns in $patcomps in the order it finds
them in the directory, which is globbing order, which is alphabetic, and
`dump' preserves this order, and _normal goes through them in the same
order.  So the user can have e.g.
 _patcomp01_X_files              # the completion is out there...
 _patcomp02_zftp
if they need ordering (looks a bit like /etc/rc.d, in fact).  It's harder
if they live in more than one directory, but at least they come up in the
order they do in $fpath, which seems entirely natural; e.g. if
fpath=(~/myfunc /usr/local/lib/zsh/functions), then first you get all your
own, then all the standard ones, if any.  We could point this out in the
rubric for #defpatcomp and I think that's the end of the matter.

-- 
Peter Stephenson <pws@xxxxxxxxxxxxxxxxx>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy