Re: Help Request/Bug Report: comparguments causes _arguments to fail in certain cases

[Daniel Shahaf <d.s@xxxxxxxxxxxxxxxxxx>]

> > > Well, for starters, if you «shift words», you should decrement CURRENT  
> > > as well.
> > >
> > > However, _python uses the «*::…» form of an _arguments optspec,
> > > which should take care of $words/$NUMERIC for you.  I assume the reason
> > > it didn't is that you used «compdef -p».  Try -P instead?  If I'm not
> > > mistaken, that would also handle «python … script.py <TAB>» for you
> > > (where the ellipsis stands for python's --options).  
> >
> >
> > I tested the 4 possible options, and found that switching -p with -P
> > doesn't change anything, but shifting words and decrementing CURRENT  
> > works!  But... I feel like it's a hack I'm not supposed to do
> > (correct me if I'm wrong).

Those parameters are documented, and manipulated by a number of
functions (including _python, as it happens), so you _are_ using the API.
See also the 'compset' builtin.

However, as mentioned, I suspect that hardcoding a skip/decrement of one
element won't DTRT if python is invoked with options (e.g., «python
--foo script.py <TAB>») — not unless you reimplement _python's parsing
of those options and any arguments to them.

> > I now tried using the BUFFER variable instead of words (since it remains
> > unchanged), but found that it also has caveats: The BUFFER variable
> > contains the raw value of the line, whereas the words array contains the
> > words after expansions (I think that's the right term).

Partly.  $words has undergone alias expansion, but not further
expansions.  In particular, the words are still quoted.

> > To demonstrate this I tried running again completion for `python ps,py`
> > with `#compdef -P *`, and printing out BUFFER and words. The debug log  
> > (and my terminal) showed the following:
> > First of all _python was called and dispatched my script which printed
> > `BUFFER=python ps.py` and `words=ps.py`.

That's what I'd expect.

> > After _python returned, my script was invoked again and printed
> > `BUFFER=python ps.py` and `words=python*3* ps.py` (note the 3).

I'm not sure offhand where this comes from.  $context, $funcstack, et al
may have clues.

> > I do think the right solution is to be dispatched by _python with  
> > `#compdef -P *.py`, but to be able to run I need the words array as
> > seen by _python

Once an element is shifted off $words, it's lost forever (free() is
called on it), so there'll be no way to access a "previous" value of $words
unless a copy had been made somewhere.

Moreover, consider that python may be itself wrapped by some precommand:
for example, «env X=Y python», «sudo -u foo python», or «ssh foo python».

You might try looking up the place that originally populates $words and
making it stash a copy of $words in another, read-only array; then you'll
be able to use «compset -P» and access that array.  (Compare vcs_info's
use of *_orig keys in ${hook_com}.)

> > or a to understand how BUFFER can be expanded/converted to its correct  
> > form.  If you have any input on this, I would very much appreciate
> > it.

Something along these lines, I think:

    local -a words2=( ${(z):-"$PREBUFFER${LBUFFER}x"} )
    words2[-1]=${words2[-1]%x}
    words2[1,${words2[(I)(;|&&|…)]}]=()

To understand the second line, see addx() in Src/Zle/zle_tricky.c.  With
that out of the way, the first line's a textbook use of the ${(z)}
tokenizer.  The third line just throws out everything until the start of
the current _simple command_ (see zshmisc(1)) — or, at least, it will
once the ellipsis is filled out.  

Caveats: that snippet doesn't do alias expansion, doesn't try to deal
with ${RBUFFER}, doesn't try to deal with SHORT_LOOPS, doesn't try to
deal with precommands and precommand assignments, should be changed to
use zZ+c+ if the INTERACTIVE_COMMENTS option is set…

Although it's a plugin that I worked on myself, I should probably
mention that zsh-syntax-highlighting has faced the same problem and you
and implemented a solution to it, which handles a number of the above
caveats.

>

There's also a third approach: instead of trying to run the binary, make
it stash the output in a well-known location.  For example, /usr/bin/foo
could store the output in /usr/share/argcomplete/foo.  (I've always
wanted to standardize _some_ solution to this problem; I'm tired of
writing completion functions by hand…)

Cheers,

Daniel