still confused about completion and matching

["E. Jay Berkenbilt" <ejb@xxxxxx>]

Thanks for the tips on _all_matches.  That part is now working
perfectly for me.  Since I now have ^Xa to insert all matches, I don't
want hitting TAB with a pattern to do that anymore.  This means that I
am free to go after what I really want. :-)

So I'm now back to trying to customize pattern matching behavior.  I
can't quite get it to do what I want.  I think I may be running into
to a combination of a bug and some misunderstanding, but probably
mostly just not knowing quite enough yet.

This is all with the current CVS version...

First, I'll explain what I want.  Then I can ramble for a while about
what I've tried and what I've found.

What I want is effectively for the completion system to wait as long
as possible before replacing a pattern with a match and to leave my
cursor where it is when I hit TAB.  It should only replace a match
when it can do so unambiguously, and, when it shows choices, it should
show as many directory components as required starting from the first
one where there is an ambiguity and ending with where the cursor is.
It should not complete past a /.  It should never fall back to menu
completion.

I'll try to say this more algorithmically.  I haven't coded this for
real, so there may still be a logic error lurking here.  I hope not.

First of all, we want to do this only if we are completing filenames.
This logic probably needs to go in _path_files itself perhaps
conditionally upon some compstate[pattern_match] setting or on some
style.

I know that _path_files has lots of code to deal with filtering the
list of matches based on various criteria such as looking at only
files with certain types or attributes or that match certain
patterns.  I am going to oversimplify what I'm looking for by ignoring
this very important functionality in hopes that doing so will make it
possible for me to communicate my goal.

What I'm looking for goes something like this:

  cur_word = the word currently being completed
  tmp1 = list of matches of ${cur_word}*
  tmp2 = longest prefix common to all matches
  if tmp2 contains a /
    # Some pattern characters can be unambiguously replaced
    tmp3 = everything up to and including the last / in tmp2
    replace "the appropriate part of cur_word" with tmp3
  else
    tmp3 = ""
  fi
  the list of choices is the portion of each word in tmp1 that
      follows the common tmp3 prefix (if any)

Then the completion system should still insert after the cursor
whatever can be inserted unambiguously.  (This is the part I can't
figure out how to do in the current system.)

Figuring out what is meant by the phrase "the appropriate part of
cur_word" is obviously very complex and I see that a great deal of
code in _path_files is devoted to this problem.  It gets especially
hairy if multiple components may be matched by any pattern.

Now I'll try clarify with examples.  I haven't written this into code
so there's a chance I may be making a mistake somewhere and not saying
what I mean but I've tried very hard to avoid that.

Suppose I have the directory structure created by the following
commands:

   rm -rf /tmp/z
   mkdir /tmp/z
   cd /tmp/z
   mkdir u{1,2,3,4}
   mkdir u{1,2,3}/q1
   mkdir u4/q
   mkdir u1/q1/e1
   mkdir u2/q1/e2
   mkdir u2/q1/e2/a{1,2}
   mkdir u4/q/a{1,2}

This is the following structure:

u1
 q1
  e1
u2
 q1
  e2
   a1
   a2
u3
 q1
u4
 q
  a1
  a2

For clarity, throughout this message, I am using _ to represent the
cursor position and TAB to mean that I am hitting TAB at that point.

If I now type

zsh% ls u?_TAB

I want

zsh% ls u?/_
u1/  u2/  u3/  u4/

The ? cannot be replaced unambiguously (tmp3 above is empty) so we
leave it alone.  However, all the matches to this pattern can be
followed by a /.

Then if I hit TAB again, I want to see

zsh% ls u?/q_
u1/q  u2/q  u3/q  u4/q/

because, again it is possible to complete through to the q for all
choices without replacing the ? since all possible choices start with
q.  Note that the u4/q/ case gets a trailing / because u4/q/ is
complete and is a directory.

If I type 

zsh% ls u?/q/_TAB

I want

zsh% ls u4/q/a_
a1/  a2/

since there is now only one way to replace the ?.  After this, we no
longer have a pattern so subsequent tabs should behave "normally".  (I
don't want the completion system to imagine that there is a * after
the q.  It seems to do that now, even with
compstate[pattern_match]='-', as I discuss below.)

Now suppose I type

zsh% ls u?/q1/_TAB

I should see 

zsh% ls u?/q1/_
u1/q1/  u2/q1/  u3/q1/

since these are the possible choices.  If I now say

zsh% ls u?/q1/e_TAB

I should see

zsh% ls u?/q1/e_
u1/q1/e1/  u2/q1/e2/

because these are the choices that match this pattern.  If I type

zsh% ls u?/q1/e?/a_TAB

I should see

zsh% ls u2/q1/e2/a_
a1/  a2/

It's okay if I have to hit TAB more than once to get this output.

If I type

zsh% ls u?/**/a_TAB

ideally, I like to see

zsh% ls u?/**/a_
u2/q1/e2/a1  u2/q1/e2/a2  u4/q/a1  u4/q/a2

if I have extended_glob set (which is actually the thing that first
made me switch to zsh).  Does this make sense?  Am I asking for too
much? :-)



Now, you can ALMOST get this behavior.  With the following definition
of _qcomp:

function _qcomp {
   local tmp opm="$compstate[pattern_match]" found=0

   tmp="${${:-$PREFIX$SUFFIX}#[~=]}"
   [[ "$tmp:q" = "$tmp" ]] && return 1

   compstate[pattern_match]='-'
   _complete && found=1
   compstate[pattern_match]="$opm"

   if (( found )); then
     compstate[insert]=
   fi

   return 1-found
}

and the following completion style setting:

zstyle ':completion:*' completer _complete _qcomp _ignored

If you look at the traces, _path_files does actually have the complete
list of matches.  The choices you see are all the first components of
the choices I want to see.  If you replace 

compstate[insert]=

with

compstate[insert]=unambiguous

above then you get another behavior that is almost right.  It breaks
for the ** case though.  Also, it falls back to menu completion for
some reason that I don't understand.  With this, there is some other
behavior I get which I think is a feature but I don't really like it.
I think it has to do with compstate[pattern_match].  Given the above,
if I type

zsh% ls u?/q/_TAB

I get menu completion with the choices /u1, /u2, and /u4 and with my
command line cycling through u1/q/, u2/q/, and /u4/q/, each time I hit
TAB.  If when it says u2/q/ I hit space, backspace, TAB, then u2/q/
becomes u2/q1/e2/.  It is as if the completion system is imagining a *
after the q and before the /.  I thought this would happen only if I
had compstate[pattern_match]='*', not ='-'.  Am I confused?


As I was researching all this, I was looking at _match to see whether
I could get that to do what I wanted.  I am very confused about line
47:

   $#compstate[unambiguous] -ge ${#:-${PREFIX}${SUFFIX}} ]] && 

I can't figure out its purpose.  It seems to always be false.  This
line prevented me from getting compstate[pattern_insert]=unambiguous
even if I had 

zstyle ':completion:*' insert-unambiguous true

which is why I ultimately went to _qcomp again.


The reason that I would like this is based on the way I tend to
arrange my directories.  On my machine, I have various partitions
which I call /u1, /u2, etc.  On each partition, I create a directory
called q which I own and which I use as scratch space.  I don't always
remember which directory something is in.  I have a directory called
/u1/q/devel where I check out stuff from the CVS repository at the
office.  I have /u1/q/zsh where I currently have zsh checked out.
However, on some other system, maybe devel is in /u2/q.  I would like
to be able to say

zsh% ls /u?/q/d_TAB 

and get

zsh% ls /u1/q/devel/

On the other hand, sometimes I have a /u1/q/devel and a /u2/q/devel
with different side branches.  Maybe job 900 is in one and job 904 is
in another.  In that case, I should be able to type

zsh% ls /u?/q/d_TAB

and get 

zsh% ls /u?/q/devel/
u1/q/devel/  u2/q/devel/

and then I should hit TAB again and get

zsh% ls /u?/q/devel/
u1/q/devel/900  u2/q/devel/904

I should then be able to type 904 so that i have

zsh% ls /u?/q/devel/904

and get the shell to replace the ? with a 2 since it can now do so
unambiguously.



I hope this makes sense.  Would you believe that it took me four hours
to compose this mail message?  I've gone through about 10 different
ways of presenting this with some earlier versions referencing trace
output from ^x? with completion functions set various ways,
referencing different parts of _path_files, etc....  I hope that the
message hasn't gotten completely lost.  This would be so much easier
to communicate interactively.

The main thing that would currently stop me from being able to
implement this in a reasonable amount of time is that I still don't
really fully understand all the internals of completion widgets.  I
don't know all the options to compadd, some of the compstate keys,
compquote, etc., and I still don't fully understand how to use styles.
It seems though that it shouldn't be that hard to get _path_files to
do this at least conditionally.  What do you think?

My brain hurts and I have to get back to my real work.... :-/

Hopefully this will start some interesting discussion. :-)

                                Jay