Re: still confused about completion and matching

["E. Jay Berkenbilt" <ejb@xxxxxx>]

With this latest patch, the behavior is very close to what I was
looking for.  It's close enough in fact that even if none of the
problems I mention below are fixed, I would still find the existing
functionality from the latest patch to be quite useful.  I really
appreciate all your work on this.

Here are a few cases that don't work quite as I would expect.  None of
these cases are particularly important, but you may be interested in
them, and some of them may be trivial to fix.

Here's a copy of my testing startup environment with a few new mkdir
commands added:

   PS1='zsh%% '
   setopt noautomenu
   autoload -U compinit
   compinit
   bindkey "^I" complete-word
   rm -rf /tmp/z
   mkdir /tmp/z
   cd /tmp/z
   mkdir u{1,2,3,4,5}
   mkdir u{1,2,3,4}/q
   mkdir u5/q1
   mkdir u1/q/e1
   mkdir u1/q/devel
   mkdir u1/q/e1/b{1,2}
   mkdir u2/q/e2
   mkdir u2/q/e2/a{1,2}
   mkdir u3/q/dark
   zstyle ':completion:*' completer _complete _match
   zstyle ':completion:*:match:*' insert-unambiguous pattern
   zstyle ':completion:*:paths' list-suffixes yes
   zstyle ':completion:*:paths' expand prefix suffix

If I omit "suffix" from :completion:*:paths expand, although 

ls u?/ TAB

doesn't generate a superfluous / anymore,

ls u?/q/ TAB

still does.

ls u?/q/d TAB

lists u1/q/devel and u3/q/dark as it should.  However,

ls u?/q/de TAB

doesn't list anything.  Similarly,

ls u?/q/e TAB

lists u1/q/e1 and u2/q/e2, but

ls u?/q/e2 TAB

lists nothing.  However,

ls u?/q/e2/ TAB

actually expands to u2/q/e2/a as would be ideal.  If you

rmdir u2/q/e2/a2

though then

ls u?/q/e2/ TAB

no longer lists anything.  It seems as though there are some cases in
which there being only a single possible completion causes nothing to
be listed at all whereas the case of no ambiguity on what has been
typed but multiple possible completions works fine.  Both the
devel/dark case and the removal of a2 breaking things illustrate this.

Finally,

ls u? TAB

works fine, but

ls ./u? TAB

makes the ? disappear from the commandline.

As always, should you decide to attempt to fix the above issues, I
will happily test the patches.

I will use zsh for my normal activities with those patches applied
just to make sure nothing else breaks.  I guess it would be good for
people who don't want this new style options to also work with these
patches to make sure that they don't break anything when not turned
on.

>  The problem really is that the completion can only: 1) leave the
>  command line unchanged, 2) insert a match (menu completion like or if
>  the match is unambiguous) or 3) insert the unambiguous part of a
>  match. That last one can't work if the matches were generated using
>  pattern matching -- how should it detect which wildcards should be
>  left unchanged and which have been used to match something
>  unambiguously. And note that the stuff I'm talking about is in the
>  C-code, at the very heart of the completion code.

Thanks for this explanation.  My thoughts on the "how should it detect
which wildcards should be left unchanged...." would be to use an
approach similar to backreferencing in regular expression matches.  If
a given wildcard character (or group of contiguous wildcard
characters) matched the same string in all expansions of the overall
expression, that single group of contiguous wildcard characters could
be replaced.  The completion system could then retry if anything it
did changed the commandline.  I may be totally off the mark here since
my understanding of these internals is minimal, and I'm not really
suggesting implementing anything like that.  For me this has passed
into the realm of academic discussion on what could possibly be done.
The existing patches are really quite enough for me to get what I want
in just about every case I ever really run into.  For example, if I
want to check in all changed files except for Makefile, I can now type

cvs ci -m"my comment here" *~M* TAB

and if the list looks right I can invoke insert-all-matches.  If not,
I can edit the expression I've typed and try again.  Also, if I forget
which of my /u?/q directories something is in, I can use /u?/q/ TAB to
look at all of them at once.  If the completion system ends up being
able to help me further great; if not, I can back over to the ? (with
C-r ?), change it, and continue.  This all seems to work fine both for
my contrived examples in /tmp/z and for my real directory structure,
and is really 99% of what I want in the first place.

I really appreciate all your help in trying to accommodate my esoteric
requests.  I've found this all quite educational as well.

Now to make a gratuitous personal observation.... zsh is about the
only open source software I've used recently where it hasn't been just
as easy for me to send a patch implementing my requested changes as it
is to describe them in detail.  I'm not sure exactly why this is, but
I think it's just because there is so much power that can be expressed
so compactly that the density of code that I don't understand due to
lack of zsh knowledge is increased.  Perhaps also the software has so
much potential that I'm thinking of changes that just wouldn't occur
to me if the software couldn't already almost handle them.  I have a
rule against modifying code I don't understand (at least locally).
Reading the code in the zsh completion system, I find that there is
something mysterious in just about every line though the documentation
seems remarkably complete.  Should I decide to spend the time required
to really understand what's going on in code like that in _path_files,
looking back at these discussions and patches will no doubt be an
invaluable tool.  In the mean time, I plan to start with something a
bit more modest.  The single remaining command I used to have a
programmable completion for in tcsh that I don't in zsh is smbclient.
I plan to eventually write a completion function for smbclient and
submit it.  This should be much simpler than this _path_files stuff,
especially if I read Etc/completion-style-guide.

Thanks again for all your help and for your work on zsh in general.  I
am continually impressed with how much I can do so easily now since
I've switched to zsh!

                                Jay