Re: completion tricks

[Sven Wischnowsky <wischnow@xxxxxxxxxxxxxxxxxxxxxxx>]

Adam Spiers wrote:

> Sven Wischnowsky (wischnow@xxxxxxxxxxxxxxxxxxxxxxx) wrote:
> > On a stand-alone Linux box there probably really isn't that much
> > difference, but at least for me compiling the whole completion system
> > into a memory mapped digest file and using that makes things faster
> > (after the first function is loaded).  It also saves memory, of course 
> > because mapped files are shared by all instances of zsh running on the 
> > same host (if your OS supports that, but almost every Unix should).
> 
> Would you mind showing how you do this quickly?

I use this:

    zrec() {
      zrecompile "$@" -p -- \
        -R ~/.zshrc -- \
        -R ~/.zdump -- \
        -R ~/lib/zsh/comp/Core/compinit -- \
        -R ~/lib/zsh/comp/Core/compaudit -- \
        -R ~/lib/zsh/comp/Core/compdump -- \
        -M ~/lib/zsh/funcs.zwc \
           ~/funcs/*(.) \
           ~/lib/zsh/comp/*/_* \
           ~/lib/zsh/func/*/*
    }

Never mind the funny paths.  Since I just upgraded the zsh I use every 
day (above the development version) to 4.0.1-pre2 I can say that this
worked for me with the latest CVS, so I'm not sure why you get:

> ...
> 
> zsh: parse error near `|'
> zcompile: can't read file: /nfs-home/adams/local/share/zsh/4.0.1-pre-2/functions/Completion/Builtins/_vars_eq

There are only `||'s in _vars_eq, no `|'s.  Do you have any aliases
that might get in the way?  You'll have to use `zcompile -U ...', then.

> Also, for this to work, wouldn't I have to include ~/.zsh/comp.zwc in
> fpath? 

Right.

> Why did the example suggest only to compile Completion/*/_*,
> not Completion/*/* ?

So that one saves the (probably not too large) space for the comp*
functions which get called only seldom and unfunction themselves.  If
they are in a mapped zwc file they might still be kept in memory even
though they will never be called again through the lifetime of the shell.

> I get the impression there are several different strategies one could
> take with compilation.  For example, how much is there to be gained by
> replacing all the Completion/* directories from the fpath with a
> single compiled foo.zwc digest?  Could you even do that, given that
> presumably foo.zwc would be missing Completion/Core/comp* ?

Of course different strategies might be useful: everything into one
zwc (probably the fastest -- I even put my personal utility functions
in my master zwc file), one zwc digest per directory... Using one zwc
file per function would be a bad idea, though.  Most of them will not
be mapped unless forced with the -M option but then one would have so
many mapped files open that the kernel might get unhappy.

Using the zwc digest for only the completion functions (sans comp*)
works, of course (see above), but you'll have to put at least the Core 
directory in your $fpath.  Put it after the zwc file and the shell will 
starting searching that, find the function it's searching and not look 
at the directories, so it's quite fast (after the first function found 
in the digest file the `directory' from the zwc is mapped in memory
and searching it...).

> The more I think about it, the more I like the idea of `make install'
> (or maybe `make install.zwc') taking the most sensible compilation
> route.  The distribution's completion functions probably aren't going
> to change at all in between upgrades, after all.

I wouldn't be against that, I just wouldn't know what style to suggest:
one for all, one for each directory, and there are also the functions 
in the Functions directory... Although users still have to call
autoload with the function names they want so it isn't that big a
problem if the global zwc digest contains functions some users don't
want to use.  Hm.

> > > Completing files with TAB was noticeably slower with the new
> > > system over the old.
> > 
> > As Adam pointed out, this of course depends on the completers one uses 
> > (and the match specs, btw.).
> 
> Perhaps these performance hits should be mentioned in the manual?

Well, that putting many function into the completer style makes things 
slower shouldn't be that surprising.  And for the match specs I'm not
completely sure what to suggest (even thought I asked Peter to put
something about that into the Guide, which he did).  Specs with `*'
are slower than others, specs with `**' even more so.  But depending
on the number, length and complexity of the matches one probably
wouldn't notice that on modern computers.  Using many matchers with
the matcher-list style can make things slow, too, but I always think
that using to many different matchers isn't that good idea anyway
(because they will probably match too often) so that users will find
that out if they want or not anyway.

But of course I'm not against putting a little warning here and there
into the manual.  Maybe we could first collect some reports from
everyone here about things which they considered to be too slow (and
how they solved it).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@xxxxxxxxxxxxxxxxxxxxxxx