Re: Autoloading of compctl from dbm database file.

[Bart Schaefer <schaefer@xxxxxxx>]

On Dec 1,  8:55pm, Fung-Chai Lim wrote:
> Subject: Re: Autoloading of compctl from dbm database file.
> In <961130151728.ZM16366828@xxxxxxxxxxxxxx>
> on Sat, 30 Nov 1996 15:16:26 -0800 (PST)
> Bart Schaefer <schaefer@xxxxxxx> writes:
> 
> > Compctls aren't very expensive to store either, are they?  The slow
> > part of loading compctls and functions is parsing.  If we're going
> > to introduce a database, we might as well also introduce a binary
> > storage format and put pre-"compiled" functions, compctls, etc. in
> > there.  Using a database just to store text that zsh then has to
parse
> > and execute anyway doesn't seem very helpful.
> 
> Then you have to worry about the "binary" database being shared by
> systems of different architectures; distributed file systems are
> quite ubiquitous.  The database must be portable over networks.

Yes, but that's not a very difficult problem.

> > Further -- I admit I haven't tried the patch -- doesn't attempting
to
> > autoload compctls, even from a database, slow down default
completion
> > a lot?
> 
> I do not notice any degradation, even with "big" compctl like the
> one for `find'.  Remember, only one compctl is parsed at a time
> with autoloading, unlike in startup files.

I'm not worried about how long it takes to load any given compctl.  I'm
worried about how long it takes, every time a command that DOES NOT HAVE
A SPECIAL COMPCTL is used, to discover that there isn't an entry for it.
Completion for a command that doesn't use the autoload database should
not be forced to wait for it.  That's one nice thing about autoloaded
functions -- you have to tell zsh in advance that the function will be
autoloaded, so there's no execute-time overhead for external commands.

Compctls are seldom large/complex enough to make it worthwhile to first
tell zsh about them and then load them later; I'm sure this is why you
chose to use a database for them.  However, I'm not convinced that the
"faster, leaner" startup is worth trading off against searching the
database every time I press tab.

Given a binary database, you could put an index in the first record so
zsh could quickly discover what commands needed to be looked up later;
but with a text format that'd be little better than parsing .zshenv.

On Dec 1,  2:02pm, Zefram wrote:
> Subject: Re: Autoloading of compctl from dbm database file.
> 
> How about an interface like
> 
> % compctl -L exec
> compctl -l '' exec
> % bcompctl -L exec
> bcompctl '35!^Az^@^@^@' exec
> % bcompctl '35!^Az^@^@^@' command
> % compctl -L command
> compctl -l '' command
> 
> That is, bcompctl does the same job as compctl, but using a byte-
> compiled form instead of the normal options.  "bcompctl -L > file"
> could be used to save byte-compiled compctls in a form that can be
> sourced quickly later.

I don't think that goes far enough.  If you still have to parse and
tokenize a command line for every compctl, the binary format hasn't
won you very much.

I'd expect bcompctl to take a file name to read and bcompctl -L a
file name to write, after the manner of fc -R/-W.

Further, {b}compctl -L would have to get a lot smarter, and merge
together into one output item the whole class of commands that use the
same compctl.  E.g. stop doing this:

% compctl -l '' exec noglob nocorrect
% compctl -L
compctl -l '' exec
compctl -l '' nocorrect
compctl -l '' noglob

> The challenge is to come up with a binary form
> of compctls that is architecture- and version-independent, and can be
> turned into the internal form rapidly without risking a crash.

Version-independent will be the tough part; we'd better be darn sure
we're storing compctls internally in the form we'll always want to store
them, before we start saving them off as binary.

A text format has some of the same problems -- I for one would love to
see compctl move away from all these cryptic command-line switches and
instead become a more structured construct with keywords etc.

> This is getting pretty far from the original idea of a database.

Well, a database of some sort (if only a simple one) would be a good
input/output format for bcompctl.  Further, one could treat loading of
compctls the way zsh currently treats filling in the command hash tables
(and even make it subject to the HASH options) -- that is, don't load
the database immediately, and then load only as much as necessary each
time a completion is requested.