Re: Autoloading of compctl from dbm database file.

[Fung-Chai Lim <fclim@xxxxxxxxxxxxxx>]

In <961201094226.ZM20801243@xxxxxxxxxxxxxx>
on Sun, 1 Dec 1996 09:41:24 -0800 (PST)
Bart Schaefer <schaefer@xxxxxxx> writes:

bart> Further -- I admit I haven't tried the patch -- doesn't attempting to
bart> autoload compctls, even from a database, slow down default completion
bart> a lot?

fclim> I do not notice any degradation, even with "big" compctl like the
fclim> one for `find'.  Remember, only one compctl is parsed at a time
fclim> with autoloading, unlike in startup files.

bart> I'm not worried about how long it takes to load any given compctl.  I'm
bart> worried about how long it takes, every time a command that DOES NOT HAVE
bart> A SPECIAL COMPCTL is used, to discover that there isn't an entry for it.
bart> Completion for a command that doesn't use the autoload database should
bart> not be forced to wait for it.  That's one nice thing about autoloaded
bart> functions -- you have to tell zsh in advance that the function will be
bart> autoloaded, so there's no execute-time overhead for external commands.

`zdb -u' is equivalent to `autoload'.  I use the hitherto unused `flags'
member of the compctlp structure.  All autoloaded compctl have this flag
set to PM_UNDEFINED.  The zle checks the flag to see if it needs to
consult the database.  The flag is turned off once the compctl is fetched
from the database.

Any command that does not have a compctl is not in compctltab and so there
will be no check; the zle will immediately use the default for that command.

The extra cycles go into parsing the compctl.  So, if a binary database
is used, the extra reduce to only a database fetch.

> Compctls are seldom large/complex enough to make it worthwhile to first
> tell zsh about them and then load them later; I'm sure this is why you
> chose to use a database for them.  However, I'm not convinced that the
> "faster, leaner" startup is worth trading off against searching the
> database every time I press tab.
>
> Given a binary database, you could put an index in the first record so
> zsh could quickly discover what commands needed to be looked up later;
> but with a text format that'd be little better than parsing .zshenv.

The full syntax for `zdb -u' is `zdb -u [NAME...]'.  The user decides
which compctl to autoload.  Each autoloaded compctl is a compctlp
structure with a NULL cc (compctl struct) member.  That is the amount of
memory used until the zle references it, at which time, the cc member
is created.

> Further, {b}compctl -L would have to get a lot smarter, and merge
> together into one output item the whole class of commands that use the
> same compctl.  E.g. stop doing this:
> 
> % compctl -l '' exec noglob nocorrect
> % compctl -L
> compctl -l '' exec
> compctl -l '' nocorrect
> compctl -l '' noglob

My zdb builtin spends a lot of time on getting this list of commands.
I suggest an additional member to the compctlp structure:
    LinkList cmds;
which treads through all compctlp in the compctltab that have the same
cc member.  Like the cc member, there is only one copy of the cmds
member.

While we are on the subject of the group of commands that shares the
same compctl, let me share my experience writing zdb.  Suppose the
user does the following
    % compctl ... cmd1 cmd2 cmd3
    % compctl ... cmd4 cmd5 cmd6
    % <save to a database>
    % compctl ... cmd1 cmd4 cmd7
    % <save to the same database>
If we use `bcompctl -L ... > database' as the mechanism to save to the
database, we will have 3 records in the database.  But which one to
fetch for cmd1?  For cmd2?  Saving to the database is not trivial.
The database must be treated as persistent storage for the compctltab.

> Well, a database of some sort (if only a simple one) would be a good
> input/output format for bcompctl.  Further, one could treat loading of
> compctls the way zsh currently treats filling in the command hash tables
> (and even make it subject to the HASH options) -- that is, don't load
> the database immediately, and then load only as much as necessary each
> time a completion is requested.

I suggest letting the user decide which compctl to autoload.

Regards,
fclim.