Re: PATCH: wordcode files

[Sven Wischnowsky <wischnow@xxxxxxxxxxxxxxxxxxxxxxx>]

Bart Schaefer wrote:

> } Wordcode files can be used in two ways. One way is to put them into
> } the directories in $fpath with the .zwc extension. [...] The other way
> } is to put the name of the wordcode file into $fpath. In that case the
> } .zwc extension isn't required [...]
> 
> This all sounds fine, although I lean towards requiring that the .zwc
> extension appear in the $fpath listing.  What happens, for example, if
> I have both a ~/zshfun/ and ~/zshfun.zwc and I list ~/zshfun in $fpath?

You get the functions in ~/zshfun because you named that. But we can
change that to be more consistent (I mean: requiring the extension).

> ...
> 
> } [...] I also find it tempting to turn most of my .z* files (including
> } the completion dump file) into functions and putting them into a
> } wordcode file, but I haven't implemented any support for that either.
> } Maybe we could extend the mechanism with the *.zwc files to also apply
> } to sourced files -- creating dump files for them is simple, and can be
> } done already with zcompile, it's just that the function for sourcing a
> } file does not check if there is a *.zwc file. Should we?
> 
> What would "source" do if it encountered a function digest file?  Simply
> define all the functions it contains?
> 
> Perhaps we'd need yet another option to zcompile, to tell it to produce
> a source-able file instead of a function digest.  Then "source" ignores
> the digest files and autoloading ignores source-able files.  Unless we
> think it'd be useful to have autoloading simply execute the source-able
> files, but I think that'd have to be limited to the file-in-a-directory
> case and not the file-named-in-$fpath case.

Wordcode files really only contain the parsed code for files, not
functions. The connection functions comes only through the autoloading 
mechanism. Currently it goes like this: getfpfunc() calls
try_dump_file() with the path (from $fpath) and function name. This
then checks if the path names a digest file or if there is a
<path>/<fname>.zwc file and that contains an entry for <fname>. If it
does, it returns the Eprog for the contents of the original file which 
is then used by (the caller of) getfpfunc() to make a function out of
it.

So, to make this work for sourced files, we would only have to change
the name handling a bit. source() gets a pathname (or may get one) and 
we would have to get the trailing component from that, look if there
is a <source-file-name>.zwc file and that contains the contents of
<tail-of-source-file-name> and is younger than that. And even this
test is already there, for the automatic lookup of <fname>.zwc files,
so we only need the strrchr() to get the trailing pathname or to change 
the name-comparison (comparing the name in the wordcode file to the
name searched, which currently only compares the trailing component
even though wordcode files contain the full pathname given to
zcompile).

That's all. A call to strrchr(). On the wordcode file side, at
least. There may be some more to do to be able to execute the Eprog
returned by try_dump_file() in the same way sourced files are executed 
now. I haven't really looked at that yet, a simple test I did
yesterday evening showed, that at least the execution of my startup
files work nicely if they are taken from a wordcode file. The
setup/cleanup done in source() would be easy to share for sourced
normal and wordcode files, I think, but I'm not sure how much of the
stuff in loop() we need retain when sourcing a wordcode file. I.e. if
we really need (some of) the stuff loop() does before/after each
event, we may be in trouble because we would have to make that
executed before/after each list in the Eprog returned from
try_dump_file().

> }   % zcompile foo ../Completion/*/_*~*~
> }   % fpath=(foo)
> } 
> } And then do completion (without having it done before in the same
> } shell)... the first completions (where normally the functions are
> } loaded and parsed) should be a lot faster, especially when using the
> } larger functions like _cvs, _rpm or _pbm.
> 
> So _cvs is still a function that defines a bunch of other functions
> and then calls itself?  When such a function is loaded from a mapped
> digest file, the result of executing it is going to be a bunch of
> functions that have been copied into allocated memory, right?

Yes and the no. _cvs still defines lots of function and then calls
itself, but when the wordcode file is mapped (instead of being read),
we only need to allocate the Eprog structs and the pattern cache (the
latter would have to be allocated in any case). The wordcode block
and string table are shared with the original _cvs, i.e. not copied.
This is possible because mapped wordcode files are (of course) kept in 
a list (of funcdump structs). And these funcdump structs have a
reference counter which is incremented when executing a funcdef from a 
mapped wordcode file. So defining a function in a function which is
defined by a mapped wordcode is really cheap (and fast).


Bye
 Sven


--
Sven Wischnowsky                         wischnow@xxxxxxxxxxxxxxxxxxxxxxx