Re: Default fpath

[Sven Wischnowsky <wischnow@xxxxxxxxxxxxxxxxxxxxxxx>]

Bart Schaefer wrote:

> On Mar 14, 11:19am, Sven Wischnowsky wrote:
> } Subject: Re: Default fpath
> }
> } Bart Schaefer wrote:
> } > Actually, what would suit me fine is a way to dump out a
> } > .zwc of (a subset of the) functions defined in a running zsh.
> } 
> } That's easy. Would you prefer it as an option to `zcompile' or one for
> } `functions'?
> 
> I think it should go in zcompile, just because of potential confusion
> between "functions" and "typeset -f".  It's not *that* big a deal, I
> guess, but somehow having anything that's described as "equivalent to
> typeset" write to a file, bothers me.

Just what I preferred, so I had this patch already...

> [Tagging .zwc for zsh/ksh loading:]
> } 
> }   zcompile -z comp.zwc .../Completion/*/_*
> } 
> } to have them all loaded zsh-like.
> } 
> }   zcompile foo.zwc -z bar -k baz
> } 
> } would load bar zsh-like and baz ksh-like.
> } 
> } Ok?
> 
> Seems fine to me.  Hmm, maybe autoload should have the `k' and `z' opts,
> too, but they only work with +X (the load-it now option).  That would
> fix something that's been bugging me for a while.

Sounds good. No patch for any of this yet.

> } But maybe this is a case where adding functions to a wordcode file
> } would be handy:
> } 
> }   zcompile foo.zwc -z bar1 baz1
> }   zcompile -a foo.zwc -k bar2 baz2
> } 
> } would create a wordcode file with bar1/baz1 loaded zsh-like and then
> } add bar2/baz2 loaded ksh-like to it.
> 
> That would be reasonable, too, but with the "autoload" patches as well,
> I think the `zcompile -a ...' line would be equivalent to
> 
>     zsh -fc 'fpath=(...);\
> 	     autoload +Xw foo;\
> 	     autoload +Xk bar2 baz2;\
> 	     zcompile -c foo'
> 
> which could be provided as a utility function rather than a builtin.

Sounds even better, especially after having thought some more about
trying to append to existing wordcode files -- it would mostly look
like the above, only in C.

One question: `autoload -w' does not use $fpath to search the dump
file, should it? Or maybe with `+Xw' we should (temporarily and only
internally) prepend the dump file to $fpath so that `autoload +Xw dump'
loads all functions from a dump file without having to set $fpath
explicitly? Or would that be irritating for anyone. Or maybe the
current behaviour is irritating?

Anyway, I think, I prefer the latter... (prepending the dump file).


The patch adds the `-c' and `-M' options to autoload. `-c' means that
function definitions from the current shell are written:

  zcompile -c zwc f1 f2 ...

write f1, f2, ... to the wordcode file. If no function name is given,
all functions are written. Functions that are only marked for
autoloading are written, too (i.e. their definition), if they can be
found via $fpath. This will not make them be autoloaded by the shell
(i.e. after zcompile, they are still marked for autoloading).

`-c' can be combined with `-M' which makes the names be taken as
patterns and only functions whose names match any of these patterns
are written. zcompile is careful to write each function only once even 
if its name is matched by multiple patterns (or is it `... name
matches multiple patterns'? To repeat Andrej's question.).

That `-M' is a bit ugly, because normally `-m' is the option to make
arguments be used as patterns. Should we change zcompile's -r and -m
options to -R and -M and use -m for this pattern-thing?

Bye
 Sven

P.S.: The patch looks bigger than it is because I moved the code to
      write a wordcode file in its own function.

diff -ru ../z.old/Doc/Zsh/builtins.yo Doc/Zsh/builtins.yo
--- ../z.old/Doc/Zsh/builtins.yo	Tue Mar 14 16:50:31 2000
+++ Doc/Zsh/builtins.yo	Wed Mar 15 10:31:27 2000
@@ -1296,6 +1296,7 @@
 cindex(wordcode, creation)
 cindex(compilation)
 xitem(tt(zcompile) [ tt(-U) ] [ tt(-r) | tt(-m) ] var(file) [ var(name) ... ])
+xitem(tt(zcompile) tt(-c) [ tt(-M) ] [ tt(-r) | tt(-m) ] var(file) [ var(name) ... ])
 item(tt(zcompile -t) var(file) [ var(name) ... ])(
 This builtin command can be used to create and display files
 containing the wordcode for functions or scripts. In the first form, a wordcode
@@ -1342,7 +1343,15 @@
 it is read or mapped, only one half of the file will really be used
 (and mapped).
 
-In the second form, with the tt(-t) option, an existing wordcode file is
+If given the tt(-c) option, the names have to be names currently
+defined in the shell or marked as autoloaded. The definitions for all
+these functions will be written into the wordcode var(file). If the
+tt(-M) option is given, too, the var(name)s are used as patterns and
+all functions whose names match one of these patterns will be
+written. If no var(name) is given, the definitions of all functions
+currently defined or marked as autoloaded will be written. 
+
+In the third form, with the tt(-t) option, an existing wordcode file is
 tested. Without further arguments, the names of the original files
 used for it are listed. The first line tells the version of the shell
 the file was created with and how the file will be used (mapping or
diff -ru ../z.old/Src/Modules/parameter.c Src/Modules/parameter.c
--- ../z.old/Src/Modules/parameter.c	Wed Mar 15 10:07:20 2000
+++ Src/Modules/parameter.c	Wed Mar 15 10:09:05 2000
@@ -345,7 +345,7 @@
 	return;
     }
     shf = (Shfunc) zalloc(sizeof(*shf));
-    shf->funcdef = zdupeprog(prog);
+    shf->funcdef = dupeprog(prog, 0);
     shf->flags = dis;
 
     if (!strncmp(name, "TRAP", 4) &&
diff -ru ../z.old/Src/builtin.c Src/builtin.c
--- ../z.old/Src/builtin.c	Wed Mar 15 10:07:10 2000
+++ Src/builtin.c	Wed Mar 15 10:09:06 2000
@@ -124,7 +124,7 @@
     BUILTIN("where", 0, bin_whence, 0, -1, 0, "pmsw", "ca"),
     BUILTIN("which", 0, bin_whence, 0, -1, 0, "ampsw", "c"),
     BUILTIN("zmodload", 0, bin_zmodload, 0, -1, 0, "ILabcfdipue", NULL),
-    BUILTIN("zcompile", 0, bin_zcompile, 0, -1, 0, "tUmr", NULL),
+    BUILTIN("zcompile", 0, bin_zcompile, 0, -1, 0, "tUmrcM", NULL),
 };
 
 /****************************************/
@@ -3895,7 +3895,7 @@
 	    zwarnnam(name, "undefined signal: %s", *argv, 0);
 	    break;
 	}
-	t = zdupeprog(prog);
+	t = dupeprog(prog, 0);
 	if (settrap(sig, t))
 	    freeeprog(t);
     }
diff -ru ../z.old/Src/exec.c Src/exec.c
--- ../z.old/Src/exec.c	Wed Mar 15 10:07:10 2000
+++ Src/exec.c	Wed Mar 15 10:09:06 2000
@@ -3184,7 +3184,7 @@
 	if (prog->alloc == EA_MAP)
 	    shf->funcdef = stripkshdef(prog, shf->nam);
 	else
-	    shf->funcdef = zdupeprog(stripkshdef(prog, shf->nam));
+	    shf->funcdef = dupeprog(stripkshdef(prog, shf->nam), 0);
 	shf->flags &= ~PM_UNDEFINED;
     }
     popheap();
@@ -3217,7 +3217,7 @@
     if (prog->alloc == EA_MAP)
 	shf->funcdef = stripkshdef(prog, shf->nam);
     else
-	shf->funcdef = zdupeprog(stripkshdef(prog, shf->nam));
+	shf->funcdef = dupeprog(stripkshdef(prog, shf->nam), 0);
     shf->flags &= ~PM_UNDEFINED;
 
     popheap();
@@ -3359,7 +3359,7 @@
  * list of its contents.                                                    */
 
 /**/
-static Eprog
+Eprog
 getfpfunc(char *s)
 {
     char **pp, buf[PATH_MAX];
diff -ru ../z.old/Src/linklist.c Src/linklist.c
--- ../z.old/Src/linklist.c	Wed Mar 15 10:07:11 2000
+++ Src/linklist.c	Wed Mar 15 10:09:06 2000
@@ -239,7 +239,7 @@
 }
 
 /**/
-void
+mod_export void
 rolllist(LinkList l, LinkNode nd)
 {
     l->last->next = l->first;
@@ -251,7 +251,7 @@
 }
 
 /**/
-LinkList
+mod_export LinkList
 newsizedlist(int size)
 {
     LinkList list;
@@ -270,4 +270,17 @@
     node[-1].next = NULL;
 
     return list;
+}
+
+/**/
+mod_export int
+listcontains(LinkList list, void *dat)
+{
+    LinkNode node;
+
+    for (node = firstnode(list); node; incnode(node))
+	if (getdata(node) == dat)
+	    return 1;
+
+    return 0;
 }
diff -ru ../z.old/Src/parse.c Src/parse.c
--- ../z.old/Src/parse.c	Wed Mar 15 10:07:13 2000
+++ Src/parse.c	Wed Mar 15 10:16:09 2000
@@ -1980,7 +1980,7 @@
 
 /**/
 mod_export Eprog
-zdupeprog(Eprog p)
+dupeprog(Eprog p, int heap)
 {
     Eprog r;
     int i;
@@ -1989,12 +1989,13 @@
     if (p == &dummy_eprog)
 	return p;
 
-    r = (Eprog) zalloc(sizeof(*r));
+    r = (heap ? (Eprog) zhalloc(sizeof(*r)) : (Eprog) zalloc(sizeof(*r)));
     r->alloc = EA_REAL;
     r->dump = NULL;
     r->len = p->len;
     r->npats = p->npats;
-    pp = r->pats = (Patprog *) zcalloc(r->len);
+    pp = r->pats = (heap ? (Patprog *) hcalloc(r->len) :
+		    (Patprog *) zcalloc(r->len));
     r->prog = (Wordcode) (r->pats + r->npats);
     r->strs = ((char *) r->prog) + (p->strs - ((char *) p->prog));
     memcpy(r->prog, p->prog, r->len - (p->npats * sizeof(Patprog)));
@@ -2263,6 +2264,7 @@
 bin_zcompile(char *nam, char **args, char *ops, int func)
 {
     int map;
+    char *dump;
 
     if (ops['t']) {
 	Wordcode f;
@@ -2295,14 +2297,19 @@
 	zwarnnam(nam, "too few arguments", NULL, 0);
 	return 1;
     }
+    if ((ops['c'] && ops['U']) || (!ops['c'] && ops['M'])) {
+	zwarnnam(nam, "illegal combination of options", NULL, 0);
+	return 1;
+    }
     map = (ops['m'] ? 2 : (ops['r'] ? 0 : 1));
 
-    if (!args[1])
+    if (!args[1] && !ops['c'])
 	return build_dump(nam, dyncat(*args, FD_EXT), args, ops['U'], map);
 
-    return build_dump(nam,
-		      (strsfx(FD_EXT, *args) ? *args : dyncat(*args, FD_EXT)),
-		      args + 1, ops['U'], map);
+    dump = (strsfx(FD_EXT, *args) ? *args : dyncat(*args, FD_EXT));
+
+    return (ops['c'] ? build_cur_dump(nam, dump, args + 1, ops['M'], map) :
+	    build_dump(nam, dump, args + 1, ops['U'], map));
 }
 
 /* Load the header of a dump file. Returns NULL if the file isn't a
@@ -2374,16 +2381,73 @@
 
 /* Write a dump file. */
 
+static void
+write_dump(int dfd, LinkList names, LinkList progs, int map,
+	   int hlen, int tlen)
+{
+    LinkNode name, node;
+    int other = 0, ohlen, tmp;
+    wordcode pre[FD_PRELEN];
+    char *tail, *n;
+    struct fdhead head;
+    Eprog prog;
+
+    if (map == 1)
+	map = (tlen >= FD_MINMAP);
+
+    for (ohlen = hlen; ; hlen = ohlen) {
+	fdmagic(pre) = (other ? FD_OMAGIC : FD_MAGIC);
+	fdsetflags(pre, ((map ? FDF_MAP : 0) | other));
+	fdsetother(pre, tlen);
+	strcpy(fdversion(pre), ZSH_VERSION);
+	write(dfd, pre, FD_PRELEN * sizeof(wordcode));
+
+	for (node = firstnode(progs), name = firstnode(names); node;
+	     incnode(node), incnode(name)) {
+	    n = (char *) getdata(name);
+	    prog = (Eprog) getdata(node);
+	    head.start = hlen;
+	    hlen += (prog->len - (prog->npats * sizeof(Patprog)) +
+		     sizeof(wordcode) - 1) / sizeof(wordcode);
+	    head.len = prog->len - (prog->npats * sizeof(Patprog));
+	    head.npats = prog->npats;
+	    head.strs = prog->strs - ((char *) prog->prog);
+	    head.hlen = (sizeof(struct fdhead) / sizeof(wordcode)) +
+		(strlen(n) + sizeof(wordcode)) / sizeof(wordcode);
+	    if ((tail = strrchr(n, '/')))
+		tail++;
+	    else
+		tail= n;
+	    head.tail = tail - n;
+	    if (other)
+		fdswap((Wordcode) &head, sizeof(head) / sizeof(wordcode));
+	    write(dfd, &head, sizeof(head));
+	    tmp = strlen(n) + 1;
+	    write(dfd, n, tmp);
+	    if ((tmp &= (sizeof(wordcode) - 1)))
+		write(dfd, &head, sizeof(wordcode) - tmp);
+	}
+	for (node = firstnode(progs); node; incnode(node)) {
+	    prog = (Eprog) getdata(node);
+	    tmp = (prog->len - (prog->npats * sizeof(Patprog)) +
+		   sizeof(wordcode) - 1) / sizeof(wordcode);
+	    if (other)
+		fdswap(prog->prog, (((Wordcode) prog->strs) - prog->prog));
+	    write(dfd, prog->prog, tmp * sizeof(wordcode));
+	}
+	if (other)
+	    break;
+	other = FDF_OTHER;
+    }
+}
+
 /**/
 static int
 build_dump(char *nam, char *dump, char **files, int ali, int map)
 {
-    int dfd, fd, hlen, tlen, flen, tmp, ona = noaliases, other = 0, ohlen;
-    LinkList progs;
-    LinkNode node;
-    struct fdhead head;
-    wordcode pre[FD_PRELEN];
-    char *file, **ofiles = files, **oofiles = files, *tail;
+    int dfd, fd, hlen, tlen, flen, ona = noaliases;
+    LinkList progs, names;
+    char *file;
     Eprog prog;
 
     if (!strsfx(FD_EXT, dump))
@@ -2394,6 +2458,7 @@
 	return 1;
     }
     progs = newlinklist();
+    names = newlinklist();
     noaliases = ali;
 
     for (hlen = FD_PRELEN, tlen = 0; *files; files++) {
@@ -2434,9 +2499,10 @@
 	zfree(file, flen);
 
 	addlinknode(progs, prog);
+	addlinknode(names, *files);
 
 	flen = (strlen(*files) + sizeof(wordcode)) / sizeof(wordcode);
-	hlen += (sizeof(head) / sizeof(wordcode)) + flen;
+	hlen += (sizeof(struct fdhead) / sizeof(wordcode)) + flen;
 
 	tlen += (prog->len - (prog->npats * sizeof(Patprog)) +
 		 sizeof(wordcode) - 1) / sizeof(wordcode);
@@ -2444,53 +2510,138 @@
     noaliases = ona;
 
     tlen = (tlen + hlen) * sizeof(wordcode);
-    if (map == 1)
-	map = (tlen >= FD_MINMAP);
 
-    for (ohlen = hlen; ; hlen = ohlen) {
-	fdmagic(pre) = (other ? FD_OMAGIC : FD_MAGIC);
-	fdsetflags(pre, ((map ? FDF_MAP : 0) | other));
-	fdsetother(pre, tlen);
-	strcpy(fdversion(pre), ZSH_VERSION);
-	write(dfd, pre, FD_PRELEN * sizeof(wordcode));
+    write_dump(dfd, names, progs, map, hlen, tlen);
 
-	for (node = firstnode(progs), ofiles = oofiles; node;
-	     ofiles++, incnode(node)) {
-	    prog = (Eprog) getdata(node);
-	    head.start = hlen;
-	    hlen += (prog->len - (prog->npats * sizeof(Patprog)) +
-		     sizeof(wordcode) - 1) / sizeof(wordcode);
-	    head.len = prog->len - (prog->npats * sizeof(Patprog));
-	    head.npats = prog->npats;
-	    head.strs = prog->strs - ((char *) prog->prog);
-	    head.hlen = (sizeof(struct fdhead) / sizeof(wordcode)) +
-		(strlen(*ofiles) + sizeof(wordcode)) / sizeof(wordcode);
-	    if ((tail = strrchr(*ofiles, '/')))
-		tail++;
-	    else
-		tail= *ofiles;
-	    head.tail = tail - *ofiles;
-	    if (other)
-		fdswap((Wordcode) &head, sizeof(head) / sizeof(wordcode));
-	    write(dfd, &head, sizeof(head));
-	    tmp = strlen(*ofiles) + 1;
-	    write(dfd, *ofiles, tmp);
-	    if ((tmp &= (sizeof(wordcode) - 1)))
-		write(dfd, &head, sizeof(wordcode) - tmp);
+    close(dfd);
+
+    return 0;
+}
+
+static int
+cur_add_func(Shfunc shf, LinkList names, LinkList progs, int *hlen, int *tlen)
+{
+    Eprog prog;
+
+    if (shf->flags & PM_UNDEFINED) {
+	int ona = noaliases;
+
+	noaliases = (shf->flags & PM_UNALIASED);
+	if (!(prog = getfpfunc(shf->nam)) || prog == &dummy_eprog) {
+	    noaliases = ona;
+
+	    return 1;
 	}
-	for (node = firstnode(progs); node; incnode(node)) {
-	    prog = (Eprog) getdata(node);
-	    tmp = (prog->len - (prog->npats * sizeof(Patprog)) +
-		   sizeof(wordcode) - 1) / sizeof(wordcode);
-	    if (other)
-		fdswap(prog->prog, (((Wordcode) prog->strs) - prog->prog));
-	    write(dfd, prog->prog, tmp * sizeof(wordcode));
+	if (prog->dump)
+	    prog = dupeprog(prog, 1);
+	noaliases = ona;
+    } else
+	prog = dupeprog(shf->funcdef, 1);
+
+    addlinknode(progs, prog);
+    addlinknode(names, shf->nam);
+
+    *hlen += ((sizeof(struct fdhead) / sizeof(wordcode)) +
+	      ((strlen(shf->nam) + sizeof(wordcode)) / sizeof(wordcode)));
+    *tlen += (prog->len - (prog->npats * sizeof(Patprog)) +
+	      sizeof(wordcode) - 1) / sizeof(wordcode);
+
+    return 0;
+}
+
+/**/
+static int
+build_cur_dump(char *nam, char *dump, char **names, int match, int map)
+{
+    int dfd, hlen, tlen;
+    LinkList progs, lnames;
+    Shfunc shf;
+
+    if (!strsfx(FD_EXT, dump))
+	dump = dyncat(dump, FD_EXT);
+
+    if ((dfd = open(dump, O_WRONLY|O_CREAT, 0600)) < 0) {
+	zwarnnam(nam, "can't write dump file: %s", dump, 0);
+	return 1;
+    }
+    progs = newlinklist();
+    lnames = newlinklist();
+
+    hlen = FD_PRELEN;
+    tlen = 0;
+
+    if (!*names) {
+	int i;
+	HashNode hn;
+
+	for (i = 0; i < shfunctab->hsize; i++)
+	    for (hn = shfunctab->nodes[i]; hn; hn = hn->next)
+		if (cur_add_func((Shfunc) hn, lnames, progs, &hlen, &tlen)) {
+		    zwarnnam(nam, "can't load function: %s", shf->nam, 0);
+		    errflag = 0;
+		    close(dfd);
+		    unlink(dump);
+		    return 1;
+		}
+    } else if (match) {
+	char *pat;
+	Patprog pprog;
+	int i;
+	HashNode hn;
+
+	for (; *names; names++) {
+	    tokenize(pat = dupstring(*names));
+	    if (!(pprog = patcompile(pat, PAT_STATIC, NULL))) {
+		zwarnnam(nam, "bad pattern: %s", *names, 0);
+		close(dfd);
+		unlink(dump);
+		return 1;
+	    }
+	    for (i = 0; i < shfunctab->hsize; i++)
+		for (hn = shfunctab->nodes[i]; hn; hn = hn->next)
+		    if (!listcontains(lnames, hn->nam) &&
+			pattry(pprog, hn->nam) &&
+			cur_add_func((Shfunc) hn, lnames, progs,
+				     &hlen, &tlen)) {
+			zwarnnam(nam, "can't load function: %s", shf->nam, 0);
+			errflag = 0;
+			close(dfd);
+			unlink(dump);
+			return 1;
+		    }
 	}
-	if (other)
-	    break;
-	other = FDF_OTHER;
+    } else {
+	for (; *names; names++) {
+	    if (errflag ||
+		!(shf = (Shfunc) shfunctab->getnode(shfunctab, *names))) {
+		zwarnnam(nam, "unknown function: %s", *names, 0);
+		errflag = 0;
+		close(dfd);
+		unlink(dump);
+		return 1;
+	    }
+	    if (cur_add_func(shf, lnames, progs, &hlen, &tlen)) {
+		zwarnnam(nam, "can't load function: %s", shf->nam, 0);
+		errflag = 0;
+		close(dfd);
+		unlink(dump);
+		return 1;
+	    }
+	}
+    }
+    if (empty(progs)) {
+	zwarnnam(nam, "no functions", NULL, 0);
+	errflag = 0;
+	close(dfd);
+	unlink(dump);
+	return 1;
     }
+    tlen = (tlen + hlen) * sizeof(wordcode);
+
+    write_dump(dfd, lnames, progs, map, hlen, tlen);
+
     close(dfd);
+
     return 0;
 }
 

--
Sven Wischnowsky                         wischnow@xxxxxxxxxxxxxxxxxxxxxxx