Re: User-defined zle widgets and built-in widget failure

[Zefram <zefram@xxxxxxxxx>]

Bart Schaefer wrote:
>I think we should tie the most common cases to standard scripting concepts;
>e.g., failure should be a nonzero $?, and the numeric value of $? should
>indicate the reason for failure.  It shouldn't be necessary to invoke "zle"
>multiple times to find out this kind of basic information; further, it
>makes it obvious how a widget writer sets the value, i.e., with "return".

Good points.  I think that's worth handling as a special case, even
though there are many other bits of state that will need a more
complete mechanism.  This special case can be defined pretty well:
zle returns non-zero iff the feep flag is set at the time of return,
and the feep flag is set on return from a user-defined widget iff the
function returns non-zero.

>} Or a more general possibility would be thingy array variables
>} accessible via thingy names
>
>Just how many thingys are we talking about here?

I was thinking about a general array mechanism, using thingies to make
pushing them onto the zle stack particularly efficient.  However, it
would essentially be duplicating the existing array parameter mechanism,
so I'd rather use what we've already got.

>I suggest defining one array variable $ZLE (or whatever name you want),
>and define the meanings of the array positions.  E.g., ZLE could be
>equivlent to ($BUFFER $CURSOR $LBUFFER $RBUFFER) in the current scheme;
>instead of BUFFER="the text here" you'd say ZLE[1]="the text here".

Yuck.  This arbitrary index issue is one of the things I dislike about
Perl (the stat() interface, specifically).  Let's stick to names.  And,
of course, separate parameters is the only way to handle array values,
since we don't have nested arrays.

>} (though I'd want to do a proper namespace separation, e.g., ${zle.feep}).
>
>You're creeping dangerously close to Perl associative arrays, there.

I don't see that as an associative array.  It's just a structured
namespace for parameters.

>Unless we're talking about a huge amount of data, just use numbered
>positions in an array with a "sufficiently unique" name.

Really, we're talking about *arbitrary* amounts of data.  Non-builtin
widgets whould be able to store their own data in the same manner as
the standard widgets, and ideally be able to keep that data out of the
way of normal user variables.

>I think I'd rather not have to rewrite my .z* scripts again.  How much
>"moving" are we talking about?

Actually, I don't anticipate permanently removing names established
in 3.0.  An option, on by default in zsh mode, would enable the current
names.  Those that take the time to rewrite their .z* scripts can then
disable the current names, and get a parameter namespace as clean as
possible.  Of course, the POSIX-mandated parameters have to retain their
current names in any case, but all the zsh extensions could be renamed.

If we *don't* introduce a proper namespace scheme, I hate to think what
will happen when modules gain the ability to define special parameters.


This is probably a good time to talk about what I've been doing for the
past year.  (Standard disclaimer applies.)  I work for Tao Systems Ltd.,
which is in the business of writing a rather interesting portable OS
called Elate.  One of the main features of this OS is that each utility
and each library function is dynamically loaded, in a shareable manner
-- the equivalent under Unix would be to put every library function in
a separate shared library, though of course it's much more efficient
in Elate.

Among other things, I was assigned to design and write the standard
Elate shell.  (There are many zsh fans here.)  I have had the opportunity
to write a shell that has really clean syntax and consistent behaviour,
initially without any pressure to make the shell practically usable.
(It finished up pretty usable *and* clean, because I had the time to
achieve both aims.)  I also had the opportunity to establish many of
the conventions for this shell environment.  Having experimented with
a lot of my shell ideas on this project, I think some of them are worth
trying to get into zsh in some form.

One of the conventions I established was that all environment variables,
except for strictly user-generated ones, have structured names.
For example, the equivalent of the Unix "PATH" variable is called
"shell.path".  The equivalent of "LOCALE" is called "user.locale".
This means that scripts, or the user at the shell prompt, can use *any*
alphanumeric variable name, without any risk of colliding with anyone
else's variables, now or in the future.  And utility writers have a
guaranteed way to define variables special to a particular utility
without clashing with anyone else's variables, so there's none of this
"MAILFILE" nonsense.

What I like most about this convention is that, as the shell writer, I
can invent new special variables without colliding with anyone else.  If I
add a feature like zsh's $WATCH, I can call the parameter "shell.watch",
and there is no problem even if an existing script happened to call a
variable "watch", which is not an unlikely occurrence.

Another feature worth examining is the design of the command line editor.
I actually designed it to be much like I wanted ZLE to be.  The key
feature is that the editor is almost entirely written in shell code --
the only builtin code written for it is (1) the display updating code and
(2) an interface to the parser (for completion).  The buffer is actually
stored in shell variables ("sle.lbuffer" et al), and so is trivially
manipulable by user-defined editor functions.  In fact, user-defined
editing functions are precisely on level footing with the standard
functions, because *all* of them are simply shell functions.

I'm not suggesting that we go to that extreme with ZLE -- there is an
enormous performance hit in doing the simplest editing actions in shell
functions, and completion suffers somewhat -- but I do want to achieve
three basic aims:

	(1) as little as possible should be built into the base zle module

	(2) built-in widgets should use the same interfaces as
	    user-defined ones, so that they behave *as if* they were
	    shell functions

	(3) as much of the ZLE state as possible should be directly
	    manipulable by user-defined widgets

My experience with the Elate shell has shown that building in nothing
(even the ability to run the various types of commands is dynamically
loaded), and doing as much as possible without inventing new mechanisms,
makes a shell very easily extensible, and very radically configurable
for any specific purpose.

-zefram