Re: [PATCH] Proposal: stat -> zstat

[Peter Stephenson <pws@xxxxxxx>]

Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:
> } Each module can advertise a set of features it supports. These will
> } be variables, builtins, conditions or math functions; we'll need a
> } convention for how to disambiguate. I wouldn't propose to build the
> } convention into the API, however, just represent each feature by a
> } string
>
> Do I correctly understand that the naming convention has nothing to do
> with what happens when one asks that feature X of module Z be loaded
> or enabled?  Your suggested -F option implies as much, but presently
> there are different flags for
> builtins/conditions/parameters/mathfuncs.

In my reply to Phil I suggested using a convention like b:zstat,
p:functions, f:sin, c:-pcre-match, etc., to get around namespace problems.
It's quite possible we don't have clashes at the moment, but it's entirely
conceivable a module may export both a builtin and a parameter with the
same name.  I think ":" is a good choice here---doesn't need quoting, used
in a similar way for styles---but I'd be happy to entertain alternatives.
I said m: for math functions before, but f: makes more sense
given the next point below.

This ought to be transparent to the main shell; it just passes down a
feature name to be handled by the module...

> I guess the real question is, is there any explicit interaction
> between -ab/-ac/-af/-ap and -F?  Could something bad happen if I ask
> for a parameter feature but say it should be loaded as a builtin?

...however, this is an interesting point.  It does seem a pointless extra
complexity to keep this separate from the feature stuff.

Suppose:

zmodload -ab zsh/nosh fishfingers

is defined to be equivalent to

zmodload -Fa zsh/nosh b:fishfingers

and we document that the the second form is preferred?  In other words, the
-a addition to the feature mechanism *does* require the name to be in the
form <codechar>:<featurename>, and generates an autoload request of the
appropriate type (or an error if it doesn't recognise the prefix).  This
uses the same codes, but as part of the feature name rather than an
additional option.

One extra tweak at least is needed: if we loaded a module through the new
form of autoload, we only enable any features explicitly requested in that
list.  For compatibility the old form would fully load the module (it may
well be that a function assumes that autoloading one builtin will bring in a
whole load of others).  So I think we need a list separate from the
autoload stubs put into the relevant hash table (that's just an
implementation detail; we could alternatively search for autoload stubs
when we load the module without enabling all features).

Presumably

zmodload -ab zsh/nosh b:fishfingers

should generate an error for sanity checking; since this would be
essentially a legacy interface it would probably be OK to veto any colon
in the "feature" part.

> Conversely (sort of), can features be collections of several builtins/
> conditions/etc.?  Hypothetically, if I ask zmodload for the
> "bash-regex" feature, do I get both the =~ condtional and the
> BASH_REMATCH parameter? (I know, those aren't directly linked in the
> current implementation, but I needed an example.)

Hypothetically, I think this is OK and the documentation for the module can
make clear what is actually going on; by not using the <codechar>:
prefix it should also be clear in the naming that this is an abstract
feature rather than a specific shell builtin etc.  We could probably
suggest (as a design guideline) that such features don't contain a ":" at
all.

It's then entirely up to the module what it does with such a feature.  It
could even, say, set all the options for a particular emulation.
I would propose to document zmodload to make this clear.

> } The set of enabled features can be an arbitrary-length array of
> } unsigned integers with bits giving the enabled status. We can
> } explicitly limit ourselves to the first 32 bits of each integer for
> } future proofing. Obviously we need one bit for each feature array
> } element.
> 
> I'm not following how this maps to the individual modules. Is there
> one integer in the array per module, so each module is limited to 32
> features?  

No, that's why I said "an arbitrary-length array" of integers.  However,
I'm coming round to the view that it's much simpler to have just an
array the same length as the feature list; the additional memory
requirement isn't so great when we have strings floating around anyway.

Nonetheless, if anyone can think of a neat way of not having to pass up and
down every single feature on every call that would be interesting.
I don't want to make the API too complex, however.

> } zmodload <module>
> }   loads everything (as at present).  If the module is already
> loaded, and }   supports features, zmodload will check to see if all
> features are loaded. }   If they are, it's an error.  If not, it will
> enable them all.
> 
> I've always wondered why zmodload doesn't simply behave by default as
> if the -i option were provided.  What's useful about generating an
> error if the module is already loaded?

That's a good question I can't answer.  I've simply kept this because it's
always behaved that way, but I can't think of a good reason to use it.
Unless anyone can see one, I'd be perfectly happy to make -i the default.
There's no point adding an option in the opposite sense, since if you have
to add new code to check anyway you might as well use zmodload -e.

> } zmodload -F <module> feature1 feature2
> }    will load the module or cause an error if it's already loaded,
> }    and turn on feature1 and feature2 (only---all others are
> disabled).
> 
> How does this interact with autoloading?>

Basic suggestion above, but...

> Can features (or the user,
> with zmodload -d) declare that they depend on other features (of the
> same or other modules)?  

Urk.  The current dependency mechanism is per-module and if we're going to
keep it then to do it properly we need it to be per-feature.
If each *feature* in the target module can depend on a feature in another
module it gets very hairy even to specify.  I'm not sure we want to go down
that road immediately, but something like

zsh -dF target-module feature1... -- dependency-module featureN+1...

could do it.

Although this is a significant question, I think we could implement module
features without this initially; at the moment the dependencies used by the
shell are at a fairly gross level and I don't think any of the supplied
functions specify dependencies.  Typically you need all of a support module
(e.g. zsh/zftp and zsh/net/tcp).

> What happens if you try to turn off a feature
> that some other feature depends on?

If we do something like the above, we can keep the specified line as a
database (instead of dumping the info when the target module has been
loaded, as at present).  Then, as long as we get the internal structures
right, it ought to be easy to check; e.g.

% zmodload -F dependency-module -featureN-1
zsh: zmodload: featureN-1 of dependency-module is required by target-module

We may need to improve listing of dependencies, though the basics for there
are that (running "zmodload -d target-module" will explain the error
message above).  We already have -u to remove dependencies and that can be
added to the syntax suggested above.

(Unnecessarily detailed point of implementation: by implication, the
features are defined by there position in an array returned by the module.
So we can use that index internally to refer to the feature.  This would
make it useful to have completely separate calls to the module for features
(which are read-only) and feature-enables.

  char **features_(Module m);

Returns the NULL-terminated array of features.  I've a vague feeling it
would be beneficial to have this zalloc'd for the reasons given below.

  int *feature_enables_(Module m, int *intarrptr);

If intarrptr is non-NULL, set the features from the array (*intarrptr).  In
any case, return a point to the array of current feature enables; if
intarrptr was passed, this is returned, else zalloc'd memory.

This API doesn't impose the burden on the module of a permanent array of
feature enables in the expanded form, one integer per feature.  That might
be a significant saving for a module like zsh/mathfunc with a lot of
builtins.  With a query/set, we get the benefit that the array is only
allocated once.

> } zmodload -iF <module> -feature1 feature3
> }    will load the module if it's not already loaded, turn off
> feature1 }    and turn on feature3, leaving all other features alone.
>
> I suggest that this work like options, where a "no" prefix disables
> the feature.  (Hyphens and underscores ignored like options, too.)
>
> I'm not strongly opposed to using a leading "-" but if we do I think a
> leading "+" should also be allowed.

I'm perfectly happy with allowing +.  I'm not that keen on "no" for general
feature use because of the "notify"/"nomatch" problem: we need to test
everything twice just in case.  (I don't like limitations on feature names
based on details of the English language, either.)

Ignoring punctuation is also a bit problematic in the case where the
feature maps directly to a builtin etc.; underscores are natural in builtin
names and "-" is a natural part of a condition.

I can certainly see why you'd want name munging for abstract/collective
features.  However, leaving this up the module is a bit fraught since the
main shell needs to be able to do the comparison to manipulate the feature
enables (unless we handle the feature list parsing at a lower level, which
I don't like).  So I don't see any easy way to get this unless we have some
convention for names of certain types.  Hmm... if the feature enables is
now a full integer, perhaps we could define a bit in it to say "supports
name munging"?

> Also, what about the -u option?
> Does "zmodload -uF ..." mean anything?

I don't think so (except as described for dependencies).  It doesn't make
sense to unload a module unless you're turning off all features, and we
don't need a special syntax to say that's what's happening when you do
unload a module.

-- 
Peter Stephenson <pws@xxxxxxx>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


To access the latest news from CSR copy this link into a web browser:  http://www.csr.com/email_sig.php

To get further information regarding CSR, please visit our Investor Relations page at http://ir.csr.com/csr/about/overview