Re: GH:zsh-users/zsh-completions.

[Oliver Kiddle <okiddle@xxxxxxxxxxx>]

Firstly, thanks for the detailed reply.

Julien Nicoulaud wrote:
> As a side note, I named the organization "zsh-users" and I realized after a
> while this was a poor choice, as it sounds too "official". Many users get
> confused and think this is part of the zsh project (especially with the git
> mirror of the zsh repo). I am open to suggestions regarding this, and if the
> zsh team wants me to rename the org, I will do it.

I'd leave the name as it is. Renaming now would cause more confusion
than it solves.

>   • I believe the release rythm of zsh (~once a year) is a bit too slow for
>     completions. We do ~monthly releases, and I think most users importing it
>     via a "plugin manager" actually directly use the master branch, which is
>     fine IMHO for completions. By the time a compdef lands in a zsh release, it
>     can be already obsolete or need bugfixes. I think this is the biggest point
>     for having compdefs in a separate repo than the shell itself.

The time it takes for updates to reach a release is not something I
viewed as a problem that needed fixing. It reduces the number of my
silly mistakes that end up in a release. If someone wants, they can take
completion functions from git without upgrading zsh itself - and that's
not especially more effort than pulling in an external collection of
functions.

Ideally, at the time of a zsh release, completion definitions are
up-to-date. So the only descrepancies a user will experience are for
changes between that time and when their distribution released (plus
missing out on new functions). But as Eric pointed out, this cuts both
ways: if you're running something like RHEL or Ubuntu LTS and getting
completions from git you'll have big descrepancies in the opposite
direction.

It is of course Peter's prerogative to determine the zsh release
schedule. If there is general agreement that we want to get completion
changes out faster, we might do that while accomodating the needs of
the rest of the shell by branching for each x.y line and continue more
regular x.y.z releases containing only completion (and perhaps bugfix)
changes. That would also need someone willing to do the associated work
that would impose.

>   • I believe a significant part of the contributions we had were because the
>     project is hosted on github It attracts an audience of contributors that
>     are not familiar with the "classic way" (mailing list, IRC, patches, and
>     different process for a each project), and it makes it very easy for
>     "drive-by" contributions and learning by example, which fits well to
>     completions. It also easy to review/provide feedback and iterate on patches
>     (eg: https://github.com/zsh-users/zsh-completions/pull/512)

I'm sure you're right here.

Does anyone else have views on allowing a github model for completion
functions? Most of my arguments against github usage don't really apply
if we're only talking about completions for commands.

And is there a simple way to provide this without separating the repository?

>   • The initial idea was indeed for zsh-completions to be an "incubator", and
>     contribute the compdefs on the mailing-list. The previous point also
>     applies to me, it asks me more work to extract a compdef from the repo,
>     create a patch against zsh, send it to the mailing list, and follow up. I
>     do not spend a lot of time on this project, and I quickly realized that was
>     not something I would be able to do on a regular basis, so I did not do it
>     at all.

If it's to be an "incubator" going forward, I'm happy to handle the, er, "hatching".

>   • If I send a compdef to zsh, what happens then ? We have to wait a least for
>     the next release of zsh to delete it from zsh-completions. But then, not
>     all distros update to the latest release of zsh, or the compdef could have
>     evolved in zsh-completions in the meanwhile, so I can see this could be
>     become quite a headache to manage (for users as well).

We could alleviate some of those concerns by scheduling "hatching" of functions
for shortly before a zsh release and if zsh-completions does releases shortly
after a zsh release.

>   • zsh-completions has a lower quality standard than the completions in zsh
>     itself. For instance, some compdefs are partial, some compdefs use awk/grep
>     /sed/tr instead of the shell builtins, some compdefs use the low level API
>     (compadd, etc) instead of the high level one. We often accept it as "better
>     than nothing".

There's some overlap in that. There's some pretty decent functions in
zsh-completions and a few in zsh that are not. One or two ought perhaps
to be relegated.

>   • Some completions are for projects with a very small userbase. Do we really
>     want to bloat the zsh repo, and user installations with hundreds of
>     compdefs they don't use ?

I have vague memories of this having come up as a question on the lists
in the past with the conclusion being that we'd be generous in accepting
such functions. They don't tend to be large and it is notable that few
distributions go to the effort of stripping functions intended for other
operating systems. 

A small userbase is perhaps a strong criteria for suggesting that the
function should be contributed upstream....

>   • My opinion about including completions directly in the upstream project is
>     mixed. Either the project already does it, and this is fine. But in my

I agree with these points. Going forward, we should perhaps decide that
on a case-by-case basis.

> So the main points are:

>   • release cycle different than zsh's one
>   • newbie/occasional/one time contributor friendly thanks to:
>       □ github
>       □ lower quality standard

I'd add to that the unclear status of zsh-completions. Is it to be an
incubator, an additional repository for obscure commands, both or the
repository of all completions?

> Here are some ideas to improve things:

>   • Get some core zsh contributors involved in zsh-completions

I'd be happy to help with reviewing new contributions.

>   • Split completions from the main zsh repo to another one, with:

I can't say I'm entirely enthusiastic about this. It was sort-of before
my time but there are good reasons why they were included in the first
place. I'd be interested to hear if anyone else has any views.

We'd have to start worrying about backward compatibility with old releases of
zsh, virtually precluding the use of any new features. Most zsh users want the
completion functions to be there. I really value being able to go to something
like Solaris, RHEL or macOS and the functions are just there even if a tiny bit
dated.

>     This idea of having a "zsh-contrib" repo actually applies to more than just
>     completions: prompts, ZLE widgets, etc.

oh-my-zsh and other projects already occupy that niche.

>   • (slightly unrelated) Improve the tooling around completions:

>      1. Maybe define some standard metadatas that can be included in compdefs,
>         and that can be exploited by other tooling (eg: version of program)

I prefer the metadata to be outside of the functions so that it can be
updated without cluttering the version control for the functions with
non-functional changes. It also allows the metadata to be in one place
even if functions are not all in the same repository.

>      2. I could see your project used to build a permanent webapp/dashboard,
>         with the status for each command (does it have a completion ? where ?
>         for which version ?).

Yes, if someone wants to build on the repository I have in that or some other
way, then that would be great.

>      3. Work on tooling to lint and unit test completions (eg to invoke a
>         compdef programatically, and return matches in an array). I have some
>         unpushed stuff I started working on for zsh-completions, to run basic
>         checks on pull requests.

Some sort of lint tool is a good idea. I'd be inclined to gradually collect
regular expressions into a perl script as I think of them but am open to any
better ideas someone may have on how to do this.

> I realize that's a lot of points, but I hope it helps moving things forward :)

Thanks again for taking the time to think about it and respond.

Oliver