Re: GH:zsh-users/zsh-completions.

[Julien Nicoulaud <julien.nicoulaud@xxxxxxxxx>]

Hi Oliver,

Thanks for reaching out to me, those are some valid points and I have been
aware of the issues with this repo for a long time, but could not really
find the time to address it, sorry about that. I added zsh-workers in copy,
I hope you don't mind.

zsh-completions started eight years ago now, with zsh-syntax-highlighting.
Those were just some personal experimentations, but at some points both
started to get a bit a popularity, so I decided to try to make it
"community" projects and moved it to an organization. Both projects
continued evolving from this point and got many contributions, and that is
pretty much it, there is no agenda.

As a side note, I named the organization "zsh-users" and I realized after a
while this was a poor choice, as it sounds too "official". Many users get
confused and think this is part of the zsh project (especially with the git
mirror of the zsh repo). I am open to suggestions regarding this, and if
the zsh team wants me to rename the org, I will do it.

Anyway, a few thoughts about zsh-completions:

   - I believe the release rythm of zsh (~once a year) is a bit too slow
   for completions. We do ~monthly releases, and I think most users importing
   it via a "plugin manager" actually directly use the master branch, which is
   fine IMHO for completions. By the time a compdef lands in a zsh release, it
   can be already obsolete or need bugfixes. I think this is the biggest point
   for having compdefs in a separate repo than the shell itself.
   - I believe a significant part of the contributions we had were because
   the project is hosted on github. It attracts an audience of contributors
   that are not familiar with the "classic way" (mailing list, IRC, patches,
   and different process for a each project), and it makes it very easy for
   "drive-by" contributions and learning by example, which fits well to
   completions. It also easy to review/provide feedback and iterate on patches
   (eg: https://github.com/zsh-users/zsh-completions/pull/512)
   - The initial idea was indeed for zsh-completions to be an "incubator",
   and contribute the compdefs on the mailing-list. The previous point also
   applies to me, it asks me more work to extract a compdef from the repo,
   create a patch against zsh, send it to the mailing list, and follow up. I
   do not spend a lot of time on this project, and I quickly realized that was
   not something I would be able to do on a regular basis, so I did not do it
   at all.
   - If I send a compdef to zsh, what happens then ? We have to wait a
   least for the next release of zsh to delete it from zsh-completions. But
   then, not all distros update to the latest release of zsh, or the compdef
   could have evolved in zsh-completions in the meanwhile, so I can see this
   could be become quite a headache to manage (for users as well).
   - zsh-completions has a lower quality standard than the completions in
   zsh itself. For instance, some compdefs are partial, some compdefs use
   awk/grep/sed/tr instead of the shell builtins, some compdefs use the low
   level API (compadd, etc) instead of the high level one. We often accept it
   as "better than nothing".
   - Some completions are for projects with a very small userbase. Do we
   really want to bloat the zsh repo, and user installations with hundreds of
   compdefs they don't use ?
   - I do not have a strong opinion about licensing. I used this rule
   because we got many pull requests from people just copy-pasting someone
   else's work without any attribution. I think your proposition is good, we
   can use the Zsh license as the default one.
   - My opinion about including completions directly in the upstream
   project is mixed. Either the project already does it, and this is fine. But
   in my experience, when they do not by themselves, contributing a compdef
   often results in endless discussions in bug trackers and poor
   maintainance... IMHO, to write a completion, the knowledge of zsh is more
   important than the knowledge of the command itself, so it feels natural to
   me that completions are written by "zsh experts" rather than "project XXXX
   experts". Another point against completions in upstream project is that it
   makes work a bit harder for packaging (for instance, there is no real clean
   way to handle this in python's setup.py, so each project ends with a more
   or less custom solution).

So the main points are:

   - release cycle different than zsh's one
   - newbie/occasional/one time contributor friendly thanks to:
      - github
      - lower quality standard

Here are some ideas to improve things:

   - Get some core zsh contributors involved in zsh-completions


   - Split completions from the main zsh repo to another one, with:


   1. frequent, regular releases
      2. some clear guidelines for contributing compdefs, and for becoming
      a patch reviewer
      3. github contributing workflow (either by hosting it on github, or
      auto-posting patches from a github mirror, etc)

This idea of having a "zsh-contrib" repo actually applies to more than just
completions: prompts, ZLE widgets, etc.


   - (slightly unrelated) Improve the tooling around completions:


   1. Maybe define some standard metadatas that can be included in
      compdefs, and that can be exploited by other tooling (eg: version of
      program)
      2. I could see your project used to build a permanent
      webapp/dashboard, with the status for each command (does it have a
      completion ? where ? for which version ?).
      3. Work on tooling to lint and unit test completions (eg to invoke a
      compdef programatically, and return matches in an array). I have some
      unpushed stuff I started working on for zsh-completions, to run basic
      checks on pull requests.

I realize that's a lot of points, but I hope it helps moving things forward
:)

Cheers,
Julien

2017-10-14 11:37 GMT+02:00 Oliver Kiddle <okiddle@xxxxxxxxxxx>:

> I'm one of the people who regularly maintain the completion functions
> distributed with zsh. I've been aware of your function collection on
> github for a while and I think it would be worth giving some thoughts to
> how we can best improve the completion situation for zsh as a whole.
>
> Right at the start of README.md is the assertion that "scripts are meant
> to be contributed to the Zsh project when stable enough". As far as I
> can tell there hasn't been much, or even any, flow of functions to zsh.
>
> I'll start by pointing out something which is perhaps a minor nitpick:
> licensing. You encourage people to include a license which is good but
> if eventual inclusion in zsh is an aim then why not encourage the use
> of the zsh license. Above any arguments about copyleft vs. permissive
> licences, I always prefer to keep things simple by sticking to whatever
> licence a project is currently using. I know Peter has accepted some
> functions with alternate licenses but I'm not fond it. It's also sort
> of ugly having a license at the top of every file. Why not nominate
> the zsh licence as the default license and include a LICENSE file for
> it. Then simply state that where a function deviates from it that it
> should include its own license. If the intention is to include it in the
> upstream project then their license should be preferred.
>
> One of the problems when people contribute a function, either to zsh,
> zsh-users/zsh-completions, or some other project is that they often view
> it as a job done and never bother to come back with improvements or
> fixes. An advantage of keeping them in the author's own github repo is
> that they might be more inclined to maintain it. The disadvantage is
> that it is harder for users to find, especially if there are multiple
> forks. Perhaps something like git submodules or subtree could be used to
> link them in from a master repository?
>
> I've experimented with contributing some functions upstream over the
> past year. Sometimes it works great and the function gets updated by the
> people who know the upstream command best. All too often it gets little
> attention or my pull requests sit ignored. A staging repository such as
> zsh-completions could be good as a temporary home for these.
>
> I'm also concerned about how much feedback people get. Or more
> specifically encouraging a one-time committer to become more involved
> while also improving the quality of each newly submitted function by
> making a few iterations right at the beginning. We've not always done a
> great job on zsh-workers but the people best placed to give feedback are
> there.
>
> Did you create zsh-completions in the first place or did you
> inherit it? If so, why? - as opposed to sending contributions to the
> mailing list.
>
> If you're not aware of it, I'll just mention
>   https://github.com/okapia/zsh-helpoutput
> I had been keeping the output of --help for commands in CVS going back
> years. The idea is to automate detection of changes in upstream
> commands. Things had got fairly bad but I've been trying to get the
> functions included in zsh into better shape.
>
> I'd be interested in any ideas you may have on how we can do things
> better.
>
> Oliver
>