Re: ZWJ paste from clipboard problem (unicode)

[Oliver Kiddle <opk@xxxxxxx>]

"Daniil Rozanov" wrote:
> As far as I know it is possible on some bunch of terminals. This
> possibility called "mode 2027", briefly described here:
>
> https://mitchellh.com/writing/grapheme-clusters-in-terminals
>
> So at least zsh can test terminal to know does it support grapheme
> clustering or not and work around this.

The problem for zsh remains that wcwidth is the libc interface that
provides a way to find a correspondence between wide characters (i.e.
unicode code points) and graphemes. There's a lot of limitations in
what libc provides. Most terminals also use wcwidth() so share the same
limitations meaning that behaviour is at least consistent.

So zsh would need to either depend on some additional external library
or we need to fetch tables of unicode code points and generate our own
tables to classify them. libICU is the long-standing C library for this
but it's a huge monster dating from a time when people expected 16-bits
to be enough. I have used libutf8proc before and my impression was
positive but I didn't think it covered this particular functionality.
However, foot is using it (utf8proc) for grapheme-clustering and the
licence (MIT "expat" variant) is compatible so perhaps it would be an
option. Or perhaps someone else is more knowledgeable on this than me
or has strong objections to adding a dependency given zsh currently has
nearly none.

It still needs someone to do the working of adding this. If someone is
motivated to do so, the areas of code that need changing might be fairly
easily identified by looking for where the COMBINING_CHARS option is
checked.

I'm aware of mode 2027 having come across it when looking at the more
general possibilities involved with terminal queries that subscribers to
-workers will have seen.

Is changing COMBINING_CHARS to be enabled by default (at least for zsh
emulation) something that might sensibly be changed for a future
(major?) release?

Oliver