Re: The q parameter expansion flag does not support Unicode

[Mikael Magnusson <mikachu@xxxxxxxxx>]

On Sun, Jul 12, 2026 at 6:53 PM Vincent Lefevre <vincent@xxxxxxxxxx> wrote:
>
> On 2026-07-12 13:59:46 +0100, Stephane Chazelas wrote:
> > On 2026-07-12 11:04, Vincent Lefevre wrote:
> > [...]
> > >   var=$(printf "A\bB\u2060C")
> > >   printf "%s\n" ${(q)var}
> > >   printf "%s\n" $var:q
> > >
> > > the \b is quoted as $'\b', but the U+2060 WORD JOINER character
> > > is not quoted though it is not printable. I would have expected
> > > it to be quoted as $'\u2060'.
> >
> > [[ $'\u2060' = [[:print:]] ]] returns true for me in the en_GB.UTF-8 locale
> > of a Debian system (with glibc 2.42).
>
> Indeed, but there is something strange with zsh, because if I type
>
>   printf "%s\n" $var
>
> then [Tab] just after "$var", I get
>
>   printf "%s\n" A$'\b'B<2060>C
>
> with <2060> in reverse video.

This is how all zerowidth characters (and other special characters)
are shown on the commandline when the combiningchars option is not
enabled.

> Moreover, in xterm, in the output of
>
>   printf "%s\n" $var:q
>
> when I copy "BC", the U+2060 character is dropped, but this seems
> to be a bug in xterm as both rxvt and GNOME Terminal are fine.
>
> In any case, there is a visibility issue.

The problem is that the character in question is a format character
(in the Cf class). These return 0 for wcwidth and 1 for iswprint, so
there's not much we can do in terms of posix library calls to
distinguish them. We could query a custom library like utf8proc, and
apparently glibc also has an extension that lets you use iswctype(x,
wctype("combining")) to determine if it's a Cf or other combining
character.

-- 
Mikael Magnusson