UTF-8 FAQs

[Clint Adams <clint@xxxxxxx>]

The number of questions I've been getting about zsh and UTF-8
have skyrocketed over the past couple of weeks.

Perhaps some of these belong in the zsh FAQ.  Corrections to my stock
answers are welcome.

Q: Does zsh support UTF-8?

A: zsh's built-in printf command supports "\u" and "\U" escapes
to output arbitrary Unicode characters.  ZLE (the Zsh Line Editor) has
no concept of character encodings, and is confused by multi-octet
encodings.

Q: Why doesn't zsh have proper UTF-8 support?

A: The code has not been written yet.

Q: What makes UTF-8 support difficult to implement?

A: In order to handle arbitrary encodings the correct way, significant
and intrusive changes must be made to the shell.

Q: Why can't zsh just use readline?

A: ZLE is not encapsulated from the rest of the shell.  Isolating it
such that it could be replaced by readline would be a significant
effort.  Furthermore, using readline would effect a significant loss of
features.

Q: How can I help implement UTF-8 support in zsh?

A: One approach would be to have ZLE use wide characters internally.
Character based widgets could then operate on a single wide character
instead of a single byte, and the proper display width can be calculated
with wcswidth().