Re: zsh 4.3.0-dev-2

[Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>]

William Scott wrote:
> Compiles fine on OS X v. 10.4.3, Xcode 2.2 (with gcc 4.0.1).
> 
> How do I test to see if myltibyte is indeed working?

That's a good point.  I suppose this could go into the FAQ, so please
correct any bits I've got wrong.

Multibyte support is compiled in if the line

  #define MULTIBYTE_SUPPORT 1

appears in config.h.  If you don't have the source, you could try
running

(bindkey -m)

and see if you get a warning message about doing this with multibyte
enabled.  (The parentheses protect you from the results.)

To test it, you need an appropriate environment:

(1) a locale with multibyte characters.  Unless you already know that
you need a different character set, look for a variant of your usual
locale with UTF-8 in it.  Nowadays it's quite possible $LANG is already
set to a UTF-8 locale.  For example, mine is en_GB.UTF-8; en_US.UTF-8 is
an obvious one.  zsh will complete locales on assignment to the variable
LANG, so

export LANG=^D

will show possible values.  A reasonable modern Unix distribution should
let you set this up consistently for your desktop environment, but
exporting the variable, then starting a suitable terminal emulator is
enough to get things going.

(2) a corresponding font.  If you use an old-style X font with lots of
dashes, you need one with iso10646-1 at the end (not, for example,
iso8859-1).  If your terminal emulator allows you to pick fonts from a
menu, it's probably smart enough to match the character set to the
locale and you can forget about this.

(3) a terminal emulator that handles multibyte mode.  I've had mixed
results with xterm, possibly out of ignorance.  gnome-terminal is the
best I've tried but I haven't searched too far.  (This is the sort of
thing Apple will almost certainly have got right in whatever they
provide.)

Once zsh is running in this environment, you need to be able to enter
multibyte characters.

If your keyboard provides such characters, don't use the "bindkey -m"
option, which treats the high bit in 8-bit characters as if it were an
escape character.  A shell with multibyte enabled will warn if you set
the option.  You can remove all meta bindings with:

bindkey -L | grep '\\M' | 
  sed -e '/^bindkey \(-R \|\)"\\M/s/[^ ]*$/undefined-key/' |
  while read -r line; do eval $line; done

(that's the simplest I could come up with; the shell doesn't provide a
direct antidote), but that doesn't seem to undo the effect completely,
so there may be a bug lurking.

There are two zle widgets provided with 4.3 to allow you to enter
characters not on the keyboard; load them in the normal way with

autoload -Uz insert-unicode-char insert-composed-char
zle -N insert-unicode-char
zle -N insert-composed-char

Bind them with bindkey as appropriate.  They're described in the
zshcontrib manual page.

insert-unicode-char takes the Unicode value for the character as
a hex number (execute the widget before and after the hex number).

insert-composed-char is followed by a more memorable two-character
mnemonic described in RFC 1345, http://www.faqs.org/rfcs/rfc1345.html,
for various accented characters, non-Latin European and related
character sets, and special characters such as punctuation and
mathematical symbols.  For example, try cH and see if you get a playing
card heart.  Not all of RFC 1345 is implemented but it's easy to extend
the function.

Ideally, editing with multibyte characters should behave identically to
editing with ASCII, even if the characters have double width.  There's
one bug in cd listings reported by Mikael Magnusson in zsh-users/9741
that hasn't been looked at yet, but mostly it seems to be working OK.

It's taken me six years until I can use the Euro character that came
on my keyboard directly in zsh, but it's quite satisfying...

-- 
Peter Stephenson <p.w.stephenson@xxxxxxxxxxxx>
Web page still at http://www.pwstephenson.fsnet.co.uk/