Re: Encoding bug?

["Benjamin R. Haskell" <zsh@xxxxxxxxxx>]

On Fri, 7 Sep 2012, ☈king wrote:

> In v5.0.0 (revision 361e171), if I do:
>
> % echo ♔
> % r
>
> I get Mojibake.
>
> If I instead do:
>
> % echo ♔
> % fc
>
> I also get a case of the 'baks.
>
> My locale settings are no different than from when I ran an older zsh:
> LANG=en_US.utf8
> LC_CTYPE="en_US.utf8"
> LC_NUMERIC="en_US.utf8"
> LC_TIME="en_US.utf8"
> LC_COLLATE="en_US.utf8"
> LC_MONETARY="en_US.utf8"
> LC_MESSAGES="en_US.utf8"
> LC_PAPER="en_US.utf8"
> LC_NAME="en_US.utf8"
> LC_ADDRESS="en_US.utf8"
> LC_TELEPHONE="en_US.utf8"
> LC_MEASUREMENT="en_US.utf8"
> LC_IDENTIFICATION="en_US.utf8"
> LC_ALL=en_US.utf8
>
>
> I did talk to one other user on #zsh that is using 5.0.0 and didn't 
> experience the problem, and one who did. I have no clue what the 
> difference between the odd man out might be.

Not sure where something's not being encoded properly, but I get the 
same results here under 4.3.12 patch 1.5346 w/ LANG/LC_* set to 
en_US.UTF-8.  Also fails on git tag zsh-4.3.10.  So, it's not a new 
issue, AFAICT.

♔ = \U2654 = UTF-8: 0xe2 0x99 0x94

In my HISTORYFILE it ends up being encoded as:

0xe2 0x83 0xb9 0x83 0xb4

If I run:

$ echo ♔
$ r

and then arrow-up, it ends up displaying as: echo <20f9><83><b4> (which 
jives with the HISTORYFILE encoding, but seems notable inasmuch as zsh 
is 'aware' that the characters are different).

In the process of git-bisecting, but I won't be able to finish til 
tomorrow sometime (probably).

--
Best,
Ben