Zsh Mailing List Archive
Messages sorted by:
Reverse Date,
Date,
Thread,
Author
Re: [PATCH] zle_refresh multibyte fix
- X-seq: zsh-workers 20870
- From: Andrey Borzenkov <arvidjaar@xxxxxxxxxx>
- To: zsh-workers@xxxxxxxxxx
- Subject: Re: [PATCH] zle_refresh multibyte fix
- Date: Fri, 25 Feb 2005 22:17:45 +0300
- In-reply-to: <200502231457.j1NEvfBI032390@xxxxxxxxxxxxxx>
- Mailing-list: contact zsh-workers-help@xxxxxxxxxx; run by ezmlm
- References: <200502231727.58923.arvidjaar@xxxxxxxxxx> <200502231457.j1NEvfBI032390@xxxxxxxxxxxxxx>
On Wednesday 23 February 2005 17:57, Peter Stephenson wrote:
>
> > Actually I find wc stuff very easy and suitable for using as internal
> > representation in zsh core. But this is separate topic.
>
> Apart from the inefficiency of extending every byte that comes into the
> shell into (typically) a four-byte integer, we can't rely on input and
> output bytes being a valid wide character in the current locale at all.
> I think the shell has to handle arbitrary strings of bytes without
> mutilating them. Consider, for example:
>
> # Pass secret byte to my utility
> my_utility $'\xff'
>
> (or any other string you like, the only point being that it isn't a
> valid multibyte character string). I don't see why we should
> arbitrarily decide that doesn't work because it doesn't convert to a
> wide character. It will simply break far too many things.
>
I do not have easy answer, but what would be semantic of
- regexps ([[:print:]] et al.)?
- $foo[n,m] for scalar?
- Upper/Lower conversion?
- comparison (collating)?
apparently at least bash treats things byte-oriented:
bash-3.00$ echo к$(zsh -c "echo $'\xff\xff\xff'") | xxd
0000000: d0ba ffff ff0a ......
bash-3.00$ echo к$(zsh -c "echo $'\xff\xff\xff'") | { read foo; case "$foo"
in ???? ) echo yes;; * ) echo no; esac; }
no
but
bash-3.00$ zsh -c "echo $'\xff\xff\xff'" | { read foo; case "$foo" in ??? )
echo yes;; * ) echo no; esac; }
yes
and has definite problem editing non-ASCII input line (actually the same as
zsh had - cursor position is wrong).
But it is hardly acceptable for zsh. It depends (completion code in the first
place) too much on correct character handling.
OTOH does mb -> wc -> mb preserves original byte sequence? It is true for
UTF-8 (at least in current revision), but I am not sure that it is true for
arbitrary encoding.
-andrey
Messages sorted by:
Reverse Date,
Date,
Thread,
Author