Zsh Mailing List Archive
Messages sorted by:
Reverse Date,
Date,
Thread,
Author
Re: mb_metacharlenconv vs. tokens
- X-seq: zsh-workers 22769
- From: Peter Stephenson <pws@xxxxxxx>
- To: zsh-workers@xxxxxxxxxx
- Subject: Re: mb_metacharlenconv vs. tokens
- Date: Tue, 26 Sep 2006 10:20:04 +0100
- In-reply-to: <200609252146.00249.arvidjaar@xxxxxxxxxx>
- Mailing-list: contact zsh-workers-help@xxxxxxxxxx; run by ezmlm
- References: <200609252146.00249.arvidjaar@xxxxxxxxxx>
Andrey Borzenkov wrote:
> Apparently mb_metacharlenconv gets passed tokenized string (how is it called
> properly in zsh lingua?) It means that the fact length(any-token-char) == 1
> is actually side effect of mbrtowc failing miserably and mb_metacharlenconv
> returning fallback 1 (at least using UTF-8).
>
> Should not it untokenize character first? If yes, I will provide a fix as
> part of larger patch; if no, I fail to see how it works then.
It's a bug if it's getting a tokenized character. It should have been
untokenized at some point in the sequence leading to the call, since
mb_metacharlenconv() should already be dealing with a printable
string---by implication, we're looking at its width or length in
characters because we're about to do something to it which is
appropriate to a printable string. However, this being zsh, the exact
point at which untokenization should be done is not necessarily obvious.
This is particularly true if we're using the string as a pattern.
I had a case like this in substitution recently:
2006-09-12 Peter Stephenson <pws@xxxxxxx>
* 22689: Src/subst.c, Test/D04parameter.ztst: untokenize
strings for substitution in cases like
${${~:-*}//(#m)*/$MATCH=$MATCH}. The pattern code tried
to metafy the tokens, which caused chaos.
We're a victim of the fact that metafied strings are used both to
protect tokens and to protect embedded NULs; it tends to hide the logic
indicating that tokens are still around when they shouldn't be.
--
Peter Stephenson <pws@xxxxxxx> Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
To access the latest news from CSR copy this link into a web browser: http://www.csr.com/email_sig.php
Messages sorted by:
Reverse Date,
Date,
Thread,
Author