Re: PATCH: parse from even deeper in hell

[Peter Stephenson <p.stephenson@xxxxxxxxxxx>]

Apparently this message from Mikael didn't go to the list...

I've shortened the quotation.

I'll reply to it separately.

pws

On Sun, Feb 22, 2015 at 7:26 PM, Peter Stephenson
<p.w.stephenson@xxxxxxxxxxxx> wrote:
> On Fri, 20 Feb 2015 11:12:39 +0100
> Mikael Magnusson <mikachu@xxxxxxxxx> wrote:
>> > The question is where to put this in on history read.  I think it's
>> > going to affect non-lexical history, too, but the error on reading won't
>> > be flagged up.
>>
>> I don't think so, unmetafy() doesn't care about the table. And as I
>> checked earlier, both the old and new version of the string in my
>> history file is unmetafied to the correct UTF-8 string. The 'only'
>> problem is that the lexer is looking at some bytes before it's
>> unmetafied and some stuff that should have been metafied to avoid
>> being parsed as tokens, isn't, because they weren't special in the old
>> version. That's why I think running unmetafy before lexing is
>> needed... And if the lexer wants metafied text then we'd just have to
>> metafy it again right away.
>
> See if this fixes the problems, then.

I actually thought the patch in 34587 had fixed it, turns out I just
lost my --enable-zsh-debug flag. This patch doesn't fix it either
though, and I get a different set of errors every time I open a
terminal now, so that's fun. Which is to say it's all the same
wordsplit error, but it's printed for a different set of lines. Some
of them are the same though.

I noticed another thing from these errors too, they're printed also
when I exit zsh. There's not much point in lexing the history at
write-out time, is there?

I tried playing around with the code a bit. The thing that looks
suspicious to me is
    if (*ptr != Meta && imeta(*ptr)) {
Shouldn't they check ptr[0] and ptr[1] or so? I tried this,
    if ((ptr == lineptr || ptr[-1] != Meta) && imeta(ptr[0])) {
(in both places) but it didn't improve matters.

I tried the following instead of the for+if:
    unmetafy(lineptr, NULL);
    lineptr = metafy(lineptr, -1, META_USEHEAP);
and it gets rid of the errors, but it also causes insert-last-word to
do nothing. So maybe my whole theory is wrong.

-- 
Mikael Magnusson