Re: 'patch -Rp0<!$' loses the < in expansion

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

On Aug 16, 12:46pm, Daniel Shahaf wrote:
} Subject: 'patch -Rp0<!$' loses the < in expansion
}
} [[[
} % echo >foo
} % : -Rp0<!$
} : -Rp0foo
} % echo >foo
} % : -Rp0 <!$
} : -Rp0 <foo
} ]]]
} 
} This reproduces using 4.3.10 (me) and HEAD (Frank Terbeck).

It reproduces back to 4.2.0 at least.

} Shouldn't the < be retained?

Presumably.

What's going on here is that the lexer is looking ahead to discover
whether "<" is introducing a numeric glob instead of a redirection.  
It actually tracks back and forth across "<f" a couple of times before
deciding it's safe to move ahead to "o".  Right up until the point
where Src/hist.c:ihwaddc('o') is called, the history line (chline in
the C code) looks fine; somewhere in processing "o" it backtracks too
far and overwrites "<f" with "fo".

But only in the history, not in the lexer state; so the redirection
is correctly processed and except for a garbled history entry it all
works fine.

Here's the first spot where hptr backs up to the "<":

(gdb) where
#0  ihungetc (c=60) at ../../zsh-4.0/Src/hist.c:785
#1  0x080871cb in gettokstr (c=60, sub=0) at ../../zsh-4.0/Src/lex.c:1435
#2  0x0808652e in gettok () at ../../zsh-4.0/Src/lex.c:1001
#3  0x08085344 in zshlex () at ../../zsh-4.0/Src/lex.c:395
#4  0x080a3620 in par_simple (complex=0xbff8a45c, nr=0)
    at ../../zsh-4.0/Src/parse.c:1678
#5  0x080a15ed in par_cmd (complex=0xbff8a45c) at ../../zsh-4.0/Src/parse.c:879
#6  0x080a104b in par_pline (complex=0xbff8a45c)
    at ../../zsh-4.0/Src/parse.c:728
#7  0x080a0ff8 in par_sublist2 (complex=0xbff8a45c)
    at ../../zsh-4.0/Src/parse.c:709
#8  0x080a0e8b in par_sublist (complex=0xbff8a488)
    at ../../zsh-4.0/Src/parse.c:664
#9  0x080a0910 in par_event () at ../../zsh-4.0/Src/parse.c:477
#10 0x080a0886 in parse_event () at ../../zsh-4.0/Src/parse.c:454
#11 0x0807aa68 in loop (toplevel=1, justonce=0) at ../../zsh-4.0/Src/init.c:132
#12 0x0807da85 in zsh_main (argc=2, argv=0xbff8a5d4)
    at ../../zsh-4.0/Src/init.c:1528

It appears to go forward from there OK, but then backs up again here:

(gdb) where
#0  0x08076d57 in ihwend () at ../../zsh-4.0/Src/hist.c:1362
#1  0x08087def in exalias () at ../../zsh-4.0/Src/lex.c:1801
#2  0x08085357 in zshlex () at ../../zsh-4.0/Src/lex.c:395
#3  0x080a3620 in par_simple (complex=0xbff8a45c, nr=0)
    at ../../zsh-4.0/Src/parse.c:1678
#4  0x080a15ed in par_cmd (complex=0xbff8a45c) at ../../zsh-4.0/Src/parse.c:879
#5  0x080a104b in par_pline (complex=0xbff8a45c)
    at ../../zsh-4.0/Src/parse.c:728
#6  0x080a0ff8 in par_sublist2 (complex=0xbff8a45c)
    at ../../zsh-4.0/Src/parse.c:709
#7  0x080a0e8b in par_sublist (complex=0xbff8a488)
    at ../../zsh-4.0/Src/parse.c:664
#8  0x080a0910 in par_event () at ../../zsh-4.0/Src/parse.c:477
#9  0x080a0886 in parse_event () at ../../zsh-4.0/Src/parse.c:454
#10 0x0807aa68 in loop (toplevel=1, justonce=0) at ../../zsh-4.0/Src/init.c:132
#11 0x0807da85 in zsh_main (argc=2, argv=0xbff8a5d4)
    at ../../zsh-4.0/Src/init.c:1528

That's this:

1358                if (hwgetword > -1) {
1359                    /* We want to reuse the current word position */
1360                    chwordpos = hwgetword;
1361                    /* Start from where previous word ended, if possible */
1362                    hptr = chline + chwords[chwordpos ? chwordpos - 1 : 0];
1363                }

This makes me suspect that hwgetword was not incremented at some place
where it should have been, but I don't know where -- or whether that is
really the problem.