[PATCH] (slightly) improve "compset -q"

["Jun T." <takimoto-j@xxxxxxxxxxxxxxxxx>]

The patch below addresses two problems.

[1] the 1st hunk.

zsh% sh -c 'for ((<TAB>
_sh:1: maximum nested function level reached; increase FUNCNEST?

The string passed to the lexer is " for ((x", and the last "x" stands for
the cursor. After finding two tokens "for" and "((", ctxtlex() (line 1619)
returns LEXERR with tokstr="x". Then the execution exits from the 'do ... while'
loop by 'break' at line 1641. This means the block
   if (!got && !lexflags) {...}    (line 1667)
is not executed, and this seems to be causing the trouble.

In the patch below, the 'break' is simply removed. I *hope* this has no bad
side effects.

[2] 2nd and 3rd hunks; I think these are OK.

Original problem: users/22958: sh -c '. foo;<TAB>
Current master: workers/42180 (by dana)

dana's patch seems to work; my patch just fixes the problem more cleanly.

> 2017/12/28 16:47, dana <dana@xxxxxxx> wrote:
> 
> The problem seems related to the fact that set_comp_sep() (which is called via
> `compset -q` in _cmdstring) removes semi-colons and other meta-characters from
> compwords (the user-land words array), but doesn't decrement compcurrent (the
> user-land CURRENT integer) accordingly.

Yes. And what is causing this mismatch is the following:

A word is added to the array 'foo' only if there is a tokstr (line 1663).
But the counter 'i' is incremented for every token in the string (line 1680),
so it is not the "number of words before cursor" but the number of tokens
(including separators like SEMI).

The patch below fixes this by using countlinknodes(foo).
With this fix, I *think* compcurrent will never exceed the number of words,
but I added a DUPTS warning in case something still goes wrong.

# I'm preparing another patch for "compset -q", but it is far more subtle and will
# be posted separately, after the 1st hunk is found to be OK.


diff --git a/Src/Zle/compcore.c b/Src/Zle/compcore.c
index c3b971e0d..f733e0ee5 100644
--- a/Src/Zle/compcore.c
+++ b/Src/Zle/compcore.c
@@ -1638,7 +1638,7 @@ set_comp_sep(void)
 		    p[-1] = '\0';
 	    }
 	}
-	if (tok == ENDINPUT || tok == LEXERR)
+	if (tok == ENDINPUT)
 	    break;
 	if (tokstr && *tokstr) {
             for (p = tokstr; dq && *p; p++) {
@@ -1667,7 +1667,7 @@ set_comp_sep(void)
 	if (!got && !lexflags) {
 	    DPUTS(!p, "no current word in substr");
 	    got = 1;
-	    cur = i;
+	    cur = countlinknodes(foo) - 1;  /* cur is 0 offset */
 	    swb = wb - 1 - dq - sq - dolq;
 	    swe = we - 1 - dq - sq - dolq;
             sqq = lsq;
@@ -1902,7 +1902,10 @@ set_comp_sep(void)
 	    untokenize(p);
 	}
 	/* The current position shouldn't exceed the new word count */
-	compcurrent = cur + 1 > i ? i : cur + 1;
+	if ((compcurrent = cur + 1) > i) {
+	    DPUTS2(1, "compcurrent=%d > number_of_words=%d", compcurrent, i);
+	    compcurrent = i;
+	}
 	compwords[i] = NULL;
     }
     instring = ois;