Re: PATCH: parse from even deeper in hell

[Peter Stephenson <p.stephenson@xxxxxxxxxxx>]

On Mon, 23 Feb 2015 12:36:07 +0000
Peter Stephenson <p.stephenson@xxxxxxxxxxx> wrote:
> On Mon, 23 Feb 2015 12:35:51 +0100
> Mikael Magnusson <mikachu@xxxxxxxxx> wrote:
> > I figured out that when we assign lineptr after fiddling with it, we
> > also need to update start, it records the location of *lineptr on
> > entry to the function, and is used to calculate things later on. With
> > that addition, the unmetafy + metafy mostly works. insert-last-word
> > still gets "stuck" on words that came from the old metafication and
> > starts over from the end of history, leaving the old word in place.
> 
> What's the line this happens with?  Is it the same as the one that
> originally failed?

Yes, I see it is.

> I wonder if it's getting information by some other path, though it seems
> strange the last word wouldn't come from the bit we've apparently fixed
> up.

Ah, simply moving the hunk up seems to help this problem, at least the
one I'm seeing.

I've also implemented the suggestion of skipping the character
after any Meta found, though we need to check that's not '\0' since
this is has come from the big wide world.

Anything still broken?

diff --git a/Src/hist.c b/Src/hist.c
index acc4259..da79771 100644
--- a/Src/hist.c
+++ b/Src/hist.c
@@ -3372,47 +3372,44 @@ mod_export void
 histsplitwords(char *lineptr, short **wordsp, int *nwordsp, int *nwordposp,
 	       int uselex)
 {
-    int nwords = *nwordsp, nwordpos = 0;
+    int nwords = *nwordsp, nwordpos = 0, remeta = 0;
     short *words = *wordsp;
-    char *start = lineptr;
+    char *start, *ptr;
+
+    /*
+     * Handle the special case that we're reading from an
+     * old shell with fewer meta characters, so we need to
+     * metafy some more.  (It's not clear why the history
+     * file is metafied at all; some would say this is plain
+     * stupid.  But we're stuck with it now without some
+     * hairy workarounds for compatibility).
+     *
+     * This is rare so doesn't need to be that efficient; just
+     * allocate space off the heap.
+     *
+     * Note that our it's currently believed this all comes out in
+     * the wash in the non-uselex case owing to where unmetafication
+     * and metafication happen.
+     */
+    for (ptr = lineptr; *ptr; ptr++) {
+	if (*ptr == Meta && ptr[1])
+	    ptr++;
+	else if (imeta(*ptr)) {
+	    remeta = 1;
+	    break;
+	}
+    }
+    if (remeta) {
+	unmetafy(lineptr, &remeta);
+	lineptr = metafy(lineptr, remeta, META_USEHEAP);
+    }
+
+    start = lineptr;
 
     if (uselex) {
 	LinkList wordlist;
 	LinkNode wordnode;
-	int nwords_max, remeta = 0;
-	char *ptr;
-
-	/*
-	 * Handle the special case that we're reading from an
-	 * old shell with fewer meta characters, so we need to
-	 * metafy some more.  (It's not clear why the history
-	 * file is metafied at all; some would say this is plain
-	 * stupid.  But we're stuck with it now without some
-	 * hairy workarounds for compatibility).
-	 *
-	 * This is rare so doesn't need to be that efficient; just
-	 * allocate space off the heap.
-	 *
-	 * Note that our it's currently believed this all comes out in
-	 * the wash in the non-uselex case owing to where unmetafication
-	 * and metafication happen.
-	 */
-	for (ptr = lineptr; *ptr; ptr++) {
-	    if (*ptr != Meta && imeta(*ptr))
-		remeta++;
-	}
-	if (remeta) {
-	    char *ptr2, *line2;
-	    ptr2 = line2 = (char *)zhalloc((ptr - lineptr) + remeta + 1);
-	    for (ptr = lineptr; *ptr; ptr++) {
-		if (*ptr != Meta && imeta(*ptr)) {
-		    *ptr2++ = Meta;
-		    *ptr2++ = *ptr ^ 32;
-		} else
-		    *ptr2++ = *ptr;
-	    }
-	    lineptr = line2;
-	}
+	int nwords_max;
 
 	wordlist = bufferwords(NULL, lineptr, NULL,
 			       LEXFLAGS_COMMENTS_KEEP);