Re: Let's finish this new completion stuff

["Bart Schaefer" <schaefer@xxxxxxxxxxxxxxxx>]

Preliminary remark:  I think it's premature to be talking about "finishing"
this stuff.  There are, what, three? people actually trying this out right
now?  And one of them is the implementor of 90% of the underlying code.
This is not the way to do a usability study.  I imagine we're going to be
tweaking things for a long while.

On Feb 19,  4:21pm, Sven Wischnowsky wrote:
} Subject: Let's finish this new completion stuff
}
} - Using the positional parameters as we do now has some drawbacks. We
}   could make the completion widget store them in another array but
}   then the modifying tests (see below) would still change `argv'

Let me see if I understand this.  Referring to the tests that modify the
word list as a side-effect:

As of the current date, those tests (a) expect to find the previous word
list in the positionals, and (b) always stuff the new word list into the
positional parameters of whatever function calls them.  So if you forget
to pass the previous word list to a sub-function as its arguments, none
of the tests work as expected.

Have I got that right?

That was not what I envisioned when I suggested passing the word list in
the positionals, but then, I didn't envision tests with side-effects.

I was about to suggest that we put the words in an associative array with
elements named "0", "1", etc., but it occurred to me that you wouldn't be
able to do arithmetic in the subscripts, or slice assignment.

}   unless we make the test get the name of the array to work upon as an 
}   argument.

If we don't use the positionals, we should use another special variable
with a hardwired name, as PWS suggested.

Fortunately, because of some magic zsh does, $words[0] would always be
the command name, although with ksharrays NOT set, so would $words[1].

I just had a really wild idea for how we could handle -string, -iprefix,
and so on, with one or more special associative arrays.  Hmm, again.

} - I would like to remove the parameter `COMMAND' since this has
}   already become yet another way to describe the context (`COMMAND' is 
}   a bit of a misnomer currently). When using an array to report the
}   words on the line we could store it in this array as the first word

What do you do in cases where COMMAND presently doesn't contain a real
command name?

} - We could also put the `COMMAND' into the `CONTEXT' parameter which I 
}   would like to replace with a associative array with keys:

So whichever of these was uninteresting would simply not be set?

Thinking about that caused me to notice this bit of oddness:

zagzig% typeset -A foo
zagzig% echo ${+foo[x]}
1
zagzig% typeset -a bar
zagzig% echo ${+bar[x]}
0

That is, all fields of associative arrays currently appear to be set,
even when they aren't; this is not true of plain arrays.

Anyway, let's call this hypothetical associative array `compstate' for
purposes of discussion.

}   - `command'   if this is used as the replacement for `COMMAND'.
}   - `redirect'  for the string of the redirection operator
}   - `parameter' for the `value' and `subscript' contexts where it will 
}                 contain the name of the parameter
}   - `type'      as the replacement for the old `CONTEXT'

I like `context' better.  `Type' means too many different things.

}   - `quote'     if we want to do this, this could be set to `single',
}                 `double', or the empty string giving information about 
} 		the quotes the completion code thinks we are in

I suggest putting the actual quote character in there.  Then you don't
even have to test the value if all you want to do is close the quote;
you just append $compstate[quote] to whatever you're compadd-ing.

}   We may find other interesting things here and with an associative
}   array it would be easy to add new types of context information.

Yes.  Hmm, hmm, hmm.

Why not put everything that's not another array in here?

	CURRENT		-->	compstate[current]
	PREFIX		-->	compstate[prefix]
	IPREFIX		-->	compstate[ignored]
	SUFFIX		-->	compstate[suffix]
	NMATCHES	-->	compstate[nmatches]
	MATCHER		-->	compstate[matcher]

} - The example code currently uses the return value of the shell
}   functions or a parameter (in the new version I just sent) to decide
}   whether more completion definitions should be used. This can not be
}   combined with `compcall' which currently gives no information about
}   things like that.

I'm not sure exactly what you're asking about.

} The testing problem:
} 
} Some of the condition codes we currently have could easily be replaced
} by shell code so we should probably remove them. The question was/is
} with what we would replace them.

I have a radical suggestion:  Replace them with magic elements of the
compstate array.  A special associative predefined by zsh doesn't have
to be implemented by a simple hash table; it can do its own special
interpretation of the subscripts.

Then we can invent a whole lot of new syntax for the completion stuff,
and hide it where the lexer already skipparens() it.

} There seems to be support for doing this with a builtin [....]  That
} builtin would have several options (again):
} 
} - Options to control the special parameters. For the modifications we
}   could either add one option per modification (e.g. using the
}   characters known from `compctl -x': `-s <str>' and the like), or use 
}   (nearly) the syntax Bart sometimes used (although I don't know if
}   this was suggested syntax): `-m ignored-prefix <str>'.

Let's think about this a bit.  These are all easy:

[[ -nmatches x ]]	((compstate[nmatches] == x))
[[ -matcher x ]]	((compstate[matcher] == x))

[[ -word x str ]]	[[ $words[x] == 'str' ]]
[[ -mword x str ]]	[[ $words[x] == str ]]
[[ -current x str ]]	[[ $words[compstate[current]+offset] == 'str' ]]
[[ -mcurrent x str ]]	[[ $words[compstate[current]+offset] == str ]]

These aren't too bad:

[[ -prefix str ]]	[[ -z "${compstate[prefix]%%str*}" ]]
[[ -iprefix str ]]	[[ -z "${compstate[prefix]%%str*}" ]] &&
			    { compstate[ignored]="$compstate[ignored]str"
			      compstate[prefix]="${compstate[prefix]#str}" }

These are OK (but handling the negative numbers gets tricky, not shown):

[[ -words x ]]		(($#words == x))
[[ -words x y ]]	(($#words >= x && $#words <= y))

[[ -position x ]]	[[ $compstate[current] == x ]]
[[ -position x y ]]	((compstate[current] >= &&
			  compstate[current] <= y)) &&
			    words=($words[x,y])
			    # Can assignment to $words change $compstate?
			    # If not, this gets messy.

These are so messy I'm not going to try work them out on the fly here:

[[ -string str ]]
[[ -string x str ]]
[[ -class str ]]
[[ -class x pat ]]
[[ -after str ]]
[[ -mafter pat ]]
[[ -between str1 str2 ]]
[[ -mbetween pat1 pat2 ]]

So, let's invent a magic subscript for each of those last two groups.
Choosing `position' as an example, we can do:

    $compstate[position(x)]	if compstate[current] == x, expands
    				to x, else expands to nothing

    $compstate[position(x,y)]	if compstate[current] is between x
				and y, expands to x,y

Now, hey presto:

[[ -position x y ]]		words=($words[$compstate[position(x,y)]])

(That doesn't quite work because of the way tokenization happens in the
[] of $words[...], but you get the idea.)

An alternate idea is to require the (r) or (R) subscript flags to make
the magic syntax active, so you get something like:

    $compstate[(r)position(x)]	 if compstate[current] == x, expands
    				 to $words[x], else expands to nothing

    $compstate[(R)position(x,y)] if compstate[current] is between x
				 and y, expands to $words[x,y]

Similar things could be invented for the rest of the conditions, to make
it simple to have tests without the side-effects and then duplicate those
side-effects in shell code.

}   With these/this option(s) the builtin would set it's return value in 
}   a way that it can be used for testing, but that would only be a
}   `side effect'.

By the way, your doc for the new completion stuff doesn't mention any
of the $argv-modifying etc. side-effects of the condition codes (except
for -iprefix changing $IPREFIX), as far as I can tell.

}   Here we could also add a built-in replacement for the `compsave' and 
}   `compreset' aliases from the example code. Let's give it two options 
}   to save and restore the state of all the special parameters. It
}   would be the user's responsibility to make those calls symmetrical.

I'm still not thrilled with this.

} - Options to get information from the calling completion code,
}   e.g. things like `is there already a valid list', `will menu-
}   completion be used' (which would also test for automenu) and the
}   like.

I suggest making the existing list available in a parameter that can
be tested and modified directly; the sets.afn of that array can do the
equivalent of `compadd`.  For booleans like "are we menu-completing,"
see the AA idea above.

} - Options to say what should be done with the matches genereated,
}   e.g.: `list them', `don't list them', `use menu-completion', `insert 
}   it into the line', etc.

Don't those options belong on complist/compadd?  Or is it not possible
to have some of the matches listed and others not?  (That is, is it a
requirement that there be a single global switch for this stuff?)

By the way, again, one thing I haven't figured out how to do with the
new completion stuff is the equivalent of `compctl -X`.  How does one
add an explanation string, and when does it get shown?

On Feb 20,  4:01pm, Peter Stephenson wrote:
} Subject: Re: Let's finish this new completion stuff
}
} Another issue: ordering of pattern completions.  I think this can already
} be done quite naturally.

Your scheme sounds OK to me, though I reserve judgement until I see it
in action.

Now I have to run.  More long messages next weekend, maybe.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com