Re: Bug#419832: zsh: expanding non-ASCII filenames with <TAB>

[Clint Adams <schizo@xxxxxxxxxx>]

On Fri, Aug 17, 2007 at 05:05:02AM -0400, Alan Curry wrote:
> >> In the following demonstration, the first <TAB> keypress inserted the $'\300'
> >> for me. The second <TAB> keypress, typed immediately after the asterisk,
> >> should expand the glob into $'\300' also, but instead it just erases the
> >> asterisk, replacing it with nothing at all. If Return is pressed after the
> >> tab, the cat is executed with no arguments and reads from the tty.

> >> Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

> >Non-ASCII characters don't exist in the C locale; maybe you want to pick
> >a better one.

> That's a pretty lame brush-off.

> My locale is set correctly (to be precise, it is unset correctly; none of
> those environment variables are set). It represents the type of output I want
> to get from all programs that recognize locales: text in English if possible,
> and traditional sort order, not that new-fangled chaotic LANG=en order, where
> ls hides your Makefile in the middle of all your lowercase source files! (Why
> do you think they made make(1) recognize Makefiles with a capital M? Because
> it belongs at the start of the listing, that's why.)

> If you think this behavior is justified, for what am I being punished? Using
> the default ("C") locale? It accurately describes what language I can read.
> Having a file that is not a valid sequence of characters in that locale?
> Maybe I should go file bug reports on all the programs that allow me to
> create a file with such a name. That will be a lot of bug reports.

> Or maybe we could admit that regardless of one's preferred locale, it is
> inevitable that one will occasionally obtain files whose names are not valid
> character strings in that locale. It would be nice if our tools would not
> choke on those, would it not?

> The $'\300' notation is a vast improvement over what older zsh versions did,
> just dump the wacky bytes directly to the terminal. The current version
> already automatically inserts $'\300' when completing; I only suggest that it
> behave identically when expanding.

> Expanding a glob to an empty list, when in fact it matched something, surely
> can't be considered acceptable behavior. Even worse if it matched several
> things and only one of them had a nasty byte and got omitted, you might not
> notice and then go ahead and act on the wrong set of files.
> 
> Come on.

There is apparently a bit of inconsistency here.  expand-or-complete and
expand-word will eat the asterisk, but _expand_word won't.

What's worse is that if you `touch a b$'\300' c` in an empty directory,
cat *<TAB> will only expand to "a b".