Re: Re: variable expansion and "word"s (fwd)

[Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx>]

Discussion about how ${...} behaves with respect to quotes.  The direction
being chosen here may disagree with what zsh implements.  (I think it's
insane that "${var-"*} would ever have been syntactically sensible, but
oh well.)


---------- Forwarded message ----------
Date: Wed, 3 Mar 2010 10:41:01 -0500
From: David Korn <dgk@xxxxxxxxxxxxxxxx>
To: austin-group-l@xxxxxxxxxxxxx
Subject: Re: Re: variable expansion and "word"s
Resent-Date: 3 Mar 2010 15:41:38 -0000
Resent-From: austin-group-l@xxxxxxxxxxxxx
Resent-To: austin-group-l@xxxxxxxxxxxxx

Subject: Re: Re: variable expansion and "word"s
--------

> I'm working on a new proposal for this section, but I've hit a
> stumbling block over single-quotes.  There are two possible
> behaviours (remember this is the case where the whole expansion
> is within double-quotes):
>
> 1. Single-quotes are not special.  In this case, there should
> be no need to disallow unmatched single-quotes.  (Unless they
> are within nested double-quotes, but my proposal will say the
> behaviour is unspecified if there are nested double-quotes.)
>
> 2. Single-quotes are special.  In this case they must match,
> but since they are special they must also be removed during
> quote removal.
>
> I believe all the modern shells exhibit one or other of these
> behaviours with one exception: bash.  In some situations it
> treats single-quotes as special but it doesn't remove them
> during quote removal.  E.g. here the single-quotes quote
> the } inside them but they are not removed:
>
> $ printf '%s\n' "${IFS+a'}'b}"
> a'}'b
>
> The choices for updating the standard are either to allow both
> behaviours 1 and 2 or to require behaviour 1.  Since bash will
> need to change either way, my inclination is that we should
> require behaviour 1.
>
> --
> Geoff Clare <g.clare@xxxxxxxxxxxxx>
> The Open Group, Thames Tower, Station Road, Reading, RG1 1LX, England
>
>

I don't think that either of these are correct.

First of all there are two issues:
1.	How is the closing brace found when tokenizing.
	I think that this should be independent of whether the ${} is inside
	"" or not.  Rule 2 should be used and braces must nest unless quoted
	or escaped.
2.	How are quote characters treated when the expansion occurs.
	I think that in this case +, - = and ? have to be treated differently
	than %, %%, # and ##.
	 %, %%, # and ## should follow rule 2.
The other operators need to be treated differently for backward compatibility.

In the Bourne Shell,
	"${var op word}"
was treated as if it was
	"${var}" op "word"
where op determined whether word was was used or skipped.  In other words
implied double quotes was placed around word.  Bourne did not require matching
quotes inside ${} (unlike the standard), so that
	"${var-"*}
would be used to get either
	"${var}" or *
with file name generation performed on *.

Therefore, for the operators + - ? and =, word should be treated with
the double quote rules except that \ can be used to esacpe a brace
character that is not a closing brace.

For the example you gave,
	printf '%s\n' "${IFS+a'}'b}"
the Bourne shell would give
	a''b}

Since the first } would terminate the ${} and the 'b} would just be
part of the quoted string.

ksh88 and ksh93 are compatible with Bourne shell.

It uses rule 2 for parsing so that it thinks that the last } terminates
${...}, but when it goes to expand, it finds the first brace and uses
that as the terminator.

My recommendation is to adopt rule 1 as far as tokenizing is concerned
and for the operators + - = ?, leave it unspecified as to whether
' is special or not.

David Korn
dgk@xxxxxxxxxxxxxxxx