Re: MAX_ARRLEN

[Peter Stephenson <Peter.Stephenson@xxxxxxx>]

On Tue, 24 Apr 2012 12:45:23 -0700
Bart Schaefer <schaefer@xxxxxxxxxxxxxxxx> wrote:

> On Apr 24,  2:37pm, Peter Stephenson wrote:
> } 
> } As something to do now, I'd be tempted either to "#if 0" the code until
> } someone can come up with a replacement that is demonstrably useful, or
> } implement $ZSH_MAX_ARRAY_LENGTH and initialise it to 0 (no limit),
> } applying it at the current definitely non-optimal location.  Either
> } option at least gives us something basic usable, which the current code
> } isn't really.  Anything beyond that still seems to be somewhat
> } ill-defined and I'd like finally to have something non-broken ASAP.
> 
> I'm OK with just removing the check entirely.

I think I'll do that, pending any fully developed proposal for doing it
better.

> (I think the strange number for the limit was based on
> the assumption that you were creating a mostly empty array. If we used
> linked lists [ala bash] this would never come up.)

The trade off with linked lists is in favour of pointer arrays if the large
arrays you're creating aren't going to be sparse, though, which was the
case I ran up against this week.

I've changed references to "parameter expansion" to "parameter
substitution" below, since that's more standard.

===================================================================
RCS file: /cvsroot/zsh/zsh/NEWS,v
retrieving revision 1.54
diff -p -u -r1.54 NEWS
--- NEWS	15 Apr 2012 19:35:29 -0000	1.54
+++ NEWS	25 Apr 2012 08:57:51 -0000
@@ -52,31 +52,38 @@ Expansion (parameters, globbing, etc.) a
   This is useful for expanding paths with many variable components as
   commonly found in software development.
 
-- Parameter expansion has the ${NAME:OFFSET} and ${NAME:OFFSET:LENGTH}
+- Parameter substitution has the ${NAME:OFFSET} and ${NAME:OFFSET:LENGTH}
   syntax for compatibility with other shells (and zero-based indexing
   is used to enhance compatibility).  LENGTH may be negative to count
   from the end.
 
-- The parameter expansion flag (D) abbreviates directories in parameters
+- The arbitrary limit on parameter subscripts (262144) has been removed.
+  As it was not configurable and tested in an inconvenient place it
+  was deemed preferable to remove it completely.  The limit was originally
+  introduced to prevent accidental creation of a large parameter array
+  by typos that generated assignments along the lines of "12345678=0".
+  The general advice is not to do that.
+
+- The parameter substitution flag (D) abbreviates directories in parameters
   using the familiar ~ form.
 
-- The parameter expansion flag (g) can take delimited arguments o, e and
+- The parameter substitution flag (g) can take delimited arguments o, e and
   c to provide echo- and print-style expansion: (g::) provides basic
   echo-style expansion; (g:e:) provides the extended capabilities of
   print; (g:o:) provides octal escapes without a leading zero; (g:c:)
   additionally expands "^c" style control characters as for bindkey.
   Options may be combined, e.g. (g:eoc:).
 
-- The parameter expansion flag (m) indicates that string lengths used
+- The parameter substitution flag (m) indicates that string lengths used
   calculated by the (l) and (r) flags or the # operator should take
   account of the printing width of characters in multibyte mode, whether
   0, 1 or more.  (mm) causes printing characters to count as 1 and
   non-printing chracters to count as 0.
 
-- The parameter expansion flag (q-) picks the most minimal way of
+- The parameter substitution flag (q-) picks the most minimal way of
   quoting the parameter words, to make the result as readable as possible.
 
-- The parameter expansion flag (Z), a variant of (z), takes arguments
+- The parameter substitution flag (Z), a variant of (z), takes arguments
   describing how to split a variable using shell syntax: (Z:c:) parses
   comments as strings (the default is not to treat comment characters
   specially); (Z:C:) parses comments and strips them; (Z:n:) treats
Index: Src/params.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/params.c,v
retrieving revision 1.181
diff -p -u -r1.181 params.c
--- Src/params.c	22 Apr 2012 18:10:43 -0000	1.181
+++ Src/params.c	25 Apr 2012 08:57:52 -0000
@@ -1905,6 +1905,18 @@ fetchvalue(Value v, char **pptr, int bra
     if (!bracks && *s)
 	return NULL;
     *pptr = s;
+#if 0
+    /*
+     * Check for large subscripts that might be erroneous.
+     * This code is too gross in several ways:
+     * - the limit is completely arbitrary
+     * - the test vetoes operations on existing arrays
+     * - it's not at all clear a general test on large arrays of
+     *   this kind is any use.
+     *
+     * Until someone comes up with workable replacement code it's
+     * therefore commented out.
+     */
     if (v->start > MAX_ARRLEN) {
 	zerr("subscript too %s: %d", "big", v->start + !isset(KSHARRAYS));
 	return NULL;
@@ -1921,6 +1933,7 @@ fetchvalue(Value v, char **pptr, int bra
 	zerr("subscript too %s: %d", "small", v->end);
 	return NULL;
     }
+#endif
     return v;
 }
 
-- 
Peter Stephenson <pws@xxxxxxx>            Software Engineer
Tel: +44 (0)1223 692070                   Cambridge Silicon Radio Limited
Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, UK


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog