[PATCH] Change documentation, dedication, loose ends

[dana <dana@xxxxxxx>]

I've gone through the history since 5.8 and updated the documentation
for all of the changes we missed before. Attaching the patch here just
to make sure i've correctly understood what they all do.

I've also attached a patch for the dedication we discussed off-list.

And, unposted here, i've added the new -s option to _fc.

Some other minor loose ends:

* In workers/47922 Daniel had suggested some changes to the documentation
  for the CASE_PATHS option Bart added; these changes were never merged or
  commented on. Are we satisfied with Bart's original documentation?

* The ${name:offset:length} expansion documentation was not updated with
  Jun's change from workers/49853. Does it need to be?

PS: Trying a different mail client, please let me know if it messes
the text up. Patches also included as attachments just in case

dana


From 200d3209e591839de391752d330575adb7ada8ea Mon Sep 17 00:00:00 2001
From: dana <dana@xxxxxxx>
Date: Sat, 2 Apr 2022 22:19:04 -0500
Subject: [PATCH 1/2] NEWS/README: Add missing change documentation for 5.9

This covers the following changes:

users/24971: ${(-)var} sorts on signed integers

47704: POSIX export and readonly ignore "-p" when parameter names also appear

47913: implement CASE_PATHS option to make NO_CASE_GLOB more sensible

48073: Add fc -s as POSIX way of rerunning command without starting editor

49307 with doc update: POSIX_TRAPS fix.

49528: allow multiple -D options to compadd

49561: add zformat -F option, similar to -f but ternary expressions check for
existence instead of doing math evaluation

49597: add a helper for completing numbers with unit suffixes and separate out
defaults, ranges and units in completion descriptions

49611 based on 49590 (Martijn Dekker): disable Inf and NaN in math expressions
for sh emulation

49646: allow colors in WATCHFMT with %F/%K

49694 + doc: Allow using empty STTY= to freeze tty for a single command

49853 + 49882/49883: make "${arr[*]:off}" compatible with ksh/bash
---
 NEWS   | 72 ++++++++++++++++++++++++++++++++++++++++++----------------
 README | 33 +++++++++++++++++++--------
 2 files changed, 76 insertions(+), 29 deletions(-)

diff --git a/NEWS b/NEWS
index 8441610b0..61ee32ef1 100644
--- a/NEWS
+++ b/NEWS
@@ -4,25 +4,8 @@ CHANGES FROM PREVIOUS VERSIONS OF ZSH
 
 Note also the list of incompatibilities in the README file.
 
-Changes since 5.8
------------------
-
-CVE-2021-45444: Some prompt expansion sequences, such as %F, support
-'arguments' which are themselves expanded in case they contain colour
-values, etc. This additional expansion would trigger PROMPT_SUBST
-evaluation, if enabled. This could be abused to execute code the user
-didn't expect. e.g., given a certain prompt configuration, an attacker
-could trick a user into executing arbitrary code by having them check
-out a Git branch with a specially crafted name.
-
-This is fixed in the shell itself by no longer performing PROMPT_SUBST
-evaluation on these prompt-expansion arguments.
-
-Users who are concerned about an exploit but unable to update their
-binaries may apply the partial work-around described in the file
-Etc/CVE-2021-45444-VCS_Info-workaround.patch included with the shell
-source. [ Reported by RyotaK <security@xxxxxxxxx>. Additional thanks to
-Marc Cornellà <hello@xxxxxxxxxxxxx>. ]
+Changes since 5.8.1
+-------------------
 
 When unsetting a hash element, the string enclosed in square brackets is
 interpreted literally after any normal command-line-argument expansions.
@@ -54,6 +37,9 @@ fractional seconds.
 The option CLOBBER_EMPTY was added to enable the overwrite behaviour
 of CLOBBER for empty files only. It is disabled by default.
 
+A (-) expansion flag was added. It works like (n) but correctly sorts
+negative numbers.
+
 The compinit function learnt a -w option to explain why compdump runs.
 When run without the -i or -u options and compaudit discovers security
 issues, answering "y" to the "Ignore insecure ..." prompt removes the
@@ -69,11 +55,37 @@ widgets.  This corresponds to long-standing behavior of other user ZLE
 widgets.  Use the _complete_debug widget to capture XTRACE output, or
 use "functions -T" to enable tracing of specific completion functions.
 
+The fc builtin learnt an -s option which is a POSIX equivalent to the
+`fc -e-` method of re-executing a command without invoking an editor.
+
+The option CASE_PATHS was added to control how NO_CASE_GLOB behaves.
+NO_CASE_GLOB + NO_CASE_PATHS is equivalent to the current NO_CASE_GLOB
+behaviour. NO_CASE_GLOB + CASE_PATHS treats only path components that
+contain globbing characters as case-insensitive; this behaviour may
+yield more predictable results on case-sensitive file systems.
+NO_CASE_PATHS is the default.
+
 With the new TYPESET_TO_UNSET option set, "typeset foo" leaves foo unset,
 in contrast to the default behavior which assigns foo="".  Any parameter
 attributes such as numeric type, sorting, and padding are retained until
 the parameter is explicitly unset or a conflicting value is assigned.
-This is similar to default behavior of bash and ksh.
+This is similar to default behavior of bash and ksh.  This option is
+disabled by default.
+
+The compadd builtin's -D option can now be specified more than once.
+
+The zsh/zutil module's zformat builtin learnt an -F option which behaves
+like -f except that ternary expressions check for existence instead of
+doing math evaluation.
+
+A _numbers helper function has been added to help completion functions
+complete numbers with unit suffixes, etc.
+
+The WATCHFMT parameter now supports colours via the %F and %K escapes.
+
+The STTY parameter can now be set to an empty string before running a
+command to automatically restore terminal settings after the command
+finishes.
 
 The "jobs" command and "$jobstates" and related parameters can report on
 parent shell jobs even in subshells.  This is a snapshot of the parent
@@ -81,6 +93,26 @@ state, frozen at the point the subshell started.  However, if a subshell
 starts its own background jobs, the parent state is discarded in order
 to report on those new jobs.
 
+Changes from 5.8 to 5.8.1
+-------------------------
+
+CVE-2021-45444: Some prompt expansion sequences, such as %F, support
+'arguments' which are themselves expanded in case they contain colour
+values, etc. This additional expansion would trigger PROMPT_SUBST
+evaluation, if enabled. This could be abused to execute code the user
+didn't expect. e.g., given a certain prompt configuration, an attacker
+could trick a user into executing arbitrary code by having them check
+out a Git branch with a specially crafted name.
+
+This is fixed in the shell itself by no longer performing PROMPT_SUBST
+evaluation on these prompt-expansion arguments.
+
+Users who are concerned about an exploit but unable to update their
+binaries may apply the partial work-around described in the file
+Etc/CVE-2021-45444-VCS_Info-workaround.patch included with the shell
+source. [ Reported by RyotaK <security@xxxxxxxxx>. Additional thanks to
+Marc Cornellà <hello@xxxxxxxxxxxxx>. ]
+
 Changes from 5.7.1-test-3 to 5.8
 --------------------------------
 
diff --git a/README b/README
index c27d6881a..21142e17c 100644
--- a/README
+++ b/README
@@ -5,11 +5,12 @@ THE Z SHELL (ZSH)
 Version
 -------
 
-This is version 5.8.1 of the shell.  This is a security and bugfix release.
+This is version 5.9 of the shell.  This is a security and feature release.
+There are several visible improvements since 5.8.1, as well as bug fixes.
 All zsh installations are encouraged to upgrade as soon as possible.
 
 Note in particular the changes highlighted under "Incompatibilities since
-5.8" below.  See NEWS for more information.
+5.8.1" below.  See NEWS for more information.
 
 Installing Zsh
 --------------
@@ -30,16 +31,13 @@ Zsh is a shell with lots of features.  For a list of some of these, see the
 file FEATURES, and for the latest changes see NEWS.  For more
 details, see the documentation.
 
-Incompatibilities since 5.8
----------------------------
+Incompatibilities since 5.8.1
+-----------------------------
 
 compinit: A "y" response to the "Ignore ... and continue?" prompt removes
 insecure elements from the set of completion functions, where previously
 it ignored the compaudit result and included all elements.
 
-PROMPT_SUBST expansion is no longer performed on arguments to prompt-
-expansion sequences such as %F.
-
 Build-time change: The default value of the --enable-gdbm configure
 argument has changed from "yes" to "no".  Thus, the zsh/db/gdbm module will
 not be built unless --enable-gdbm is passed explicitly.
@@ -105,11 +103,25 @@ emulate sh: When zsh emulates sh, the final command in a pipeline is now run in
 a subshell.  This differs from the behavior in the native (zsh) mode, but is
 consistent with most other sh implementations.
 
+The export and readonly builtins now ignore the -p option when there are
+operands given and POSIX_BUILTINS is enabled. This more closely matches the
+behaviour of bash and ksh.
+
 getopts now calculates OPTIND in a similar manner to other shells when the
 POSIX_BUILTINS option is enabled.
 
-Incompatibilities between 5.7.1 and 5.8
----------------------------------------
+Ignored-signal traps are now inherited by subshells when the POSIX_TRAPS
+option is enabled.
+
+emulate sh: Inf and NaN are now treated as parameter names in arithmetic
+context when zsh is emulating sh.
+
+The ${name:offset:length} expansion syntax now behaves more similarly to
+other shells in that the offset and length are applied as array indices
+prior to scalar conversion in e.g. "${*:0:2}".
+
+Incompatibilities between 5.7.1 and 5.8.1
+-----------------------------------------
 
 The history expansion !:1:t2 used to be interpreted such that the 2
 was a separate character added after the history expansion.  Now
@@ -140,6 +152,9 @@ changes made in the course of fixing CVE-2019-20044.  Please report this
 to the zsh-workers mailing list if your system is affected.  See NEWS for
 more.
 
+PROMPT_SUBST expansion is no longer performed on arguments to prompt-
+expansion sequences such as %F.
+
 Incompatibilities between 5.6.2 and 5.7.1
 -----------------------------------------
 
-- 
2.34.1


From 1ffc4b86c4d76d91d18661bcf12d0b306deb472a Mon Sep 17 00:00:00 2001
From: dana <dana@xxxxxxx>
Date: Sat, 2 Apr 2022 22:20:55 -0500
Subject: [PATCH 2/2] NEWS: Dedicate zsh 5.9 to Sven Guckes

---
 NEWS | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/NEWS b/NEWS
index 61ee32ef1..6c9112ad6 100644
--- a/NEWS
+++ b/NEWS
@@ -6,6 +6,11 @@ Note also the list of incompatibilities in the README file.
 
 Changes since 5.8.1
 -------------------
+zsh 5.9 is dedicated in memory of Sven Guckes, who was, amongst other
+things, a long-time zsh advocate.
+
+  https://linuxnews.de/2022/02/sven-guckes-verstorben/
+  https://groups.google.com/g/vim_announce/c/MJBKVd-xrEE/m/joVNaDgAAgAJ
 
 When unsetting a hash element, the string enclosed in square brackets is
 interpreted literally after any normal command-line-argument expansions.
-- 
2.34.1

From 200d3209e591839de391752d330575adb7ada8ea Mon Sep 17 00:00:00 2001
From: dana <dana@xxxxxxx>
Date: Sat, 2 Apr 2022 22:19:04 -0500
Subject: [PATCH 1/2] NEWS/README: Add missing change documentation for 5.9

This covers the following changes:

users/24971: ${(-)var} sorts on signed integers

47704: POSIX export and readonly ignore "-p" when parameter names also appear

47913: implement CASE_PATHS option to make NO_CASE_GLOB more sensible

48073: Add fc -s as POSIX way of rerunning command without starting editor

49307 with doc update: POSIX_TRAPS fix.

49528: allow multiple -D options to compadd

49561: add zformat -F option, similar to -f but ternary expressions check for
existence instead of doing math evaluation

49597: add a helper for completing numbers with unit suffixes and separate out
defaults, ranges and units in completion descriptions

49611 based on 49590 (Martijn Dekker): disable Inf and NaN in math expressions
for sh emulation

49646: allow colors in WATCHFMT with %F/%K

49694 + doc: Allow using empty STTY= to freeze tty for a single command

49853 + 49882/49883: make "${arr[*]:off}" compatible with ksh/bash
---
 NEWS   | 72 ++++++++++++++++++++++++++++++++++++++++++----------------
 README | 33 +++++++++++++++++++--------
 2 files changed, 76 insertions(+), 29 deletions(-)

diff --git a/NEWS b/NEWS
index 8441610b0..61ee32ef1 100644
--- a/NEWS
+++ b/NEWS
@@ -4,25 +4,8 @@ CHANGES FROM PREVIOUS VERSIONS OF ZSH
 
 Note also the list of incompatibilities in the README file.
 
-Changes since 5.8
------------------
-
-CVE-2021-45444: Some prompt expansion sequences, such as %F, support
-'arguments' which are themselves expanded in case they contain colour
-values, etc. This additional expansion would trigger PROMPT_SUBST
-evaluation, if enabled. This could be abused to execute code the user
-didn't expect. e.g., given a certain prompt configuration, an attacker
-could trick a user into executing arbitrary code by having them check
-out a Git branch with a specially crafted name.
-
-This is fixed in the shell itself by no longer performing PROMPT_SUBST
-evaluation on these prompt-expansion arguments.
-
-Users who are concerned about an exploit but unable to update their
-binaries may apply the partial work-around described in the file
-Etc/CVE-2021-45444-VCS_Info-workaround.patch included with the shell
-source. [ Reported by RyotaK <security@xxxxxxxxx>. Additional thanks to
-Marc Cornellà <hello@xxxxxxxxxxxxx>. ]
+Changes since 5.8.1
+-------------------
 
 When unsetting a hash element, the string enclosed in square brackets is
 interpreted literally after any normal command-line-argument expansions.
@@ -54,6 +37,9 @@ fractional seconds.
 The option CLOBBER_EMPTY was added to enable the overwrite behaviour
 of CLOBBER for empty files only. It is disabled by default.
 
+A (-) expansion flag was added. It works like (n) but correctly sorts
+negative numbers.
+
 The compinit function learnt a -w option to explain why compdump runs.
 When run without the -i or -u options and compaudit discovers security
 issues, answering "y" to the "Ignore insecure ..." prompt removes the
@@ -69,11 +55,37 @@ widgets.  This corresponds to long-standing behavior of other user ZLE
 widgets.  Use the _complete_debug widget to capture XTRACE output, or
 use "functions -T" to enable tracing of specific completion functions.
 
+The fc builtin learnt an -s option which is a POSIX equivalent to the
+`fc -e-` method of re-executing a command without invoking an editor.
+
+The option CASE_PATHS was added to control how NO_CASE_GLOB behaves.
+NO_CASE_GLOB + NO_CASE_PATHS is equivalent to the current NO_CASE_GLOB
+behaviour. NO_CASE_GLOB + CASE_PATHS treats only path components that
+contain globbing characters as case-insensitive; this behaviour may
+yield more predictable results on case-sensitive file systems.
+NO_CASE_PATHS is the default.
+
 With the new TYPESET_TO_UNSET option set, "typeset foo" leaves foo unset,
 in contrast to the default behavior which assigns foo="".  Any parameter
 attributes such as numeric type, sorting, and padding are retained until
 the parameter is explicitly unset or a conflicting value is assigned.
-This is similar to default behavior of bash and ksh.
+This is similar to default behavior of bash and ksh.  This option is
+disabled by default.
+
+The compadd builtin's -D option can now be specified more than once.
+
+The zsh/zutil module's zformat builtin learnt an -F option which behaves
+like -f except that ternary expressions check for existence instead of
+doing math evaluation.
+
+A _numbers helper function has been added to help completion functions
+complete numbers with unit suffixes, etc.
+
+The WATCHFMT parameter now supports colours via the %F and %K escapes.
+
+The STTY parameter can now be set to an empty string before running a
+command to automatically restore terminal settings after the command
+finishes.
 
 The "jobs" command and "$jobstates" and related parameters can report on
 parent shell jobs even in subshells.  This is a snapshot of the parent
@@ -81,6 +93,26 @@ state, frozen at the point the subshell started.  However, if a subshell
 starts its own background jobs, the parent state is discarded in order
 to report on those new jobs.
 
+Changes from 5.8 to 5.8.1
+-------------------------
+
+CVE-2021-45444: Some prompt expansion sequences, such as %F, support
+'arguments' which are themselves expanded in case they contain colour
+values, etc. This additional expansion would trigger PROMPT_SUBST
+evaluation, if enabled. This could be abused to execute code the user
+didn't expect. e.g., given a certain prompt configuration, an attacker
+could trick a user into executing arbitrary code by having them check
+out a Git branch with a specially crafted name.
+
+This is fixed in the shell itself by no longer performing PROMPT_SUBST
+evaluation on these prompt-expansion arguments.
+
+Users who are concerned about an exploit but unable to update their
+binaries may apply the partial work-around described in the file
+Etc/CVE-2021-45444-VCS_Info-workaround.patch included with the shell
+source. [ Reported by RyotaK <security@xxxxxxxxx>. Additional thanks to
+Marc Cornellà <hello@xxxxxxxxxxxxx>. ]
+
 Changes from 5.7.1-test-3 to 5.8
 --------------------------------
 
diff --git a/README b/README
index c27d6881a..21142e17c 100644
--- a/README
+++ b/README
@@ -5,11 +5,12 @@ THE Z SHELL (ZSH)
 Version
 -------
 
-This is version 5.8.1 of the shell.  This is a security and bugfix release.
+This is version 5.9 of the shell.  This is a security and feature release.
+There are several visible improvements since 5.8.1, as well as bug fixes.
 All zsh installations are encouraged to upgrade as soon as possible.
 
 Note in particular the changes highlighted under "Incompatibilities since
-5.8" below.  See NEWS for more information.
+5.8.1" below.  See NEWS for more information.
 
 Installing Zsh
 --------------
@@ -30,16 +31,13 @@ Zsh is a shell with lots of features.  For a list of some of these, see the
 file FEATURES, and for the latest changes see NEWS.  For more
 details, see the documentation.
 
-Incompatibilities since 5.8
----------------------------
+Incompatibilities since 5.8.1
+-----------------------------
 
 compinit: A "y" response to the "Ignore ... and continue?" prompt removes
 insecure elements from the set of completion functions, where previously
 it ignored the compaudit result and included all elements.
 
-PROMPT_SUBST expansion is no longer performed on arguments to prompt-
-expansion sequences such as %F.
-
 Build-time change: The default value of the --enable-gdbm configure
 argument has changed from "yes" to "no".  Thus, the zsh/db/gdbm module will
 not be built unless --enable-gdbm is passed explicitly.
@@ -105,11 +103,25 @@ emulate sh: When zsh emulates sh, the final command in a pipeline is now run in
 a subshell.  This differs from the behavior in the native (zsh) mode, but is
 consistent with most other sh implementations.
 
+The export and readonly builtins now ignore the -p option when there are
+operands given and POSIX_BUILTINS is enabled. This more closely matches the
+behaviour of bash and ksh.
+
 getopts now calculates OPTIND in a similar manner to other shells when the
 POSIX_BUILTINS option is enabled.
 
-Incompatibilities between 5.7.1 and 5.8
----------------------------------------
+Ignored-signal traps are now inherited by subshells when the POSIX_TRAPS
+option is enabled.
+
+emulate sh: Inf and NaN are now treated as parameter names in arithmetic
+context when zsh is emulating sh.
+
+The ${name:offset:length} expansion syntax now behaves more similarly to
+other shells in that the offset and length are applied as array indices
+prior to scalar conversion in e.g. "${*:0:2}".
+
+Incompatibilities between 5.7.1 and 5.8.1
+-----------------------------------------
 
 The history expansion !:1:t2 used to be interpreted such that the 2
 was a separate character added after the history expansion.  Now
@@ -140,6 +152,9 @@ changes made in the course of fixing CVE-2019-20044.  Please report this
 to the zsh-workers mailing list if your system is affected.  See NEWS for
 more.
 
+PROMPT_SUBST expansion is no longer performed on arguments to prompt-
+expansion sequences such as %F.
+
 Incompatibilities between 5.6.2 and 5.7.1
 -----------------------------------------
 
-- 
2.34.1

From 1ffc4b86c4d76d91d18661bcf12d0b306deb472a Mon Sep 17 00:00:00 2001
From: dana <dana@xxxxxxx>
Date: Sat, 2 Apr 2022 22:20:55 -0500
Subject: [PATCH 2/2] NEWS: Dedicate zsh 5.9 to Sven Guckes

---
 NEWS | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/NEWS b/NEWS
index 61ee32ef1..6c9112ad6 100644
--- a/NEWS
+++ b/NEWS
@@ -6,6 +6,11 @@ Note also the list of incompatibilities in the README file.
 
 Changes since 5.8.1
 -------------------
+zsh 5.9 is dedicated in memory of Sven Guckes, who was, amongst other
+things, a long-time zsh advocate.
+
+  https://linuxnews.de/2022/02/sven-guckes-verstorben/
+  https://groups.google.com/g/vim_announce/c/MJBKVd-xrEE/m/joVNaDgAAgAJ
 
 When unsetting a hash element, the string enclosed in square brackets is
 interpreted literally after any normal command-line-argument expansions.
-- 
2.34.1