[PATCH] compat: give conjoining Hangul jamo zero Unicode 9 width

[Link Kim <link@xxxxxxx>]

From: Hangil Kim <uoneway@xxxxxxxxx>

With --enable-unicode9, wcwidth9() assigns width 1 to medial and final
Hangul jamo.  An NFD syllable consequently occupies more columns in ZLE's
model than it does in the terminal.  For example, the five code points
U+1103 U+1162 U+1107 U+1169 U+11AB occupy four terminal columns, but zsh
counts seven.  Recalling and clearing such a history entry repeatedly
moves the input into the prompt and leaves stale characters behind.

Return zero for positive widths in the Hangul medial/final jamo ranges,
including Jamo Extended-B.  Preserve negative widths for unassigned code
points and retain the existing widths of leading and compatibility jamo.

Add Unicode 9 regression tests for NFC/NFD padding, assigned jamo range
boundaries, and the non-printable classification of reserved Extended-B
code points.  Gate these table-specific expectations on --enable-unicode9
rather than prescribing the widths supplied by an arbitrary system libc.
---

Based on upstream master 7e57394d42e20160158364929c7bf68f7969f72b.

Minimal non-interactive reproducer, after configuring with
--enable-multibyte --enable-unicode9 and building Src/zsh:

  LC_ALL=en_US.UTF-8 ./Src/zsh -f <<'EOF'
  setopt multibyte
  s=$'\u1103\u1162\u1107\u1169\u11ab'
  p=${(ml:8::.:)s}
  print ${#${p%%$s}}
  EOF

Expected: 4 padding columns.  Before the patch: 1.

For an interactive reproduction, start zsh -f in a UTF-8 locale, enable
combiningchars, and run:

  print -s -- $'echo "/tmp/\u1103\u1162\u1107\u1169\u11ab.xlsx"'

Recall and clear that history entry with Up/Down repeatedly.  With the
unpatched Unicode 9 build, the input drifts three columns into the prompt
on each cycle.  The patched build remains aligned.

Validation on macOS 27.2 arm64:

- Added D07multibyte cases reproduce the width errors before the fix.
- make TESTNUM=D07 check passes with --enable-unicode9.
- The same suite passes with --disable-unicode9; the three new tests
  are skipped because they specifically exercise the built-in table.
- An isolated real ZLE/PTY replay through Herdr 0.9.3's VT core passes
  24/24 cursor checks and preserves all 3/3 history command byte strings.
- Direct u9_wcwidth checks confirm the reserved Extended-B code points
  U+D7C7, U+D7CA, U+D7FC, and U+D7FF still return -1.

Please Cc me on replies.

 Src/compat.c           | 14 ++++++++---
 Test/D07multibyte.ztst | 56 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 4 deletions(-)

diff --git a/Src/compat.c b/Src/compat.c
index 918d98e..38b3f0a 100644
--- a/Src/compat.c
+++ b/Src/compat.c
@@ -759,10 +759,16 @@ strtoul(nptr, endptr, base)
 int
 u9_wcwidth(wchar_t ucs)
 {
-  int w = wcwidth9(ucs);
-  if (w < -1)
-    return 1;
-  return w;
+    int w = wcwidth9(ucs);
+
+    /* Medial and final Hangul jamo add no columns to a syllable.
+     * Preserve negative widths for unassigned code points. */
+    if (w > 0 && ((ucs >= 0x1160 && ucs <= 0x11ff) ||
+		  (ucs >= 0xd7b0 && ucs <= 0xd7ff)))
+	return 0;
+    if (w < -1)
+	return 1;
+    return w;
 }

 /**/
diff --git a/Test/D07multibyte.ztst b/Test/D07multibyte.ztst
index 28cb9d9..43ecc63 100644
--- a/Test/D07multibyte.ztst
+++ b/Test/D07multibyte.ztst
@@ -349,6 +349,62 @@
 0:Multibyte characters in printf widths
 > főo

+  [[ $(<$ZTST_testdir/../config.h) == *'#define ENABLE_UNICODE9 1'* ]] ||
+    ZTST_skip='requires --enable-unicode9'
+  () {
+    local hangul padded
+    for hangul in $'\ub300\ubcf8' $'\u1103\u1162\u1107\u1169\u11ab'; do
+      padded=${(ml:8::.:)hangul}
+      print ${#${padded%%$hangul}}
+      padded=${(mr:8::.:)hangul}
+      print ${#${padded#$hangul}}
+    done
+  }
+0:NFC and decomposed Hangul have equal widths with --enable-unicode9
+>4
+>4
+>4
+>4
+
+  [[ $(<$ZTST_testdir/../config.h) == *'#define ENABLE_UNICODE9 1'* ]] ||
+    ZTST_skip='requires --enable-unicode9'
+  () {
+    local jamo padded
+    for jamo in $'\u1100' $'\u115f' $'\u1160' $'\u1161' \
+                $'\u11a8' $'\u11ff' $'\ud7b0' $'\ud7c6' \
+                $'\ud7cb' $'\ud7fb' $'\u3131' $'\uffa1'; do
+      padded=${(ml:4::.:)jamo}
+      print $(( 4 - ${#${padded%%$jamo}} ))
+    done
+  }
+0:Only medial and final Hangul jamo have zero display width
+>2
+>2
+>0
+>0
+>0
+>0
+>0
+>0
+>0
+>0
+>2
+>1
+
+  [[ $(<$ZTST_testdir/../config.h) == *'#define ENABLE_UNICODE9 1'* ]] ||
+    ZTST_skip='requires --enable-unicode9'
+  () {
+    local unassigned
+    for unassigned in $'\ud7c7' $'\ud7ca' $'\ud7fc' $'\ud7ff'; do
+      print -r -- ${(q+)unassigned}
+    done
+  }
+0:Unassigned Hangul Jamo Extended-B code points remain non-printable
+>$'\ud7c7'
+>$'\ud7ca'
+>$'\ud7fc'
+>$'\ud7ff'
+
 # TODO?: POSIX requires that printf should always compute width and
 # precision of '%s' conversion in bytes, while zsh computes them in
 # characters if multi-byte locale is in use.
-- 
2.54.0 (Apple Git-157)