260913.1 · 8c45e3b4485db63a055f12892adc478af5a6fb3c
fix(vt): let the segmenter decide clustering, not Go's unicode tables (#1170)
* fix(vt): ask the segmenter about clustering instead of Go's unicode tables
putRune decided whether a rune could extend a grapheme cluster by testing it against unicode.M plus a few hand-listed ranges. Go's tables are Unicode 15; the uax29 segmenter used two lines later is newer, so 199 runes that do cluster after an ASCII base were split into their own cell.
Cluster width had the same problem: it took the widest rune in the cluster and then patched up regional indicators and U+FE0F by hand. displaywidth measures the cluster directly.
Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
* perf(vt): skip the segmenter for runes that cannot join a cluster
Handing every non-ascii rune to the segmenter cost about 10% in putRune. Only 2619 runes above U+007F can extend a cluster, so a bitmap with one bit per 64 runes rules the rest out in two instructions, and the segmenter still decides the cases that matter.
BenchmarkPutRuneCurrent, go1.26.3 darwin/arm64, n=10, vs main: width -8.7%, wide -8.0%, line -17.5%, mixed32 -13.9%, ascii unchanged.
The table is derived from the segmenter rather than from Go's unicode tables, and TestGraphemeJoinerBlocks re-derives it, so a uax29 upgrade cannot leave it stale.
Co-Authored-By: claude-opus-5 <noreply@anthropic.com>
---------
Co-authored-by: Dylan Pulver <dylanpulver@users.noreply.github.com> Co-authored-by: claude-opus-5 <noreply@anthropic.com>
reading declarations
Changes to vt/graphemetable.go. All files
@@ -0,0 +1,41 @@+// Copyright 2026 The TCell Authors+//+// Licensed under the Apache License, Version 2.0 (the "License");+// you may not use this file except in compliance with the License.+// You may obtain a copy of the License at+//+// http://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing, software+// distributed under the License is distributed on an "AS IS" BASIS,+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+// See the License for the specific language governing permissions and+// limitations under the License.++package vt++// graphemeJoinerBlocks has one bit per 64 runes, set if any rune in that+// block can extend a grapheme cluster. A clear bit means the segmenter+// cannot join anything in the block, so putRune may skip it. Only 2619 of+// the 1113984 runes above U+007F can join, so nearly every bit is clear.+//+// The table is derived from the segmenter, not from Go's unicode tables,+// and TestGraphemeJoinerBlocks re-derives it to catch drift.+var graphemeJoinerBlocks = [272]uint64{+ 0: 0xfffffffffbc43000,+ 1: 0x0089ff15f0002007,+ 2: 0x00a8000000000009,+ 3: 0x0000000000000005,+ 10: 0x00008ffd0e000000,+ 15: 0x4100100000000000,+ 16: 0x6c30090000002880,+ 17: 0x387527b117cffbff,+ 19: 0x0000000000020000,+ 22: 0xe000180000000010,+ 27: 0x0004000000000000,+ 28: 0x3000000000000000,+ 29: 0x0000070000000260,+ 30: 0x0000002808880c15,+ 31: 0x0000000000008000,+ 224: 0x00000000000000f3,+}