Skip to content

refactor(core): extract shared sanitizeUnicodeText from session-name/… - #2190

Merged
Astro-Han merged 1 commit into
apache:mainfrom
chinawch007:refactor/extract-sanitize-unicode-text
Aug 5, 2026
Merged

refactor(core): extract shared sanitizeUnicodeText from session-name/…#2190
Astro-Han merged 1 commit into
apache:mainfrom
chinawch007:refactor/extract-sanitize-unicode-text

Conversation

@chinawch007

Copy link
Copy Markdown
Contributor

Summary

  • foreign-session.ts:sanitizeForeignText duplicated the session-name.ts unicode pipeline and the two had drifted — session-name was missing 8 code points that foreign-session (and its own FOREIGN_UNSAFE_CHARS id guard) already covered: U+061C, U+200E, U+200F (bidi marks) and U+2060–U+2064 (invisible format chars).

  • I extracted the pipeline into a shared pure helper sanitizeUnicodeText(text, {maxCodePoints, truncatedSuffix}) in a new packages/core/src/text-sanitize.ts leaf module, which now holds the three character-class regexes as the single source of truth. The character classes take the union — the stricter foreign-session coverage — so session-name picks up the 8 previously-missing code points as a pure coverage gain (they never appear in legitimate CJK/emoji text, so no false positives).

  • Because the two callers disagree on truncation behavior and both are pinned by existing tests, I parameterized the suffix: foreign-session appends … (the default), while session-name passes '' to preserve its silent-cap behavior. The helper boundary is clean-and-truncate only; the type guard, empty-string policy, and Result-vs-string return shape stay with each caller, so both export signatures are unchanged and no downstream caller needs edits.

  • foreign-session.ts:sanitizeForeignText is now a thin wrapper around the helper, and session-name.ts:normalizeUserSessionName calls it with truncatedSuffix: '' (net −59 lines). I also exported sanitizeUnicodeText from index.ts and added one session-name test case locking in the newly-covered bidi marks and invisible operators so they can't silently drift back out of sync. FOREIGN_UNSAFE_CHARS and isSafeForeignId (the verbatim-rendered id guard) are a separate concern and left untouched.

Refs: #1404

Verification

  • npm run format:check
  • npm run lint
  • npm run typecheck (packages/core) — passed, 0 errors
  • npm --workspace @maka/core run build — passed
  • npm --workspace @maka/core run test:dist -- 782 passed

…foreign-session (apache#1404)

foreign-session.ts:sanitizeForeignText duplicated the session-name.ts
unicode pipeline and the two had drifted: session-name was missing 8
code points foreign-session (and its own FOREIGN_UNSAFE_CHARS id guard)
already covered — U+061C/200E/200F (bidi marks) and U+2060-2064
(invisible format chars). The control-char classes were written
differently but are byte-equivalent (not real drift).

Extract the pipeline into a shared pure helper sanitizeUnicodeText in a
new text-sanitize.ts leaf module:

  - char classes = union (the stricter foreign coverage)
  - truncatedSuffix parameter: foreign appends '…' (default),
    session-name passes '' to preserve its silent-cap behavior (its
    test asserts the capped length is exactly 80)
  - helper boundary = clean+truncate only; type guard, empty-string
    policy, and Result-vs-string return stay with each caller, so
    signatures are unchanged and no downstream caller needs edits

session-name gets a pure coverage gain (the 8 chars never appear in
legitimate CJK/emoji text). Add one session-name test case locking in
the newly-covered bidi marks and invisible operators so they can't
silently drift back out of sync. Existing tests stay green.

FOREIGN_UNSAFE_CHARS / isSafeForeignId (verbatim id guard) are out of
scope and untouched.
@Astro-Han

Copy link
Copy Markdown
Contributor

Thanks for this — I verified the extraction at character level against the pre-PR pipelines: the foreign call site is byte-identical (same regex sets, same ordering, same '…' suffix, same non-string handling) across a 60+ input battery, and the session-name site differs on exactly the 8 documented code points (ALM/LRM/RLM → space, U+2060–U+2064 → removed) — matching the commit message precisely, with the new test cases failing on the old code as they should. The scope is right too: UNSAFE_REVIEW_CHARACTER and the other sanitize-ish code are deliberately different guards, not candidates for unification. All 33 tests pass on the PR sources.

Three optional notes:

  • The "one source of truth" claim for FOREIGN_UNSAFE_CHARS now spans two files (the sanitizer's classes live in text-sanitize.ts); nothing enforces that the id-guard set stays equal to the sanitizer union. A small contract test (every code point the sanitizer accepts is rejected by the guard) would pin it.
  • U+FEFF and U+200C removal through the shared function are untested (pre-existing gap, worth two lines in the new union-coverage test).
  • The session-name change is user-visible (marks render as spaces) — it's pinned by the new test, just worth an explicit mention in the PR body/release notes.

Nothing blocking — happy to approve.

@Astro-Han
Astro-Han merged commit 317d2d5 into apache:main Aug 5, 2026
20 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants