Skip to content

regression: combined emojis displayed as separate emojis and some flags displayed as a black flag - #41441

Merged
dionisio-bot[bot] merged 1 commit into
developfrom
fix/emoji-zwj-and-tag-sequences-split
Jul 18, 2026
Merged

dionisio-bot[bot] merged 1 commit into
developfrom
fix/emoji-zwj-and-tag-sequences-split

Conversation

@abhinavkrin

@abhinavkrin abhinavkrin commented Jul 17, 2026 •

Copy link
Copy Markdown
Member

Proposed changes (including videos or screenshots)

The message parser was splitting multi-codepoint emoji sequences into multiple emojis:

  • ZWJ sequences anchored by a face emoji (😶‍🌫️, 😮‍💨, 😵‍💫) were displayed as two separate emojis. The grammar tried the standalone emoticon rule before the ZWJ-sequence branch, so the face was matched alone and the remainder of the sequence was left behind. Additionally, the emoticon range was not a valid ZWJ component, so a face could never anchor a sequence even when the branch was reached.
  • Tag-sequence flags (England 🏴󠁧󠁢󠁥󠁮󠁧󠁿, Scotland 🏴󠁧󠁢󠁳󠁣󠁴󠁿, Wales 🏴󠁧󠁢󠁷󠁬󠁳󠁿) were displayed as a plain black flag. The grammar had no rule for Unicode tag characters, so only the Black Flag base was matched and the invisible tag letters fell through as plain text. This was previously masked by emojione shipping its own assets for these flags and was exposed when it was removed (chore: remove emojione in favor of native emojis #39411).

Changes in packages/message-parser/src/grammar.pegjs:

  • Added a new UnicodeEmojiTagSequence rule (Black Flag + tag characters + Cancel Tag), tried first so the standalone black flag cannot preempt it.
  • Moved the ZWJ-sequence branch ahead of the standalone emoticon rule.
  • Added emoticons (with optional skin-tone modifier) as valid ZWJ components.

No existing rule's character ranges were narrowed, so anything that matched before still matches — the only behavioral change is that sequences previously producing two tokens now produce one.

Issue(s)

CORE-2311
CORE-2416

Steps to test or reproduce

  1. Send a message containing 😶‍🌫️, 😮‍💨 or 😵‍💫 — each should render as a single emoji instead of two.
  2. Send a message containing 🏴󠁧󠁢󠁥󠁮󠁧󠁿, 🏴󠁧󠁢󠁳󠁣󠁴󠁿 or 🏴󠁧󠁢󠁷󠁬󠁳󠁿 — the correct flag should render instead of a plain black flag.
  3. Verify no regressions: 🏴 (black flag), 🏴‍☠️, 🏳️‍🌈, 👨‍👩‍👧‍👦, 🇺🇸 and standalone emojis like 👍 and 😀 still render as before.

Further comments

CORE-2311 · CORE-2416

Review in cubic

Summary by CodeRabbit

  • Bug Fixes
    • Improved rendering of complex emoji sequences, including combined ZWJ emojis.
    • Corrected display of England, Scotland, and Wales flag emojis.
    • Improved handling of adjacent emoji sequences to prevent incorrect character grouping.

…iple emojis

Signed-off-by: Abhinav Kumar <abhinav@avitechlab.com>
@dionisio-bot

dionisio-bot Bot commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

Looks like this PR is ready to merge! 🎉
If you have any trouble, please check the PR guidelines

@changeset-bot

changeset-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f4333e2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@rocket.chat/message-parser Patch
@rocket.chat/gazzodown Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The message parser grammar now recognizes emoji tag sequences and emoticon-based ZWJ components. Tests cover flag variants, adjacent sequences, and mixed text, while a changeset records a patch release for the rendering fixes.

Changes

Emoji sequence parsing

Layer / File(s) Summary
Emoji sequence grammar support
packages/message-parser/src/grammar.pegjs
UnicodeEmoji recognizes tag sequences first, and ZWJ components now accept emoticons alongside existing components and modifiers.
Sequence validation and release metadata
packages/message-parser/tests/emoji.test.ts, .changeset/emoji-zwj-tag-sequences.md
Tests cover complex ZWJ and tag-based flag sequences, adjacency boundaries, and mixed text; changeset metadata records a patch release.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: type: bug

Suggested reviewers: ggazzo, martinschoeler

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main regression fixed: combined emojis and some flags rendering incorrectly.

Warning

Review ran into problems

🔥 Problems

Errors were encountered while retrieving linked issues.

Errors (2)
  • CORE-2311: Request failed with status code 401
  • CORE-2416: Request failed with status code 401

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/message-parser/src/grammar.pegjs (1)

797-808: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove unreachable branches.

The explicitly listed UnicodeEmojiEmoticon, UnicodeEmojiMiscellaneousSymbols, and UnicodeEmojiDingbats alternatives are now completely subsumed by the preceding ZWJ sequence branch (which naturally matches a single component when the * quantifier matches zero times).

Because PEG parsing checks alternatives in order, these subsumed branches will never be reached and can be safely removed to eliminate dead code and slightly optimize parsing performance.

♻️ Proposed refactor
 UnicodeEmoji
   = UnicodeEmojiTagSequence
   / $(
     (UnicodeEmojiZwjComponent [\u200D])*
     UnicodeEmojiZwjComponent
   )
-  / UnicodeEmojiEmoticon
   / UnicodeEmojiTransportAndMapSymbols
   / UnicodeEmojiMiscellaneousTechnical
-  / UnicodeEmojiMiscellaneousSymbols
-  / UnicodeEmojiDingbats
   / UnicodeEmojiFlags
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/message-parser/src/grammar.pegjs` around lines 797 - 808, Remove the
unreachable UnicodeEmojiEmoticon, UnicodeEmojiMiscellaneousSymbols, and
UnicodeEmojiDingbats alternatives from the emoji parsing rule. Keep the
preceding UnicodeEmojiZwjComponent sequence and the remaining distinct
alternatives unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/message-parser/src/grammar.pegjs`:
- Around line 797-808: Remove the unreachable UnicodeEmojiEmoticon,
UnicodeEmojiMiscellaneousSymbols, and UnicodeEmojiDingbats alternatives from the
emoji parsing rule. Keep the preceding UnicodeEmojiZwjComponent sequence and the
remaining distinct alternatives unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a8b90f9f-6305-4603-8ce3-188afd846b47

📥 Commits

Reviewing files that changed from the base of the PR and between 65a366e and f4333e2.

📒 Files selected for processing (3)
  • .changeset/emoji-zwj-tag-sequences.md
  • packages/message-parser/src/grammar.pegjs
  • packages/message-parser/tests/emoji.test.ts
📜 Review details
⏰ Context from checks skipped due to timeout. (5)
  • GitHub Check: 📦 Build Packages
  • GitHub Check: cubic · AI code reviewer
  • GitHub Check: Hacktron Security Check
  • GitHub Check: CodeQL-Build
  • GitHub Check: CodeQL-Build
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js}

📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)

**/*.{ts,tsx,js}: Write concise, technical TypeScript/JavaScript with accurate typing in Playwright tests
Avoid code comments in the implementation

Files:

  • packages/message-parser/tests/emoji.test.ts
🧠 Learnings (4)
📚 Learning: 2026-03-16T21:50:37.589Z
Learnt from: amitb0ra
Repo: RocketChat/Rocket.Chat PR: 39676
File: .changeset/migrate-users-register-openapi.md:3-3
Timestamp: 2026-03-16T21:50:37.589Z
Learning: For changes related to OpenAPI migrations in Rocket.Chat/OpenAPI, when removing endpoint types and validators from rocket.chat/rest-typings (e.g., UserRegisterParamsPOST, /v1/users.register) document this as a minor changeset (not breaking) per RocketChat/Rocket.Chat-Open-API#150 Rule 7. Note that the endpoint type is re-exposed via a module augmentation .d.ts in the consuming package (e.g., packages/web-ui-registration/src/users-register.d.ts). In reviews, ensure the changeset clearly states: this is a non-breaking change, the major version should not be bumped, and the changeset reflects a minor version bump. Do not treat this as a breaking change during OpenAPI migrations.

Applied to files:

  • .changeset/emoji-zwj-tag-sequences.md
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In the Rocket.Chat repository, do not reference Biome lint rules in code review feedback. Biome is not used even if biome.json exists; only reference Biome rules if there is explicit, project-wide usage documented. For TypeScript files, review lint implications without Biome guidance unless the project enables Biome rules.

Applied to files:

  • packages/message-parser/tests/emoji.test.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In this repository (RocketChat/Rocket.Chat), Biome lint rules are not used even if a biome.json exists. When reviewing TypeScript files (e.g., packages/ui-voip/src/providers/useMediaSession.ts), ensure lint suggestions do not reference Biome-specific rules. Rely on general ESLint/TypeScript lint rules and project conventions instead.

Applied to files:

  • packages/message-parser/tests/emoji.test.ts
📚 Learning: 2026-05-06T12:21:44.083Z
Learnt from: juliajforesti
Repo: RocketChat/Rocket.Chat PR: 40256
File: apps/meteor/client/components/CreateDiscussion/CreateDiscussion.tsx:121-149
Timestamp: 2026-05-06T12:21:44.083Z
Learning: Field wrappers in rocket.chat/fuselage-forms (Field, FieldLabel, FieldRow, FieldError, FieldHint) auto-create htmlFor/id associations, aria-describedby, and role="alert" for errors. Do not manually set htmlFor, id, aria-describedby, or role attributes when using these wrappers. This automatic wiring does not apply to plain rocket.chat/fuselage components, which require explicit ID wiring per the accessibility docs. In code reviews, prefer using fuselage-forms wrappers for form fields and verify there is no unnecessary manual ID/aria wiring in files that use these wrappers. If a component uses plain fuselage components, ensure proper id wiring as per docs.

Applied to files:

  • packages/message-parser/tests/emoji.test.ts
🔇 Additional comments (3)
packages/message-parser/src/grammar.pegjs (1)

814-820: LGTM!

packages/message-parser/tests/emoji.test.ts (1)

91-115: LGTM!

.changeset/emoji-zwj-tag-sequences.md (1)

1-6: LGTM!

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Re-trigger cubic

@abhinavkrin abhinavkrin changed the title fix: combined emojis displayed as separate emojis and England/Scotland/Wales flags displayed as a black flag regression: combined emojis displayed as separate emojis and England/Scotland/Wales flags displayed as a black flag Jul 17, 2026
@abhinavkrin abhinavkrin changed the title regression: combined emojis displayed as separate emojis and England/Scotland/Wales flags displayed as a black flag regression: combined emojis displayed as separate emojis and some flags displayed as a black flag Jul 17, 2026
@codecov

codecov Bot commented Jul 17, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.54%. Comparing base (65a366e) to head (f4333e2).
⚠️ Report is 7 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop   #41441      +/-   ##
===========================================
- Coverage    68.56%   68.54%   -0.02%     
===========================================
  Files         4124     4124              
  Lines       160128   160246     +118     
  Branches     29166    29183      +17     
===========================================
+ Hits        109788   109841      +53     
- Misses       45240    45295      +55     
- Partials      5100     5110      +10     
Flag Coverage Δ
e2e 58.99% <ø> (-0.03%) ⬇️
e2e-api 45.35% <ø> (-0.30%) ⬇️
unit 70.50% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@abhinavkrin abhinavkrin added the stat: QA assured Means it has been tested and approved by a company insider label Jul 17, 2026
@abhinavkrin abhinavkrin added this to the 8.7.0 milestone Jul 17, 2026
@dionisio-bot dionisio-bot Bot added the stat: ready to merge PR tested and approved waiting for merge label Jul 17, 2026
@dionisio-bot
dionisio-bot Bot added this pull request to the merge queue Jul 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 17, 2026
@dionisio-bot
dionisio-bot Bot added this pull request to the merge queue Jul 17, 2026
Merged via the queue into develop with commit adc1570 Jul 18, 2026
48 checks passed
@dionisio-bot
dionisio-bot Bot deleted the fix/emoji-zwj-and-tag-sequences-split branch July 18, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stat: QA assured Means it has been tested and approved by a company insider stat: ready to merge PR tested and approved waiting for merge type: bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants