Skip to content

regression: combined picker emojis split into separate symbols - #41534

Merged
dionisio-bot[bot] merged 2 commits into
release-8.7.0from
fix/emoji-parsing
Jul 28, 2026
Merged

regression: combined picker emojis split into separate symbols#41534
dionisio-bot[bot] merged 2 commits into
release-8.7.0from
fix/emoji-parsing

Conversation

@ricardogarim

@ricardogarim ricardogarim commented Jul 23, 2026

Copy link
Copy Markdown
Member

Proposed changes (including videos or screenshots)

@rocket.chat/message-parser recognizes emojis via hand-maintained Unicode block rules, and several classes weren't covered — so they were split into pieces (🚶‍♀️ → 🚶 + ♀️) or left as text, and didn't enlarge when sent alone. CORE-2466 reported the "facing right" combined emojis, but the gap was wider.

This extends the grammar to recognize combined/ZWJ sequences, keycaps, geometric squares, squared badges, and variation-selector (VS16) forms — every emoji emojibase-data ships (283 → 0), guarded by a coverage test. Text-default symbols (©, ™, ▶) require their VS16, so bare punctuation stays text. The only exclusions are the 26 regional indicators — flag-building letters that form a flag only in a pair (🇺 + 🇸 = 🇺🇸); real letter emojis (🅰️ Ⓜ️) are supported.

Issue(s)

Steps to test or reproduce

Send each emoji in a message (paste, or pick from the emoji picker). Each should render as a single emoji and enlarge when sent alone.

Combined / ZWJ sequences: 🚶‍♀️ 🚶‍♂️ 🚶‍➡️ 🚶‍♀️‍➡️ 🚶‍♂️‍➡️ 🚣‍♀️ 🚴‍♂️ 🚵‍♀️ 🙂‍↔️ 🙂‍↕️ 🧑‍🚀 👨‍🚒 🐈‍⬛ 🐦‍⬛ 🍋‍🟩 🍄‍🟫

Geometric squares / circles: 🟠 🟡 🟢 🟣 🟤 🟥 🟦 🟧 🟨 🟩 🟪 🟫 ⬛ ⬜ 🟰

Keycaps: 0️⃣ 1️⃣ 2️⃣ 3️⃣ 4️⃣ 5️⃣ 6️⃣ 7️⃣ 8️⃣ 9️⃣ #️⃣ *️⃣

Squared badges / CJK / joker: 🆎 🆑 🆒 🆓 🆔 🆕 🆖 🆗 🆘 🆙 🆚 🈁 🈶 🉐 🈹 🈲 🉑 🈸 🈴 🈳 🈺 🈵 🃏

Variation-selector (VS16) forms: ©️ ®️ ™️ ‼️ ⁉️ ℹ️ Ⓜ️ ▶️ ◀️ ⭐️ ⭕️ ⬆️ ⬇️ ⬅️ ↖️ ↗️ ↘️ ↙️ ↩️ ↪️ ⤴️ ⤵️ ▪️ ▫️ ◻️ ◼️ ◽️ ◾️ 〰️ 〽️ ㊗️ ㊙️ 🅰️ 🅱️ 🅾️ 🅿️ 🈂️ 🈚️ 🈯️ 🀄️ 🚇️ 🚑️ 🛳️ 🛎️ 🚹️ 🚺️ 😐️

Should still render as text (not emoji): bare © # → *, plus *bold* and #channel.

Flags still render: 🇺🇸 🇧🇷

Further comments

The coverage test sweeps a committed fixture (tests/fixtures/allEmoji.ts) — every emoji emojibase-data ships — generated by scripts/emoji/generateEmojiFixture.mjs (see its README). It's a snapshot: after bumping emojibase-data, re-run the script and the sweep flags any new emoji the grammar doesn't recognize.

Review in cubic

Summary by CodeRabbit

  • New Features
    • Enhanced emoji parsing to better recognize keycaps, flags, transport/map symbols, arrows, geometric shapes, enclosed badges, and text-style characters.
    • Improved native emoji rendering resolution using “bare” aliases, including correct behavior with and without variation selector-16 (VS16).
  • Documentation
    • Added guidance on generating and updating the comprehensive emoji test fixture.
  • Tests
    • Added regression coverage ensuring documented emojis (including skin-tone variants) parse as single tokens and verified native rendering differences with VS16.

@dionisio-bot

dionisio-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Looks like this PR is ready to merge! 🎉
If you have any trouble, please check the PR guidelines

@changeset-bot

changeset-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: e40fb3e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The emoji PEG grammar recognizes additional Unicode emoji sequences and refined symbol ranges. A generated emojibase fixture validates parser coverage, while native emoji data now resolves bare Unicode aliases alongside VS16-qualified forms.

Changes

Emoji parsing and native rendering

Layer / File(s) Summary
Expand grammar and parser coverage
packages/message-parser/src/grammar.pegjs, packages/message-parser/tests/emoji.test.ts
Adds keycap, geometric square, enclosed badge, and text-presentation rules; expands ZWJ components; refines emoji ranges; and validates the generated fixture as single tokens.
Generate emoji coverage fixture
scripts/emoji/generateEmojiFixture.mjs, packages/message-parser/tests/fixtures/allEmoji.ts, scripts/emoji/README.md
Generates and documents a versioned snapshot of base and skin-tone emojibase entries.
Resolve native bare aliases
apps/meteor/app/emoji-native/lib/generateEmojiData.ts, apps/meteor/app/emoji-native/lib/getEmojiConfig.ts, apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
Adds bare aliases to emoji data and Unicode lookup, with tests for VS16 and text-default rendering behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested labels: type: bug

Suggested reviewers: sampaiodiego, martinschoeler

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the regression being fixed: combined emojis were being split into separate symbols.

Warning

Review ran into problems

🔥 Problems

Errors were encountered while retrieving linked issues.

Errors (2)
  • CORE-2466: Request failed with status code 401
  • CORE-2462: Request failed with status code 401

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ricardogarim ricardogarim added this to the 8.7.0 milestone Jul 23, 2026
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.67%. Comparing base (fcf244e) to head (e40fb3e).
⚠️ Report is 3 commits behind head on release-8.7.0.

Additional details and impacted files

Impacted file tree graph

@@                Coverage Diff                @@
##           release-8.7.0   #41534      +/-   ##
=================================================
+ Coverage          68.61%   68.67%   +0.06%     
=================================================
  Files               4138     4139       +1     
  Lines             159416   159678     +262     
  Branches           28206    28313     +107     
=================================================
+ Hits              109384   109662     +278     
+ Misses             44893    44871      -22     
- Partials            5139     5145       +6     
Flag Coverage Δ
e2e 58.87% <90.00%> (+0.02%) ⬆️
e2e-api 45.65% <ø> (+0.03%) ⬆️
unit 70.65% <100.00%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ricardogarim
ricardogarim marked this pull request as ready for review July 23, 2026 18:45

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed against the latest diff

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/message-parser/src/grammar.pegjs
Comment thread packages/message-parser/tests/emoji.test.ts
@ricardogarim
ricardogarim requested review from a team as code owners July 27, 2026 00:46
@ricardogarim
ricardogarim removed request for a team July 27, 2026 00:47

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/message-parser/src/grammar.pegjs`:
- Line 837: Update UnicodeEmojiTransportAndMapSymbols to exclude the reserved
U+1F6ED–U+1F6EF code points instead of accepting the entire surrogate range,
using explicit emoji-data ranges or a generated whitelist. Add parser coverage
confirming reserved values are rejected while valid transport/map emoji continue
to parse.
- Around line 848-855: Update the Unicode emoji grammar near
UnicodeEmojiTextPresentation to recognize bare ⭐ (U+2B50) and ⭕ (U+2B55) as
emoji-default characters, while preserving the existing VS16-required handling
for other text-presentation characters.
- Line 824: Restrict the standalone arrow grammar rule to arrows with required
VS16, rather than allowing the broad UnicodeEmojiArrows rule to match bare
text-default arrows. Preserve UnicodeEmojiArrows for UnicodeEmojiZwjComponent so
arrows remain valid within ZWJ sequences, while standalone ↔ and ↕ require VS16.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1de64390-008c-443d-92b6-26a6b8ccd732

📥 Commits

Reviewing files that changed from the base of the PR and between 0c05727 and e40fb3e.

📒 Files selected for processing (8)
  • apps/meteor/app/emoji-native/lib/generateEmojiData.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.ts
  • packages/message-parser/src/grammar.pegjs
  • packages/message-parser/tests/emoji.test.ts
  • packages/message-parser/tests/fixtures/allEmoji.ts
  • scripts/emoji/README.md
  • scripts/emoji/generateEmojiFixture.mjs
🚧 Files skipped from review as they are similar to previous changes (4)
  • packages/message-parser/tests/emoji.test.ts
  • scripts/emoji/README.md
  • packages/message-parser/tests/fixtures/allEmoji.ts
  • scripts/emoji/generateEmojiFixture.mjs
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
  • GitHub Check: 📦 Build Packages
  • GitHub Check: CodeQL-Build
  • GitHub Check: CodeQL-Build
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx,js}

📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)

**/*.{ts,tsx,js}: Write concise, technical TypeScript/JavaScript with accurate typing in Playwright tests
Avoid code comments in the implementation

Files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.ts
  • apps/meteor/app/emoji-native/lib/generateEmojiData.ts
**/*.spec.ts

📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)

**/*.spec.ts: Use descriptive test names that clearly communicate expected behavior in Playwright tests
Use .spec.ts extension for test files (e.g., login.spec.ts)

Files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
🧠 Learnings (6)
📚 Learning: 2026-02-24T19:22:48.358Z
Learnt from: juliajforesti
Repo: RocketChat/Rocket.Chat PR: 38493
File: apps/meteor/tests/e2e/omnichannel/omnichannel-send-pdf-transcript.spec.ts:66-67
Timestamp: 2026-02-24T19:22:48.358Z
Learning: In Playwright end-to-end tests (e.g., under apps/meteor/tests/e2e/...), prefer locating elements by translated text (getByText) and ARIA roles (getByRole) over data-qa attributes. If translation values change, update the corresponding test locators accordingly. Never use data-qa locators. This guideline applies to all Playwright e2e test specs in the repository and helps keep tests robust to UI text changes and accessible semantics.

Applied to files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In the Rocket.Chat repository, do not reference Biome lint rules in code review feedback. Biome is not used even if biome.json exists; only reference Biome rules if there is explicit, project-wide usage documented. For TypeScript files, review lint implications without Biome guidance unless the project enables Biome rules.

Applied to files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.ts
  • apps/meteor/app/emoji-native/lib/generateEmojiData.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In this repository (RocketChat/Rocket.Chat), Biome lint rules are not used even if a biome.json exists. When reviewing TypeScript files (e.g., packages/ui-voip/src/providers/useMediaSession.ts), ensure lint suggestions do not reference Biome-specific rules. Rely on general ESLint/TypeScript lint rules and project conventions instead.

Applied to files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.ts
  • apps/meteor/app/emoji-native/lib/generateEmojiData.ts
📚 Learning: 2026-03-06T18:10:15.268Z
Learnt from: tassoevan
Repo: RocketChat/Rocket.Chat PR: 39397
File: packages/gazzodown/src/code/CodeBlock.spec.tsx:47-68
Timestamp: 2026-03-06T18:10:15.268Z
Learning: In tests (especially those using testing-library/dom/jsdom) for Rocket.Chat components, the HTML <code> element has an implicit ARIA role of 'code'. Therefore, screen.getByRole('code') or screen.findByRole('code') will locate <code> elements even without a role attribute. Do not flag findByRole('code') as invalid in reviews; prefer using the implicit role instead of adding role="code" unless necessary for accessibility.

Applied to files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
📚 Learning: 2026-05-06T12:21:44.083Z
Learnt from: juliajforesti
Repo: RocketChat/Rocket.Chat PR: 40256
File: apps/meteor/client/components/CreateDiscussion/CreateDiscussion.tsx:121-149
Timestamp: 2026-05-06T12:21:44.083Z
Learning: Field wrappers in rocket.chat/fuselage-forms (Field, FieldLabel, FieldRow, FieldError, FieldHint) auto-create htmlFor/id associations, aria-describedby, and role="alert" for errors. Do not manually set htmlFor, id, aria-describedby, or role attributes when using these wrappers. This automatic wiring does not apply to plain rocket.chat/fuselage components, which require explicit ID wiring per the accessibility docs. In code reviews, prefer using fuselage-forms wrappers for form fields and verify there is no unnecessary manual ID/aria wiring in files that use these wrappers. If a component uses plain fuselage components, ensure proper id wiring as per docs.

Applied to files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.ts
  • apps/meteor/app/emoji-native/lib/generateEmojiData.ts
📚 Learning: 2026-07-22T19:03:06.706Z
Learnt from: KevLehman
Repo: RocketChat/Rocket.Chat PR: 41520
File: apps/meteor/app/emoji-native/lib/getEmojiConfig.ts:59-62
Timestamp: 2026-07-22T19:03:06.706Z
Learning: In Rocket.Chat’s native emoji pipeline, keep the legacy-parity behavior for ASCII emoticons that appear inside `<code>`/`<pre>`: `apps/meteor/app/emoji-native/lib/getEmojiConfig.ts` may convert matching ASCII emoticons into emoji-span nodes, but the downstream DOM pass in `apps/meteor/app/emoji/client/emojiParser.ts` must restore the emoji span’s literal `title` text when the span’s parent is a `CODE` element. When changing emoji config or parsing, ensure this `<code>` protection behavior is preserved; the planned structural HTML protection should be handled by the post-release message-parser/gazzodown migration rather than by removing/altering this restore logic ad-hoc.

Applied to files:

  • apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts
  • apps/meteor/app/emoji-native/lib/getEmojiConfig.ts
  • apps/meteor/app/emoji-native/lib/generateEmojiData.ts
🔇 Additional comments (4)
packages/message-parser/src/grammar.pegjs (1)

798-798: LGTM!

Also applies to: 808-812, 828-833, 858-859

apps/meteor/app/emoji-native/lib/generateEmojiData.ts (1)

19-19: LGTM!

Also applies to: 59-61, 75-75, 77-78, 92-95, 118-118, 138-138

apps/meteor/app/emoji-native/lib/getEmojiConfig.ts (1)

27-27: LGTM!

Also applies to: 36-41

apps/meteor/app/emoji-native/lib/getEmojiConfig.spec.ts (1)

55-77: LGTM!

Comment thread packages/message-parser/src/grammar.pegjs
Comment thread packages/message-parser/src/grammar.pegjs
Comment thread packages/message-parser/src/grammar.pegjs
@jessicaschelly jessicaschelly added the stat: QA assured Means it has been tested and approved by a company insider label Jul 28, 2026
@dionisio-bot dionisio-bot Bot added the stat: ready to merge PR tested and approved waiting for merge label Jul 28, 2026
@dionisio-bot
dionisio-bot Bot merged commit 987f8d6 into release-8.7.0 Jul 28, 2026
92 of 94 checks passed
@dionisio-bot
dionisio-bot Bot deleted the fix/emoji-parsing branch July 28, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stat: QA assured Means it has been tested and approved by a company insider stat: ready to merge PR tested and approved waiting for merge type: bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants