fix(security): skip ZWJ in legitimate emoji sequences during injection scan - #59503
Open
LavyaTandel wants to merge 3 commits into
Open
fix(security): skip ZWJ in legitimate emoji sequences during injection scan#59503LavyaTandel wants to merge 3 commits into
LavyaTandel wants to merge 3 commits into
Conversation
alt-glitch
added
type/bug
…n scan What: Context files (SOUL.md, AGENTS.md, .cursorrules) containing ZWJ emoji (e.g. 👨💻, 🐈⬛) were blocked with 'invisible_unicode_U+200D' because the injection scanner flagged every U+200D occurrence. Why: U+200D is used both in injection attacks (hiding text with zero-width joiners) and in legitimate emoji ZWJ sequences. The scanner had no distinction. Fix: When U+200D is found, check if it joins two emoji code points. If every ZWJ in the content is between emoji code points (emoji ZWJ sequence), skip the finding. Non-emoji ZWJ is still flagged. Closes NousResearch#59492
LavyaTandel
force-pushed
the
fix/zwj-emoji-false-positive
branch
from
July 6, 2026 09:21
94e1aa2 to
fb8afad
Compare
This was referenced Jul 6, 2026
Closed
teknium1
reviewed
Jul 15, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for isolating the real U+200D context-file false positive; current main still flags every U+200D in tools/threat_patterns.py:235-237, and agent/prompt_builder.py:61-64 blocks the affected context file.
Problems
tools/threat_patterns.py:217classifies variation selectors and tag characters as emoji. Combined with the immediate-neighbour check at proposed line 267,a\uFE0F\u200D🔥is accepted even though its left base is ASCII. Skip extensions before validating actual emoji bases, as the existing cron helper does intools/cronjob_tools.py:145-157.- The diff has no regression tests.
tests/agent/test_prompt_builder.py:63-113covers blocked context content but not this acceptance path or malformed ZWJ cases.
Suggested changes
- Add scanner and context-path tests for valid emoji sequences and for text/malformed-selector ZWJ sequences that must remain blocked.
Automated hermes-sweeper review.
| # Variation Selectors (FE00-FE0F), Tag characters (E0020-E007F). | ||
| return ( | ||
| (0x2600 <= cp <= 0x27BF) or | ||
| (0xFE00 <= cp <= 0xFE0F) or |
Contributor
There was a problem hiding this comment.
Variation selectors are extensions, not emoji bases. With this classification, a\uFE0F\u200D🔥 passes the later immediate-neighbour check although the ZWJ is not between two emoji. Skip selectors/tags while walking to the actual bases, then validate those bases as emoji.
teknium1
added
sweeper:risk-security-boundary
…ousResearch#59503) Tag chars U+E0020-U+E007F (flag sequences, e.g. England flag) and variation selectors U+FE0F (keycap sequences, e.g. '1' + VS16 + keycap) were flagged as invisible unicode. Now skipped before the emoji-neighbour check so legitimate emoji graphemes pass. Tests added.
23 tasks
Author
|
Hi @b Mentioning for visibility. This PR is ready for review. All checks are red due to the first-time-contributor Could you please approve a workflow run so the checks can execute? Rebased onto latest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Context files (SOUL.md, AGENTS.md, .cursorrules) containing ZWJ emoji (e.g. 👨💻, 🐈⬛) are no longer blocked by the injection scanner. Previously, every U+200D was flagged as
invisible_unicode_U+200Dand the entire file was dropped.Why
U+200D serves dual purpose: injection attacks (hiding text with zero-width joiners) and legitimate emoji ZWJ sequences. The scanner had no distinction — a decorative emoji in SOUL.md caused the agent to silently boot without its persona/instructions.
Fix
When U+200D is found, check if it joins two emoji code points (ranges: Misc Symbols 2600-26FF, Dingbats 2700-27BF, Emoticons/Transport/Supplemental 1F000-1FFFF, Variation Selectors FE00-FE0F). If every ZWJ in the content is between emoji code points, skip the finding. Non-emoji ZWJ is still flagged.
1 file changed, 36 insertions, 1 deletion.
Runtime Proof
Before:
SOUL.mdwith🐈⬛ (black cat)→ blocked, agent boots without personaAfter: same file loads normally, ZWJ in emoji sequences ignored
Duplicate Scan
No existing PRs for #59492.
Closes #59492