Skip to content

fix(agent): allow ZWJ emoji sequences in threat scanner (#59492) - #59701

Closed
webtecnica wants to merge 1 commit into
NousResearch:mainfrom
webtecnica:fix/59492-zwj-emoji-false-positive
Closed

fix(agent): allow ZWJ emoji sequences in threat scanner (#59492)#59701
webtecnica wants to merge 1 commit into
NousResearch:mainfrom
webtecnica:fix/59492-zwj-emoji-false-positive

Conversation

@webtecnica

Copy link
Copy Markdown
Contributor

Summary

U+200D (zero-width joiner) in INVISIBLE_CHARS was causing false positives for legitimate emoji ZWJ sequences (e.g. ZWJ emoji sequences) in SOUL.md / AGENTS.md context files. This PR makes the threat scanner context-aware for ZWJ: it only flags U+200D when it does not join two Extended_Pictographic (emoji) characters.

Root cause

tools/threat_patterns.py lists U+200D in INVISIBLE_CHARS, and scan_for_threats() flags any occurrence via set-intersection. The set-based approach loses position information, so legitimate ZWJ emoji sequences are indistinguishable from ZWJ-based injection tactics.

Fix

  1. Add _is_extended_pictographic() helper that checks unicodedata.category() == "So" plus Variation Selectors (U+FE00-U+FE0F) and Regional Indicators (U+1F1E6-U+1F1FF).
  2. In scan_for_threats(): skip U+200D in the fast set-based intersection, then run a second pass that examines each ZWJ occurrence in context. ZWJ between two emoji => skip (legitimate sequence); ZWJ anywhere else => flag as before.

Changes

File Change
tools/threat_patterns.py Add _is_extended_pictographic() + context-aware ZWJ scanning
tests/tools/test_threat_patterns.py 6 new tests for ZWJ emoji false-positive fix

Closes #59492

…#59492)

U+200D (zero-width joiner) in INVISIBLE_CHARS was causing false
positives for legitimate emoji ZWJ sequences like 🐈‍⬛ and 👨‍💻
in SOUL.md / AGENTS.md context files.

- Add _is_extended_pictographic() helper checking Unicode category
  "So" + Variation Selectors + Regional Indicators
- Replace blind set-based flagging of U+200D with a context-aware
  scan that only flags ZWJ when it does NOT join two emoji
- U+200D between non-emoji characters or at boundaries is still
  flagged (preserving the injection-detection signal)
- All 46 existing tests pass; 6 new tests added for ZWJ emoji
  sequences
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 6, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #59503 — both make scan_for_threats() in tools/threat_patterns.py context-aware for U+200D (skip the finding only when the ZWJ joins two Extended_Pictographic codepoints). #59503 is the earliest open variant of this same-mechanism fix; the extra helper/tests here don't change the underlying change. Also related to the broader #12673 and issue #59492.

@webtecnica

Copy link
Copy Markdown
Contributor Author

Closing as duplicate — the sweeper identified this as already covered by another PR. Thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

False positive: ZWJ emoji (U+200D) in context files (SOUL.md/AGENTS.md) is flagged as injection and drops the whole file

2 participants