Skip to content

fix(agent): tolerate lone UTF-16 surrogates in tool-guardrail hashing - #66650

Closed
NimbleCoAI wants to merge 1 commit into
NousResearch:mainfrom
NimbleCoAI:fix/tool-guardrail-surrogatepass
Closed

fix(agent): tolerate lone UTF-16 surrogates in tool-guardrail hashing#66650
NimbleCoAI wants to merge 1 commit into
NousResearch:mainfrom
NimbleCoAI:fix/tool-guardrail-surrogatepass

Conversation

@NimbleCoAI

Copy link
Copy Markdown

Hit this from a tool result with a stray unpaired surrogate in it (looked like scraped web text with half of a mathematical-bold character pair). _sha256() in the tool guardrail controller does a strict utf-8 encode of the tool result before hashing it, which raises UnicodeEncodeError on that input — and since this runs on every tool call, it took the whole conversation loop down.

The hash only needs deterministic bytes out of the string, not a valid UTF-8 roundtrip, so encode("utf-8", "surrogatepass") is a safe swap. Added a regression test that reproduces the crash on an unpatched _sha256 and also checks the hash stays stable enough for the guardrail's exact-failure dedup to still work.

I noticed a few other spots in this codebase already carry surrogatepass-style fixes for similar sites (e.g. Signal/Telegram delivery), so this looked like the same bug class landing in one more place.

Tool results scraped from the web/social platforms can carry unpaired
UTF-16 surrogates (e.g. half of a mathematical-bold character pair).
_sha256() did a strict utf-8 encode, which raises UnicodeEncodeError on
that input and took down the whole conversation loop — the hash only
needs deterministic bytes, not valid UTF-8, so encode with
surrogatepass instead.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
"event": "APPROVE",

Code Review Summary

Verdict: Approved

Fixes a crash when tool-result hashing encounters unpaired UTF-16 surrogates (e.g. first half of a mathematical-bold pair like \\ud835). Web-scraped text can carry these and was taking down the entire conversation loop. Fix: value.encode(\"utf-8\", \"surrogatepass\") instead of default strict encode. New regression test covers the exact failure mode.

Looks Good

  • Targeted fix at the exact crash site (_sha256 in tool_guardrails.py)
  • Comment clearly explains why surrogatepass is correct here
  • Regression test validates both hashing stability and exact-failure tripping

Reviewed by Hermes Agent",
"comments": []
}

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P1 High — major feature broken, no workaround labels Jul 18, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Merged via #66822 — your commit was cherry-picked onto current main with your authorship preserved in git log (rebase-merge, commit e5afc0d).

Clean fix, well spotted: the single _sha256 behind both the args-hash and result-hash paths meant your one-line change covered the whole bug class in the module, and the regression test proving hash stability for the exact-failure dedup was exactly the right thing to include. Thanks for the contribution!

@teknium1 teknium1 closed this Jul 18, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
prmartinow pushed a commit to prmartinow/hermes-agent that referenced this pull request Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P1 High — major feature broken, no workaround type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants