fix(agent): tolerate lone UTF-16 surrogates in tool-guardrail hashing - #66650
fix(agent): tolerate lone UTF-16 surrogates in tool-guardrail hashing#66650NimbleCoAI wants to merge 1 commit into
Conversation
Tool results scraped from the web/social platforms can carry unpaired UTF-16 surrogates (e.g. half of a mathematical-bold character pair). _sha256() did a strict utf-8 encode, which raises UnicodeEncodeError on that input and took down the whole conversation loop — the hash only needs deterministic bytes, not valid UTF-8, so encode with surrogatepass instead.
tonydwb
left a comment
There was a problem hiding this comment.
{
"event": "APPROVE",
Code Review Summary
Verdict: Approved
Fixes a crash when tool-result hashing encounters unpaired UTF-16 surrogates (e.g. first half of a mathematical-bold pair like \\ud835). Web-scraped text can carry these and was taking down the entire conversation loop. Fix: value.encode(\"utf-8\", \"surrogatepass\") instead of default strict encode. New regression test covers the exact failure mode.
Looks Good
- Targeted fix at the exact crash site (
_sha256intool_guardrails.py) - Comment clearly explains why
surrogatepassis correct here - Regression test validates both hashing stability and exact-failure tripping
Reviewed by Hermes Agent",
"comments": []
}
|
Merged via #66822 — your commit was cherry-picked onto current main with your authorship preserved in git log (rebase-merge, commit e5afc0d). Clean fix, well spotted: the single |
Hit this from a tool result with a stray unpaired surrogate in it (looked like scraped web text with half of a mathematical-bold character pair).
_sha256()in the tool guardrail controller does a strictutf-8encode of the tool result before hashing it, which raisesUnicodeEncodeErroron that input — and since this runs on every tool call, it took the whole conversation loop down.The hash only needs deterministic bytes out of the string, not a valid UTF-8 roundtrip, so
encode("utf-8", "surrogatepass")is a safe swap. Added a regression test that reproduces the crash on an unpatched_sha256and also checks the hash stays stable enough for the guardrail's exact-failure dedup to still work.I noticed a few other spots in this codebase already carry
surrogatepass-style fixes for similar sites (e.g. Signal/Telegram delivery), so this looked like the same bug class landing in one more place.