Skip to content

fix(platforms,gateway,tools): add encoding='utf-8' to write_text() calls - #54240

Closed
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:fix/write-text-utf8-encoding-user-content
Closed

fix(platforms,gateway,tools): add encoding='utf-8' to write_text() calls#54240
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:fix/write-text-utf8-encoding-user-content

Conversation

@AlexFucuson9

Copy link
Copy Markdown
Contributor

Problem

Path.write_text() without an explicit encoding parameter uses the platform's default encoding. On Windows this is typically cp1252 or mbcs, which corrupts non-ASCII characters (emoji, CJK text, accented characters) in user-facing content.

This is a cross-platform data corruption bug — content written on Windows with non-ASCII characters will be silently corrupted, and reading it back with read_text(encoding="utf-8") (which the codebase consistently uses) will produce garbled output or raise UnicodeDecodeError.

Affected Locations

File Context Content Type
plugins/platforms/discord/adapter.py:6209 Temp file for reply User/agent response text
plugins/platforms/telegram/adapter.py:4808 Temp file for reply User/agent response text
plugins/platforms/feishu/adapter.py:2103 Temp file for reply User/agent response text
gateway/run.py:8004 Response temp file Agent response text
gateway/delivery.py:281,294 Message output files Message content
tools/skills_hub.py (3 locations) Skill cache files JSON with ensure_ascii=False

Fix

Add encoding="utf-8" to each write_text() call:

# Before
tmp.write_text(answer)

# After
tmp.write_text(answer, encoding="utf-8")

This follows the existing pattern where read_text() calls consistently specify encoding="utf-8" throughout the codebase.

Impact

  • Severity: P1 — data corruption on Windows for non-ASCII content
  • Scope: 6 files, 9 lines changed
  • Risk: Minimal — explicit encoding matches what Linux already defaults to

Path.write_text() without an explicit encoding uses the platform's
default encoding. On Windows this is typically cp1252 or mbcs, which
corrupts non-ASCII characters (emoji, CJK text, accented characters)
in user-facing content.

Fixed the most critical locations where user/agent text is written:
- Platform adapters (discord, telegram, feishu): temp file for reply text
- gateway/run.py: agent response temp file
- gateway/delivery.py: message content output files
- tools/skills_hub.py: skill cache with ensure_ascii=False (Unicode data)

This follows the existing pattern used throughout the codebase where
read_text() calls consistently specify encoding='utf-8'.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins comp/tools Tool registry, model_tools, toolsets platform/discord Discord bot adapter platform/feishu Feishu / Lark adapter platform/telegram Telegram bot adapter platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages P3 Low — cosmetic, nice to have labels Jun 28, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved

Mechanical fix adding explicit encoding='utf-8' to write_text() calls across platform adapters and core modules (7+ files). Same rationale as #54241 for read_text -- ensures consistent UTF-8 encoding on Windows.

Strengths:

  • Consistent application across all identified call sites
  • No behavioral change on Unix
  • Prevents encoding mismatches when writing response files, cron output, and skill cache

No concerns.


Reviewed by Hermes Agent

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Mechanical change adding encoding='utf-8' to write_text() calls across 6 files. Consistent with #54241 (read_text encoding fix). No logic changes.

Reviewed by Hermes Agent

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for identifying a real Windows portability issue. Current main still has the six unencoded writes this PR targets, including gateway/run.py:9040, the Discord/Feishu/Telegram update-response paths, gateway/delivery.py:360/:373, and the three skills-hub cache writes.

Problems

  • gateway/platforms/qqbot/adapter.py:1204 implements the same atomic .update_response answer-file contract as the three platform adapters, but still uses tmp.write_text(answer) without an encoding. Please include it so the update-response fix covers all current adapters.
  • The PR has no regression coverage for a non-ASCII response/cache value or explicit UTF-8 writer behavior.

Suggested changes

  • Add encoding="utf-8" at gateway/platforms/qqbot/adapter.py:1204.
  • Add a narrow UTF-8 regression test for an update-response path and a skills-hub cache path.

Automated hermes-sweeper review.

response_path = home / ".update_response"
tmp = response_path.with_suffix(".tmp")
tmp.write_text(answer)
tmp.write_text(answer, encoding="utf-8")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include the equivalent QQBot update-response writer at gateway/platforms/qqbot/adapter.py:1204; it uses the same tmp.write_text(answer) pattern and remains vulnerable on Windows.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 15, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Closing as resolved by PR #71078 (merged, commit d372fda): the class-wide close-out salvaged your #50655/#54241/#56385/#66856/#65440 series as the backbone (authorship preserved in git log) and swept the remaining sites, so every read_text/write_text call this PR touches is now guarded on current main — verified per-site. A CI linter rule in check-windows-footguns.py plus the AST guard test now prevent regressions. Your overlapping/split variants of the same series are being closed together; the credit for the class rests on your commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have platform/discord Discord bot adapter platform/feishu Feishu / Lark adapter platform/telegram Telegram bot adapter platform/windows Native Windows-specific behavior or breakage sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants