Skip to content

fix(hindsight): specify UTF-8 encoding for file I/O on Windows - #56725

Closed
jeeaay wants to merge 1 commit into
NousResearch:mainfrom
jeeaay:fix/hindsight-utf8-encoding-windows
Closed

jeeaay wants to merge 1 commit into
NousResearch:mainfrom
jeeaay:fix/hindsight-utf8-encoding-windows

Conversation

@jeeaay

@jeeaay jeeaay commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Problem

On Windows with CJK system locales (e.g. Chinese GBK), pathlib.Path.read_text() defaults to the system encoding instead of UTF-8. This causes a UnicodeDecodeError when the hindsight memory provider reads .env or JSON config files containing non-ASCII characters.

UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 661: illegal multibyte sequence

Fix

Explicitly pass encoding="utf-8" to all 3 read_text() and 1 write_text() calls in plugins/memory/hindsight/__init__.py that handle .env and JSON config files.

This is consistent with the existing write_text(..., encoding="utf-8") call already present at line 556 in the same file.

Affected lines

Line Change
749 config_path.read_text()read_text(encoding="utf-8")
898 env_path.read_text()read_text(encoding="utf-8")
928 env_path.read_text()read_text(encoding="utf-8")
941 env_path.write_text(...)write_text(..., encoding="utf-8")

Environment

  • OS: Windows 10, Chinese (GBK) locale
  • Python: 3.11.15
  • Hermes: v2026.7.1

On Windows with CJK locales (e.g. Chinese/GBK), pathlib.Path.read_text()
defaults to the system encoding instead of UTF-8, causing UnicodeDecodeError
when reading .env or .json config files that contain non-ASCII characters.

Explicitly pass encoding='utf-8' to all read_text() and write_text() calls
in the hindsight memory provider plugin.
@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers platform/windows Native Windows-specific behavior or breakage sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows P3 Low — cosmetic, nice to have labels Jul 2, 2026
@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the targeted Windows compatibility fix. The premise is confirmed on current main: HindsightMemoryProvider.save_config still reads JSON without an encoding at plugins/memory/hindsight/__init__.py:749, and post_setup still has default-encoding .env reads/writes at lines 898, 928, and 941. This matches the four lines in commit d0dbdd807a3a.

Problems

  • The PR does not add regression coverage. Existing setup coverage is in tests/plugins/memory/test_hindsight_provider.py:503-625, but no test exercises UTF-8 non-ASCII JSON/.env content across the changed paths.

Suggested changes

  • Add a focused temporary-HERMES_HOME regression covering save_config and local-embedded setup with UTF-8 non-ASCII fixtures, ensuring these paths continue to specify UTF-8.

This is an automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users area/memory Memory subsystem: store, providers, sync, background reviews labels Jul 15, 2026
@teknium1

Copy link
Copy Markdown
Collaborator

Merged via PR #71078 — your commit(s) were cherry-picked onto current main with your authorship preserved in git log (rebase merge). This PR was part of the class-wide close-out of bare read_text/write_text calls: all 139 remaining sites now pass explicit encoding, and a new CI linter rule prevents regressions. Thanks for the contribution!

@teknium1 teknium1 closed this Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have platform/windows Native Windows-specific behavior or breakage sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants