fix(gateway): pass encoding="utf-8" to read_text/write_text in update path (#37423) by rodboev · Pull Request #38985 · NousResearch/hermes-agent

rodboev · 2026-06-04T12:20:47Z

Summary

On native Windows with a non-UTF-8 locale such as cp1252 or GBK, the gateway update flow could crash or corrupt file-based IPC. Several gateway code paths still used bare Path.read_text() or Path.write_text() calls for update response files, pending-session JSON, exit codes, captured subprocess output, prompt JSON, PID files, and small restart or state markers. Python defaults those calls to the active locale codec instead of UTF-8, which can raise UnicodeEncodeError when writing emoji or CJK input, UnicodeDecodeError when reading UTF-8 subprocess output, or silently mojibake the payload.

This branch keeps the original fix intent on current main: every remaining gateway read_text() and write_text() call in the update path now passes encoding="utf-8" explicitly, and the regression guard enforces that invariant across gateway/. The maintainer follow-up on this PR found one bundled sibling outside gateway/: the Discord update-response writer still used a bare write_text(), and the AST guard only scanned gateway/, so it could not detect that omission. This rework adds the missing Discord encoding and extends the guard to the bundled Discord, Telegram, and Feishu .update_response adapters alongside the recursive gateway tree.

Fixes #37423

Changes

gateway/run.py: add encoding="utf-8" to update-path and gateway-state read_text() / write_text() calls
gateway/slash_commands.py: add encoding="utf-8" to the /update pending-session marker write
gateway/delivery.py: add encoding="utf-8" to delivery output writes
gateway/dead_targets.py: add encoding="utf-8" to dead-target registry text I/O
gateway/status.py: add encoding="utf-8" to the PID record read
gateway/platforms/qqbot/adapter.py: add encoding="utf-8" to the update-prompt answer write
plugins/platforms/telegram/adapter.py: add encoding="utf-8" to the update-prompt answer write
plugins/platforms/feishu/adapter.py: add encoding="utf-8" to the update-prompt answer write
plugins/platforms/discord/adapter.py: add encoding="utf-8" to the update-prompt answer write that the review thread flagged
plugins/platforms/whatsapp/adapter.py: add encoding="utf-8" to the PID and dependency-stamp reads and writes
tests/gateway/test_gateway_utf8_encoding.py: keep the AST regression guard for bare read_text() / write_text() calls under gateway/ and extend it to the bundled Discord, Telegram, and Feishu update-response adapters

Validation

Scenario	Before	After
User replies to an update prompt with emoji on cp1252 Windows	`UnicodeEncodeError` could crash the gateway command handler	UTF-8 write succeeds
`hermes update` emits UTF-8 box-drawing, emoji, or CJK output	`UnicodeDecodeError` could crash the update watcher, or output could mojibake under a non-UTF-8 locale	UTF-8 read succeeds and streams correctly
Pending `/update` metadata is written on a non-UTF-8 locale	pending-session JSON could be locale-encoded	pending-session JSON is always UTF-8
Telegram, Discord, QQ, or Feishu update-prompt answers contain non-ASCII text	adapter write could fail with `UnicodeEncodeError`	adapter write succeeds
WhatsApp bridge PID or dependency stamp is read on a non-UTF-8 locale	locale decoding could be inconsistent	read and write are explicit UTF-8
Future bare `read_text()` / `write_text()` call appears in `gateway/` or the bundled update-response adapters	the Discord sibling sat outside the old guard	the AST test fails immediately

Test plan

python -m pytest tests/gateway/test_gateway_utf8_encoding.py -v --timeout=0 — 1 passed

Not in scope

The surrounding except OSError guards still do not catch UnicodeError subclasses. This change keeps those errors from arising by forcing UTF-8 at the file boundary, but it does not broaden the exception handling itself.

… path (NousResearch#37423)

teknium1

Thanks for the focused Windows compatibility fix. The current-main /update file-IPC paths still use locale-default Path text I/O, so the core premise remains valid.

Problems

plugins/platforms/discord/adapter.py:7143 still writes the same .update_response IPC file with bare tmp.write_text(answer). This is the Discord sibling of the Telegram/Feishu/QQ paths changed here and remains locale-dependent on Windows.
tests/gateway/test_gateway_utf8_encoding.py:10 scans only gateway/, so it cannot catch that bundled-plugin omission or future regressions in the plugin response writers changed by this PR.

Suggested changes

Add encoding="utf-8" at plugins/platforms/discord/adapter.py:7143.
Cover the bundled platform update-response writers in the regression test scope.

Automated hermes-sweeper review.

…37423)

Follow-up to the salvaged #38985: guard the 4 bare read_text/write_text sites its allowlist missed (google_chat thread-count store + oauth JSON) and add whatsapp/google_chat to the AST guard test's file list.

teknium1 · 2026-07-24T22:47:50Z

Merged via PR #71014 — all three of your commits were cherry-picked onto current main with your authorship preserved in git log (rebase merge). The AST guard test is the standout piece: it now also covers the whatsapp and google_chat adapters (we guarded the 4 google_chat sites your allowlist missed as a follow-up commit on top). This closes the gateway update-path half of #37423; the remaining ~134 bare read_text/write_text sites elsewhere in the repo are queued as a follow-up campaign using your guard as the template. Thanks for the thorough work and the Jul 19 refresh.

…r rule AST-driven pass over every Path.read_text()/write_text() without an explicit encoding= across non-test code: 71 sites in 34 files (skills_hub, hermes_cli/main+profiles+service_manager+container_boot, mem0/hindsight/honcho plugins, achievements dashboard, release/CI scripts, productivity+comfyui skill helpers, agent/*). Verified zero positional-encoding collisions before insertion; per-file compile() check after. Adds a check-windows-footguns rule flagging bare single-line read_text/write_text (multi-line forms stay covered by the AST guard test from #38985). Together with the salvaged contributor commits this retires the ~169-site bare file-I/O class (#37423's long tail).

* fix(memory-setup): sanitize .env values in the core writer too Widens the salvaged .env injection fix (#50315) to the sibling site it missed: hermes_cli/memory_setup.py::_write_env_vars is the near-identical core writer the openviking plugin's copy was forked from, is fed directly by interactive _prompt() (pasted API keys), and is reused by other memory plugins (e.g. supermemory imports it). A pasted secret with an embedded CR/LF injected an arbitrary extra KEY=VALUE line on the next read. Same _env_line_safe() treatment as the plugin writer (strip every str.splitlines() separator + NUL), matching config.save_env_value's existing newline strip. Mutation-checked: reverting the sanitizer makes the new regression tests fail. * fix(cron): tick every served profile's cron store under multiplex_profiles (#69377) Under multiplex_profiles, the gateway starts a single InProcessCronScheduler bound to the process-global HERMES_HOME (the default profile's home), so only that profile's cron/jobs.json is ticked. A job registered from a secondary-profile session lands in <profile>/cron/jobs.json, reports a valid next_run_at — and never fires. Changes: 1. cron/scheduler_provider.py — InProcessCronScheduler.start() now accepts an optional profile_homes kwarg (list of (name, Path) tuples). When set, _start_multiplex() iterates tick() over each profile home using use_cron_store(), so every served profile's cron store is ticked on every tick cycle. Heartbeats and interrupted-execution recovery are also scoped per profile via use_cron_store(). 2. gateway/run.py — start_gateway() now resolves profiles_to_serve(multiplex=True) when multiplex_profiles is on and passes them to the cron scheduler as profile_homes. Only applies to InProcessCronScheduler (the built-in); external providers are unchanged. 3. cron/jobs.py — record_ticker_heartbeat(), get_ticker_heartbeat_age(), and get_ticker_success_age() now resolve paths via _current_cron_store() instead of module-level TICKER_HEARTBEAT_FILE / TICKER_SUCCESS_FILE constants. This makes heartbeats correctly scoped per profile, so 'hermes cron status' reflects liveness for every profile independently under multiplex_profiles. 4. tests/cron/test_scheduler_provider.py — two new tests: - test_multiplex_ticker_ticks_each_profile_once: verifies tick() is called once per profile per tick cycle. - test_multiplex_heartbeat_scoped_per_profile: verifies heartbeat files are written to each profile's cron store. * fix(cron): scope hermes_home override per-profile in multiplex ticker The multiplex cron path only used use_cron_store() to scope storage paths (jobs.json, heartbeat files), but _get_lock_paths() and the agent execution path in cron/scheduler.py resolve via _get_hermes_home() → get_hermes_home() which checks _HERMES_HOME_OVERRIDE, a separate ContextVar. Without set_hermes_home_override(), the .tick.lock, config.yaml, .env, and secrets all resolved to the default profile instead of the per-profile home. This matches the web_server.py pattern (line 11994) which sets both set_hermes_home_override(home) AND use_cron_store(home), and the _profile_runtime_scope pattern used for the multiplexed inbound path. Found via 3-agent parallel review of salvaged PR #69529. * feat(honcho): add OAuth device-code login (RFC 8628) for headless environments Adds a device authorization grant flow alongside the existing loopback OAuth flow, so `hermes setup` can connect to Honcho cloud from SSH and other no-browser environments. - oauth.py: new HTTP seams — _http_post_form_status (non-raising, since RFC 8628 polling reads the OAuth error off a 400) and _http_get_json for the RFC 8414 metadata probe - oauth_flow.py: DeviceCode, request_device_code, poll_for_token with slow_down backoff (+5s, capped at 60s) bounded by expires_in, typed errors (AccessDenied, DeviceCodeExpired, AuthorizationTimeout), and supports_device_login (fail-closed metadata gate); device flow ends in the same install_grant tail as loopback so refresh/status work unchanged - oauth_flow.py: loopback callback now serves a "sign-in was not completed" page on consent cancel instead of the success page - cli.py: cloud menu offers oauth / device / apikey; the device option only appears when the host advertises the grant, and becomes the default when no browser is detected - 18 new tests covering the full flow against a local fake AS, backoff schedule, error mapping, deadline bound, metadata gate, and wizard branches Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(honcho): default device-code poll interval to 5s when AS omits it RFC 8628 §3.2 makes the device-authorization `interval` optional with a client-side default of 5 seconds. request_device_code required it, so a compliant AS that omitted it hit the malformed-response path and the flow could never complete. Fall back to 5s and cover it with a regression test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(skills): align skill directory names with frontmatter name Salvaged from PR #42788 by @Love-JourneY, re-applied at current locations (audiocraft and segment-anything have since moved to optional-skills/): - skills/mlops/inference/vllm -> serving-llms-vllm - skills/mlops/evaluation/lm-evaluation-harness -> evaluating-llms-harness - optional-skills/mlops/models/segment-anything -> segment-anything-model - optional-skills/creative/audiocraft -> audiocraft-audio-generation Directory name != frontmatter name breaks skill_view() lookup by dir name and causes hermes update sync re-seeding duplicates (#42786). The authoring guide calls this out as Pitfall #8. Fixes #42786 * docs(skills): update pages, catalogs, sidebar for skill dir renames Auto-gen page slugs, catalog rows/paths, sidebar entries, and zh-Hans mirrors follow the directory renames. Also updates the install path official/creative/audiocraft -> official/creative/audiocraft-audio-generation in the songwriting-and-ai-music pointer section. * docs(skills): fix broken related_skills references (#37338) Salvaged from PR #38820 by @bedirhancode, re-applied at current skill locations (obliteratus and s6 moved to optional-skills/ since the PR): - research-paper-writing: drop ml-paper-writing (never existed) - touchdesigner-mcp: drop native-mcp (consolidated) + hermes-video (never existed) - obliteratus: vllm -> serving-llms-vllm, gguf -> llama-cpp - s6-container-supervision: drop hermes-agent-dev (not a repo skill) 4 of the original 8 hunks were dropped: heartmula already fixed in #70453; native-mcp SKILL.md deleted from main; architecture-diagram and comfyui hunks removed refs to concept-diagrams and stable-diffusion-image-generation, which are valid optional skills. * docs(skills): fix remaining 13 broken related_skills refs repo-wide Widening pass on top of the #38820 salvage: a full-graph audit of every SKILL.md (bundled + optional) found 13 more references to skills that no longer exist. Classes: - deleted in the 38d3c49aaf bundled-skill cleanup: generative-widgets, spotify, cloudflared-quick-tunnel, webhook-subscriptions, debugging-hermes-tui-commands -> dropped - native-mcp absorbed into the hermes-agent hub skill -> re-pointed - toolset names that were never skills: browser, image_gen -> dropped Audit now reports zero broken related_skills references. * docs(skills): sync generated pages + zh-Hans mirrors for related_skills fixes * fix(tools): enforce 60-char description limit for skills MAX_DESCRIPTION_LENGTH was set to 1024, but the documented skill- authoring standard specifies <=60 characters. The model generates descriptions up to 202 chars because the validation allows 1024. Lower MAX_DESCRIPTION_LENGTH from 1024 to 60 to match the documented standard. The system-prompt skill index already truncates to 60 chars, so over-length descriptions lose their routing signal past char 60. Fixes #52367 * fix(skills): scope 60-char description enforcement to the create path The blanket MAX_DESCRIPTION_LENGTH=1024->60 change is narrowed: create-time validation now rejects new skills whose description exceeds SKILL_PROMPT_DESC_LIMIT (60) with actionable guidance, while edit/patch paths stay permissive (warning via system_prompt_preview) so existing over-limit skills remain maintainable. Runtime display truncation in skills_tool is left at 1024 (display behavior is a separate concern from authoring validation). Boundary tests: 60 accepted, 61 rejected at create; edit/patch on over-budget skills still succeed. * fix(tui_gateway): bind the branched agent to the parent profile's home + state.db session.branch wrote the child ROW into the parent's profile db but built the live agent with the launch defaults: _make_agent fell back to _get_db() and no HERMES_HOME override was active. The branched agent's own message flushes — and any later compression rotation it performed — therefore landed back on the launch profile, splitting the lineage one turn after the branch. Mirror session.create/resume: open the parent profile's SessionDB for the agent and hold the home override across the build, so config/skills/memory resolve to the profile too. Spotted in #70605's sibling implementation of the same fix. Co-authored-by: HexLab98 <liruixinch@outlook.com> * fix(skills): sync mlops structured-output/vectordb skills to current APIs Five optional mlops skills documented removed pre-major-version APIs. Verified each against upstream and rewrote to the current form: - outlines: pre-1.0 outlines.generate.*/models.transformers -> v1 from_transformers + model(prompt, output_type) - guidance: models.Anthropic (nonexistent in 0.3.x) -> Transformers backend; grammar-string -> guidance.json(); noted constrained gen needs local logits - pinecone: pip install pinecone-client (deprecated) -> pinecone; removed bogus alpha= query kwarg, pre-scale hybrid vectors - qdrant: client.search()/search_batch() (removed) -> query_points()/query_batch_points() - modal: container_idle_timeout/concurrency_limit/allow_concurrent_inputs -> scaledown_window/max_containers/@modal.concurrent; floor bumped to modal>=1.0 * fix(skills): sync mlops training/model-infra skills to current APIs Seven optional mlops training skills had stale APIs, config paths, image locations, and requirement pins. Verified against upstream and corrected: - torchtitan: removed TOML train_configs paths (replaced upstream by config registry) - trl-fine-tuning: PPO removed from TRL 1.x -> GRPO/RLOO; SFTTrainer tokenizer= -> processing_class - flash-attention: torch.backends.cuda.sdp_kernel (deprecated) -> torch.nn.attention.sdpa_kernel; corrected false FA3/FP8-in-pip claim (FA2 only) - accelerate: DeepSpeedPlugin instance not raw dict; --config_file expects accelerate YAML; auto_wrap_policy -> transformer_based_wrap - saelens: v6 nested training config (sae=/logger=); from_pretrained tuple -> from_pretrained_with_cfg_and_sparsity - tensorrt-llm: Docker Hub image 404 -> NGC nvcr.io; rc pin -> GA; CUDA req updated - nemo-curator: pip extras renamed; repo moved to NVIDIA-NeMo/Curator; 1.x pipeline rewrite noted * fix(skills): sync coding-agent CLI skills to current flags/packages Four coding-agent CLI skills drifted from their live CLIs. Verified against live --help/npm and corrected: - codex: --full-auto deprecated -> --sandbox workspace-write; --yolo -> --dangerously-bypass-approvals-and-sandbox (yolo kept as noted alias) - claude-code: --effort levels low/medium/high/xhigh/max (dropped removed 'auto', added 'xhigh'); fixed stray table cell - grok: --session-id is UUID-only for new sessions (cannot resume by name); rewrote the Session Continuation example; noted --max-turns now exists - blackbox: wrong npm package (@blackboxai/cli is unrelated) -> @blackbox_ai/blackbox-cli; removed dead source-repo link and phantom session/info subcommands * docs(design-md): sync skill with @google/design.md CLI 0.3.0 The design-md skill documented the Apr 2026 (0.1.x) CLI behavior, which has since drifted: - Lint rules: the skill listed 7 rules that no longer exist by those names (duplicate-section, invalid-color, wcag-contrast, unknown-component-property); the 0.3.0 linter runs 9 rules (contrast-ratio, orphaned-tokens, missing-primary, missing-typography, section-order, unknown-key, token-summary, missing-sections, broken-ref). Verified against live lint output. - Colors: any CSS color is now valid (oklch/rgb/named), not hex-only. - Export: json-tailwind (v3) + css-tailwind (Tailwind v4 @theme CSS) formats; 'tailwind' is a back-compat alias. New exit-code semantics (export exits 0 regardless of source lint findings). - Section order / duplicate headings are lint warnings, not file rejection (verified: duplicate + out-of-order sections exit 0). - Windows: documented the designmd dot-free bin alias (the design.md bin name collides with the .md file association); skill declares platforms: [windows]. - New pitfall: typography sub-property typos (fontwight) are silently dropped with no finding as of 0.3.0. All claims verified by running @google/design.md 0.3.0 live (lint, export, duplicate-section, oklch token, starter template lints clean). Docs page regenerated via generate-skill-docs.py. * fix(skills): sync bundled + misc CLI skills to current upstream Nine bundled and optional skills had stale flags, install URLs, packages, and paths. Verified each against upstream and corrected: - vllm: removed bogus --enable-metrics/--metrics-port (metrics at /metrics on API port); --speculative-model -> --speculative-config; canonical HF model IDs - lm-evaluation-harness: --tasks list -> lm-eval ls tasks; --allow_code_execution -> --confirm_run_unsafe_code - weights-and-biases: wandb.keras import removed -> wandb.integration.keras (WandbMetricsLogger); log_uniform -> log_uniform_values for raw values - huggingface-hub: upload-large-folder now deprecated; hf papers list -> ls - openhue: Linux install 404 -> openhue_Linux_x86_64.tar.gz tarball (release repo openhue/openhue-cli, v0.24) - apple-notes: memo notes -a is a bare flag, no positional title - excalidraw: upload.py path skills/diagramming/... -> skills/creative/... - searxng-search: removed Method 3 (searxng-data pip package is a PyPI 404) - sketch: noted get-shit-done upstream is archived/unmaintained * feat(desktop): date dividers in the sessions sidebar Group the flat recents list and entered-project lanes by recency: an unlabelled head of the newest run of sessions (cut at a real break in activity, sized toward the most recent handful), then one divider per coarse calendar range — Earlier today / Yesterday / Earlier this week / Last week / Earlier this month / month / month + year. Empty ranges are skipped, the first rendered group is never labelled, branch clusters never split, and hand-ordered lists / pinned / project previews stay divider-free. * fix(desktop): let the pinned sidebar section grow to fit all pins The pinned list was hard-capped at max-h-44 with an invisible scrollbar; cap it at half the viewport instead so every pin is visible. * feat(sessions): opt-in auto-archive of stale sessions + durable pin flag New sessions.auto_archive / auto_archive_days config: soft-hide (never delete) sessions with no activity for N days, aging on last activity rather than creation so an old-but-active chat is spared. Sweeps are throttled through state_meta and fire from CLI startup, gateway startup + hourly housekeeping, and the serve/dashboard backend (opportunistic on session list + an hourly lifespan ticker), so every surface honours one setting. A new pinned column (declaratively migrated) exempts sessions from the sweep; PATCH /api/sessions/{id} accepts pinned and flips the whole compression lineage as a unit, mirroring set_session_archived. * feat(desktop): auto-archive toggle + mirror sidebar pins to the backend Sessions settings gain an "Auto-archive stale chats" toggle with a configurable idle threshold, persisted to sessions.* in config.yaml so the backend sweep owns the policy. Sidebar pins (localStorage) are mirrored to the backend pinned flag at boot and on every change — pre-existing pins migrate transparently — so the sweep can never hide a pinned chat. * test(sessions): use mock.patch for the config gate, matching file idiom * test(gateway): account for the auto-archive construction-time sync escape The gateway startup maintenance block gained a maybe_auto_archive call in the same provably-off-loop __init__ site as maybe_auto_prune_and_vacuum; bump the reviewed sync-escape count from 3 to 4. * fmt(js): `npm run fix` on merge (#70845) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix: harden tui_gateway subprocess reads against Windows locale UnicodeDecodeError (#53137) * fix: address review — add regression test, revert cosmetic churn, cross-link #61595 * fix(windows): widen utf-8 subprocess decode guard to sibling desktop-backend sites The salvaged #61978 covers tui_gateway/server.py. The crash reported on Jul 24 came from a sibling site it doesn't touch: the desktop update panel's _recent_upstream_commits() in hermes_cli/web_server.py runs git log with text=True and no encoding. Commit 84db32484f put a bug emoji (UTF-8 f0 9f 90 9b) in a subject on main; byte 0x90 is undefined in cp1252, so every Windows desktop install behind that commit crashed in subprocess._readerthread during the update check (#52649). Guard every text=True capture site in the desktop-backend process with encoding='utf-8', errors='replace': - hermes_cli/web_server.py: git log update panel, memory-provider setup runner, WhatsApp bridge npm install, docker probe - hermes_cli/banner.py: all 5 git sites (update check runs at startup) - tui_gateway/host_supervisor.py: build-sha probe, ps probe, compute-host Popen drain threads - tui_gateway/compute_host.py: build-sha probe, ps rss probe * Reapply "Merge pull request #30179 from NousResearch/feat/iron-proxy" This reverts commit c6dc7c03c355fb3a407c1309aabebb13520c9efd. * test(docker): update network-reuse harness fake ps output for egress-aware 3-field probe test_docker_network_config.py landed on main after the #58489 revert and stubbed docker ps with the 2-field ID\tState format. The re-landed egress-aware reuse probe requests ID\tState\tEgressLabel when egress is off, so the fake line failed to parse and the reuse path never fired. Fixture-only change; production behavior is unchanged. * feat(api): honor provider-aware request routing Carry model, provider, and model_options through the API server's execution surfaces (session chat, Chat Completions, Responses, /v1/runs) without mutating global configuration. Precedence: session /model override -> model_routes alias -> direct request selection -> global defaults. Conflicting route/provider mixes fail closed with 400. model_options stays request-scoped regardless of which selection wins. Salvaged from PR #54426 by @abundantbeing. * fix(api): gate bare-model passthrough + route-alias model leak Follow-ups on the salvaged #54426 routing contract: - Bare `model` without `provider` on the OpenAI-compatible endpoints (/v1/chat/completions, /v1/responses) is now opt-in via gateway.platforms.api_server.direct_model_requests (default off) — generic OpenAI clients hardcode model names ('gpt-4o', ...) and existing deployments rely on those falling back to the gateway default. Explicit `provider` requests and the Hermes-native session-chat + /v1/runs surfaces are always honored. Idea credit: PR #22825 by @mssteuer. - A model_routes alias with no `model` key can no longer leak the alias string as the executing model name (defensive; parse-time validation already drops such routes). - Fix mis-indented _run_agent call args in _handle_session_chat_stream. - Docs: document the opt-in flag. * feat(desktop): add Arabic (ar) locale with RTL support Arabic is the desktop app's first right-to-left locale. The i18n provider now sets `document.dir`/`lang` from the active locale so Tailwind logical utilities flip automatically, and `ar` is registered in the catalog, language options, and alias table. The catalog is a partial `defineLocale` so keys added to English later fall back cleanly. Co-authored-by: 3ssiri <assiri@gmail.com> Co-authored-by: Da7-Tech <286182457+Da7-Tech@users.noreply.github.com> * feat(web): add Arabic (ar) locale with RTL support Adds the Arabic catalog to the dashboard, registers it in the locale list and picker, and flips the document direction to RTL when Arabic is active. Introduces a `defineLocale` merge helper (mirroring the desktop app) so the Arabic catalog can be a partial override that falls back to English for any untranslated key instead of hand-porting every future string. Co-authored-by: morolab <ahmedmoro@gmail.com> * feat(i18n): add Arabic (ar) catalog for agent/CLI messages Registers `ar` in the supported-language set and alias table and ships locales/ar.yaml at full key and placeholder parity with en.yaml, covering approval prompts and gateway slash-command replies. Identifiers, commands, paths, config keys, model/provider names, and {placeholder} tokens are kept verbatim. Co-authored-by: Da7-Tech <286182457+Da7-Tech@users.noreply.github.com> * fix(auth): stop stale-key credential recovery loops Track the selected credential by stable pool entry ID so token refreshes and shared cursor movement cannot detach failures from the entry that issued them. Stop unmatched single-entry pools from reporting a no-op rotation as successful recovery. Co-authored-by: Maxim Esipov <maksesipov@gmail.com> * refactor: extract sync_credential_pool_entry_id helper Replace 3 duplicated entry_id resolution blocks (try/except + entry_id_for_api_key + fallback to None) in agent_init.py, chat_completion_helpers.py, and switch_model with a single sync_credential_pool_entry_id(agent) function in agent_runtime_helpers. Follow-up to #70323. * fix(url_safety): allow DNS failure in proxy/sandbox environments When the runtime blocks direct DNS (NVIDIA OpenShell, Docker + Squid, corporate proxy with DNS-only-via-proxy), socket.getaddrinfo() fails and is_safe_url() blocks *all* requests — including legitimate public URLs via the configured proxy. Add _proxy_is_configured() helper that checks HTTPS_PROXY, HTTP_PROXY, http_proxy, https_proxy, ALL_PROXY, all_proxy. When DNS fails AND a proxy is configured, delegate DNS resolution to the proxy rather than blocking outright. Blocked hostnames (metadata.google.internal, 169.254.169.254, etc.) are checked BEFORE DNS resolution, so cloud metadata endpoints remain blocked regardless of proxy status. Fixes #32217 * fix(url_safety): harden proxy DNS delegation — literal IPs stay fail-closed + regression tests Follow-up on the salvaged #68469 commit: - Literal-IP hostnames never take the proxy DNS-delegation path (a getaddrinfo failure on a literal IP is not a proxy-environment symptom, and IPs need no DNS) — keeps the private-IP/metadata floor intact under proxy env vars. - Adds TestProxyEnvironmentDnsDelegation: delegation fires only for hostnames, metadata hostname/IP floor holds, DNS-success path unchanged, empty proxy var ignored. - Guards the three pre-existing DNS-failure tests against ambient proxy env vars so they don't flake on developer machines. * fix: apply _rewrite_compound_background in spawn_local to prevent worker deadlock on server backgrounding Issue #68915: when the agent runs a compound command with trailing & (e.g. `cd /app && node server.js &`), bash parses it as `(A && B) &` — a subshell that holds the stdout pipe open forever when B is a long-running server. The existing _rewrite_compound_background in terminal_tool.py correctly rewrites this to `A && { B & }` to avoid the subshell fork, but it was only applied in the foreground execute() path (tools/environments/base.py). The background spawn_local() path bypasses base.py entirely and passed the raw command directly to Popen/PTY, leaving the deadlock unmitigated. Fix: apply _rewrite_compound_background in spawn_local() before the command is passed to Popen or PTY spawn. Uses a lazy import to avoid circular dependency (terminal_tool imports process_registry). - PTY spawn path: now uses safe_command (rewritten) - Popen spawn path: now uses safe_command (rewritten) - Session.command still stores the original (unrewritten) command for display - Simple `cmd &` is left unchanged (no subshell bug) Tests: 4 regression tests verifying (1) compound is rewritten, (2) simple bg is preserved, (3) multi-line compounds are rewritten, (4) session.command stores original. * fix(telegram): prevent connect hang with retry watchdog and fresh app per attempt (#67498) The Telegram adapter's connect retry loop could silently stall after 'Connecting to Telegram (attempt 1/8)...' with the event loop permanently parked in select() — all threads idle, no attempt 2/8 ever scheduled. Root cause analysis: - The retry loop reused the same Application object across all 8 attempts. After a failed initialize() the app could be in a partially- initialized state (closed httpx transports from , or flag set before the hang) causing subsequent calls to silently skip real initialization. - CancelledError (a BaseException, not an Exception) propagated silently through all except handlers with no logging — the task driving the retry loop could exit without any trace. - No total watchdog bound existed for the entire retry loop; only per-attempt timeouts via _await_with_thread_deadline. If the loop itself stalled between attempts (between-attempt sleep, cleanup, or scheduling), there was no timeout to catch it. Fixes: 1. **Total watchdog deadline**: Compute a total deadline for the entire connect loop (8 attempts × init_timeout + 120s margin). Before each attempt, check the wall clock; if exceeded, raise OSError immediately instead of attempting another initialize(). 2. **Fresh Application per retry**: On each failed attempt, rebuild via and re-register all handlers. The old app is best-effort shutdown with . This ensures each retry starts with a clean slate — no stale transports, no stale flag, no leaked state from the previous attempt. 3. **BaseException logging + propagation**: Added (placed LAST after all other handlers) to log CancelledError and other non-Exception signals before propagating. Previously these exited the retry loop silently with no log message. 4. ** block for app rebuild**: The clause runs after every failed attempt that isn't the last, rebuilding the app and discarding the old one regardless of which exception class caused the failure. * chore: add contributor email mapping for agent@hermes.dev -> webtecnica * fix(tui): refuse empty prompt.submit truncation without confirm Stale truncate_before_user_ordinal=0 from a desynced Desktop client resolved to history[:0] and replace_messages() wiped the durable transcript. Require confirm_empty_truncate for that edge and have intentional first-turn restore/regenerate paths send it. * test(tui): cover empty truncate guard on prompt.submit Refuse ordinal-0 wipes without confirm_empty_truncate; allow the opt-in path used by first-turn restore/regenerate. * fix: use error code 4028 (4025 already taken by session.handoff) * fix(gateway): deliver relay-backed homes after restart * fix(tui_gateway): recover custom provider identity from the session's model name A session pinned to a named custom provider could silently reroute to the user's default provider on resume/rebuild. Session rows persist the RESOLVED provider — bare "custom" for every named providers:/custom_providers: entry — and when no base_url survived in model_config, the existing heal (canonical_custom_identity) had only the config.model.provider fallback left. For users whose global default is a BUILT-IN provider (e.g. nous) that tier cannot fire, so the bare provider was dropped, resume fell back to the default provider with the session's custom model name, and the default endpoint 404'd with "Model '<x>' not found. The requested model does not exist in our configuration or OpenRouter catalog." Re-selecting via /model fixed it until the next resume — the reported symptom. Add a model-name recovery tier between the base_url reverse-lookup and the config fallback: find_custom_provider_identity_by_model() maps the stored model back to the entry that serves it (model/default_model/models catalog, dict and legacy list shapes). The session row always stores the model, so the entry identity survives even when the row has no base_url AND the global default points elsewhere. All five bare-custom heal sites in tui_gateway/server.py now pass the model: _ensure_session_db_row, _stored_session_runtime_overrides, _runtime_model_config, _make_agent, and _model_picker_context. * fix(fallback): allow xai-oauth → xai failover on shared host/model Base-url+model dedup was meant for custom shim aliases, but it also skipped first-class providers that share an inference host while using different credentials. That stranded xai-oauth spending-limit failover to the xai API-key provider when both used the same model slug. * test(fallback): cover xai-oauth → xai same-host same-model failover Pin that a configured xai API-key fallback still activates when the primary xai-oauth runtime shares api.x.ai and the same model slug. * test: tighten spawn-rewrite assertions and add PTY-path coverage Follow-up to salvaged PR #70549: - Replace fragile 'or' assertions with single precise checks that catch partial-rewrite regressions (would have masked a missing closing brace) - Add test_pty_path_uses_rewritten_command covering the PTY spawn path that was modified but previously untested * chore: fix contributor attribution for desktop PR * feat(desktop): add "Connect to existing Hermes" option to first-run onboarding Adds a first-run Desktop choice between installing Hermes locally and connecting to an existing remote Hermes gateway. The choice appears after backend resolution but before ensureRuntime(), so selecting remote cannot accidentally trigger local bootstrap. New modules: - first-run-setup-gate: concurrent first-run decision gate and reset semantics - primary-backend-startup: Electron-free orchestration seam (saved remote resolution, gate decision, remote re-resolution, local continuation) - primary-connection-rehome: prevents dual-owner race where both cold boot() and renderer softSwitch() could connect simultaneously - first-run-remote-form: extracted remote form with stale-result guards Reuses existing connection-config IPC, encrypted token storage, OAuth session partition, and primary backend resolution. Fixes #38602 Fixes #36970 * feat(desktop): add Webhooks page for subscription CRUD (#69687) * feat(desktop): add Webhooks page for subscription CRUD Brings the desktop GUI to parity with the dashboard's Webhooks page. Adds a /webhooks route that lists webhook subscriptions, enables the webhook gateway platform, and creates/toggles/deletes subscriptions, hitting the same /api/webhooks* endpoints the dashboard and CLI use. - types/hermes.ts: WebhookRoute, WebhooksResponse, WebhookCreatePayload, WebhookCreateResponse, WebhookEnableResponse - hermes.ts: getWebhooks, enableWebhooks, createWebhook, deleteWebhook, setWebhookEnabled (profile-scoped) + type re-exports - app/webhooks/index.tsx: WebhooksView (enable card, restart banner, subscription list with copy/toggle/delete, create dialog with one-time secret reveal); optimistic toggle, profile re-home - routing: routes.ts, contrib/surfaces.tsx, chat/route-tile.tsx - nav: command palette, keybinds (nav.webhooks), sidebar row - i18n: en + zh full, types interface; ja/zh-hant fall back to English - test: webhooks-rest.test.ts covers the REST helper contracts * feat(desktop): surface Webhooks in the status bar instead of nav Moves the Webhooks entry point from the sidebar nav / command palette / keybind to a status bar action next to Cron, matching where scheduled jobs live. The /webhooks route, page, and REST helpers are unchanged. - use-statusbar-items.tsx: add webhooks action (Globe icon) after cron - i18n: shell.statusbar.webhooks / openWebhooks (en, zh, types) - revert nav wiring: sidebar row, command palette entry, nav.webhooks keybind action + label, commandCenter.nav webhooks entry * feat(desktop): add skills field to webhook create form Exposes the backend's per-subscription skills list in the create dialog (comma-separated) and shows skill badges on subscription rows, so this page covers the skill-backed endpoint case as well as general CRUD. - create form: Skills input; passes skills[] to createWebhook - rows: render skill badges - i18n: fieldSkills / fieldSkillsPlaceholder (en, zh, types) Consolidates the skill-endpoint framing from #42817. Co-authored-by: LionGateOS <98371158+LionGateOS@users.noreply.github.com> * feat(desktop): render Webhooks as an overlay instead of a full page Webhooks took over the whole workspace/chat pane because 'webhooks' was missing from OVERLAY_VIEWS, so it routed through PageSearchShell while cron rendered through the Panel overlay. Add it to OVERLAY_VIEWS and wire it up like cron: mount WebhooksView as a floating overlay in wiring.tsx, render null for the webhooks route in the workspace table, and convert WebhooksView from PageSearchShell to the Panel primitive with an onClose. Drop webhooks from route-tile BUILTIN_PAGES so it can't be tiled as a page either. * fix(desktop): place webhook URL copy button next to the URL The URL span used flex-1, stretching it across the row and pushing the copy button to the far right. Drop flex-1 so the span sizes to content and the copy button sits directly after the URL. * fix(desktop): top-align the deliver-only checkbox with its wrapped label The label used h-9 items-center, centering the checkbox against the two-line hint text. Switch to items-start so the checkbox aligns to the first line. * feat(desktop): give Webhooks a cron-style master/detail layout Replace the single-column subscription list with the same Panel master/detail cron uses: a left PanelList of subscription rows (status dot + kebab menu) capped by a PanelAddButton, and a right PanelDetail showing the selected subscription's deliver/events/skills, URL with copy, description, and prompt. Enable/restart banners sit above the body; the empty state keeps its own New subscription action. * fix(desktop): drop header on the Webhooks empty state to match cron The zero-subscriptions state still rendered the PanelHeader with the title and the refresh/new buttons on the right. Remove it so the empty state is just the centered PanelEmpty (icon, message, New subscription), matching the cron empty state. Also drop the header from the loading state. * fix(desktop): top-align deliver-only checkbox and its wrapped label Drop the min-h-9/pt-1.5 baseline shim that pushed the row down and made the wrapped hint look misaligned. Use plain items-start with a mt-0.5 on the checkbox so it sits at the first line, and wrap the hint in a leading-snug span. * fix(desktop): drop header refresh/new buttons; + button owns create flow The populated Webhooks header carried a refresh icon and a New subscription button. Remove both — the PanelAddButton at the bottom of the list is the create flow, matching cron. Profile-change reload and the refresh hotkey still run; the restart banner keeps its own refresh. * fix(desktop): nudge deliver-only checkbox down 2px Bump the checkbox top margin from mt-0.5 to mt-[4px] so it sits level with the first line of the wrapped hint. * refactor(desktop): reuse shared primitives on the Webhooks page Address OutThisLife's review — stop reinventing primitives the app already ships: - copy: drop the local navigator.clipboard button for the shared CopyButton (routes through the Electron clipboard bridge + haptic + error state instead of swallowing failures) - banners: enable/restart callouts now use Alert variant=warning (primary color-mix tokens) instead of a hand-rolled amber palette - toggle: detail Enable/Disable is a Switch (messaging idiom), not a text ghost button - checkbox: create dialog uses the Checkbox primitive, not a raw input Rows/chips already moved to PanelListRow/PanelDetail/PanelPill in the earlier cron-layout pass. Left the main-list fetch on manual load and the local Field helper: cron itself does both, so useQuery here would diverge from the reference idiom rather than align with it. * refactor(desktop): move Webhooks fetch to the react-query layer Replace the manual useState/useEffect load with useQuery keyed by ['webhooks', profileScope] — profile change re-fetches automatically, no effect. reload() invalidates the query; the optimistic toggle writes the cache via queryClient.setQueryData then invalidates so backend truth wins. Load failures surface via an error-watching effect (react-query v5 dropped useQuery onError). Refresh hotkey calls refetch(). Left the create-dialog Field helper as-is: settings ListRow is a side-by-side settings row, wrong for a stacked dialog form, and cron's editor dialog (the reference) defines the same local Field. * refactor(desktop): drop bespoke Field for shared ListRow/ToggleRow Remove the local Field wrapper entirely. Every create-dialog field now uses settings ListRow (wide, so label stacks over the full-width control), the deliver-only pref uses ToggleRow (ListRow + Switch, haptic baked in) instead of a bare Checkbox, and the created URL/secret reveal rows use ListRow too. No component in this file is hand-rolled anymore. * fix(desktop): pair webhook create fields into a 2-column layout The single-column dialog scrolled awkwardly. Group fields: name + description side by side, prompt full-width under them, events + skills side by side, deliver-to + deliver-only side by side. Drop the deliver-only help text and rename the label to 'Deliver payload only' (remove the now-unused fieldDeliverOnlyHint i18n key from en/zh/types). * fix(desktop): align Webhooks page with cron conventions and DESIGN.md - Delete copy uses deleteDescPrefix + bolded name + deleteDescSuffix (no em-dash) - Drop the duplicated detail-header Switch + Trash2; enable/disable and delete live only in the row kebab, matching CronJobDetail - Collapse three copyable-value chromes into one flat token-backed CopyValueRow - Delete success toast gains a w.deleted title - Replace hand-rolled delete Dialog with shared ConfirmDialog --------- Co-authored-by: LionGateOS <98371158+LionGateOS@users.noreply.github.com> * fmt(js): `npm run fix` on merge (#70914) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(api-server): expose model options inventory Add authenticated GET /api/model/options to the gateway API server, sharing the dashboard/TUI picker payload builder so external clients can sync to the user's configured Hermes provider catalog instead of scraping the single OpenAI-compatible /v1/models alias. - new shared hermes_cli.inventory.build_model_options_payload() wraps build_models_payload with the stable picker shape and safe custom-provider probe policy (probe current only on normal open, probe all + cache bust on explicit refresh) - dashboard web_server and TUI gateway model.options refactored onto the shared builder; dashboard build moved off the event loop via run_in_threadpool - capabilities endpoint advertises model_options - docs for both API server and programmatic integration Salvaged from PR #54689 by @abundantbeing. * feat(desktop): add Cron Blueprints to the GUI (#70066) * feat(desktop): add Cron Blueprints to the GUI The desktop app had a Cron jobs panel but no Blueprints tab, so the parameterized automation templates the dashboard offers were unreachable there (parity matrix: Dashboard=Y, GUI=N). Adds a Jobs/Blueprints segmented toggle to the cron panel. The Blueprints tab renders the catalog from the existing GET /api/cron/blueprints endpoint, one card per template with a typed form (time/enum/weekdays/text slots). Submitting POSTs to /api/cron/blueprints/instantiate, which fills the slots and creates a real cron job via the same create_job path as a hand-written one. The new job is merged into the shared atom so the Jobs tab and sidebar reflect it immediately. Backend already served both endpoints; this is desktop frontend only. Strings added to all four locales. * fix(desktop): stop cron blueprint cards clipping and tabs overlapping close X Blueprint cards were wrapped in PanelBlock (a max-h-48 overflow-auto <pre> for monospace code), which capped each card and forced an inner scroll, clipping the copy. Use a plain auto-height card div so rows grow with their content and the gallery scrolls as one. PanelHeader actions sat under the overlay's absolutely-positioned close X (no layout space reserved). Reserve pr-8 clearance when actions are present. * refactor(desktop): narrow blueprint card i18n dep, document intentional scoping Address review nits on the Cron Blueprints PR: - BlueprintCard's submit useCallback depended on the whole t.cron object; it only uses the blueprints slice. Bind const b = c.blueprints and use it (plus narrow the dep) throughout the card. - Document the intentional GET-vs-POST profile asymmetry on the blueprint endpoints (global catalog vs per-profile instantiate) — the prior comment claimed both were profile-scoped. - Note why the blueprints tab collapses 'all' scope to 'default' (a blueprint creates a real per-profile job; 'all' is not a writable target). * fix(desktop): default blueprint delivery to This desktop, not origin The blueprint catalog is shared with the dashboard, so its deliver slot defaults to 'origin' (the chat/home-channel a dashboard or gateway job was created from). Desktop has no origin chat and no home-channel picker, so the seeded 'origin' rendered unlabeled and, at delivery, fell through the home-channel fallback to nowhere when no gateway was configured. Seed the deliver slot to 'local' (This desktop) when the backend default is 'origin' or empty, drop the origin option from the desktop dropdown, and label the remaining options with the desktop's own delivery labels — matching the manual cron editor (local/telegram/discord/slack/email). Also skip the backend's origin-centric deliver help, which contradicts desktop semantics. * refactor(desktop): align blueprint cards with the Panel/settings idiom Address PR review (UI consistency with neighboring surfaces): - Card container: drop the standalone-card look (bg-foreground/5) for the shared in-panel grouping token bg-(--ui-bg-quinary), matching the cron editor's in-surface groupings so blueprints sit in the Panel family. - Form fields: replace the bespoke <label>+<Input> rows with the shared ListRow primitive from settings/primitives (label+help on the left, control on the right, stacks in a narrow pane) — same idiom as settings/messaging. No behavior change; blueprint deliver remap and $cronJobs merge untouched. * fix(desktop): satisfy eslint on the cron blueprint files CI check:lint failed on import/export ordering and an unused import in the blueprint changes: - hermes.ts: sort AutomationBlueprint before AuxiliaryModelsResponse in the type import + re-export blocks, and drop the unused AutomationBlueprintField import (still re-exported for blueprints.tsx). - cron/index.tsx: alphabetize the dialog/segmented-control imports and the ./blueprints vs ../shell/statusbar-controls group. - blueprints.tsx: add the required blank lines between statements. eslint --fix only; no behavior change. typecheck + blueprint tests green. * refactor(desktop): blueprints reuse the cron editor dialog + shared card Address review: the blueprint UI was still going its own way on the card and form. Reuse the app's canonical pieces instead of a bespoke surface. - CronEditorDialog gains a 'blueprint' mode: EditorState carries the blueprint + target profile, the dialog renders the typed slots with the same Field/FieldHint/DialogFooter/error-block chrome as manual New cron, and submit routes to instantiateAutomationBlueprint. One dialog, one editor state machine. Resolves the accordion, the border-t divider, ListRow-vs-Field, the ad-hoc buttons, and the plain error <p> in one move. - Blueprint cards use selectableCardClass({ prominent: true }) (the shared theme/pet/gateway card idiom), caller owns padding (p-2), whole card is a button that opens the dialog pre-filled. No inline form. - Gallery renders via PanelDetail, not PanelBody's master/detail row. - i18n: drop the now-unused blueprints.setUp/cancel, add blueprints.dialogDesc across en/ja/zh/zh-hant + types. - Dropped the stray \u2014 literal comment. Logic (origin->local deliver, desktopDeliverOptions, merge into $cronJobs) is unchanged and stays unit-tested. typecheck + eslint + tests green. * fix(desktop): dropdown no longer closes the cron dialog; unify deliver targets Two cron-dialog bugs: 1. Dismissing any Select dropdown inside the cron editor dialog closed the whole dialog. Radix portals Select/Popover content outside the dialog, so the dismiss pointerdown reached the Dialog's DismissableLayer as an outside-interaction. Guard DialogContent.onInteractOutside: swallow interactions originating from a [data-radix-popper-content-wrapper] (a dropdown dismiss inside our own dialog), compose with any caller handler. Fix is at the shared Dialog level so every dialog benefits. 2. Blueprint deliver only offered 'This desktop'. The blueprint used the backend blueprint field.options (configured gateways only) while the manual editor hardcoded local/telegram/discord/slack/email regardless of what's connected. Wire the desktop to GET /api/cron/delivery-targets (the documented single source of truth, already used by the dashboard) via getCronDeliveryTargets, and render both the manual editor and the blueprint deliver slot through one shared DeliverSelect. Now all three surfaces agree and only offer connected platforms; unconfigured-home-channel targets show a hint. i18n: add cron.deliverNeedsHomeChannel across en/ja/zh/zh-hant + types. typecheck + eslint + vitest green. * fix(desktop): clicking away from an open dropdown no longer closes the dialog The onInteractOutside guard only caught pointerdowns whose target was inside the popper wrapper. But dismissing an open Select by clicking elsewhere inside the dialog also closes the popover, which moves focus — and Radix Dialog reads that as focusOutside and closes the whole dialog. (Radix Select 2.3.1 has no modal prop, so that escape hatch isn't available.) Guard both paths at the shared DialogContent level: onInteractOutside AND onFocusOutside now swallow the event when it originates from a Radix popper OR when any [data-radix-popper-content-wrapper] is open at event time (covers the focus/re-dispatch case where the target is no longer the popper). A genuine backdrop click with no dropdown open still closes the dialog. Export isInteractionFromPopper + unit-test the three cases. typecheck + eslint + vitest green (7 dialog tests). * fix(desktop): portal popovers into their dialog so dropdowns don't close it Root cause (affected every dialog, not just cron): Radix Select/Popover/ DropdownMenu portal to document.body — a SIBLING of the dialog, outside its DOM subtree. Dismissing a dropdown (or clicking another field) moves focus out of the dialog subtree, which the Dialog's modal FocusScope/DismissableLayer reads as an outside interaction and closes the whole dialog. Separate body-level portals also make z-index across the two fragile. The earlier onInteractOutside/onFocusOutside guards treated symptoms and didn't hold (and Radix Select 2.3.1 has no modal prop to disable its layer). Real fix is a layering system: DialogContent publishes its content node via DialogPortalContainerContext; SelectContent/PopoverContent/DropdownMenuContent call usePopoverPortalContainer() and portal INTO that node when inside a dialog (document.body otherwise). The popover is then a true DOM descendant of the dialog — focus stays in, dismissal no longer closes the dialog, and both share one stacking context so z-index is deterministic. Test: with a Dialog open, an open Select's item is a descendant of the dialog (portalled in), verified in jsdom. typecheck + eslint + component tests green. * fix(desktop): bump radix-ui so dismissing a dropdown can't close its dialog Upstream bug, not app-layer: with radix-ui 1.6.0 (dismissable-layer 1.1.13), an open modal Select sets pointer-events: none on the dialog body, so a click anywhere inside the dialog hit-tests through to the overlay. The Dialog's DismissableLayer defers its outside-pointerdown decision to the click, but the overlay is a registered dismissable surface exempt from the interception check — so the Select swallowing the press didn't count, the deferred onDismiss fired, and the dialog closed along with the dropdown. dismissable-layer 1.1.17 adds shouldHandlePointerDownOutside, which makes the dialog's layer ignore the press entirely while a higher layer has its pointer events disabled. Bump radix-ui ^1.4.3 -> ^1.6.5 (dismissable-layer 1.1.17, dialog 1.1.21, select 2.3.5); lockfile diff is Radix-only. Repro test fires the pointerdown/up/click sequence on the overlay with a Select open inside the dialog: red on 1.6.0 (dialog closed), green on 1.6.5. Counterpart test keeps a genuine overlay click closing the dialog. typecheck + dialog/select component tests green. Full-suite failures on this Windows host reproduce identically without the bump (pre-existing env issues). * fmt(js): `npm run fix` on merge (#70927) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix: add explicit UTF-8 encoding to all subprocess text=True calls (#53428) On Windows with Chinese locale (GBK), subprocess.run(text=True) without explicit encoding causes UnicodeDecodeError crashes. This fix adds encoding='utf-8', errors='replace' to all subprocess.run() and subprocess.Popen() calls that use text=True across 76 non-test Python files. Fixes #53428 (master tracker for Windows GBK locale crash). Note: credential_pool.py and electron changes excluded per reviewer request — those will be submitted as separate focused PRs. * fix: add explicit UTF-8 encoding to _op_whoami subprocess call (#53428) PR #55339 adds encoding='utf-8', errors='replace' to 26 subprocess.run(text=True) call sites across the codebase. The triage review (thanks @alt-glitch) diffed this PR against #55339 and found that 5 of the 6 originally-touched call sites are already covered there byte-identically: - hermes_cli/main.py::_probe_container - hermes_cli/setup.py SSH probe - tools/tts_tool.py::_generate_neutts - tools/transcription_tools.py::_prepare_local_audio - tools/transcription_tools.py::_transcribe_local_command (both branches) The one genuinely net-new site — hermes_cli/onepassword_secrets_cli.py::_op_whoami (the 1Password op CLI whoami probe) — is NOT in #55339 and is fixed here. Without explicit encoding=, text=True decodes child output with locale.getpreferredencoding(False) — cp936 on Chinese Windows — which crashes _readerthread on non-GBK bytes, cascading into pipe buffer fills, event loop stalls, and TUI freezes (issues #47939, #53428, #57238). Scope narrowed per triage feedback: the other 5 sites should land via #55339. Refs #53428 (together with #55339). * fix: extend UTF-8 encoding to _op_version probe (#53428) Hermes-sweeper review on #60741 flagged that _op_version (the paired op probe used by the same setup/status CLI flow at lines 127 and 205) still ran text=True without explicit encoding/errors. Add encoding='utf-8', errors='replace' to match _op_whoami and the production op read path at agent/secret_sources/onepassword.py:271-278. Also extend the regression test to cover _op_version alongside _op_whoami, and update the module docstring to reflect the widened scope. Test sensitivity verified: reverting the source change makes test_op_version_passes_utf8_encoding fail with encoding=None. * feat(linter): detect subprocess text=True without explicit encoding= Adds a new rule to scripts/check-windows-footguns.py that flags subprocess.run/Popen/call/check_output/check_call(..., text=True, ...) calls missing an explicit encoding= kwarg. On Chinese Windows (cp936/GBK) and other non-UTF-8 default codepages, text=True without encoding= decodes child output with locale.getpreferredencoding(False), crashing _readerthread with UnicodeDecodeError on non-default-codepage bytes (issues #47939, #53428, rule prevents future regressions. Rule design: - Pattern matches 'text=True' / 'text = True' - post_filter skips lines that: - already pass encoding= on the same line - are method definitions (def text) - contain text=True inside string literals - are not subprocess-shaped calls (heuristic via _is_likely_subprocess_call) - Two helper functions: _is_likely_subprocess_call, _looks_like_string_literal - Multi-line calls where subprocess.X( and text=True are on different lines are not flagged (acceptable false negative for a line-based scanner) Also fixes the linter's own footgun: get_staged_files() and get_diff_files() used subprocess.check_output(text=True) without encoding= — now fixed. Suppresses 4 false positives on non-Windows platform-exclusive calls: - tools/voice_mode.py (Termux/Android) - tools/environments/singularity.py (Linux HPC) - plugins/google_meet/cli.py (macOS system_profiler) Test plan: - 21 unit tests in tests/scripts/test_footgun_subprocess_encoding.py - TestDetection: 6 cases verifying the rule flags real subprocess calls - TestSuppression: 7 cases verifying false-positive avoidance - TestHelpers: 7 cases for the two helper functions - TestFullRepoScan: scans the whole tree and asserts the new rule finds only the 7 call sites that PR #60741 fixes (or zero, once #60741 merges) Verified: full-repo scan reports 7 matches on main (the #60741 sites), 4 platform-exclusive calls correctly suppressed, zero false positives. * fix(windows): sweep remaining unguarded text-mode subprocess sites codebase-wide AST-driven pass over every subprocess.run/Popen/check_output/check_call/call with text=True (or universal_newlines=True) and no explicit encoding=: append encoding='utf-8', errors='replace' at the kwarg site. 136 call sites across 28 files (cli.py, hermes_cli/main.py, tools_config.py, environments, computer_use, gateway, scripts, skills helpers, agent/*). Together with the salvaged #55339/#60741 commits this closes out issue #53428's bug class; the salvaged #60751 linter rule in check-windows-footguns.py now enforces it repo-wide (verified: 807 files scanned, zero findings). * chore: map jinglun010@gmail.com -> jinglun010-cpu * fix: repair sweep fallout — duplicate encoding kwargs, non-subprocess call sites, kwarg-snapshot tests - Strip the salvaged commit's inline encoding kwargs where main had since gained its own (process_registry, local env, cua doctor, gateway, commands, gateway_windows — the latter keeps its locale-aware _schtasks_encoding() from #38186) - Revert encoding kwargs mistakenly applied to non-subprocess APIs (exa get_contents, tempfile.mkstemp in webhook.py) - Guard the ddgs worker Popen (new on main since #55339) - Update two kwarg-snapshot test assertions for the new kwargs * chore: map stoltemberg@users.noreply.github.com -> Stoltemberg * test: update kwarg-snapshot assertions for the utf-8 subprocess guard - whatsapp taskkill + webhook gh-comment assert_called_with: add the two new kwargs - test_status fake_run: accept **kwargs so signature-strict stub doesn't TypeError on encoding/errors * fix(cron): respect the platform-conditional decode design in _run_job_script + taskkill kwarg snapshot cron/scheduler.py deliberately applies utf-8/replace only on Windows via popen_kwargs (non-Windows keeps locale default per its test contract) — drop the sweep's unconditional inline kwargs there. Update the gateway force-kill kwarg snapshot for the new guard. * fix(api_server): close divergence gaps from gateway/run.py Three parity fixes between the API server and the native gateway's agent-runtime resolution, integrated with the provider-aware request routing that landed in #70853: - Session-persisted model is honored: POST /api/sessions {"model": ...} stores a model that the chat handlers previously fetched and threw away. A stored value that matches a model_routes alias goes through the route path (route provider/credentials apply); a raw model string threads through as session_model, pinning the session's turns ahead of per-request body values but below an explicit session /model override. - Empty-model recovery: provider-catalog default when config has no model.default but a provider resolved, plus last-known-good model recovery (#35314) keyed on gateway_session_key only (never ephemeral session_id — no unbounded growth from one-off requests). - Provider auth failures surface as controlled responses: RuntimeError from _resolve_runtime_agent_kwargs() is re-raised as a dedicated _ProviderAuthResolutionError at the call site, caught narrowly in _run_agent() and the /v1/runs executor to return run.py's response shape instead of an undifferentiated 500 (session-chat endpoints previously returned a raw aiohttp 500 with no JSON body). Salvaged from PR #57947 by @FvanW; session-model route-alias resolution from PR #59941 by @kaishi00. Co-authored-by: kaishi00 <kaishi00@users.noreply.github.com> * feat(models): add anthropic/claude-opus-5 to OpenRouter and Nous Portal catalogs Anthropic released Claude Opus 5 (+ -fast variant) — both are live on OpenRouter and the Nous Portal /models endpoint (verified against both live APIs). Opus 4.8 entries are kept. - hermes_cli/models.py: opus-5 + opus-5-fast in OPENROUTER_MODELS; opus-5 in _PROVIDER_MODELS[nous] (Portal serves both, curated list carries the base model like the rest of the Nous Anthropic block). Ordering: below fable-5 flagship, above opus-4.8. - agent/model_metadata.py: claude-opus-5 -> 1M context (matches live OpenRouter metadata). - agent/reasoning_timeouts.py: claude-opus-5 -> 240s stale-timeout floor (same as the opus-4.x thinking family). - website/static/api/model-catalog.json: regenerated via scripts/build_model_catalog.py. Both providers bill via official_models_api (live pricing), so no _OFFICIAL_DOCS_PRICING snapshot entry is needed for these routes. * feat(api): backend-acknowledged session model lock with runtime routing Add a persisted, backend-confirmed provider/model lock for Hermes Browser and other session API clients. A confirmed lock is an execution contract rather than response metadata: - POST /api/sessions/{session_id}/model validates and persists a confirmed browser_model_lock (advertised in /v1/capabilities) - session chat + chat/stream consume the persisted lock on body-only follow-up turns; a confirmed lock wins over an older gateway session /model override and the session-persisted model - a later successful session /model switch explicitly clears and replaces the lock while preserving lineage markers (_branched_from) and invalidating cached system-prompt model/provider metadata - ordinary one-off request overrides never replace a confirmed lock - provider-resolution failure fails closed as a typed provider-auth error (controlled response, never global-credential reuse) - confirmed locks disable the global fallback model chain - the completed agent's actual provider/model must match the locked route or the turn fails with a runtime-mismatch error - responses carry sanitized runtime metadata reporting actual vs requested provider/model and lock state Rebased onto the provider-aware request routing (#70853) and session-model parity (#70931) that landed since the original branch; the lock now slots into that precedence chain as the top rung. Salvaged from PR #61236 by @abundantbeing. * fix(mcp): use encoding_error_handler='replace' for stdio transport On Windows, pipe I/O can deliver non-UTF-8 bytes at chunk boundaries, causing `UnicodeDecodeError` when the MCP SDK's `TextReceiveStream` uses `errors="strict"`. Set `encoding_error_handler="replace"` on `StdioServerParameters` so undecodable bytes become U+FFFD instead of crashing. * fix(gateway): pass encoding="utf-8" to read_text/write_text in update path (#37423) * fix(gateway): add utf-8 encoding to dead target registry * fix(gateway): cover discord update-response utf-8 path (#37423) * fix(gateway): extend the utf-8 file-I/O guard to google_chat + whatsapp Follow-up to the salvaged #38985: guard the 4 bare read_text/write_text sites its allowlist missed (google_chat thread-count store + oauth JSON) and add whatsapp/google_chat to the AST guard test's file list. * fix(tools): utf-8 decode for STT/TTS command-provider popen_kwargs Salvaged from PR #45099 — the two popen_kwargs dict sites the #70875 AST sweep missed because the kwargs are built indirectly (_run_command_stt, _run_command_tts). * fix(windows): platform._syscmd_ver stub in bootstrap + PYTHONUTF8 in desktop backend env Two gaps found auditing the decode-crash cluster: 1. suppress_platform_ver_console() only ran in hermes_cli.main processes; slash workers, tui_gateway/entry, run_agent, batch_runner, and cli.py import only hermes_bootstrap and were exposed to both the console flash and (on Python 3.11.0/3.11.1, which lack CPython's encoding='locale' fix) a UnicodeDecodeError inside platform.win32_ver() under PEP 540 — the crash #69413 reported. Move the stub into hermes_bootstrap so every entry point gets it; the _subprocess_compat copy stays for non-bootstrap callers. 2. The desktop Electron spawn built the backend env without PYTHONUTF8, so anything the Python child emitted before hermes_bootstrap ran (interpreter startup errors, pre-bootstrap tracebacks) decoded with the locale default. Re-port of PR #56499's env half (echoriver89) to backend-env.ts (original targeted the deleted backend-env.cjs); explicit user setting wins. * fix(compaction): strip proactive section headers from summary template Remove three directive-heavy section headers from both the LLM and deterministic summary templates that caused the agent to resume stale tasks after context compression: - Historical In-Progress State - Historical Pending User Asks - Historical Remaining Work These sections read as actionable instructions even within a REFERENCE-ONLY wrapper, hijacking the user's latest message. The remaining sections are purely descriptive/past-tense. Frozen prefix copies in _HISTORICAL_SUMMARY_PREFIXES updated to match. Test 8/8 passed. * fix(compaction): freeze pre-change SUMMARY_PREFIX generation, restore mutated entry Address review on #69619: the previous commit mutated the newest frozen entry in _HISTORICAL_SUMMARY_PREFIXES and never froze the live prefix it retired (the generation with both the four-heading discard clause and the tools-active clause). A summary persisted immediately before upgrading was therefore treated as an ordinary message on resume/re-compaction, keeping the old handoff text embedded in the body. - Prepend the exact pre-change live prefix as a new frozen entry (newest-first), leaving all existing frozen entries byte-identical - Restore the Jul 2026 (#65848 class) frozen entry to its original four-heading text - Pin the retired generation as a literal in test_summary_prefix_semantics.py so mutating or dropping it fails CI - Make the #65848 tool-use regression position-agnostic (match the pre-clause generation by content, not tuple index) Verified byte-identity of both rescued generations against the parent commit. 233 focused prefix/resume/compressor tests pass. * test(compaction): byte-pin every frozen prefix generation Hardening follow-up to the #69619 review fix. The previous regression byte-pinned only the rescued pre-#69619 generation; older frozen entries were covered solely by fragment assertions and a self-matching loop that cannot detect a frozen entry mutating (the loop tests each entry against itself). - Pin all four _HISTORICAL_SUMMARY_PREFIXES generations as literals in _FROZEN_PREFIX_GENERATIONS and assert order-sensitive tuple equality plus detect/strip for each - State the prepend-only contract explicitly on the tuple: never mutate or reorder existing entries Negative controls verified: mutating, dropping, or reordering a frozen entry each fail the new test, while the legacy self-matching loop still passes under mutation — confirming the closed coverage gap. * chore: map contributor akb4q * fix(macos): use launchctl submit instead of start_new_session for plist reload helper (#69098) The deferred launchd reload helper used start_new_session=True to detach from the gateway's process group. However, setsid(2) alone does NOT move the child outside the launchd job's process coalition — whe…

#25) * fix(checkpoints): bind an empty orphan preview to an empty deletion allowlist Follow-up for salvaged PR #69141, addressing the last open review point: cmd_prune() only set orphan_allowlist inside 'if orphans or pre_v2_orphans', so a zero-orphan preview passed the unrestricted None sentinel down to prune_checkpoints(), authorizing deletion of any project that became orphaned between the preview and the rescan — with zero confirmation calls. The allowlist is now bound unconditionally for every non-force run (empty preview => empty allowlist); --force keeps None. Adds the zero-orphan-preview timing regression plus allowlist-identity tests. * fix(agent): cache static system prompt prefixes * fix(prompt-caching): inject cache breakpoints after message normalization The conversation loop normalizes message text right before the API call so the request prefix is byte-identical across turns -- the stated reason is KV cache reuse on local inference servers and better cache hit rates on cloud providers. Cache breakpoints were injected *before* that pass, which defeats it. `_apply_cache_marker` rewrites a plain-string `content` into a `[{"type": "text", ...}]` block. The normalization pass is guarded on `isinstance(content, str)`, so every message that just got marked is silently skipped by it and keeps its raw leading/trailing whitespace. A message is only marked while it sits in the last-3 window, so: turn N in the window -> marked, content "file1\nfile2\n" turn N+1 rolled out -> plain, content "file1\nfile2" The same logical message is sent with different bytes on consecutive turns. The prefix stops matching at that position -- which is inside the span the breakpoints were placed to protect -- so the reusable prefix collapses back toward the system breakpoint on every turn. Tool results carry a trailing newline almost by default (any shell command output), so this is the common case, not an edge case. Move the injection below every message mutation. Besides fixing the whitespace divergence this stops breakpoints from being spent on messages that the orphan sweep or the thinking-only drop is about to remove or merge away -- a marker on a dropped message is a wasted breakpoint out of the four available. Nothing between the old and new call sites reads `cache_control`, and the mutators now see the plain-string shapes they were written against. * fix(caching): reconstruct static system prefix on session restore and post-compression reuse Follow-up to the cherry-picked #68258 base: the cross-session-stable prefix (_cached_system_prompt_static) was only recorded on fresh builds, so two paths silently degraded to the legacy single-breakpoint layout (flagged in review of #68258/#69341/#69704): - Session restore: gateway surfaces build a fresh AIAgent per turn and restore the persisted prompt verbatim from the session DB; the static prefix stayed None from turn 2 onward, flip-flopping the wire layout. - Post-compression cached-prompt reuse: _invalidate_system_prompt() clears the static prefix, and the keep-cached-prompt branch never restored it. Both sites now reconstruct the stable tier and adopt it ONLY when the authoritative prompt string literally startswith() it — stable-tier drift (skills edited, identity changed) falls back to the legacy layout with the stored bytes untouched. Fail-open on any builder error. The restore-path rebuild is gated on _use_prompt_caching so non-Anthropic routes skip it entirely. Refs #68191 Co-authored-by: JonthanaHanh <92574114+JonthanaHanh@users.noreply.github.com> Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com> Co-authored-by: webtecnica <webtecnica@users.noreply.github.com> * fix(config): preserve opaque .env values The .env sanitizer inferred missing newlines from known KEY= substrings inside existing values. Plain secrets containing those bytes could therefore be split into synthetic assignments and rewritten to disk. Treat each physical line as the only assignment boundary and keep bytes after the first equals sign opaque for boundary discovery. Preserve safe formatting, null-byte removal, BOM handling, and normal one-assignment-per-line parsing. Cover direct loading, dotenv loading, sanitization, writers, and migration with behavioral regressions. Fixes #29155 * fix(web): resolve per-profile gateway state for ?profile= in /api/status When ?profile=<name> was passed to /api/status, the handler used _config_profile_scope to set the HERMES_HOME contextvar override, but the gateway liveness check (get_running_pid_cached) and runtime status read (read_runtime_status) both resolve _get_process_hermes_home(), which deliberately ignores contextvar overrides (issue #56986) — it always reads os.environ['HERMES_HOME'] or the platform default. A named profile's gateway identity files (~/.hermes/profiles/<name>/gateway.pid, gateway_state.json) were therefore never found and the endpoint always reported the profile's gateway as stopped. Fix: when ?profile=<name> is requested, resolve the profile directory and pass explicit profile-scoped paths: - get_running_pid_cached(pid_path=profile_dir / 'gateway.pid') - read_runtime_status(path=profile_dir / 'gateway_state.json') - get_runtime_status_running_pid(..., expected_home=profile_dir) This is the same explicit-path pattern _collect_profile_gateway_topology already uses for per-profile gateway state, and it works within the #56986 constraint (no HERMES_HOME env mutation; read-only cross-profile access). Plain /api/status without ?profile= keeps the exact zero-arg calls, so its behavior — including the pid-cache signature and runtime-status fallback — is byte-for-byte unchanged. Fixes #69143 * test(web): pin per-profile gateway state scoping on /api/status Follow-up for the salvaged #70498 fix: replace the original PR's mock-signature churn (28 lambda **kw edits, needed only because it changed the no-profile call shape) with two targeted regression tests: - ?profile=<name> must pass the profile's gateway.pid / gateway_state.json paths and expected_home to the gateway status readers (HOME-anchored per-profile state under ~/.hermes/profiles/<name>/) - ?profile=<unknown> must 404 via _resolve_profile_dir The production change keeps plain /api/status on the exact zero-arg calls, so every pre-existing test passes unmodified. * test: accept the new profile-scoped kwargs in status fakes /api/status?profile= now passes pid_path=/path=/expected_home= to the PID and runtime-status readers; the profile-unification fakes had zero-arg signatures and raised TypeError. Plain /api/status call shapes are unchanged (pinned by the existing zero-arg tests in test_web_server.py). * fix(gateway): prevent reconnect watcher wedge after network-loss fatal error (#70344) Three-part fix for the gateway going silently deaf after a retryable fatal adapter error (e.g. httpx.ConnectError on Telegram): 1. **Detach-on-timeout in _connect_adapter_with_timeout** — Replaced plain asyncio.wait_for with the task-detach pattern used by _await_adapter_cleanup_with_timeout. asyncio.wait_for cancels the overdue task but then waits for it to exit, so a connect() that catches CancelledError can block recovery forever. The detach pattern releases the runner at the deadline via consume_detached_task_result. 2. **Ensure reconnect watcher always runs after escalation** — Added _ensure_reconnect_watcher_running(), called after queueing a retryable fatal error. If the reconnect watcher task has died (exhausted restart budget, terminal exception), it is respawned so queued platforms are never permanently stranded. 3. **Faulthandler at gateway startup** — Enabled faulthandler + SIGUSR2 dump to a rotating file under HERMES_HOME/logs/ for post-mortem diagnosis of future event-loop freezes. Tests added for _ensure_reconnect_watcher_running (alive, dead, not-started, not-running), fatal-error integration (retryable calls ensure, non-retryable does not), and _connect_adapter_with_timeout (timeout raises, success returns). * fix: explicit encoding for faulthandler file open (ruff PLW1514) * fix(gateway): stay alive on mixed retryable + non-retryable startup failures When connected_count == 0 and at least one platform failed with a non-retryable error, the runner exited with GATEWAY_FATAL_CONFIG_EXIT_CODE (78) even if OTHER platforms failed for merely transient reasons. Real-world shape (NS-609, hosted instance): WhatsApp enabled but never paired (non-retryable whatsapp_not_paired) + Telegram TimedOut during polling startup (retryable) => exit 78 => the gateway either goes permanently down (supervisors honoring the exit-78 contract via RestartPreventExitStatus / the s6 finish->125 translation from #51228) or crash-loops (anything else). Either way Telegram never gets its retry and the dashboard drops with every exit, so a single unpaired platform plus one network blip disconnected every channel on the instance. Now exit 78 is reserved for the case where ALL startup failures are non-retryable (true config error, nothing to wait for). With mixed failures the gateway stays alive in degraded state: the reconnect watcher recovers the retryable platforms and the misconfigured ones stay fatal-parked and visible in runtime status. * fix: gate SIGUSR2 faulthandler registration behind POSIX check signal.SIGUSR2 and faulthandler.register() don't exist on Windows; the bare reference raised AttributeError at import time per the windows-footgun checker. faulthandler.enable() still covers fatal-error dumps on all platforms. * fix(gateway): detect and escape silent event-loop freezes - A self-rescheduling 5s call_later floor timer, armed before any adapter connects, guarantees the selector always has a finite timeout, so the existing async defenses (polling heartbeat, timeout guards) regain a chance to run after a zero-pending-timer stall. - A resident daemon-thread liveness watchdog probes the loop via call_soon_threadsafe every 30s; after 3 consecutive 10s-timeout misses (~120s of total unresponsiveness) it dumps all thread tracebacks and exits with the established GATEWAY_SERVICE_RESTART_EXIT_CODE (75) so a supervisor restarts the gateway - async-level recovery cannot run on a frozen loop. - stop() disarms both guards before any teardown await so a busy shutdown is never misjudged as a freeze. HERMES_GATEWAY_LOOP_WATCHDOG=0 disables; _INTERVAL/_TIMEOUT/_STRIKES tune the thresholds. Fixes #69089 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gateway): close watchdog shutdown race against final-strike exit - Re-check stop_event after a missed probe (before the strike increment) and again on entering the final-strike branch (before the critical log, dump, and hard exit), so a normal stop() landing between the last timeout check and the exit path can no longer be misclassified as a freeze and trigger a supervisor restart. - Deterministic boundary tests pin both re-checks independently (mutation-verified: removing either check turns its own test red); frozen-loop semantics are unchanged. Addresses the shutdown-race review on #69164. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gateway): recheck stop immediately before watchdog hard exit - A stop() landing while the final diagnostics (critical log, traceback dump) are executing could still reach os._exit(75) after the pre-diagnostic check. Add a third stop_event recheck immediately before the hard exit: diagnostics may complete, but a disarmed watchdog never exits. - Deterministic regressions for both windows (stop triggered from inside logger.critical and from inside faulthandler.dump_traceback); mutation-verified (removing the check turns both red). Frozen-loop semantics unchanged. Addresses the second round of the shutdown-race review on #69164. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(gateway): gate loop-liveness watchdog via config.yaml, drop HERMES_* env knobs Follow-up to the salvaged #69164 commits: policy forbids introducing new HERMES_* environment variables, so the four watchdog env knobs (HERMES_GATEWAY_LOOP_WATCHDOG / _INTERVAL / _TIMEOUT / _STRIKES) are replaced with a single config.yaml boolean: gateway: loop_watchdog: true # default; false disables both guards - gateway/config.py: new GatewayConfig.loop_watchdog field (default True), parsed from top-level or nested gateway: form, round-trips via to_dict/from_dict. - gateway/run.py: _start_loop_liveness_guards() checks config.loop_watchdog before arming the floor timer + watchdog (getattr-guarded for bare object.__new__ runners). - gateway/shutdown_watchdog.py: start_loop_liveness_watchdog() no longer reads the environment; probe interval/timeout/strikes are module constants (30s/10s/3 — ~90s to restart, matching the systemd watchdog layer's posture). - hermes_cli/config.py: documented gateway.loop_watchdog default so 'hermes config set gateway.loop_watchdog false' validates. - tests: env-knob tests replaced with config-gate + round-trip tests; the final-strike boundary test injects its probe via max_strikes directly instead of patching the removed env helper. * fix: getattr-guard _stop_loop_liveness_guards in GatewayRunner.stop Teardown-path tests build bare runners via object.__new__ without the liveness-guard machinery; the unguarded call raised AttributeError in 8 tests. Same guard pattern as the start path. * fix(desktop): close cross-session leak windows in composer + session refs (#59305) Two React passive-effect timing bugs let a session switch land in the wrong chat: activeSessionIdRef/selectedStoredSessionIdRef (use-session-state-cache) and the composer's attachment-scope swap (use-composer-draft) both mirrored their source props via useEffect, which fires one commit AFTER the new session's view has already painted — a synchronous read/submit in that window observed the outgoing session's ids/attachments. - use-session-state-cache.ts: mirror the session refs synchronously during render instead of a useEffect, guarded to fire only when the prop itself changed (not unconditionally) so an imperative pin from submit.ts / use-session-actions (e.g. a freshly resumed runtime id, intentionally not synced to the source atom) survives an unrelated re-render. - use-composer-draft.ts: the per-thread attachment-scope-swap effect is now a useLayoutEffect, closing the window before paint. - submit.ts / session-context-drift.ts: add a 3rd drift prong comparing the composer's loaded scope (SubmitTextOptions.composerScope) against the submit target, resolved into the same lineage-root domain (resolveComposerSessionKey) the composer itself uses — comparing against the raw tip id would false-positive-abort every submit into any session that has ever auto-compressed. - routes.ts / chat/index.tsx: the primary composer's durable scope key now prefers the route over a possibly-stale store selection (primaryRouteSelectedSessionId). - use-composer-draft.ts: redacted [composer-rehydrate] diagnostic log (counts/kinds/scope only, never raw refs) for future reports in this class. - chat-runtime.ts: normalize attachment id values (url/path) before hashing so a re-attach with a trailing slash or backslash path dedupes correctly. 16 files, 286 tests across the touched/dependent suites (17 files) green, including new regression coverage for each fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(desktop): satisfy eslint import-order rules in use-composer-draft.test.tsx CI's check:lint failed on two perfectionist rule violations introduced by the new test file: type import ordering and missing blank line between the parent-relative and same-directory import groups. No behavior change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(agent): prevent shared OpenAI client FD-recycle corruption from stale stream watchdog The streaming stale watchdog was calling _replace_primary_openai_client() from its polling thread, which closes the shared client's connection pool. Worker threads from previous stale-killed attempts may still be unwinding their SSL BIOs, causing TLS application-data to overwrite SQLite file headers via FD reuse. This is the same corruption vector documented in #67142 for Anthropic, where the fix was to never close the shared client from a non-owner thread. Apply the same pattern to the OpenAI-wire path: - Stale stream watchdog: skip shared client replacement - Mid-tool-retry cleanup: skip shared client replacement - Stream retry cleanup: skip shared client replacement The request-local client is already closed via _close_request_client_once. The shared client is replaced lazily by _ensure_primary_openai_client on the next request, which runs on the owning thread. Closes #70773. * fix(agent): retire replaced shared OpenAI clients instead of cross-thread pool close Widen the #70773 fix beyond the three in-request cleanup sites removed in the cherry-picked commit: every remaining path that swaps out the shared OpenAI client could still hard-close its pool from a thread that doesn't own the in-flight sockets (credential rotation/refresh on the turn thread, dead-connection cleanup, gateway cache eviction, transport recovery) — the same FD-recycle corruption vector, just rarer. Add AIAgent._retire_shared_openai_client(): shutdown(SHUT_RDWR) all pooled sockets (FD-safe from any thread, unblocks in-flight readers) but never call client.close() — FD release is deferred to GC, which cannot run until every borrowing thread has unwound its SSL BIO. Refcounting is the ownership handshake; with no borrowers the FDs are released immediately. Wired into: - _replace_primary_openai_client (rotation/refresh/dead-conn cleanup) - try_recover_primary_transport (primary_recovery) - release_clients (gateway cache_evict) agent.close() keeps the hard close: full teardown is a real session boundary where no request may be in flight. Tests: new tests/run_agent/test_70773_shared_client_fd_corruption.py covers the three watchdog/retry sites plus retire semantics; existing close-assertions updated to pin retire-not-close. * test: update credential-refresh tests for retire-not-close contract The three refresh tests asserted the replaced shared client gets close()d — the exact cross-thread close #70773 removes. They now pin the new contract: close() is NOT called from the refresh path; the old client is retired (sockets shutdown, FD release deferred to GC). * fix(doctor): UTF-8/latin-1 fallback when scanning .env Prefer UTF-8 for ~/.hermes/.env provider scans, then latin-1 for cp1252/Notepad files. Add regression test for invalid UTF-8 bytes. * fix: handle non-UTF-8 files in OpenClaw migration script * fix: decode config and state files as UTF-8 on non-UTF-8 locales Several file-I/O call sites still use open() / Path.read_text() / Path.write_text() without an explicit encoding, so they fall back to the platform default. On Windows CN/JP/KR locales (GBK/CP932/CP949) any non-ASCII byte in a config/state/user-content file raises UnicodeDecodeError or UnicodeEncodeError and crashes the caller. to the remaining hot paths: - agent/copilot_acp_client.py: fs/read_text_file and fs/write_text_file (Copilot's read_file / write_file tools, directly reported in #18637 bug 2) - agent/model_metadata.py: context-length YAML cache load + two save sites (context probing is on the call path of every model invocation) - agent/nous_rate_guard.py: cross-session rate-limit JSON state (read + atomic write via os.fdopen) - cron/scheduler.py: user config.yaml read in run_job - gateway/delivery.py: cron output writes for AI-generated content, very likely non-ASCII yaml.dump call sites also gain allow_unicode=True so the emitted YAML preserves non-ASCII chars as-is instead of emitting \u escape sequences. Adds regression tests that monkeypatch builtins.open / Path.read_text / Path.write_text to simulate a GBK locale: each test raises UnicodeDecodeError / UnicodeEncodeError unless the caller explicitly passes encoding='utf-8'. Verified that the tests fail on main and pass with this change, on Linux as well as on Windows. Refs #18637 * fix(cli): add explicit encoding to read_text/write_text calls Path.read_text() and Path.write_text() without explicit encoding default to the system locale encoding. On Windows this is typically cp1252, which causes UnicodeDecodeError for UTF-8 content (JSON configs, user data, service scripts). Add encoding="utf-8" to all read_text() and write_text() calls across 8 CLI files, matching the pattern established in PR #50534 (security_audit_startup.py) and ruff rule PLW1514. Fixed files: - main.py: 4 read_text calls - auth.py: 3 read_text calls - banner.py: 1 read_text + 1 write_text - service_manager.py: 1 read_text + 4 write_text - container_boot.py: 1 read_text + 4 write_text - doctor.py: 3 read_text calls - uninstall.py: 2 read_text calls - gateway.py: 1 write_text call * fix(core,cli,gateway,plugins): add encoding='utf-8' to read_text() calls Path.read_text() without an explicit encoding uses the platform's default encoding. On Windows this is typically cp1252 or mbcs, which causes UnicodeDecodeError or silent data corruption when reading UTF-8 content (JSON files, user text, config with non-ASCII chars). This is the read-side companion to the write_text() encoding fix. Fixed the most critical locations that read JSON data, user content, and config files across 14 files with 31 call sites. Pattern: .read_text() → .read_text(encoding='utf-8') json.loads(path.read_text()) → json.loads(path.read_text(encoding='utf-8')) * fix(install): emit UTF-8 from skills_sync on non-UTF-8 Windows locales On Windows with a non-UTF-8 system locale (e.g. CP936/GBK on zh-CN), Python defaults stdout/stderr to the active codepage. tools/skills_sync.py prints glyphs such as checkmark (U+2713) and up-arrow (U+2191) that GBK cannot encode, raising UnicodeEncodeError mid-run. The installer (scripts/install.ps1) captures this script's stdout and the Rust bootstrap parses it as UTF-8 expecting a JSON result frame. A GBK byte stream (or the traceback it triggers) surfaces as: WARN stdout read error: stream did not contain valid UTF-8 stage=config-templates state=Failed error=install.ps1 -Stage config-templates produced no JSON result frame (exit=Some(0)) i.e. the stage fails even though the script exits 0. install.ps1 already sets [Console]::OutputEncoding = UTF8, but that does not propagate to the python.exe child (Python reads PYTHONIOENCODING / locale, not the console encoding). Fix in two places for defense in depth: - tools/skills_sync.py: reconfigure sys.stdout/stderr to UTF-8 at import so output is valid UTF-8 regardless of caller or active codepage. - scripts/install.ps1: set PYTHONIOENCODING=utf-8 and PYTHONUTF8=1 (scoped to the call, restored afterwards) around the skills_sync.py invocation. * test(install): add UTF-8 regression guard for skills_sync child path Addresses hermes-sweeper review on PR #54866: the installer runs tools/skills_sync.py as a child python.exe whose PYTHONIOENCODING / PYTHONUTF8 the scoped install.ps1 block sets, but there was no regression test for this child-Python UTF-8 path. The existing test_child_process_inherits_utf8_mode covers a different (bootstrap entry-point) flow. Add TestSkillsSyncUtf8Guard: three subprocess tests that import skills_sync (triggering its import-time stdout/stderr reconfigure) and assert the checkmark/up-arrow glyphs the script prints at tools/skills_sync.py:596,675 emit valid UTF-8 and exit 0 even when the child env is left unset or explicitly hostile (gbk). A third test proves the guard is load-bearing by reproducing the crash without it. Also keep the new install.ps1 comment ASCII-only (the checkmark spelled out as U+2713) per the file's PS 5.1 parser-compatibility contract at scripts/install.ps1:79-80; the literal glyph in the comment violated that contract. * fix: add encoding="utf-8" to Path.write_text() calls (P1) Path.write_text() without encoding defaults to system locale encoding. On Windows (cp1252), this silently corrupts non-ASCII content written to JSON files, config files, and cache files. This is the write-side counterpart to the read_text() encoding fix (PR #56115). PLW1514 only covers open() calls — Path methods are unguarded by ruff. 39 instances across 16 files, all passing py_compile. Files changed: - agent/copilot_acp_client.py (1) - tools/web_tools.py (1) - tools/xai_http.py (1) - tools/skills_hub.py (8) - gateway/slash_commands.py (1) - gateway/run.py (5) - gateway/dead_targets.py (1) - gateway/delivery.py (2) - gateway/platforms/qqbot/adapter.py (1) - hermes_cli/gateway.py (1) - hermes_cli/banner.py (1) - hermes_cli/service_manager.py (5) - hermes_cli/container_boot.py (5) - hermes_cli/uninstall.py (1) - hermes_cli/main.py (2) - hermes_cli/profiles.py (3) * fix(hindsight): specify UTF-8 encoding for file I/O on Windows On Windows with CJK locales (e.g. Chinese/GBK), pathlib.Path.read_text() defaults to the system encoding instead of UTF-8, causing UnicodeDecodeError when reading .env or .json config files that contain non-ASCII characters. Explicitly pass encoding='utf-8' to all read_text() and write_text() calls in the hindsight memory provider plugin. * fix(memory): read/write .env as UTF-8 in mem0 and hindsight setup The mem0 and hindsight memory-provider setup routines round-trip the user's ~/.hermes/.env: they read existing lines, update the keys they manage, and rewrite the whole file preserving every other line verbatim. Both used env_path.read_text() / write_text() with no encoding. read_text()/write_text() with no encoding fall back to the system locale (cp1252/GBK on Windows), so on a non-UTF-8 host the preserved lines get mangled or the call crashes on any non-ASCII value, and — because the reader never strips a BOM — a Notepad-edited .env makes the first key fail the in-place match and get duplicated instead of updated. Match the canonical .env readers in hermes_cli/config.py: read with encoding='utf-8-sig' (BOM-tolerant) and write with encoding='utf-8'. mem0/_setup.py already pins utf-8 for mem0.json, so this just aligns the .env path in the same file. Fixes both memory plugins in one class fix. Adds regression tests: a BOM'd .env updates the first key in place (locale-independent, fails without the fix) and non-ASCII existing lines survive the round-trip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(memory): cover the remaining setup-time .env reads with utf-8-sig Follow-up to review feedback: - mem0 _prompt_api_key read .env with the locale default, so a Notepad BOM hid the first key from the masked current-value lookup; read it with utf-8-sig + errors=replace like the canonical readers in hermes_cli/config.py. - hindsight _load_simple_env used plain utf-8; it also parses the Hermes .env during post_setup, where a BOM stuck to the first key. Switch to utf-8-sig + errors=replace. - Add hindsight regressions: BOM key matching in _load_simple_env and in the cloud post_setup writer, plus non-ASCII round-trip preservation, and a mem0 regression for the BOM'd masked-key lookup. The BOM tests fail without the fix on any platform. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(profile): read .env as utf-8-sig in the distribution-install preview `_render_distribution_plan` reads the target profile's `.env` to decide whether a required env var is already set (so it doesn't nag the user), using `Path.read_text()` with no encoding. Two bugs: 1. `Path.read_text()` defaults to the system locale (cp1251/GBK on Windows), which raises `UnicodeDecodeError` on any non-ASCII byte. The surrounding `except OSError` does NOT catch that — `UnicodeDecodeError` is a `ValueError` — so a mis-encoded `.env` aborts the entire install preview. 2. Even on a UTF-8 locale, a Notepad-added BOM prefixes the first key (`KEY`), so the very first required env var is mis-reported as "needs setting" when it is actually present. `.env` is written as UTF-8 everywhere in the codebase. Read it as `utf-8-sig` (tolerates the BOM) and also catch `UnicodeDecodeError` so a genuinely un-decodable file skips the pre-check instead of crashing. Regression tests: a BOM-prefixed `.env` whose first key must still read as "set", and an invalid-UTF-8 `.env` that must not abort the preview. * fix: add UTF-8 encoding to read_text/write_text in tools/ and agent/ Path.read_text() and Path.write_text() without encoding= default to the system locale (cp1252 on Windows), which corrupts non-ASCII JSON content. Coverage-gap fix for files not addressed by prior encoding PRs: - tools/skills_hub.py: 6 read_text + 8 write_text (cache, index, lock files) - tools/skills_sync.py: 1 read_text (lock file) - tools/xai_http.py: 1 read_text + 1 write_text (auth store, marker) - agent/shell_hooks.py: 1 read_text (allowlist) - gateway/status.py: 1 read_text (PID file) - hermes_cli/banner.py: 1 read_text + 1 write_text (update cache) All sites read/write JSON or short text. No behavioral change on Linux (already UTF-8); fixes silent data corruption on Windows. * fix(skills): tolerate non-UTF-8 bytes in hub lock.json _read_hub_installed_names() reads ~/.hermes/skills/.hub/lock.json with a strict utf-8 decode. Hub skill descriptions can carry Windows-1252 typographic bytes (em-dash 0x97, smart quotes, bullets) as single high bytes; read_text(encoding="utf-8") then raises UnicodeDecodeError, which is a ValueError sibling not caught by the function's except (OSError, json.JSONDecodeError). It escapes and 500s the whole /api/skills endpoint, blanking the desktop Skills panel. Decode with errors="replace" so the offending byte degrades to U+FFFD and the structurally valid JSON — and every other skill — stays readable. Fixes #68053 * fix(cli): read .worktreeinclude and .gitignore as UTF-8 in worktree setup _setup_worktree read both files with the locale default encoding. On a cp1251/GBK Windows machine a UTF-8 include list either decodes to mojibake paths (non-ASCII entries silently not copied) or raises UnicodeDecodeError, which the enclosing handler logs at DEBUG and swallows — no include is copied at all, so the worktree starts without .env/keys and the agent breaks invisibly. A Notepad BOM likewise glues to the first include entry on every platform, and to the first .gitignore line, defeating the '.worktrees/' membership check and appending a duplicate entry on each run. Read both files with utf-8-sig + errors=replace, matching the canonical .env readers in hermes_cli/config.py (utf-8-sig because Notepad adds a BOM) and the UTF-8 append this same block already performs on .gitignore. Regression tests exercise the real cli._setup_worktree: the two BOM tests fail without the fix on any platform, the non-ASCII include test additionally reproduces the Windows locale failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(skills): read OOXML parts as bytes and form JSON as UTF-8 in office skill scripts The bundled office skills (#68595) read user documents and agent-authored payloads with the locale-default codec: - docx/powerpoint validators/base.py opened OOXML part XML in text mode before handing it to lxml. On Windows (cp1251/GBK) the bytes decode to mojibake that lxml then parses, so validation runs against silently corrupted document text; on locales where the UTF-8 bytes don't decode the validator crashes with UnicodeDecodeError instead of validating. Opening as bytes lets lxml honor the encoding declared in the XML prolog. - The pdf form scripts (fill_fillable_fields, fill_pdf_form_with_annotations, create_validation_image, check_bounding_boxes) read the fields JSON the agent authors — UTF-8 by construction — with the locale codec, so non-ASCII form values (any Cyrillic/CJK/accented input) either crash or get written into the user's PDF as mojibake. The json.dump writers use ensure_ascii=True and were already safe; only the readers needed pinning. Adds a contract test asserting every document/payload reader is locale-independent, plus a live regression test that runs check_bounding_boxes.py on a non-ASCII fields.json under a forced non-UTF-8 locale — it fails without the fix on both POSIX (C locale) and Windows (cp1251 chokes on the 0x98 byte of U+2018). * fix(windows): sweep remaining bare read_text/write_text sites + linter rule AST-driven pass over every Path.read_text()/write_text() without an explicit encoding= across non-test code: 71 sites in 34 files (skills_hub, hermes_cli/main+profiles+service_manager+container_boot, mem0/hindsight/honcho plugins, achievements dashboard, release/CI scripts, productivity+comfyui skill helpers, agent/*). Verified zero positional-encoding collisions before insertion; per-file compile() check after. Adds a check-windows-footguns rule flagging bare single-line read_text/write_text (multi-line forms stay covered by the AST guard test from #38985). Together with the salvaged contributor commits this retires the ~169-site bare file-I/O class (#37423's long tail). * fix: restore utf-8-sig BOM tolerance at .env readers the sweep normalized The cherry-pick auto-resolution + AST sweep applied plain utf-8 at three .env reader sites where the salvaged PRs (#62617, #62123) deliberately use utf-8-sig — a Notepad BOM must not hide/duplicate the first key. Restore the contract (tests pin it). * chore: contributor email mappings for the file-I/O salvage * refactor(desktop): add shared Field form-dialog primitive Dialog forms each hand-rolled their own label+control+hint stack (or borrowed the settings-surface ListRow), so gaps and hint styling drifted between the profile, cron, and webhook dialogs. Add a single Field / FieldHint primitive for label-over-control dialog fields and adopt it in the create/rename profile dialogs as the first consumers. * fix(desktop): unify overlay-pane padding and add primary PanelAction Overlay panes each set their own top padding, so the Settings sidebar and Panel headers sat at different heights than System/Agents and the close X (the #67759 regression). Hoist the shared beside-the-X clearance into OVERLAY_TOP_CLEARANCE, keep the taller pad only on OverlayMain (which sits under the X), tighten OverlayMain's gutters, and drop the one-off Settings override. Also give PanelAction a `primary` variant so a detail header can promote its main action to a filled button. * refactor(desktop): fold cron Blueprints into the New Job dialog Blueprints lived behind a separate Jobs/Blueprints tab with its own card gallery — a bespoke surface no other overlay uses. Remove the tab and make blueprints a "Start from" dropdown at the top of the New Job dialog (default "Custom" = the manual editor); picking one swaps the form for that blueprint's typed slots. Also promote the detail-view "Trigger now" button to a primary action and adopt the shared Field primitive. * refactor(desktop): webhooks create form uses shared Field; drop status pill The create dialog used the settings-surface ListRow/ToggleRow inside a modal, which read differently from every other form dialog, and the detail header carried an enabled/disabled pill that rendered as a stray dash. Switch the form to the shared Field primitive (+ Switch) and remove the pill. * fmt(js): `npm run fix` on merge (#71099) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(dashboard): add lightweight /api/health liveness endpoint /api/status is the only public liveness route, and its handler loads the gateway config, probes gateway health, and counts sessions before it can answer. That work is wrong for a readiness probe: a caller that only needs to know the process is up pays for a cold plugin import tree. Add /api/health, which returns process liveness, version, and the auth-gate shape and touches nothing else. * fix(desktop): probe /api/health for boot readiness, and survive a stalled loop Desktop boot polls /api/status, so readiness waits on gateway config and a cold plugin import tree. On Windows that regularly outlives the probe and Desktop kills a backend that is already listening, respawns it, and re-pays the same import cost — the reported crash loop. Probe /api/health instead, falling back to /api/status only for the missing-route shapes the fetch helpers emit (404, or HTML from the SPA), so an older remote backend still connects. Timeouts and server errors keep polling health rather than dropping to the heavyweight route. A cheap route is not enough on its own. Warming the gateway import holds the GIL, so the event loop can stall for tens of seconds and starve /api/health too. At the default 15s socket timeout only three attempts fit in the 45s budget; give each probe 5s so the loop keeps retrying across the stall. Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com> Co-authored-by: DESXIE <78300229+DESXIE@users.noreply.github.com> Co-authored-by: frohsinnllc <231045016+frohsinnllc@users.noreply.github.com> * fix(state): decode display_metadata at every message read path get_messages(), get_messages_around() and get_anchored_view() returned the raw display_metadata column instead of the dict every caller expects. The desktop paints a resumed transcript from the REST prefetch, which reads through get_messages(), so any session holding an async_delegation_complete event failed resume with "Cannot use 'in' operator to search for 'task_count'" — on every such session, not just corrupted ones. Route all four read paths through one shared codec that also unwraps rows carrying a second JSON layer, so sessions already broken on disk recover on read rather than needing a migration. Co-authored-by: Studio729 <Studio729@users.noreply.github.com> Co-authored-by: aml1973 <aml1973@users.noreply.github.com> Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> * fix(state): stop double-encoding display_metadata on write export_session() reads through get_messages(), so before the read fix an already-serialized string went straight back into _insert_message_rows() and got re-dumped — an export/import round trip permanently corrupted the row. Guard the three write paths the same way tool_calls already is: parse a string argument before storing it, and drop metadata that isn't an object rather than persisting something no reader can use. Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> Co-authored-by: aml1973 <aml1973@users.noreply.github.com> * fix(desktop): tolerate unparsed display_metadata from an older backend The desktop and the Hermes backend it talks to version independently — a remote VM running an older build still serves display_metadata as JSON text. Indexing into that string with `in` threw and failed the whole resume, so narrow the type to admit a string and parse it before reading task_count. Falling back to the generic label keeps a delegation event renderable even when the metadata is unusable. Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> Co-authored-by: Studio729 <Studio729@users.noreply.github.com> * fix(checkpoints): don't prune a project whose volume is merely unmounted Orphan pruning decides a project is gone from a single probe: if delete_orphans and (not workdir or not Path(workdir).exists()): reason = "orphan" then deletes its ref, index, and metadata — the project's entire checkpoint history. `Path.exists()` is False for a deleted directory, but it is equally False for one whose storage is not attached right now: an unplugged external drive, a share behind a downed VPN, a bind-mount absent from this container, an offline Windows mapped drive. The project is fine; only our view of it is. This is not an opt-in maintenance command. `maybe_auto_prune_checkpoints` runs unattended at startup from both `cli.py` and `gateway/run.py`, with `delete_orphans=True` by default. So starting Hermes once while the drive is unplugged silently destroys the restore points for every project on it — the one thing checkpoints exist to provide, and there is nothing to restore from afterwards. Reproduced against the real store: a project registered under an unmounted path and one on local disk, then a startup prune — prune: {'scanned': 2, 'deleted_orphan': 1} unreachable project index still on disk: False The legacy pre-v2 branch has the same flaw plus a second one: a `HERMES_WORKDIR` marker that exists but cannot be read leaves `workdir = None`, which the same condition treats as an orphan. Failing to read a file is not evidence that a project was deleted. Require corroboration before deleting: the workdir's parent must be present, so its absence is something we actually observed. A missing parent means the volume is not there and we know nothing, so the entry is left alone — and an unreadable marker never deletes at all. Genuinely abandoned projects are still reclaimed, both by the unchanged orphan path (parent present, project gone) and by the retention/stale rule, which runs off `last_touch` rather than a filesystem probe. tests/tools/test_checkpoint_manager.py: a project whose whole mount disappears keeps its history; controls prove a genuinely deleted project is still pruned and a live project is untouched. The data-loss test fails on main; both controls pass there. 81 passed across the checkpoint suites (2 failures in test_checkpoint_manager.py are pre-existing and fail identically on clean main). * fix(checkpoints): an empty surviving mount point is not evidence of deletion Addresses @egilewski's review: the parent-directory check still deleted checkpoint history for the most common unmount layout. Detaching storage removes the parent outright in some layouts (`/Volumes/Ext/proj` on macOS, `/media/<user>/<label>/proj`), which the first commit handles. But in the classic static layout — `/mnt/volume/proj`, an fstab entry, a container bind-mount — unmounting removes the contents and leaves the mount point behind as an empty directory. `parent.is_dir()` is then true, the project is absent, and the startup sweep deletes its ref, index and metadata: exactly the case this PR set out to protect. Reproduced against the real predicate before this commit: mount root vanished (macOS) -> False ok empty surviving mount point -> True <-- history deleted really deleted (siblings) -> True ok An empty parent carries no information: it looks identical whether the volume was detached or the project was deleted. So require the parent to actually say something — it holds some other entry (we observed a populated directory that does not contain the project), or it is itself a live mount point (the volume is attached right now and demonstrably does not hold the project). The cost is that a project deleted out of an otherwise-empty parent is no longer reclaimed by the orphan rule. It is not leaked: the retention rule reads `last_touch` rather than probing the filesystem and still collects it, so reclamation is deferred, not lost. That is the right direction for a predicate whose false positive destroys a user's restore points unattended. `_dir_has_any_entry` stops at the first entry via `os.scandir` instead of materializing a listing, since a project root can hold a large tree. tests/tools/test_checkpoint_manager.py: `test_surviving_empty_mountpoint_ keeps_its_checkpoints` pins the reviewed case, and `test_empty_parent_project_ is_still_reclaimed_by_retention` pins the deferral above so the safety valve cannot silently regress into a leak. Both fail on the previous commit. The real-orphan control now seeds a sibling so it exercises a populated parent rather than the ambiguous empty one. 80 passed in the checkpoint suite; the 2 remaining failures (`TestGitEnvIsolation`, `TestClearFunctions`) fail identically on clean main. * fix(checkpoints): require positive volume-attachment evidence before orphan classification Follow-up to the cherry-picked #69063: egilewski's review found that the _dir_has_any_entry(parent) guard treats ANY entry in the mount point's parent as proof the volume is attached — but unmounting exposes the UNDERLAY directory's own files (e.g. a .keep placeholder), so a populated underlying mount-point dir still classified the project as an orphan and deleted its ref/index/metadata. Reproduced on both main and the PR head. Attachment evidence is now positive instead of circumstantial: * _volume_evidence() records the parent directory's (st_dev, st_ino) identity in the project's metadata while the workdir is observably live (at _register_project/_touch_project time). A mount point resolves to the mounted filesystem's root while attached and to the underlay directory after detach — same path, different directory, different identity. * _workdir_is_observably_gone() now requires the parent visible at prune time to match that recorded identity before the populated-parent check can classify an orphan. A mismatch means a different directory (the underlay) is showing through — a detached volume, not an observed deletion. * Metadata without a recorded identity (written by older versions) is never orphan-classified — unsure never deletes; the retention/stale rule still reclaims genuinely abandoned projects off last_touch. * The frozen pre-v2 layout has no metadata channel for the identity, so it keeps the structural checks only (require_parent_identity=False). * A failed evidence probe on re-registration preserves the previously recorded identity — stale evidence can only make pruning MORE conservative. Windows: st_dev/st_ino of 0 (filesystems without file IDs, some network shares) is treated as "no evidence recorded", which falls into the conservative never-orphan path. os.path.ismount and Path.stat are cross-platform; no POSIX-only calls added. tests/tools/test_checkpoint_manager.py: adds egilewski's exact regression (checkpoint history for mnt/volume/project, detach exposes mnt/volume/.keep, prune with orphan deletion enabled → NOT deleted; fails on the bare cherry-pick, passes with this fix), plus no-recorded-identity conservatism and probe-failure identity preservation. His absent-parent/empty-parent/retention/genuine-deletion/ live-project controls all still pass. Reported-by: egilewski (review on #69063) * fix(telegram): require initial polling readiness Use wall deadlines for deleteWebhook and start_polling, then fail cold startup unless getUpdates proves progress. This lets the gateway discard partial PTB state and retry with a fresh adapter.\n\nRefs #67498 * fix(gateway): allow Telegram readiness budget Give Telegram a 180s default outer connect budget so cold polling can prove getUpdates readiness. Preserve the 30s default for other platforms and all explicit config/env overrides.\n\nRefs #67498 * fix(telegram): bind strict cold-start readiness to its own polling generation Follow-up hardening for the salvaged #69240 readiness gate (#67498): - _start_polling_once now returns its (generation, progress_event) pair so the strict cold-start gate binds to exactly the generation it started, instead of re-reading self._polling_progress_event which a concurrent recovery task may have replaced with a newer generation's event (the G1/G2 race flagged in the #69240 review). - Strict cold start no longer schedules background polling recovery: a polling error during the readiness wait is captured by a strict callback and fails the connect attempt immediately with a loud OSError, so GatewayRunner disposes the partial adapter and retries with a fresh one — no more waiting out the full readiness deadline on a generation that already errored, and no G2-on-partial-app healing. - After readiness is proven the strict callback delegates every later polling error to the real background-recovery callback, preserving the existing degraded/reconnect semantics for the polling lifetime. - The readiness-timeout error message now states the deadline and that the gateway will retry with a fresh adapter (loud failure, not a silent wait). - Regression tests: current-generation progress connects; a polling error during strict cold start fails fast without scheduling background recovery (the #67498 idle-threads shape); stale-generation progress is rejected. Progresses #67498 * test: record getUpdates progress in mocked cold-connect polling flows The strict cold-start readiness gate (#67498) means adapter.connect() no longer returns True until the mocked start_polling records a successful getUpdates round trip for its generation. Update the conflict-suite Application mocks accordingly: - fake_start_polling side effects call adapter._record_polling_progress(adapter._polling_generation) on the initial connect (retry generations intentionally do NOT auto-progress where a test asserts the conflict count survives an unproven retry). - _build_polling_app takes the adapter so its start_polling mock can record progress. Without this, the cold connects in these tests wait out the full 60s readiness deadline and fail — which is exactly the fail-closed behavior the gate is supposed to provide when polling shows no progress. * fix(config): add a collision-safe env var name for custom endpoint keys Both the Desktop panel and the CLI setup flow need somewhere in .env to put a custom endpoint's API key. Deriving the name from the endpoint's hostname collapses two servers on one machine onto a single slot, and every IP-based local endpoint slugs to a digit-leading name that save_env_value rejects outright. Key off the endpoint's own identity and keep a fixed prefix. Co-authored-by: asorry75 <33794789+asorry75@users.noreply.github.com> Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com> * fix(windows): verify rebuilt Hermes.exe integrity before shipping it as an update (#69179) The desktop self-update chain (Desktop -> hermes-setup --update -> hermes update -> hermes desktop --build-only -> relaunch) rebuilds Hermes.exe on the user's machine and declared success on bare file EXISTENCE. A truncated PE (corrupt cached Electron zip / interrupted extraction or rcedit rewrite / full disk) or a wrong-architecture unpacked tree therefore shipped as the 'updated' app, which Windows refuses to load with 'This app can't run on your computer' (此应用无法在你的电脑上运行) — and the previous working build had already been wiped by before-pack.mjs, leaving nothing to fall back to. Fix, in three parts: - hermes_cli/main.py: post-build integrity gate on Windows (_ensure_desktop_exe_launchable). Parses the PE header of the freshly built Hermes.exe — MZ/PE magic, section-table completeness vs file size (catches truncation), and COFF machine vs the host arch (catches arm64/x64 mixups). On failure it purges the (likely corrupt) cached Electron zip, invalidates the content-hash build stamp so the updater's retry-once genuinely re-downloads and rebuilds, restores the previous build from the .bak tree when one exists (keeping the corrupt tree as .corrupt for diagnostics), tells the user the update was aborted and their old version kept, and exits nonzero. _desktop_packaged_executable also now prefers a host-loadable PE over pure newest-mtime when multiple win-*-unpacked trees coexist. - apps/desktop/scripts/before-pack.mjs: on win32, the previous unpacked tree is preserved as <appOutDir>.bak (only when it holds the product exe — partial/corrupt trees still get the plain wipe) instead of being destroyed, providing the rollback material for the gate above. Non-Windows behavior is unchanged. - Behavior-contract tests: tests/hermes_cli/test_desktop_exe_integrity.py (23 tests — synthetic PE fixtures for truncation/non-PE/arch-mismatch, rollback semantics, and the build-only exit contract) and 6 new vitest cases in before-pack.test.mjs for the .bak preservation rules. Progresses #69179 * fix(desktop): persist the whole discovered model list when saving an endpoint Test enumerates a custom provider's catalogue and the panel holds the result in discoveredModels, but the save payload never carried it, so only the one model the user hand-typed reached providers.<id>.models. Every downstream picker reads that map straight from config.yaml with no live probe, which is why a proxy serving 18 models offered exactly one. Send the discovered list and merge it onto the entry, so models already known keep their context lengths. Fixes #69988 Co-authored-by: asorry75 <33794789+asorry75@users.noreply.github.com> * fix(web_server): keep Desktop custom endpoint API keys out of config.yaml The Custom Endpoints panel wrote the raw key to providers.<id>.api_key, so the credential sat in plaintext in a file users routinely share and commit. The input is masked, so nothing warned them. Write the key to .env and reference it via key_env, the same indirection built-in providers use and that runtime_provider already resolves. The read side has to move with it: reporting has_api_key from api_key alone would show "no API key" for every migrated endpoint, and activate copying only api_key would drop the credential entirely. Delete now clears the .env slot too, and an entry still carrying a pre-fix plaintext key is migrated on its next save so existing users get cleaned up without re-entering anything — unless the key is a hand-written ${VAR} template, which is already safe and must not be duplicated into a second env var. Fixes #69449 Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com> Co-authored-by: asorry75 <33794789+asorry75@users.noreply.github.com> * fix(cli): store custom endpoint API key in .env instead of config.yaml hermes model's custom-endpoint flow is the other write path that produced a plaintext key, on both the model block and the custom_providers entry. Route it through the same .env indirection as the Desktop panel, and swap an existing entry's inline key for the reference when the URL is re-saved. Co-authored-by: liuhao1024 <sunsky.lau@gmail.com> * test: cover custom endpoint key storage and model-list persistence Bug-class coverage for both fixes: the full catalogue survives Save, context lengths are preserved, the key never lands in config.yaml on either write path, blank clears it, a pre-fix plaintext key migrates while a ${VAR} template is left alone, two endpoints on one host keep separate credentials, and an IP-derived name is still a valid POSIX env var. The two delete tests asserted on the plaintext mirror; they now assert the same invariants against the credential reference. * fix(desktop): persist @image: refs instead of the vision-enrichment text The desktop gateway passed the vision-enriched, model-only message text (carrying an `image_url:<path>` hint) straight into run_conversation as the persisted user turn. The renderer only parses `@image:<path>`, so it could not rebuild the attachment from history: after a restart the image was gone and only the caption survived, and on a live session switch the warm cache disagreed with the authoritative text and the frontend "rescued" the image by appending it after the caption. run_conversation already supports persist_user_message for exactly this "what the model sees" vs "what gets stored" split; it was simply never wired up for the attachment path. * fix(desktop): keep cached attachment refs on session resume Persisted history carries no attachment metadata for non-image refs, so resume reconciliation dropped `@file:` chips off a user turn whose text matched. Carry the warm cache's refs forward when the resumed message has none of its own, never replacing refs that are already present. (cherry picked from commit eac5b0a8ac39eab242a5d571531e386ec70e2435) * fix(desktop): quote persisted @image: paths so spaced paths render The unquoted alternative in the directive pattern is `\S+`, so a ref built by string interpolation truncates at the first space and strands the tail as loose text next to a broken thumbnail. Composer images live in the app's userData dir, which on macOS is `~/Library/Application Support/<App>/` — so every pasted or dropped image hit this. Adds format_reference_value next to REFERENCE_PATTERN, mirroring formatRefValue in the desktop's directive-text.tsx, and covers the round-trip through the parser. * fix(desktop): persist the image ref for natively-vision-capable models too A turn routed to a model that takes pixels directly sends `content` as a parts list, and the session store deliberately ignores a plain-string persist override for a list payload — a text override must not erase a turn's image summary. So the override was dropped for every user on a vision-capable main model, and the durable row kept only the caption plus a literal `[Image attached at: ...]` / `[screenshot]`, which the renderer cannot turn back into an image. Only vision-preprocessed (text-mode) turns were actually fixed. Mirror the shape instead: swap the text part for the `@image:` ref form and keep the image parts, so the model still has the pixels for the rest of the session, and drop the `[screenshot]` stand-in on the way into the bubble when a ref was lifted from the same message. * refactor(desktop): memoize the directive image-segment filter Matches the two derived values above it and fixes the indentation. * fix(desktop): lead persisted image turns with the caption Session previews are the first 60 characters of the first user message, so persisting the @image: directives ahead of the caption labelled the session with a truncated file path in the sidebar, session switcher, and command palette. Clients lift the refs out of the body line by line, so moving them after the caption changes nothing about how the turn renders. * test(desktop): cover attached-image resume end to end The unit tests cover each layer in isolation, but nothing exercised the whole chain the bug lived in: the real gateway persisting an attachment, SessionDB holding it after the process exits, and the renderer rebuilding a thumbnail from the stored turn. Seeds a session through the real gateway with an image attached, then launches desktop against it — so the first render is already the relaunch case. Pins native image routing (the majority path, and the one where a text-only persist override is dropped) and stages the file behind directory and file names with spaces, mirroring the macOS composer's Application Support path. * fix(models): resolve custom provider model ids Map picker-prefixed custom provider selections back to their configured model IDs before validation, persistence, and API requests. Fixes #68347 * chore(contributors): map jevin@jevin.org to ijevin Attribution check needs a mapping for the cherry-picked commit's author so release notes credit them correctly. * fix(relay): normalize forwarded Discord interactions to leading-slash commands (#71048) A real APPLICATION_COMMAND interaction forwarded over the relay arrived slash-less: _discord_interaction_to_event set text = data['name'] ("new", not "/new"), MessageType.TEXT, and dropped options entirely — so a registered /new dispatched as plain chat instead of a command (MessageEvent.is_command() is text.startswith("/")). Port the connector's Slack slash-command precedent (normalizeSlackCommand builds `${command} ${args}`.trim() with a leading slash and explicit command type): for type-2 interactions build "/" + name, append rendered options space-separated (scalar options contribute their value, matching the native adapter's f"/model {name}" shape; SUB_COMMAND/ SUB_COMMAND_GROUP contribute their name then recurse into nested options), and set MessageType.COMMAND. Type-3 (custom_id) and other interaction types are unchanged. This implements the interaction->command sub-design previously flagged as deferred in the _on_passthrough docstring. Companion connector fix in gateway-gateway: fix(relay): strip own-mention prefix so addressed slash commands dispatch. * feat(desktop): add session link title resolver Resolve @session:<profile>/<id> reference values to the session's title: the in-memory sidebar list answers most lookups, and an unknown id falls back to GET /api/sessions/{id}. Cache, in-flight dedupe, and subscriber fan-out mirror the external-link title resolver. An untitled row resolves to empty rather than "Untitled session" so the caller's short-id fallback stays the chip label. * feat(desktop): show resolved titles on @session chips Route session refs in the transcript through the title resolver so a dropped session reads as its title instead of a truncated id, and use Tabler's funnel for the session chip icon. * feat(desktop): render agent-written @session links as chips Assistant text goes through the markdown renderer, not DirectiveContent, so a session reference an agent wrote came out as literal text. Rewrite bare refs into `#session/<value>` links during markdown preprocessing and dispatch that href to the shared chip in MarkdownLink, alongside the existing media and preview hrefs. Preprocessing already skips code fences and inline code, so a ref being discussed in code stays literal. The pure parsing/href helpers move to session-refs.ts to keep the resolver's React and API imports out of the per-flush preprocess path. * fix(sessions): export delegate cascade before deletion * refactor: extract lineage_is_logical local + document TOCTOU re-query Follow-up cleanup for PR #71123: - Extract getattr(args, 'lineage', 'single') == 'logical' to a local (appeared 3x in the export block) - Document that the double _collect_delegate_child_ids traversal in delete_session is an intentional TOCTOU guard inside the write txn * feat(session-search): give the agent a link to hand back Asked to link to a session, the agent had no way to know the @session reference syntax exists — every mention in the tool schema described consuming a link the user dropped, never writing one — so it answered with the title and timestamp as prose and the desktop had nothing to render. Every result now carries a ready-to-copy `link`, and the schema says to write it inline instead of restating the title around it. The profile segment is omitted when the active profile can't be named confidently; a bare id still resolves. Also skip linkifying a ref a model already wrapped in a markdown link, which would otherwise rewrite into a nested link. * fix(tui_gateway): retain failed turns as replayable inflight snapshots A turn that ended in error cleared inflight_turn and emitted its terminal frame in the same breath. If the client was disconnected during that window (the exact case for a failure like a network drop), the frame went to the detached drop-transport and the in-memory state was already gone — the desktop reconnected to a session with no trace of the failure. Failed turns now retain a compact error snapshot (user prompt, partial assistant text, error, recoverable) that session.resume's inflight payload carries to a reconnecting client. Covers all three loss sites: the returned-error result path, the turn exception path (which now closes with the same status:"error" message.complete frame shape instead of a bare error event), and agent-init failure. The snapshot lives until the next turn starts or the session closes; _run_prompt_submit replaces a retained error leftover instead of appending onto it. Co-authored-by: Reza Sayar <rsayar@uvic.ca> * feat(desktop): crash-survivable in-flight turn journal The renderer's session-state cache is memory-only and the backend's inflight snapshot dies with the backend process, so nothing survived a full app or machine death mid-turn: reopening the session showed the transcript up to the last committed turn and silently dropped everything the crashed turn had streamed. While a turn runs, the visible tail (user prompt + streamed assistant rows, tool calls included) is now journaled to localStorage — throttled off the delta-flush hot path, bounded (24 entries / 7 days), cleared the moment the turn settles. Session resume folds the journaled tail back onto the restored transcript. When the backend also has a live text-only inflight projection for the same turn, the journal overlays its richer structure onto that row (longer text wins, base row id kept so live deltas keep landing) instead of treating it as caught up — the ordering defect that dropped locally recorded tool progress in the original PR. Co-authored-by: Omar Baradei <omar@kostudios.io> * fix(desktop): surface terminal error frames as failed bubbles message.complete frames with status "error" were detected only by a text regex heuristic, which misses the gateway's "Error: <detail>" texts and partial-text failures — a failed turn rendered as a healthy reply. The structured e…

* test(runtime): cover managed SQLite cutover (E-949) * fix(runtime): preserve cutover lifecycle on retry (E-949) * fix(runtime): request minor line for SQLite runtime repair + tests Follow-up on the #70186 salvage. The cherry-picked repair pinned the candidate to the exact current CPython patch (e.g. 3.11.14). Verified live with uv 0.11.19: every published python-build-standalone artifact for 3.11.14 links vulnerable SQLite 3.50.4 — even with --reinstall — so the exact-patch pin made the repair permanently impossible on the installs that need it most (repair_vulnerable_runtime returned 'failed: could not provision a fixed private Python runtime'). Request the minor line (3.11) instead — the same resolution a fresh 'uv python install' would make, still inside requires-python — and tighten the drift gate to 'same minor, no downgrade'. E2E-verified end-to-end on a real vulnerable venv: repair_vulnerable_runtime() provisioned 3.11.15, built + smoke-tested the sibling venv, cut over, and reported SQLite 3.50.4 → 3.53.1 with the old venv parked for rollback. * test: accept kwargs in managed_uv fixture fakes The runtime-repair change passes repair_observer= to update_managed_uv/ ensure_uv; the autouse fixture fakes had zero-arg signatures and raised TypeError through the mock. Sibling test file to the PR's own suite. * fix(compression): recover rotated session lineage * chore: map contributor ruizanthony * fix: pre-lease drift guard must not fire on in-place compaction or mutated snapshots The salvaged drift check compared durable rows to the in-memory snapshot by content and ran in both modes. Two problems: 1. In-place compaction (the default) archives non-destructively — drift cannot lose data there, and the strict-prefix content comparison failed against seeded histories, aborting every in-place compaction (5 test failures in test_in_place_compaction.py). 2. Content equality wedges on sessions with legal in-memory mutation of past turns (multimodal compression, retry replacement) — the same permanent-abort shape as #14694. Now rotation-only and length-based: abort only when the durable parent has MORE rows than the snapshot (a writer committed in the lease window). Dead helper _durable_history_matches_snapshot removed. * test: order compression-tip fixtures around the closed-parent write guard Two compression-tip hydration tests simulated legacy state by emptying the parent AFTER end_session(compression) — exactly the durable write the new closed-parent guard refuses. Reordered: empty first, close second. The tests' actual contract (old id hydrates from the live tip) is unchanged and still pinned. * fix(checkpoints): never auto-delete orphans on unattended startup sweep Builds on this PR's diagnosis by @Frowtek: a missing workdir is ambiguous (deleted project vs. an unmounted external volume / network share / VPN not yet up), so it's not safe evidence for a destructive GC sweep — especially one that runs unattended at startup. - cli.py / gateway/run.py: the startup auto-maintenance sweep now always passes delete_orphans=False to maybe_auto_prune_checkpoints(). It still prunes by retention_days, size cap, and legacy archives — none of which require guessing whether a project was deleted or is just temporarily unreachable. - hermes_cli/config.py: drop the now-unused delete_orphans default. - hermes_cli/checkpoints.py: `hermes checkpoints prune` (the explicit, human-invoked path) now previews the orphan project list and asks for confirmation before deleting, unless -f/--force is passed. - Docs updated (EN + zh-Hans) to match. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(checkpoints): include pre-v2 shadow repos in orphan preview store_status()["projects"] only ever covered v2 metadata, so the `hermes checkpoints prune` confirmation prompt was blind to pre-v2 base/<hash>/HEAD shadow repos that prune_checkpoints() deletes separately via shutil.rmtree — a pre-v2-only or mixed store could lose checkpoint history without ever hitting the confirmation. Extract the pre-v2 scan into _pre_v2_shadow_repos() and have both store_status() (preview, new pre_v2_projects key) and prune_checkpoints() (deletion) read from it, so the CLI prompt can no longer diverge from what actually gets removed. Addresses review from egilewski on #69141. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test(checkpoints): cover prune decline/accept/--force for pre-v2-only and mixed stores Requested by egilewski on #69141: the orphan confirmation flow had no test coverage at all before this. Exercises hermes_cli.checkpoints.cmd_prune directly against pre-v2-only and mixed (v2 + pre-v2) fake stores — decline aborts with nothing deleted, accept deletes both layouts, --force and --keep-orphans skip the prompt as expected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(checkpoints): bind orphan confirmation to previewed identities Address P1 from PR review: cmd_prune()'s y/N preview reads store_status() but the confirmed deletion re-scans both the v2 and pre-v2 layouts from scratch. A workdir that goes missing while the human is answering the prompt gets swept in as if it had been shown and approved. prune_checkpoints() now accepts orphan_allowlist — a set of v2 project hashes and/or pre-v2 shadow repo paths. When set, only orphans whose identity is in the set are deleted; anything newly orphaned since the scan survives the run. cmd_prune() builds this set from the exact projects it just displayed and passed confirmation for. --force still passes None (no preview shown, so nothing to bind to). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * doc(checkpoints): add cyberpunk infographic for startup sweep safety * doc(checkpoints): update infographic to show 8 files * fix(checkpoints): bind an empty orphan preview to an empty deletion allowlist Follow-up for salvaged PR #69141, addressing the last open review point: cmd_prune() only set orphan_allowlist inside 'if orphans or pre_v2_orphans', so a zero-orphan preview passed the unrestricted None sentinel down to prune_checkpoints(), authorizing deletion of any project that became orphaned between the preview and the rescan — with zero confirmation calls. The allowlist is now bound unconditionally for every non-force run (empty preview => empty allowlist); --force keeps None. Adds the zero-orphan-preview timing regression plus allowlist-identity tests. * fix(agent): cache static system prompt prefixes * fix(prompt-caching): inject cache breakpoints after message normalization The conversation loop normalizes message text right before the API call so the request prefix is byte-identical across turns -- the stated reason is KV cache reuse on local inference servers and better cache hit rates on cloud providers. Cache breakpoints were injected *before* that pass, which defeats it. `_apply_cache_marker` rewrites a plain-string `content` into a `[{"type": "text", ...}]` block. The normalization pass is guarded on `isinstance(content, str)`, so every message that just got marked is silently skipped by it and keeps its raw leading/trailing whitespace. A message is only marked while it sits in the last-3 window, so: turn N in the window -> marked, content "file1\nfile2\n" turn N+1 rolled out -> plain, content "file1\nfile2" The same logical message is sent with different bytes on consecutive turns. The prefix stops matching at that position -- which is inside the span the breakpoints were placed to protect -- so the reusable prefix collapses back toward the system breakpoint on every turn. Tool results carry a trailing newline almost by default (any shell command output), so this is the common case, not an edge case. Move the injection below every message mutation. Besides fixing the whitespace divergence this stops breakpoints from being spent on messages that the orphan sweep or the thinking-only drop is about to remove or merge away -- a marker on a dropped message is a wasted breakpoint out of the four available. Nothing between the old and new call sites reads `cache_control`, and the mutators now see the plain-string shapes they were written against. * fix(caching): reconstruct static system prefix on session restore and post-compression reuse Follow-up to the cherry-picked #68258 base: the cross-session-stable prefix (_cached_system_prompt_static) was only recorded on fresh builds, so two paths silently degraded to the legacy single-breakpoint layout (flagged in review of #68258/#69341/#69704): - Session restore: gateway surfaces build a fresh AIAgent per turn and restore the persisted prompt verbatim from the session DB; the static prefix stayed None from turn 2 onward, flip-flopping the wire layout. - Post-compression cached-prompt reuse: _invalidate_system_prompt() clears the static prefix, and the keep-cached-prompt branch never restored it. Both sites now reconstruct the stable tier and adopt it ONLY when the authoritative prompt string literally startswith() it — stable-tier drift (skills edited, identity changed) falls back to the legacy layout with the stored bytes untouched. Fail-open on any builder error. The restore-path rebuild is gated on _use_prompt_caching so non-Anthropic routes skip it entirely. Refs #68191 Co-authored-by: JonthanaHanh <92574114+JonthanaHanh@users.noreply.github.com> Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com> Co-authored-by: webtecnica <webtecnica@users.noreply.github.com> * fix(config): preserve opaque .env values The .env sanitizer inferred missing newlines from known KEY= substrings inside existing values. Plain secrets containing those bytes could therefore be split into synthetic assignments and rewritten to disk. Treat each physical line as the only assignment boundary and keep bytes after the first equals sign opaque for boundary discovery. Preserve safe formatting, null-byte removal, BOM handling, and normal one-assignment-per-line parsing. Cover direct loading, dotenv loading, sanitization, writers, and migration with behavioral regressions. Fixes #29155 * fix(web): resolve per-profile gateway state for ?profile= in /api/status When ?profile=<name> was passed to /api/status, the handler used _config_profile_scope to set the HERMES_HOME contextvar override, but the gateway liveness check (get_running_pid_cached) and runtime status read (read_runtime_status) both resolve _get_process_hermes_home(), which deliberately ignores contextvar overrides (issue #56986) — it always reads os.environ['HERMES_HOME'] or the platform default. A named profile's gateway identity files (~/.hermes/profiles/<name>/gateway.pid, gateway_state.json) were therefore never found and the endpoint always reported the profile's gateway as stopped. Fix: when ?profile=<name> is requested, resolve the profile directory and pass explicit profile-scoped paths: - get_running_pid_cached(pid_path=profile_dir / 'gateway.pid') - read_runtime_status(path=profile_dir / 'gateway_state.json') - get_runtime_status_running_pid(..., expected_home=profile_dir) This is the same explicit-path pattern _collect_profile_gateway_topology already uses for per-profile gateway state, and it works within the #56986 constraint (no HERMES_HOME env mutation; read-only cross-profile access). Plain /api/status without ?profile= keeps the exact zero-arg calls, so its behavior — including the pid-cache signature and runtime-status fallback — is byte-for-byte unchanged. Fixes #69143 * test(web): pin per-profile gateway state scoping on /api/status Follow-up for the salvaged #70498 fix: replace the original PR's mock-signature churn (28 lambda **kw edits, needed only because it changed the no-profile call shape) with two targeted regression tests: - ?profile=<name> must pass the profile's gateway.pid / gateway_state.json paths and expected_home to the gateway status readers (HOME-anchored per-profile state under ~/.hermes/profiles/<name>/) - ?profile=<unknown> must 404 via _resolve_profile_dir The production change keeps plain /api/status on the exact zero-arg calls, so every pre-existing test passes unmodified. * test: accept the new profile-scoped kwargs in status fakes /api/status?profile= now passes pid_path=/path=/expected_home= to the PID and runtime-status readers; the profile-unification fakes had zero-arg signatures and raised TypeError. Plain /api/status call shapes are unchanged (pinned by the existing zero-arg tests in test_web_server.py). * fix(gateway): prevent reconnect watcher wedge after network-loss fatal error (#70344) Three-part fix for the gateway going silently deaf after a retryable fatal adapter error (e.g. httpx.ConnectError on Telegram): 1. **Detach-on-timeout in _connect_adapter_with_timeout** — Replaced plain asyncio.wait_for with the task-detach pattern used by _await_adapter_cleanup_with_timeout. asyncio.wait_for cancels the overdue task but then waits for it to exit, so a connect() that catches CancelledError can block recovery forever. The detach pattern releases the runner at the deadline via consume_detached_task_result. 2. **Ensure reconnect watcher always runs after escalation** — Added _ensure_reconnect_watcher_running(), called after queueing a retryable fatal error. If the reconnect watcher task has died (exhausted restart budget, terminal exception), it is respawned so queued platforms are never permanently stranded. 3. **Faulthandler at gateway startup** — Enabled faulthandler + SIGUSR2 dump to a rotating file under HERMES_HOME/logs/ for post-mortem diagnosis of future event-loop freezes. Tests added for _ensure_reconnect_watcher_running (alive, dead, not-started, not-running), fatal-error integration (retryable calls ensure, non-retryable does not), and _connect_adapter_with_timeout (timeout raises, success returns). * fix: explicit encoding for faulthandler file open (ruff PLW1514) * fix(gateway): stay alive on mixed retryable + non-retryable startup failures When connected_count == 0 and at least one platform failed with a non-retryable error, the runner exited with GATEWAY_FATAL_CONFIG_EXIT_CODE (78) even if OTHER platforms failed for merely transient reasons. Real-world shape (NS-609, hosted instance): WhatsApp enabled but never paired (non-retryable whatsapp_not_paired) + Telegram TimedOut during polling startup (retryable) => exit 78 => the gateway either goes permanently down (supervisors honoring the exit-78 contract via RestartPreventExitStatus / the s6 finish->125 translation from #51228) or crash-loops (anything else). Either way Telegram never gets its retry and the dashboard drops with every exit, so a single unpaired platform plus one network blip disconnected every channel on the instance. Now exit 78 is reserved for the case where ALL startup failures are non-retryable (true config error, nothing to wait for). With mixed failures the gateway stays alive in degraded state: the reconnect watcher recovers the retryable platforms and the misconfigured ones stay fatal-parked and visible in runtime status. * fix: gate SIGUSR2 faulthandler registration behind POSIX check signal.SIGUSR2 and faulthandler.register() don't exist on Windows; the bare reference raised AttributeError at import time per the windows-footgun checker. faulthandler.enable() still covers fatal-error dumps on all platforms. * fix(gateway): detect and escape silent event-loop freezes - A self-rescheduling 5s call_later floor timer, armed before any adapter connects, guarantees the selector always has a finite timeout, so the existing async defenses (polling heartbeat, timeout guards) regain a chance to run after a zero-pending-timer stall. - A resident daemon-thread liveness watchdog probes the loop via call_soon_threadsafe every 30s; after 3 consecutive 10s-timeout misses (~120s of total unresponsiveness) it dumps all thread tracebacks and exits with the established GATEWAY_SERVICE_RESTART_EXIT_CODE (75) so a supervisor restarts the gateway - async-level recovery cannot run on a frozen loop. - stop() disarms both guards before any teardown await so a busy shutdown is never misjudged as a freeze. HERMES_GATEWAY_LOOP_WATCHDOG=0 disables; _INTERVAL/_TIMEOUT/_STRIKES tune the thresholds. Fixes #69089 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gateway): close watchdog shutdown race against final-strike exit - Re-check stop_event after a missed probe (before the strike increment) and again on entering the final-strike branch (before the critical log, dump, and hard exit), so a normal stop() landing between the last timeout check and the exit path can no longer be misclassified as a freeze and trigger a supervisor restart. - Deterministic boundary tests pin both re-checks independently (mutation-verified: removing either check turns its own test red); frozen-loop semantics are unchanged. Addresses the shutdown-race review on #69164. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gateway): recheck stop immediately before watchdog hard exit - A stop() landing while the final diagnostics (critical log, traceback dump) are executing could still reach os._exit(75) after the pre-diagnostic check. Add a third stop_event recheck immediately before the hard exit: diagnostics may complete, but a disarmed watchdog never exits. - Deterministic regressions for both windows (stop triggered from inside logger.critical and from inside faulthandler.dump_traceback); mutation-verified (removing the check turns both red). Frozen-loop semantics unchanged. Addresses the second round of the shutdown-race review on #69164. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(gateway): gate loop-liveness watchdog via config.yaml, drop HERMES_* env knobs Follow-up to the salvaged #69164 commits: policy forbids introducing new HERMES_* environment variables, so the four watchdog env knobs (HERMES_GATEWAY_LOOP_WATCHDOG / _INTERVAL / _TIMEOUT / _STRIKES) are replaced with a single config.yaml boolean: gateway: loop_watchdog: true # default; false disables both guards - gateway/config.py: new GatewayConfig.loop_watchdog field (default True), parsed from top-level or nested gateway: form, round-trips via to_dict/from_dict. - gateway/run.py: _start_loop_liveness_guards() checks config.loop_watchdog before arming the floor timer + watchdog (getattr-guarded for bare object.__new__ runners). - gateway/shutdown_watchdog.py: start_loop_liveness_watchdog() no longer reads the environment; probe interval/timeout/strikes are module constants (30s/10s/3 — ~90s to restart, matching the systemd watchdog layer's posture). - hermes_cli/config.py: documented gateway.loop_watchdog default so 'hermes config set gateway.loop_watchdog false' validates. - tests: env-knob tests replaced with config-gate + round-trip tests; the final-strike boundary test injects its probe via max_strikes directly instead of patching the removed env helper. * fix: getattr-guard _stop_loop_liveness_guards in GatewayRunner.stop Teardown-path tests build bare runners via object.__new__ without the liveness-guard machinery; the unguarded call raised AttributeError in 8 tests. Same guard pattern as the start path. * fix(desktop): close cross-session leak windows in composer + session refs (#59305) Two React passive-effect timing bugs let a session switch land in the wrong chat: activeSessionIdRef/selectedStoredSessionIdRef (use-session-state-cache) and the composer's attachment-scope swap (use-composer-draft) both mirrored their source props via useEffect, which fires one commit AFTER the new session's view has already painted — a synchronous read/submit in that window observed the outgoing session's ids/attachments. - use-session-state-cache.ts: mirror the session refs synchronously during render instead of a useEffect, guarded to fire only when the prop itself changed (not unconditionally) so an imperative pin from submit.ts / use-session-actions (e.g. a freshly resumed runtime id, intentionally not synced to the source atom) survives an unrelated re-render. - use-composer-draft.ts: the per-thread attachment-scope-swap effect is now a useLayoutEffect, closing the window before paint. - submit.ts / session-context-drift.ts: add a 3rd drift prong comparing the composer's loaded scope (SubmitTextOptions.composerScope) against the submit target, resolved into the same lineage-root domain (resolveComposerSessionKey) the composer itself uses — comparing against the raw tip id would false-positive-abort every submit into any session that has ever auto-compressed. - routes.ts / chat/index.tsx: the primary composer's durable scope key now prefers the route over a possibly-stale store selection (primaryRouteSelectedSessionId). - use-composer-draft.ts: redacted [composer-rehydrate] diagnostic log (counts/kinds/scope only, never raw refs) for future reports in this class. - chat-runtime.ts: normalize attachment id values (url/path) before hashing so a re-attach with a trailing slash or backslash path dedupes correctly. 16 files, 286 tests across the touched/dependent suites (17 files) green, including new regression coverage for each fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(desktop): satisfy eslint import-order rules in use-composer-draft.test.tsx CI's check:lint failed on two perfectionist rule violations introduced by the new test file: type import ordering and missing blank line between the parent-relative and same-directory import groups. No behavior change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(agent): prevent shared OpenAI client FD-recycle corruption from stale stream watchdog The streaming stale watchdog was calling _replace_primary_openai_client() from its polling thread, which closes the shared client's connection pool. Worker threads from previous stale-killed attempts may still be unwinding their SSL BIOs, causing TLS application-data to overwrite SQLite file headers via FD reuse. This is the same corruption vector documented in #67142 for Anthropic, where the fix was to never close the shared client from a non-owner thread. Apply the same pattern to the OpenAI-wire path: - Stale stream watchdog: skip shared client replacement - Mid-tool-retry cleanup: skip shared client replacement - Stream retry cleanup: skip shared client replacement The request-local client is already closed via _close_request_client_once. The shared client is replaced lazily by _ensure_primary_openai_client on the next request, which runs on the owning thread. Closes #70773. * fix(agent): retire replaced shared OpenAI clients instead of cross-thread pool close Widen the #70773 fix beyond the three in-request cleanup sites removed in the cherry-picked commit: every remaining path that swaps out the shared OpenAI client could still hard-close its pool from a thread that doesn't own the in-flight sockets (credential rotation/refresh on the turn thread, dead-connection cleanup, gateway cache eviction, transport recovery) — the same FD-recycle corruption vector, just rarer. Add AIAgent._retire_shared_openai_client(): shutdown(SHUT_RDWR) all pooled sockets (FD-safe from any thread, unblocks in-flight readers) but never call client.close() — FD release is deferred to GC, which cannot run until every borrowing thread has unwound its SSL BIO. Refcounting is the ownership handshake; with no borrowers the FDs are released immediately. Wired into: - _replace_primary_openai_client (rotation/refresh/dead-conn cleanup) - try_recover_primary_transport (primary_recovery) - release_clients (gateway cache_evict) agent.close() keeps the hard close: full teardown is a real session boundary where no request may be in flight. Tests: new tests/run_agent/test_70773_shared_client_fd_corruption.py covers the three watchdog/retry sites plus retire semantics; existing close-assertions updated to pin retire-not-close. * test: update credential-refresh tests for retire-not-close contract The three refresh tests asserted the replaced shared client gets close()d — the exact cross-thread close #70773 removes. They now pin the new contract: close() is NOT called from the refresh path; the old client is retired (sockets shutdown, FD release deferred to GC). * fix(doctor): UTF-8/latin-1 fallback when scanning .env Prefer UTF-8 for ~/.hermes/.env provider scans, then latin-1 for cp1252/Notepad files. Add regression test for invalid UTF-8 bytes. * fix: handle non-UTF-8 files in OpenClaw migration script * fix: decode config and state files as UTF-8 on non-UTF-8 locales Several file-I/O call sites still use open() / Path.read_text() / Path.write_text() without an explicit encoding, so they fall back to the platform default. On Windows CN/JP/KR locales (GBK/CP932/CP949) any non-ASCII byte in a config/state/user-content file raises UnicodeDecodeError or UnicodeEncodeError and crashes the caller. to the remaining hot paths: - agent/copilot_acp_client.py: fs/read_text_file and fs/write_text_file (Copilot's read_file / write_file tools, directly reported in #18637 bug 2) - agent/model_metadata.py: context-length YAML cache load + two save sites (context probing is on the call path of every model invocation) - agent/nous_rate_guard.py: cross-session rate-limit JSON state (read + atomic write via os.fdopen) - cron/scheduler.py: user config.yaml read in run_job - gateway/delivery.py: cron output writes for AI-generated content, very likely non-ASCII yaml.dump call sites also gain allow_unicode=True so the emitted YAML preserves non-ASCII chars as-is instead of emitting \u escape sequences. Adds regression tests that monkeypatch builtins.open / Path.read_text / Path.write_text to simulate a GBK locale: each test raises UnicodeDecodeError / UnicodeEncodeError unless the caller explicitly passes encoding='utf-8'. Verified that the tests fail on main and pass with this change, on Linux as well as on Windows. Refs #18637 * fix(cli): add explicit encoding to read_text/write_text calls Path.read_text() and Path.write_text() without explicit encoding default to the system locale encoding. On Windows this is typically cp1252, which causes UnicodeDecodeError for UTF-8 content (JSON configs, user data, service scripts). Add encoding="utf-8" to all read_text() and write_text() calls across 8 CLI files, matching the pattern established in PR #50534 (security_audit_startup.py) and ruff rule PLW1514. Fixed files: - main.py: 4 read_text calls - auth.py: 3 read_text calls - banner.py: 1 read_text + 1 write_text - service_manager.py: 1 read_text + 4 write_text - container_boot.py: 1 read_text + 4 write_text - doctor.py: 3 read_text calls - uninstall.py: 2 read_text calls - gateway.py: 1 write_text call * fix(core,cli,gateway,plugins): add encoding='utf-8' to read_text() calls Path.read_text() without an explicit encoding uses the platform's default encoding. On Windows this is typically cp1252 or mbcs, which causes UnicodeDecodeError or silent data corruption when reading UTF-8 content (JSON files, user text, config with non-ASCII chars). This is the read-side companion to the write_text() encoding fix. Fixed the most critical locations that read JSON data, user content, and config files across 14 files with 31 call sites. Pattern: .read_text() → .read_text(encoding='utf-8') json.loads(path.read_text()) → json.loads(path.read_text(encoding='utf-8')) * fix(install): emit UTF-8 from skills_sync on non-UTF-8 Windows locales On Windows with a non-UTF-8 system locale (e.g. CP936/GBK on zh-CN), Python defaults stdout/stderr to the active codepage. tools/skills_sync.py prints glyphs such as checkmark (U+2713) and up-arrow (U+2191) that GBK cannot encode, raising UnicodeEncodeError mid-run. The installer (scripts/install.ps1) captures this script's stdout and the Rust bootstrap parses it as UTF-8 expecting a JSON result frame. A GBK byte stream (or the traceback it triggers) surfaces as: WARN stdout read error: stream did not contain valid UTF-8 stage=config-templates state=Failed error=install.ps1 -Stage config-templates produced no JSON result frame (exit=Some(0)) i.e. the stage fails even though the script exits 0. install.ps1 already sets [Console]::OutputEncoding = UTF8, but that does not propagate to the python.exe child (Python reads PYTHONIOENCODING / locale, not the console encoding). Fix in two places for defense in depth: - tools/skills_sync.py: reconfigure sys.stdout/stderr to UTF-8 at import so output is valid UTF-8 regardless of caller or active codepage. - scripts/install.ps1: set PYTHONIOENCODING=utf-8 and PYTHONUTF8=1 (scoped to the call, restored afterwards) around the skills_sync.py invocation. * test(install): add UTF-8 regression guard for skills_sync child path Addresses hermes-sweeper review on PR #54866: the installer runs tools/skills_sync.py as a child python.exe whose PYTHONIOENCODING / PYTHONUTF8 the scoped install.ps1 block sets, but there was no regression test for this child-Python UTF-8 path. The existing test_child_process_inherits_utf8_mode covers a different (bootstrap entry-point) flow. Add TestSkillsSyncUtf8Guard: three subprocess tests that import skills_sync (triggering its import-time stdout/stderr reconfigure) and assert the checkmark/up-arrow glyphs the script prints at tools/skills_sync.py:596,675 emit valid UTF-8 and exit 0 even when the child env is left unset or explicitly hostile (gbk). A third test proves the guard is load-bearing by reproducing the crash without it. Also keep the new install.ps1 comment ASCII-only (the checkmark spelled out as U+2713) per the file's PS 5.1 parser-compatibility contract at scripts/install.ps1:79-80; the literal glyph in the comment violated that contract. * fix: add encoding="utf-8" to Path.write_text() calls (P1) Path.write_text() without encoding defaults to system locale encoding. On Windows (cp1252), this silently corrupts non-ASCII content written to JSON files, config files, and cache files. This is the write-side counterpart to the read_text() encoding fix (PR #56115). PLW1514 only covers open() calls — Path methods are unguarded by ruff. 39 instances across 16 files, all passing py_compile. Files changed: - agent/copilot_acp_client.py (1) - tools/web_tools.py (1) - tools/xai_http.py (1) - tools/skills_hub.py (8) - gateway/slash_commands.py (1) - gateway/run.py (5) - gateway/dead_targets.py (1) - gateway/delivery.py (2) - gateway/platforms/qqbot/adapter.py (1) - hermes_cli/gateway.py (1) - hermes_cli/banner.py (1) - hermes_cli/service_manager.py (5) - hermes_cli/container_boot.py (5) - hermes_cli/uninstall.py (1) - hermes_cli/main.py (2) - hermes_cli/profiles.py (3) * fix(hindsight): specify UTF-8 encoding for file I/O on Windows On Windows with CJK locales (e.g. Chinese/GBK), pathlib.Path.read_text() defaults to the system encoding instead of UTF-8, causing UnicodeDecodeError when reading .env or .json config files that contain non-ASCII characters. Explicitly pass encoding='utf-8' to all read_text() and write_text() calls in the hindsight memory provider plugin. * fix(memory): read/write .env as UTF-8 in mem0 and hindsight setup The mem0 and hindsight memory-provider setup routines round-trip the user's ~/.hermes/.env: they read existing lines, update the keys they manage, and rewrite the whole file preserving every other line verbatim. Both used env_path.read_text() / write_text() with no encoding. read_text()/write_text() with no encoding fall back to the system locale (cp1252/GBK on Windows), so on a non-UTF-8 host the preserved lines get mangled or the call crashes on any non-ASCII value, and — because the reader never strips a BOM — a Notepad-edited .env makes the first key fail the in-place match and get duplicated instead of updated. Match the canonical .env readers in hermes_cli/config.py: read with encoding='utf-8-sig' (BOM-tolerant) and write with encoding='utf-8'. mem0/_setup.py already pins utf-8 for mem0.json, so this just aligns the .env path in the same file. Fixes both memory plugins in one class fix. Adds regression tests: a BOM'd .env updates the first key in place (locale-independent, fails without the fix) and non-ASCII existing lines survive the round-trip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(memory): cover the remaining setup-time .env reads with utf-8-sig Follow-up to review feedback: - mem0 _prompt_api_key read .env with the locale default, so a Notepad BOM hid the first key from the masked current-value lookup; read it with utf-8-sig + errors=replace like the canonical readers in hermes_cli/config.py. - hindsight _load_simple_env used plain utf-8; it also parses the Hermes .env during post_setup, where a BOM stuck to the first key. Switch to utf-8-sig + errors=replace. - Add hindsight regressions: BOM key matching in _load_simple_env and in the cloud post_setup writer, plus non-ASCII round-trip preservation, and a mem0 regression for the BOM'd masked-key lookup. The BOM tests fail without the fix on any platform. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(profile): read .env as utf-8-sig in the distribution-install preview `_render_distribution_plan` reads the target profile's `.env` to decide whether a required env var is already set (so it doesn't nag the user), using `Path.read_text()` with no encoding. Two bugs: 1. `Path.read_text()` defaults to the system locale (cp1251/GBK on Windows), which raises `UnicodeDecodeError` on any non-ASCII byte. The surrounding `except OSError` does NOT catch that — `UnicodeDecodeError` is a `ValueError` — so a mis-encoded `.env` aborts the entire install preview. 2. Even on a UTF-8 locale, a Notepad-added BOM prefixes the first key (`KEY`), so the very first required env var is mis-reported as "needs setting" when it is actually present. `.env` is written as UTF-8 everywhere in the codebase. Read it as `utf-8-sig` (tolerates the BOM) and also catch `UnicodeDecodeError` so a genuinely un-decodable file skips the pre-check instead of crashing. Regression tests: a BOM-prefixed `.env` whose first key must still read as "set", and an invalid-UTF-8 `.env` that must not abort the preview. * fix: add UTF-8 encoding to read_text/write_text in tools/ and agent/ Path.read_text() and Path.write_text() without encoding= default to the system locale (cp1252 on Windows), which corrupts non-ASCII JSON content. Coverage-gap fix for files not addressed by prior encoding PRs: - tools/skills_hub.py: 6 read_text + 8 write_text (cache, index, lock files) - tools/skills_sync.py: 1 read_text (lock file) - tools/xai_http.py: 1 read_text + 1 write_text (auth store, marker) - agent/shell_hooks.py: 1 read_text (allowlist) - gateway/status.py: 1 read_text (PID file) - hermes_cli/banner.py: 1 read_text + 1 write_text (update cache) All sites read/write JSON or short text. No behavioral change on Linux (already UTF-8); fixes silent data corruption on Windows. * fix(skills): tolerate non-UTF-8 bytes in hub lock.json _read_hub_installed_names() reads ~/.hermes/skills/.hub/lock.json with a strict utf-8 decode. Hub skill descriptions can carry Windows-1252 typographic bytes (em-dash 0x97, smart quotes, bullets) as single high bytes; read_text(encoding="utf-8") then raises UnicodeDecodeError, which is a ValueError sibling not caught by the function's except (OSError, json.JSONDecodeError). It escapes and 500s the whole /api/skills endpoint, blanking the desktop Skills panel. Decode with errors="replace" so the offending byte degrades to U+FFFD and the structurally valid JSON — and every other skill — stays readable. Fixes #68053 * fix(cli): read .worktreeinclude and .gitignore as UTF-8 in worktree setup _setup_worktree read both files with the locale default encoding. On a cp1251/GBK Windows machine a UTF-8 include list either decodes to mojibake paths (non-ASCII entries silently not copied) or raises UnicodeDecodeError, which the enclosing handler logs at DEBUG and swallows — no include is copied at all, so the worktree starts without .env/keys and the agent breaks invisibly. A Notepad BOM likewise glues to the first include entry on every platform, and to the first .gitignore line, defeating the '.worktrees/' membership check and appending a duplicate entry on each run. Read both files with utf-8-sig + errors=replace, matching the canonical .env readers in hermes_cli/config.py (utf-8-sig because Notepad adds a BOM) and the UTF-8 append this same block already performs on .gitignore. Regression tests exercise the real cli._setup_worktree: the two BOM tests fail without the fix on any platform, the non-ASCII include test additionally reproduces the Windows locale failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(skills): read OOXML parts as bytes and form JSON as UTF-8 in office skill scripts The bundled office skills (#68595) read user documents and agent-authored payloads with the locale-default codec: - docx/powerpoint validators/base.py opened OOXML part XML in text mode before handing it to lxml. On Windows (cp1251/GBK) the bytes decode to mojibake that lxml then parses, so validation runs against silently corrupted document text; on locales where the UTF-8 bytes don't decode the validator crashes with UnicodeDecodeError instead of validating. Opening as bytes lets lxml honor the encoding declared in the XML prolog. - The pdf form scripts (fill_fillable_fields, fill_pdf_form_with_annotations, create_validation_image, check_bounding_boxes) read the fields JSON the agent authors — UTF-8 by construction — with the locale codec, so non-ASCII form values (any Cyrillic/CJK/accented input) either crash or get written into the user's PDF as mojibake. The json.dump writers use ensure_ascii=True and were already safe; only the readers needed pinning. Adds a contract test asserting every document/payload reader is locale-independent, plus a live regression test that runs check_bounding_boxes.py on a non-ASCII fields.json under a forced non-UTF-8 locale — it fails without the fix on both POSIX (C locale) and Windows (cp1251 chokes on the 0x98 byte of U+2018). * fix(windows): sweep remaining bare read_text/write_text sites + linter rule AST-driven pass over every Path.read_text()/write_text() without an explicit encoding= across non-test code: 71 sites in 34 files (skills_hub, hermes_cli/main+profiles+service_manager+container_boot, mem0/hindsight/honcho plugins, achievements dashboard, release/CI scripts, productivity+comfyui skill helpers, agent/*). Verified zero positional-encoding collisions before insertion; per-file compile() check after. Adds a check-windows-footguns rule flagging bare single-line read_text/write_text (multi-line forms stay covered by the AST guard test from #38985). Together with the salvaged contributor commits this retires the ~169-site bare file-I/O class (#37423's long tail). * fix: restore utf-8-sig BOM tolerance at .env readers the sweep normalized The cherry-pick auto-resolution + AST sweep applied plain utf-8 at three .env reader sites where the salvaged PRs (#62617, #62123) deliberately use utf-8-sig — a Notepad BOM must not hide/duplicate the first key. Restore the contract (tests pin it). * chore: contributor email mappings for the file-I/O salvage * refactor(desktop): add shared Field form-dialog primitive Dialog forms each hand-rolled their own label+control+hint stack (or borrowed the settings-surface ListRow), so gaps and hint styling drifted between the profile, cron, and webhook dialogs. Add a single Field / FieldHint primitive for label-over-control dialog fields and adopt it in the create/rename profile dialogs as the first consumers. * fix(desktop): unify overlay-pane padding and add primary PanelAction Overlay panes each set their own top padding, so the Settings sidebar and Panel headers sat at different heights than System/Agents and the close X (the #67759 regression). Hoist the shared beside-the-X clearance into OVERLAY_TOP_CLEARANCE, keep the taller pad only on OverlayMain (which sits under the X), tighten OverlayMain's gutters, and drop the one-off Settings override. Also give PanelAction a `primary` variant so a detail header can promote its main action to a filled button. * refactor(desktop): fold cron Blueprints into the New Job dialog Blueprints lived behind a separate Jobs/Blueprints tab with its own card gallery — a bespoke surface no other overlay uses. Remove the tab and make blueprints a "Start from" dropdown at the top of the New Job dialog (default "Custom" = the manual editor); picking one swaps the form for that blueprint's typed slots. Also promote the detail-view "Trigger now" button to a primary action and adopt the shared Field primitive. * refactor(desktop): webhooks create form uses shared Field; drop status pill The create dialog used the settings-surface ListRow/ToggleRow inside a modal, which read differently from every other form dialog, and the detail header carried an enabled/disabled pill that rendered as a stray dash. Switch the form to the shared Field primitive (+ Switch) and remove the pill. * fmt(js): `npm run fix` on merge (#71099) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(dashboard): add lightweight /api/health liveness endpoint /api/status is the only public liveness route, and its handler loads the gateway config, probes gateway health, and counts sessions before it can answer. That work is wrong for a readiness probe: a caller that only needs to know the process is up pays for a cold plugin import tree. Add /api/health, which returns process liveness, version, and the auth-gate shape and touches nothing else. * fix(desktop): probe /api/health for boot readiness, and survive a stalled loop Desktop boot polls /api/status, so readiness waits on gateway config and a cold plugin import tree. On Windows that regularly outlives the probe and Desktop kills a backend that is already listening, respawns it, and re-pays the same import cost — the reported crash loop. Probe /api/health instead, falling back to /api/status only for the missing-route shapes the fetch helpers emit (404, or HTML from the SPA), so an older remote backend still connects. Timeouts and server errors keep polling health rather than dropping to the heavyweight route. A cheap route is not enough on its own. Warming the gateway import holds the GIL, so the event loop can stall for tens of seconds and starve /api/health too. At the default 15s socket timeout only three attempts fit in the 45s budget; give each probe 5s so the loop keeps retrying across the stall. Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com> Co-authored-by: DESXIE <78300229+DESXIE@users.noreply.github.com> Co-authored-by: frohsinnllc <231045016+frohsinnllc@users.noreply.github.com> * fix(state): decode display_metadata at every message read path get_messages(), get_messages_around() and get_anchored_view() returned the raw display_metadata column instead of the dict every caller expects. The desktop paints a resumed transcript from the REST prefetch, which reads through get_messages(), so any session holding an async_delegation_complete event failed resume with "Cannot use 'in' operator to search for 'task_count'" — on every such session, not just corrupted ones. Route all four read paths through one shared codec that also unwraps rows carrying a second JSON layer, so sessions already broken on disk recover on read rather than needing a migration. Co-authored-by: Studio729 <Studio729@users.noreply.github.com> Co-authored-by: aml1973 <aml1973@users.noreply.github.com> Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> * fix(state): stop double-encoding display_metadata on write export_session() reads through get_messages(), so before the read fix an already-serialized string went straight back into _insert_message_rows() and got re-dumped — an export/import round trip permanently corrupted the row. Guard the three write paths the same way tool_calls already is: parse a string argument before storing it, and drop metadata that isn't an object rather than persisting something no reader can use. Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> Co-authored-by: aml1973 <aml1973@users.noreply.github.com> * fix(desktop): tolerate unparsed display_metadata from an older backend The desktop and the Hermes backend it talks to version independently — a remote VM running an older build still serves display_metadata as JSON text. Indexing into that string with `in` threw and failed the whole resume, so narrow the type to admit a string and parse it before reading task_count. Falling back to the generic label keeps a delegation event renderable even when the metadata is unusable. Co-authored-by: xxxigm <xxxigm@users.noreply.github.com> Co-authored-by: Studio729 <Studio729@users.noreply.github.com> * fix(checkpoints): don't prune a project whose volume is merely unmounted Orphan pruning decides a project is gone from a single probe: if delete_orphans and (not workdir or not Path(workdir).exists()): reason = "orphan" then deletes its ref, index, and metadata — the project's entire checkpoint history. `Path.exists()` is False for a deleted directory, but it is equally False for one whose storage is not attached right now: an unplugged external drive, a share behind a downed VPN, a bind-mount absent from this container, an offline Windows mapped drive. The project is fine; only our view of it is. This is not an opt-in maintenance command. `maybe_auto_prune_checkpoints` runs unattended at startup from both `cli.py` and `gateway/run.py`, with `delete_orphans=True` by default. So starting Hermes once while the drive is unplugged silently destroys the restore points for every project on it — the one thing checkpoints exist to provide, and there is nothing to restore from afterwards. Reproduced against the real store: a project registered under an unmounted path and one on local disk, then a startup prune — prune: {'scanned': 2, 'deleted_orphan': 1} unreachable project index still on disk: False The legacy pre-v2 branch has the same flaw plus a second one: a `HERMES_WORKDIR` marker that exists but cannot be read leaves `workdir = None`, which the same condition treats as an orphan. Failing to read a file is not evidence that a project was deleted. Require corroboration before deleting: the workdir's parent must be present, so its absence is something we actually observed. A missing parent means the volume is not there and we know nothing, so the entry is left alone — and an unreadable marker never deletes at all. Genuinely abandoned projects are still reclaimed, both by the unchanged orphan path (parent present, project gone) and by the retention/stale rule, which runs off `last_touch` rather than a filesystem probe. tests/tools/test_checkpoint_manager.py: a project whose whole mount disappears keeps its history; controls prove a genuinely deleted project is still pruned and a live project is untouched. The data-loss test fails on main; both controls pass there. 81 passed across the checkpoint suites (2 failures in test_checkpoint_manager.py are pre-existing and fail identically on clean main). * fix(checkpoints): an empty surviving mount point is not evidence of deletion Addresses @egilewski's review: the parent-directory check still deleted checkpoint history for the most common unmount layout. Detaching storage removes the parent outright in some layouts (`/Volumes/Ext/proj` on macOS, `/media/<user>/<label>/proj`), which the first commit handles. But in the classic static layout — `/mnt/volume/proj`, an fstab entry, a container bind-mount — unmounting removes the contents and leaves the mount point behind as an empty directory. `parent.is_dir()` is then true, the project is absent, and the startup sweep deletes its ref, index and metadata: exactly the case this PR set out to protect. Reproduced against the real predicate before this commit: mount root vanished (macOS) -> False ok empty surviving mount point -> True <-- history deleted really deleted (siblings) -> True ok An empty parent carries no information: it looks identical whether the volume was detached or the project was deleted. So require the parent to actually say something — it holds some other entry (we observed a populated directory that does not contain the project), or it is itself a live mount point (the volume is attached right now and demonstrably does not hold the project). The cost is that a project deleted out of an otherwise-empty parent is no longer reclaimed by the orphan rule. It is not leaked: the retention rule reads `last_touch` rather than probing the filesystem and still collects it, so reclamation is deferred, not lost. That is the right direction for a predicate whose false positive destroys a user's restore points unattended. `_dir_has_any_entry` stops at the first entry via `os.scandir` instead of materializing a listing, since a project root can hold a large tree. tests/tools/test_checkpoint_manager.py: `test_surviving_empty_mountpoint_ keeps_its_checkpoints` pins the reviewed case, and `test_empty_parent_project_ is_still_reclaimed_by_retention` pins the deferral above so the safety valve cannot silently regress into a leak. Both fail on the previous commit. The real-orphan control now seeds a sibling so it exercises a populated parent rather than the ambiguous empty one. 80 passed in the checkpoint suite; the 2 remaining failures (`TestGitEnvIsolation`, `TestClearFunctions`) fail identically on clean main. * fix(checkpoints): require positive volume-attachment evidence before orphan classification Follow-up to the cherry-picked #69063: egilewski's review found that the _dir_has_any_entry(parent) guard treats ANY entry in the mount point's parent as proof the volume is attached — but unmounting exposes the UNDERLAY directory's own files (e.g. a .keep placeholder), so a populated underlying mount-point dir still classified the project as an orphan and deleted its ref/index/metadata. Reproduced on both main and the PR head. Attachment evidence is now positive instead of circumstantial: * _volume_evidence() records the parent directory's (st_dev, st_ino) identity in the project's metadata while the workdir is observably live (at _register_project/_touch_project time). A mount point resolves to the mounted filesystem's root while attached and to the underlay directory after detach — same path, different directory, different identity. * _workdir_is_observably_gone() now requires the parent visible at prune time to match that recorded identity before the populated-parent check can classify an orphan. A mismatch means a different directory (the underlay) is showing through — a detached volume, not an observed deletion. * Metadata without a recorded identity (written by older versions) is never orphan-classified — unsure never deletes; the retention/stale rule still reclaims genuinely abandoned projects off last_touch. * The frozen pre-v2 layout has no metadata channel for the identity, so it keeps the structural checks only (require_parent_identity=False). * A failed evidence probe on re-registration preserves the previously recorded identity — stale evidence can only make pruning MORE conservative. Windows: st_dev/st_ino of 0 (filesystems without file IDs, some network shares) is treated as "no evidence recorded", which falls into the conservative never-orphan path. os.path.ismount and Path.stat are cross-platform; no POSIX-only calls added. tests/tools/test_checkpoint_manager.py: adds egilewski's exact regression (checkpoint history for mnt/volume/project, detach exposes mnt/volume/.keep, prune with orphan deletion enabled → NOT deleted; fails on the bare cherry-pick, passes with this fix), plus no-recorded-identity conservatism and probe-failure identity preservation. His absent-parent/empty-parent/retention/genuine-deletion/ live-project controls all still pass. Reported-by: egilewski (review on #69063) * fix(telegram): require initial polling readiness Use wall deadlines for deleteWebhook and start_polling, then fail cold startup unless getUpdates proves progress. This lets the gateway discard partial PTB state and retry with a fresh adapter.\n\nRefs #67498 * fix(gateway): allow Telegram readiness budget Give Telegram a 180s default outer connect budget so cold polling can prove getUpdates readiness. Preserve the 30s default for other platforms and all explicit config/env overrides.\n\nRefs #67498 * fix(telegram): bind strict cold-start readiness to its own polling generation Follow-up hardening for the salvaged #69240 readiness gate (#67498): - _start_polling_once now returns its (generation, progress_event) pair so the strict cold-start gate binds to exactly the generation it started, instead of re-reading self._polling_progress_event which a concurrent recovery task may have replaced with a newer generation's event (the G1/G2 race flagged in the #69240 review). - Strict cold start no longer schedules background polling recovery: a polling error during the readiness wait is captured by a strict callback and fails the connect attempt immediately with a loud OSError, so GatewayRunner disposes the partial adapter and retries with a fresh one — no more waiting out the full readiness deadline on a generation that already errored, and no G2-on-partial-app healing. - After readiness is proven the strict callback delegates every later polling error to the real background-recovery callback, preserving the existing degraded/reconnect semantics for the polling lifetime. - The readiness-timeout error message now states the deadline and that the gateway will retry with a fresh adapter (loud failure, not a silent wait). - Regression tests: current-generation progress connects; a polling error during strict cold start fails fast without scheduling background recovery (the #67498 idle-threads shape); stale-generation progress is rejected. Progresses #67498 * test: record getUpdates progress in mocked cold-connect polling flows The strict cold-start readiness gate (#67498) means adapter.connect() no longer returns True until the mocked start_polling records a successful getUpdates round trip for its generation. Update the conflict-suite Application mocks accordingly: - fake_start_polling side effects call adapter._record_polling_progress(adapter._polling_generation) on the initial connect (retry generations intentionally do NOT auto-progress where a test asserts the conflict count survives an unproven retry). - _build_polling_app takes the adapter so its start_polling mock can record progress. Without this, the cold connects in these tests wait out the full 60s readiness deadline and fail — which is exactly the fail-closed behavior the gate is supposed to provide when polling shows no progress. * fix(config): add a collision-safe env var name for custom endpoint keys Both the Desktop panel and the CLI setup flow need somewhere in .env to put a custom endpoint's API key. Deriving the name from the endpoint's hostname collapses two servers on one machine onto a single slot, and every IP-based local endpoint slugs to a digit-leading name that save_env_value rejects outright. Key off the endpoint's own identity and keep a fixed prefix. Co-authored-by: asorry75 <33794789+asorry75@users.noreply.github.com> Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com> * fix(windows): verify rebuilt Hermes.exe integrity before shipping it as an update (#69179) The desktop self-update chain (Desktop -> hermes-setup --update -> hermes update -> hermes desktop --build-only -> relaunch) rebuilds Hermes.exe on the user's machine and declared success on bare file EXISTENCE. A truncated PE (corrupt cached Electron zip / interrupted extraction or rcedit rewrite / full disk) or a wrong-architecture unpacked tree therefore shipped as the 'updated' app, which Windows refuses to load with 'This app can't run on your computer' (此应用无法在你的电脑上运行) — and the previous working build had already been wiped by before-pack.mjs, leaving nothing to fall back to. Fix, in three parts: - hermes_cli/main.py: post-build integrity gate on Windows (_ensure_desktop_exe_launchable). Parses the PE header of the freshly built Hermes.exe — MZ/PE magic, section-table completeness vs file size (catches truncation), and COFF machine vs the host arch (catches arm64/x64 mixups). On failure it purges the (likely corrupt) cached Electron zip, invalidates the content-hash build stamp so the updater's retry-once genuinely re-downloads and rebuilds, restores the previous build from the .bak tree when one exists (keeping the corrupt tree as .corrupt for diagnostics), tells the user the update was aborted and their old version kept, and exits nonzero. _desktop_packaged_executable also now prefers a host-loadable PE over pure newest-mtime when multiple win-*-unpacked trees coexist. - apps/desktop/scripts/before-pack.mjs: on win32, the previous unpacked tree is preserved as <appOutDir>.bak (only when it holds the product exe — partial/corrupt trees still get the plain wipe) instead of being destroyed, providing the rollback material for the gate above. Non-Windows behavior is unchanged. - Behavior-contract tests: tests/hermes_cli/test_desktop_exe_integrity.py (23 tests — synthetic PE fixtures for truncation/non-PE/arch-mismatch, rollback semantics, and the build-only exit contract) and 6 new vitest cases in before-pack.test.mjs for the .bak preservation rules. Progresses #69179 * fix(desktop): persist the whole discovered model list when saving an endpoint Test enumerates a custom provider's catalogue and the panel holds the result in discoveredModels, but the save payload never carried it, so only the one model the user hand-typed reached providers.<id>.models. Every downstream picker reads that map straight from config.yaml with no live probe, which is why a proxy serving 18 models offered exactly one. Send the discovered list and merge it onto the entry, so models already known keep their context lengths. Fixes #69988 Co-authored-by: asorry75 <33794789+asorry75@users.noreply.github.com> * fix(web_server): keep Desktop custom endpoint API keys out of config.yaml The Custom Endpoints panel wrote the raw key to providers.<id>.api_key, so the credential sat in plaintext in a file users routinely share and commit. The input is masked, so nothing warned them. Write the key to .env and reference it via key_env, the same indirection built-in providers use and that runtime_provider already resolves. The read side has to move with it: reporting has_api_key from api_key alone would show "no API key" for every migrated endpoint, and activate copying only api_key would drop the credential entirely. Delete now clears the .env slot too, and an entry still carrying a pre-fix plaintext key is migrated on its next save so existing users get cleaned up without re-entering anything — unless the key is a hand-written ${VAR} template, which is already safe and must not be duplicated into a second env var. Fixes #69449 Co-authored-by: webtecnica <75556242+webtecnica@users.noreply.github.com> Co-authored-by: asorry75 <33794789+asorry75@users.noreply.github.com> * fix(cli): store custom endpoint API key in .env instead of config.yaml hermes model's custom-endpoint flow is the other write path that produced a plaintext key, on both the model block and the custom_providers entry. Route it through the same .env indirection as the Desktop panel, and swap an existing entry's inline key for the reference when the URL is re-saved. Co-authored-by: liuhao1024 <sunsky.lau@gmail.com> * test: cover custom endpoint key storage and model-list persistence Bug-class coverage for both fixes: the full catalogue survives Save, context lengths are preserved, the key never lands in config.yaml on either write path, blank clears it, a pre-fix plaintext key migrates while a ${VAR} template is left alone, two endpoints on one host keep separate credentials, and an IP-derived name is still a valid POSIX env var. The two delete tests asserted on the plaintext mirror; they now assert the same invariants against the credential reference. * fix(desktop): persist @image: refs instead of the vision-enrichment text The desktop gateway passed the vision-enriched, model-only message text (carrying an `image_url:<path>` hint) straight into run_conversation as the persisted user turn. The renderer only parses `@image:<path>`, so it could not rebuild the attachment from history: after a restart the image was gone and only the caption survived, and on a live session switch the warm cache disagreed with the authoritative text and the frontend "rescued" the image by appending it after the caption. run_conversation already supports persist_user_message for exactly this "what the model sees" vs "what gets stored" split; it was simply never wired up for the attachment path. * fix(desktop): keep cached attachment refs on session resume Persisted history carries no attachment metadata for non-image refs, so resume reconciliation dropped `@file:` chips off a user turn whose text matched. Carry the warm cache's refs forward when the resumed message has none of its own, never replacing refs that are already present. (cherry picked from commit eac5b0a8ac39eab242a5d571531e386ec70e2435) * fix(desktop): quote persisted @image: paths so spaced paths render The unquoted alternative in the directive pattern is `\S+`, so a ref built by string interpolation truncates at the first space and strands the tail as loose text next to a broken thumbnail. Composer images live in the app's userData dir, which on macOS is `~/Library/Application Support/<App>/` — so every pasted or dropped image hit this. Adds format_reference_value next to REFERENCE_PATTERN, mirroring formatRefValue in the desktop's directive-text.tsx, and covers the round-trip through the parser. * fix(desktop): persist the image ref for natively-vision-capable models too A turn routed to a model that takes pixels directly sends `content` as a parts list, and the session store deliberately ignores a plain-string persist override for a list payload — a text override must not erase a turn's image summary. So the override was dropped for every user on a vision-capable main model, and the durable row kept only the caption plus a literal `[Image attached at: ...]` / `[screenshot]`, which the renderer cannot turn back into an image. Only vision-preprocessed (text-mode) turns were actually fixed. Mirror the shape instead: swap the text part for the `@image:` ref form and keep the image parts, so the model still has the pixels for the rest of the session, and drop the `[screenshot]` stand-in on the way into the bubble when a ref was lifted from the same message. * refactor(desktop): memoize the directive image-segment filter Matches the two derived values above it and fixes the indentation. * fix(desktop): lead persisted image turns with the caption Session previews are the first 60 characters of the first user message, so persisting the @image: directives ahead of the caption labelled the session with a truncated file path in the sidebar, session switcher, and command palette. Clients lift the refs out of the body line by line, so moving them after the caption changes nothing about how the turn renders. * test(desktop): cover attached-image resume end to end The unit tests cover each layer in isolation, but nothing exercised the whole chain the bug lived in: the real gateway persisting an attachment, SessionDB holding it after the process exits, and the renderer rebuilding a thumbnail from the stored turn. Seeds a session through the real gateway with an image attached, then launches desktop against it — so the first render is already the relaunch case. Pins native image routing (the majority path, and the one where a text-only persist override is dropped) and stages the file behind directory and file names with spaces, mirroring the macOS composer's Application Support path. * fix(models): resolve custom provider model ids Map picker-prefixed custom provider selections back to their configured model IDs before validation, persistence, and API requests. Fixes #68347 * chore(contributors): map jevin@jevin.org to ijevin Attribution check needs a mapping for the cherry-picked commit's author so release notes credit them correctly. * fix(relay): normalize forwarded Discord interactions to leading-slash commands (#71048) A real APPLICATION_COMMAND interaction forwarded over the relay ar…

Follow-up to the salvaged NousResearch#38985: guard the 4 bare read_text/write_text sites its allowlist missed (google_chat thread-count store + oauth JSON) and add whatsapp/google_chat to the AST guard test's file list.

…r rule AST-driven pass over every Path.read_text()/write_text() without an explicit encoding= across non-test code: 71 sites in 34 files (skills_hub, hermes_cli/main+profiles+service_manager+container_boot, mem0/hindsight/honcho plugins, achievements dashboard, release/CI scripts, productivity+comfyui skill helpers, agent/*). Verified zero positional-encoding collisions before insertion; per-file compile() check after. Adds a check-windows-footguns rule flagging bare single-line read_text/write_text (multi-line forms stay covered by the AST guard test from NousResearch#38985). Together with the salvaged contributor commits this retires the ~169-site bare file-I/O class (NousResearch#37423's long tail).

alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery area/config Config system, migrations, profiles labels Jun 4, 2026

rodboev force-pushed the pr/gateway-utf8-update-path branch 2 times, most recently from 0554b73 to 6510597 Compare June 10, 2026 14:04

rodboev force-pushed the pr/gateway-utf8-update-path branch from 6510597 to c1f6823 Compare June 12, 2026 11:47

rodboev force-pushed the pr/gateway-utf8-update-path branch from c1f6823 to 7cb9fde Compare June 28, 2026 16:29

fix(gateway): pass encoding="utf-8" to read_text/write_text in update…

7c04046

… path (NousResearch#37423)

rodboev force-pushed the pr/gateway-utf8-update-path branch from 5d3031f to 7c04046 Compare July 7, 2026 04:35

fix(gateway): add utf-8 encoding to dead target registry

0bfdf49

teknium1 reviewed Jul 14, 2026

View reviewed changes

Comment thread plugins/platforms/telegram/adapter.py

fix(gateway): cover discord update-response utf-8 path (NousResearch#…

f757652

…37423)

teknium1 mentioned this pull request Jul 24, 2026

fix(windows): residual encoding gaps — MCP stdio, gateway update file-I/O, STT/TTS, desktop spawn, bootstrap ver-stub #71014

Merged

teknium1 closed this Jul 24, 2026

GottZ mentioned this pull request Jul 26, 2026

hermes doctor: Mem0 OSS mode falsely reports "API key not set" + setup wizard defaults to built-in #57931

Open

GottZ mentioned this pull request Aug 3, 2026

fix: add encoding="utf-8" to read_text() calls in auth/config readers #66782

Closed

2 tasks

This was referenced Aug 3, 2026

fix: add encoding='utf-8' to read_text/fdopen in core agent and tools files #65565

Closed

fix: add encoding="utf-8" to dead_targets read/write #66791

Closed

Navigation Menu

Uh oh!

fix(gateway): pass encoding="utf-8" to read_text/write_text in update path (#37423) - #38985

fix(gateway): pass encoding="utf-8" to read_text/write_text in update path (#37423)#38985
rodboev wants to merge 3 commits into
NousResearch:mainfrom
rodboev:pr/gateway-utf8-update-path

rodboev commented Jun 4, 2026 •

edited

Loading

Uh oh!

teknium1 left a comment

Uh oh!

Uh oh!

teknium1 commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

rodboev commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Validation

Test plan

Not in scope

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

Uh oh!

teknium1 commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rodboev commented Jun 4, 2026 •

edited

Loading