fix(cron): managed-cron fires execute in the gateway process (live adapters + dashboard forwarder) - #84339
Merged
Merged
Conversation
The Chronos fire webhook (/api/cron/fire) called provider.fire_due(job_id, adapters=None, loop=loop), so every externally-triggered fire delivered through the standalone path even with a live gateway in-process. E2EE platforms and relay-fronted logical platforms (whose ONLY send path is the live relay adapter — no native credential exists on the box) failed every external fire with "platform 'X' not configured/enabled", while the same job delivered fine under the built-in ticker (gateway/run.py passes runner.adapters). Resolve the runner (self.gateway_runner → app['gateway_runner'] → _gateway_runner_ref(), the same chain the drain check uses) and forward its adapters. No runner → adapters=None, preserving the historical standalone path byte-identically. Note: does not by itself fix Fly-hosted scale-to-zero deployments where NAS's callback lands on the DASHBOARD process (internal_port 9119) — _fire_cron_job_for_profile there has no gateway runner in-process. That topology needs a separate fire handoff (design pending).
…unreachable) The dashboard's /api/cron/fire executed cron jobs in the DASHBOARD process via _fire_cron_job_for_profile with adapters=None. On hosted deployments (Fly proxy exposes only the dashboard's port) that made every managed-cron fire deliver through the standalone send path, which cannot serve relay-fronted logical platforms (their only sender is the live relay adapter in the gateway process — no native credential exists on the box) or E2EE rooms. It also ran the whole agent turn inside the dashboard: wrong process for memory/session ownership and fire-claim attribution. Restore the invariant that the GATEWAY owns cron execution: - Dashboard route: after verifying the NAS JWT and resolving the job's profile, FORWARD the fire to the gateway api_server's own /api/cron/fire on loopback, NAS bearer preserved (the gateway re-verifies the JWT — defense in depth, no new trust link), and pass the gateway's response through. Gateway unreachable → 503 so NAS retries per the Chronos contract (non-2xx = retryable; the store CAS de-dupes the eventual double fire). Deliberately NO local-execution fallback. - Endpoint resolution mirrors gateway/config.py's api_server load order per target profile (config.yaml extra.port → API_SERVER_PORT from process env or the profile's .env → 8642), with /p/<profile>/ prefix routing under multiplex. - docker/stage2-hook.sh: generate a strong API_SERVER_KEY into .env on first boot when absent (never overwrites an operator value), so the loopback api_server passes its startup guard on hosted images. The fire route itself is NAS-JWT-authed; the key gates the rest of the api_server surface. The listener binds 127.0.0.1 by default and the Fly service exposes only the dashboard port. - _fire_cron_job_for_profile kept but deprecated (late-binding seam compatibility); no route calls it. - docs/chronos-managed-cron-contract.md: document the two-hop inbound topology and the 503-retry semantics. Depends on the previous commit (fire webhook passes live adapters to fire_due) — together they make NAS→dashboard→gateway fires deliver over relay end to end.
Contributor
૮ >ﻌ< ა ci reviewran on 2ea2e20 — Merge remote-tracking branch 'origin/main' into fix/chronos-
|
alt-glitch
added
type/bug
…loader
CI guard test_config_read_guard flagged the new _gateway_fire_endpoint
for a raw yaml.safe_load of the profile's config.yaml — the exact drift
class the guard exists to kill (raw reads miss the managed-scope
overlay, ${ENV_VAR} expansion, and root-model normalization).
Read through load_config() under a HERMES_HOME override scoped to the
target profile instead (the same pattern the deprecated
_fire_cron_job_for_profile uses for its store scope), and pull the port
with cfg_get. Test updated to stub load_config rather than write a raw
config.yaml.
…m gate The stage2 hook now generates API_SERVER_KEY for every Docker container, and key presence force-enables the api_server platform. The scale-to-zero arm gate counted every enabled platform, so the loopback api_server listener made messaging_is_relay_only_or_absent False on every hosted instance — silently disarming the feature (the not-armed log would show enabled platforms=['relay','api_server']). The arm gate and the not-armed logger now share one helper that filters to enabled MESSAGING platforms, excluding LOCAL/API_SERVER/WEBHOOK — the same non-messaging exclusion set _connect_platforms already uses. A genuinely enabled direct-socket platform (Discord/Telegram) still disarms. Two of the three new tests fail without this fix.
…-adapters # Conflicts: # tests/gateway/test_scale_to_zero_watcher.py
batumilove
added a commit
to batumilove/hermes-agent
that referenced
this pull request
Aug 12, 2026
* fix(gateway): run channel-directory write off the event loop
atomic_json_write() calls os.fsync(), which blocks until the write
reaches stable storage. build_channel_directory() already offloads its
builders with asyncio.to_thread (#60794) but still called the persist
step directly on the loop, so the Discord heartbeat waited on a disk
flush.
* test(gateway): assert the channel-directory write leaves the event loop
Mirrors test_discord_builder_runs_off_event_loop_thread. Verified to FAIL
against unpatched v0.19.0 and pass with the fix.
* fix(gateway): offload remaining atomic_json_write calls in async paths
Completes the bug class from #83906 — the same blocking fsync-on-event-loop
pattern existed in two more async gateway paths:
- slash_commands.py _handle_restart_command: two atomic_json_write calls
for .restart_notify.json and .restart_last_processed.json were blocking
on fsync inside an async function. Now offloaded via asyncio.to_thread.
- run.py _clear_restart_failure_count: called from
_handle_message_with_agent (async, per-turn path) after a successful
agent turn. Made the method async and offloaded the atomic_json_write
call via asyncio.to_thread. Caller updated to await.
Shutdown-path calls in _stop_impl_body (_increment_restart_failure_counts,
planned restart notification marker) are intentionally left synchronous —
the event loop is draining/stopping and offloading adds complexity for no
benefit.
* chore: add landaun to contributor email map for #83906 salvage
* fix(ci): repair red main — busy-mode test + missing checkout in skills-index workflows
Three separate reds on main. Two are fixed here; the third needs no code.
1. tests/gateway/test_multiplex_busy_input_mode.py (blocks every merge)
Fails "Python tests / Run tests slice 5/12" and therefore "All required
checks pass". Semantic merge conflict between two PRs merged ~1h apart:
a31be480 fix(gateway): respect routed profile busy modes (added the test)
c8f235a1 feat(gateway): allow selective multiplex profile serving (added the gate)
c8f235a1 taught _profile_name_for_source to reject a route whose target
profile is not in the served set (profiles_to_serve). Each PR was green on
its own base; neither ran against the other's merge result.
The test asserts a route to profile "research" resolves to that profile's
busy mode, but never patches profiles_to_serve — so it reads the runner's
REAL on-disk profiles. "research" is not among them, the route is rejected
before the busy-mode snapshot is consulted, and the assertion gets the
gateway default:
WARNING gateway.run: Rejecting profile route 'research-chat':
target profile 'research' is not served
AssertionError: assert 'interrupt' == 'steer'
Patch profiles_to_serve for the assertion — the same seam every sibling
test in tests/gateway/test_profile_resolution.py already patches
(test_route_inside_allowlist_resolves, test_route_outside_allowlist_rejects).
This also removes an ambient-state dependency: the test previously passed
or failed based on which profiles happened to exist on the machine running
it. Verified passing under an empty HERMES_HOME.
Test-only. The serving gate from c8f235a1 is correct and left intact.
2. Skills-index workflows: local action used without actions/checkout
check-freshness has failed on all 12 of its last 12 scheduled runs:
##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under
'.../.github/actions/get-app-token'. Did you forget to run
actions/checkout before running your local action?
./.github/actions/get-app-token is a LOCAL composite action and cannot
resolve without the repo on disk. skills-index-freshness.yml had no
checkout step at all. The step is gated on `status != 'ok'`, so the
watchdog broke exactly when it was supposed to file its issue — the live
index is currently 521.4h stale (limit 26h) and nobody was told.
An audit of all workflows for this bug class found one more instance:
skills-index.yml's `trigger-deploy` job, which re-triggers the docs deploy
so a refreshed index reaches the live site. Its sibling `build-index` job
checks out; this one did not. That is plausibly why the index went stale
in the first place. Both are fixed; the audit now reports zero remaining
jobs that use a local action without a prior checkout.
Pinned to the same actions/checkout SHA used by the other 35 call sites.
3. "Publish inline E2E evidence" — no fix needed
Failed once at 13:33Z on a transient TLS error reaching api.github.com
("certificate is not valid for any names") while installing a gh
extension. The last 25 runs of that workflow are 25/25 success. Infra
blip, not a code defect.
* fix(desktop/windows): quiet minimal update hand-off window
The hand-off script's WinForms window was a 720x420 dashboard: streaming
log box, wide marquee, warning label. Updating is a wait, not a dashboard
-- it is now the same shape as the other update surfaces (#75895): a fixed
280x320 panel, marquee loader, one title, one static line, following the
OS light/dark theme (charcoal #232323 seeds, never brand blue).
Failure gets a terse finale instead of a wall of log: 'Failed to update' +
'Run "hermes debug share" in a terminal to send a report' + Close (held
max 5 minutes, then the relaunched Desktop re-surfaces the result banner
as before). The result-json message points at debug share too.
With nothing streamed to the window, the per-line stdout pump is gone:
Invoke-HermesStep drains both pipes async (no deadlock on chatty children,
no frozen marquee on quiet ones) and writes full output to the hand-off
log afterwards, where hermes debug share picks it up.
* feat(update): shim UI + event channel for the Windows hand-off
scripts/desktop-update.ps1 moves to scripts/desktop-update/windows.ps1 (a
compat forwarder stays at the old path for one asar/checkout skew cycle)
and gains the shim: scripts/desktop-update/ui.html rendered in a
chromeless Edge app window, fed done|error over a loopback /progress
endpoint. The page is #75895's hand-off screen ported verbatim (Fourier
Flow loader, one title, one line, OS light/dark, charcoal dark seeds);
failure is the terse card pointing at hermes debug share. The WinForms
card stays as the no-Edge fallback, same shape.
Salvaged from the web-shell spike: TcpListener runspace server, Edge
--app spawn with throwaway profile, degradation ladder, -SelfTestUi.
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
* feat(update): posix hand-off orchestrator (mac/linux quit-first updates)
scripts/desktop-update/posix.sh is the mac/linux twin of windows.ps1:
the Desktop spawns it detached and QUITS; it waits the app out, runs
plain hermes update (retry-once across the update boundary, truthful
desktop-rebuild completion), swaps/relaunches the .app bundle (mac) or
the release/*-unpacked binary when its sandbox helper is launchable
(linux), writes .hermes-update-result.json, and drives the same shim.
Repo-owned, so every update refreshes the code that drives the next one.
resolvePosixScriptHandoff mirrors the Windows resolver (with the
flat-path fallback covering the scripts/ reorg skew).
* refactor(desktop): replace the in-app posix updater with the hand-off
applyUpdatesPosixInApp is gone: mac/linux Update now quits into the
detached posix orchestrator, same shape as Windows. Deletes everything
the in-app path dragged into main.ts -- runStreamedUpdate, the rebuild
retry, the relaunch-outcome matrix (update-relaunch.ts/update-rebuild.ts
and tests), shellQuote, resolveHermesCliBinary -- and with the app dead
before the update starts, the HERMES_DESKTOP_CHILD_PID reaper-exclusion
dance (#37532) is structurally unnecessary on the desktop path.
* test(update): sandboxed repro paths as npm scripts
scripts/desktop-update/repro.sh drives the real code paths against a
disposable HERMES_HOME under /tmp: shim/shim-fail (UI dry runs), fresh
(literal install.sh), behind N (rewound checkout driven forward by the
orchestrator), error (broken venv -> abort + result file). Exposed as
npm run update:shim / update:shim:fail / update:repro:* from
apps/desktop.
* fix(update): posix hand-off truth ordering, relaunch-gate port, JSON escaping
Address helix4u's review:
- finish() now delivers the outcome BEFORE publishing it: mac bundle swap
and the linux relaunch gate run first, then the result file, marker
removal, and the shim event -- the app launch itself goes last so it
can't race the result write. A gated/skewed linux install (AppImage/
deb/rpm, broken sandbox helper) surfaces its message in the result file
AND holds the shim window open with it instead of closing on a false
'Opening Hermes...'.
- mac swap is transactional with a checked rollback; a failed install
restores the previous bundle and the result says so (exit 7 when even
rollback fails). Failed 'open' rewrites the result truthfully.
- linux gate is an exact port of the deleted update-relaunch.ts logic:
anchored path-segment match on <root>/apps/desktop/release/linux-unpacked,
chrome-sandbox absent = namespace build = fine, present = root+setuid
required, with the real opt-outs (ELECTRON_DISABLE_SANDBOX, --no-sandbox
among replayed args, or the Desktop vouching) instead of the invented
HERMES_DESKTOP_NO_SANDBOX. collectRelaunchArgs/sandboxFallbackFromEnv
live in updater-process.ts again; the Desktop passes filtered launch
args (after --) and --relaunch-cwd so a deep-link or --no-sandbox
launch survives the update.
- result/status JSON strings are escaped (git permits '"' in branch
names) and the result write is atomic (tmp + rename).
- coverage: resolvePosixScriptHandoff + ported helpers in
updater-process.test.ts (19 pass); repro.sh gate / npm run
update:repro:gate asserts the whole gate matrix and round-trips a
hostile branch name through the result JSON.
* fix(update): run hermes update from the install root + unbreak fresh repro
The posix orchestrator inherited the Desktop's cwd, and parts of the
update pipeline resolve the tree they mutate from the working directory
-- the sandboxed behind-repro caught it updating the DEVELOPER'S primary
checkout (cwd at spawn time) while reporting success against the
sandbox. cd "$INSTALL_ROOT" before running hermes update, matching the
cwd:updateRoot contract of the deleted in-app path. Verified: rerun
leaves the outside checkout untouched (reflog clean).
repro.sh fresh used a --no-interactive flag install.sh doesn't have;
non-TTY stdin (</dev/null) + --skip-setup is the real non-interactive
contract.
* fix(update): launch acceptance before the terminal event, on both orchestrators
gille's round-2 review: the terminal lifecycle claimed outcomes the
launch hadn't delivered yet.
- posix finish() reorders: outcome -> durable result+marker -> LAUNCH
WITH ACCEPTANCE -> terminal event. mac acceptance is open's exit code
(launchd rejects broken bundles loudly); linux verifies the setsid
child is still alive 1.5s after spawn, so an instant exec failure
downgrades to a held 'manual' state + truthful result instead of a
vanished 'done'. Gated skew/manual outcomes publish a real 'manual'
event (new third shim state -- still zero logic in the page).
- Renderer-free linux recovery: when no chromium-family browser exists,
manual/error outcomes fire notify-send/zenity/kdialog best-effort so a
gated non-relaunch is never a silent disappearance.
- windows.ps1 mirrors the contract: Start-DesktopRelaunch returns
verified acceptance (WMI pid alive / fallback process alive; dying
before the window appears counts as failure), and the finally block
downgrades to Show-ManualFinale + rewritten result when the launch
didn't land. Error path still relaunches after showing itself.
- repro.sh launch / npm run update:repro:launch: real-orchestrator
matrix for instant-exit relaunch downgrade and skew-message surfacing.
- posix.sh cds into the install root before hermes update (found by the
sandboxed behind-repro: parts of the update resolve the mutated tree
from cwd, which is the Desktop's cwd -- it updated the DEVELOPER'S
checkout while reporting success against the sandbox).
* fix(update): fail-closed cd, rejected-launch semantics, guaranteed recovery surface
gille's round 3:
- cd into the install root FAILS CLOSED (set -u without set -e let a
failed cd continue hermes update in the caller's tree -- the exact
wrong-tree class the correction exists to kill). Honest result, exit 3.
- A supplied mac relaunch target that is missing is a REJECTED launch ->
manual downgrade; the launch matrix asserts the downgrade instead of
codifying the old false success. A mac swap-failure DONE_NOTE now still
relaunches the kept/rolled-back bundle before publishing manual.
- notify_fallback: every rung falls through on EXECUTION failure (a
notify-send that can't reach D-Bus no longer eats the message), mac
gets osascript (present on every macOS -- Safari-only machines have no
chromium shim), and the no-surface terminal case is an explicit logged
contract: the result file carries the outcome to the next boot.
- update:repro:fresh passes --non-interactive explicitly (prompt_yes_no
falls back to /dev/tty, so </dev/null was not equivalent).
* fix(update): manual-result protocol so gated outcomes reach the user
Round 4 of helix4u's review — the durable fallback is now real:
- Result protocol gains `manual`: an ok result the user still must act
on (reopen the app, reinstall the GUI package, fix the sandbox helper).
Both orchestrators set it on every DONE_NOTE/downgrade path; the Desktop
consumer surfaces manual results in a real dialog on next boot instead
of a log line — the browserless-Linux disappearance now ends at a
visible dialog, worst case one boot later. Older result files without
the field parse as manual:false (covered).
- notify ladder verifies EXECUTION, not existence: zenity/kdialog must
survive their first second (an instant death means no display and falls
through); the no-surface case is an explicit best-effort contract whose
guaranteed channel is the result dialog.
- mac DONE_NOTE + failed relaunch of the kept/rolled-back bundle is no
longer swallowed (`|| true` dropped): the durable message carries both
facts.
- launch/gate matrices assert `manual` in the result JSON; consumer
round-trip tested in handoff-result.test.ts.
* fix(auth): /auth/native/authorize 空 provider 自动选择不再统计会被拒绝的密码 provider
Fix #78906
当部署同时启用 basic 密码 provider 与一个 OAuth/OIDC session provider 时，
list_session_providers() 会把密码 provider 也计入 "exactly one candidate"
判断（密码 provider 虽是 session provider，但下一行就会因 supports_password
被原生 OAuth broker 流程拒绝），导致 len == 2、自动选择被跳过，桌面端
空 provider 登录返回 404 "Unknown provider: ''"。
修复：自动选择只在可 broker 的 provider（supports_session 且非
supports_password）中计数，与 /api/status 的 native_pkce 能力宣告使用同一
"brokerable" 定义；当没有任何可 broker provider 时保留原有选择逻辑，
让显式的 400 错误继续解释密码 provider 不支持原生 OAuth。
新增回归测试：basic+OIDC 并存时自动选中 OIDC、单 OAuth provider 自动
选中、多 OAuth provider 歧义 404、纯密码部署保留 400。
* fix(update): exempt manual results from the hand-off freshness window
A manual:true hand-off result is the durable action-required channel: on a
browserless Linux box with no working notifier, the boot dialog is the first
and only place the message ever surfaces. The 30-minute freshness gate
discarded it if the user reopened Hermes later, stranding exactly the machine
the channel exists to serve. Parse before the age check and skip the window
for manual results; the file is still unlinked before any age check, so it's
surfaced at most once. Ordinary results still expire.
Regression: a stale ordinary result is discarded (and consumed) while a stale
manual result is still returned once.
* feat(browser): auto-install the Browser Use CLI instead of silently downgrading
The Browser Use CLI became the default browser backend, but nothing
provisioned it: users without uv/uvx (field report from DongyangHe on
macOS) silently fell back to the built-in browser tools with no notice.
- install_cli() in tools/browser_use_cli.py: uv tool install browser-use
via the managed uv (bootstrapped on demand), linked into
$HERMES_HOME/bin (UV_TOOL_BIN_DIR)
- _find_cli() now also probes $HERMES_HOME/bin for browser-use/uvx —
Hermes' managed uv is not on the user's PATH
- hermes tools post_setup actually installs (Camofox standard) instead
of printing instructions
- install.sh / install.ps1 provision the CLI at install time
(best-effort, non-fatal, honors --skip-browser)
- CLI startup shows a one-line notice (24h rate-limited) when the
default backend downgraded to the built-in tools
* fix: ASCII-only install.ps1 comment; allow-list install_cli's uv PATH fallback
- install.ps1 must stay pure ASCII (PowerShell 5.1 ANSI code-page
decoding, #66994/#67000): em-dash -> '--'
- tests/test_managed_runtime_resolution.py: install_cli()'s
shutil.which('uv') is a reviewed fallback AFTER ensure_uv() misses
* fix(relay): stop sibling gateways answering another instance's button press (#83677)
* fix(relay): stop sibling gateways answering another instance's button press
A Discord button press arrives on the passthrough plane, and the connector
fans a passthrough forward out to EVERY live gateway session of the tenant
(relayServer.routeBusMessage delivers `passthrough` via sessionsByTenant),
unlike a message, which it narrows to the admitted instance set. The prompt
went out from exactly one instance and _pending_prompts is process-local, so
every sibling gateway saw an answer for a prompt it never minted, could not
tell that from its own prompt expiring, and fell through to chat dispatch --
where the option-shaped text ("/c1") is not a real command and run.py replied
"Unknown command `/c1`". One copy per sibling, under the single real ack.
Prompt ids are now minted as `<per-process nonce>.<8 hex>`, so an answer can
be attributed to the process that minted it. A prompt answer is always
consumed, never re-dispatched as chat: a sibling's prompt and a repeat answer
are both dropped silently, and an expired prompt of our own gets a short
"no longer waiting" notice from the owning gateway only.
Ids stay inside the connector codec's contract ([A-Za-z0-9_.-], <=32 chars,
64-byte callback budget -- verified against promptCodec.ts: 52 bytes worst
case with a full-length option id). An id with no nonce segment (a prompt in
flight across an in-place upgrade) is still treated as ours.
Tests: 4 added, each verified to fail without the fix. Full relay suite green
(160 tests).
* style(tests): ruff-format the added relay prompt tests
* feat(relay): ambient token endpoint mode for gateway.idp.token_url (#84074)
* feat(relay): ambient token endpoint mode for gateway.idp.token_url
When gateway.idp.token_url is configured WITHOUT client_id/client_secret,
treat the URL as a metadata-server-style ambient credential endpoint:
plain GET, response body is the token (raw JWT or {"access_token": ...}
JSON envelope). Covers workload-identity proxies such as Domino's
$DOMINO_API_PROXY/access-token, which mint short-lived user-scoped OIDC
tokens with no client registration.
Previously this configuration was a hard error (client_id/client_secret
missing), so no working deployment changes behaviour: creds present keeps
the OAuth2 client_credentials POST, no token_url keeps Nous Portal. The
misconfig error now self-diagnoses (names the ambient fallback and how to
select the client_credentials grant instead).
* fix(relay): reject short plain-text bodies in ambient token shape gate
Review finding: the shape gate accepted any base64url-alphabet word, so an
IdP answering the ambient GET with a terse error body ('unauthorized',
'error', 'null') had that word returned as a bearer token instead of the
fail-closed misconfiguration error. Tighten the gate to JWT-like dotted
tokens (3+ segments) or long opaque tokens (>= 32 chars); short bare words
now raise the self-diagnosing ambient error.
* fix(relay): partial IdP client credentials keep the loud error, never select ambient GET
The ambient-endpoint dispatch used 'not client_id or not client_secret',
so configuring exactly one credential (a mistyped client_credentials
setup) silently issued a GET at the IdP token endpoint and then raised
'no client_id/client_secret configured' — factually wrong for that
operator, and a stray request the old hard error never made.
Ambient mode now requires NEITHER credential; a partial pair raises
immediately, names the missing key, and issues no HTTP request (tests
assert urlopen is never called). Docstring and relay.md now say
'neither' instead of 'without'.
* fix(relay): ambient JSON envelope requires a string access_token, no coercion
Review finding (P2): the JSON-envelope branch accepted any truthy
access_token via str() coercion — a number became '12345…', a boolean
became 'True', an object became its Python repr — bypassing the fail-
closed contract and deferring the failure to the connector, where it
hides the real endpoint problem.
The envelope value must now be a non-empty string, the same contract the
client_credentials path enforces on its token response. Deliberately NO
shape gate on envelope values: an envelope is an intentional token
response (mode-1 symmetry), and opaque tokens may use the standard-base64
alphabet the raw-body gate rejects. Mutation check: reverting the branch
to str() coercion sends the 3 coercion tests red (3 failed, 15 passed).
---------
Co-authored-by: Ben Barclay <ben@nousresearch.com>
* fmt(js): `npm run fix` on merge (#84193)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(auxiliary): honor main model for title generation (#83636)
* fix(gateway): scale-to-zero gateway self-suspends via flaps socket instead of relying on Fly autostop (#84295)
Fly Proxy autostop judges idle exclusively on inbound proxied connections.
It cannot see an in-flight agent turn (outbound-only LLM traffic), and since
Fly's mid-2026 proxy change an open outbound socket (the relay WS) no longer
holds a machine awake. With autostop:"suspend", Fly suspended machines while
they were still processing long-running jobs, and could suspend before the
gateway flipped the relay destination (the buffered-event black hole).
The scale-to-zero watcher now owns the suspend: after the idle predicate
holds (no running agents, no live background work, inbound-quiet) and the
go_dormant() quiesce completes (relay drained + flipped), it POSTs
/v1/apps/{app}/machines/{id}/suspend on the local /.fly/api flaps socket.
Suspend is skipped when the quiesce fails or inbound lands mid-quiesce
(flip-before-freeze), and off-Fly the step is a no-op (fail-awake).
Pairs with the NAS change that provisions scale-to-zero machines with
autostop:"off" (gateway-owned suspend); wake is unchanged (Fly-proxied
wakeUrl poke + autostart).
* fix(cron): deliver to relay-fronted platforms via canonical home_channel (#84300)
Cron jobs targeting a relay-fronted logical platform (e.g. Discord behind
the relay connector) failed twice over:
1. Target resolution read only the legacy <PLATFORM>_HOME_CHANNEL env
mirror. The canonical home_channel block that /sethome persists to
config.yaml — the only store that exists in a relay-fronted deployment,
where no native env var is exported — was never consulted, so
deliver='discord' silently resolved to nothing and the job fell back
to local-only.
2. Even with a resolved target, the delivery loop's native
configured/enabled gate rejected the platform ('not configured/enabled')
although resolve_delivery_transport had already produced a live relay
transport fronting it. A relay-fronted platform is deliberately NOT
natively enabled (its credential lives in the connector), so the native
gate must not apply to a relay transport.
Resolution now falls back from the env mirror to
config.get_home_channel(platform) for both chat_id and thread_id (thread
affinity only when the chat id came from the same config block), which
also makes the 'all' routing token pick up relay-fronted platforms. The
delivery gate honours a resolved relay transport, mirroring the
enablement rule resolve_delivery_transport already applied; the standalone
(no-relay) path keeps the historical gate byte-identical.
* fix(gateway): exclude permanent supervised watchers from the scale-to-zero busy check (#84327)
_scale_to_zero_has_live_background_work() counted every task in
_background_tasks — but _spawn_supervised parks all permanent watchers
there (session-expiry, kanban, reconnect, the scale-to-zero watcher
itself, ...). An armed gateway therefore considered itself busy forever
and never went dormant or suspended. Verified live on staging
(hermes-agent-stg-test-6698, 2026-08-12): armed at 05:25, fully idle for
25+ minutes, zero 'going dormant' lines. Fly's coarse proxy autostop used
to mask the bug; once the gateway took ownership of the suspend (#84295)
it became load-bearing.
_spawn_supervised now tags its tasks and the busy check skips them.
Transient tasks (startup-resume events, delegation, tracked processes)
still block suspend. New tests exercise the REAL _spawn_supervised path
rather than a stubbed _background_tasks set — the stubbing is exactly why
the earlier tests missed this (same call-site trap as the F25 arm bug);
the key test fails on main and passes with the fix.
* fix(relay): stamp logical platform + relay trust on Discord interaction events (#84318)
The relay interactions passthrough lane (_discord_interaction_to_event)
built its SessionSource with platform=Platform.RELAY and no
delivered_via_upstream_relay marker — unlike the relay text lane
(ws_transport._event_from_wire), which maps the connector's platform to
the logical enum and stamps the authenticated-upstream flag.
Consequences of the mismatch:
- /sethome sent as a Discord slash command persisted the home channel
under platforms.relay.home_channel (invisible to cron delivery, which
looks up the logical platform) and mirrored it into the dead
RELAY_HOME_CHANNEL env var — so cron jobs with deliver='discord' kept
falling back to local-only even after the resolution/delivery fixes.
The absent trust marker also meant via_relay=False, so the handler's
'Relay does not authenticate this logical home target' guard —
designed to reject exactly this misfiled shape — never engaged.
- Session keys forked: the connector binds the interaction's follow-up
capability under buildSessionKey with platform 'discord' and
chat_type 'group' (interactionSessionSource), while the gateway keyed
the same interaction as relay/channel.
- _capture_scope skipped recording _platform_by_chat (it ignores the
generic 'relay'), losing the egress sender hint for the chat.
Stamp Platform.DISCORD (the lane statically parses Discord interaction
wire payloads), chat_type 'group' for guild channels (native-adapter and
connector parity), and delivered_via_upstream_relay=True (parity with
the text lane; set locally, never read off the wire).
With this, slash-command /sethome files under platforms.discord and
passes the via_relay guard legitimately, and cron delivery over relay
works end to end with the #84300 resolution fixes.
* fix(cron): managed-cron fires execute in the gateway process (live adapters + dashboard forwarder) (#84339)
* fix(gateway): pass live adapters to cron fire webhook's fire_due
The Chronos fire webhook (/api/cron/fire) called
provider.fire_due(job_id, adapters=None, loop=loop), so every
externally-triggered fire delivered through the standalone path even
with a live gateway in-process. E2EE platforms and relay-fronted
logical platforms (whose ONLY send path is the live relay adapter — no
native credential exists on the box) failed every external fire with
"platform 'X' not configured/enabled", while the same job delivered
fine under the built-in ticker (gateway/run.py passes runner.adapters).
Resolve the runner (self.gateway_runner → app['gateway_runner'] →
_gateway_runner_ref(), the same chain the drain check uses) and forward
its adapters. No runner → adapters=None, preserving the historical
standalone path byte-identically.
Note: does not by itself fix Fly-hosted scale-to-zero deployments where
NAS's callback lands on the DASHBOARD process (internal_port 9119) —
_fire_cron_job_for_profile there has no gateway runner in-process. That
topology needs a separate fire handoff (design pending).
* fix(cron): dashboard forwards Chronos fires to the gateway (503 when unreachable)
The dashboard's /api/cron/fire executed cron jobs in the DASHBOARD
process via _fire_cron_job_for_profile with adapters=None. On hosted
deployments (Fly proxy exposes only the dashboard's port) that made
every managed-cron fire deliver through the standalone send path, which
cannot serve relay-fronted logical platforms (their only sender is the
live relay adapter in the gateway process — no native credential exists
on the box) or E2EE rooms. It also ran the whole agent turn inside the
dashboard: wrong process for memory/session ownership and fire-claim
attribution.
Restore the invariant that the GATEWAY owns cron execution:
- Dashboard route: after verifying the NAS JWT and resolving the job's
profile, FORWARD the fire to the gateway api_server's own
/api/cron/fire on loopback, NAS bearer preserved (the gateway
re-verifies the JWT — defense in depth, no new trust link), and pass
the gateway's response through. Gateway unreachable → 503 so NAS
retries per the Chronos contract (non-2xx = retryable; the store CAS
de-dupes the eventual double fire). Deliberately NO local-execution
fallback.
- Endpoint resolution mirrors gateway/config.py's api_server load order
per target profile (config.yaml extra.port → API_SERVER_PORT from
process env or the profile's .env → 8642), with /p/<profile>/ prefix
routing under multiplex.
- docker/stage2-hook.sh: generate a strong API_SERVER_KEY into .env on
first boot when absent (never overwrites an operator value), so the
loopback api_server passes its startup guard on hosted images. The
fire route itself is NAS-JWT-authed; the key gates the rest of the
api_server surface. The listener binds 127.0.0.1 by default and the
Fly service exposes only the dashboard port.
- _fire_cron_job_for_profile kept but deprecated (late-binding seam
compatibility); no route calls it.
- docs/chronos-managed-cron-contract.md: document the two-hop inbound
topology and the 503-retry semantics.
Depends on the previous commit (fire webhook passes live adapters to
fire_due) — together they make NAS→dashboard→gateway fires deliver over
relay end to end.
* fix(cron): read the profile api_server port via the canonical config loader
CI guard test_config_read_guard flagged the new _gateway_fire_endpoint
for a raw yaml.safe_load of the profile's config.yaml — the exact drift
class the guard exists to kill (raw reads miss the managed-scope
overlay, ${ENV_VAR} expansion, and root-model normalization).
Read through load_config() under a HERMES_HOME override scoped to the
target profile instead (the same pattern the deprecated
_fire_cron_job_for_profile uses for its store scope), and pull the port
with cfg_get. Test updated to stub load_config rather than write a raw
config.yaml.
* fix(gateway): only messaging platforms count for the scale-to-zero arm gate
The stage2 hook now generates API_SERVER_KEY for every Docker container,
and key presence force-enables the api_server platform. The scale-to-zero
arm gate counted every enabled platform, so the loopback api_server
listener made messaging_is_relay_only_or_absent False on every hosted
instance — silently disarming the feature (the not-armed log would show
enabled platforms=['relay','api_server']).
The arm gate and the not-armed logger now share one helper that filters
to enabled MESSAGING platforms, excluding LOCAL/API_SERVER/WEBHOOK —
the same non-messaging exclusion set _connect_platforms already uses.
A genuinely enabled direct-socket platform (Discord/Telegram) still
disarms. Two of the three new tests fail without this fix.
* fix(agent): log Codex transport failure details
* fix(agent): tolerate transport errors without requests
* fix: widen APIConnectionError handling to finalization drain loop
The PR added APIConnectionError handling to the main request and
iteration try blocks but missed the finalization drain loop (line ~1492).
That site catches httpx transport errors to preserve an already-completed,
already-billed response when the drain iterator fails. Without the
APIConnectionError handler, an SDK-wrapped transport error during drain
would propagate uncaught and discard the completed response.
Also strengthens the test's no-payload-leak assertion to check the full
request body and URL are absent from the log message, not just the
literal string 'payload'.
* fix(kanban): query show graph before closing database
* chore: map contributor email cmoiccool
* fix(nix): set HERMES_BIN default in wrapped binaries
The TUI resolves the CLI via process.env.HERMES_BIN (externalCli.ts) and
falls back to a bare 'hermes', which is not on PATH for nix run / nix
profile installs that only expose the wrapped binaries. Set a
--set-default so the wrapper advertises its own hermes while an explicit
operator override (documented in kanban_db.py) still wins.
* fix: warn agents off driving interactive console TUIs via pty on Windows (#84364)
* fix: warn agents off driving interactive console TUIs via pty on Windows
Driving 'gh auth login' (and other survey-style console TUIs) through a
pty background process on Windows silently hangs: these programs read
Win32 console key events via ReadConsoleInput, not the stdin byte
stream, so Enter keypresses submitted over process stdin never register.
The agent-visible symptom is a prompt frozen at 'Press Enter to open
browser...' while the user sees nothing, and a turn interrupt then kills
the process, invalidating any device code the user already entered on
github.com.
Two guidance fixes, both proven in a live session on Windows 10:
- agent/prompt_builder.py: extend _WINDOWS_BASH_SHELL_HINT to steer
agents toward non-interactive paths (flags, --with-token, config
files, curl-polled OAuth device flow) instead of answering console
prompts programmatically.
- skills/github/github-auth: document the pitfall and add the manual
OAuth device-flow procedure (curl against gh's public client_id,
poll for the token, finish with 'gh auth login --with-token'), which
succeeded first try after two interactive attempts hung.
* fix: send CRLF for Enter on Windows PTY submit; correct root cause in guidance
Review feedback (helix4u) was right on both counts:
1. Root cause correction. gh's 'Press Enter to open browser' prompt is
waitForEnter -> bufio.Scanner reading stdin, not a survey/console-API
prompt. The real bug is ours: submit_stdin appended a bare \n, and
through pywinpty/ConPTY a lone \n is not delivered as a line
terminator, so the child's blocking line read never returns. Verified
empirically against pywinpty 2.0.15 with a readline() child:
\n -> hang, \r -> line delivered, \r\n -> line delivered.
Fix: submit_stdin now appends \r\n for Windows PTY sessions (POSIX
PTYs and Popen pipes keep \n). Windows-only regression tests cover
the PTY and pipe branches.
2. Prompt hint rewritten: instead of claiming Windows console TUIs
cannot be driven, it now says to use process(submit) rather than raw
writes with bare \n, and to prefer non-interactive paths when a CLI
offers one.
3. Skill device flow rewritten as an executable script: parses the
device-code response, polls per the returned interval, handles
authorization_pending / slow_down (+5s per GitHub docs) /
expired_token / access_denied / unexpected responses, pipes the token
straight into gh without echoing it, and drops the undocumented
workflow scope (repo,read:org,gist is the documented minimum for
gh auth login --with-token). The pitfall note is narrowed to the
reproduced condition.
* fix: make verify_on_stop opt-in everywhere (default False, not auto) (#84383)
* fix: make verify_on_stop opt-in everywhere (default False, not auto)
The verify-on-stop nudge was already judged more noise than signal: the
v31 migration flips existing installs off, the v32 migration catches the
baked-in literal-true population, and the docs tell users to 'treat off
as the effective default and opt in explicitly'. But DEFAULT_CONFIG still
shipped the "auto" sentinel, so exactly one population kept getting the
nudges: fresh installs (and any config missing the key), where "auto"
resolves ON for CLI/TUI/desktop surfaces. Live symptom: repeated
'[System: You edited code ... run verification]' interruptions the user
never asked for and had to hunt down in source to disable.
- DEFAULT_CONFIG: agent.verify_on_stop "auto" -> False (opt-in).
- verify_on_stop_enabled(): missing/unrecognized value now falls back
OFF instead of surface-aware; explicit "auto" still selects the
legacy surface-aware behavior, explicit bools unchanged, and the
HERMES_VERIFY_ON_STOP env override is untouched.
- No migration needed: v31/v32 already normalized existing installs,
and this only changes the merged default for configs without the key.
- Docs updated; default-path E2E test now asserts OFF, plus a new
missing-value regression test. Also added the standard win32 skip
marker to the symlink-based temp-dir test (pre-existing Windows
failure, same class as tests/cron/test_cron_script.py).
* test: update config goldens — verify_on_stop=False is now stripped as default
With the DEFAULT_CONFIG flip to False, the migration-write invariant
(_persist_migration / save_config strip_defaults) no longer materialises
verify_on_stop: false to disk unless the user explicitly set the key:
- V20 floor fixture (agent: {} on disk): v31's write is stripped —
agent stays {} and load_config() supplies False at read time.
- V12 floor fixture (explicit verify_on_stop: true on disk): the key is
a user-set path, so the v32 flip stays materialised as false.
- Partial-write and _persist_migration regressions now assert the key is
absent from disk and (for the merge case) that the merged view still
resolves False.
Behavior verified with a one-shot migrate_config run against both
fixture shapes.
* fix: Windows path handling in search_files rg calls and patch escape drift (#84378)
* fix: Windows path handling in search_files rg calls and patch escape drift
Two related Windows failures from a live session (Windows 10, git-bash
terminal backend, winget-installed native ripgrep):
1. search_files was unusable on drive-letter paths. _escape_shell_arg
rewrites C:\... to the MSYS form /c/... so bash builtins resolve it,
but rg is a native Windows binary and Hermes disables MSYS argument
conversion for its bash subprocesses (MSYS_NO_PATHCONV=1 /
MSYS2_ARG_CONV_EXCL=*, see _apply_windows_msys_bash_env_defaults) —
so nothing ever translated /c/... back and every search failed with
'The system cannot find the path specified. (os error 3)'.
Fix: new _escape_native_tool_arg emits the forward-slash NATIVE form
(C:/Users/...), which native binaries accept, bash passes through
untouched, and MSYS builds also handle. Applied to the six rg call
sites (content search, --files search x2, zero-match probe x3); the
grep fallback keeps the MSYS form since MSYS grep wants it.
2. The patch tool silently doubled backslash runs when tool-call args
arrived JSON-escaped one extra time (file had \ where old_string
had \\). Similarity strategies (context_aware) matched the region
anyway and wrote new_string verbatim, corrupting every backslash run
(reproduced: 6 backslashes on the line became 12). _detect_escape_drift
now also blocks when every backslash run in old_string is exactly twice
its counterpart in the matched region and new_string repeats the
doubling — with guardrails so exact matches, intentional backslash
edits, model-corrected new_strings, and single weak-signal runs all
still apply. Blocking returns the standard escape-drift guidance so
the model re-reads and retries with correct counts.
Tests: TestEscapeNativeToolArg (5 cases, including an end-to-end
_search_with_rg command capture) and TestBackslashDoublingDrift (6
cases). The 8 pre-existing failures in tests/tools/test_file_operations.py
on a Windows host (umask/symlink POSIX assumptions) are identical on
unmodified main and unrelated.
* fix: shell linters get native Windows paths too (node C:\c\... double-prefix)
Same class as the rg fix: LINTERS commands (python -m py_compile,
node --check, npx tsc, go vet, rustfmt) invoke native Windows binaries,
but _check_lint interpolated the MSYS /c/... form. node resolves that
as C:\c\Users\... (double-prefixed), so on Windows hosts every .js
write reported a phantom ENOENT lint failure that could mask real
syntax errors (issue #84303). Route the {file} arg through
_escape_native_tool_arg like the rg call sites.
Regression test asserts node --check receives 'C:/...' and never
'/c/...'.
* delete tmp file lol
* feat: add Nemotron Lightning to reasoning timeout (#83982)
* fix(tools): strip heredoc bodies before background-'&' detection
_strip_quotes documented that it stripped heredoc bodies but only handled
single/double/backtick quotes. As a result _foreground_background_guidance
scanned heredoc body text for a backgrounding '&' and wrongly rejected valid
foreground commands whose heredoc body contained a spaced ampersand — e.g.
AppleScript string concat (osascript <<'EOF' ... "a" & b ... EOF), Python
bitwise-and, or literal UI text like 'FaceTime & Privacy'.
Add a _strip_heredocs pass (runs before quote-stripping, since a heredoc
delimiter may itself be quoted) covering <<EOF, <<-EOF, <<'EOF', <<"EOF".
The same-line tail after the opener (redirects/args) is preserved and the
opener token is blanked so a real backgrounding '&' after the heredoc is
still detected.
Adds tests/tools/test_terminal_heredoc_background_guard.py.
* fix(tools): harden heredoc masking into a conservative shared helper
The previous commit's regex-based stripper removed EVERY heredoc body,
which review flagged as bypassable: a fake '<<EOF' marker inside a
comment or quoted string enters the unterminated path and swallows a
later REAL background operator, and unquoted ('cat <<EOF' — expansion
runs) or shell-consumed ('bash <<'EOF'' — body IS shell) bodies are
executable content that must stay visible to the guard.
Replace it with tools/shell_heredoc.strip_inert_heredoc_bodies(), a
conservative shell-state scanner: a body is masked ONLY when every
delimiter on the opener is quoted (no expansion), every heredoc is
terminated by an exact delimiter line, the opener composes a single
command (no list/pipeline operators, no nested $()/backtick/process-
substitution scope), and the consumer is an allowlisted non-shell
interpreter (python/osascript/cat). Anything ambiguous is returned
unchanged — a false positive on exotic syntax is acceptable; hiding a
real background operator is not. Masked bodies become newlines so line
structure is preserved for MULTILINE regexes.
The helper is a standalone stdlib-only module (precedent:
tools/ansi_strip.py) because the same heredoc-as-data false-positive
class exists in the blocked-command regex checks (#83104) and the
gateway lifecycle guard (#81721/#79835, cron/lifecycle_guard.py) —
which must not import the terminal-tool module graph.
Adapted from Wolfram Ravenwolf's security-hardened rework of #63788
(69c7663c6de6b6cb05bf99203fa39673efe01ccf); test scenarios for the
bypass cases derive from his suite.
Co-authored-by: Wolfram Ravenwolf <github.com@wolfram.ravenwolf.de>
* perf(tools): linear-time masking rebuild + last-opener early exit
Efficiency review (measured with timeit probes) found two unbounded
costs on adversarial inputs:
- The masked-range rebuild copied the whole string once per range
(O(n*k)): 50k tiny heredocs took 1.7s. Replaced with a single-pass
segment join over the (sorted, non-overlapping) ranges: 152ms, and
newlines are now counted on the original command instead of
re-slicing.
- After the last '<<' occurrence no opener can start, but the scanner
still walked the remaining text per-char: one heredoc followed by a
1MB tail cost ~150ms. An rfind bound breaks out of the unit loop
once the scan passes it: 0.3ms.
Typical commands are unaffected (the '<<' fast path already returns
first). 30/30 guard tests pass; mutation check re-run on the final
stack (no-op mutation -> 11 tests fail, restore -> green).
* fix(cache): opt M3 out of cache_control markers on Anthropic wire
MiniMax-M3 ships server-side automatic prefix caching on the
Anthropic-compatible endpoint (content-keyed, no marker needed —
see platform.minimax.io/docs/api-reference/text-prompt-caching).
cache_control markers are NOT on its explicit-cache support list
(which covers only M2.7/M2.5/M2.1/M2).
Emitting markers on M3:
- wasted serialization overhead
- risked perturbing the server-side prefix hash
- gave users a false sense of explicit-cache savings (the
cache_read_input_tokens field carries a +128 constant floor
and cache_creation_input_tokens is always 0 for M3)
Also add an opt-in debug=True parameter to normalize_usage() that
emits a debug-level log line carrying the observable cache fields.
This is the only reliable cache signal for M3 — off by default,
debug-level, scoped to the anthropic_messages wire, so production
callers see no impact.
Pin both changes with 8 new tests:
- 4 M3 tests covering provider, host, and custom-provider paths
- 1 regression guard ensuring M2.x caching is unaffected
- 3 observability tests (off-by-default, on-with-M3, on-with-Claude)
Verified end-to-end against api.minimaxi.com/anthropic/v1/messages
with MiniMax-M3[1m]: identical system prompt hit-rate with and
without markers; cache_read field is unreliable (128 floor),
input_tokens drop (8467 -> 1) is the real hit signal.
* fix: close provider-anthropic MiniMax proxy bypass + rework cache observability
Follow-up fixes on top of the salvaged #83678 commit:
1. Hoist the MiniMax-M3 marker exclusion ABOVE the native-Anthropic
early return. provider="anthropic" pointed at a MiniMax /anthropic
proxy is a supported override (_anthropic_base_url_override_ok), and
the is_native_anthropic branch matched on provider alone — returning
(True, True) before the M3 exclusion was reached. Two regression
tests pin the proxy route (M3 off, M2.7 still on).
2. Reuse the existing _model_name_suggests_minimax_m3() helper from
agent/model_metadata.py instead of a second inline substring copy.
3. Drop the debug kwarg on normalize_usage() — it had zero production
callers and duplicated standard logging level gating. The
cache-observability line is now a plain logger.debug scoped to
MiniMax providers on the Anthropic wire only, so the "+128 floor"
note can no longer appear for native Anthropic where it is false.
Tests updated accordingly (MiniMax logs, native Anthropic does not).
* chore: map hermes-agent@nous.local commit identity to @C-EXCITE-STUDIO
Salvaged PR #83678's commit is authored under a generic local agent
identity with no linked GitHub account; map it to the PR opener for
release attribution (same pattern as hermes-agent@users.noreply.local).
* fix: Windows agent-loop papercuts — path splitting, hashing, autocomplete, screenshots, OS detection (#84419)
Sweep of open Windows issues affecting day-to-day agent operation
(explicitly excluding install/setup and locale classes):
- hermes_cli/_subprocess_compat.py: new split_command_line() — Windows-
safe command-line tokenizer (posix=False + quote stripping) so
backslash paths survive. POSIX behavior unchanged (plain shlex.split).
- hermes_cli/console_engine.py (#83934): console commands like
'sessions export C:\Users\me\out.jsonl' no longer silently mangle the
path into a relative filename in the cwd.
- agent/shell_hooks.py (#78293): hook commands with backslash paths now
spawn, resolve their script path, and pass hooks doctor instead of
reporting 'not executable'. All three shlex sites routed through the
shared splitter.
- agent/prompt_builder.py (#51755): system prompt now reports
Windows (11) on Windows 11 — platform.release() returns 10 for both;
distinguish via sys.getwindowsversion().build >= 22000.
- hermes_cli/commands.py (#42016): @ autocomplete no longer crashes the
prompt_toolkit event loop when rg emits a path on a different mount
(device paths \.\nul, other drive letters) — relpath ValueError is
skipped per-entry.
- tools/browser_use_cli.py (#83884): screenshot-path detection now
matches Windows drive-letter paths (C:\... and C:/...) in addition to
POSIX; Browser Use screenshots attach on Windows.
- tools/skills_hub.py + tools/skills_guard.py (#62310): the two 'MUST
stay symmetric' skill content hashes actually agree on Windows now.
Bundle keys are normalized to POSIX separators before hashing, and the
disk digest sorts by rel-posix STRING (case-sensitive) instead of Path
objects (case-insensitive on Windows). Fixes permanent false-positive
update_available for every installed skill.
Tests: tests/tools/test_windows_agent_loop_papercuts.py — 16 cases
covering each fix, including a disk-vs-bundle hash symmetry check built
with native Windows separators and a mixed-case filename.
* fix: steer agents off MSYS paths for native tools; pin line-ending preservation (#84426)
Two follow-ups from live Windows sessions:
1. agent/prompt_builder.py: extend the Windows shell hint with the
native-binary path rule. Hermes disables MSYS path conversion for its
bash, so agents passing /c/Users/... or /tmp/... to NATIVE programs
(git -C, node, python, rg) hit 'cannot change to' / 'not found' while
the same path works in bash builtins — observed repeatedly in a live
session (git -C failures, git apply /tmp/x.patch failures). The hint
now says: forward-slash native form (C:/Users/x) for native tools,
$LOCALAPPDATA/Temp over /tmp for scratch files native tools read.
(/tmp is pure model habit from Linux training data — nothing
instructs it — so the hint is the right layer.)
2. tests: pin LF/CRLF preservation through write_file and patch_replace.
A live session saw a repo-LF file come back full-CRLF after an edit
(4699-line diff churn); not reproducible through current tool APIs,
so pin the correct behavior — LF files stay LF, CRLF files stay CRLF,
no mixed endings — to catch any regression on the Windows write path.
* fix(security): approval system covers Windows destructive commands and paths (#84428)
Fixes #69472. On a Windows host every destructive native command passed
approval silently — DANGEROUS_PATTERNS were POSIX-shaped, and the
normalizer strips backslashes as shell escapes so no Windows path could
ever match a path rule. Probed live before the fix: 15 of 15 destructive
Windows commands (Remove-Item -Recurse -Force, del /s /q, iwr | iex,
taskkill /F, Format-Volume, diskpart, icacls /grant Everyone, vssadmin
delete shadows, bcdedit /set, reg delete, cipher /w, ...) sailed through
undetected.
Two changes:
1. Windows destructive tier in DANGEROUS_PATTERNS: PowerShell deletes
(bare Remove-Item -Recurse/-Force), cmd builtins with /s|/q switches,
iwr|iex remote execution (pipe and subexpression forms), taskkill /F /
Stop-Process -Force, volume/disk destruction (Format-Volume,
Clear-Disk, diskpart, format.com, cipher /w), icacls Everyone-grant /
/reset, backup destruction (vssadmin delete shadows, wbadmin delete,
bcdedit /set), reg delete / Remove-ItemProperty -Force, and service
stop/delete (Stop-Service -Force, sc stop|delete). Each pattern
requires the destructive flag so graceful/read-only usage (taskkill
/IM without /F, reg query, icacls inspect, sc query, plain del file)
does not prompt. Patterns live in the main list, not a win32-gated
tier: a Linux-hosted Hermes can drive a Windows box over SSH.
2. Windows-path detection variant in _command_detection_variants: when
the raw command contains a drive-letter/UNC backslash path, also
yield a variant with backslashes flattened to forward slashes BEFORE
normalization strips them, plus Windows spellings of the credential
path rules (Users/<u>/.ssh, AppData/{Local,Roaming}/hermes .env).
Gated on a real path shape so POSIX escape semantics are untouched.
Tests: tests/tools/test_approval_windows.py — 48 cases (27 destructive
flagged, 13 benign not flagged, 5 credential paths in both separator
spellings, 4 POSIX-escape non-regressions). The 8 pre-existing failures
under '-k approval' on this Windows host are identical on unmodified
main (ordering artifacts + known symlink cases) and unrelated.
* fix: Windows MCP PATHEXT resolution + python3 -> python in cross-platform skills (#84429)
Two Windows agent-loop friction fixes:
1. tools/mcp_tool.py (#56536): shutil.which(cmd, path=env_path) reads
executable extensions from the PARENT process PATHEXT, not the MCP
subprocess env — a stdio MCP config supplying both PATH and PATHEXT
could fail to resolve a command its own env can locate, and startup
then got a bare command name. On Windows, when the first which() call
misses and the config env carries PATHEXT (any key casing), retry the
resolution with the config's PATHEXT temporarily applied.
2. skills/ + optional-skills/ (#50606): 42 SKILL.md files that declare
platforms: [.., windows] used python3 in their command examples.
python3 does not exist on native Windows (the toolchain probe in the
system prompt reports python3=missing), so every copy-pasted example
burned a failed agent turn before self-correction. Replaced the
command word python3 -> python (python3-config / python3.x version
strings untouched). python is the spelling that exists in every
Hermes-managed environment (Windows native, uv-managed venvs on all
three OSes); agents on POSIX hosts additionally see the probed
toolchain line and adapt either way.
* fix(tools): clarify identical old and new string error
* fix(tools): improve patch tool parameter description
* refactor(tools): extract IDENTICAL_STRINGS_ERROR constant
The 3-sentence identical-edit message was snapshot-asserted verbatim in
two tests. House style avoids exact-string change-detector assertions;
both tests now import the constant from tools/fuzzy_match so rewording
the message can't silently break them.
* fix(tools): mirror must-differ guidance in skill_manage new_string schema
skill_manage's patch action uses the same fuzzy_find_and_replace engine
as the file patch tool and surfaces the identical-strings error verbatim
— and unlike the file path it has NO is_already_applied no-op rescue, so
identical old/new ALWAYS errors there. Mirror the new_string description
so the schema warns before the error fires (sibling-site parity with
tools/file_tools.py PATCH_SCHEMA).
* fix(tools): skip degenerate identical hunks in V4A validation
The apply phase already skips a hunk whose -/+ lines are identical
(patch_parser.py '(search_lines == replace_lines): continue'), but the
validation phase lacked the guard: such a hunk reached
fuzzy_find_and_replace, whose identical-strings error names
old_string/new_string — parameters that don't exist in patch mode — and
failed the whole atomic patch that apply would have accepted. Mirror
the apply-phase skip in validation; regression test drives a mixed
degenerate+live patch end-to-end (short text dodges the
is_already_applied >=8-char rescue).
* fix(windows): SSH ControlMaster gating + stop hijacking the user's python (#84452)
* fix(windows): SSH ControlMaster gating + stop hijacking the user's python
Two Windows environment-integrity fixes:
1. tools/environments/ssh.py (#73927): Windows OpenSSH has no
Unix-domain-socket ControlMaster support, so unconditionally passing
ControlPath/ControlMaster/ControlPersist failed EVERY tool call on a
Windows-hosted ssh terminal backend with 'getsockname failed: Not a
socket'. Gate the three multiplexing options behind a module-level
_SSH_MULTIPLEX = (os.name != 'nt'); the scp upload path is gated the
same way. On Windows the backend now works without connection pooling
(each command a fresh connection); POSIX behavior is unchanged. The
teardown 'ssh -O exit' is naturally inert because the socket never
exists on Windows.
2. scripts/install.ps1 (#83797): the installer put the whole
venv\Scripts directory on the user PATH, which contains python.exe /
pythonw.exe / pip.exe and so silently hijacked the 'python' command in
every terminal on the machine — unrelated projects started resolving
python to Hermes' runtime interpreter. Now copy only the launchers
(hermes.exe, hermes-acp.exe) into a dedicated $InstallDir\bin and put
THAT on PATH. Existing installs are migrated: the legacy venv\Scripts
entry is stripped from the user PATH on the next install/update. The
new bin dir is under $InstallDir (…\hermes-agent), which the uninstall
PATH sweep already matches via its \hermes-agent marker.
Updated the stale hermes_cli/update_cmd.py docstring that described the
old venv\Scripts-on-PATH layout.
Tests: SSH ControlMaster gating pinned both directions (multiplex on →
flags present; off → absent but BatchMode/StrictHostKeyChecking retained).
install.ps1 parses clean via the PowerShell AST parser.
* docs: update windows-native install docs for the bin\ launcher layout
CI (test_windows_native_docs) pins the docs and installer to the same
PATH layout. The #83797 fix moved the PATH entry from venv\Scripts to a
dedicated $InstallDir\bin holding only the hermes launchers, so update
the Windows-native guide to match: PATH-after-install section, the
install-steps list, the directory-layout table, the Get-Command
verification line, and the 'command not found' pitfall. Test now asserts
the bin\ layout and guards against a regression back to venv\Scripts on
PATH.
* fix: keep install.ps1 pure ASCII (PowerShell 5.1 codepage safety)
The two comments I added in the #83797 PATH-hijack fix used em-dashes,
tripping tests/test_install_ps1_ascii_only.py — Windows PowerShell 5.1
reads a BOM-less .ps1 in the system ANSI codepage (not UTF-8), so a
non-ASCII byte can misdecode into a stray quote and desync the parser
(issues #66994/#67000). Replace the em-dashes with ASCII '--'.
* fix(tools): improve error message when wrong args
* feat(tests): add tests for execute_code error mesages
* fix(tools): redirect non-string code payloads in execute_code handler
Review follow-up on the salvaged handler: a non-string 'code' (int,
dict, list) reached code.strip() and surfaced as a generic
'Tool execution failed: AttributeError' — the same unrecoverable shape
the salvage exists to eliminate. Add an isinstance guard beside the
'command' check that names the received type and shows the correct
call form; narrow the docstring to what the handler actually does.
Regression test drives int/dict/list through registry.dispatch and
asserts no AttributeError leaks (mutation-checked: removing the guard
fails 3 subtests).
* fix(tools): mirror misplaced-arg recovery on the terminal side
Whole-bug-class sibling of the execute_code fix: terminal(code=...) —
the reverse confusion — fell through to command=None and failed with
'Invalid command: expected string, got NoneType', naming neither the
stray 'code' argument nor execute_code as the right tool. Mirror the
guard in _handle_terminal (verified live: the opaque NoneType error
reproduces on main). Mutation-checked: removing the guard fails the
new regression test.
* fix(tools): isolate external project environments
* feat(tests): add tests to cover external-venv PYTHONPATH isolation
* fix(tools): harden interpreter-environment probe for the strict-mode default
Follow-up to the salvaged #81201 commits:
- Short-circuit _uses_hermes_python_environment when the child IS the
running interpreter (path or realpath match). The default strict-mode
path no longer spawns a probe subprocess at all, and a flaky probe of
sys.executable can never drop the hermes root from PYTHONPATH
(protects the test_repo_root_modules_are_importable invariant). The
realpath leg also covers uv-style venvs whose bin/python resolves to
the same binary.
- Stop caching failed probes: _python_environment_prefix now uses a
success-only dict cache instead of lru_cache, so one transient
timeout under load no longer sticks for the process lifetime.
- Deduplicate the subprocess probe scaffolding shared with
_is_usable_python into _probe_python().
- Log once when the hermes root is omitted so import-behavior changes
are diagnosable from user reports.
- Tests: fail the composition tests loudly if execute_code never
reaches Popen (was vacuously passing on exceptions); assert the
staging dir is literally first in PYTHONPATH (was truthiness only);
add guards for probe-failure retry and the no-probe short-circuit.
* refactor(tools): unify probe caches and dedupe the exclusion log
/simplify-code findings on the full PR diff:
- _is_usable_python had the same sticky-failure bug the previous commit
fixed in _python_environment_prefix: lru_cache pinned a transient
probe failure (fork pressure, timeout) as False forever, silently
locking project mode to sys.executable. Both probes now share a
success-only bounded dict cache via _cache_probe_result() with FIFO
eviction at _PROBE_CACHE_MAX (the old < cap guard stopped caching new
entries instead of evicting, re-probing entry 33+ on every call).
- The hermes-root-omitted logger.info fired on every external-env call
in project mode; now deduped once per interpreter path per process
(matching the tirith/mcp warn-once convention).
- Regression test: _is_usable_python probe failures are retried, not
cached (mutation-verified).
---------
Co-authored-by: landaun <landaun@gmail.com>
Co-authored-by: kshitij <82637225+kshitijk4poor@users.noreply.github.com>
Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
Co-authored-by: x7peeps <xtpeeps@qq.com>
Co-authored-by: Ben Barclay <ben@nousresearch.com>
Co-authored-by: victor-kyriazakos <93273468+victor-kyriazakos@users.noreply.github.com>
Co-authored-by: hermes-seaeye[bot] <307254004+hermes-seaeye[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: fangliquanflq <fangliquan@oppo.com>
Co-authored-by: Gille <4317663+helix4u@users.noreply.github.com>
Co-authored-by: cmoiccool <cmoiccool@users.noreply.github.com>
Co-authored-by: alt-glitch <balyan.sid@gmail.com>
Co-authored-by: ethernet <arilotter@gmail.com>
Co-authored-by: elisam0 <elisam@nvidia.com>
Co-authored-by: Taylor Mingos <54285+tmingos@users.noreply.github.com>
Co-authored-by: Wolfram Ravenwolf <github.com@wolfram.ravenwolf.de>
Co-authored-by: Hermes Agent <hermes-agent@nous.local>
14 tasks
vashkartik
added a commit
to vashkartik/hermes-agent
that referenced
this pull request
Aug 13, 2026
* fix(gateway): offload remaining atomic_json_write calls in async paths
Completes the bug class from #83906 — the same blocking fsync-on-event-loop
pattern existed in two more async gateway paths:
- slash_commands.py _handle_restart_command: two atomic_json_write calls
for .restart_notify.json and .restart_last_processed.json were blocking
on fsync inside an async function. Now offloaded via asyncio.to_thread.
- run.py _clear_restart_failure_count: called from
_handle_message_with_agent (async, per-turn path) after a successful
agent turn. Made the method async and offloaded the atomic_json_write
call via asyncio.to_thread. Caller updated to await.
Shutdown-path calls in _stop_impl_body (_increment_restart_failure_counts,
planned restart notification marker) are intentionally left synchronous —
the event loop is draining/stopping and offloading adds complexity for no
benefit.
* chore: add landaun to contributor email map for #83906 salvage
* fix(ci): repair red main — busy-mode test + missing checkout in skills-index workflows
Three separate reds on main. Two are fixed here; the third needs no code.
1. tests/gateway/test_multiplex_busy_input_mode.py (blocks every merge)
Fails "Python tests / Run tests slice 5/12" and therefore "All required
checks pass". Semantic merge conflict between two PRs merged ~1h apart:
a31be480 fix(gateway): respect routed profile busy modes (added the test)
c8f235a1 feat(gateway): allow selective multiplex profile serving (added the gate)
c8f235a1 taught _profile_name_for_source to reject a route whose target
profile is not in the served set (profiles_to_serve). Each PR was green on
its own base; neither ran against the other's merge result.
The test asserts a route to profile "research" resolves to that profile's
busy mode, but never patches profiles_to_serve — so it reads the runner's
REAL on-disk profiles. "research" is not among them, the route is rejected
before the busy-mode snapshot is consulted, and the assertion gets the
gateway default:
WARNING gateway.run: Rejecting profile route 'research-chat':
target profile 'research' is not served
AssertionError: assert 'interrupt' == 'steer'
Patch profiles_to_serve for the assertion — the same seam every sibling
test in tests/gateway/test_profile_resolution.py already patches
(test_route_inside_allowlist_resolves, test_route_outside_allowlist_rejects).
This also removes an ambient-state dependency: the test previously passed
or failed based on which profiles happened to exist on the machine running
it. Verified passing under an empty HERMES_HOME.
Test-only. The serving gate from c8f235a1 is correct and left intact.
2. Skills-index workflows: local action used without actions/checkout
check-freshness has failed on all 12 of its last 12 scheduled runs:
##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under
'.../.github/actions/get-app-token'. Did you forget to run
actions/checkout before running your local action?
./.github/actions/get-app-token is a LOCAL composite action and cannot
resolve without the repo on disk. skills-index-freshness.yml had no
checkout step at all. The step is gated on `status != 'ok'`, so the
watchdog broke exactly when it was supposed to file its issue — the live
index is currently 521.4h stale (limit 26h) and nobody was told.
An audit of all workflows for this bug class found one more instance:
skills-index.yml's `trigger-deploy` job, which re-triggers the docs deploy
so a refreshed index reaches the live site. Its sibling `build-index` job
checks out; this one did not. That is plausibly why the index went stale
in the first place. Both are fixed; the audit now reports zero remaining
jobs that use a local action without a prior checkout.
Pinned to the same actions/checkout SHA used by the other 35 call sites.
3. "Publish inline E2E evidence" — no fix needed
Failed once at 13:33Z on a transient TLS error reaching api.github.com
("certificate is not valid for any names") while installing a gh
extension. The last 25 runs of that workflow are 25/25 success. Infra
blip, not a code defect.
* fix(desktop/windows): quiet minimal update hand-off window
The hand-off script's WinForms window was a 720x420 dashboard: streaming
log box, wide marquee, warning label. Updating is a wait, not a dashboard
-- it is now the same shape as the other update surfaces (#75895): a fixed
280x320 panel, marquee loader, one title, one static line, following the
OS light/dark theme (charcoal #232323 seeds, never brand blue).
Failure gets a terse finale instead of a wall of log: 'Failed to update' +
'Run "hermes debug share" in a terminal to send a report' + Close (held
max 5 minutes, then the relaunched Desktop re-surfaces the result banner
as before). The result-json message points at debug share too.
With nothing streamed to the window, the per-line stdout pump is gone:
Invoke-HermesStep drains both pipes async (no deadlock on chatty children,
no frozen marquee on quiet ones) and writes full output to the hand-off
log afterwards, where hermes debug share picks it up.
* feat(update): shim UI + event channel for the Windows hand-off
scripts/desktop-update.ps1 moves to scripts/desktop-update/windows.ps1 (a
compat forwarder stays at the old path for one asar/checkout skew cycle)
and gains the shim: scripts/desktop-update/ui.html rendered in a
chromeless Edge app window, fed done|error over a loopback /progress
endpoint. The page is #75895's hand-off screen ported verbatim (Fourier
Flow loader, one title, one line, OS light/dark, charcoal dark seeds);
failure is the terse card pointing at hermes debug share. The WinForms
card stays as the no-Edge fallback, same shape.
Salvaged from the web-shell spike: TcpListener runspace server, Edge
--app spawn with throwaway profile, degradation ladder, -SelfTestUi.
Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>
* feat(update): posix hand-off orchestrator (mac/linux quit-first updates)
scripts/desktop-update/posix.sh is the mac/linux twin of windows.ps1:
the Desktop spawns it detached and QUITS; it waits the app out, runs
plain hermes update (retry-once across the update boundary, truthful
desktop-rebuild completion), swaps/relaunches the .app bundle (mac) or
the release/*-unpacked binary when its sandbox helper is launchable
(linux), writes .hermes-update-result.json, and drives the same shim.
Repo-owned, so every update refreshes the code that drives the next one.
resolvePosixScriptHandoff mirrors the Windows resolver (with the
flat-path fallback covering the scripts/ reorg skew).
* refactor(desktop): replace the in-app posix updater with the hand-off
applyUpdatesPosixInApp is gone: mac/linux Update now quits into the
detached posix orchestrator, same shape as Windows. Deletes everything
the in-app path dragged into main.ts -- runStreamedUpdate, the rebuild
retry, the relaunch-outcome matrix (update-relaunch.ts/update-rebuild.ts
and tests), shellQuote, resolveHermesCliBinary -- and with the app dead
before the update starts, the HERMES_DESKTOP_CHILD_PID reaper-exclusion
dance (#37532) is structurally unnecessary on the desktop path.
* test(update): sandboxed repro paths as npm scripts
scripts/desktop-update/repro.sh drives the real code paths against a
disposable HERMES_HOME under /tmp: shim/shim-fail (UI dry runs), fresh
(literal install.sh), behind N (rewound checkout driven forward by the
orchestrator), error (broken venv -> abort + result file). Exposed as
npm run update:shim / update:shim:fail / update:repro:* from
apps/desktop.
* fix(update): posix hand-off truth ordering, relaunch-gate port, JSON escaping
Address helix4u's review:
- finish() now delivers the outcome BEFORE publishing it: mac bundle swap
and the linux relaunch gate run first, then the result file, marker
removal, and the shim event -- the app launch itself goes last so it
can't race the result write. A gated/skewed linux install (AppImage/
deb/rpm, broken sandbox helper) surfaces its message in the result file
AND holds the shim window open with it instead of closing on a false
'Opening Hermes...'.
- mac swap is transactional with a checked rollback; a failed install
restores the previous bundle and the result says so (exit 7 when even
rollback fails). Failed 'open' rewrites the result truthfully.
- linux gate is an exact port of the deleted update-relaunch.ts logic:
anchored path-segment match on <root>/apps/desktop/release/linux-unpacked,
chrome-sandbox absent = namespace build = fine, present = root+setuid
required, with the real opt-outs (ELECTRON_DISABLE_SANDBOX, --no-sandbox
among replayed args, or the Desktop vouching) instead of the invented
HERMES_DESKTOP_NO_SANDBOX. collectRelaunchArgs/sandboxFallbackFromEnv
live in updater-process.ts again; the Desktop passes filtered launch
args (after --) and --relaunch-cwd so a deep-link or --no-sandbox
launch survives the update.
- result/status JSON strings are escaped (git permits '"' in branch
names) and the result write is atomic (tmp + rename).
- coverage: resolvePosixScriptHandoff + ported helpers in
updater-process.test.ts (19 pass); repro.sh gate / npm run
update:repro:gate asserts the whole gate matrix and round-trips a
hostile branch name through the result JSON.
* fix(update): run hermes update from the install root + unbreak fresh repro
The posix orchestrator inherited the Desktop's cwd, and parts of the
update pipeline resolve the tree they mutate from the working directory
-- the sandboxed behind-repro caught it updating the DEVELOPER'S primary
checkout (cwd at spawn time) while reporting success against the
sandbox. cd "$INSTALL_ROOT" before running hermes update, matching the
cwd:updateRoot contract of the deleted in-app path. Verified: rerun
leaves the outside checkout untouched (reflog clean).
repro.sh fresh used a --no-interactive flag install.sh doesn't have;
non-TTY stdin (</dev/null) + --skip-setup is the real non-interactive
contract.
* fix(update): launch acceptance before the terminal event, on both orchestrators
gille's round-2 review: the terminal lifecycle claimed outcomes the
launch hadn't delivered yet.
- posix finish() reorders: outcome -> durable result+marker -> LAUNCH
WITH ACCEPTANCE -> terminal event. mac acceptance is open's exit code
(launchd rejects broken bundles loudly); linux verifies the setsid
child is still alive 1.5s after spawn, so an instant exec failure
downgrades to a held 'manual' state + truthful result instead of a
vanished 'done'. Gated skew/manual outcomes publish a real 'manual'
event (new third shim state -- still zero logic in the page).
- Renderer-free linux recovery: when no chromium-family browser exists,
manual/error outcomes fire notify-send/zenity/kdialog best-effort so a
gated non-relaunch is never a silent disappearance.
- windows.ps1 mirrors the contract: Start-DesktopRelaunch returns
verified acceptance (WMI pid alive / fallback process alive; dying
before the window appears counts as failure), and the finally block
downgrades to Show-ManualFinale + rewritten result when the launch
didn't land. Error path still relaunches after showing itself.
- repro.sh launch / npm run update:repro:launch: real-orchestrator
matrix for instant-exit relaunch downgrade and skew-message surfacing.
- posix.sh cds into the install root before hermes update (found by the
sandboxed behind-repro: parts of the update resolve the mutated tree
from cwd, which is the Desktop's cwd -- it updated the DEVELOPER'S
checkout while reporting success against the sandbox).
* fix(update): fail-closed cd, rejected-launch semantics, guaranteed recovery surface
gille's round 3:
- cd into the install root FAILS CLOSED (set -u without set -e let a
failed cd continue hermes update in the caller's tree -- the exact
wrong-tree class the correction exists to kill). Honest result, exit 3.
- A supplied mac relaunch target that is missing is a REJECTED launch ->
manual downgrade; the launch matrix asserts the downgrade instead of
codifying the old false success. A mac swap-failure DONE_NOTE now still
relaunches the kept/rolled-back bundle before publishing manual.
- notify_fallback: every rung falls through on EXECUTION failure (a
notify-send that can't reach D-Bus no longer eats the message), mac
gets osascript (present on every macOS -- Safari-only machines have no
chromium shim), and the no-surface terminal case is an explicit logged
contract: the result file carries the outcome to the next boot.
- update:repro:fresh passes --non-interactive explicitly (prompt_yes_no
falls back to /dev/tty, so </dev/null was not equivalent).
* fix(update): manual-result protocol so gated outcomes reach the user
Round 4 of helix4u's review — the durable fallback is now real:
- Result protocol gains `manual`: an ok result the user still must act
on (reopen the app, reinstall the GUI package, fix the sandbox helper).
Both orchestrators set it on every DONE_NOTE/downgrade path; the Desktop
consumer surfaces manual results in a real dialog on next boot instead
of a log line — the browserless-Linux disappearance now ends at a
visible dialog, worst case one boot later. Older result files without
the field parse as manual:false (covered).
- notify ladder verifies EXECUTION, not existence: zenity/kdialog must
survive their first second (an instant death means no display and falls
through); the no-surface case is an explicit best-effort contract whose
guaranteed channel is the result dialog.
- mac DONE_NOTE + failed relaunch of the kept/rolled-back bundle is no
longer swallowed (`|| true` dropped): the durable message carries both
facts.
- launch/gate matrices assert `manual` in the result JSON; consumer
round-trip tested in handoff-result.test.ts.
* fix(auth): /auth/native/authorize 空 provider 自动选择不再统计会被拒绝的密码 provider
Fix #78906
当部署同时启用 basic 密码 provider 与一个 OAuth/OIDC session provider 时，
list_session_providers() 会把密码 provider 也计入 "exactly one candidate"
判断（密码 provider 虽是 session provider，但下一行就会因 supports_password
被原生 OAuth broker 流程拒绝），导致 len == 2、自动选择被跳过，桌面端
空 provider 登录返回 404 "Unknown provider: ''"。
修复：自动选择只在可 broker 的 provider（supports_session 且非
supports_password）中计数，与 /api/status 的 native_pkce 能力宣告使用同一
"brokerable" 定义；当没有任何可 broker provider 时保留原有选择逻辑，
让显式的 400 错误继续解释密码 provider 不支持原生 OAuth。
新增回归测试：basic+OIDC 并存时自动选中 OIDC、单 OAuth provider 自动
选中、多 OAuth provider 歧义 404、纯密码部署保留 400。
* fix(update): exempt manual results from the hand-off freshness window
A manual:true hand-off result is the durable action-required channel: on a
browserless Linux box with no working notifier, the boot dialog is the first
and only place the message ever surfaces. The 30-minute freshness gate
discarded it if the user reopened Hermes later, stranding exactly the machine
the channel exists to serve. Parse before the age check and skip the window
for manual results; the file is still unlinked before any age check, so it's
surfaced at most once. Ordinary results still expire.
Regression: a stale ordinary result is discarded (and consumed) while a stale
manual result is still returned once.
* feat(browser): auto-install the Browser Use CLI instead of silently downgrading
The Browser Use CLI became the default browser backend, but nothing
provisioned it: users without uv/uvx (field report from DongyangHe on
macOS) silently fell back to the built-in browser tools with no notice.
- install_cli() in tools/browser_use_cli.py: uv tool install browser-use
via the managed uv (bootstrapped on demand), linked into
$HERMES_HOME/bin (UV_TOOL_BIN_DIR)
- _find_cli() now also probes $HERMES_HOME/bin for browser-use/uvx —
Hermes' managed uv is not on the user's PATH
- hermes tools post_setup actually installs (Camofox standard) instead
of printing instructions
- install.sh / install.ps1 provision the CLI at install time
(best-effort, non-fatal, honors --skip-browser)
- CLI startup shows a one-line notice (24h rate-limited) when the
default backend downgraded to the built-in tools
* fix: ASCII-only install.ps1 comment; allow-list install_cli's uv PATH fallback
- install.ps1 must stay pure ASCII (PowerShell 5.1 ANSI code-page
decoding, #66994/#67000): em-dash -> '--'
- tests/test_managed_runtime_resolution.py: install_cli()'s
shutil.which('uv') is a reviewed fallback AFTER ensure_uv() misses
* fix(relay): stop sibling gateways answering another instance's button press (#83677)
* fix(relay): stop sibling gateways answering another instance's button press
A Discord button press arrives on the passthrough plane, and the connector
fans a passthrough forward out to EVERY live gateway session of the tenant
(relayServer.routeBusMessage delivers `passthrough` via sessionsByTenant),
unlike a message, which it narrows to the admitted instance set. The prompt
went out from exactly one instance and _pending_prompts is process-local, so
every sibling gateway saw an answer for a prompt it never minted, could not
tell that from its own prompt expiring, and fell through to chat dispatch --
where the option-shaped text ("/c1") is not a real command and run.py replied
"Unknown command `/c1`". One copy per sibling, under the single real ack.
Prompt ids are now minted as `<per-process nonce>.<8 hex>`, so an answer can
be attributed to the process that minted it. A prompt answer is always
consumed, never re-dispatched as chat: a sibling's prompt and a repeat answer
are both dropped silently, and an expired prompt of our own gets a short
"no longer waiting" notice from the owning gateway only.
Ids stay inside the connector codec's contract ([A-Za-z0-9_.-], <=32 chars,
64-byte callback budget -- verified against promptCodec.ts: 52 bytes worst
case with a full-length option id). An id with no nonce segment (a prompt in
flight across an in-place upgrade) is still treated as ours.
Tests: 4 added, each verified to fail without the fix. Full relay suite green
(160 tests).
* style(tests): ruff-format the added relay prompt tests
* feat(relay): ambient token endpoint mode for gateway.idp.token_url (#84074)
* feat(relay): ambient token endpoint mode for gateway.idp.token_url
When gateway.idp.token_url is configured WITHOUT client_id/client_secret,
treat the URL as a metadata-server-style ambient credential endpoint:
plain GET, response body is the token (raw JWT or {"access_token": ...}
JSON envelope). Covers workload-identity proxies such as Domino's
$DOMINO_API_PROXY/access-token, which mint short-lived user-scoped OIDC
tokens with no client registration.
Previously this configuration was a hard error (client_id/client_secret
missing), so no working deployment changes behaviour: creds present keeps
the OAuth2 client_credentials POST, no token_url keeps Nous Portal. The
misconfig error now self-diagnoses (names the ambient fallback and how to
select the client_credentials grant instead).
* fix(relay): reject short plain-text bodies in ambient token shape gate
Review finding: the shape gate accepted any base64url-alphabet word, so an
IdP answering the ambient GET with a terse error body ('unauthorized',
'error', 'null') had that word returned as a bearer token instead of the
fail-closed misconfiguration error. Tighten the gate to JWT-like dotted
tokens (3+ segments) or long opaque tokens (>= 32 chars); short bare words
now raise the self-diagnosing ambient error.
* fix(relay): partial IdP client credentials keep the loud error, never select ambient GET
The ambient-endpoint dispatch used 'not client_id or not client_secret',
so configuring exactly one credential (a mistyped client_credentials
setup) silently issued a GET at the IdP token endpoint and then raised
'no client_id/client_secret configured' — factually wrong for that
operator, and a stray request the old hard error never made.
Ambient mode now requires NEITHER credential; a partial pair raises
immediately, names the missing key, and issues no HTTP request (tests
assert urlopen is never called). Docstring and relay.md now say
'neither' instead of 'without'.
* fix(relay): ambient JSON envelope requires a string access_token, no coercion
Review finding (P2): the JSON-envelope branch accepted any truthy
access_token via str() coercion — a number became '12345…', a boolean
became 'True', an object became its Python repr — bypassing the fail-
closed contract and deferring the failure to the connector, where it
hides the real endpoint problem.
The envelope value must now be a non-empty string, the same contract the
client_credentials path enforces on its token response. Deliberately NO
shape gate on envelope values: an envelope is an intentional token
response (mode-1 symmetry), and opaque tokens may use the standard-base64
alphabet the raw-body gate rejects. Mutation check: reverting the branch
to str() coercion sends the 3 coercion tests red (3 failed, 15 passed).
---------
Co-authored-by: Ben Barclay <ben@nousresearch.com>
* fmt(js): `npm run fix` on merge (#84193)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(auxiliary): honor main model for title generation (#83636)
* fix(gateway): scale-to-zero gateway self-suspends via flaps socket instead of relying on Fly autostop (#84295)
Fly Proxy autostop judges idle exclusively on inbound proxied connections.
It cannot see an in-flight agent turn (outbound-only LLM traffic), and since
Fly's mid-2026 proxy change an open outbound socket (the relay WS) no longer
holds a machine awake. With autostop:"suspend", Fly suspended machines while
they were still processing long-running jobs, and could suspend before the
gateway flipped the relay destination (the buffered-event black hole).
The scale-to-zero watcher now owns the suspend: after the idle predicate
holds (no running agents, no live background work, inbound-quiet) and the
go_dormant() quiesce completes (relay drained + flipped), it POSTs
/v1/apps/{app}/machines/{id}/suspend on the local /.fly/api flaps socket.
Suspend is skipped when the quiesce fails or inbound lands mid-quiesce
(flip-before-freeze), and off-Fly the step is a no-op (fail-awake).
Pairs with the NAS change that provisions scale-to-zero machines with
autostop:"off" (gateway-owned suspend); wake is unchanged (Fly-proxied
wakeUrl poke + autostart).
* fix(cron): deliver to relay-fronted platforms via canonical home_channel (#84300)
Cron jobs targeting a relay-fronted logical platform (e.g. Discord behind
the relay connector) failed twice over:
1. Target resolution read only the legacy <PLATFORM>_HOME_CHANNEL env
mirror. The canonical home_channel block that /sethome persists to
config.yaml — the only store that exists in a relay-fronted deployment,
where no native env var is exported — was never consulted, so
deliver='discord' silently resolved to nothing and the job fell back
to local-only.
2. Even with a resolved target, the delivery loop's native
configured/enabled gate rejected the platform ('not configured/enabled')
although resolve_delivery_transport had already produced a live relay
transport fronting it. A relay-fronted platform is deliberately NOT
natively enabled (its credential lives in the connector), so the native
gate must not apply to a relay transport.
Resolution now falls back from the env mirror to
config.get_home_channel(platform) for both chat_id and thread_id (thread
affinity only when the chat id came from the same config block), which
also makes the 'all' routing token pick up relay-fronted platforms. The
delivery gate honours a resolved relay transport, mirroring the
enablement rule resolve_delivery_transport already applied; the standalone
(no-relay) path keeps the historical gate byte-identical.
* fix(gateway): exclude permanent supervised watchers from the scale-to-zero busy check (#84327)
_scale_to_zero_has_live_background_work() counted every task in
_background_tasks — but _spawn_supervised parks all permanent watchers
there (session-expiry, kanban, reconnect, the scale-to-zero watcher
itself, ...). An armed gateway therefore considered itself busy forever
and never went dormant or suspended. Verified live on staging
(hermes-agent-stg-test-6698, 2026-08-12): armed at 05:25, fully idle for
25+ minutes, zero 'going dormant' lines. Fly's coarse proxy autostop used
to mask the bug; once the gateway took ownership of the suspend (#84295)
it became load-bearing.
_spawn_supervised now tags its tasks and the busy check skips them.
Transient tasks (startup-resume events, delegation, tracked processes)
still block suspend. New tests exercise the REAL _spawn_supervised path
rather than a stubbed _background_tasks set — the stubbing is exactly why
the earlier tests missed this (same call-site trap as the F25 arm bug);
the key test fails on main and passes with the fix.
* fix(relay): stamp logical platform + relay trust on Discord interaction events (#84318)
The relay interactions passthrough lane (_discord_interaction_to_event)
built its SessionSource with platform=Platform.RELAY and no
delivered_via_upstream_relay marker — unlike the relay text lane
(ws_transport._event_from_wire), which maps the connector's platform to
the logical enum and stamps the authenticated-upstream flag.
Consequences of the mismatch:
- /sethome sent as a Discord slash command persisted the home channel
under platforms.relay.home_channel (invisible to cron delivery, which
looks up the logical platform) and mirrored it into the dead
RELAY_HOME_CHANNEL env var — so cron jobs with deliver='discord' kept
falling back to local-only even after the resolution/delivery fixes.
The absent trust marker also meant via_relay=False, so the handler's
'Relay does not authenticate this logical home target' guard —
designed to reject exactly this misfiled shape — never engaged.
- Session keys forked: the connector binds the interaction's follow-up
capability under buildSessionKey with platform 'discord' and
chat_type 'group' (interactionSessionSource), while the gateway keyed
the same interaction as relay/channel.
- _capture_scope skipped recording _platform_by_chat (it ignores the
generic 'relay'), losing the egress sender hint for the chat.
Stamp Platform.DISCORD (the lane statically parses Discord interaction
wire payloads), chat_type 'group' for guild channels (native-adapter and
connector parity), and delivered_via_upstream_relay=True (parity with
the text lane; set locally, never read off the wire).
With this, slash-command /sethome files under platforms.discord and
passes the via_relay guard legitimately, and cron delivery over relay
works end to end with the #84300 resolution fixes.
* fix(cron): managed-cron fires execute in the gateway process (live adapters + dashboard forwarder) (#84339)
* fix(gateway): pass live adapters to cron fire webhook's fire_due
The Chronos fire webhook (/api/cron/fire) called
provider.fire_due(job_id, adapters=None, loop=loop), so every
externally-triggered fire delivered through the standalone path even
with a live gateway in-process. E2EE platforms and relay-fronted
logical platforms (whose ONLY send path is the live relay adapter — no
native credential exists on the box) failed every external fire with
"platform 'X' not configured/enabled", while the same job delivered
fine under the built-in ticker (gateway/run.py passes runner.adapters).
Resolve the runner (self.gateway_runner → app['gateway_runner'] →
_gateway_runner_ref(), the same chain the drain check uses) and forward
its adapters. No runner → adapters=None, preserving the historical
standalone path byte-identically.
Note: does not by itself fix Fly-hosted scale-to-zero deployments where
NAS's callback lands on the DASHBOARD process (internal_port 9119) —
_fire_cron_job_for_profile there has no gateway runner in-process. That
topology needs a separate fire handoff (design pending).
* fix(cron): dashboard forwards Chronos fires to the gateway (503 when unreachable)
The dashboard's /api/cron/fire executed cron jobs in the DASHBOARD
process via _fire_cron_job_for_profile with adapters=None. On hosted
deployments (Fly proxy exposes only the dashboard's port) that made
every managed-cron fire deliver through the standalone send path, which
cannot serve relay-fronted logical platforms (their only sender is the
live relay adapter in the gateway process — no native credential exists
on the box) or E2EE rooms. It also ran the whole agent turn inside the
dashboard: wrong process for memory/session ownership and fire-claim
attribution.
Restore the invariant that the GATEWAY owns cron execution:
- Dashboard route: after verifying the NAS JWT and resolving the job's
profile, FORWARD the fire to the gateway api_server's own
/api/cron/fire on loopback, NAS bearer preserved (the gateway
re-verifies the JWT — defense in depth, no new trust link), and pass
the gateway's response through. Gateway unreachable → 503 so NAS
retries per the Chronos contract (non-2xx = retryable; the store CAS
de-dupes the eventual double fire). Deliberately NO local-execution
fallback.
- Endpoint resolution mirrors gateway/config.py's api_server load order
per target profile (config.yaml extra.port → API_SERVER_PORT from
process env or the profile's .env → 8642), with /p/<profile>/ prefix
routing under multiplex.
- docker/stage2-hook.sh: generate a strong API_SERVER_KEY into .env on
first boot when absent (never overwrites an operator value), so the
loopback api_server passes its startup guard on hosted images. The
fire route itself is NAS-JWT-authed; the key gates the rest of the
api_server surface. The listener binds 127.0.0.1 by default and the
Fly service exposes only the dashboard port.
- _fire_cron_job_for_profile kept but deprecated (late-binding seam
compatibility); no route calls it.
- docs/chronos-managed-cron-contract.md: document the two-hop inbound
topology and the 503-retry semantics.
Depends on the previous commit (fire webhook passes live adapters to
fire_due) — together they make NAS→dashboard→gateway fires deliver over
relay end to end.
* fix(cron): read the profile api_server port via the canonical config loader
CI guard test_config_read_guard flagged the new _gateway_fire_endpoint
for a raw yaml.safe_load of the profile's config.yaml — the exact drift
class the guard exists to kill (raw reads miss the managed-scope
overlay, ${ENV_VAR} expansion, and root-model normalization).
Read through load_config() under a HERMES_HOME override scoped to the
target profile instead (the same pattern the deprecated
_fire_cron_job_for_profile uses for its store scope), and pull the port
with cfg_get. Test updated to stub load_config rather than write a raw
config.yaml.
* fix(gateway): only messaging platforms count for the scale-to-zero arm gate
The stage2 hook now generates API_SERVER_KEY for every Docker container,
and key presence force-enables the api_server platform. The scale-to-zero
arm gate counted every enabled platform, so the loopback api_server
listener made messaging_is_relay_only_or_absent False on every hosted
instance — silently disarming the feature (the not-armed log would show
enabled platforms=['relay','api_server']).
The arm gate and the not-armed logger now share one helper that filters
to enabled MESSAGING platforms, excluding LOCAL/API_SERVER/WEBHOOK —
the same non-messaging exclusion set _connect_platforms already uses.
A genuinely enabled direct-socket platform (Discord/Telegram) still
disarms. Two of the three new tests fail without this fix.
* fix(agent): log Codex transport failure details
* fix(agent): tolerate transport errors without requests
* fix: widen APIConnectionError handling to finalization drain loop
The PR added APIConnectionError handling to the main request and
iteration try blocks but missed the finalization drain loop (line ~1492).
That site catches httpx transport errors to preserve an already-completed,
already-billed response when the drain iterator fails. Without the
APIConnectionError handler, an SDK-wrapped transport error during drain
would propagate uncaught and discard the completed response.
Also strengthens the test's no-payload-leak assertion to check the full
request body and URL are absent from the log message, not just the
literal string 'payload'.
* fix(kanban): query show graph before closing database
* chore: map contributor email cmoiccool
* fix(nix): set HERMES_BIN default in wrapped binaries
The TUI resolves the CLI via process.env.HERMES_BIN (externalCli.ts) and
falls back to a bare 'hermes', which is not on PATH for nix run / nix
profile installs that only expose the wrapped binaries. Set a
--set-default so the wrapper advertises its own hermes while an explicit
operator override (documented in kanban_db.py) still wins.
* fix: warn agents off driving interactive console TUIs via pty on Windows (#84364)
* fix: warn agents off driving interactive console TUIs via pty on Windows
Driving 'gh auth login' (and other survey-style console TUIs) through a
pty background process on Windows silently hangs: these programs read
Win32 console key events via ReadConsoleInput, not the stdin byte
stream, so Enter keypresses submitted over process stdin never register.
The agent-visible symptom is a prompt frozen at 'Press Enter to open
browser...' while the user sees nothing, and a turn interrupt then kills
the process, invalidating any device code the user already entered on
github.com.
Two guidance fixes, both proven in a live session on Windows 10:
- agent/prompt_builder.py: extend _WINDOWS_BASH_SHELL_HINT to steer
agents toward non-interactive paths (flags, --with-token, config
files, curl-polled OAuth device flow) instead of answering console
prompts programmatically.
- skills/github/github-auth: document the pitfall and add the manual
OAuth device-flow procedure (curl against gh's public client_id,
poll for the token, finish with 'gh auth login --with-token'), which
succeeded first try after two interactive attempts hung.
* fix: send CRLF for Enter on Windows PTY submit; correct root cause in guidance
Review feedback (helix4u) was right on both counts:
1. Root cause correction. gh's 'Press Enter to open browser' prompt is
waitForEnter -> bufio.Scanner reading stdin, not a survey/console-API
prompt. The real bug is ours: submit_stdin appended a bare \n, and
through pywinpty/ConPTY a lone \n is not delivered as a line
terminator, so the child's blocking line read never returns. Verified
empirically against pywinpty 2.0.15 with a readline() child:
\n -> hang, \r -> line delivered, \r\n -> line delivered.
Fix: submit_stdin now appends \r\n for Windows PTY sessions (POSIX
PTYs and Popen pipes keep \n). Windows-only regression tests cover
the PTY and pipe branches.
2. Prompt hint rewritten: instead of claiming Windows console TUIs
cannot be driven, it now says to use process(submit) rather than raw
writes with bare \n, and to prefer non-interactive paths when a CLI
offers one.
3. Skill device flow rewritten as an executable script: parses the
device-code response, polls per the returned interval, handles
authorization_pending / slow_down (+5s per GitHub docs) /
expired_token / access_denied / unexpected responses, pipes the token
straight into gh without echoing it, and drops the undocumented
workflow scope (repo,read:org,gist is the documented minimum for
gh auth login --with-token). The pitfall note is narrowed to the
reproduced condition.
* fix: make verify_on_stop opt-in everywhere (default False, not auto) (#84383)
* fix: make verify_on_stop opt-in everywhere (default False, not auto)
The verify-on-stop nudge was already judged more noise than signal: the
v31 migration flips existing installs off, the v32 migration catches the
baked-in literal-true population, and the docs tell users to 'treat off
as the effective default and opt in explicitly'. But DEFAULT_CONFIG still
shipped the "auto" sentinel, so exactly one population kept getting the
nudges: fresh installs (and any config missing the key), where "auto"
resolves ON for CLI/TUI/desktop surfaces. Live symptom: repeated
'[System: You edited code ... run verification]' interruptions the user
never asked for and had to hunt down in source to disable.
- DEFAULT_CONFIG: agent.verify_on_stop "auto" -> False (opt-in).
- verify_on_stop_enabled(): missing/unrecognized value now falls back
OFF instead of surface-aware; explicit "auto" still selects the
legacy surface-aware behavior, explicit bools unchanged, and the
HERMES_VERIFY_ON_STOP env override is untouched.
- No migration needed: v31/v32 already normalized existing installs,
and this only changes the merged default for configs without the key.
- Docs updated; default-path E2E test now asserts OFF, plus a new
missing-value regression test. Also added the standard win32 skip
marker to the symlink-based temp-dir test (pre-existing Windows
failure, same class as tests/cron/test_cron_script.py).
* test: update config goldens — verify_on_stop=False is now stripped as default
With the DEFAULT_CONFIG flip to False, the migration-write invariant
(_persist_migration / save_config strip_defaults) no longer materialises
verify_on_stop: false to disk unless the user explicitly set the key:
- V20 floor fixture (agent: {} on disk): v31's write is stripped —
agent stays {} and load_config() supplies False at read time.
- V12 floor fixture (explicit verify_on_stop: true on disk): the key is
a user-set path, so the v32 flip stays materialised as false.
- Partial-write and _persist_migration regressions now assert the key is
absent from disk and (for the merge case) that the merged view still
resolves False.
Behavior verified with a one-shot migrate_config run against both
fixture shapes.
* fix: Windows path handling in search_files rg calls and patch escape drift (#84378)
* fix: Windows path handling in search_files rg calls and patch escape drift
Two related Windows failures from a live session (Windows 10, git-bash
terminal backend, winget-installed native ripgrep):
1. search_files was unusable on drive-letter paths. _escape_shell_arg
rewrites C:\... to the MSYS form /c/... so bash builtins resolve it,
but rg is a native Windows binary and Hermes disables MSYS argument
conversion for its bash subprocesses (MSYS_NO_PATHCONV=1 /
MSYS2_ARG_CONV_EXCL=*, see _apply_windows_msys_bash_env_defaults) —
so nothing ever translated /c/... back and every search failed with
'The system cannot find the path specified. (os error 3)'.
Fix: new _escape_native_tool_arg emits the forward-slash NATIVE form
(C:/Users/...), which native binaries accept, bash passes through
untouched, and MSYS builds also handle. Applied to the six rg call
sites (content search, --files search x2, zero-match probe x3); the
grep fallback keeps the MSYS form since MSYS grep wants it.
2. The patch tool silently doubled backslash runs when tool-call args
arrived JSON-escaped one extra time (file had \ where old_string
had \\). Similarity strategies (context_aware) matched the region
anyway and wrote new_string verbatim, corrupting every backslash run
(reproduced: 6 backslashes on the line became 12). _detect_escape_drift
now also blocks when every backslash run in old_string is exactly twice
its counterpart in the matched region and new_string repeats the
doubling — with guardrails so exact matches, intentional backslash
edits, model-corrected new_strings, and single weak-signal runs all
still apply. Blocking returns the standard escape-drift guidance so
the model re-reads and retries with correct counts.
Tests: TestEscapeNativeToolArg (5 cases, including an end-to-end
_search_with_rg command capture) and TestBackslashDoublingDrift (6
cases). The 8 pre-existing failures in tests/tools/test_file_operations.py
on a Windows host (umask/symlink POSIX assumptions) are identical on
unmodified main and unrelated.
* fix: shell linters get native Windows paths too (node C:\c\... double-prefix)
Same class as the rg fix: LINTERS commands (python -m py_compile,
node --check, npx tsc, go vet, rustfmt) invoke native Windows binaries,
but _check_lint interpolated the MSYS /c/... form. node resolves that
as C:\c\Users\... (double-prefixed), so on Windows hosts every .js
write reported a phantom ENOENT lint failure that could mask real
syntax errors (issue #84303). Route the {file} arg through
_escape_native_tool_arg like the rg call sites.
Regression test asserts node --check receives 'C:/...' and never
'/c/...'.
* delete tmp file lol
* feat: add Nemotron Lightning to reasoning timeout (#83982)
* fix(tools): strip heredoc bodies before background-'&' detection
_strip_quotes documented that it stripped heredoc bodies but only handled
single/double/backtick quotes. As a result _foreground_background_guidance
scanned heredoc body text for a backgrounding '&' and wrongly rejected valid
foreground commands whose heredoc body contained a spaced ampersand — e.g.
AppleScript string concat (osascript <<'EOF' ... "a" & b ... EOF), Python
bitwise-and, or literal UI text like 'FaceTime & Privacy'.
Add a _strip_heredocs pass (runs before quote-stripping, since a heredoc
delimiter may itself be quoted) covering <<EOF, <<-EOF, <<'EOF', <<"EOF".
The same-line tail after the opener (redirects/args) is preserved and the
opener token is blanked so a real backgrounding '&' after the heredoc is
still detected.
Adds tests/tools/test_terminal_heredoc_background_guard.py.
* fix(tools): harden heredoc masking into a conservative shared helper
The previous commit's regex-based stripper removed EVERY heredoc body,
which review flagged as bypassable: a fake '<<EOF' marker inside a
comment or quoted string enters the unterminated path and swallows a
later REAL background operator, and unquoted ('cat <<EOF' — expansion
runs) or shell-consumed ('bash <<'EOF'' — body IS shell) bodies are
executable content that must stay visible to the guard.
Replace it with tools/shell_heredoc.strip_inert_heredoc_bodies(), a
conservative shell-state scanner: a body is masked ONLY when every
delimiter on the opener is quoted (no expansion), every heredoc is
terminated by an exact delimiter line, the opener composes a single
command (no list/pipeline operators, no nested $()/backtick/process-
substitution scope), and the consumer is an allowlisted non-shell
interpreter (python/osascript/cat). Anything ambiguous is returned
unchanged — a false positive on exotic syntax is acceptable; hiding a
real background operator is not. Masked bodies become newlines so line
structure is preserved for MULTILINE regexes.
The helper is a standalone stdlib-only module (precedent:
tools/ansi_strip.py) because the same heredoc-as-data false-positive
class exists in the blocked-command regex checks (#83104) and the
gateway lifecycle guard (#81721/#79835, cron/lifecycle_guard.py) —
which must not import the terminal-tool module graph.
Adapted from Wolfram Ravenwolf's security-hardened rework of #63788
(69c7663c6de6b6cb05bf99203fa39673efe01ccf); test scenarios for the
bypass cases derive from his suite.
Co-authored-by: Wolfram Ravenwolf <github.com@wolfram.ravenwolf.de>
* perf(tools): linear-time masking rebuild + last-opener early exit
Efficiency review (measured with timeit probes) found two unbounded
costs on adversarial inputs:
- The masked-range rebuild copied the whole string once per range
(O(n*k)): 50k tiny heredocs took 1.7s. Replaced with a single-pass
segment join over the (sorted, non-overlapping) ranges: 152ms, and
newlines are now counted on the original command instead of
re-slicing.
- After the last '<<' occurrence no opener can start, but the scanner
still walked the remaining text per-char: one heredoc followed by a
1MB tail cost ~150ms. An rfind bound breaks out of the unit loop
once the scan passes it: 0.3ms.
Typical commands are unaffected (the '<<' fast path already returns
first). 30/30 guard tests pass; mutation check re-run on the final
stack (no-op mutation -> 11 tests fail, restore -> green).
* fix(cache): opt M3 out of cache_control markers on Anthropic wire
MiniMax-M3 ships server-side automatic prefix caching on the
Anthropic-compatible endpoint (content-keyed, no marker needed —
see platform.minimax.io/docs/api-reference/text-prompt-caching).
cache_control markers are NOT on its explicit-cache support list
(which covers only M2.7/M2.5/M2.1/M2).
Emitting markers on M3:
- wasted serialization overhead
- risked perturbing the server-side prefix hash
- gave users a false sense of explicit-cache savings (the
cache_read_input_tokens field carries a +128 constant floor
and cache_creation_input_tokens is always 0 for M3)
Also add an opt-in debug=True parameter to normalize_usage() that
emits a debug-level log line carrying the observable cache fields.
This is the only reliable cache signal for M3 — off by default,
debug-level, scoped to the anthropic_messages wire, so production
callers see no impact.
Pin both changes with 8 new tests:
- 4 M3 tests covering provider, host, and custom-provider paths
- 1 regression guard ensuring M2.x caching is unaffected
- 3 observability tests (off-by-default, on-with-M3, on-with-Claude)
Verified end-to-end against api.minimaxi.com/anthropic/v1/messages
with MiniMax-M3[1m]: identical system prompt hit-rate with and
without markers; cache_read field is unreliable (128 floor),
input_tokens drop (8467 -> 1) is the real hit signal.
* fix: close provider-anthropic MiniMax proxy bypass + rework cache observability
Follow-up fixes on top of the salvaged #83678 commit:
1. Hoist the MiniMax-M3 marker exclusion ABOVE the native-Anthropic
early return. provider="anthropic" pointed at a MiniMax /anthropic
proxy is a supported override (_anthropic_base_url_override_ok), and
the is_native_anthropic branch matched on provider alone — returning
(True, True) before the M3 exclusion was reached. Two regression
tests pin the proxy route (M3 off, M2.7 still on).
2. Reuse the existing _model_name_suggests_minimax_m3() helper from
agent/model_metadata.py instead of a second inline substring copy.
3. Drop the debug kwarg on normalize_usage() — it had zero production
callers and duplicated standard logging level gating. The
cache-observability line is now a plain logger.debug scoped to
MiniMax providers on the Anthropic wire only, so the "+128 floor"
note can no longer appear for native Anthropic where it is false.
Tests updated accordingly (MiniMax logs, native Anthropic does not).
* chore: map hermes-agent@nous.local commit identity to @C-EXCITE-STUDIO
Salvaged PR #83678's commit is authored under a generic local agent
identity with no linked GitHub account; map it to the PR opener for
release attribution (same pattern as hermes-agent@users.noreply.local).
* fix: Windows agent-loop papercuts — path splitting, hashing, autocomplete, screenshots, OS detection (#84419)
Sweep of open Windows issues affecting day-to-day agent operation
(explicitly excluding install/setup and locale classes):
- hermes_cli/_subprocess_compat.py: new split_command_line() — Windows-
safe command-line tokenizer (posix=False + quote stripping) so
backslash paths survive. POSIX behavior unchanged (plain shlex.split).
- hermes_cli/console_engine.py (#83934): console commands like
'sessions export C:\Users\me\out.jsonl' no longer silently mangle the
path into a relative filename in the cwd.
- agent/shell_hooks.py (#78293): hook commands with backslash paths now
spawn, resolve their script path, and pass hooks doctor instead of
reporting 'not executable'. All three shlex sites routed through the
shared splitter.
- agent/prompt_builder.py (#51755): system prompt now reports
Windows (11) on Windows 11 — platform.release() returns 10 for both;
distinguish via sys.getwindowsversion().build >= 22000.
- hermes_cli/commands.py (#42016): @ autocomplete no longer crashes the
prompt_toolkit event loop when rg emits a path on a different mount
(device paths \.\nul, other drive letters) — relpath ValueError is
skipped per-entry.
- tools/browser_use_cli.py (#83884): screenshot-path detection now
matches Windows drive-letter paths (C:\... and C:/...) in addition to
POSIX; Browser Use screenshots attach on Windows.
- tools/skills_hub.py + tools/skills_guard.py (#62310): the two 'MUST
stay symmetric' skill content hashes actually agree on Windows now.
Bundle keys are normalized to POSIX separators before hashing, and the
disk digest sorts by rel-posix STRING (case-sensitive) instead of Path
objects (case-insensitive on Windows). Fixes permanent false-positive
update_available for every installed skill.
Tests: tests/tools/test_windows_agent_loop_papercuts.py — 16 cases
covering each fix, including a disk-vs-bundle hash symmetry check built
with native Windows separators and a mixed-case filename.
* fix: steer agents off MSYS paths for native tools; pin line-ending preservation (#84426)
Two follow-ups from live Windows sessions:
1. agent/prompt_builder.py: extend the Windows shell hint with the
native-binary path rule. Hermes disables MSYS path conversion for its
bash, so agents passing /c/Users/... or /tmp/... to NATIVE programs
(git -C, node, python, rg) hit 'cannot change to' / 'not found' while
the same path works in bash builtins — observed repeatedly in a live
session (git -C failures, git apply /tmp/x.patch failures). The hint
now says: forward-slash native form (C:/Users/x) for native tools,
$LOCALAPPDATA/Temp over /tmp for scratch files native tools read.
(/tmp is pure model habit from Linux training data — nothing
instructs it — so the hint is the right layer.)
2. tests: pin LF/CRLF preservation through write_file and patch_replace.
A live session saw a repo-LF file come back full-CRLF after an edit
(4699-line diff churn); not reproducible through current tool APIs,
so pin the correct behavior — LF files stay LF, CRLF files stay CRLF,
no mixed endings — to catch any regression on the Windows write path.
* fix(security): approval system covers Windows destructive commands and paths (#84428)
Fixes #69472. On a Windows host every destructive native command passed
approval silently — DANGEROUS_PATTERNS were POSIX-shaped, and the
normalizer strips backslashes as shell escapes so no Windows path could
ever match a path rule. Probed live before the fix: 15 of 15 destructive
Windows commands (Remove-Item -Recurse -Force, del /s /q, iwr | iex,
taskkill /F, Format-Volume, diskpart, icacls /grant Everyone, vssadmin
delete shadows, bcdedit /set, reg delete, cipher /w, ...) sailed through
undetected.
Two changes:
1. Windows destructive tier in DANGEROUS_PATTERNS: PowerShell deletes
(bare Remove-Item -Recurse/-Force), cmd builtins with /s|/q switches,
iwr|iex remote execution (pipe and subexpression forms), taskkill /F /
Stop-Process -Force, volume/disk destruction (Format-Volume,
Clear-Disk, diskpart, format.com, cipher /w), icacls Everyone-grant /
/reset, backup destruction (vssadmin delete shadows, wbadmin delete,
bcdedit /set), reg delete / Remove-ItemProperty -Force, and service
stop/delete (Stop-Service -Force, sc stop|delete). Each pattern
requires the destructive flag so graceful/read-only usage (taskkill
/IM without /F, reg query, icacls inspect, sc query, plain del file)
does not prompt. Patterns live in the main list, not a win32-gated
tier: a Linux-hosted Hermes can drive a Windows box over SSH.
2. Windows-path detection variant in _command_detection_variants: when
the raw command contains a drive-letter/UNC backslash path, also
yield a variant with backslashes flattened to forward slashes BEFORE
normalization strips them, plus Windows spellings of the credential
path rules (Users/<u>/.ssh, AppData/{Local,Roaming}/hermes .env).
Gated on a real path shape so POSIX escape semantics are untouched.
Tests: tests/tools/test_approval_windows.py — 48 cases (27 destructive
flagged, 13 benign not flagged, 5 credential paths in both separator
spellings, 4 POSIX-escape non-regressions). The 8 pre-existing failures
under '-k approval' on this Windows host are identical on unmodified
main (ordering artifacts + known symlink cases) and unrelated.
* fix: Windows MCP PATHEXT resolution + python3 -> python in cross-platform skills (#84429)
Two Windows agent-loop friction fixes:
1. tools/mcp_tool.py (#56536): shutil.which(cmd, path=env_path) reads
executable extensions from the PARENT process PATHEXT, not the MCP
subprocess env — a stdio MCP config supplying both PATH and PATHEXT
could fail to resolve a command its own env can locate, and startup
then got a bare command name. On Windows, when the first which() call
misses and the config env carries PATHEXT (any key casing), retry the
resolution with the config's PATHEXT temporarily applied.
2. skills/ + optional-skills/ (#50606): 42 SKILL.md files that declare
platforms: [.., windows] used python3 in their command examples.
python3 does not exist on native Windows (the toolchain probe in the
system prompt reports python3=missing), so every copy-pasted example
burned a failed agent turn before self-correction. Replaced the
command word python3 -> python (python3-config / python3.x version
strings untouched). python is the spelling that exists in every
Hermes-managed environment (Windows native, uv-managed venvs on all
three OSes); agents on POSIX hosts additionally see the probed
toolchain line and adapt either way.
* fix(tools): clarify identical old and new string error
* fix(tools): improve patch tool parameter description
* refactor(tools): extract IDENTICAL_STRINGS_ERROR constant
The 3-sentence identical-edit message was snapshot-asserted verbatim in
two tests. House style avoids exact-string change-detector assertions;
both tests now import the constant from tools/fuzzy_match so rewording
the message can't silently break them.
* fix(tools): mirror must-differ guidance in skill_manage new_string schema
skill_manage's patch action uses the same fuzzy_find_and_replace engine
as the file patch tool and surfaces the identical-strings error verbatim
— and unlike the file path it has NO is_already_applied no-op rescue, so
identical old/new ALWAYS errors there. Mirror the new_string description
so the schema warns before the error fires (sibling-site parity with
tools/file_tools.py PATCH_SCHEMA).
* fix(tools): skip degenerate identical hunks in V4A validation
The apply phase already skips a hunk whose -/+ lines are identical
(patch_parser.py '(search_lines == replace_lines): continue'), but the
validation phase lacked the guard: such a hunk reached
fuzzy_find_and_replace, whose identical-strings error names
old_string/new_string — parameters that don't exist in patch mode — and
failed the whole atomic patch that apply would have accepted. Mirror
the apply-phase skip in validation; regression test drives a mixed
degenerate+live patch end-to-end (short text dodges the
is_already_applied >=8-char rescue).
* fix(windows): SSH ControlMaster gating + stop hijacking the user's python (#84452)
* fix(windows): SSH ControlMaster gating + stop hijacking the user's python
Two Windows environment-integrity fixes:
1. tools/environments/ssh.py (#73927): Windows OpenSSH has no
Unix-domain-socket ControlMaster support, so unconditionally passing
ControlPath/ControlMaster/ControlPersist failed EVERY tool call on a
Windows-hosted ssh terminal backend with 'getsockname failed: Not a
socket'. Gate the three multiplexing options behind a module-level
_SSH_MULTIPLEX = (os.name != 'nt'); the scp upload path is gated the
same way. On Windows the backend now works without connection pooling
(each command a fresh connection); POSIX behavior is unchanged. The
teardown 'ssh -O exit' is naturally inert because the socket never
exists on Windows.
2. scripts/install.ps1 (#83797): the installer put the whole
venv\Scripts directory on the user PATH, which contains python.exe /
pythonw.exe / pip.exe and so silently hijacked the 'python' command in
every terminal on the machine — unrelated projects started resolving
python to Hermes' runtime interpreter. Now copy only the launchers
(hermes.exe, hermes-acp.exe) into a dedicated $InstallDir\bin and put
THAT on PATH. Existing installs are migrated: the legacy venv\Scripts
entry is stripped from the user PATH on the next install/update. The
new bin dir is under $InstallDir (…\hermes-agent), which the uninstall
PATH sweep already matches via its \hermes-agent marker.
Updated the stale hermes_cli/update_cmd.py docstring that described the
old venv\Scripts-on-PATH layout.
Tests: SSH ControlMaster gating pinned both directions (multiplex on →
flags present; off → absent but BatchMode/StrictHostKeyChecking retained).
install.ps1 parses clean via the PowerShell AST parser.
* docs: update windows-native install docs for the bin\ launcher layout
CI (test_windows_native_docs) pins the docs and installer to the same
PATH layout. The #83797 fix moved the PATH entry from venv\Scripts to a
dedicated $InstallDir\bin holding only the hermes launchers, so update
the Windows-native guide to match: PATH-after-install section, the
install-steps list, the directory-layout table, the Get-Command
verification line, and the 'command not found' pitfall. Test now asserts
the bin\ layout and guards against a regression back to venv\Scripts on
PATH.
* fix: keep install.ps1 pure ASCII (PowerShell 5.1 codepage safety)
The two comments I added in the #83797 PATH-hijack fix used em-dashes,
tripping tests/test_install_ps1_ascii_only.py — Windows PowerShell 5.1
reads a BOM-less .ps1 in the system ANSI codepage (not UTF-8), so a
non-ASCII byte can misdecode into a stray quote and desync the parser
(issues #66994/#67000). Replace the em-dashes with ASCII '--'.
* fix(tools): improve error message when wrong args
* feat(tests): add tests for execute_code error mesages
* fix(tools): redirect non-string code payloads in execute_code handler
Review follow-up on the salvaged handler: a non-string 'code' (int,
dict, list) reached code.strip() and surfaced as a generic
'Tool execution failed: AttributeError' — the same unrecoverable shape
the salvage exists to eliminate. Add an isinstance guard beside the
'command' check that names the received type and shows the correct
call form; narrow the docstring to what the handler actually does.
Regression test drives int/dict/list through registry.dispatch and
asserts no AttributeError leaks (mutation-checked: removing the guard
fails 3 subtests).
* fix(tools): mirror misplaced-arg recovery on the terminal side
Whole-bug-class sibling of the execute_code fix: terminal(code=...) —
the reverse confusion — fell through to command=None and failed with
'Invalid command: expected string, got NoneType', naming neither the
stray 'code' argument nor execute_code as the right tool. Mirror the
guard in _handle_terminal (verified live: the opaque NoneType error
reproduces on main). Mutation-checked: removing the guard fails the
new regression test.
* fix(tools): isolate external project environments
* feat(tests): add tests to cover external-venv PYTHONPATH isolation
* fix(tools): harden interpreter-environment probe for the strict-mode default
Follow-up to the salvaged #81201 commits:
- Short-circuit _uses_hermes_python_environment when the child IS the
running interpreter (path or realpath match). The default strict-mode
path no longer spawns a probe subprocess at all, and a flaky probe of
sys.executable can never drop the hermes root from PYTHONPATH
(protects the test_repo_root_modules_are_importable invariant). The
realpath leg also covers uv-style venvs whose bin/python resolves to
the same binary.
- Stop caching failed probes: _python_environment_prefix now uses a
success-only dict cache instead of lru_cache, so one transient
timeout under load no longer sticks for the process lifetime.
- Deduplicate the subprocess probe scaffolding shared with
_is_usable_python into _probe_python().
- Log once when the hermes root is omitted so import-behavior changes
are diagnosable from user reports.
- Tests: fail the composition tests loudly if execute_code never
reaches Popen (was vacuously passing on exceptions); assert the
staging dir is literally first in PYTHONPATH (was truthiness only);
add guards for probe-failure retry and the no-probe short-circuit.
* refactor(tools): unify probe caches and dedupe the exclusion log
/simplify-code findings on the full PR diff:
- _is_usable_python had the same sticky-failure bug the previous commit
fixed in _python_environment_prefix: lru_cache pinned a transient
probe failure (fork pressure, timeout) as False forever, silently
locking project mode to sys.executable. Both probes now share a
success-only bounded dict cache via _cache_probe_result() with FIFO
eviction at _PROBE_CACHE_MAX (the old < cap guard stopped caching new
entries instead of evicting, re-probing entry 33+ on every call).
- The hermes-root-omitted logger.info fired on every external-env call
in project mode; now deduped once per interpreter path per process
(matching the tirith/mcp warn-once convention).
- Regression test: _is_usable_python probe failures are retried, not
cached (mutation-verified).
* docs(browser): document Lightpanda local engine
* fix: correct Lightpanda fallback docs — remove nonexistent PDF/upload/clipboard actions
Hermes has no browser PDF, file upload, or clipboard tools. The fallback
mechanism only covers commands in _FALLBACK_ELIGIBLE (open, snapshot,
screenshot, eval, click, fill, scroll, back, press, console, errors).
The original docs described Lightpanda's general limitations, not
Hermes's actual behavior.
* add grok 4.6 (#84661)
* docs: present /export and /import as the second way to share a profile
The distributions guide framed export/import as local backup only, so the
new slash commands read as a competing path instead of the lightweight
half of one story. Give profile-distributions.md a comparison table up
front (git repo vs single file: updates, versioning, setup cost, what
each carries), rewrite the Not-a-fit bullets that mislabeled export, and
add a full Export/import section covering the CLI, TUI, and desktop
entry points, the desktop.json overlay, and what an archive actually
contains — including that it can carry memories and sessions, which a
distribution never does.
Also register /export and /import in the slash-command reference (they
shipped undocumented), point the profile-command entries at their chat
and desktop doors, and cover the desktop Export/Import UI on the desktop
page.
* fix(file-safety): approval-gate ~/.ssh/config writes instead of hard-denying (#84663)
The write_file / patch file tools hard-denied ~/.ssh/config as a
"protected system/credential file", while the terminal tool only
*asked* for approval on ~/.ssh writes. That inconsistency meant a write
to ~/.ssh/config was refused via write_file but succeeded via terminal
after an approval prompt -- the same operation flip-flopping between
denied and OK depending on which tool ran it.
The SSH client config carries no private-key material, and editing it
(host aliases, ProxyJump, VS Code Remote-SSH targets) is a routine,
user-initiated task. It CAN carry ProxyCommand / Match exec directives
that run commands, so a free write is still inappropriate -- approval,
not a flat refusal, is the right policy, matching what the terminal tool
already does.
Changes:
- agent/file_safety.py: remove ~/.ssh/config from the flat credential
deny; add build_write_approval_paths() + is_write_approval_required(),
and short-circuit it out of the ~/.ssh/ prefix deny so the file is
allowed at the classifier layer. Private keys, authorized_keys, and
everything else under ~/.ssh/ stay hard-denied.
- tools/file_tools.py: _check_approval_required_write() routes ssh config
writes through the shared _run_approval_gate (once/session/always,
honors --yolo, fail-closed with no human), wired into write_file_tool
and patch_tool right after the protected-instruction gate.
- Non-interactive consumers fail closed: the ACP file bridge
(copilot_acp_client) rejects approval-required paths outright, and the
TTS output-path picker refuses them as before.
- Docs + tests updated (security.md exception note;
TestSshConfigApprovalGate covers config approval-gated, keys still
hard-denied).
* fix(desktop): keep config/structured code blocks fenced instead of unwrapping to prose (#84664)
* fi…
teknium1
pushed a commit
that referenced
this pull request
Aug 17, 2026
… (OOF-266) Since the managed-cron redesign (#84339, v2026.8.13) the dashboard fire webhook forwards fires to the gateway process and returns 503 when it is unreachable so NAS/QStash retries. Correct for transient windows — but an operator-STOPPED gateway can never be fixed by retrying: every fire on every job burns the full scheduler retry budget, NAS converts each 503 to a retryable 502, and the resulting storms page on-call for a non-incident (OOF-266 and its five duplicate tickets; +93% relay callback failures as the fleet adopted v2026.8.13). Split the unreachable path by durable operator intent: - desired_state == "stopped" (written only by the s6 lifecycle commands; the same intent signal container-boot reconciliation trusts) -> drop the fire with 200 + a structured log line, mirroring NAS's own instance_stopped drop. Jobs are not lost: the Chronos provider reconciles and re-arms every job on the next gateway start. - Anything else (crash loop, scale-to-zero wake, restart, legacy state file without desired_state) -> keep the retryable 503, now stamped with Retry-After: 60 so a scheduler that honors it spaces retries past the wake/restart window instead of exhausting them inside it. The gateway's own pass-through 503s (draining) get the same hint. The intent check fails open (any parse/resolution error -> retryable path) and is only consulted when the gateway is actually unreachable, so a stale state file can never shadow a live gateway.
vashkartik
added a commit
to vashkartik/hermes-agent
that referenced
this pull request
Aug 17, 2026
* fix(cli): bound the Windows process-scan probes so a slow WMI scan cannot wedge hermes update (#87134)
subprocess.run(capture_output=True, timeout=N) is not hang-safe on
Windows: after the timeout fires, run()'s cleanup kills the direct child
and then joins the pipe reader threads with an UNBOUNDED communicate().
A descendant (conhost.exe under wmic/powershell) holding duplicated pipe
handles keeps the pipes from EOF and the join never returns.
_scan_gateway_pids() runs its wmic / Get-CimInstance Win32_Process scans
exactly that way, and on machines where the full process scan genuinely
exceeds its 10/15s budget (cold WMI on first boot, ARM VMs, heavy
Update/AV activity) hermes update wedged forever inside
_pause_windows_gateways_for_update() before printing a single line —
observed live on a fresh Windows 11 ARM64 VM with a faulthandler stack
pinning the main thread in subprocess._communicate and only a conhost.exe
child surviving. The single-flight update lock then blocks retries until
the wedged process is killed by hand.
This is the same deadlock class bounded_git_probe already fixed for git
probes (#68609 / #66037). Generalize that proven pattern into a shared
bounded_probe_run() — explicit communicate(timeout), kill_process_tree on
failure, bounded 1s drain, then abandon the daemonic readers — and
migrate the whole call-site class onto it:
- hermes_cli/gateway.py _scan_gateway_pids (the site that hung; reached
from hermes update, cron, gateway restart/status, dashboard)
- hermes_cli/dashboard_procs.py wmic scan (same shape, reached on update)
- hermes_cli/claw.py tasklist + PowerShell probes (same shape; its
try/except cannot catch a hang because a hang raises nothing)
- bounded_git_probe now delegates to bounded_probe_run (identical
contract, one copy of the cleanup logic)
Unlike bounded_git_probe, bounded_probe_run returns the CompletedProcess
(or None) rather than collapsing to stdout, because the gateway scan
branches on returncode to trip its wmic -> powershell fallback.
Tests: tests/hermes_cli/test_bounded_probe_run.py covers success,
nonzero-exit passthrough, spawn failure, bounded timeout (fails against
the old unbounded semantics — verified by sabotage), errors= decoding,
DEVNULL stdin, POSIX process-group placement, and the bounded_git_probe
delegation contract. Existing test_git_probe_tree_kill.py passes
unchanged against the delegated implementation.
Closes #87134
* test(cli): retarget the wmic-encoding regression test at bounded_probe_run
The Windows-only test asserted encoding/errors kwargs on a mocked
subprocess.run, but the scan now routes through bounded_probe_run
(#87134), so subprocess.run is never invoked. Assert the probe call's
contract instead (errors='ignore', finite timeout), verify the parsed
PIDs, and add a fail-open case for probe failure. The test no longer
needs a Windows host once the probe is mocked, so the windows_only
gate is dropped.
* fix(agent): attribute background-review usage and add cost controls
Persist fork token usage under session_model_usage task=background_review,
emit a per-fork completion log line, and expose enabled/max_iterations/
prompt_file so operators can see and bound the automatic review cost.
Address review feedback: load auxiliary.background_review once per spawn,
classify completion logs by summarize action prefixes, treat explicit
api_call_count=None as the documented default of 1, and WARNING on the
fail-open enabled-gate path.
* fix(desktop): route registry 'local' entry to the genuinely-local runtime
ensureRegistryBackend delegated kind==='local' to ensureBackend(), which
follows the v1 connection.json routing table — under a v1 REMOTE global
mode (the migration keeps the mandatory 'local' entry AND makes that
remote the registry primary) the roster's 'This device' rows enumerated
and dialed the REMOTE primary: every profile appeared twice (forcing
-slug handles) and clicking a local agent talked to the remote box.
resolveRegistryLocalRoute() (pure, colocated with the registry helpers)
now decides the local entry's path: delegate to the legacy route only
when v1 is itself local (single-source behavior byte-identical);
otherwise spawn/reuse a forced-local pool child via spawnPoolBackend's
new forceLocal option, pooled under the composite conn:local::<profile>
key so it cannot collide with the v1 remote descriptor cached at the
bare profile key.
* fix(desktop): key fan-out event consumption by (connectionId, profile)
Secondary-gateway events were tagged with connectionId (store/gateway
fan-out) but no consumer read it: working/attention tracking, the
pruneSecondaryGateways keep-set, and the profile-scoped event gates
(skin.changed / change-watcher broadcasts / approval-mode reconcile)
all keyed by session id + bare profile name. Every registered source
exposes a 'default' profile (the roster force-unshifts it), so two
connected gateways collided — gateway B's 'default' activity was
attributed to gateway A's 'default', keeping the wrong socket alive
and applying the wrong source's config/skin/cron changes.
Thread connectionId through consumption using the existing composite
backendScopeKey helper:
- session-states records each registry-tagged event's (connectionId,
profile) scope per runtime session; liveSessionScopes() projects the
busy/needs-input ones as composite keys for the gateway keep-set.
- recomputeKeptGateways (use-gateway-boot) seeds the keep-set with
those scopes; pruneSecondaryGateways matches registry-scoped entries
ONLY on their composite key, while local entries keep matching bare
profile names (single-source path unchanged).
- gateway-event's 'from the active profile' gates now compare the
event's composite scope against the active gateway's connection via
the new activeGatewayConnectionId(); untagged local/primary events
behave byte-identically.
Display-only surfaces that already use roster handles are untouched.
* style(desktop): order @hermes/shared import before nanostores (perfectionist/sort-imports)
* fmt(js): `npm run fix` on merge (#87839)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fmt(js): `npm run fix` on merge (#87844)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(agent): trim background_review to the enabled switch
Follow-up to #87400: drop the max_iterations and prompt_file knobs from
auxiliary.background_review. The aux model routing (provider/model/
base_url/...) predates #87400 and stays; the enabled switch and the
usage telemetry stay. The fork's iteration budget returns to the
historical hardcoded 16.
* fix(update): reload _subprocess_compat and dashboard_procs after git pull
hermes update runs in the PRE-pull Python process. After git pull updates
source files on disk, modules already in sys.modules still hold the OLD
code. The existing _reload_config_modules() reloaded only config modules,
but the post-update dashboard cleanup path (_finish_dashboard_update_cleanup
-> _scan_dashboard_processes) imports hermes_cli._subprocess_compat lazily;
a new symbol added there (e.g. bounded_probe_run) is invisible to the
cached module object, causing ImportError during the cleanup step.
Extend the reload list to include hermes_cli._subprocess_compat and
hermes_cli.dashboard_procs so the cleanup uses freshly-pulled code.
* fix(update): reload process-scan modules at the dashboard-cleanup entry point
Widen PR #87757 to cover the ZIP path: _update_via_zip() also calls
_finish_dashboard_update_cleanup() but never runs _reload_config_modules,
so the Windows git-broken fallback would still crash with the same
ImportError (cannot import name 'bounded_probe_run' from the stale cached
hermes_cli._subprocess_compat).
- new _reload_process_scan_modules() called inside
_finish_dashboard_update_cleanup itself, so every current and future
call site is covered; reloads dependency-first
(_subprocess_compat, then dashboard_procs)
- reload failures log at warning (a miss surfaces seconds later as an
ImportError in the same process)
- regression tests: reload-before-kill ordering, node-failure skip,
stale-module symbol restoration (the exact #87134 boundary state),
nonfatal reload failure, and the #87757 reload-list contract
* chore: release v0.20.2 (2026.8.16)
* fix(tui): modified Enter and bare LF insert a newline in the composer across IDE and macOS terminals (#87854)
* fix(tui): send atomic CSI u for modified Enter in IDE terminals
VS Code/Cursor/Windsurf terminals bound Shift/Ctrl/Cmd+Enter to the
legacy \\r\n sequence, which Ink's parse-keypress split into a
backslash keypress plus a plain Return — inserting a stray backslash and
submitting instead of adding a newline. Emit Kitty CSI u sequences that
encode the modifier atomically, and migrate keybindings users already
have on disk.
Co-authored-by: yatesjalex <yatesjalex@users.noreply.github.com>
* fix(tui): treat a bare LF as a newline in macOS composer terminals
Terminals that can't send a distinct Shift+Enter collapse a modified
Enter / Ctrl+J down to a bare LF. shouldPreserveCtrlJNewline() already
handles the env-detectable cases (SSH, Windows Terminal, Ghostty, WSL),
but plain macOS terminals (Terminal.app, iTerm2 defaults) do the same and
aren't env-detectable, leaving no keyboard-driven newline there. Fold the
return-key decision into shouldInsertNewlineOnReturn() and accept a bare
LF as a multiline fallback on macOS too, keeping CR as submit everywhere.
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
---------
Co-authored-by: yatesjalex <yatesjalex@users.noreply.github.com>
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
* fix(state): classify structural DB corruption as its own persistence cause
'database disk image is malformed' contains the word 'disk', so
classify_persistence_error bucketed SQLITE_CORRUPT / SQLITE_NOTADB
failures as 'disk' and the turn-completion explainer told users to
free disk space for a structurally damaged state.db (the #77386-family
misdiagnosis, reproduced in the v0.20.0 malformed-DB incident report).
- hermes_state: new 'corrupt' bucket in PERSISTENCE_ERROR_CAUSES,
matched via _DB_CORRUPTION_MARKERS BEFORE the locked/disk buckets
- run_agent: explainer text for 'corrupt' points at hermes doctor and
explicitly says freeing space will not help
- cron explainer-variant suppression picks the new variant up
automatically (it iterates PERSISTENCE_ERROR_CAUSES)
* fix(openviking): strip PYTHONPATH from autostarted server child env (#78153)
(cherry picked from commit 7afd99155667cde480c0ab4ee31e242dab849d40)
* fix(openviking): read .env BOM-tolerantly when rewriting credentials
f1ea4a56c ("cover the remaining setup-time .env reads with utf-8-sig",
following 75afc47ba for mem0/hindsight) swept this class; openviking's
_write_env_vars was missed and still reads with strict utf-8.
It copies every existing line through on each update, so the read decides
whether a credential update lands:
BOM'd .env -> the first key never matches, so the old line survives and
the new value is appended as a duplicate. .env loaders keep
the first occurrence, so the update silently does nothing.
cp1252 .env -> UnicodeDecodeError aborts setup outright.
Read exactly like the canonical hermes_cli/config.py save_env_value
(utf-8-sig + errors="replace"). A plain UTF-8 file rewrites byte-identically.
Scope: hermes_cli/memory_setup.py has the same read but is already the
subject of #30281 / #60587, so it is left alone here.
(cherry picked from commit 175c6852c2c255b3219575b5de0b1b70f1f0efcb)
* fix(openviking): preserve non-UTF-8 env bytes on update
* docs(openviking): correct environment handling explanations
Clarify that the Desktop backend can add Hermes venv packages to PYTHONPATH and that current .env loaders use the last duplicate value.
* Revert "fix(agent): preserve local reasoning timeout opt-out"
This reverts commit 26b2b475935d5f5f369142fe1648cf5c95e7b056.
* Revert "fix(agent): harden canonical tool call deduplication"
This reverts commit 8fc4189edd23dde055232cc07ea14d1d525e44ee.
* fix(update): restart hermes-serve systemd units alongside gateways
hermes update discovered and restarted hermes-gateway* systemd units but
never looked for hermes-serve* — the Desktop app's backend — so it kept
running stale pre-update code until the user restarted it by hand (#83438).
Extend the systemd unit discovery/restart loop to also match hermes-serve*
units. They don't wire SIGUSR1 to a graceful drain (only gateway/run.py
does), so restart eligibility for the graceful path is now gated on unit
name via a small, directly-tested helper; hermes-serve units fall straight
to the existing blunt systemctl restart path, matching the workaround the
issue already documents.
* fix(update): tighten hermes-serve unit gate, dedupe fleet/cleanup restarts
Review on #83595 flagged two service-lifecycle gaps in the hermes-serve
restart support:
- The unit-name gate accepted anything starting with "hermes-serve",
which also matched the unrelated hermes-server.service. Require the
exact base unit or the hyphenated profile family instead.
- The fleet-restart loop and _finish_dashboard_update_cleanup() could
both restart the same hermes-serve unit — the loop restarts it
directly, then cleanup's PID scan finds the fresh process and
restarts its owning unit again. Thread the fleet loop's restarted
unit names through to _kill_stale_dashboard_processes() so it skips
units already handled.
* fix(update): tighten gateway-side unit gates to exact/hyphenated shape
Mirror the strict unit-name shape from the hermes-serve gate (review on
PR #83595) on the gateway side too: the discovery gate and the SIGUSR1
eligibility helper now accept only `hermes-gateway.service` or the
`hermes-gateway-<profile>` family, so a near-prefix unit like
`hermes-gatewayd.service` can neither enter the restart path nor be sent
a SIGUSR1 it does not handle.
* fix(desktop): ignore stale remote connection attempts
* chore: map contributor email for xkam7ar
* fix(apps): dial primary sleep/wake reconnect at window backend not active profile
* fix(desktop): scope pluginSocket's connection to the active profile
pluginSocket (hermes.ts) is documented as "the live twin of pluginRest,
scoped the same way", but it calls window.hermesDesktop.getConnection()
with no profile argument, while pluginRest passes the active profile via
profileScoped(). getConnection's IPC handler (ensureBackend in
electron/main.ts) falls back to the primary profile whenever the profile
argument is empty, so an unscoped call always resolves to the primary
profile's backend regardless of which profile is actually active.
For a plugin used from a non-primary profile (e.g. kanban), this means REST
calls go to the correct pooled backend while the plugin's WebSocket silently
connects to the wrong one — a multi-profile user sees one profile's data
with another profile's live events.
Fix (adapted to the post-#87600 registry-agent store shape during salvage):
resolve the plugin socket's connection through the same (connectionId,
profile) source of truth ensureGatewayProfile/ensureGatewayAgent maintain
for $connection — store/gateway's setActive now pushes the active scope's
registry connection id into the hermes module (setApiRequestConnection,
the no-store-import twin of setApiRequestProfile), and pluginSocket
resolves via getConnectionFor for registry-agent scopes and
getConnection(profile) for the local pool. The plugin socket therefore
follows registry-agent activations too, not just profile switches.
voice-playback.ts's resolveSpeakStreamUrl had the same gap originally, but
main has since fixed it independently (via the getApiRequestProfile()
getter rather than direct store access) — dropped from this PR as
redundant, keeping only the still-open pluginSocket gap.
Co-authored-by: Hermes Agent <hermes@nousresearch.com>
* fmt(js): `npm run fix` on merge (#87880)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(image_gen): disable default-on upscaling everywhere — opt-in only
The Aug 8 default-on upscaling policy (66ea4e686) chained the Clarity
Upscaler after every sub-2MP generation. Clarity is an SD1.5 creative
tile-diffusion enhancer (creativity 0.35, "masterpiece" prompt prefix) —
it redraws content, which degraded output on 100% of generations for
models like GPT Image 2 and Ideogram whose value is precise text
rendering, CJK, and photorealistic detail.
Policy now: no model upscales by default, on FAL or Krea. The `upscale`
tool param remains as a per-call opt-in (`upscale: true`); explicit
requests still chain Clarity (FAL) / Krea Enhance as before.
- FAL catalog: all 17 default-on entries flipped to upscale=False
- Krea plugin: medium + medium-turbo per-model defaults flipped off
- Tool schema: upscale param described as opt-in with a fidelity warning
- Tests updated: catalog invariant now pins all-off; default-on cases
now assert no upscaler call
- Docs (en + zh) updated to the opt-in policy
* fix: make every tool interruptible — sequential executor abandons on user interrupt
The sequential tool path only noticed a user interrupt after the running
tool returned: with the deadline disabled it ran the tool inline (fully
blocking), and with a deadline it waited in 5s slices without ever
checking agent._interrupt_requested. Any tool without cooperative
is_interrupted() polling (image_generate, tts, transcription, skills
sync, ...) held the whole turn hostage — the reported symptom was a
redirect queued ~40s behind a FAL image generation + upscale pass.
Executor backstop (class fix, covers ALL tools):
- _run_sequential_tool_execution_middleware always dispatches on the
daemon worker (timeout None no longer means inline blocking) and polls
the interrupt flag every 1s.
- On interrupt: 3s cooperative grace (mirrors the concurrent path), then
synthesize a cancelled tool result (_ToolCancelledResult), emit the
terminal post_tool_call with status=cancelled, and abandon the worker.
- _ToolCancelledResult suppresses downstream post-hook double emission
exactly like _ToolTimeoutResult, so an abandoned worker finishing late
cannot report success for a cancelled call.
- clarify (interactive, _NEVER_PARALLEL_TOOLS) keeps the inline path —
it owns its own human wait.
Cooperative layer in the reported offender:
- image_generation_tool: blind handler.get() (generation + Clarity
upscale) replaced with _wait_fal_result(), which polls is_interrupted()
in 0.5s slices and raises ImageGenerationInterrupted immediately.
- _upscale_image propagates the interrupt instead of swallowing it into
the "upscale failed, use original" fallback.
Message alternation is preserved: the cancelled result is a normal tool
result for the call_id. Sabotage-verified: with the old wait loop
restored, the new tests fail (tool blocks full runtime); with the fix
they pass in ~4s.
* feat(computer-use): support Cua Driver 0.20 runtime contracts
* fix(computer-use): reconcile existing cua-driver installs
* fix(computer-use): enforce existing-profile grant, unblock the opt-in
Live-testing the Cua Driver 0.20 convergence on Windows 11 (session 2,
cua-driver 0.20.0) surfaced three defects in the existing-profile browser
path and in install status.
1. The config grant was silently nullified by an approval bypass.
`--yolo` / `-z` map onto a private unrestricted daemon, which answers every
browser_prepare. Because the host delegated the entire existing-profile
decision to the driver, that bypass also nullified
`computer_use.grant_existing_profile: false`: a plain `hermes -z` attached
to the user's real Chrome profile and read live page content over CDP, with
the driver reporting it as "the approved existing Chromium profile". It was
never approved.
An approval bypass is consent to skip prompts, not consent to read an
existing profile's pages, cookies, and storage. CuaTypedBrowserRoute.prepare
now enforces the key itself, regardless of permission mode. bounded stays
exempt - its reviewed capability manifest is the authorization boundary.
The authorization inputs are resolved in the backend from config and the
backend's immutable mode, never from model-supplied kwargs.
2. The grant, once set, still could not be used.
With `grant_existing_profile: true` the runtime is launched
`--grant existing-profile` correctly, but cua_browser_prepare then hit a
runtime approval prompt anyway - re-asking the user to authorize what the
config already authorized, and making the documented opt-in unusable on any
non-interactive run, where the prompt has nobody to answer it and the call
dies on approval timeout. The durable, file-backed grant now stands in for
that prompt. Scope is narrow: only the existing-profile prepare, only when
the grant is present; isolated launches still prompt and any resolution
failure falls closed to prompting.
3. `computer-use status` hid a custom override and spliced its output.
With HERMES_CUA_DRIVER_CMD pointed at cmd.exe, status printed the child's
multi-line banner and prompt inside the one-line version field, never
mentioned the override, and advised `hermes computer-use install` - which
install itself (correctly) refuses to run against an overridden path. It now
names the override and mirrors install's update-or-unset guidance, and
version output is reduced to one bounded line.
Verified on the reported host: `-z` existing-profile attach now refuses and
names the key; `grant: true` no longer prompts (33s vs a 300s approval
timeout); status names the override and prints one line. No change to the
reconciliation path - driver SHA256 unchanged end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): make the typed-browser bind/snapshot split discoverable
`cua_browser_state` has two branches, chosen implicitly: any call carrying
pid or window_id is a *binding* (browser_route.py:252), anything else is a
*snapshot*. A binding clears session state, mints fresh tab_ids, returns
binding metadata with no page content, and sets verification_required.
Nothing in the response says that. A caller that keeps passing pid/window_id
- the natural reading of "bind to this window, then read it" - re-binds
forever: the tab_id it just received is unbound by the next bind, so every
cua_browser_navigate comes back browser_verification_required, and the
refusal ("take a fresh snapshot") points at the same call that just re-bound.
Observed live as 11 consecutive refused navigates before the model gave up
and fell back to foreground SendInput on the address bar.
The same confusion silently swallowed include_screenshot: both calls that
requested one were bindings, which carry no page content, so the flag had
nothing to attach to and was dropped without comment.
A binding response now reports snapshot_required, next_step
(fresh_browser_state, matching the existing token convention) and a hint
naming the exact next call; requesting a screenshot on a binding reports
screenshot_deferred instead of dropping it. The verification refusal now
says to call cua_browser_state WITHOUT pid/window_id and why re-sending them
does not help. The schema documents that include_screenshot applies to
snapshots.
Behavior of the bind and snapshot branches themselves is unchanged - this is
purely about making the split legible to the caller.
Unit-tested. Not verified end to end on the reporting host: the driver
refuses the bind upstream there (`browser_requires_setup: no owned DevTools
endpoint`, and it does not accept a user-launched --remote-debugging-port),
so the typed route never reaches this branch. That attach failure is a
separate cua-driver issue.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): keep a v3 capability manifest on approval-bypassed runs
`--yolo` / `-z` route the session onto a private embedded daemon in
`unrestricted` mode. That daemon was constructed without the configured
capability manifest, and the serve command only attached
`--capability-manifest` when the mode was exactly `bounded`. So the moment a
run was bypassed, the user's declared ceiling was dropped:
without -z: --permission-mode bounded --capability-manifest ...
with -z: --permission-mode unrestricted --dangerously-bypass-approvals
No manifest, no warning. The most carefully configured run - a reviewed
ceiling, written by hand - became the least constrained one, silently, and
it failed open.
That was never a driver limitation. cua-driver documents the manifest as a
ceiling across modes ("A manifest can narrow a profile but never widen it";
its own authorization table calls it `optional_capability_manifest_ceiling`),
and accepts it alongside `--permission-mode unrestricted`.
The forwarding is version-aware, because the two manifest schemas differ
(cua-driver session_manifest.rs):
* v1/v2 are legacy and must declare `mode: bounded`. Handing one to an
unrestricted runtime aborts startup with "legacy capability manifest mode
must be bounded", so a naive forward would turn a working session into a
hard failure. These are forwarded for bounded only, and a warning names
the migration when one cannot apply.
* v3 must not declare a mode. It is the mode-independent ceiling, and it now
rides along with unrestricted.
Unreadable or unparseable manifests are not forwarded outside bounded, on
the same fail-safe reasoning; bounded still forwards unconditionally and
lets the driver be the authority there.
Verified against cua-driver 0.20.0 on Windows. Launch args now carry
`--permission-mode unrestricted --dangerously-bypass-approvals
--capability-manifest <v3> --approve-capability-manifest`, and the ceiling
is enforced in the bypassed run - a tool outside the manifest is refused
("outside the capability manifest for this session ... blocked as a
protected resource") where the same config previously ran unbounded. A
legacy manifest was confirmed to abort driver startup when forwarded, which
is what the version gate prevents.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): warn when an approval bypass widens the driver mode
`--yolo` / `-z` read as "don't prompt me", but they also swap computer_use
onto a private `unrestricted` daemon, dropping the ceilings the configured
mode would have applied. Nothing said so. A script picks up `-z` for quiet
output and loses its limits as a side effect, and the only trace is a driver
process nobody inspects.
The mapping itself stays. It is deliberate, and `unrestricted` is reachable
no other way: it is intentionally not a config value so a stale config line
can never silently bypass approvals (see `_cua_configured_permission_mode`).
Removing the mapping would delete the capability rather than fix it, and
splitting it onto a second CLI flag was declined to avoid growing the
surface.
So the widening is now stated instead: one warning per session naming the
configured mode it left, what stopped applying, and the two ways to keep a
ceiling - drop the bypass flag, or declare a version-3 capability manifest,
which now rides along with unrestricted as of the previous commit.
Once per session, not per dispatch: the resolver runs on every tool call.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): report unusable driver exit status
(cherry picked from commit 8bda6191ca548d65648263dfe30d2d18950a6e60)
* fix(computer-use): preserve missing driver overrides
* fix(computer-use): verify Windows driver repair
* fix(computer-use): align browser guidance and screenshots
* fix(desktop): keep the local pack out of electron-builder's publish path
`hermes desktop` runs `npm run pack` through _npm_lifecycle_env(), which
sets CI=1. electron-builder 26 reads that as an implicit publish request
(`onTagOrDraft`) when --publish is absent, so a local --dir build enters
publish resolution it has no business being in.
Pin `--publish never` on the pack script. This is also what electron-builder
asks for directly -- the implicit CI behavior is removed in v27.
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: fangliquanflq <fangliquanflq@users.noreply.github.com>
* fix(desktop): declare the repository so publish resolution can succeed
With a GH_TOKEN/GITHUB_TOKEN in the environment, electron-builder auto-selects
the github provider and resolves owner/repo from the repository field, falling
back to reading <projectDir>/.git/config. projectDir is apps/desktop, which has
no .git of its own, and app-builder-lib does not walk up to the workspace root
-- so resolution returned null and threw "Cannot detect repository by
.git/config".
On Linux this fires from onAfterPack for a plain `dir` target: the darwin and
Windows branches return early for non-installer targets, Linux has no such
guard. That is why the same build worked elsewhere.
--publish never keeps `pack` from reaching this at all, but `dist:*` and
test-desktop.mjs still resolve publish config on a machine with a token, so
declare the field too.
Tests call the real app-builder-lib resolver rather than asserting on the text
of package.json, so they track electron-builder's behavior instead of our
formatting.
Co-authored-by: airo7 <airo7@users.noreply.github.com>
Co-authored-by: frankmendes1979 <frankmendes1979@users.noreply.github.com>
* fix(computer-use): auto-repair an installed driver that fails the runtime contract
A same-day version-floor bump (0.20 runtime contract) left every install
with an older cua-driver hard-failing on all computer_use calls: the
start() gate fails closed, while the `hermes update` refresh defers to the
driver's own check-update verb — whose ~20h cache routinely answers "no
update available" right after we raise the floor. Hermes knew it required
0.20+ but never acted on that knowledge.
Two changes:
- tools_config.install_cua_driver(): a contract-failed installed driver is
repaired on the upgrade=True path too (previously only upgrade=False).
The contract failure itself is the confirmation, so the
require_confirmed_update gate and the check-update short-circuit are
bypassed for repairs — an indeterminate or stale-cached check can no
longer pin users on an unusable driver.
- cua_backend.CuaDriverBackend.start(): when the contract gate fails on an
installed binary, attempt one automatic repair per process via the
standard install path, then re-probe. HERMES_CUA_DRIVER_CMD overrides
are never repaired (explicit override is authoritative even when broken)
and a missing binary still just reports the install hint. A failing
installer can't loop: the second start() surfaces the original error.
Tests: contract-repair coverage in test_computer_use.py (auto-repair
success, failed repair surfaces the original error, once-per-process
guard, override never repaired, missing binary never repaired) and
test_install_cua_driver.py (incompatible driver repairs despite an
indeterminate check-update, check-update not consulted). All new tests
verified to fail against the unfixed source (sabotage run).
* docs(computer-use): note driver contract auto-repair at update and runtime
The runtime-contract repair now also runs during hermes update and once
per session at the first computer_use call (PR #87923); the docs only
mentioned setup and toolset enablement.
* feat: raise Codex OAuth context to live-verified 350K for gpt-5.6 family and gpt-5.4
The Codex /models catalog advertises 272K for the gpt-5.6 (sol/terra/luna)
and gpt-5.4 slugs, but the backend actually accepts ~371K input tokens
(verified live against chatgpt.com/backend-api/codex/responses, Aug 16 2026:
~371K completed OK on all four slugs; ~382K+ rejected with
context_length_exceeded). 350K keeps ~22K margin under the observed ~372K
enforcement.
The bump applies ONLY when the resolved value is exactly the known-stale
272,000 advertisement — any other advertised value (higher or lower) is
trusted as a real server-side change, so a future catalog correction
deactivates the override automatically. gpt-5.5 and gpt-5.4-mini both
genuinely enforce 272K (rejected 360K live) and are excluded.
* fix(tui): restore Alt+Enter for newlines (#87066)
* fix(tui): restore Alt+Enter for newlines
Restore Alt+Enter support for inserting a new line in the TUI after the behavior was lost during newer input-handling updates.
Legacy terminals encode Alt+Enter as ESC followed by carriage return. Preserve those bytes as a single tokenizer sequence and parse the result as Return with the Meta modifier so TextInput inserts a newline instead of submitting.
Keep plain CR and LF mapped to unmodified Return, and cover the legacy ESC+CR sequence with a regression test.
* fix(tui): scope legacy Alt+Enter tokenization
* feat(desktop): expose connection-aware plugin routing
* fix(desktop): report remote plugin target profiles
* fix(desktop): route plugin profiles through registry
* fix(desktop): harden plugin route lifecycle
* fix(desktop): preserve registry route identity
* chore: add contributor email mapping for addelh
* fix(desktop): scope session/pin lists per connection across windows
Multiple Desktop windows share one renderer origin (one localStorage
area) while each window can be connected to a DIFFERENT gateway. The
sidebar pin set (hermes.desktop.pinnedSessions), the manual session
order, and the remembered last-session/route navigation keys were all
persisted under single global (or profile-only) keys, so two windows on
different gateways read and reconciled the same lists: pin-sync's
pullRemotePins() in one window adopted/dropped pins belonging to the
other window's backend, producing the overlapping mixed PINNED/SESSIONS
lists reported after the v0.19.1 update relaunch.
Introduce a connection-scope persistence layer (connectionScopedAtom in
src/lib/connection-scoped.ts): the local connection keeps the bare
legacy key (byte-identical for single-backend users, same contract as
backendScopeKey), while remote connections persist under
`<key>.remote.<encoded baseUrl>.<encoded profile>` — the shape
workspaceCwdKey already established. setConnection() rescopes every
scoped atom when the window's connection changes (null descriptors keep
the current scope, as with syncCronModelImpactConnection), and pin-sync
resets its mirrored/pending/unconfirmed bookkeeping on rescope so a
reconcile never PATCHes one gateway's pins to another.
Legacy globally-keyed values are deliberately not migrated into remote
scopes: ownership of rows accumulated by every window is unknowable
(the #67709 precedent), and backend-mirrored pins self-heal from the
gateway's own `pinned` rows.
Fixes #77318
* fix(desktop): keep profile rail alive across remote/Cloud connection switches
A connection/mode apply (soft re-home) moves /api/profiles routing to a new
backend, but nothing deterministically re-fetched the rail's $profiles list
and a stale in-flight response from the previous backend could land last and
collapse the rail to Home (#85731).
- store/profile: epoch-guard refreshProfiles/refreshActiveProfile so a
response fetched against the previous backend never writes the shared cache
(invalidateProfileListFetches), and bump the epoch on live profile swaps.
- store/gateway-switch: strand in-flight profile-list fetches in the same
wipe every connection/mode apply funnels through.
- use-gateway-boot: explicitly re-pull the active profile + list from the NEW
backend during softSwitch, best-effort like its sibling fetches.
Fixes #85731
* fix(desktop): read cron run-history from the owning gateway
When Hermes Desktop works against a REGISTERED gateway connection, cron
jobs execute on that gateway and persist their run sessions in the
gateway's state.db. But every REST call in the app — the cron surface
included — carried only `profile`, so `hermes:api` routed it through the
local profile pool and `_list_cron_job_runs_sync` read a local state.db
with zero `source='cron'` rows. Every job showed "No runs yet" while the
same endpoint on the gateway returned the real runs (#87882).
Fix at the routing seam:
- HermesApiRequest gains an optional `connectionId`. The renderer's cron
helpers (list/get/runs/delivery-targets/create/update/pause/resume/
trigger/delete/blueprints) now tag the active registry connection via a
new connectionScoped() twin of profileScoped(), fed from the same
setApiRequestConnection seam store/gateway already maintains for the
plugin socket.
- The hermes:api main-process handler resolves a tagged request through
ensureRegistryBackend — the SAME pool the job list and WS traffic use —
instead of the legacy profile route. Shared remote/cloud hosts (one
gateway, many profiles) get the path scoped with ?profile= via the new
pathWithProfileScope helper, factored out of pathWithGlobalRemoteProfile.
- '' / 'local' / absent connectionId keep the byte-identical v1 route, so
single-source and connection-config-remote users are unaffected.
This covers the run-history panel, the sidebar cron peek, and every other
cron surface in one place, since they all funnel through the same helpers.
Fixes #87882
* fmt(js): `npm run fix` on merge (#88014)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* test(desktop): pin steered-turn transcript order end-to-end
A steered turn's contract — pre-steer output above the correction bubble,
post-steer output and the settled reply below it — was fixed across
several PRs (#73793/#83151 class, settle fixes) but only covered piecewise:
the mid-turn insert as a unit, the settle math as a unit. Nothing drove the
real stream reducer through a whole steered turn, and nothing asserted the
durable-row hydration renders the same order after reload.
Two suites close that:
- steer-arrival-order: full event sequences through useMessageStream's real
handler + the real optimistic insert — single steer with tool activity,
steer racing message.complete, double steer in one turn.
- steered-turn-hydration-order: toChatMessages over persisted row shapes
copied from a real state.db steered turn, including a tool result that
lands after the correction row.
* test(desktop): harden steer-order suite against fake-timer id collisions
Review follow-ups: steer ids now come from a monotonic counter instead of
Date.now() (frozen under fake timers — two steers without a clock advance
would have collided), and the settle-above assertion documents its
load-bearing sealed-bubble assumption.
* test(desktop): steer suite drives the real redirectPrompt path; hydration fixture carries durable row shape
The live suite previously called appendMidTurnUserMessage directly, leaving
redirectPrompt's appendAfterActiveReply guard — the production decision of
WHERE a correction lands — outside the harness. Both hooks now mount together
sharing one state map, exactly as the desktop wires them, so a regression in
the caller (not just the insert) goes red. Verified by mutation: disabling the
guard fails 2/4.
Also covers the rejected-redirect path: a not_running response discards the
optimistic bubble instead of stranding a correction the model never saw.
The hydration fixture now carries the durable row shape the client actually
receives (row_id, reasoning, provider call_id/response_item_id on tool_calls)
instead of a hand-simplified echo, so the 'mirrors real state.db rows' claim
is honest. The fake-timer steer id counter is gone with the local insert —
ids come from redirectPrompt itself.
* fmt(js): `npm run fix` on merge (#88016)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* feat(desktop): bundle Bot Mode (hermes-bots) as a built-in, default-on plugin
Adopts the Hermes-Bot-Mode desktop plugin (NousResearch/Hermes-Bot-Mode)
into apps/desktop/src/plugins/hermes-bots/, registered by the bundled
vite glob and ON by default. It stays a pure @hermes/plugin-sdk consumer
in plain-ESM plugin.js form; users disable it live in Settings > Plugins.
- contrib/plugins.ts: bundled glob accepts plugin.js entries
- contrib/runtime-loader.ts: a disk/runtime copy of an id that ships
bundled is skipped (standalone installs predating adoption cannot
double-register)
- package.json: check:test:plugins runs the plugin's node:test suite in
CI (138 tests)
- source: Hermes-Bot-Mode @ c19baba, incl. today's #107/#103/#99 merges
* feat(agent): core Bot Mode teammate protocol — stable-tier prompt section
Replaces the plugin-side SOUL.md protocol append: on Bot-Mode-managed
installs (any profile carrying ui_meta['hermes-bots']) the prompt builder
injects the "Messaging other agents" section into every session of every
profile — including headless `hermes -p <bot> chat` sessions a teammate
starts — so bot handoffs work without mutating user-authored SOUL files.
- tools/bot_mode_probe.py: silent-when-unmanaged probe, cached per
(process, home), keyed off the agent's OWN home (not ambient
HERMES_HOME); silent when SOUL.md already carries the legacy section
- agent/system_prompt.py + agent_init.py + config_defaults.py: wired as
agent.bot_mode_protocol (default True), stable tier, byte-stable
across rebuilds (E2E-verified against the real build_system_prompt)
- tui_gateway profiles.list gains bot_mode_protocol capability flag;
the bundled plugin gates ALL SOUL protocol writes on it (backfill,
composeSoul, Edit save) — older gateways keep the SOUL-append path
- overhead: ~916 bytes, only on Bot-Mode installs; zero elsewhere
Supersedes the SOUL backfill half of Hermes-Bot-Mode#99 (credit
@kaduxo — the handle fix, `hermes profile list` correction, and
idempotent-append guards from that PR ship in the bundled plugin).
* fix: track bundled plugin.js sources past the tsc-artifact gitignore
apps/desktop/src/**/*.js is gitignored (stale tsc output shadows .tsx),
which silently dropped the hermes-bots plugin.js from the adoption
commit — tests shipped, source didn't, CI ENOENT'd. Negate the pattern
for src/plugins/*/plugin.js: adopted plain-ESM plugins have no .tsx
sibling, so the shadow hazard cannot apply.
* fix(agent): scope the Bot Mode protocol section to canonical Bot Chat sessions
Per review: the protocol belongs only in official Bot Mode interactions,
not every session on a managed install. The prompt builder now injects
the section only when the agent's session row is titled "Bot Chat"
(BOT_CHAT_TITLE, matching the desktop's createCanonicalChat pin and the
`hermes -p <bot> chat -c "Bot Chat"` resume target). Regular sessions
never carry it; the desktop composer middleware owns @mention sends.
Title is read once at first prompt build and the rendered prompt is
cached + DB-restored — cache-safe. E2E against the real AIAgent +
SessionDB: absent in an untitled session, present in Bot Chat,
byte-stable across rebuilds, absent after retitle, absent with the
flag off. Overhead unchanged (~916B, Bot Chat sessions only).
* fix(hermes-bots): composeSoul honors the bot_mode_protocol capability
Found in live desktop E2E: the generated-identity path of composeSoul
still appended the protocol section even when the backend injects it
into the system prompt. New agents now get a clean identity-only SOUL
against capable backends; older gateways keep the append. Covered in
the capability-suppression test.
* fix(agent): Bot Chat gate reads a session-title hint before the DB
Live desktop E2E caught a write-ordering bug the automated E2E missed:
tui_gateway applies pending_title to state.db AFTER the first turn, but
the system prompt builds at turn START — the DB-title gate saw nothing
and the Bot Chat was cached protocol-less forever. The gateway now
hands the agent its intended title at construction and the gate checks
the hint first, DB second (CLI/messaging-gateway paths unchanged).
Live-verified on the running desktop: fresh bot's Bot Chat persisted
with the protocol section, handle, and roster in its system prompt;
regular sessions and SOUL.md untouched.
* feat(agent): capability-refresh + timeless prompts for eternal Bot Chat sessions
Bot Chats break the "new sessions come often" assumption behind
build-once system prompts: capability edits used to sit invisible until
/new or compression, and the frozen birth date became misinformation.
- tools/bot_mode_probe.py: capability_fingerprint() hashes the profile's
capability surface (disabled skills, toolset pins, MCP config, SOUL.md,
installed skills, Bot-Mode roster); Bot Chat prompts embed the 12-hex
epoch stamp
- agent/conversation_loop.py restore path: stored Bot Chat prompt whose
epoch mismatches disk → ONE rebuild (through a cleared skills-prompt
cache so new installs appear), persisted so the next turn reuses the
new bytes verbatim. Prompts without a stamp — every non-Bot-Chat
session — never take the branch; probe failure fails closed to reuse
- agent/system_prompt.py: Bot Chat prompts are timeless — the
"Conversation started:" date is dropped (timezone kept); no ticking
fields in an eternal session
- tui_gateway: _sync_bot_capabilities at turn start rebuilds the live
agent (tool definitions are construction-baked) when the fingerprint
moves, same session id/history, with a user-visible notice
Cache stance: this is the /model exception applied to capabilities — a
loud, user-initiated, once-per-change prefix break. Unchanged state
hashes identically and stored bytes are reused verbatim (E2E-proven).
Validation: 9 probe unit tests incl. per-axis fingerprint changes;
E2E v3 against the real restore path (fresh build → verbatim reuse →
skill install → single refresh w/ new skill in index → verbatim reuse;
regular sessions dated, unstamped, never refreshed); tests/agent/
4647/4647.
* feat(agent): one-time protocol upgrade for legacy Bot Chat sessions
Bot Chats created before the epoch mechanism persisted prompts with no
protocol section and no stamp — the staleness check only fires on
stamped prompts, so pre-existing bots would never learn to message
teammates. stored_bot_chat_prompt_needs_upgrade() migrates them: one
rebuild, title-gated to Bot Chat, only when the probe would actually
emit a section (SOUL-append legacies and unmanaged installs are left
alone — rebuilding those would loop). The rebuilt prompt carries the
stamp, so the upgrade can never re-fire.
E2E v3b through the real restore path: legacy Bot Chat upgraded once
then verbatim-reused; legacy regular sessions byte-untouched.
tests/agent/ 4648/4648.
* fix: capability fingerprint reads config via the canonical loader
The config-read guard (test_config_read_guard) correctly flagged the
probe's raw yaml.safe_load of config.yaml — raw reads miss the managed
overlay, env expansion, and normalization. Use load_config_readonly()
under a scoped HERMES_HOME override instead. E2E v3/v3b and the guard
both green.
* feat: sync bundled Bot Mode with multi-source roster (Hermes-Bot-Mode#68)
Pulls the multi-source roster into the bundled plugin: profiles.list rows
from the active gateway are merged with the host.agents() union roster
(hermes-agent #86875), so the Bots panel shows agents from every registered
Desktop connection with @name-device handles for duplicates. Feature-detected
and best-effort — an older Desktop build or roster failure leaves the
single-source list untouched.
Adapted for the bundle:
- useRoster queryFn combines the bot_mode_protocol capability read (which
landed after #68 was cut) with the multi-source merge
- multi-source-roster tests updated for the namespace SDK import harness
- soul-protocol-backfill anchor widened for the new botHandle(name, bot)
signature
Plugin suite: 143/143.
* feat: raise Codex OAuth context to 900K for gpt-5.6 family and gpt-5.4 (subscription 1M rollout)
OpenAI enabled the large-context window for ChatGPT-subscription Codex
accounts (announced by @thsottiaux Aug 16 2026; previously API-key-only).
Live re-probe the same day: 911,276 input tokens completed OK on
gpt-5.6-sol; ~925K+ rejected with context_length_exceeded (1.05M window
minus reserved output headroom). terra, luna, and gpt-5.4 all completed
900,026 tokens OK. The Codex catalog still advertises 272K, so the
stale-advertisement override from #87981 is the right lever — this just
raises its value 350K -> 900K.
gpt-5.5 and gpt-5.4-mini still enforce 272K live (rejected 500K) and
remain excluded. Override semantics unchanged: fires only on an
exactly-272,000 advertisement; any live catalog change is trusted
verbatim.
* fix(desktop): map SSH profile aliases in REST paths
* chore: map contributor email for attribution audit
* fmt(js): `npm run fix` on merge (#88079)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(cron): stop retry storms when the gateway is deliberately stopped (OOF-266)
Since the managed-cron redesign (#84339, v2026.8.13) the dashboard fire
webhook forwards fires to the gateway process and returns 503 when it is
unreachable so NAS/QStash retries. Correct for transient windows — but an
operator-STOPPED gateway can never be fixed by retrying: every fire on
every job burns the full scheduler retry budget, NAS converts each 503 to
a retryable 502, and the resulting storms page on-call for a non-incident
(OOF-266 and its five duplicate tickets; +93% relay callback failures as
the fleet adopted v2026.8.13).
Split the unreachable path by durable operator intent:
- desired_state == "stopped" (written only by the s6 lifecycle commands;
the same intent signal container-boot reconciliation trusts) -> drop
the fire with 200 + a structured log line, mirroring NAS's own
instance_stopped drop. Jobs are not lost: the Chronos provider
reconciles and re-arms every job on the next gateway start.
- Anything else (crash loop, scale-to-zero wake, restart, legacy state
file without desired_state) -> keep the retryable 503, now stamped
with Retry-After: 60 so a scheduler that honors it spaces retries
past the wake/restart window instead of exhausting them inside it.
The gateway's own pass-through 503s (draining) get the same hint.
The intent check fails open (any parse/resolution error -> retryable
path) and is only consulted when the gateway is actually unreachable, so
a stale state file can never shadow a live gateway.
* feat(state): support the context-manager protocol on SessionDB
A SessionDB handle cannot be released by dropping the last reference.
Once its background token writer starts, the instance pins ITSELF two
ways: the writer thread's target is a bound method, and
queue_token_counts registers atexit.register(_drain_token_queue_at_exit),
which only close() unregisters. A dropped-but-pinned handle keeps its
state.db/-wal/-shm descriptors for the life of the process, and __del__
never runs for it, so the existing safety net is dead code for exactly
the instances that leak.
That is why owning call sites are expected to close explicitly, in those
words, in the ownership comments in run_agent.py and
tui_gateway/methods_session.py. This adds the ergonomic half of that
contract so an owner can scope a handle and be exception-safe by
construction:
with SessionDB(path) as db:
db.append_message(...)
Purely additive. __enter__ returns self, __exit__ closes and returns
False so a caller's exception always propagates, and close() is already
idempotent, so a scope that closes early still exits cleanly. Nothing
changes for callers that already close directly.
Four regressions cover the scope closing the handle, __enter__ returning
the instance itself, the failure path closing while still propagating,
and an early close leaving the exit clean. They assert on the
sqlite_safe_read tracking registry rather than raw descriptor counts,
matching test_session_db_read_conn_pool.py, because SQLite's unix VFS
parks a closed descriptor on a per-inode reuse list and makes raw counts
lag the real connection count.
Refs #88033
* fix(state): release abandoned session database handles
* fix(state): avoid overlapping context manager change
* fix(gateway): ignore invalid managed Node directories
Signed-off-by: Shawn Wang <32839114+enwaiax@users.noreply.github.com>
* fix(gateway): accept CJK full-width punctuation as MEDIA path terminators
MEDIA_TAG_CLEANUP_RE (and MEDIA_EXTENSIONLESS_TAG_RE) only recognized
ASCII terminators after a MEDIA:<path> tag. Chinese-language agent
output naturally writes MEDIA:D:\...\zhibao.pdf（782.6 KB）or ...pdf：内容 —
the full-width punctuation failed the trailing lookahead and the
attachment was silently dropped (cron even reported 'delivered') (#88038).
Both lookaheads now accept a CJK full-width terminator set (（）〈〉《》：，。；
！？、curly quotes【】) alongside the ASCII set. The #68773 adjacent-tag
splitting guard is covered by a regression test.
* fix(skills): rescan skill commands cache when active profile changes
Switching Desktop profiles mid-session changes HERMES_HOME but not the
platform scope, so get_skill_commands() kept serving the previous
profile's skill list. A skill only available under the new profile then
looked like a cache miss to callers such as slash.exec, which fall
through to the slash_worker dead path (#88023).
* fix(gateway): scope slash.exec's skill-command check to the session's profile
Independent review of the prior commit found the cache-invalidation key
alone doesn't fix the reported #88023 dead path: slash.exec runs as a
_LONG_HANDLER on the pool with a copied context, and no binding of
_HERMES_HOME_OVERRIDE happens between the transport read and the handler
body, so get_skill_commands() there always fell back to the process-level
HERMES_HOME regardless of which profile's session issued the request.
Bind the session's own profile_home around the get_skill_commands() check,
mirroring the same bind/reset-in-finally pattern already used at every
other per-turn HERMES_HOME scoping site (e.g. server.py's prompt-turn and
system-prompt-rebuild paths). This makes the #88023 dead path actually
reachable by the fix instead of only exercising the cache primitive in
isolation.
* feat(desktop): add status bar reconnect for offline gateways
Expose the existing profile-aware gateway boot reconnect path through a
single-flight renderer action, and surface a Reconnect button in the
gateway status menu panel whenever the socket is not open. Repeated
clicks share one in-flight reconnect; failures surface through the
existing non-destructive notification UI. Localized copy for all
supported Desktop locales.
Salvaged from PR #80694 (net diff re-applied onto current main; panel
code lives in app/shell/gateway-menu-panel.tsx now).
* fix(desktop): self-heal dropped SSH/HTTP registered remote connections
A dropped registered remote connection (SSH or HTTP) never recovered on
its own: the next boot attempt failed with a transient transport error
("Could not verify the existing SSH backend", ERR_CONNECTION_RESET,
mint timeout), the failure was correctly NOT latched, but nothing ever
re-attempted the boot — the renderer's reconnect machinery only arms
after a completed boot. The app parked on "Desktop boot failed" until
the user manually deleted and re-entered the same connection details,
which merely forced the fresh bootstrap an automatic retry would have
performed (issue 82679, feature ask 80430).
Root causes and fixes:
- electron/backend-start-failure.ts: new isRetryableRemoteBootFailure()
predicate — a remote, non-reauth boot failure is transient and may be
retried; local failures and confirmed 401/403 rejections are not
(a missing capability differs from a transient failure).
- electron/main.ts: the boot-failure progress broadcast now carries
`retryable` (rides with `error` through updateBootProgress), and a
failed reuse probe against a cached SSH master tears the stale
master/tunnel down so the next attempt bootstraps fresh — exactly
what manual re-entry did.
- use-gateway-boot.ts: bounded self-heal loop for a failed boot whose
progress is marked retryable — up to 5 re-attempts with the same
full-jitter backoff as the socket reconnect loop (2s base, 15s cap).
Exhausted retries end in the real boot-failure recovery overlay,
never an infinite spinner. Reset on success and on soft switch;
timer cleared on unmount.
- store/boot.ts: resumeDesktopBootForRetry() re-arms the overlay with a
retry status while an automatic retry is in flight.
Secondaries already had full-jitter backoff (store/gateway.ts); this
closes the same class for the PRIMARY/registered-connection path.
Tests: predicate matrix (retryable vs reauth-latch mutually exclusive),
plus renderer hook tests proving a transient SSH failure self-heals on
the next attempt, retries are bounded (6 total dials then the recovery
overlay, no further attempts), and non-retryable failures never enter
the loop. Sabotage-verified (disabling either half fails 4 tests).
Fixes #82679
Fixes #80430
* feat(desktop): support remote gateway headers
* feat(desktop): carry remote gateway headers through the connections registry, test probes, and Settings UI
Completes PR #74468 (remote gateway headers for Cloudflare Access, #74466)
against the v2 multi-connection registry that landed after the PR was
authored, and closes the review blockers:
- connection-registry: additive optional `headers` field on remote/cloud
entries (normalized through the same forbidden-name filter, secret
envelopes like `token`); inherited on edit, treated as dial material by
connectionDialFieldsChanged, preserved by normalizeRegistry, and carried
through migrateV1ToRegistry. v2 registries without the field load
unchanged — no version bump.
- main.ts registry paths: connectRegistryBackend dials with the entry's
headers (readiness probe, ticket mint, descriptor REST via
getJsonForBackend/fetchJsonForBackend, registry ws-url minting with
rememberRemoteWsHeaders so renderer upgrades get them injected).
- saveRegistryConnection encrypts incoming plaintext header values with the
same safeStorage/allowPlainText seam as tokens; sanitizeRegistryConnection
exposes only header NAMES to the renderer — values never cross IPC.
- Connection tests exercise the leg they validate: both
hermes:connection-config:test and hermes:connections:test now send the
configured headers on the HTTP status call, the ws-ticket mint, AND the
live WebSocket probe (probeGatewayWebSocket grew an injectable `headers`
option passed as the undici WebSocket constructor's second argument).
- Settings → Connections gains an "Extra gateway headers" editor for
remote/cloud entries (name + secret value rows, stored values shown as
saved-but-hidden, clearable), with i18n keys (en + zh; other locales fall
back through defineLocale).
* chore: map contributor email for tigercraft4 (PR #74468 salvage)
* feat(delegation): record model/provider in live-transcript manifest (#telemetry)
* fix(gateway): attribute scoped credential lock conflicts to the owning profile (OOF-3)
Scoped credential locks (Telegram bot token, Discord bot token, etc.) are
machine-global, but the conflict error only reported the holder's PID:
Telegram bot token already in use (PID 559). Stop the other gateway first.
On multi-profile hosts (e.g. hosted instances running 13 profiles), a bare
PID gives the operator no way to tell WHICH profile owns the credential —
the exact failure mode observed on zerocool-9781, where the 'default'
profile was misconfigured with the same bot token as 'lead-gen-outreach'
and logged an unattributable conflict every ~5 minutes (4,602 rows).
Fix:
- acquire_scoped_lock() now stamps a 'profile' label on lock records,
inferred from the process HERMES_HOME (<root>/profiles/<name> layouts,
'default' for the root home). Omitted when not inferable.
- New scoped_lock_owner_label() resolves the owning profile from a lock
record: prefers the explicit field, falls back to inferring from the
persisted hermes_home for locks written before the field existed.
Labels are validated against the profile-id grammar before use (lock
files are plain JSON on disk and the label flows into log lines and a
suggested CLI command).
- _acquire_platform_lock() conflict message now names the owning profile
and gives the correct remedy:
Telegram bot token already in use by the 'lead-gen-outreach' profile
gateway (PID 559). Stop that gateway first
(hermes --profile lead-gen-outreach gateway stop).
Records with no attribution signal keep the original PID-only wording.
Testing:
- New TestScopedLockOwnerLabel suite covering label inference (named,
Docker, root/default, unknown layouts), grammar validation, explicit-
field preference, hermes_home fallback, and legacy/malformed records.
- acquire_scoped_lock tests for profile stamping and omission.
- Adapter-level tests for profile-attributed, legacy-home-inferred, and
PID-only conflict messages.
- 76/76 targeted gateway tests pass; broad gateway suite failures are
baseline-identical (verified via git stash comparison). Ruff clean.
* fix(gateway): surface multiplex profile failures (OOF-3)
* fix(status): aggregate independent per-profile gateway failures; harden key filter (OOF-3)
- /api/status now folds LIVE independent per-profile gateways' platform
failures (gateway_mode == 'multiple', the OOF-3 deployment mode) into
gateway_platforms under the validated <profile>:<platform> grammar, so
NAS fleet health sees them without a schema change. ?profile= requests
stay unmerged (single-profile view).
- Namespaced-key validation no longer fails open: colon-containing keys
are grammar-checked even when configured-platform loading throws.
- Platform key segment now accepts hyphens, matching plugin platform IDs
(plugins/platforms/<dir> names, e.g. foo-bar).
* fix(status): freshness-filter aggregated per-profile platform entries (OOF-3)
Gateway startup deliberately preserves plain platform entries in
gateway_state.json across restarts, and the active-profile endpoint
compensates by filtering against current configuration. The cross-profile
aggregation copied raw maps, so a fatal entry for a platform the operator
had since disabled/removed could keep NAS reporting the instance degraded
indefinitely.
The aggregation has no cheap per-profile config context (platform sets
depend on tokens in each profile's .env behind its secret scope), so use
freshness instead: an entry is aggregatable only when its updated_at is
at/after the live gateway process's create time (validated PID via
get_runtime_status_running_pid + psutil create_time; the record's own
start_time field is a PID-reuse fingerprint in clock ticks, not a
timestamp). Config changes require a restart to take effect, so
restart-anchored freshness is exactly the config filter's semantics.
Fail closed: unparseable timestamps or no live process exclude the entry
— a false 'degraded forever' is the worse failure mode.
* fix(status): strict writer-identity ownership for aggregated platform entries (OOF-3)
The freshness window (updated_at >= live process create_time - 2s) had a
P1 boundary hole: a stale failure written by the PREVIOUS process
immediately before a fast restart landed inside the slack and was
aggregated; if that platform was then removed, the new process never
replaces the entry and NAS stays degraded indefinitely.
Replace clock heuristics with persisted writer identity:
- write_runtime_status now stamps every platform entry with the writing
process's (writer_pid, writer_start_time) — the same PID-reuse
fingerprint the liveness checks use, so a recycled PID never
masquerades as the original writer.
- The aggregation ownership filter requires exact equality between an
entry's stamp and the profile's validated live gateway process
(get_runtime_status_running_pid + _get_process_start_time). No slack,
no timestamps. Legacy entries without a stamp fail closed.
- Writer stamps are process recon (same class as the auth-gated
gateway_pid) and are stripped from all /api/status projections, both
active-profile and merged cross-profile entries.
Near-boundary regression test: prior-process entry stamped 100ms before
restart is excluded; recycled-pid-different-fingerprint excluded;
legacy no-stamp excluded; current-process entry kept.
* docs(state): soften stale SessionDB self-pin wording after #88063
#88048 documented the token-writer self-pin (bound-method thread target +
strong atexit hook) as a permanent contract: "__del__ never runs for
exactly the instances that leak". #88063 then removed both pins (idle
writer retirement + weakref atexit hook), making abandoned handles
eventually collectible.
Reword the __enter__ docstring and the context-manager test module
docstring to describe the pin as historical motivation, note the #88063
behavior, and keep the guidance that owners close deterministically.
No code changes.
* fix(desktop): keep cloud bot avatar eye catchlights inside the eyes
The white catchlight dots in BotFace were static circles pinned at the
circle-face eye line (cy 16.5), whi…
mtbitcr
added a commit
to mtbitcr/hermes-agent
that referenced
this pull request
Aug 17, 2026
* fix(desktop): route registry 'local' entry to the genuinely-local runtime
ensureRegistryBackend delegated kind==='local' to ensureBackend(), which
follows the v1 connection.json routing table — under a v1 REMOTE global
mode (the migration keeps the mandatory 'local' entry AND makes that
remote the registry primary) the roster's 'This device' rows enumerated
and dialed the REMOTE primary: every profile appeared twice (forcing
-slug handles) and clicking a local agent talked to the remote box.
resolveRegistryLocalRoute() (pure, colocated with the registry helpers)
now decides the local entry's path: delegate to the legacy route only
when v1 is itself local (single-source behavior byte-identical);
otherwise spawn/reuse a forced-local pool child via spawnPoolBackend's
new forceLocal option, pooled under the composite conn:local::<profile>
key so it cannot collide with the v1 remote descriptor cached at the
bare profile key.
* fix(desktop): key fan-out event consumption by (connectionId, profile)
Secondary-gateway events were tagged with connectionId (store/gateway
fan-out) but no consumer read it: working/attention tracking, the
pruneSecondaryGateways keep-set, and the profile-scoped event gates
(skin.changed / change-watcher broadcasts / approval-mode reconcile)
all keyed by session id + bare profile name. Every registered source
exposes a 'default' profile (the roster force-unshifts it), so two
connected gateways collided — gateway B's 'default' activity was
attributed to gateway A's 'default', keeping the wrong socket alive
and applying the wrong source's config/skin/cron changes.
Thread connectionId through consumption using the existing composite
backendScopeKey helper:
- session-states records each registry-tagged event's (connectionId,
profile) scope per runtime session; liveSessionScopes() projects the
busy/needs-input ones as composite keys for the gateway keep-set.
- recomputeKeptGateways (use-gateway-boot) seeds the keep-set with
those scopes; pruneSecondaryGateways matches registry-scoped entries
ONLY on their composite key, while local entries keep matching bare
profile names (single-source path unchanged).
- gateway-event's 'from the active profile' gates now compare the
event's composite scope against the active gateway's connection via
the new activeGatewayConnectionId(); untagged local/primary events
behave byte-identically.
Display-only surfaces that already use roster handles are untouched.
* style(desktop): order @hermes/shared import before nanostores (perfectionist/sort-imports)
* fmt(js): `npm run fix` on merge (#87839)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fmt(js): `npm run fix` on merge (#87844)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(agent): trim background_review to the enabled switch
Follow-up to #87400: drop the max_iterations and prompt_file knobs from
auxiliary.background_review. The aux model routing (provider/model/
base_url/...) predates #87400 and stays; the enabled switch and the
usage telemetry stay. The fork's iteration budget returns to the
historical hardcoded 16.
* fix(update): reload _subprocess_compat and dashboard_procs after git pull
hermes update runs in the PRE-pull Python process. After git pull updates
source files on disk, modules already in sys.modules still hold the OLD
code. The existing _reload_config_modules() reloaded only config modules,
but the post-update dashboard cleanup path (_finish_dashboard_update_cleanup
-> _scan_dashboard_processes) imports hermes_cli._subprocess_compat lazily;
a new symbol added there (e.g. bounded_probe_run) is invisible to the
cached module object, causing ImportError during the cleanup step.
Extend the reload list to include hermes_cli._subprocess_compat and
hermes_cli.dashboard_procs so the cleanup uses freshly-pulled code.
* fix(update): reload process-scan modules at the dashboard-cleanup entry point
Widen PR #87757 to cover the ZIP path: _update_via_zip() also calls
_finish_dashboard_update_cleanup() but never runs _reload_config_modules,
so the Windows git-broken fallback would still crash with the same
ImportError (cannot import name 'bounded_probe_run' from the stale cached
hermes_cli._subprocess_compat).
- new _reload_process_scan_modules() called inside
_finish_dashboard_update_cleanup itself, so every current and future
call site is covered; reloads dependency-first
(_subprocess_compat, then dashboard_procs)
- reload failures log at warning (a miss surfaces seconds later as an
ImportError in the same process)
- regression tests: reload-before-kill ordering, node-failure skip,
stale-module symbol restoration (the exact #87134 boundary state),
nonfatal reload failure, and the #87757 reload-list contract
* chore: release v0.20.2 (2026.8.16)
* fix(tui): modified Enter and bare LF insert a newline in the composer across IDE and macOS terminals (#87854)
* fix(tui): send atomic CSI u for modified Enter in IDE terminals
VS Code/Cursor/Windsurf terminals bound Shift/Ctrl/Cmd+Enter to the
legacy \\r\n sequence, which Ink's parse-keypress split into a
backslash keypress plus a plain Return — inserting a stray backslash and
submitting instead of adding a newline. Emit Kitty CSI u sequences that
encode the modifier atomically, and migrate keybindings users already
have on disk.
Co-authored-by: yatesjalex <yatesjalex@users.noreply.github.com>
* fix(tui): treat a bare LF as a newline in macOS composer terminals
Terminals that can't send a distinct Shift+Enter collapse a modified
Enter / Ctrl+J down to a bare LF. shouldPreserveCtrlJNewline() already
handles the env-detectable cases (SSH, Windows Terminal, Ghostty, WSL),
but plain macOS terminals (Terminal.app, iTerm2 defaults) do the same and
aren't env-detectable, leaving no keyboard-driven newline there. Fold the
return-key decision into shouldInsertNewlineOnReturn() and accept a bare
LF as a multiline fallback on macOS too, keeping CR as submit everywhere.
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
---------
Co-authored-by: yatesjalex <yatesjalex@users.noreply.github.com>
Co-authored-by: LeonSGP43 <LeonSGP43@users.noreply.github.com>
* fix(state): classify structural DB corruption as its own persistence cause
'database disk image is malformed' contains the word 'disk', so
classify_persistence_error bucketed SQLITE_CORRUPT / SQLITE_NOTADB
failures as 'disk' and the turn-completion explainer told users to
free disk space for a structurally damaged state.db (the #77386-family
misdiagnosis, reproduced in the v0.20.0 malformed-DB incident report).
- hermes_state: new 'corrupt' bucket in PERSISTENCE_ERROR_CAUSES,
matched via _DB_CORRUPTION_MARKERS BEFORE the locked/disk buckets
- run_agent: explainer text for 'corrupt' points at hermes doctor and
explicitly says freeing space will not help
- cron explainer-variant suppression picks the new variant up
automatically (it iterates PERSISTENCE_ERROR_CAUSES)
* fix(openviking): strip PYTHONPATH from autostarted server child env (#78153)
(cherry picked from commit 7afd99155667cde480c0ab4ee31e242dab849d40)
* fix(openviking): read .env BOM-tolerantly when rewriting credentials
f1ea4a56c ("cover the remaining setup-time .env reads with utf-8-sig",
following 75afc47ba for mem0/hindsight) swept this class; openviking's
_write_env_vars was missed and still reads with strict utf-8.
It copies every existing line through on each update, so the read decides
whether a credential update lands:
BOM'd .env -> the first key never matches, so the old line survives and
the new value is appended as a duplicate. .env loaders keep
the first occurrence, so the update silently does nothing.
cp1252 .env -> UnicodeDecodeError aborts setup outright.
Read exactly like the canonical hermes_cli/config.py save_env_value
(utf-8-sig + errors="replace"). A plain UTF-8 file rewrites byte-identically.
Scope: hermes_cli/memory_setup.py has the same read but is already the
subject of #30281 / #60587, so it is left alone here.
(cherry picked from commit 175c6852c2c255b3219575b5de0b1b70f1f0efcb)
* fix(openviking): preserve non-UTF-8 env bytes on update
* docs(openviking): correct environment handling explanations
Clarify that the Desktop backend can add Hermes venv packages to PYTHONPATH and that current .env loaders use the last duplicate value.
* Revert "fix(agent): preserve local reasoning timeout opt-out"
This reverts commit 26b2b475935d5f5f369142fe1648cf5c95e7b056.
* Revert "fix(agent): harden canonical tool call deduplication"
This reverts commit 8fc4189edd23dde055232cc07ea14d1d525e44ee.
* fix(update): restart hermes-serve systemd units alongside gateways
hermes update discovered and restarted hermes-gateway* systemd units but
never looked for hermes-serve* — the Desktop app's backend — so it kept
running stale pre-update code until the user restarted it by hand (#83438).
Extend the systemd unit discovery/restart loop to also match hermes-serve*
units. They don't wire SIGUSR1 to a graceful drain (only gateway/run.py
does), so restart eligibility for the graceful path is now gated on unit
name via a small, directly-tested helper; hermes-serve units fall straight
to the existing blunt systemctl restart path, matching the workaround the
issue already documents.
* fix(update): tighten hermes-serve unit gate, dedupe fleet/cleanup restarts
Review on #83595 flagged two service-lifecycle gaps in the hermes-serve
restart support:
- The unit-name gate accepted anything starting with "hermes-serve",
which also matched the unrelated hermes-server.service. Require the
exact base unit or the hyphenated profile family instead.
- The fleet-restart loop and _finish_dashboard_update_cleanup() could
both restart the same hermes-serve unit — the loop restarts it
directly, then cleanup's PID scan finds the fresh process and
restarts its owning unit again. Thread the fleet loop's restarted
unit names through to _kill_stale_dashboard_processes() so it skips
units already handled.
* fix(update): tighten gateway-side unit gates to exact/hyphenated shape
Mirror the strict unit-name shape from the hermes-serve gate (review on
PR #83595) on the gateway side too: the discovery gate and the SIGUSR1
eligibility helper now accept only `hermes-gateway.service` or the
`hermes-gateway-<profile>` family, so a near-prefix unit like
`hermes-gatewayd.service` can neither enter the restart path nor be sent
a SIGUSR1 it does not handle.
* fix(desktop): ignore stale remote connection attempts
* chore: map contributor email for xkam7ar
* fix(apps): dial primary sleep/wake reconnect at window backend not active profile
* fix(desktop): scope pluginSocket's connection to the active profile
pluginSocket (hermes.ts) is documented as "the live twin of pluginRest,
scoped the same way", but it calls window.hermesDesktop.getConnection()
with no profile argument, while pluginRest passes the active profile via
profileScoped(). getConnection's IPC handler (ensureBackend in
electron/main.ts) falls back to the primary profile whenever the profile
argument is empty, so an unscoped call always resolves to the primary
profile's backend regardless of which profile is actually active.
For a plugin used from a non-primary profile (e.g. kanban), this means REST
calls go to the correct pooled backend while the plugin's WebSocket silently
connects to the wrong one — a multi-profile user sees one profile's data
with another profile's live events.
Fix (adapted to the post-#87600 registry-agent store shape during salvage):
resolve the plugin socket's connection through the same (connectionId,
profile) source of truth ensureGatewayProfile/ensureGatewayAgent maintain
for $connection — store/gateway's setActive now pushes the active scope's
registry connection id into the hermes module (setApiRequestConnection,
the no-store-import twin of setApiRequestProfile), and pluginSocket
resolves via getConnectionFor for registry-agent scopes and
getConnection(profile) for the local pool. The plugin socket therefore
follows registry-agent activations too, not just profile switches.
voice-playback.ts's resolveSpeakStreamUrl had the same gap originally, but
main has since fixed it independently (via the getApiRequestProfile()
getter rather than direct store access) — dropped from this PR as
redundant, keeping only the still-open pluginSocket gap.
Co-authored-by: Hermes Agent <hermes@nousresearch.com>
* fmt(js): `npm run fix` on merge (#87880)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(image_gen): disable default-on upscaling everywhere — opt-in only
The Aug 8 default-on upscaling policy (66ea4e686) chained the Clarity
Upscaler after every sub-2MP generation. Clarity is an SD1.5 creative
tile-diffusion enhancer (creativity 0.35, "masterpiece" prompt prefix) —
it redraws content, which degraded output on 100% of generations for
models like GPT Image 2 and Ideogram whose value is precise text
rendering, CJK, and photorealistic detail.
Policy now: no model upscales by default, on FAL or Krea. The `upscale`
tool param remains as a per-call opt-in (`upscale: true`); explicit
requests still chain Clarity (FAL) / Krea Enhance as before.
- FAL catalog: all 17 default-on entries flipped to upscale=False
- Krea plugin: medium + medium-turbo per-model defaults flipped off
- Tool schema: upscale param described as opt-in with a fidelity warning
- Tests updated: catalog invariant now pins all-off; default-on cases
now assert no upscaler call
- Docs (en + zh) updated to the opt-in policy
* fix: make every tool interruptible — sequential executor abandons on user interrupt
The sequential tool path only noticed a user interrupt after the running
tool returned: with the deadline disabled it ran the tool inline (fully
blocking), and with a deadline it waited in 5s slices without ever
checking agent._interrupt_requested. Any tool without cooperative
is_interrupted() polling (image_generate, tts, transcription, skills
sync, ...) held the whole turn hostage — the reported symptom was a
redirect queued ~40s behind a FAL image generation + upscale pass.
Executor backstop (class fix, covers ALL tools):
- _run_sequential_tool_execution_middleware always dispatches on the
daemon worker (timeout None no longer means inline blocking) and polls
the interrupt flag every 1s.
- On interrupt: 3s cooperative grace (mirrors the concurrent path), then
synthesize a cancelled tool result (_ToolCancelledResult), emit the
terminal post_tool_call with status=cancelled, and abandon the worker.
- _ToolCancelledResult suppresses downstream post-hook double emission
exactly like _ToolTimeoutResult, so an abandoned worker finishing late
cannot report success for a cancelled call.
- clarify (interactive, _NEVER_PARALLEL_TOOLS) keeps the inline path —
it owns its own human wait.
Cooperative layer in the reported offender:
- image_generation_tool: blind handler.get() (generation + Clarity
upscale) replaced with _wait_fal_result(), which polls is_interrupted()
in 0.5s slices and raises ImageGenerationInterrupted immediately.
- _upscale_image propagates the interrupt instead of swallowing it into
the "upscale failed, use original" fallback.
Message alternation is preserved: the cancelled result is a normal tool
result for the call_id. Sabotage-verified: with the old wait loop
restored, the new tests fail (tool blocks full runtime); with the fix
they pass in ~4s.
* feat(computer-use): support Cua Driver 0.20 runtime contracts
* fix(computer-use): reconcile existing cua-driver installs
* fix(computer-use): enforce existing-profile grant, unblock the opt-in
Live-testing the Cua Driver 0.20 convergence on Windows 11 (session 2,
cua-driver 0.20.0) surfaced three defects in the existing-profile browser
path and in install status.
1. The config grant was silently nullified by an approval bypass.
`--yolo` / `-z` map onto a private unrestricted daemon, which answers every
browser_prepare. Because the host delegated the entire existing-profile
decision to the driver, that bypass also nullified
`computer_use.grant_existing_profile: false`: a plain `hermes -z` attached
to the user's real Chrome profile and read live page content over CDP, with
the driver reporting it as "the approved existing Chromium profile". It was
never approved.
An approval bypass is consent to skip prompts, not consent to read an
existing profile's pages, cookies, and storage. CuaTypedBrowserRoute.prepare
now enforces the key itself, regardless of permission mode. bounded stays
exempt - its reviewed capability manifest is the authorization boundary.
The authorization inputs are resolved in the backend from config and the
backend's immutable mode, never from model-supplied kwargs.
2. The grant, once set, still could not be used.
With `grant_existing_profile: true` the runtime is launched
`--grant existing-profile` correctly, but cua_browser_prepare then hit a
runtime approval prompt anyway - re-asking the user to authorize what the
config already authorized, and making the documented opt-in unusable on any
non-interactive run, where the prompt has nobody to answer it and the call
dies on approval timeout. The durable, file-backed grant now stands in for
that prompt. Scope is narrow: only the existing-profile prepare, only when
the grant is present; isolated launches still prompt and any resolution
failure falls closed to prompting.
3. `computer-use status` hid a custom override and spliced its output.
With HERMES_CUA_DRIVER_CMD pointed at cmd.exe, status printed the child's
multi-line banner and prompt inside the one-line version field, never
mentioned the override, and advised `hermes computer-use install` - which
install itself (correctly) refuses to run against an overridden path. It now
names the override and mirrors install's update-or-unset guidance, and
version output is reduced to one bounded line.
Verified on the reported host: `-z` existing-profile attach now refuses and
names the key; `grant: true` no longer prompts (33s vs a 300s approval
timeout); status names the override and prints one line. No change to the
reconciliation path - driver SHA256 unchanged end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): make the typed-browser bind/snapshot split discoverable
`cua_browser_state` has two branches, chosen implicitly: any call carrying
pid or window_id is a *binding* (browser_route.py:252), anything else is a
*snapshot*. A binding clears session state, mints fresh tab_ids, returns
binding metadata with no page content, and sets verification_required.
Nothing in the response says that. A caller that keeps passing pid/window_id
- the natural reading of "bind to this window, then read it" - re-binds
forever: the tab_id it just received is unbound by the next bind, so every
cua_browser_navigate comes back browser_verification_required, and the
refusal ("take a fresh snapshot") points at the same call that just re-bound.
Observed live as 11 consecutive refused navigates before the model gave up
and fell back to foreground SendInput on the address bar.
The same confusion silently swallowed include_screenshot: both calls that
requested one were bindings, which carry no page content, so the flag had
nothing to attach to and was dropped without comment.
A binding response now reports snapshot_required, next_step
(fresh_browser_state, matching the existing token convention) and a hint
naming the exact next call; requesting a screenshot on a binding reports
screenshot_deferred instead of dropping it. The verification refusal now
says to call cua_browser_state WITHOUT pid/window_id and why re-sending them
does not help. The schema documents that include_screenshot applies to
snapshots.
Behavior of the bind and snapshot branches themselves is unchanged - this is
purely about making the split legible to the caller.
Unit-tested. Not verified end to end on the reporting host: the driver
refuses the bind upstream there (`browser_requires_setup: no owned DevTools
endpoint`, and it does not accept a user-launched --remote-debugging-port),
so the typed route never reaches this branch. That attach failure is a
separate cua-driver issue.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): keep a v3 capability manifest on approval-bypassed runs
`--yolo` / `-z` route the session onto a private embedded daemon in
`unrestricted` mode. That daemon was constructed without the configured
capability manifest, and the serve command only attached
`--capability-manifest` when the mode was exactly `bounded`. So the moment a
run was bypassed, the user's declared ceiling was dropped:
without -z: --permission-mode bounded --capability-manifest ...
with -z: --permission-mode unrestricted --dangerously-bypass-approvals
No manifest, no warning. The most carefully configured run - a reviewed
ceiling, written by hand - became the least constrained one, silently, and
it failed open.
That was never a driver limitation. cua-driver documents the manifest as a
ceiling across modes ("A manifest can narrow a profile but never widen it";
its own authorization table calls it `optional_capability_manifest_ceiling`),
and accepts it alongside `--permission-mode unrestricted`.
The forwarding is version-aware, because the two manifest schemas differ
(cua-driver session_manifest.rs):
* v1/v2 are legacy and must declare `mode: bounded`. Handing one to an
unrestricted runtime aborts startup with "legacy capability manifest mode
must be bounded", so a naive forward would turn a working session into a
hard failure. These are forwarded for bounded only, and a warning names
the migration when one cannot apply.
* v3 must not declare a mode. It is the mode-independent ceiling, and it now
rides along with unrestricted.
Unreadable or unparseable manifests are not forwarded outside bounded, on
the same fail-safe reasoning; bounded still forwards unconditionally and
lets the driver be the authority there.
Verified against cua-driver 0.20.0 on Windows. Launch args now carry
`--permission-mode unrestricted --dangerously-bypass-approvals
--capability-manifest <v3> --approve-capability-manifest`, and the ceiling
is enforced in the bypassed run - a tool outside the manifest is refused
("outside the capability manifest for this session ... blocked as a
protected resource") where the same config previously ran unbounded. A
legacy manifest was confirmed to abort driver startup when forwarded, which
is what the version gate prevents.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): warn when an approval bypass widens the driver mode
`--yolo` / `-z` read as "don't prompt me", but they also swap computer_use
onto a private `unrestricted` daemon, dropping the ceilings the configured
mode would have applied. Nothing said so. A script picks up `-z` for quiet
output and loses its limits as a side effect, and the only trace is a driver
process nobody inspects.
The mapping itself stays. It is deliberate, and `unrestricted` is reachable
no other way: it is intentionally not a config value so a stale config line
can never silently bypass approvals (see `_cua_configured_permission_mode`).
Removing the mapping would delete the capability rather than fix it, and
splitting it onto a second CLI flag was declined to avoid growing the
surface.
So the widening is now stated instead: one warning per session naming the
configured mode it left, what stopped applying, and the two ways to keep a
ceiling - drop the bypass flag, or declare a version-3 capability manifest,
which now rides along with unrestricted as of the previous commit.
Once per session, not per dispatch: the resolver runs on every tool call.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(computer-use): report unusable driver exit status
(cherry picked from commit 8bda6191ca548d65648263dfe30d2d18950a6e60)
* fix(computer-use): preserve missing driver overrides
* fix(computer-use): verify Windows driver repair
* fix(computer-use): align browser guidance and screenshots
* fix(desktop): keep the local pack out of electron-builder's publish path
`hermes desktop` runs `npm run pack` through _npm_lifecycle_env(), which
sets CI=1. electron-builder 26 reads that as an implicit publish request
(`onTagOrDraft`) when --publish is absent, so a local --dir build enters
publish resolution it has no business being in.
Pin `--publish never` on the pack script. This is also what electron-builder
asks for directly -- the implicit CI behavior is removed in v27.
Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
Co-authored-by: fangliquanflq <fangliquanflq@users.noreply.github.com>
* fix(desktop): declare the repository so publish resolution can succeed
With a GH_TOKEN/GITHUB_TOKEN in the environment, electron-builder auto-selects
the github provider and resolves owner/repo from the repository field, falling
back to reading <projectDir>/.git/config. projectDir is apps/desktop, which has
no .git of its own, and app-builder-lib does not walk up to the workspace root
-- so resolution returned null and threw "Cannot detect repository by
.git/config".
On Linux this fires from onAfterPack for a plain `dir` target: the darwin and
Windows branches return early for non-installer targets, Linux has no such
guard. That is why the same build worked elsewhere.
--publish never keeps `pack` from reaching this at all, but `dist:*` and
test-desktop.mjs still resolve publish config on a machine with a token, so
declare the field too.
Tests call the real app-builder-lib resolver rather than asserting on the text
of package.json, so they track electron-builder's behavior instead of our
formatting.
Co-authored-by: airo7 <airo7@users.noreply.github.com>
Co-authored-by: frankmendes1979 <frankmendes1979@users.noreply.github.com>
* fix(computer-use): auto-repair an installed driver that fails the runtime contract
A same-day version-floor bump (0.20 runtime contract) left every install
with an older cua-driver hard-failing on all computer_use calls: the
start() gate fails closed, while the `hermes update` refresh defers to the
driver's own check-update verb — whose ~20h cache routinely answers "no
update available" right after we raise the floor. Hermes knew it required
0.20+ but never acted on that knowledge.
Two changes:
- tools_config.install_cua_driver(): a contract-failed installed driver is
repaired on the upgrade=True path too (previously only upgrade=False).
The contract failure itself is the confirmation, so the
require_confirmed_update gate and the check-update short-circuit are
bypassed for repairs — an indeterminate or stale-cached check can no
longer pin users on an unusable driver.
- cua_backend.CuaDriverBackend.start(): when the contract gate fails on an
installed binary, attempt one automatic repair per process via the
standard install path, then re-probe. HERMES_CUA_DRIVER_CMD overrides
are never repaired (explicit override is authoritative even when broken)
and a missing binary still just reports the install hint. A failing
installer can't loop: the second start() surfaces the original error.
Tests: contract-repair coverage in test_computer_use.py (auto-repair
success, failed repair surfaces the original error, once-per-process
guard, override never repaired, missing binary never repaired) and
test_install_cua_driver.py (incompatible driver repairs despite an
indeterminate check-update, check-update not consulted). All new tests
verified to fail against the unfixed source (sabotage run).
* docs(computer-use): note driver contract auto-repair at update and runtime
The runtime-contract repair now also runs during hermes update and once
per session at the first computer_use call (PR #87923); the docs only
mentioned setup and toolset enablement.
* feat: raise Codex OAuth context to live-verified 350K for gpt-5.6 family and gpt-5.4
The Codex /models catalog advertises 272K for the gpt-5.6 (sol/terra/luna)
and gpt-5.4 slugs, but the backend actually accepts ~371K input tokens
(verified live against chatgpt.com/backend-api/codex/responses, Aug 16 2026:
~371K completed OK on all four slugs; ~382K+ rejected with
context_length_exceeded). 350K keeps ~22K margin under the observed ~372K
enforcement.
The bump applies ONLY when the resolved value is exactly the known-stale
272,000 advertisement — any other advertised value (higher or lower) is
trusted as a real server-side change, so a future catalog correction
deactivates the override automatically. gpt-5.5 and gpt-5.4-mini both
genuinely enforce 272K (rejected 360K live) and are excluded.
* fix(tui): restore Alt+Enter for newlines (#87066)
* fix(tui): restore Alt+Enter for newlines
Restore Alt+Enter support for inserting a new line in the TUI after the behavior was lost during newer input-handling updates.
Legacy terminals encode Alt+Enter as ESC followed by carriage return. Preserve those bytes as a single tokenizer sequence and parse the result as Return with the Meta modifier so TextInput inserts a newline instead of submitting.
Keep plain CR and LF mapped to unmodified Return, and cover the legacy ESC+CR sequence with a regression test.
* fix(tui): scope legacy Alt+Enter tokenization
* feat(desktop): expose connection-aware plugin routing
* fix(desktop): report remote plugin target profiles
* fix(desktop): route plugin profiles through registry
* fix(desktop): harden plugin route lifecycle
* fix(desktop): preserve registry route identity
* chore: add contributor email mapping for addelh
* fix(desktop): scope session/pin lists per connection across windows
Multiple Desktop windows share one renderer origin (one localStorage
area) while each window can be connected to a DIFFERENT gateway. The
sidebar pin set (hermes.desktop.pinnedSessions), the manual session
order, and the remembered last-session/route navigation keys were all
persisted under single global (or profile-only) keys, so two windows on
different gateways read and reconciled the same lists: pin-sync's
pullRemotePins() in one window adopted/dropped pins belonging to the
other window's backend, producing the overlapping mixed PINNED/SESSIONS
lists reported after the v0.19.1 update relaunch.
Introduce a connection-scope persistence layer (connectionScopedAtom in
src/lib/connection-scoped.ts): the local connection keeps the bare
legacy key (byte-identical for single-backend users, same contract as
backendScopeKey), while remote connections persist under
`<key>.remote.<encoded baseUrl>.<encoded profile>` — the shape
workspaceCwdKey already established. setConnection() rescopes every
scoped atom when the window's connection changes (null descriptors keep
the current scope, as with syncCronModelImpactConnection), and pin-sync
resets its mirrored/pending/unconfirmed bookkeeping on rescope so a
reconcile never PATCHes one gateway's pins to another.
Legacy globally-keyed values are deliberately not migrated into remote
scopes: ownership of rows accumulated by every window is unknowable
(the #67709 precedent), and backend-mirrored pins self-heal from the
gateway's own `pinned` rows.
Fixes #77318
* fix(desktop): keep profile rail alive across remote/Cloud connection switches
A connection/mode apply (soft re-home) moves /api/profiles routing to a new
backend, but nothing deterministically re-fetched the rail's $profiles list
and a stale in-flight response from the previous backend could land last and
collapse the rail to Home (#85731).
- store/profile: epoch-guard refreshProfiles/refreshActiveProfile so a
response fetched against the previous backend never writes the shared cache
(invalidateProfileListFetches), and bump the epoch on live profile swaps.
- store/gateway-switch: strand in-flight profile-list fetches in the same
wipe every connection/mode apply funnels through.
- use-gateway-boot: explicitly re-pull the active profile + list from the NEW
backend during softSwitch, best-effort like its sibling fetches.
Fixes #85731
* fix(desktop): read cron run-history from the owning gateway
When Hermes Desktop works against a REGISTERED gateway connection, cron
jobs execute on that gateway and persist their run sessions in the
gateway's state.db. But every REST call in the app — the cron surface
included — carried only `profile`, so `hermes:api` routed it through the
local profile pool and `_list_cron_job_runs_sync` read a local state.db
with zero `source='cron'` rows. Every job showed "No runs yet" while the
same endpoint on the gateway returned the real runs (#87882).
Fix at the routing seam:
- HermesApiRequest gains an optional `connectionId`. The renderer's cron
helpers (list/get/runs/delivery-targets/create/update/pause/resume/
trigger/delete/blueprints) now tag the active registry connection via a
new connectionScoped() twin of profileScoped(), fed from the same
setApiRequestConnection seam store/gateway already maintains for the
plugin socket.
- The hermes:api main-process handler resolves a tagged request through
ensureRegistryBackend — the SAME pool the job list and WS traffic use —
instead of the legacy profile route. Shared remote/cloud hosts (one
gateway, many profiles) get the path scoped with ?profile= via the new
pathWithProfileScope helper, factored out of pathWithGlobalRemoteProfile.
- '' / 'local' / absent connectionId keep the byte-identical v1 route, so
single-source and connection-config-remote users are unaffected.
This covers the run-history panel, the sidebar cron peek, and every other
cron surface in one place, since they all funnel through the same helpers.
Fixes #87882
* fmt(js): `npm run fix` on merge (#88014)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* test(desktop): pin steered-turn transcript order end-to-end
A steered turn's contract — pre-steer output above the correction bubble,
post-steer output and the settled reply below it — was fixed across
several PRs (#73793/#83151 class, settle fixes) but only covered piecewise:
the mid-turn insert as a unit, the settle math as a unit. Nothing drove the
real stream reducer through a whole steered turn, and nothing asserted the
durable-row hydration renders the same order after reload.
Two suites close that:
- steer-arrival-order: full event sequences through useMessageStream's real
handler + the real optimistic insert — single steer with tool activity,
steer racing message.complete, double steer in one turn.
- steered-turn-hydration-order: toChatMessages over persisted row shapes
copied from a real state.db steered turn, including a tool result that
lands after the correction row.
* test(desktop): harden steer-order suite against fake-timer id collisions
Review follow-ups: steer ids now come from a monotonic counter instead of
Date.now() (frozen under fake timers — two steers without a clock advance
would have collided), and the settle-above assertion documents its
load-bearing sealed-bubble assumption.
* test(desktop): steer suite drives the real redirectPrompt path; hydration fixture carries durable row shape
The live suite previously called appendMidTurnUserMessage directly, leaving
redirectPrompt's appendAfterActiveReply guard — the production decision of
WHERE a correction lands — outside the harness. Both hooks now mount together
sharing one state map, exactly as the desktop wires them, so a regression in
the caller (not just the insert) goes red. Verified by mutation: disabling the
guard fails 2/4.
Also covers the rejected-redirect path: a not_running response discards the
optimistic bubble instead of stranding a correction the model never saw.
The hydration fixture now carries the durable row shape the client actually
receives (row_id, reasoning, provider call_id/response_item_id on tool_calls)
instead of a hand-simplified echo, so the 'mirrors real state.db rows' claim
is honest. The fake-timer steer id counter is gone with the local insert —
ids come from redirectPrompt itself.
* fmt(js): `npm run fix` on merge (#88016)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* feat(desktop): bundle Bot Mode (hermes-bots) as a built-in, default-on plugin
Adopts the Hermes-Bot-Mode desktop plugin (NousResearch/Hermes-Bot-Mode)
into apps/desktop/src/plugins/hermes-bots/, registered by the bundled
vite glob and ON by default. It stays a pure @hermes/plugin-sdk consumer
in plain-ESM plugin.js form; users disable it live in Settings > Plugins.
- contrib/plugins.ts: bundled glob accepts plugin.js entries
- contrib/runtime-loader.ts: a disk/runtime copy of an id that ships
bundled is skipped (standalone installs predating adoption cannot
double-register)
- package.json: check:test:plugins runs the plugin's node:test suite in
CI (138 tests)
- source: Hermes-Bot-Mode @ c19baba, incl. today's #107/#103/#99 merges
* feat(agent): core Bot Mode teammate protocol — stable-tier prompt section
Replaces the plugin-side SOUL.md protocol append: on Bot-Mode-managed
installs (any profile carrying ui_meta['hermes-bots']) the prompt builder
injects the "Messaging other agents" section into every session of every
profile — including headless `hermes -p <bot> chat` sessions a teammate
starts — so bot handoffs work without mutating user-authored SOUL files.
- tools/bot_mode_probe.py: silent-when-unmanaged probe, cached per
(process, home), keyed off the agent's OWN home (not ambient
HERMES_HOME); silent when SOUL.md already carries the legacy section
- agent/system_prompt.py + agent_init.py + config_defaults.py: wired as
agent.bot_mode_protocol (default True), stable tier, byte-stable
across rebuilds (E2E-verified against the real build_system_prompt)
- tui_gateway profiles.list gains bot_mode_protocol capability flag;
the bundled plugin gates ALL SOUL protocol writes on it (backfill,
composeSoul, Edit save) — older gateways keep the SOUL-append path
- overhead: ~916 bytes, only on Bot-Mode installs; zero elsewhere
Supersedes the SOUL backfill half of Hermes-Bot-Mode#99 (credit
@kaduxo — the handle fix, `hermes profile list` correction, and
idempotent-append guards from that PR ship in the bundled plugin).
* fix: track bundled plugin.js sources past the tsc-artifact gitignore
apps/desktop/src/**/*.js is gitignored (stale tsc output shadows .tsx),
which silently dropped the hermes-bots plugin.js from the adoption
commit — tests shipped, source didn't, CI ENOENT'd. Negate the pattern
for src/plugins/*/plugin.js: adopted plain-ESM plugins have no .tsx
sibling, so the shadow hazard cannot apply.
* fix(agent): scope the Bot Mode protocol section to canonical Bot Chat sessions
Per review: the protocol belongs only in official Bot Mode interactions,
not every session on a managed install. The prompt builder now injects
the section only when the agent's session row is titled "Bot Chat"
(BOT_CHAT_TITLE, matching the desktop's createCanonicalChat pin and the
`hermes -p <bot> chat -c "Bot Chat"` resume target). Regular sessions
never carry it; the desktop composer middleware owns @mention sends.
Title is read once at first prompt build and the rendered prompt is
cached + DB-restored — cache-safe. E2E against the real AIAgent +
SessionDB: absent in an untitled session, present in Bot Chat,
byte-stable across rebuilds, absent after retitle, absent with the
flag off. Overhead unchanged (~916B, Bot Chat sessions only).
* fix(hermes-bots): composeSoul honors the bot_mode_protocol capability
Found in live desktop E2E: the generated-identity path of composeSoul
still appended the protocol section even when the backend injects it
into the system prompt. New agents now get a clean identity-only SOUL
against capable backends; older gateways keep the append. Covered in
the capability-suppression test.
* fix(agent): Bot Chat gate reads a session-title hint before the DB
Live desktop E2E caught a write-ordering bug the automated E2E missed:
tui_gateway applies pending_title to state.db AFTER the first turn, but
the system prompt builds at turn START — the DB-title gate saw nothing
and the Bot Chat was cached protocol-less forever. The gateway now
hands the agent its intended title at construction and the gate checks
the hint first, DB second (CLI/messaging-gateway paths unchanged).
Live-verified on the running desktop: fresh bot's Bot Chat persisted
with the protocol section, handle, and roster in its system prompt;
regular sessions and SOUL.md untouched.
* feat(agent): capability-refresh + timeless prompts for eternal Bot Chat sessions
Bot Chats break the "new sessions come often" assumption behind
build-once system prompts: capability edits used to sit invisible until
/new or compression, and the frozen birth date became misinformation.
- tools/bot_mode_probe.py: capability_fingerprint() hashes the profile's
capability surface (disabled skills, toolset pins, MCP config, SOUL.md,
installed skills, Bot-Mode roster); Bot Chat prompts embed the 12-hex
epoch stamp
- agent/conversation_loop.py restore path: stored Bot Chat prompt whose
epoch mismatches disk → ONE rebuild (through a cleared skills-prompt
cache so new installs appear), persisted so the next turn reuses the
new bytes verbatim. Prompts without a stamp — every non-Bot-Chat
session — never take the branch; probe failure fails closed to reuse
- agent/system_prompt.py: Bot Chat prompts are timeless — the
"Conversation started:" date is dropped (timezone kept); no ticking
fields in an eternal session
- tui_gateway: _sync_bot_capabilities at turn start rebuilds the live
agent (tool definitions are construction-baked) when the fingerprint
moves, same session id/history, with a user-visible notice
Cache stance: this is the /model exception applied to capabilities — a
loud, user-initiated, once-per-change prefix break. Unchanged state
hashes identically and stored bytes are reused verbatim (E2E-proven).
Validation: 9 probe unit tests incl. per-axis fingerprint changes;
E2E v3 against the real restore path (fresh build → verbatim reuse →
skill install → single refresh w/ new skill in index → verbatim reuse;
regular sessions dated, unstamped, never refreshed); tests/agent/
4647/4647.
* feat(agent): one-time protocol upgrade for legacy Bot Chat sessions
Bot Chats created before the epoch mechanism persisted prompts with no
protocol section and no stamp — the staleness check only fires on
stamped prompts, so pre-existing bots would never learn to message
teammates. stored_bot_chat_prompt_needs_upgrade() migrates them: one
rebuild, title-gated to Bot Chat, only when the probe would actually
emit a section (SOUL-append legacies and unmanaged installs are left
alone — rebuilding those would loop). The rebuilt prompt carries the
stamp, so the upgrade can never re-fire.
E2E v3b through the real restore path: legacy Bot Chat upgraded once
then verbatim-reused; legacy regular sessions byte-untouched.
tests/agent/ 4648/4648.
* fix: capability fingerprint reads config via the canonical loader
The config-read guard (test_config_read_guard) correctly flagged the
probe's raw yaml.safe_load of config.yaml — raw reads miss the managed
overlay, env expansion, and normalization. Use load_config_readonly()
under a scoped HERMES_HOME override instead. E2E v3/v3b and the guard
both green.
* feat: sync bundled Bot Mode with multi-source roster (Hermes-Bot-Mode#68)
Pulls the multi-source roster into the bundled plugin: profiles.list rows
from the active gateway are merged with the host.agents() union roster
(hermes-agent #86875), so the Bots panel shows agents from every registered
Desktop connection with @name-device handles for duplicates. Feature-detected
and best-effort — an older Desktop build or roster failure leaves the
single-source list untouched.
Adapted for the bundle:
- useRoster queryFn combines the bot_mode_protocol capability read (which
landed after #68 was cut) with the multi-source merge
- multi-source-roster tests updated for the namespace SDK import harness
- soul-protocol-backfill anchor widened for the new botHandle(name, bot)
signature
Plugin suite: 143/143.
* feat: raise Codex OAuth context to 900K for gpt-5.6 family and gpt-5.4 (subscription 1M rollout)
OpenAI enabled the large-context window for ChatGPT-subscription Codex
accounts (announced by @thsottiaux Aug 16 2026; previously API-key-only).
Live re-probe the same day: 911,276 input tokens completed OK on
gpt-5.6-sol; ~925K+ rejected with context_length_exceeded (1.05M window
minus reserved output headroom). terra, luna, and gpt-5.4 all completed
900,026 tokens OK. The Codex catalog still advertises 272K, so the
stale-advertisement override from #87981 is the right lever — this just
raises its value 350K -> 900K.
gpt-5.5 and gpt-5.4-mini still enforce 272K live (rejected 500K) and
remain excluded. Override semantics unchanged: fires only on an
exactly-272,000 advertisement; any live catalog change is trusted
verbatim.
* fix(desktop): map SSH profile aliases in REST paths
* chore: map contributor email for attribution audit
* fmt(js): `npm run fix` on merge (#88079)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(cron): stop retry storms when the gateway is deliberately stopped (OOF-266)
Since the managed-cron redesign (#84339, v2026.8.13) the dashboard fire
webhook forwards fires to the gateway process and returns 503 when it is
unreachable so NAS/QStash retries. Correct for transient windows — but an
operator-STOPPED gateway can never be fixed by retrying: every fire on
every job burns the full scheduler retry budget, NAS converts each 503 to
a retryable 502, and the resulting storms page on-call for a non-incident
(OOF-266 and its five duplicate tickets; +93% relay callback failures as
the fleet adopted v2026.8.13).
Split the unreachable path by durable operator intent:
- desired_state == "stopped" (written only by the s6 lifecycle commands;
the same intent signal container-boot reconciliation trusts) -> drop
the fire with 200 + a structured log line, mirroring NAS's own
instance_stopped drop. Jobs are not lost: the Chronos provider
reconciles and re-arms every job on the next gateway start.
- Anything else (crash loop, scale-to-zero wake, restart, legacy state
file without desired_state) -> keep the retryable 503, now stamped
with Retry-After: 60 so a scheduler that honors it spaces retries
past the wake/restart window instead of exhausting them inside it.
The gateway's own pass-through 503s (draining) get the same hint.
The intent check fails open (any parse/resolution error -> retryable
path) and is only consulted when the gateway is actually unreachable, so
a stale state file can never shadow a live gateway.
* feat(state): support the context-manager protocol on SessionDB
A SessionDB handle cannot be released by dropping the last reference.
Once its background token writer starts, the instance pins ITSELF two
ways: the writer thread's target is a bound method, and
queue_token_counts registers atexit.register(_drain_token_queue_at_exit),
which only close() unregisters. A dropped-but-pinned handle keeps its
state.db/-wal/-shm descriptors for the life of the process, and __del__
never runs for it, so the existing safety net is dead code for exactly
the instances that leak.
That is why owning call sites are expected to close explicitly, in those
words, in the ownership comments in run_agent.py and
tui_gateway/methods_session.py. This adds the ergonomic half of that
contract so an owner can scope a handle and be exception-safe by
construction:
with SessionDB(path) as db:
db.append_message(...)
Purely additive. __enter__ returns self, __exit__ closes and returns
False so a caller's exception always propagates, and close() is already
idempotent, so a scope that closes early still exits cleanly. Nothing
changes for callers that already close directly.
Four regressions cover the scope closing the handle, __enter__ returning
the instance itself, the failure path closing while still propagating,
and an early close leaving the exit clean. They assert on the
sqlite_safe_read tracking registry rather than raw descriptor counts,
matching test_session_db_read_conn_pool.py, because SQLite's unix VFS
parks a closed descriptor on a per-inode reuse list and makes raw counts
lag the real connection count.
Refs #88033
* fix(state): release abandoned session database handles
* fix(state): avoid overlapping context manager change
* fix(gateway): ignore invalid managed Node directories
Signed-off-by: Shawn Wang <32839114+enwaiax@users.noreply.github.com>
* fix(gateway): accept CJK full-width punctuation as MEDIA path terminators
MEDIA_TAG_CLEANUP_RE (and MEDIA_EXTENSIONLESS_TAG_RE) only recognized
ASCII terminators after a MEDIA:<path> tag. Chinese-language agent
output naturally writes MEDIA:D:\...\zhibao.pdf（782.6 KB）or ...pdf：内容 —
the full-width punctuation failed the trailing lookahead and the
attachment was silently dropped (cron even reported 'delivered') (#88038).
Both lookaheads now accept a CJK full-width terminator set (（）〈〉《》：，。；
！？、curly quotes【】) alongside the ASCII set. The #68773 adjacent-tag
splitting guard is covered by a regression test.
* fix(skills): rescan skill commands cache when active profile changes
Switching Desktop profiles mid-session changes HERMES_HOME but not the
platform scope, so get_skill_commands() kept serving the previous
profile's skill list. A skill only available under the new profile then
looked like a cache miss to callers such as slash.exec, which fall
through to the slash_worker dead path (#88023).
* fix(gateway): scope slash.exec's skill-command check to the session's profile
Independent review of the prior commit found the cache-invalidation key
alone doesn't fix the reported #88023 dead path: slash.exec runs as a
_LONG_HANDLER on the pool with a copied context, and no binding of
_HERMES_HOME_OVERRIDE happens between the transport read and the handler
body, so get_skill_commands() there always fell back to the process-level
HERMES_HOME regardless of which profile's session issued the request.
Bind the session's own profile_home around the get_skill_commands() check,
mirroring the same bind/reset-in-finally pattern already used at every
other per-turn HERMES_HOME scoping site (e.g. server.py's prompt-turn and
system-prompt-rebuild paths). This makes the #88023 dead path actually
reachable by the fix instead of only exercising the cache primitive in
isolation.
* feat(desktop): add status bar reconnect for offline gateways
Expose the existing profile-aware gateway boot reconnect path through a
single-flight renderer action, and surface a Reconnect button in the
gateway status menu panel whenever the socket is not open. Repeated
clicks share one in-flight reconnect; failures surface through the
existing non-destructive notification UI. Localized copy for all
supported Desktop locales.
Salvaged from PR #80694 (net diff re-applied onto current main; panel
code lives in app/shell/gateway-menu-panel.tsx now).
* fix(desktop): self-heal dropped SSH/HTTP registered remote connections
A dropped registered remote connection (SSH or HTTP) never recovered on
its own: the next boot attempt failed with a transient transport error
("Could not verify the existing SSH backend", ERR_CONNECTION_RESET,
mint timeout), the failure was correctly NOT latched, but nothing ever
re-attempted the boot — the renderer's reconnect machinery only arms
after a completed boot. The app parked on "Desktop boot failed" until
the user manually deleted and re-entered the same connection details,
which merely forced the fresh bootstrap an automatic retry would have
performed (issue 82679, feature ask 80430).
Root causes and fixes:
- electron/backend-start-failure.ts: new isRetryableRemoteBootFailure()
predicate — a remote, non-reauth boot failure is transient and may be
retried; local failures and confirmed 401/403 rejections are not
(a missing capability differs from a transient failure).
- electron/main.ts: the boot-failure progress broadcast now carries
`retryable` (rides with `error` through updateBootProgress), and a
failed reuse probe against a cached SSH master tears the stale
master/tunnel down so the next attempt bootstraps fresh — exactly
what manual re-entry did.
- use-gateway-boot.ts: bounded self-heal loop for a failed boot whose
progress is marked retryable — up to 5 re-attempts with the same
full-jitter backoff as the socket reconnect loop (2s base, 15s cap).
Exhausted retries end in the real boot-failure recovery overlay,
never an infinite spinner. Reset on success and on soft switch;
timer cleared on unmount.
- store/boot.ts: resumeDesktopBootForRetry() re-arms the overlay with a
retry status while an automatic retry is in flight.
Secondaries already had full-jitter backoff (store/gateway.ts); this
closes the same class for the PRIMARY/registered-connection path.
Tests: predicate matrix (retryable vs reauth-latch mutually exclusive),
plus renderer hook tests proving a transient SSH failure self-heals on
the next attempt, retries are bounded (6 total dials then the recovery
overlay, no further attempts), and non-retryable failures never enter
the loop. Sabotage-verified (disabling either half fails 4 tests).
Fixes #82679
Fixes #80430
* feat(desktop): support remote gateway headers
* feat(desktop): carry remote gateway headers through the connections registry, test probes, and Settings UI
Completes PR #74468 (remote gateway headers for Cloudflare Access, #74466)
against the v2 multi-connection registry that landed after the PR was
authored, and closes the review blockers:
- connection-registry: additive optional `headers` field on remote/cloud
entries (normalized through the same forbidden-name filter, secret
envelopes like `token`); inherited on edit, treated as dial material by
connectionDialFieldsChanged, preserved by normalizeRegistry, and carried
through migrateV1ToRegistry. v2 registries without the field load
unchanged — no version bump.
- main.ts registry paths: connectRegistryBackend dials with the entry's
headers (readiness probe, ticket mint, descriptor REST via
getJsonForBackend/fetchJsonForBackend, registry ws-url minting with
rememberRemoteWsHeaders so renderer upgrades get them injected).
- saveRegistryConnection encrypts incoming plaintext header values with the
same safeStorage/allowPlainText seam as tokens; sanitizeRegistryConnection
exposes only header NAMES to the renderer — values never cross IPC.
- Connection tests exercise the leg they validate: both
hermes:connection-config:test and hermes:connections:test now send the
configured headers on the HTTP status call, the ws-ticket mint, AND the
live WebSocket probe (probeGatewayWebSocket grew an injectable `headers`
option passed as the undici WebSocket constructor's second argument).
- Settings → Connections gains an "Extra gateway headers" editor for
remote/cloud entries (name + secret value rows, stored values shown as
saved-but-hidden, clearable), with i18n keys (en + zh; other locales fall
back through defineLocale).
* chore: map contributor email for tigercraft4 (PR #74468 salvage)
* feat(delegation): record model/provider in live-transcript manifest (#telemetry)
* fix(gateway): attribute scoped credential lock conflicts to the owning profile (OOF-3)
Scoped credential locks (Telegram bot token, Discord bot token, etc.) are
machine-global, but the conflict error only reported the holder's PID:
Telegram bot token already in use (PID 559). Stop the other gateway first.
On multi-profile hosts (e.g. hosted instances running 13 profiles), a bare
PID gives the operator no way to tell WHICH profile owns the credential —
the exact failure mode observed on zerocool-9781, where the 'default'
profile was misconfigured with the same bot token as 'lead-gen-outreach'
and logged an unattributable conflict every ~5 minutes (4,602 rows).
Fix:
- acquire_scoped_lock() now stamps a 'profile' label on lock records,
inferred from the process HERMES_HOME (<root>/profiles/<name> layouts,
'default' for the root home). Omitted when not inferable.
- New scoped_lock_owner_label() resolves the owning profile from a lock
record: prefers the explicit field, falls back to inferring from the
persisted hermes_home for locks written before the field existed.
Labels are validated against the profile-id grammar before use (lock
files are plain JSON on disk and the label flows into log lines and a
suggested CLI command).
- _acquire_platform_lock() conflict message now names the owning profile
and gives the correct remedy:
Telegram bot token already in use by the 'lead-gen-outreach' profile
gateway (PID 559). Stop that gateway first
(hermes --profile lead-gen-outreach gateway stop).
Records with no attribution signal keep the original PID-only wording.
Testing:
- New TestScopedLockOwnerLabel suite covering label inference (named,
Docker, root/default, unknown layouts), grammar validation, explicit-
field preference, hermes_home fallback, and legacy/malformed records.
- acquire_scoped_lock tests for profile stamping and omission.
- Adapter-level tests for profile-attributed, legacy-home-inferred, and
PID-only conflict messages.
- 76/76 targeted gateway tests pass; broad gateway suite failures are
baseline-identical (verified via git stash comparison). Ruff clean.
* fix(gateway): surface multiplex profile failures (OOF-3)
* fix(status): aggregate independent per-profile gateway failures; harden key filter (OOF-3)
- /api/status now folds LIVE independent per-profile gateways' platform
failures (gateway_mode == 'multiple', the OOF-3 deployment mode) into
gateway_platforms under the validated <profile>:<platform> grammar, so
NAS fleet health sees them without a schema change. ?profile= requests
stay unmerged (single-profile view).
- Namespaced-key validation no longer fails open: colon-containing keys
are grammar-checked even when configured-platform loading throws.
- Platform key segment now accepts hyphens, matching plugin platform IDs
(plugins/platforms/<dir> names, e.g. foo-bar).
* fix(status): freshness-filter aggregated per-profile platform entries (OOF-3)
Gateway startup deliberately preserves plain platform entries in
gateway_state.json across restarts, and the active-profile endpoint
compensates by filtering against current configuration. The cross-profile
aggregation copied raw maps, so a fatal entry for a platform the operator
had since disabled/removed could keep NAS reporting the instance degraded
indefinitely.
The aggregation has no cheap per-profile config context (platform sets
depend on tokens in each profile's .env behind its secret scope), so use
freshness instead: an entry is aggregatable only when its updated_at is
at/after the live gateway process's create time (validated PID via
get_runtime_status_running_pid + psutil create_time; the record's own
start_time field is a PID-reuse fingerprint in clock ticks, not a
timestamp). Config changes require a restart to take effect, so
restart-anchored freshness is exactly the config filter's semantics.
Fail closed: unparseable timestamps or no live process exclude the entry
— a false 'degraded forever' is the worse failure mode.
* fix(status): strict writer-identity ownership for aggregated platform entries (OOF-3)
The freshness window (updated_at >= live process create_time - 2s) had a
P1 boundary hole: a stale failure written by the PREVIOUS process
immediately before a fast restart landed inside the slack and was
aggregated; if that platform was then removed, the new process never
replaces the entry and NAS stays degraded indefinitely.
Replace clock heuristics with persisted writer identity:
- write_runtime_status now stamps every platform entry with the writing
process's (writer_pid, writer_start_time) — the same PID-reuse
fingerprint the liveness checks use, so a recycled PID never
masquerades as the original writer.
- The aggregation ownership filter requires exact equality between an
entry's stamp and the profile's validated live gateway process
(get_runtime_status_running_pid + _get_process_start_time). No slack,
no timestamps. Legacy entries without a stamp fail closed.
- Writer stamps are process recon (same class as the auth-gated
gateway_pid) and are stripped from all /api/status projections, both
active-profile and merged cross-profile entries.
Near-boundary regression test: prior-process entry stamped 100ms before
restart is excluded; recycled-pid-different-fingerprint excluded;
legacy no-stamp excluded; current-process entry kept.
* docs(state): soften stale SessionDB self-pin wording after #88063
#88048 documented the token-writer self-pin (bound-method thread target +
strong atexit hook) as a permanent contract: "__del__ never runs for
exactly the instances that leak". #88063 then removed both pins (idle
writer retirement + weakref atexit hook), making abandoned handles
eventually collectible.
Reword the __enter__ docstring and the context-manager test module
docstring to describe the pin as historical motivation, note the #88063
behavior, and keep the guidance that owners close deterministically.
No code changes.
* fix(desktop): keep cloud bot avatar eye catchlights inside the eyes
The white catchlight dots in BotFace were static circles pinned at the
circle-face eye line (cy 16.5), while the animation clock moves the
pupils to the shape-aware eye line (cy 22 for the cloud). On the cloud
avatar the highlights floated above the eyes instead of inside them.
- Tag the catchlights (data-hb-hl-l/r) and move them with the pupils in
paintMathFace, offset upper-left of each pupil center.
- Render the initial eyes/catchlights/shut-lids at the shape-aware eye
line so the first frame matches the animated frames.
* fix(desktop): make git worktrees work end-to-end on a remote gateway backend
Cmd/Ctrl+Shift+B worktree flows on a remote gateway route through the
backend's /api/git mirror (hermes_cli/web_git.py), but that mirror had
drifted behind the Electron-local git ops the same UI drives locally, so
the flows broke exactly and only on remote connections:
- Convert-a-branch: the picker offers remote-tracking refs, and the
Electron op turns "origin/feature" into a local tracking branch. The
mirror ran `git worktree add <dir> origin/feature` verbatim, which
either fails or detaches HEAD. It now resolves the ref's remote via
git (never assuming "origin"), fetches best-effort, and creates the
worktree with `--track -b <short-name>`.
- branch_list omitted remote-tracking refs entirely and never set the
`isRemote` flag the renderer's HermesGitBranch contract requires —
the convert picker on a remote gateway couldn't reach a teammate's
branch and mislabeled every row's action.
- Branching off an `origin/…` base silently wired the new branch to the
remote upstream; the mirror now passes `--no-track` like the Electron
op does.
Renderer side, replace the silent degradation with a capability gate:
when a remote backend predates the /api/git worktree routes, worktree
creation failed with an opaque "Expected JSON … got HTML" toast. The
route-missing shapes now surface a clear "update the Hermes backend"
message (isGitEndpointMissingError, mirroring the sidebar batch-endpoint
detector); real git errors still pass through untouched.
Sibling audit (documented, no code change needed): repo status / review /
file-diff / git-root / default-cwd already route through desktopGit()'s
REST bridge or /api/fs on remote; repo scan is deliberately a no-op there.
Stale comments claiming "empty/false on a remote backend" in projects.ts
and coding-status.ts updated to describe the backend-routed reality.
Fixes #81724
* fix(terminal): avoid FileProvider reads in lifecycle guard
* fix(cron): move cloud-placeholder refusal into _read_referenced_script and cover ~/Library/CloudStorage
Widen #88052 per review:
- The walk-level short-circuit only protected _contains_unsafe_gateway_action;
the sibling caller _read_script_for_scanning still opened cloud-resident cron
scripts and could hang preflight. Move the check into _read_referenced_script,
the shared choke point, so every caller fails closed without opening.
- Generalize _is_apple_file_provider_path -> _is_cloud_placeholder_path: detect
~/Library/CloudStorage (Dropbox/OneDrive/Google Drive third-party FileProvider
domains) alongside iCloud's Library/Mobile Documents.
- Regression tests: CloudStorage lexical path blocked without open; the choke
point itself refuses cloud paths with os.open forbidden.
* fix(cron): attribute cloud-path refusals to the cloud-synced script, not a lifecycle command
When check_gateway_lifecycle refuses a cron script that lives on a
FileProvider path, the generic error implied the job contained a dangerous
gateway lifecycle command. Surface th…
nikehagent2026
pushed a commit
to Ming-s-Agents/hermes-agent
that referenced
this pull request
Aug 19, 2026
…apters + dashboard forwarder) (NousResearch#84339) * fix(gateway): pass live adapters to cron fire webhook's fire_due The Chronos fire webhook (/api/cron/fire) called provider.fire_due(job_id, adapters=None, loop=loop), so every externally-triggered fire delivered through the standalone path even with a live gateway in-process. E2EE platforms and relay-fronted logical platforms (whose ONLY send path is the live relay adapter — no native credential exists on the box) failed every external fire with "platform 'X' not configured/enabled", while the same job delivered fine under the built-in ticker (gateway/run.py passes runner.adapters). Resolve the runner (self.gateway_runner → app['gateway_runner'] → _gateway_runner_ref(), the same chain the drain check uses) and forward its adapters. No runner → adapters=None, preserving the historical standalone path byte-identically. Note: does not by itself fix Fly-hosted scale-to-zero deployments where NAS's callback lands on the DASHBOARD process (internal_port 9119) — _fire_cron_job_for_profile there has no gateway runner in-process. That topology needs a separate fire handoff (design pending). * fix(cron): dashboard forwards Chronos fires to the gateway (503 when unreachable) The dashboard's /api/cron/fire executed cron jobs in the DASHBOARD process via _fire_cron_job_for_profile with adapters=None. On hosted deployments (Fly proxy exposes only the dashboard's port) that made every managed-cron fire deliver through the standalone send path, which cannot serve relay-fronted logical platforms (their only sender is the live relay adapter in the gateway process — no native credential exists on the box) or E2EE rooms. It also ran the whole agent turn inside the dashboard: wrong process for memory/session ownership and fire-claim attribution. Restore the invariant that the GATEWAY owns cron execution: - Dashboard route: after verifying the NAS JWT and resolving the job's profile, FORWARD the fire to the gateway api_server's own /api/cron/fire on loopback, NAS bearer preserved (the gateway re-verifies the JWT — defense in depth, no new trust link), and pass the gateway's response through. Gateway unreachable → 503 so NAS retries per the Chronos contract (non-2xx = retryable; the store CAS de-dupes the eventual double fire). Deliberately NO local-execution fallback. - Endpoint resolution mirrors gateway/config.py's api_server load order per target profile (config.yaml extra.port → API_SERVER_PORT from process env or the profile's .env → 8642), with /p/<profile>/ prefix routing under multiplex. - docker/stage2-hook.sh: generate a strong API_SERVER_KEY into .env on first boot when absent (never overwrites an operator value), so the loopback api_server passes its startup guard on hosted images. The fire route itself is NAS-JWT-authed; the key gates the rest of the api_server surface. The listener binds 127.0.0.1 by default and the Fly service exposes only the dashboard port. - _fire_cron_job_for_profile kept but deprecated (late-binding seam compatibility); no route calls it. - docs/chronos-managed-cron-contract.md: document the two-hop inbound topology and the 503-retry semantics. Depends on the previous commit (fire webhook passes live adapters to fire_due) — together they make NAS→dashboard→gateway fires deliver over relay end to end. * fix(cron): read the profile api_server port via the canonical config loader CI guard test_config_read_guard flagged the new _gateway_fire_endpoint for a raw yaml.safe_load of the profile's config.yaml — the exact drift class the guard exists to kill (raw reads miss the managed-scope overlay, ${ENV_VAR} expansion, and root-model normalization). Read through load_config() under a HERMES_HOME override scoped to the target profile instead (the same pattern the deprecated _fire_cron_job_for_profile uses for its store scope), and pull the port with cfg_get. Test updated to stub load_config rather than write a raw config.yaml. * fix(gateway): only messaging platforms count for the scale-to-zero arm gate The stage2 hook now generates API_SERVER_KEY for every Docker container, and key presence force-enables the api_server platform. The scale-to-zero arm gate counted every enabled platform, so the loopback api_server listener made messaging_is_relay_only_or_absent False on every hosted instance — silently disarming the feature (the not-armed log would show enabled platforms=['relay','api_server']). The arm gate and the not-armed logger now share one helper that filters to enabled MESSAGING platforms, excluding LOCAL/API_SERVER/WEBHOOK — the same non-messaging exclusion set _connect_platforms already uses. A genuinely enabled direct-socket platform (Discord/Telegram) still disarms. Two of the three new tests fail without this fix.
This was referenced Aug 20, 2026
This was referenced Aug 20, 2026
lisajlau
pushed a commit
to lisajlau/hermes-agent
that referenced
this pull request
Aug 20, 2026
… (OOF-266) Since the managed-cron redesign (NousResearch#84339, v2026.8.13) the dashboard fire webhook forwards fires to the gateway process and returns 503 when it is unreachable so NAS/QStash retries. Correct for transient windows — but an operator-STOPPED gateway can never be fixed by retrying: every fire on every job burns the full scheduler retry budget, NAS converts each 503 to a retryable 502, and the resulting storms page on-call for a non-incident (OOF-266 and its five duplicate tickets; +93% relay callback failures as the fleet adopted v2026.8.13). Split the unreachable path by durable operator intent: - desired_state == "stopped" (written only by the s6 lifecycle commands; the same intent signal container-boot reconciliation trusts) -> drop the fire with 200 + a structured log line, mirroring NAS's own instance_stopped drop. Jobs are not lost: the Chronos provider reconciles and re-arms every job on the next gateway start. - Anything else (crash loop, scale-to-zero wake, restart, legacy state file without desired_state) -> keep the retryable 503, now stamped with Retry-After: 60 so a scheduler that honors it spaces retries past the wake/restart window instead of exhausting them inside it. The gateway's own pass-through 503s (draining) get the same hint. The intent check fails open (any parse/resolution error -> retryable path) and is only consulted when the gateway is actually unreachable, so a stale state file can never shadow a live gateway.
benbarclay
added a commit
that referenced
this pull request
Aug 21, 2026
…v existing (OOF-285) (#88926) * fix(docker): stage2 API_SERVER_KEY bootstrap no longer depends on .env existing (OOF-285) Fleet sweep found 144/351 started hosted instances (41%) on v2026.8.13+ with no API_SERVER_KEY: the loopback gateway api_server (which serves /api/cron/fire on :8642) never started, so every scheduled cron fire was silently lost until the NAS retry budget exhausted. Root cause chain: - .dockerignore excludes .env.example (image-size optimization), so /opt/hermes/.env.example does not exist in shipped images - stage2's first-boot seed `seed_one ".env" ".env.example"` is a silent no-op when the source is missing -> fresh volumes never get a .env - the API_SERVER_KEY generation added in #84339 was gated on `[ -f "$HERMES_HOME/.env" ]` -> never ran on those instances Fixes: - stage2-hook.sh: keygen now creates an owner-only .env when missing instead of requiring it to exist; still append-only w.r.t. operator keys, still refuses symlinked paths - .dockerignore: re-include .env.example (negation after the .env.* exclusion) so the first-boot template seed works again - tests: new tests/tools/test_stage2_hook_api_server_keygen.py covers create-when-missing, append-without-clobber, operator-key preservation, symlink refusal, and a .dockerignore contract test for .env.example * fix(docker): container-provided API_SERVER_KEY wins over stage2 keygen (review) The bootstrap generated a key whenever .env lacked one, without checking the inherited container environment. That broke the documented `docker run -e API_SERVER_KEY=...` flow: Hermes loads $HERMES_HOME/.env with override=True (hermes_cli/env_loader.py), so the generated key silently shadowed the operator's env key and 401'd existing clients. - stage2-hook.sh: skip generation when API_SERVER_KEY is present in the container environment; if BOTH the env and .env carry keys, warn that the .env value wins at runtime and touch nothing - tests: regression tests for the env-provided path (skip + no .env write; env+file conflict warns without clobbering); sandbox runner now pins/unsets API_SERVER_KEY explicitly so results don't depend on the host environment * fix(docker): drop stale empty API_SERVER_KEY= line when container env provides the key A leftover empty 'API_SERVER_KEY=' assignment in .env clobbers a container-provided key at runtime (.env loads with override=True and python-dotenv sets the empty string), so the api_server startup guard fails and every scheduled cron fire is silently lost — the exact symptom class this PR fixes, reintroduced in the env-key branch. Remove the stale empty line (behind the existing symlink guard) before skipping generation, so the operator's env key actually wins. Addresses the IMPORTANT finding both reviewers converged on. Test: env-key + stale-empty-line combination now covered; strict removal assertion gated on GNU sed (BSD sed on macOS dev hosts skips the -i invocation, same caveat as the append test). * fix(docker): warn at boot when a container-provided API_SERVER_KEY is too weak to start the api_server The startup guard refuses keys under 16 chars. Now that a container-provided key suppresses stage2 generation, a weak `docker run -e API_SERVER_KEY=...` value means the api_server stays down (cron fires unavailable) instead of clients getting 401s against a generated key. Say so in the boot log, where the operator will look. * fix(docker): create .env under umask 077 instead of touch+chmod touch created the file with the inherited umask (typically 0644), then a silenced chmod tightened it to 0600 — a brief group/world-readable window, and no warning if the chmod failed. Creating under umask 077 makes the file owner-only from the first instant with no dependence on a second command succeeding. Covered by the existing 0600 mode assertion in test_keygen_creates_env_when_missing. * fix(docker): guard the API_SERVER_KEY append so a read-only .env degrades to a warning, not a failed boot stage2 runs under set -eu; the unguarded printf append meant a keyless .env on a read-only volume (or full disk) aborted the whole cont-init phase and the container boot. Guard it and emit the same loud warning the create-failure path uses. Test harness now runs the extracted block under set -eu to match production (it ran set -u only, so it could not see this defect class); new read-only regression test verified RED against the unguarded append via mutation. * fix(docker): only warn about a weak container API_SERVER_KEY when it is actually the effective key The <16-chars warning fired before the .env inspection, so a weak container key alongside a strong .env key produced a false boot-log claim that the api_server 'will refuse to start' — immediately followed by the both-keys warning saying the .env value wins, and the server in fact starts. Move the check into the branch where the env key really is the effective key on this boot (round-2 review finding, verified by execution against python-dotenv last-wins semantics). --------- Co-authored-by: Ben Barclay <ben@nousresearch.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third and final piece of the relay-cron delivery chain (follows #84300 resolution fixes and #84318 interactions-lane fix). Staging (
hermes-agent-stg-test-6698) still failed after both because managed-cron fires never execute in the gateway process at all.Root cause (verified on the staging box)
internal_port 9119= the dashboard. Listening sockets on the box:[22, 9119]— the gateway's api_server isn't running.cron.provider: chronos→ NAS POSTs the fire callback to the dashboard's/api/cron/fire, which executed the job in the dashboard process via_fire_cron_job_for_profile(..., adapters=None)(docstring even said "Runs with no live adapters").last_delivery_error: "platform 'discord' not configured/enabled"(confirmed on all 8 jobs in the box's jobs.json).Fix — gateway owns cron execution; dashboard is only the public door
Commit 1 —
fix(gateway): pass live adapters to cron fire webhook's fire_dueThe gateway api_server's own
/api/cron/firecalledfire_due(job_id, adapters=None, loop=loop)even with a live gateway in-process. Resolve the runner (self.gateway_runner→app['gateway_runner']→_gateway_runner_ref(), the chain the drain check already uses) and forwardrunner.adapters— delivery parity with the built-in ticker (gateway/run.pypassesrunner.adapters). No runner →adapters=None, historical standalone path byte-identical.Commit 2 —
fix(cron): dashboard forwards Chronos fires to the gateway (503 when unreachable)/api/cron/fire: verify the NAS JWT, resolve the job's profile, then forward the fire to the gateway api_server's/api/cron/fireon loopback with the NAS bearer preserved. The gateway re-verifies the JWT itself (defense in depth — no new trust link; the route is NAS-JWT-authed, not API-key-authed). Gateway response passes through.claim_job_for_firestore CAS de-dupes the eventual double fire. Deliberately no in-dashboard execution fallback — delivering from the wrong process is worse than a delayed retry.gateway/config.py's api_server load order per target profile:platforms.api_server.extra.portin the profile's config.yaml →API_SERVER_PORT(process env for the active profile, the profile's own.envotherwise) → default 8642. Multiplex mode routes non-default profiles through the/p/<profile>/mirror.docker/stage2-hook.sh: generate a strongAPI_SERVER_KEYinto.envon first boot when absent (never overwrites an operator-provided value), so the loopback api_server passes its ≥16-char startup guard on hosted images. Bind stays the adapter default127.0.0.1; the Fly service exposes only the dashboard port, so the listener is never publicly reachable._fire_cron_job_for_profilekept but deprecated (external callers may resolve it via the web_deps late-binding seam); no route calls it.docs/chronos-managed-cron-contract.md: documents the two-hop inbound topology and 503-retry semantics.Validation
All via
./scripts/run_tests.sh(CI-equivalent per-file isolation):fire_due(adapters is runner.adapters), and no-runner still fires withadapters=None..envport, multiplex/p/<profile>/prefix).tests/cron/+test_web_server.py+ cron profiles: 675 passed, 0 failed, 1 skipped.sh -n docker/stage2-hook.shclean. (tests/docker/build tests error in my environment for an unrelated pre-existing reason: local docker lacks BuildKit.)Ops notes
API_SERVER_KEY; until then the gateway api_server stays down and fires return 503 → NAS keeps retrying. Operators can also setAPI_SERVER_KEYmanually.