chore: sync CloudSeed Hermes fork with upstream - #4
Merged
Conversation
…sion switch A queue drain pairs two identifiers from different clocks: the queue key (flips with the route) and the explicit runtime id (lags a resume behind). Mid-switch the composer can fire a drain with storedSessionId=B but sessionId=A-runtime, and prompt.submit then lands B's queued prompt — and its whole answer turn — inside session A. Make the central runtime binding authoritative for queued sends: when the explicit runtime id no longer matches the binding recorded for the target stored session, adopt the binding (or drop to the stored-id resume path when none exists yet). The identity pair (storedSessionId === sessionId) is the fresh-chat fallback and stays untouched. Tests: re-home-via-resume and rebind-to-central-runtime; existing background-drain and sleep/wake cases declare central bindings explicitly. Null-fallback guard already on main; this PR is the remaining half.
The `clarify` tool's multiple-choice prompts flattened to a numbered text
list on Photon/iMessage, even though iMessage has a native poll bubble and
spectrum-ts already exposes it via the `poll()` content builder. Two gaps
caused the flattening:
* Outbound: the sidecar only had `/send` (text); there was no way to send
a poll, so the base adapter's numbered-text fallback was used.
* Inbound: `normalizeContent()` handled only text/attachment/voice, so a
poll vote (`poll_option`) was dropped on the floor ("[Photon content
type not handled: poll_option]") and never resolved the clarify.
Fix, end to end:
* Sidecar: import `poll` from spectrum-ts; add a `/send-poll` route
(`space.send(poll(title, ...options))`); serialize inbound `poll_option`
(the vote: chosen title + selected bool) and `poll` content in
`normalizeContent()`.
* Adapter: override `send_clarify` — for choices, send a native poll via
`_sidecar_send_poll` and call `mark_awaiting_text` so the gateway's
existing pending-clarify text-intercept resolves the answer; open-ended
clarifies keep the plain-text path. Inbound `poll_option` selections are
dispatched as a plain-text MessageEvent carrying the chosen option
(deselections / empty votes are dropped). If the poll send fails (an
older sidecar without `/send-poll`, or a send error) it falls back to the
numbered-text clarify, so nothing regresses on a half-upgraded restart.
No new model tool, no new env var, no core change — the capability lives at
the platform edge. The poll vote reuses the existing clarify text-intercept
resolution path, so no new gateway resolution mechanism is introduced.
Tests: tests/plugins/platforms/photon/test_poll_clarify.py — inbound vote ->
choice text, deselection/empty-vote dropped, send_clarify sends a poll +
enables text-capture, open-ended stays text, and poll-failure falls back to
the text list. Full photon suite green.
Contributed by Vaibhav Sharma (X: @vabbyshabby).
for poll clarify Follow-up to the NousResearch#48194 pick: it was written before NousResearch#43665 landed and re-added its own /send-poll sidecar route and poll import. Collapse the duplicates: - keep NousResearch#43665's /send-poll route (>=2 trimmed string options) as the single sidecar implementation; drop NousResearch#48194's variant - drop the duplicated poll import in the sidecar destructure - make adapter.send_poll() a thin wrapper over _sidecar_send_poll(), the one /send-poll client (shared with the poll-backed clarify path), and align its validation to the sidecar's >=2-options contract
imessage.effect.message crashed the sidecar at import against SDK stubs/
builds lacking the effect surface (caught by the patch-failure health test).
Optional-chain with {} fallback; /send-effect rejects cleanly instead.
The binding check landed unscoped, so it fired for every caller passing a sessionId/storedSessionId pair — not just queue drains. A slash skill dispatch into a fresh ⌘T tab passes exactly that shape (sessionId=tab runtime, storedSessionId=tab stored) with no central binding recorded yet, so the check nulled the target and the kickoff dropped instead of landing in the tab. Only a drain pairs identifiers from two different clocks; every other explicit-target caller resolves both ids in the same tick and is authoritative by construction. Gate on fromQueue and add the scoping invariant as a test. Also refresh the two drain tests for `queued: true`, which prompt.submit started sending for queued drains in ab68c5e after this work branched. Co-authored-by: theone139344 <theone139344@users.noreply.github.com>
spectrum-ts's live-stream consumer (consumeLive in spectrum-ts 3.x) only
reconnects when its inbound async iterator throws or ends. A half-open
("zombie") gRPC socket — where the TCP connection stays ESTABLISHED but the
peer is gone (NAT idle-timeout, network blip, laptop sleep) — makes the
iterator hang forever: no error, no end. The SDK exposes no gRPC keepalive
knob (createClient takes only {address, tls, token}; grpc.keepalive_time_ms
defaults to -1 = pings off), so the inbound stream silently dies and stays
dead until the gateway is restarted. Symptom: the agent's iMessage line goes
"online but deaf" — Photon's cloud-side fallback answers users with "the agent
isn't online right now" and inbound never reaches the gateway.
Fix, entirely in the code we own (no SDK fork):
- Sidecar gains a POST /probe endpoint that drives a cheap unary read
(space.getMessage on a synthetic id) over the SAME gRPC channel the inbound
stream uses. A live channel round-trips in ms (server returns not-found,
which is success for liveness); a zombie hangs. It sends nothing to any user
and creates no chat (space.get is local in shared/dedicated mode; only the
message read touches the wire).
- The adapter runs a presence watchdog: it probes on an interval, skips the
probe when natural inbound traffic already proved liveness within the
window, and after N consecutive failed probes respawns the sidecar — a fresh
Spectrum() re-subscribes the stream and re-registers presence. Successful
probes double as application-level keepalive, helping prevent the zombie
from forming at all. Respawn is lock-guarded against double-spawn and the
watchdog is torn down cleanly on disconnect.
Behavioural settings live in config.yaml (extra), bridged to env per the
.env-is-secrets-only convention:
probe_interval_seconds (60), probe_timeout_seconds (10),
probe_max_failures (3). A non-positive interval disables the watchdog.
Tests: tests/plugins/platforms/photon/test_presence_watchdog.py covers config
resolution, the disable switch, probe alive/dead(500)/timeout/no-client, the
core N-failures->one-respawn detection, success-resets-failures, stop-then-
start respawn ordering, and lock-guarding — all without spawning Node or
hitting the network.
Contributed by Vaibhav Sharma (X: @vabbyshabby).
Follow-up to the URL markdown fix: extract the /send builder decision into sidecar/send-format.mjs and rewrite test_url_send_path.py to execute the real module under node (format+text in -> chosen builder out) instead of regex- grepping index.mjs source, which is a banned test pattern in this repo.
…ict probe semantics Maintainer rework of NousResearch#45580 (issue NousResearch#54036) on top of the contributor's cherry-pick, which targeted spectrum-ts 3.1.0 while main pins 8.0.0: Sidecar (primary detection, new): - stream-staleness.mjs: pure decision rules, executable under node. * classifyProbeRejection: only a not-found-shaped rejection of the synthetic-id read counts as a completed round-trip (ALIVE); any other rejection is INCONCLUSIVE — never alive. The original /probe treated ANY rejection as alive, which was too loose. * shouldProbe: probe only after 10+ min of stream silence (configurable via PHOTON_STREAM_SILENCE_PROBE_MS; <=0 disables) with a cooldown. * isZombieSuspect: zombie only on silence past threshold AND a probe-proven live channel. Silence alone NEVER degrades (shared lines can be quiet for hours); inconclusive probes NEVER degrade (network may be down — the iterator will throw and the re-subscribe loop recovers on its own). - index.mjs: track last inbound-iterator yield (noteInboundYield), run a 30s watchdog tick, and on a confirmed zombie feed markStreamDegraded -> the existing exit-75 restart path. /healthz gains a stream.staleness block (silentForMs, threshold, lastProbeOutcome, zombieSuspected). /probe reworked to strict semantics: 200 only on a proven round-trip, 503 with outcome hung|inconclusive otherwise. Adapter (second layer, reworked): - _probe_once returns tri-state alive|hung|inconclusive; only a hung sidecar HTTP call counts toward the respawn counter — inconclusive resets nothing and triggers nothing. - default probe_interval_seconds 60 -> 600 (conservative; avoid restart storms on quiet lines). - _monitor_sidecar_health surfaces zombieSuspected from /healthz as a warning; the fatal UPSTREAM_STREAM_DEGRADED path is unchanged and fires when the sidecar escalates. Tests: test_zombie_stream_watchdog.py executes the real node decision module and drives the adapter against mocked /healthz responses; test_presence_watchdog.py updated for the tri-state probe. Also adds contributor mappings for nickkarhan (NousResearch#53283) and vaibhavjnf (NousResearch#45580).
…ture index.mjs now imports sibling .mjs helpers (send-format, stream-staleness); the fixture copied only index.mjs so the sidecar died on module resolution before reaching the health endpoint.
…og test The lifecycle cluster made fatal notifications detached; the watchdog test still asserted synchronously.
The statusbar derived its client and backend version labels inline, in two near-identical blocks. Move the wording into a pure resolver so every surface that names an install agrees on the label, the commit diff, and the tooltip.
The command palette's row called applyBackendUpdate() directly, so in local mode it drove the backend checkout rather than the client, and nothing opened the updates overlay to report either outcome. Route it through the same target selection the statusbar and About panel use, always surface the overlay, and re-check instead of applying when the active target is already current.
The command palette named the action but not the install it acts on. Carry the version and its commit diff on the row, resolved from the shared resolver so it reads identically to the statusbar.
Commit a1bc12f added stream=True to requests.post() in _generate_xai_tts, but 4 fake_post mocks in test_tts_xai_speech_tags.py still used the old signature without the stream parameter, causing TypeError in CI slice 5/8.
Signed-off-by: Bao <nnqbao@gmail.com>
Sibling fix for NousResearch#65977 — _model_flow_bedrock_api_key used only get_env_value for AWS_BEARER_TOKEN_BEDROCK, missing pool-backed keys. Now uses _resolve_api_key_provider_secret like the other flows.
…rch#60671) Add an opt-in streaming-audio adapter seam to BasePlatformAdapter so voice-capable gateway platforms (LiveKit, Discord voice, future adapters) can consume LLM output as streaming PCM audio before the full response completes, dropping perceived voice latency from ~2-3.5s to ~500-800ms. Adapter contract (gateway/platforms/base.py): - AudioFormat dataclass: declared sample_rate, channels, sample_width - StreamingTTSHandle: opaque handle with audible/aborted flags - supports_streaming_tts / begin_streaming_tts / write_streaming_tts / finish_streaming_tts / abort_streaming_tts - All default to unsupported/no-op so existing adapters are source-compatible - Per-turn _streaming_tts_completed_chats set suppresses duplicate whole-file auto-TTS when streaming succeeded; cleared after turn completion Gateway consumer (gateway/streaming_tts_consumer.py): - StreamingTTSConsumer: bridges sync agent deltas to async adapter audio sink - Uses existing SentenceChunker (no competing parser) - Thread-safe bounded queue; on_delta never blocks the agent worker thread - Resolves configured streaming provider via resolve_streaming_provider() - Serialises clause playback in order; flushes tail on completion - Pre-audio failure: completed=False (falls back to whole-file TTS) - Post-audio failure: completed=True, partial=True (no replay from start) - Abort is idempotent; late chunks silently dropped - Per-turn state isolated across concurrent chats Gateway integration (gateway/run.py): - message_type parameter threaded through _run_agent -> _run_agent_inner - StreamingTTSConsumer created when voice input + auto-TTS + provider active - Delta callback teed to both text stream consumer and TTS consumer - TTS-only delta callback installed when text streaming is off - finish() called from executor; wait_complete() in async context after - Barge-in aborts the consumer at all three interrupt detection points - Runner-level _send_voice_reply suppressed when streaming TTS completed Tests (tests/gateway/test_streaming_tts_consumer.py): - 15 focused tests: adapter defaults, lifecycle, ordered chunks, unsupported/No-streamer fallback, abort idempotency, late-chunk drop, pre/post-audio failure, concurrent-turn isolation, think-block suppression, queue backpressure Does not touch desktop/TUI code or add config flags. Plugin TTS provider stream() metadata gap (NousResearch#47896) is explicitly out of scope — built-in ElevenLabs/OpenAI PCM streamers are the first consumers. Refs: NousResearch#60671, NousResearch#47896
…ing.provider knob, docs + E2E tests Salvaged from PR NousResearch#47588 and rebased onto the post-campaign streaming core: the StreamingTTSProvider ABC/registry and the ElevenLabs/OpenAI streamers already live on main (tools/tts_streaming.py), so this ports the pieces main lacked: - GeminiStreamer: streamGenerateContent?alt=sse -> base64 PCM chunks (24 kHz mono int16), reusing main's DEFAULT_GEMINI_TTS_* constants. - XAIStreamer: WebSocket wss://api.x.ai/v1/tts -> binary PCM frames, async->sync bridged via the _collect_async test seam. - tts.streaming.provider config knob: pin one streamer, or 'auto' to walk the priority list elevenlabs -> gemini -> openai -> xai. Unset keeps the never-swap-the-user's-voice default. - docs/streaming-tts.md: architecture, capability matrix, how to add a provider. - Unit tests for the knob, SSE parsing, and the WS bridge; key-gated E2E tests (skipped without credentials). Refs: NousResearch#47588
…ecret; bound per-sentence stream bodies at 16 MiB Follow-up integration for the NousResearch#47588 salvage, aligning the new streamers with the post-campaign invariants: - All streaming key lookups go through _resolve_key -> tts_tool. _resolve_provider_key -> resolve_provider_secret (config > env/.env > credential pool, profile-scoped) — never bare get_env_value. xAI resolves via resolve_xai_http_credentials so OAuth users stream too. - _capped(): every provider's chunk iterator is bounded at 16 MiB per sentence, mirroring _read_tts_response_bytes' bounded-upstream-body invariant on the sync paths. - Tests updated for the resolver contract + new coverage for credential routing and the cap.
…ispatcher (NousResearch#58930) speak_text (hermes_cli/voice.py — the TUI/gateway one-shot TTS entry point) now checks resolve_streaming_provider() first: when the configured provider has a chunked streamer, the reply is spoken through the same stream_tts_to_speaker pipeline CLI voice mode uses, so audio starts on sentence one instead of after whole-file synthesis. No streamer (edge/piper/etc.) or a streaming failure falls back to the existing whole-file path unchanged — one dispatcher, zero parallel streaming implementations. Refs: NousResearch#58930
Update root and Photon sidecar npm overrides and lockfile to address protobufjs security advisories.
…s-stream-kwarg fix(tts): accept stream kwarg in xAI TTS test mocks
Installing a memory provider (Honcho, mem0, hindsight, ...) from the dashboard Plugins page failed on hosted deployments with a permission error: the setup endpoint shelled out to `uv pip install --python sys.executable`, which targets the sealed read-only venv under /opt/hermes (immutable hosted image, NS-579/NousResearch#49113). The correct mechanism already exists: tools/lazy_deps.py redirects installs to the writable durable target on the data volume (HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages) when the venv is sealed (HERMES_DISABLE_LAZY_INSTALLS=1), appends the target to the END of sys.path (core venv always wins collisions), and constrains shared deps to core-venv versions. The dashboard installer simply never used it. Fix: - tools/lazy_deps.py: new public install_specs() — installs arbitrary manifest-declared pip specs through the same environment routing as ensure(): venv-scoped by default, durable-target on sealed images, refused with an actionable reason when gated off (config kill switch or sealed venv without a target — never surfaces raw EROFS/EACCES). Specs are validated with _spec_is_safe(); post-install it invalidates import/metadata caches so availability rechecks in the same process see the new packages without a restart. Never raises. - hermes_cli/web_server.py: _install_memory_provider_pip_dependencies now calls install_specs() instead of building its own uv/pip subprocess. Blocked installs surface the gate reason in the setup results; the response's status block reflects post-install availability (stale 'missing deps' state clears immediately). - hermes_cli/memory_setup.py, plugins/memory/honcho/cli.py, plugins/memory/mem0/_setup.py: CLI setup wizards routed through install_specs() too — same sealed-venv failure mode, same fix. No hosted setup path writes to /opt/hermes anymore; provider discovery and installation now use the same environment (sys.path activation is shared with the lazy-install bootstrap in hermes_bootstrap). Tests: - tests/tools/test_lazy_deps.py: TestInstallSpecs — gating matrix (sealed+no-target blocked with immutable-deployment reason, config kill switch, sealed+target proceeds), spec-safety rejection before any subprocess, venv-scoped vs --target command display, failure stderr passthrough, never-raises contract. - tests/hermes_cli/test_web_server.py: setup endpoint routes pip through lazy_deps (regression guard asserts no direct 'pip install' subprocess), blocked-reason surfacing, same-response availability recheck clears stale missing state. Fixes NS-605 (Plain T-1111).
Widen NS-605 to the two remaining direct-install sites in the hindsight plugin, which still shelled out to 'uv pip install --python sys.executable' and therefore failed (EROFS/EACCES) on immutable hosted images with sealed venvs, and lost packages on redeploy: - post_setup dependency install (~L835): install_specs() with ok / blocked-reason / stderr handling matching honcho/mem0. - initialize()-time hindsight-client auto-upgrade (~L1240): install_specs(); blocked installs log the gate reason with the manual command instead of a raw subprocess error, and init proceeds. Audited every other memory plugin (supermemory, byterover, holographic, openviking, retaindb) for direct pip/uv install subprocess calls: none remain — their deps flow through plugin.yaml pip_dependencies or lazy_deps.ensure(). Tests: TestClientAutoUpgradeRoutesThroughLazyDeps — upgrade goes through install_specs with the exact spec (regression guard asserts no subprocess.run), blocked upgrade is non-fatal and surfaces the gate reason. Updated TestPostSetupEnvEncoding stubs to the new install path.
…d/teams/telegram siblings Same class of bug as the LINE adapter (NS-603): defaulting the webhook bind to "0.0.0.0" (or hardcoding it) binds IPv4 ONLY, so the listener is unreachable over IPv6-only private networks such as Fly.io 6PN. - wecom callback_adapter: DEFAULT_HOST None; config.py env seed no longer forces 0.0.0.0 when WECOM_CALLBACK_HOST is unset. - msgraph_webhook: DEFAULT_HOST None; the allowed_source_cidrs requirement still fires for the all-interfaces default (host=None is treated as network-accessible). - whatsapp_cloud: DEFAULT_WEBHOOK_HOST None. - teams: hardcoded 0.0.0.0 TCPSite bind → _DEFAULT_HOST=None with new TEAMS_HOST / extra.host override (mirrors LINE_HOST pattern). - telegram: hardcoded listen="0.0.0.0" → default "" (tornado bind_sockets opens one socket per address family; verified against PTB 22.6/tornado) with new TELEGRAM_WEBHOOK_HOST / extra.webhook_host override. Explicit host overrides everywhere are preserved; empty/unset collapses to the dual-stack default. "::" remains a bad substitute on bindv6only=1 hosts (see LINE adapter comment).
LINE plugin.yaml plus line/wecom-callback/msgraph-webhook/ whatsapp-cloud/teams user-guide pages still documented the old IPv4-only 0.0.0.0 defaults; update to the dual-stack unset default. Telegram webhook env docs live in the adapter docstring (updated with the code change); its plugin.yaml has no webhook host entry.
One page consolidating all three Hermes×Buzz integration paths — Desktop managed runtime, buzz-acp relay bridge, and the native gateway platform — with a comparison table, per-path pointers into the detailed docs, identity guidance, and contributor credits. Registered in sidebars.ts under Integrations; Buzz added to the messaging platform list and a new Collaboration Workspaces section on the integrations index. en + zh-Hans.
Replace 20 lines of manual os.open/O_EXCL/fdopen/fsync/os.replace with the existing atomic_json_write() from utils.py, which is already used by 6+ modules and handles temp-file creation, fsync, atomic replace, mode control, and owner preservation. The only novel helper (_fsync_directory) is retained — atomic_json_write does not do directory fsync. Update test_flush_write_failure_leaves_no_recovery_file to monkeypatch utils.os.replace (the new call path) instead of gateway.shutdown_flush.os.replace.
…e fallback Follow-up docs for the July 29 salvage wave (NousResearch#73865 session filtering, NousResearch#73864 photon sidecar immutable install trees).
…tplace hub tab) The Skills Hub 'Marketplace' tab showed a single useless entry: Anthropic changed .claude-plugin/marketplace.json to bundle-shaped plugins whose source is './', so all plugins collapsed to one identifier pointing at the repo root, and the second marketplace repo (aiskillstore/marketplace) is gone (404). Everything in anthropics/skills is already surfaced by the GitHub tap as the Anthropic tab, making this source fully redundant. Removes ClaudeMarketplaceSource and all wiring: source router, index builder (crawl + floors + sort order + rate-limit messaging), extract labels/install/URL mapping, hub UI tab, web server labels, CLI limits, docs (en + zh), the legacy index-cache snapshot, and test fixtures. Stale skills-index entries with source 'claude-marketplace' still install fine: HermesIndexSource fetches via resolved GitHub paths generically.
fix(desktop): make the ⌘K Update Hermes command actually update
fix(desktop): keep queued drains out of the foreground session on session switch
Type-to-focus routes through requestComposerFocus('active'), which resolved
to a module-level activeTarget claim. Inactive tabs stay mounted under
data-pane-hidden, so typing in a session tile then clicking the main tab
left activeTarget on the buried tile: use-keybinds preventDefaults the
keystroke, the buried composer ignores the request (or is filtered out),
and the main composer never sees it. Same class of bug after the inline
edit composer unmounts with activeTarget still 'edit'.
Heal 'active' against the visible data-composer-target stamp (the same
visibility policy as every other document-wide surface lookup), release
the claim on real unmounts (useComposerDraft + user-edit-composer), and
keep getActiveComposer honest so Esc / soft / / voice agree with the
keyboard path.
The unmount release is salvaged from NousResearch#72625 (@briandevans); this PR adds
the keep-alive tab heal his unmount-only fix couldn't cover.
Co-authored-by: briandevans <252620095+briandevans@users.noreply.github.com>
…ivation + full OpenAI TTS voice/model options
Three GUI Capabilities-tab defects reported on Windows:
1. Browser rows stuck on 'Setup required' after a successful setup run.
Root causes, all in the readiness probe (not the installer):
- _has_agent_browser() never searched the Hermes-managed Node dir
(%LOCALAPPDATA%/hermes/node / $HERMES_HOME/node/bin) where the
Windows install lands, and probed node_modules/.bin/agent-browser
as the extensionless POSIX shim, which fails exec on Windows
(WinError 193) — now resolved via PATHEXT-aware shutil.which
against both rungs, mirroring _find_agent_browser().
- Cloud rows (Nous Subscription Browser Use, Browserbase, Browser
Use, Firecrawl) declared post_setup: agent_browser, whose
readiness gate requires a LOCAL Chromium build the cloud never
uses — switched to the cloud-scoped 'browserbase' hook (CLI-only).
- _agent_browser_installed() could read browser_tool's stale cached
'Chromium missing' result from before the install ran in the
spawned post-setup process — cache now dropped before probing so
the pill flips to Ready right after a successful run.
2. No way to tell which backend is active, and clicking a row to read
its details silently rewrote config. Row click now only
expands/collapses; activation is an explicit 'Use this backend'
button, the active row carries an 'Active' pill, and the expanded
active row says 'This is your active backend'.
3. OpenAI TTS showed one model and one voice. The options were always
defined but rendered through a native <datalist>, which filters by
the field's current value — a field already set to a valid option
suggested only itself. Replaced with a real combobox (Input +
dropdown) that lists every option, and voice suggestions now track
the selected model per the OpenAI TTS docs: tts-1/tts-1-hd = 9
voices, gpt-4o-mini-tts = 13 (adds ballad, verse, marin, cedar).
_has_agent_browser()'s new managed-Node rung calls
shutil.which('agent-browser', path=...); tests that monkeypatch
shutil.which globally with 1-arg lambdas raised TypeError when their
code path reached the browser readiness probe (test_post_setup_gating,
test_setup_model_provider).
…y/paste Streamed response text carried a 4-space _STREAM_PAD indent and the final-response Rich Panel used padding=(1, 4), so every line selected out of the terminal came with leading whitespace. Both now render flush-left (pad empty, panel padding=(1, 0)); the table-realignment width budgets were widened to match. /copy now writes the ORIGINAL message text through native clipboard tools (pbcopy / PowerShell Set-Clipboard via base64 / wl-copy / xclip / xsel — same fallback chain as the TUI's writeClipboardText), falling back to OSC 52 only when no native backend succeeds. This is the TUI-equivalent answer to soft-wrap mangling: the clipboard gets the raw text, not the rendered layout.
The Settings "Open plugins folder" action and the runtime disk-plugin
scanner both derived the plugin directory from getStatus().hermes_home.
Against a remote backend that value is a path on the REMOTE box (or
undefined), producing `undefined/desktop-plugins` — the folder action
errors ("Could not open the plugins folder undefined") and disk-plugin
discovery silently finds nothing, even with a valid local plugin.js.
Add an Electron-owned IPC resolver (hermes:fs:desktopPluginsRoot) that
returns <HERMES_HOME>/desktop-plugins computed from the main-process
HERMES_HOME — the local Electron path, valid in every connection mode —
creating it on demand. Route both the Settings folder action and the
runtime scanner through it, so a remote backend never determines the
local filesystem location used for Desktop runtime plugins.
Fixes NousResearch#66899
…watch path Follow-ups on top of NousResearch#66911's salvaged commit: - hermes:fs:desktopPluginsRoot now resolves the ACTIVE desktop profile (readActiveDesktopProfile) so named profiles keep their own profiles/<name>/desktop-plugins root instead of sharing the global one (profile-scope concern raised on the PR thread). - startDirWatch in runtime-loader.ts was a third sibling site still deriving the watch path from the backend's hermes_home (added by the later fs-watch commit); routed through the same Electron-local resolver, with regression coverage.
…pdate marker (NousResearch#73822) On Windows, applyUpdates kills its own backend (releaseBackendLock) BEFORE the venv-blocker preflight but only writes the on-disk update marker AFTER the scan. Killing the backend drops the renderer's WebSocket; the renderer reconnects within ~1s and the marker-only waitForUpdateToFinish gate happily spawns a fresh 'hermes serve' inside the update's own critical section. scanVenvBlockers then finds that brand-new process and aborts with 'another Hermes process is using this installation' — a different PID on every attempt, so Desktop self-update can never succeed. Fix: extract the gate into update-gate.ts (pure, DI-testable) and make it consult BOTH signals — the on-disk marker AND the in-process updateInFlight flag. The success path writes the marker before the flag clears in applyUpdates' finally, so there is no instant where both are false and a waiter can slip through. Also gate spawnPoolBackend, which previously had no waitForLocalStart at all — a background profile window could respawn a pool backend during the same window with the identical abort. Tests: update-gate.test.ts covers the open gate, the flag-only window (the NousResearch#73822 shape), the flag→marker handoff with no gap, and timeout.
…ype-to-focus-main fix(desktop): heal type-to-focus onto the visible chat surface
…oken or typed
Saying OR typing a configured stop phrase (voice.stop_phrases, default
"stop") now ends the voice chat everywhere, not just classic CLI PTT:
- hermes_cli/voice.py: new explicit on_stop_phrase callback through
start_continuous/stop_continuous. The force-transcribe path previously
DISCARDED the stop phrase silently — with auto_restart=False the client
re-arms the next capture, so the conversation never ended. Both halt
paths now fire on_stop_phrase (fallback: on_silent_limit for legacy
callers) as user intent, distinct from the no-speech timeout.
- tui_gateway/server.py: voice.record wires on_stop_phrase and emits
voice.transcript {stop_phrase: true} after flipping HERMES_VOICE(_TTS)
off and stopping streaming TTS — same teardown as /voice off. The TTS
barge-in monitor stop-checks its transcript too. prompt.submit consumes
a TYPED bare stop phrase at the server-side choke point when voice mode
is on (returns {voice_stopped: true}, no turn starts).
- ui-tui: voice.transcript {stop_phrase} ends voice mode with a clear
'voice chat ended' notice (distinct from the no-speech-limit message);
submitPrompt releases the busy latch on a consumed voice_stopped reply.
- cli.py: _typed_voice_stop in process_loop — typing a bare stop phrase
while voice mode/continuous is active ends voice mode instead of
sending 'stop' to the agent; typed 'stop' outside voice mode is
unchanged. Voice transcripts skip the check (already stop-checked).
- desktop: interceptsTypedVoiceStop — the composer's onSubmit ends the
live voice conversation (same path as clicking end on the pill) when a
bare stop command is typed with no attachments; renderer-owned loop, so
handled client-side like the existing spoken isVoiceStopCommand.
- tools/voice_mode.py: transcribe_recording never lets the Whisper
hallucination filter swallow a configured stop phrase (e.g. 'bye'
configured as a stop phrase is both a hallucination-blocklist entry and
a stop phrase — stop-phrase check now wins).
Tests: continuous-loop signal (sabotage-verified), force-transcribe stop
signal + legacy fallback, hallucination-filter ordering, typed-stop CLI
unit tests (voice on/off/longer text), prompt.submit typed-stop gateway
tests, TUI vitest for stop_phrase event handling, desktop vitest for the
typed-stop interceptor.
…stub the module without is_voice_stop_phrase)
Local faster-whisper called model.transcribe with bare {'beam_size': 5}:
no VAD, cross-window conditioning on, no confidence filtering. Pure
silence produced hallucinated tokens (E2E: 5s anullsrc WAV -> 'You',
no_speech_prob=0.705) and noisy clips could produce runs of junk, often
in other languages.
Three-layer class fix, one shared owner for every local-whisper call
site (build_local_transcribe_kwargs):
1. Silero VAD filter (bundled with faster-whisper) on by default —
silence never reaches the model. stt.local.vad: false restores the
raw behavior for music/ambient transcription.
stt.local.vad_min_silence_ms tunes chunk splitting (default 500).
2. condition_on_previous_text=False — one hallucinated token can no
longer seed a self-reinforcing run; negligible cost for
voice-note-length audio.
3. Segment confidence gate (_join_confident_segments): drop a segment
only when no_speech_prob > 0.6 AND avg_logprob < -1.0 (openai-whisper's
own heuristic shape; both must hit so quiet-but-real speech survives).
Config: stt.local.no_speech_prob_threshold / logprob_threshold.
The WHISPER_HALLUCINATIONS blocklist in voice_mode.py stays as
last-resort defense but should now almost never fire.
E2E (real faster-whisper 'base', CPU int8):
silence.wav before 'You' -> after ''
noise.wav before '' -> after ''
speech.wav before/after 'Hello World, this is a test of the
transcription system.' (unchanged)
Docs (EN + zh-Hans), DEFAULT_CONFIG, cli-config.yaml.example updated;
19 unit tests (kwargs contract, off-switch, confidence gate incl.
quiet-speech survival, _transcribe_local wiring), sabotage-verified.
The exact kwargs snapshot broke when VAD hardening added keys — the test's real contract is 'null stt.local: must not crash or force language/prompt'. Baseline kwargs are pinned by the dedicated suite.
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…etup-browser installs aren't reported as missing (NousResearch#53192) `hermes acp --setup-browser` installs agent-browser into the Hermes-managed node prefix (~/.hermes/node/bin/agent-browser), which isn't necessarily on PATH. doctor only checked PROJECT_ROOT/node_modules and PATH (shutil.which), so it false-negatived with "agent-browser not installed" even though the binary was present and runnable. Mirror dep_ensure._has_hermes_agent_browser() by also checking HERMES_HOME/node/bin and the legacy HERMES_HOME/node_modules/.bin path, each gated by agent_browser_runnable(). Tested with tests/hermes_cli/test_doctor.py (added positive + not-runnable cases) and pytest tests/hermes_cli/test_doctor.py -q (66 passed).
Follow-up on the NousResearch#53205 salvage: replace bare is_file() probes of the managed (~/.hermes/node[/bin]) and legacy (node_modules/.bin) locations with shutil.which(..., path=dir) so Windows resolves the executable .cmd shim instead of the extensionless POSIX script — the same miss class fixed for _has_agent_browser() in NousResearch#73932. Also covers the Windows managed layout where the binary sits in node/ directly.
Dannyxlm
pushed a commit
that referenced
this pull request
Aug 11, 2026
…on delegation callbacks (NousResearch#82592) * fix(gateway): stop frozen-preview finals and dropped idle-session delegation callbacks Two relay-plane delivery losses from the 2026-08-09 staging incident: 1. stream_consumer: the skip-redundant-finalize branch recorded _accumulated as the delivered turn-final payload even when the last ACKED edit was an earlier throttled preview snapshot, so delivered_final_matches reconciled True and the gateway suppressed the corrective final send — the user was left with a cut-off message ending in the streaming cursor. Extracted _mark_skip_redundant_finalize(): records the last acked wire payload (cursor-stripped), so a preview/final mismatch now returns False and the normal final send fires. 2. run.py: _classify_completion_target classified every ended parent session terminal unless it ended by compression. Idle/timeout session ends are the norm on scale-to-zero relay deployments and the chat route remains valid; completed async delegation results were terminally dropped. Ended parents now classify deliver unless the end was an explicit user boundary (session_reset / user_exit / session_switch). * fix(relay): drain in-flight outbound frames before transport teardown disconnect() failed every pending outbound future immediately with 'relay transport closed', so a trailing finalize edit racing turn teardown was lost even though the connector socket could still serve it. Bounded drain grace (5s) lets in-flight requests resolve; silent connectors still tear down promptly. asyncio.wait (not gather+wait_for) so a timeout doesn't cancel futures owned by the fail-remaining loop. * fix(gateway): route completion injection through the alias-aware transport resolver Third relay-plane delivery loss from the 2026-08-09 staging incidents: a delegation batch completed while the gateway was up, the watcher drained the event, and delivery vanished with no log line. _inject_watch_notification resolved its adapter with a literal p.value == platform_name scan of self.adapters — a relay-fronted gateway registers ONE adapter under Platform.RELAY fronting N logical platforms, so 'slack' never matched and the injection returned None ('no gateway route'), silently dropping the completion. The handoff path already documents this exact trap and uses resolve_delivery_transport; the injection path now does the same (native wins; relay eligible only when it fronts the logical platform), with the literal scan kept as fallback for stub runners and exotic platforms. * fix(relay): clamp disconnect drain grace to the runner's adapter-disconnect budget Review finding (JoaoMarcos44, NousResearch#82592): a fixed 5.0s drain in front of the three 1.0s sequential teardown awaits gives an 8.0s worst case inside the runner's 5.0s asyncio.wait_for(adapter.disconnect()) — tripping it cancels teardown mid-drain, skips the fail-pending loop, and leaves outbound callers blocked until _OUTBOUND_TIMEOUT_S (30s). The effective grace is now budget - 3*TEARDOWN - margin (env-aware via the same HERMES_GATEWAY_ADAPTER_DISCONNECT_TIMEOUT the runner reads), so the drain can never push teardown past its caller's budget; a budget too small for any drain disables it cleanly. * test(gateway): pin the final-send suppression contract across a behaviour matrix The gateway skips its own final send when the stream consumer claims the turn final already reached the user. Every incident in that family — NousResearch#71643 (stale finalize snapshot), NousResearch#78541 (payload-less multi-message split), NousResearch#82656 (frozen preview left with a visible cursor) — is the same failure: the consumer claimed delivery for text the platform never rendered, so the corrective send was suppressed and the answer was lost with no retry. Each was fixed with a scenario test pinned to one branch of GatewayStreamConsumer.run(). The got_done handler now has five sibling branches that each set the suppression flags and record a turn-final payload, and nothing checks them as a group: a new branch, or a new early `return True` in _send_or_edit, can reintroduce the class without failing a test. Pin the invariant instead of the branch — if the consumer offers the gateway any signal it would trust, the complete final text must have reached the wire — and assert it across {edit always / dies / never / lies} x {send always / never} x {fresh-final on / off} x {clean / interrupted stream}. The adapter records only frames that actually rendered, so an ACK the platform drops does not count as delivery. 24 honest-transport scenarios hold the invariant as a hard assertion. The 16 lying-transport scenarios are checked too; the single combination that still violates it is reported as an expected failure documenting the open exposure rather than asserting it away. Refs NousResearch#82656 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(gateway,relay): prime relay egress routing for synthetic injections + cap stale completion replay Defect #4 from the 2026-08-09 staging incidents (upgrade-robustness): after every gateway restart the durable async-delegation replay injected completions correctly (post-741663cf1) but their replies bounced at the connector — 'slack egress declined: target not routed to an onboarded tenant'. The relay adapter re-attaches tenant discriminators (metadata.scope_id / metadata.user_id) from per-chat caches warmed ONLY by inbound traffic; synthetic turns race those cold caches on every deploy, scale-to-zero wake, and crash recovery. - relay adapter: prime_routing_cache() — feeds a synthetic event's session-store origin through the same _capture_scope used for real inbound (never raises). - run.py injection path: prime the resolved adapter before handle_message (duck-typed; native adapters unaffected). - async_delegation: 48h staleness cap in restore_undelivered_completions — a pending completion older than the cap is terminally dropped (payload stays queryable) instead of re-run as a fresh full-context turn; the post-restart replay of a July session burned a 102K-token context. Also carried: JoaoMarcos44's suppression behaviour-matrix harness (cherry-picked from NousResearch#82676, authorship preserved) — 39 passed + 1 xfail (the documented ACK-then-drop transport-honesty residue). * test: use recent timestamps in restored-ownership fixtures test_restore_stamps_restored_flag persisted its completion with epoch-era toy timestamps (dispatched_at=1.0), which the new 48h replay staleness cap correctly classifies as stale — the fixture then exercised the cap instead of the restored-flag contract (CI slice 4 failure). Timestamps are now now-relative; the staleness behavior itself is pinned separately in test_relay_injection_egress_priming.py. * fix(gateway,relay): close four review findings on the relay delivery fixes Review follow-ups on this branch (NousResearch#82592): 1. HIGH — classifier/resolver mismatch (falsely-acknowledged loss). _classify_completion_target now returns "deliver" for idle-ended parents, but _resolve_async_delegation_session still dropped every non-compression-ended pin: the durable row was acked at adapter acceptance, then the injection died inside the pipeline with no retry — strictly worse than the honest terminal drop on main, and the delivery leg defect #2's fix depends on did not exist. The resolver now retargets non-user-boundary ends (idle/timeout/ lifecycle) to the chat's current session — session_entry already IS the routing key's current session for the same chat — while user boundaries (session_reset / new_session / user_exit / session_switch) stay fail-closed. Both sides share one module-level _USER_BOUNDARY_END_REASONS so the verdict and the routing decision cannot drift again; a coherence test asserts deliver-verdicts resolve non-None across representative end reasons. 2. HIGH — drain clamp missed adapter-level spend. The effective drain grace budgeted drain + 3x teardown, but RelayAdapter.disconnect spends revocation-monitor teardown + go_idle time BEFORE the transport drain inside the same runner wait_for; worst case still blew the budget and cancelled teardown mid-drain (skipping the fail-pending loop). The adapter now measures its own elapsed time and threads the REMAINING budget into transport.disconnect(budget_s=...); legacy/stub transports without the keyword fall back to the no-arg signature. 3. P1 — _request_response racing disconnect() could register a future after the fail-pending loop already ran, stranding the caller for the full _OUTBOUND_TIMEOUT_S (30s). Fail fast with the same "relay transport closed" error once _closing is set. 4. P1 — _build_process_event_source's last-resort reconstruction dropped scope_id, so a scoped relay completion whose session-store origin was unavailable primed no tenant discriminator and could still bounce off the connector's fail-closed egress guard. scope_id now threads through the reconstructed SessionSource, with a warning when a scoped chat reconstructs without one. All four: RED reproduced with the fix reverted, GREEN after; relay/ delegation delivery families pass (43 + 71 + 179 across the touched suites); full tests/gateway run shows only failures already failing identically on merge base 2446c8b (env/dep issues). * fix(gateway,relay): make pending-frame failure cancellation-safe; persist completion routing origin Two remaining review findings on this branch (NousResearch#82592): 1. Cancellation could strand outbound waiters past the fail-pending loop. transport.disconnect() failed pending futures only at the END of the drain + three teardown awaits; a cancellation landing mid-drain (the runner's wait_for budget, an outer cleanup deadline) skipped the loop entirely and left registered futures unresolved — their callers blocked until _OUTBOUND_TIMEOUT_S (30s). The budget threading added earlier shrinks the window but is not a hard guarantee. The fail-pending loop (and the going_idle ack failure) now run in a `finally`, so no exit path — normal, error, or cancelled — can leave a registered future unresolved. Idempotent: done futures are skipped, a second disconnect() pass is a no-op. 2. Durable completions did not persist their routing origin, so the scope_id threading in the fallback SessionSource reconstruction had nothing to carry on the exact path it exists for (restart replay with session store + source cache gone): the async-delegation event producers never populated scope_id and the durable rows never stored it. Dispatch now snapshots the originating turn's scope_id/user_id/user_name from the session context (_capture_routing_origin — a new HERMES_SESSION_SCOPE_ID contextvar bound by the gateway at session-bind time alongside the existing vars), stores them in the existing task_json payload (no schema migration), and re-attaches them to all three completion-event shapes (live single, live batch, crash-recovery rebuild). The gateway's fallback reconstruction then primes both discriminators after a restart. Tests: cancellation mid-drain -> every pending future resolves with "relay transport closed" (mutation: moving the loop out of the finally goes RED); second-pass disconnect idempotence; end-to-end dispatch -> owner-death recovery -> event carries scope_id -> fallback SessionSource primes it (mutations: dropping the dispatch capture or the task_json persistence both go RED); live completion event carries the origin. 94 passed + 1 xfailed across the delivery/delegation suites; tests/tools delegation family 73 passed (2 collection errors pre-existing on merge base 2446c8b). --------- Co-authored-by: joaomarcos <joaomarcosdias444@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Barclay <ben@nousresearch.com>
Dannyxlm
pushed a commit
that referenced
this pull request
Aug 18, 2026
refactor(relay): centralize protocol descriptors
Dannyxlm
pushed a commit
that referenced
this pull request
Aug 21, 2026
… the relay (gateway half) (NousResearch#85796) * feat(relay): live-card ops — native draft streaming + task cards over the relay (gateway half) NS-658. Three additive ops within contract v1, emitted only when the connector's negotiated descriptor advertises them: {op: draft, chat_id, draft_id, content, final, metadata} {op: task_card, chat_id, card_id, chunks, metadata} {op: task_card_stop, chat_id, card_id, metadata} The gateway side is deliberately dumb: no platform API knowledge, no new config keys. Slack mechanics (chat.startStream/appendStream/stopStream, per-workspace feature-gate cache, send+edit fallback) live connector-side where the platform adapter lives in the relay model. Semantic bridge: base send_draft is Telegram-shaped (draft clears; final is a separate send). Slack native streaming makes the stream THE message. The adapter tracks the open draft per chat and converts the turn-final send() into draft(final=true) so the connector seals the stream instead of posting a duplicate; the stream ts returns as the message identity. A failed frame disarms interception so the edit-based fallback's real send goes through untouched. BEHAVIOR CHANGE (deliberate): relay supports_draft_streaming() now requires the descriptor flag AND the draft op. Flag-only was a latent lie — send_draft inherited NotImplementedError, so a connector setting the flag without the op would have crashed the stream consumer's draft path. supported_ops stays fail-open for legacy (pre-contract) ops; draft/task_card did not exist pre-contract and must not fail open. Task cards ride NousResearch#85476's adapter-agnostic TurnRunner seam (hasattr on send_native_task_card_progress); supports_native_task_cards() is the descriptor probe. Connector half + E2E harness pair follow in the gg repo. * fix(relay): expose native_task_cards_enabled() on the relay adapter Live-canary finding (Alice, staging): the TurnRunner's task-card lane probes adapter.native_task_cards_enabled() (the native Slack adapter's opt-in contract). The relay adapter only offered supports_native_task_cards(), so the hasattr gate failed silently and tool progress stayed on the text path — draft streaming worked, cards never rendered. Alias it to the descriptor probe. * fix(relay): match task-card methods to the TurnRunner's native keyword contract Live-canary finding #2 (Alice, staging): gateway/run.py's card lane calls send/stop_native_task_card_progress with the NATIVE Slack adapter's signature (tasks/title/reply_to/metadata/fallback_text, keyword-only) — PR 85796's relay methods took a positional card_id, so every call raised TypeError('unexpected keyword argument reply_to') in the progress task, repeatedly killing the card publisher (and the retry loop resent the final delivery 4-5x). Card id now derives per turn thread (turn:<reply_to>), thread_ts anchored like draft; title/fallback_text accepted for parity, not forwarded (plan-mode stream renders chunks). * fix(relay): one draft stream per turn for stream-is-the-message adapters Live-canary finding #4 (Alice, staging): the stream consumer bumps draft_id at every tool boundary so Telegram-shaped drafts animate each text segment as a fresh preview. On relay Slack NATIVE streaming a new draft_id opens a brand-new chat.startStream — the user saw one frozen message per segment (stuck streaming cursor ▉, never sealed: only the LAST stream gets the final=true seal) plus the real final; 5-6 cumulative snapshots per turn. Adapters that mark draft_stream_is_message keep ONE stream per turn: tool progress lives in the native task card, and the connector's suffix-delta falls back to whole-text append on prefix mismatch, so segments append cleanly. Telegram-shaped drafts keep the per-segment bump. * fix(relay): don't seal the native stream at tool boundaries — only the turn-final does Live-canary finding #5 (Alice; supersedes the incomplete #4 which was necessary but not sufficient). Root cause CONFIRMED by integration trace (test_live_cards_flow_trace.py, real consumer semantics + real adapter + stub transport): at every tool boundary the consumer calls _send_or_edit(finalize=True), which skips the draft path and issues a real send(); the relay adapter's seal-interception converts THAT into draft(final=true) — sealing the stream once per segment. Timeline showed 3 seals for a 3-segment turn: exactly the frozen cumulative ▉ snapshots seen live (the replaced stream never gets stopStream, keeping its cursor). Fix: for draft_stream_is_message adapters, a segment-break finalize (finalize=True, is_turn_final=False) stays ON the draft path as another cumulative frame; only got_done (is_turn_final=True) falls through to send() and seals. Telegram-shaped platforms unchanged. Trace test now pins the invariant: ONE user-visible message per turn. * fix(relay): strip the text cursor from native draft frames Live-canary finding #6 (Alice) — the ACTUAL duplicate-content mechanism, confirmed by full-flow scan of both sides' code + logs. The consumer appends its text cursor (▉) to every non-final display_text tick. The connector's stream sender diffs CUMULATIVE frames via prefix check: 'abc▉'.startsWith → 'abc def▉' is NEVER a prefix match (the cursor sits mid-string), so deltaFor falls back to whole-text append on EVERY tick — chat.appendStream stacks each full cumulative snapshot (cursor included) into the ONE stream message. Exactly the observed thread: repeated blocks, each ending in a frozen ▉, growing per tick. Fixes #4/#5 were real (one stream per turn now) but this was the last mechanism standing. Native streams render their own typing indicator, so the text cursor is pure noise on this path: strip it from draft frames. Prefix check now holds; every tick appends only its true suffix delta. * fix(relay): seal-interception covers EVERY egress door, not just send() Live-canary finding #7 (Alice): one duplication remained after #6 — the stream froze mid-word with the live indicator (never sealed) and the final posted as a separate message. Log receipt: 'Queued follow-up: final text delivery confirmed; delivering explicit media before continuing' — the turn's final went out via the DELIVERY RESOLVER lane (gateway/delivery.py), which calls send_for_platform() DIRECTLY, bypassing send() and its seal-interception. The open stream never absorbed the final; it arrived as a plain 'send' op → chat.postMessage. Fix: hoist the open-draft check to the top of send() (ahead of the explicit-platform branch) AND add it to send_for_platform() — an open native stream absorbs the turn-final regardless of which egress door it arrives through. The stream IS the message. * fix(relay): failed seal falls back to plain send (PR 85796 AI-review point 1) A turn-final seal that fails at the transport must never swallow the final answer: the stream consumer has already disabled the draft transport for the run, so a failed _seal_open_draft returning success=False meant the user got NOTHING. Both seal-interception sites (send + send_for_platform) now fall through to the regular plain-send path on seal failure, with a warning receipt. Also mitigates AI-review point 2 (sticky _open_draft_by_chat after an abandoned turn): a stale entry's failed seal no longer blocks the next turn's delivery. * fix(relay): arm seal-interception optimistically; never disarm on ambiguous failure (audit G-D1) Deep-audit defect G-D1 (HIGH): the outbound leg is at-most-once on the wire but its ack channel is lossy — send_outbound timeout (30s) and WS-drop 'failures' frequently mean the frame WAS delivered and the connector stream is open. send_draft popped _open_draft_by_chat on any failure, disarming seal-interception while the connector stream lived: the turn-final went out as a plain send → orphaned mid-word stream + complete duplicate final (intermittent; needs a drop/timeout inside the draft window). Fix: arm the entry BEFORE the transport call and keep it armed on failure/exception. Safe in every case: sealing a non-existent stream opens+seals a single complete message connector-side, and a truly failed seal already falls back to plain send at both interception sites. Stale-entry damage is self-healing (one warning + plain send). * fix(relay): gateway-side sealed-draft tombstone — G-D1 arming must not resurrect sealed streams Regression fix on G-D1 (live: 'worse than before' — escalating frozen prefixes). Optimistic arming had no seal-awareness: a straggler frame arriving AFTER the seal re-armed _open_draft_by_chat for the already- sealed draft_id; the next send was converted to draft(final=true) on the tombstoned connector key, which CLEARED the connector tombstone (final frame = new-turn signal), re-opened a stream with cumulative content, and left it frozen — repeating per straggler: 4-5 escalating frozen snapshots. Mirror the connector: _sealed_draft_by_chat records the sealed draft_id per chat (tombstoned BEFORE the seal's transport call); send_draft for a sealed draft_id is a success no-op (content already in the sealed message) and never arms. A new turn's fresh draft_id arms normally. * fix(relay): key stream/card state per (chat, turn anchor) — parallel turns must not collide (finding #10) Live finding #10 (Alice; three concurrent turns in one flat DM): all coordination state was keyed per CHAT on a one-active-turn assumption. Three parallel turns produced: turn B's task card merged into turn A's (both were card 'turn:root' — reply_to is None in flat DMs), B left cardless, and _open/_sealed_draft_by_chat clobbered across writers (3x duplicate finals on the last turn). Per-turn machinery was correct; the keys were not. Fix: _draft_key(chat, metadata) = chat + the turn's thread anchor (inbound stamps thread_ts = event.thread_ts or ts on every top-level message, so each turn has one even in flat DMs). draft arming, seal tombstones, both interception sites, and the task-card id all derive from the same anchor. New trace test pins two interleaved turns: distinct cards, own-stream seals, no leaked plain send, no cross-turn tombstone drops (289 tests green). * fix(gateway): preserve cumulative native stream across tools * fix(gateway): consumer-declared final — the seal carries the true final Three composed fixes for the Slack live-cards duplicate-final class: 1. finish(final_text): TurnRunner passes the completed final_response (verifier footer, completion explainer included) as the authoritative finalize payload. The native-stream seal delivers the TRUE final, so post-stream mutation no longer forks a corrective plain send (#11). 2. Interim-send contract: commentary and segment-tail sends carry a gateway-internal _interim_send marker; relay seal-interception skips them at both egress doors. A mid-turn interim send can no longer seal the live stream and orphan the real final into a duplicate. 3. Queued-follow-up lane reconciles an unconfirmed final by EDITING the consumer's delivered message in place (sealed stream = regular message, chat.update live-verified); plain send only as fallback. This was the actual duplicate lane in the parallel canaries — every duplicated turn logged 'final stream delivery not confirmed; sending first response' (subagent-completion queued inbound), not parallelism. Also: draft frames stay prefix-stable gateway-side (no fence-closing, no segment state reset, no commentary reset for stream-is-the-message adapters; MagicMock-safe 'is True' guards). * test+docs: streaming-contract coverage completeness + maintenance guidelines Coverage: two gaps closed on the consumer-declared-final contract — (1) send_for_platform (the delivery-resolver egress door) honors the _interim_send contract: no seal, marker stripped before the wire; (2) finish(final_text) on a turn that never streamed does not adopt the final (delivery ownership stays with the gateway's normal send path for non-streaming models / tool-only turns). Docs: AGENTS.md 'Known Pitfalls' gains the streaming delivery contract — the four invariants of stream-is-the-message adapters (prefix-stable frames, consumer-declared final, interim-send marker, reconcile-by-edit), each traced to its live incident, plus the live-probed Slack streaming API ground truth and the MagicMock 'is True' guard-style note. * fix(relay): seal transport failure must never silently lose the final (review B1) Two halves of one silent-loss path, live-probed on the review branch: 1. adapter: _seal_open_draft did not catch transport exceptions. A socket drop at seal time raised out of send(), skipping the fail-open plain send entirely. Now: retry the SAME idempotent final frame once (the connector's sealed-key tombstone returns the original stream ts for a repeated final — a retry can never open a second stream or duplicate), then report failure so the caller's fail-open path runs. 2. consumer: the turn-final retry (elif not _already_sent) called _send_or_edit with finalize=False, which re-entered the DRAFT-FRAME branch. Its no-op dedupe compared the adopted final against the last unsealed frame, matched, and returned True with ZERO transport calls — final_response_sent went green, delivered_final_matches reconciled, the gateway suppressed its fallback, and the user never received the answer. finalize=True keeps this retry out of the draft branch. Regression suite: tests/gateway/test_relay_seal_failure.py (3 tests). Mutation evidence in follow-up verification: reverting either half sends the suite red. * fix(relay): draft ids unique across gateway incarnations (review B3) The relay connector tombstones sealed streams by (channel, draft_id) and keeps up to 512 of them; they outlive the gateway process. Relay gateways are disposable BY DESIGN (scale-to-zero), and _draft_id_counter restarted at zero every incarnation — so the first turns after every scale-from-zero in a recently-active channel replayed already-sealed wire identities. The connector answered those frames straight out of the old tombstone: zero Slack API calls, the OLD message ts returned as the new turn's identity, the new answer silently dropped while gateway-side flags recorded success. Seed the counter from wall-clock milliseconds at process start. Ids stay plain ints within the existing contract op; incarnations cannot overlap for realistic turn counts and restart gaps. Regression: tests/gateway/test_draft_id_restart_uniqueness.py — the seed test fails on the old code (seed 0 is not epoch-scale). * fix(relay): stream/card state keyed per TURN, not per thread anchor (review B2) The thread anchor is the wrong coordination identity — simultaneously: - too coarse: two parallel turns replying INSIDE ONE Slack thread share thread_ts. Live-probed on the review branch: turn A's final sealed turn B's stream with A's content while A's own stream stayed open, and B's final degraded to a plain send. - too fragile: a flat DM with no thread metadata degraded to the bare chat id, re-creating the original finding-#10 collision the anchor was meant to fix. _draft_key now prefers the triggering inbound message id (message_id / reply_to_message_id — per-turn by construction; the gateway's Slack thread metadata and the consumer's send path both stamp it), falling back to the thread anchor, then the bare chat. The consumer stamps the same reply_to_message_id on draft frames so frames and the turn-final resolve to one key. Task-card ids share the derivation via _card_key (one helper for send AND stop, so the stop always hits the stream the send opened). Legacy resolver-lane callers with placement-only metadata still seal via _match_open_draft's fallback — but ONLY when exactly one stream is open. With several open, an identity-less send stays a plain send: a duplicate message is recoverable, sealing someone else's stream is not. Regression: tests/gateway/relay/test_relay_turn_keying.py (7 tests). * fix(relay): stream-is-the-message is a Slack semantic, gate it on the descriptor (review B4) draft_stream_is_message was hardcoded True on the relay adapter class, i.e. for EVERY relay platform. The base send_draft contract is Telegram-shaped — the draft clears client-side and the final arrives as a separate real send that becomes the history message. With the flag forced on, any non-Slack connector advertising the draft op had its turn-final intercepted into draft(final=true): probed on the review branch with a telegram descriptor, the op stream was [draft(final=false), draft(final=true)] and NO send — no history message would ever be posted. Gate the flag on the negotiated descriptor platform (slack), and skip arming seal-interception entirely when it is off. A future platform with genuine stream-is-the-message native streaming should advertise it via the descriptor rather than widening the platform check by guesswork. Regression: tests/gateway/relay/test_relay_stream_semantics_gating.py (4 tests: gating both ways, telegram final is a real send, slack final still seals). * fix(gateway): mark every mid-turn status lane interim — heartbeats must not seal the stream (review B5) Seal-interception treats the first unmarked send to an armed (chat, turn) key as the turn-final. The consumer's own interim lanes (commentary, tail flush) carry _interim_send, but four gateway-side lanes that fire DURING a streaming turn did not: - long-running heartbeat (default every 180s — probed live: at 3 minutes it sealed the live stream with '⏳ Working — 3 min', the real final posted as a duplicate, and later frames were silently swallowed by the seal tombstone) - inactivity warning - plain-text approval fallback (button lane failed) - background-review notice Add _interim_metadata() beside _non_conversational_metadata and wrap all four call sites. The marker is gateway-internal; the relay adapter strips it before the wire (existing behavior, pinned by test). Note for follow-up: the opt-out shape remains fragile — any FUTURE unmarked mid-turn send lane re-creates this bug. Inverting the contract (explicitly mark the one turn-final send) is the durable fix but touches every adapter's final-delivery path; deliberately kept out of this review-fix series. Regression: tests/gateway/test_interim_send_lanes.py (4 tests). * fix(gateway): interrupted/incomplete turns must not adopt the diagnostic as the stream final (review B6) The finish(final_text) adoption gate checked only 'not failed', but the interrupt/abort returns in agent/conversation_loop.py are {completed: False, interrupted: True, final_response: 'Operation interrupted during …'} with NO failed key. Adopting that diagnostic: 1. sealed the user's streamed partial answer over with the interrupt text (stream-is-the-message: the seal rewrites the whole message), and 2. recorded the diagnostic as the turn-final payload, so delivered_final_matches reconciled and the gateway suppressed its own error-delivery path — the diagnostic became the ONLY thing delivered. Enumerated all 27 final_response-bearing return shapes in conversation_loop.py: every non-happy-path shape carries completed: False (several with a diagnostic final_response and neither failed nor interrupted — retry exhaustion, truncation, codex-incomplete); the happy path routes through turn_finalizer.finalize_turn (completed=True). Gate is therefore: not failed AND not interrupted AND completed is not False. Results lacking the completed key entirely (older callers/test doubles) keep the previous behavior. Regression: tests/gateway/test_stream_final_adoption_gate.py (6 tests, incl. a source-level pin on the run.py call site). * fix(relay): task-card transport failures degrade to failed SendResults (review B7) send_native_task_card_progress and stop_native_task_card_progress let transport exceptions escape. The stop runs inside the progress loop's finally block on the turn-cleanup path, and the post-cancel awaits in gateway/run.py caught only CancelledError — a socket drop during a card publish/stop therefore aborted cleanup BEFORE the final-delivery bookkeeping ran. Three layers, outermost defends any adapter: - both adapter methods catch transport exceptions and return failed SendResults (progress is advisory; the TurnRunner's text fallback already handles failure results) - the progress loop's finally wraps the stop (best-effort; the connector seals orphaned card streams on its own via recycling/eviction) - the cleanup awaits log-and-continue on non-cancellation errors so final-delivery bookkeeping always runs Regression: tests/gateway/relay/test_relay_task_card_failures.py. * fix(relay): a dying turn seals its native stream instead of orphaning it (review B8) Stale-generation exits (/new, /stop mid-stream) and cancellations returned from the consumer's run() with the native stream still open: - the Slack message kept its live streaming indicator forever (the cancellation best-effort edit only runs when _message_id exists, and the native draft path deliberately keeps it None); - the adapter's armed interception state survived the turn, so the next turn on the same key could inherit it and seal a dead draft_id. New adapter op abandon_open_draft(chat, content): seals in place with the text already on screen (the consumer passes its last delivered frame) — the seal adds nothing and claims nothing; delivery flags are never set, so the gateway's normal paths still own whatever happens next. Best-effort by contract (failure reported, never raised); the connector reaps truly orphaned streams via recycling/eviction. The consumer calls it from both death paths: the stale-generation early return and the CancelledError handler. Regression: tests/gateway/test_stream_abandon_on_turn_death.py (4 tests, incl. the next-turn-inheritance hazard). * fix(relay): bound the draft/seal coordination dicts (review M1) _sealed_draft_by_chat's key embeds a per-turn identity, so every completed turn wrote a permanent entry — unbounded growth for the life of a long-running gateway process (the docstring said 'one entry per chat', which stopped being true when the key gained the turn anchor). _open_draft_by_chat could grow the same way via abandoned entries. FIFO-evict both at 512 entries — the same idiom as the sibling bounded cache (_auto_thread_by_chat, capped at 256) and the same size as the connector's own tombstone store. The straggler window the tombstone exists for is seconds long; FIFO is more than enough. Regression: tests/gateway/relay/test_relay_state_bounds.py. * fix(relay): explicit connector rejection disarms interception; exceptions stay armed (review P3) The G-D1 optimistic-arming change silently dropped disarm-on-failure entirely: after an EXPLICIT connector rejection (success=False result — not a transport ambiguity), interception stayed armed even though the stream consumer disables the draft transport on that failure and falls back to edit-based streaming. Its turn-final would then be converted into a seal on a stream the connector just told us is unusable. test_draft_failure_result_propagates claimed to cover this ('must NOT leave seal-interception armed') but passed for an unrelated reason: the stub's canned failure also failed the SEAL, whose fail-open path did the plain send. Split the two semantics and pin each honestly: - explicit rejection (result success=False): disarm — turn-final is a real send (test_draft_failure_result_propagates, now testing what its comment says) - transport exception: ambiguous, stay armed — turn-final still seals (test_draft_transport_exception_keeps_interception_armed, the G-D1 contract) Also corrects commit ba3a24a's claim ('a failed frame disarms interception so the edit-based fallback's real send goes through untouched') to hold again for the rejection case it described. * fix(relay): lost acks are ambiguous, not rejections — on the RESULT channel too (review r2, finding 1) The production ws transport does not raise on ack timeout — it returns {"success": False, "error": "relay outbound timed out"}. The round-1 ambiguity handling keyed entirely on the exception channel, so the shape production actually produces was misclassified as a definite connector rejection. Probed on the head: - lost SEAL ack: skipped the idempotent retry, fell straight to a plain send — duplicate final whenever the seal had actually applied; - lost FRAME ack: the round-1 disarm-on-rejection fired — interception disarmed, frozen native stream beside a plain final. This re-created the original G-D1 ambiguous-ack defect on the result channel. Contract now spans both channels: - transport: the ack-timeout branch tags ambiguous=True. The fail-fast branches (closing / not connected) never sent anything and stay unmarked — they are definite non-delivery. - adapter frame path: ambiguous results keep interception armed (same as exceptions); only definite rejections disarm. - adapter seal path: one shared _attempt() classifier — exception and ambiguous result both mean "unknown"; the SAME idempotent frame is retried once (connector tombstone returns the original stream ts for a repeated final). Only after both attempts stay ambiguous does the caller's fail-open plain send run: a possible duplicate after double ack loss beats a silent loss, and double ack loss on one socket almost always means the transport is down for the plain send too. Regression: tests/gateway/relay/test_relay_ack_ambiguity.py (6 tests, incl. a source-of-truth check that the transport tags the timeout branch and leaves fail-fast branches unmarked). * fix(relay): stream semantics + draft capability resolve per CHAT, not per primary (review r2, finding 2) One RelayAdapter fronts N platforms (Phase 1.5): descriptors accumulate per platform on the transport and egress is tagged per chat — but the round-1 gate keyed draft_stream_is_message and supports_draft_streaming() off the PRIMARY scalar descriptor. Probed on the head: - Slack primary + Telegram chat: the Telegram chat's turn-final was intercepted into draft(final=true) — no real Telegram history message; - Telegram primary + Slack chat: the Slack chat was denied native streaming entirely. Resolve both through _descriptor_for_chat — the same per-chat machinery max_message_length already uses (added for the identical class of bug: the primary's 39000-char cap over-sending into Discord 400s): - new stream_is_message_for_chat(chat_id) on the adapter; arming and NotImplementedError gating use it. The class attribute remains as the single-platform value and legacy-probe fallback. - supports_draft_streaming() gains an optional chat_id kwarg (base signature updated; single-platform adapters ignore it). The consumer passes chat_id with a TypeError fallback for out-of-tree adapters. - the consumer's four draft_stream_is_message reads collapse into one _stream_is_message() helper that prefers the per-chat probe (class-resolved, MagicMock-safe) over the attribute. Platform-name inference ("slack") stays deliberate: a descriptor-level semantic field is the right eventual contract but is a cross-repo wire change — noted for the gg follow-up so future platforms advertise the semantic explicitly. Regression: tests/gateway/relay/test_relay_multiplatform_semantics.py (5 tests: both starvation directions, scalar fallback, per-chat capability gate). * fix(gateway): split delivery + authoritative footer reconciles by suffix, not full resend (review r2, finding 3) The _FINAL_TEXT adoption guard refuses wholesale adoption on split turns — correct (NousResearch#78541: sealed heads would repeat inside the tail) but it was absolute: a post-split verifier footer never entered the ledger, delivered_final_matches() reported a mismatch, and the gateway resent the ENTIRE body+footer after the split chunks (the #11 duplicate class, one level up). When the authoritative final strictly prefix-extends the split ledger, the missing suffix is the only undelivered content: append it to the live tail and the ledger, so the finalize carries it and the recorded payload reconciles. Non-prefix rewrites keep the full-resend fallback — a rewrite cannot be patched onto sealed heads. Regression: tests/gateway/test_split_final_suffix_reconcile.py (3 tests: suffix rides the tail + reconciles, rewrite still mismatches, unsplit adoption unchanged). * fix(relay): cancellation mid-seal restores open state so abandon can close the stream (review r2, finding 4) _seal_open_draft pops the open entry and writes the local tombstone BEFORE awaiting transport I/O — correct ordering for the straggler race, but CancelledError is not an Exception: a cancel during the await bypassed all failure handling, leaving the remote stream live (visible streaming indicator until connector eviction) while the local state said 'nothing open'. The consumer's abandon pass — added for exactly this turn-death case — found nothing to close and no-oped. On CancelledError: restore the open entry, drop the premature tombstone (only if it is still ours), re-raise. The abandon path then seals the stream in place with the on-screen text. Regression: tests/gateway/relay/test_relay_seal_cancellation.py (2 tests: state restoration, and end-to-end cancel→abandon→remote seal). * fix(relay): thread anchors are placement, not turn identity — revive the placement-only fallback (review r2, finding 5) _match_open_draft's single-open-stream fallback was dead for its primary intended callers: metadata carrying thread_ts/thread_id (placement-only resolver lanes) was classified as having 'turn identity', so those sends never reached the fallback — probed: a plain final posted beside the still-open turn-keyed stream. Only per-turn MESSAGE ids are identity now. Thread-anchored and bare callers share the fallback: absorb into the chat's open stream when EXACTLY one is open; stay a plain send when several are (duplicate is recoverable, wrong-stream seal is not). Callers WITH a message id whose key misses never fall back — their identity is authoritative and a miss means the stream belongs to a different turn. Regression: 4 new tests in test_relay_turn_keying.py (thread-anchored seal, both ambiguous-stay-plain shapes, id-mismatch never steals). * fix(relay): random process nonce for draft-id seeding (review r2, follow-up 6) The epoch-millisecond seed (round-1 B3 fix) mitigates the restart-replay class but is not a uniqueness guarantee: two gateways starting in the same millisecond, a forked process inheriting the class state, or a clock step backwards can all mint colliding wire identities against the connector's per-(channel, draft_id) tombstone store. Seed from secrets.randbits(49) instead: collision probability negligible, no clock dependence, and ids + realistic per-process turn counts stay comfortably inside the connector's JS number range (draft_id?: number, 2^53). Regression test now spawns two real interpreters and asserts their seeds differ — the exact scale-to-zero restart shape, and both start within the same second so a clock-locked seed would fail it. * fix(relay): stamp per-turn Slack egress identity — cache is fallback only (R3-5) The connector (gateway-gateway#210) fills chat.startStream's recipient_user_id / recipient_team_id — required by Slack when streaming to a channel — from metadata.user_id / metadata.scope_id. The gateway stamped only slack_team_id per-turn and left user_id (and scope_id) to RelayAdapter._with_scope, whose per-chat caches are keyed on chat_id alone and overwritten by every inbound message: with users U1 and U2 running overlapping turns in one channel, U2's arrival overwrote the cache before U1's stream opened, and U1's stream carried U2 as recipient_user_id. _thread_metadata_for_source now stamps scope_id and user_id from the turn's OWN source (setdefault — explicit values win), so identity is turn-scoped data on the wire. _with_scope is unchanged and fill-only: the caches keep serving restart/synthetic sends that carry no per-turn identity, which is all they were ever safe for. Mutation evidence: reverting the run.py hunk sends test_thread_metadata_stamps_per_turn_user_and_scope and test_concurrent_turns_carry_their_own_identity red; restore returns green. The _with_scope fill-only tests pass on both trees (existing correct behavior, now pinned against regression). --------- Co-authored-by: Ben Barclay <ben@nousresearch.com>
Dannyxlm
pushed a commit
that referenced
this pull request
Aug 25, 2026
… loop When the Python interpreter begins teardown (user closes hermes, SIGTERM, OOM-kill), every executor-backed operation raises 'cannot schedule new futures after interpreter shutdown'. The outer except handler in run_conversation caught this error but did not recognize it as fatal — it kept retrying (API calls #4, #5, #6) until max_iterations, each time hitting the same dead executor and printing another traceback. The fix adds an early check: if sys.is_finalizing() or the error matches the 'cannot schedule new futures' pattern, break immediately with a clean interpreter_shutdown exit reason instead of retrying. The codebase already had this pattern in cron/scheduler.py and agent/tool_executor.py — the conversation loop just wasn't using it.
Dannyxlm
pushed a commit
that referenced
this pull request
Aug 25, 2026
…he shell When the TUI exits while the post-turn background review fork is still mid-request, every further API attempt raises 'cannot schedule new futures after interpreter shutdown'. The conversation loop treated this as a retryable API error: un-gated ❌ prints leaked onto the user's shell AFTER the TUI exited (call #4, #5, #6...) and the loop retried a doomed request until the interpreter froze the thread. Fix the class, not the site: - tools/interpreter_shutdown.py: single shared shutdown predicate (matches both CPython message variants + sys.is_finalizing()). - cron/scheduler.py, agent/tool_executor.py: existing per-site predicates now delegate to the shared home (tool_executor previously matched only the fuller variant). - agent/conversation_loop.py: inner retry handler recognizes the shutdown signal and abandons the turn — one log warning, no print, no traceback, no debug dump, no retry; outer handler gets the same guard for shutdown errors raised outside the API call. - The outer handler's bare print() now honors suppress_status_output (set by the background-review fork) instead of bypassing it. Refs NousResearch#55924 NousResearch#58720 (same class in cron delivery), adjacent to NousResearch#90683.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges current
NousResearch/hermes-agent:maininto Danny's CloudSeed Hermes fork while preserving the fork-only product patches.Canonical topology
NousResearch/hermes-agent:mainDannyxlm/hermes-agent:main/opt/cloudseed-immutable/hermes/current/home/ubuntu/worktrees/hermes-agent/source-monitorThe publication fork is intentionally retained. It currently carries CloudSeed-specific Codex ephemeral-context behavior and managed immutable update/desktop visibility that official upstream does not yet contain.
Current distance at creation
95d303138b0ad2271ae7c34327aaed8f82c249dfc892ca25e01afd45367c5ba49f681808c8187dd7Safety boundary
This PR only reconciles Git history. It does not update the live cloud box, run
hermes updatein place, change an immutable selector, restart a gateway, or alter desktop credentials. The dashboard update button remains a request-only candidate action for managed CloudSeed runtimes.The PR stays draft until conflicts are resolved and the fork-only Codex/update suites plus relevant upstream checks pass.