Skip to content

chore: sync upstream changes from NousResearch/hermes-agent - #1

Merged
ap0ught merged 5750 commits into
mainfrom
sync-upstream-20260717-hermes-agent
Jul 17, 2026
Merged

chore: sync upstream changes from NousResearch/hermes-agent#1
ap0ught merged 5750 commits into
mainfrom
sync-upstream-20260717-hermes-agent

Conversation

@ap0ught

@ap0ught ap0ught commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

This PR syncs hermes-agent with upstream NousResearch/hermes-agent.

  • Upstream branch: main
  • Fork branch: main
  • Upstream commits brought in: 5750

Included upstream commit subjects:

This is a straight upstream sync; no fork-specific changes were introduced.

Christopher-Schulze and others added 30 commits July 16, 2026 07:17
…e multiplexer's

On a multiplexed gateway the process-level active profile is always the
multiplexer's own (usually "default"), so /profile answered "default" in
every chat regardless of which profile actually served it — making
per-chat persona routing look broken when it was working.

Report source.profile (stamped by the /p/<profile>/ URL prefix, a
per-credential adapter, or a room->profile map) and resolve the
displayed home under that profile's runtime scope, mirroring the scoped
/reset banner (NousResearch#59003). Unstamped sources fall back to the active
profile and default home, so single-profile gateways are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review follow-up: honor source.profile and enter _profile_runtime_scope
only when gateway.multiplex_profiles is on, mirroring the gating in
_run_agent, _reset_notice_session_info, and _resolve_profile_for_key.
When multiplexing is off (the default) a stamped source is ignored and
/profile reports the active profile and default home, byte-identical to
before this PR.

The stamped-source test now enables multiplexing (it previously
exercised the ungated path under the default config), and a new
regression asserts the stamp is ignored when multiplexing is off.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Feishu was unconditionally listed in _PORT_BINDING_PLATFORM_VALUES,
causing the multiplexer to reject ALL Feishu secondary profiles. But
Feishu in websocket mode (the default) uses an outbound WebSocket
connection and does NOT bind an HTTP port — only webhook/callback mode
needs a listener.

Add _platform_binds_port() helper that checks connection-dependent
platforms (currently only Feishu) against their actual config before
raising MultiplexConfigError. Feishu websocket profiles are now allowed;
Feishu webhook profiles still raise as before.

Fixes NousResearch#52563
…rch#52563 integration

With the mode-conditional check centralized, default (websocket) Feishu
no longer counts as port-binding in the secondary batch report — pin the
fixture to connection_mode=webhook so the test still exercises the
multi-platform report path.
Subset of PR NousResearch#61985: _make_adapter_auth_check gains a profile_name
parameter and secondary-profile adapters (started in
_start_one_profile_adapters) bind it, so the auth callback's
SessionSource resolves the routed profile's adapter and pairing store
instead of silently falling back to the default profile. This is the
gap left open by the NousResearch#65629 merge — adapter-internal auth checks (e.g.
Slack thread-context fetch) fire outside the wrapped message handler.

The PR's authz_mixin.py hunks are dropped: main's _auth_env (merged via
PR NousResearch#65629) already covers the scoped allowlist reads they targeted.
Rebuilt from PR NousResearch#61283 onto the /p/<profile>/ routing world (7aa21e3):
_profile_scope(None) now enters the DEFAULT profile's runtime scope when
multiplexing is active instead of returning nullcontext(). api_server is
a port-binding platform living on the default profile, so plain requests
(no /p/ prefix) are the primary path — with fail-closed get_secret they
crashed with UnscopedSecretError on the first credential read (NousResearch#61276).

All three wrapped call sites (chat-completions executor, /v1/runs agent
construction and _run_sync) inherit the fix through the one seam.
Single-profile gateways keep the no-op. Regression tests ported from
the original PR to the _profile_scope seam.

Fixes NousResearch#61276
A fallback chain entry can name its API key via key_env (or the
api_key_env alias) per the fallback-providers docs, but only the gateway
path resolved it — TUI/desktop, cron, and CLI setup fallbacks ignored it,
so a fallback provider whose key lives in a non-standard env var never
resolved on those surfaces.

Centralize the inline-api_key-then-key_env lookup in
hermes_cli/fallback_config.resolve_entry_api_key() and use it at all four
fallback resolution sites (tui_gateway, cron scheduler, gateway runner,
CLI setup mixin); the CLI mixin also gains the base_url passthrough the
other surfaces already had.

Salvaged from PR NousResearch#43861 (surgical reapply — the original branch predates
the NousResearch#65264 fallback restructuring).
… open

Follow-up to the salvaged NousResearch#65636: if the dedup UPDATE or the retried
CREATE INDEX raises, log and continue — the unique title index is an
optimization and must not block SessionDB initialization.
… max_length

BasePlatformAdapter.truncate_message() splits an over-length reply into
chunks. When max_length is 0 or 1 (and the content is longer), the split
loop makes no progress and spins forever, appending empty chunks — an
unbounded hang that pins a CPU and grows the chunk list until OOM:

  - headroom = max_length - INDICATOR_RESERVE - ... goes negative, and the
    < 1 fallback (max_length // 2) is also 0;
  - so _cp_limit is 0, the region is empty, no split point is found, and
    split_at falls back to _cp_limit (0);
  - chunk_body is remaining[:0] = "", remaining never shrinks, loop repeats.

The same stall is reachable under utf16_len (Telegram) whenever the next
char is a surrogate-pair emoji wider than the whole budget, so _cp_limit
maps to 0 codepoints even for max_length >= 2.

A pathological max_length is not hypothetical: the relay capability
descriptor's max_message_length is taken verbatim from the connector
(gateway/relay/descriptor.py from_json) and assigned straight to the
adapter's MAX_MESSAGE_LENGTH (gateway/relay/adapter.py), and 0 is a
documented "no limit" value there.

Guarantee forward progress: floor headroom at 1, and floor the
final split_at at max(1, _cp_limit) so at least one codepoint is always
consumed per iteration. Normal splitting is unaffected (both floors only
bite when the budget is already degenerate).

Adds regression tests that run truncate_message on a worker thread and
fail if it doesn't return: max_length 0/1/2 terminate and preserve every
character, and the utf16 emoji case terminates too.
…or boundary

Follow-up to the truncate_message split-loop floor. Two review points:

- CapabilityDescriptor.from_json trusted the wire max_message_length
  verbatim, so a connector advertising 0 ('no limit') — or a buggy one
  sending 0/negative — produced a descriptor whose bound flowed straight
  into the adapter's MAX_MESSAGE_LENGTH and truncate_message. Normalize
  it to the documented 4096 default (mirrors from_platform_entry's
  'or 4096' and docs/relay-connector-contract.md), fixing the degenerate
  budget at its source rather than only surviving it downstream.

- Document the truncate_message length contract for a budget too small
  for one codepoint (max_length=1 with a 2-unit surrogate pair under
  utf16_len): the chunk intentionally exceeds max_length by that one
  indivisible codepoint, because emitting it whole preserves content
  where the alternatives are data loss or an infinite loop.

Tests: from_json normalizes 0 and negative bounds to 4096 and passes a
real positive bound through unchanged; the sub-codepoint budget emits
whole codepoints with no data loss (all emojis preserved) and a chunk
that necessarily exceeds the 1-unit budget.
…crashing

_check_auth gates every OpenAI-compatible API server endpoint. It compared
the client's raw bearer token against the configured key with
hmac.compare_digest on two str values. compare_digest raises TypeError on
a str containing non-ASCII characters, and the token comes straight from
the Authorization header — so a request with a single non-ASCII byte in
the key (a stray unicode char, a smart quote, a pasted BOM) crashed the
handler with an unhandled TypeError. Every endpoint calls _check_auth
without a try/except, so the framework turned that into a 500 Internal
Server Error instead of the intended 401 Invalid API key.

Compare as bytes, matching web_server.py's dashboard-token check
(hmac.compare_digest(auth.encode(), expected.encode())). Encoding both
sides keeps the timing-safe comparison and its semantics identical for
valid keys while making a non-ASCII token fail closed with a clean 401.

Adds regression tests: a non-ASCII bearer token returns 401 (no raise),
and a non-ASCII configured key still authenticates against its exact
value.
… the endpoint

_validate_signature backs the public webhook receiver. It compared each
attacker-supplied signature/token header (GitHub X-Hub-Signature-256,
GitLab X-Gitlab-Token, generic X-Webhook-Signature / -V2, and the Svix v1
header) against a computed hex/base64 digest with hmac.compare_digest on
two str values. compare_digest raises TypeError on a str containing
non-ASCII characters, and the header is raw client input on an
unauthenticated endpoint — so any internet client could POST a single
non-ASCII byte in the signature header and raise out of the handler,
returning a 500 instead of a clean 401. Fail-closed, but an on-demand
crash of the request path.

Route all five comparisons through a small _hmac_str_equal() helper that
encodes both sides to UTF-8 bytes before the constant-time compare
(compare_digest has no ASCII restriction on bytes). Semantics are
unchanged for valid signatures; a hostile non-ASCII header now fails
closed with a rejection instead of raising.

Adds regression tests: non-ASCII GitHub/GitLab/generic/V2 signature
headers return False (no raise), and a non-ASCII configured secret still
matches its exact token value.

Also maps drexux0@gmail.com in scripts/release.py AUTHOR_MAP.
…gression

The fix routes the Svix v1 comparison through _hmac_str_equal too, but the
existing non-ASCII tests only exercised the GitHub/GitLab/generic V1/V2
branches. Add a Svix case (valid svix-id + fresh svix-timestamp so it
reaches the v1,<sig> compare) with a non-ASCII signature, which raised
TypeError before the fix and now rejects cleanly.
…g sites

Same bug class as the salvaged NousResearch#65305/NousResearch#65307: hmac.compare_digest (and
secrets.compare_digest) raise TypeError when given a str containing
non-ASCII characters, and these call sites feed it raw request input.
Compare as UTF-8 bytes everywhere:

- gateway/platforms/msgraph_webhook.py: clientState from request body
- gateway/platforms/whatsapp_cloud.py: hub.verify_token query param +
  X-Hub-Signature-256 header (comment claimed 'works on str' — it
  doesn't for non-ASCII)
- plugins/platforms/feishu: verification token + x-lark-signature
- plugins/platforms/raft: bridge token header
- plugins/platforms/line: X-Line-Signature
- plugins/platforms/sms: X-Twilio-Signature
- tools/code_execution_tool.py: sandbox RPC token (both loops)

Regression tests for the two gateway-core sites (msgraph, whatsapp).
…ntir Bearer auth

When the user's main provider is a named custom_providers entry exposing an
Anthropic Messages surface (e.g. Palantir Foundry's
/api/v2/llm/proxy/anthropic, custom LiteLLM/Bedrock proxies), auxiliary
tasks (title generation, compression, web extract, session search, etc.)
returned HTTP 404 NOT_FOUND for every call.

Root cause: `_resolve_auto` collapsed any `custom:<name>` main provider
to plain `"custom"` and passed runtime_base_url as explicit_base_url.
This landed in `resolve_provider_client`'s anonymous-custom arm
(`if provider == "custom":`), which unconditionally calls
`_to_openai_base_url` — that helper strips a trailing `/anthropic` and
substitutes `/v1` (designed for MiniMax/ZAI which expose both surfaces).
The result for Palantir is `/api/v2/llm/proxy/v1`, which does not exist
on the proxy — every auxiliary call 404s. The runtime `api_mode=
anthropic_messages` flag was discarded by this arm.

Fix: split the conditional so only the literal `"custom"` provider takes
the anonymous-custom path; `custom:<name>` keeps its full `custom:<name>`
string when handed to `resolve_provider_client`, where the
named-custom-provider arm (added in earlier work) honours the entry's
`api_mode` and routes through `AnthropicAuxiliaryClient` against the
original `/anthropic` URL.

Also: extend `_requires_bearer_auth` in `anthropic_adapter.py` to
recognise palantirfoundry hosts so the SDK sends `Authorization: Bearer`
instead of the default `x-api-key` (Palantir's proxy rejects x-api-key
with 401).

Verified end-to-end against a live Palantir Foundry deployment with both
claude-4-6-opus and claude-4-7-opus models — `generate_title` returns
real titles instead of 404ing.  Regression-tested:

  - anonymous `custom` (with base_url) still routes to OpenAI wire
  - built-in NVIDIA provider unchanged
  - custom-without-base_url still falls through to Step-2 chain
…d-custom routing, drop dead key assignment, tighten Palantir host match

Review follow-ups on the cherry-picked NousResearch#36043 commit:

1. Guard the custom:<name> passthrough with a _get_named_custom_provider
   lookup. The PR unconditionally kept the full custom:<name> string, which
   broke config-less runtime custom providers (NousResearch#34777 regression — entries
   that exist only in the live runtime, not config.yaml): the named arm
   found no entry and resolution fell through to Step 2. Now custom:<name>
   only takes the named arm when a config entry actually exists; otherwise
   it collapses to the anonymous-custom arm with the runtime endpoint,
   preserving pre-PR behavior.

2. Drop the dead 'explicit_api_key = runtime_api_key' assignment (and its
   misleading comment) in the named-entry branch. resolve_provider_client's
   named-custom arm derives the key exclusively from the entry's
   api_key/key_env and never reads explicit_api_key, so the assignment was
   a no-op. Wiring precedence in was not justified: for a named custom
   provider the runtime key IS the entry's key (set_runtime_main sources it
   from the same config), so deletion is the honest option.

3. Tighten the Palantir Bearer-auth check from a loose substring match
   ('palantirfoundry' in normalized) to a hostname match via
   base_url_host_matches(..., 'palantirfoundry.com'), so path segments or
   lookalike domains containing the string no longer trigger Bearer auth.

Tests: named-custom anthropic_messages end-to-end routing (full name kept,
AnthropicAuxiliaryClient at the original /anthropic URL, no /v1 rewrite)
plus Palantir Bearer-auth positive and substring-false-positive cases.
…gate flash

User themes (`~/.hermes/dashboard-themes/*.yaml`) reach the SPA only
after `/api/dashboard/themes` resolves at React mount.  The bundle paints
the first frame with the default Hermes Teal canvas — the
`<link rel="stylesheet">` carries `:root{--background-base:#041c1c}`,
the bundled `presets.ts` defines the same surfaces in JS — and then
`ThemeProvider.applyTheme(<user theme>)` flips the inline CSS variables
on `documentElement` once the API response lands.  Visible to the user
as a green canvas behind the loading SPA on every reload when the active
theme is non-default.

Built-in themes do not suffer the same effect because their full
definitions ship inside the bundle, so the SPA already has the palette
before first paint.

This patch closes the gap on the backend side: `_serve_index()` injects
a `<style id="hermes-theme-bootstrap">` block inside `<head>` with the
six critical CSS variables (`--background-base`, `--color-background`,
`--midground-base`, `--color-midground`, `--font-sans`,
`--font-base-size`) plus an `html, body` rule painting the body in the
target palette.  Because the inline `<style>` follows the bundle's
`<link>` in DOM order and matches the same `:root` specificity, the
later declaration wins the cascade — the static canvas behind the SPA is
already the right colour before any JavaScript runs.

`_render_active_theme_bootstrap_css()` looks up the active theme through
the existing `_discover_user_themes()` helper.  No-op for built-in
active themes (empty string returned, no `<style>` injected).  No new
API endpoints, no config flags, no frontend changes.

After `ThemeProvider` mounts and `applyTheme()` writes the same
variables as inline styles on `documentElement`, the values match what
the bootstrap block set, so there is no second-paint discrepancy on the
critical CSS variables.
…le flows through vars

Review fixes for the inline critical-CSS bootstrap (PR NousResearch#36024):

1. Variable names now match what the bundle actually consumes.
   --color-background, --color-midground, --font-sans and
   --font-base-size appear nowhere in web/src; the real tokens are:
     --background-base / --midground-base  (layerVars(), context.tsx)
     --theme-font-sans / --theme-base-size (typographyVars(), and
       index.css html{font-family:var(--theme-font-sans);
       font-size:var(--theme-base-size)})

2. Stale-rule bug: the injected html,body rule previously baked in
   literal hex/font values. Because the <style> block sits after the
   bundle's <link> at equal specificity and is never removed, switching
   themes in the picker left the old canvas/font until reload. The rule
   now references the same CSS variables instead of literals —
   applyTheme() writes those vars as inline styles on documentElement,
   which outrank this block in the cascade, so runtime theme switches
   re-resolve the rule automatically. No frontend change needed.
…ction

Server-side coverage for the critical-CSS shim (PR NousResearch#36024 salvage):

- user theme → style block emitted with ONLY real bundle variable names
  (--background-base/--midground-base from layerVars(),
  --theme-font-sans/--theme-base-size from typographyVars()/index.css),
  and an html,body rule expressed via those vars so runtime theme
  switches never leave a stale canvas/font
- built-in / unknown / non-string active theme → no block
- malformed theme YAML and load_config() exceptions → no crash, index
  still serves
- </style> breakout attempt in a theme value stays escaped
- mount_spa integration: block present in <head> for user themes,
  absent for built-ins
NousResearch#65678)

DeepSeek's own API (api.deepseek.com) reports context-cache hits as
top-level usage.prompt_cache_hit_tokens / prompt_cache_miss_tokens
(prompt_tokens = hit + miss), not the OpenAI nested
prompt_tokens_details.cached_tokens shape. Neither normalize_usage()
nor the chat_completions transport's extract_cache_stats() read those
fields, so direct DeepSeek sessions always showed 0 cache-hit tokens:
invisible in accounting, mis-billed at the full input rate, and 0%
cache display.

Both layers now fall back to prompt_cache_hit_tokens when the nested
shape is absent; the nested value wins when both are present (proxies).

Fixes NousResearch#61871.
The platform callback verifier can do blocking network I/O (e.g. the
google-chat adapter fetches Google signing certs on a cache miss), which
would stall the event loop if called inline. Run sync verifiers via
asyncio.to_thread (await coroutine verifiers directly), and treat a
crashing verifier as a 401 rather than a 500 through the dispatch path —
a broken verifier must never admit an event.
OutThisLife and others added 22 commits July 17, 2026 14:34
…itch-prewarm

perf(desktop): pre-warm profile backends and gateway sockets on hover intent
…-model-picker

fix(desktop): session-scope fast mode, surface profile ownership + pinned model override
…re-tests

test(desktop): cover the review store
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…-status-sync

refactor(desktop): derive working/attention session sets from $sessionStates
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ents re-arm busy (NousResearch#66485)

Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com>
…search#66486)

Adds a --from DIR flag to scripts/dev-sandbox.sh that copies an existing
HERMES_HOME directory into the sandbox as the starting point before the
command runs. Lets you spin up a sandbox pre-populated with your real
config, sessions, skills, etc.

  scripts/dev-sandbox.sh --from ~/.hermes hermes desktop

Design:
- cp -a dir/. dest/ — preserves perms, symlinks, hidden files
- Clobber guard: only seeds when sandbox HERMES_HOME is empty, so
  re-running --persistent doesn't blow away existing sandbox state
- Validates: errors on nonexistent dir, missing arg, flag-like arg,
  empty --from=
- Supports both --from DIR and --from=DIR forms
- Backwards compatible: no --from = unchanged behavior
…e the build path

The staleness check added in NousResearch#66052 resolved the timeout from env,
config.yaml, and the default only, while the build path also reads the
honcho.json host block (timeout/requestTimeout). With a timeout
configured in honcho.json, the two permanently disagreed: every
no-config get_honcho_client() call — i.e. every HonchoSessionManager
.honcho property access — interpreted the mismatch as a config change
and tore down and rebuilt the client, defeating the singleton on the
hot path it was meant to protect.

Teach the check to read honcho.json through the same host-aware chain
as from_global_config, memoized on the file's mtime_ns so the per-call
cost stays one stat(). A genuine honcho.json timeout change is now also
detected, extending NousResearch#57437 to that config surface.
…donly

The staleness check's bespoke mtime memo keyed only on the user
config.yaml, but load_config() merges the managed-scope config
(HERMES_MANAGED_DIR/config.yaml, /etc/hermes) whose leaf keys win. A
managed honcho.timeout with no user config.yaml made the memo cache
'no timeout' while _build resolved the managed value — the same
perpetual-rebuild mismatch this PR fixes for honcho.json. A managed
timeout edit was likewise invisible while the user file's mtime stayed
put.

load_config_readonly() is already cached on both files' signatures plus
the env-ref snapshot, so use it instead of duplicating that
invalidation logic; the defensive deepcopy the old memo existed to
avoid is skipped by the readonly variant. Drive the rebuild test
through a real config.yaml and add a HERMES_MANAGED_DIR regression
test covering stable reuse and managed-timeout edits.
…update-stream-output

fix(update): stream update child output to the live log (PYTHONUNBUFFERED)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Extends the app-server event bridge (make_codex_app_server_event_bridge)
to fire the authoritative stable-ID tool_start_callback /
tool_complete_callback alongside the existing tool_progress_callback,
and route item/reasoning/summaryDelta through the reasoning channel.

Surfaces that render structured tool cards (TUI, desktop) — not just
progress bubbles — now correlate live cards with the projected history
entry after a resume, because the call ids mirror CodexEventProjector's
_deterministic_call_id. Guarded per-callback so a broken display
consumer can't tear down the codex turn loop.

Grafted from PR NousResearch#65412 by @HaiderSultanArc onto the merged bridge (the
PR's parallel _codex_live_event implementation was reconciled into the
bridge's existing _fire_tool_started/_fire_tool_completed helpers).
- codex-app-server-runtime.md: add a Live display section covering the
  stream/reasoning/tool-card bridge and show_commentary gating.
- release.py: AUTHOR_MAP entries for HaiderSultanArc, jjadeo-oss, juanfradb
  (the latter two for forthcoming follow-up salvages of NousResearch#62396 / NousResearch#18050).
Exercise the real transport path for long session ids, including stable hashing and bounded body/header cache keys.
Fold NousResearch#62349's broader provider-boundary handling into the header fix: bound top-level and xAI override keys again at preflight after middleware, preserve unrelated headers, and cover boundaries and collisions.

Co-authored-by: Nick Taylor <nicktaylor@TheWorldofNick-Lappy.local>
Keep invalid persisted preset names fail-closed, list the valid configured choices, and classify the local lookup failure as deterministic so it reaches Desktop immediately.
Normalize stale api_base keys to each mem0 provider's accepted URL field before Memory.from_config, without mutating the saved config.
…ousResearch#66373)

* feat(attribution): conflict-free contributor mappings via contributors/emails/ directory

The AUTHOR_MAP dict in scripts/release.py was a merge-conflict magnet:
every concurrent salvage PR appended entries to the same lines of the
same file, so parallel PRs re-conflicted on every merge to main.

New system: one file per email under contributors/emails/ — filename is
the commit-author email, first non-comment line is the GitHub login.
File additions never conflict, so any number of PRs can add mappings
concurrently.

- scripts/release.py: AUTHOR_MAP is now LEGACY_AUTHOR_MAP (frozen)
  merged with the directory at import time (directory wins). All
  existing consumers (resolve_author, contributor_audit.py) unchanged.
- scripts/add_contributor.py: idempotent CLI to add a mapping; refuses
  conflicting reassignments (incl. against the legacy map), validates
  email/login shapes.
- contributor-check.yml: attribution gate now accepts a mapping file OR
  a legacy entry; failure message prints the exact add_contributor
  command. Also auto-resolves bare <login>@users.noreply.github.com
  emails is intentionally NOT added (kept id+login form only, matching
  previous behavior).
- contributor_audit.py: guidance now points at add_contributor.py.
- tests/scripts/test_contributor_map.py: 12 tests covering loader,
  merge precedence, CLI idempotency/conflict/validation, subprocess E2E.

* feat(ci): one-shot per-file flake retry in the parallel test runner

A failing test FILE is re-run once in a fresh subprocess. Pass-on-retry
counts as green but is loudly reported in a '⚠ FLAKY' summary section
(with both attempts' output preserved) so the flake gets fixed instead
of eating a full-run rerun. Deterministic failures fail both attempts —
regressions cannot be laundered green.

- --file-retries N / HERMES_TEST_FILE_RETRIES (default 1, 0 disables)
- E2E verified: simulated first-run-fail flake goes green with banner;
  deterministic failure still exits 1; retries=0 restores old behavior.

This converts the dominant CI failure mode (one timing-sensitive test
flaking a 4600-test shard, requiring a manual 10-minute rerun and an
agent triage loop) into a self-healing retry that costs one file's
runtime.

* test(approval): loosen wall-clock perf bounds 0.15s -> 2.0s

These guard against catastrophic regex backtracking (seconds-to-minutes
class), but 0.15s is within scheduler-stall noise on loaded shared CI
runners — test_max_accepted_separator_free_input_is_fast failed a CI
shard this week on runner load alone. 2.0s still catches the regression
class with zero flake surface.

* fix(ci): job timeouts everywhere + retries on all network installs

Reliability pass over every workflow:
- timeout-minutes on all 21 jobs that lacked one (a hung job previously
  burned the 6-hour default runner budget)
- ./.github/actions/retry wrapped around every network-fetching install
  that lacked it: pip installs (deploy-site, skills-index), npm ci
  (deploy-site website, upload_to_pypi web + ui-tui), uv sync (docker
  test deps). Deterministic build steps (npm run build) deliberately
  NOT retried — split into separate steps so a real build failure fails
  fast instead of retrying 3x.

* docs(agents): document the file-retry flake policy

* fix(ci): curl retries on deploy hook + skills-index probe

* fix(ci): kill the remaining transient-failure classes in workflows + Dockerfile

From the workflow reliability audit:
- tests.yml: duration-cache restore had NO restore-keys while saves use
  run_id-suffixed keys — the cache never matched once, so LPT slicing
  always ran blind and unbalanced slices pushed heavy files toward the
  per-file timeout. One-line restore-keys fixes slice balancing.
- Label gates (lint ci-reviewed, supply-chain mcp-catalog-reviewed):
  'gh pr view || true' turned an API blip into 'label absent' → false
  BLOCKING failure. Now 3x retry, and API failure is reported as an API
  failure instead of a missing label.
- detect-changes action: compare API retried before failing open (was
  silently running all lanes on any blip).
- uv-lockfile-check: 'uv lock --check' resolves against PyPI — retried
  so registry blips don't read as 'lockfile stale'.
- docker.yml merge job: imagetools create retried (Docker Hub eventual
  consistency on just-pushed digests).
- Dockerfile: apt-get Acquire::Retries=3; s6-overlay ADDs converted to
  curl --retry 3 (ADD cannot retry; checksums still enforced); npm
  --fetch-retries=5; playwright chromium fetch retried 3x.
- Advisory artifact uploads (per-slice durations, ci-timings report)
  get continue-on-error so an artifact-service blip can't fail a green
  test slice.

* fix(tests): kill the two root-cause flakes — leaking pre-warm timer + env-dependent provider list

- test_tui_gateway_server.py: session.create / non-eager session.resume
  arm a 50ms threading.Timer (_schedule_agent_build) that outlives its
  test and fires into the NEXT test's _make_agent mock, racily
  corrupting captured state (the recurring session_resume shard
  failures). Replaced the per-test whack-a-mole stub with a module-wide
  autouse fixture; the 3 worker-lifecycle tests that genuinely need the
  deferred build opt back in via @pytest.mark.real_agent_prewarm (new
  marker in pyproject).
- test_api_key_providers.py: PROVIDER_ENV_VARS is now derived from the
  live PROVIDER_REGISTRY instead of a hand-list that had drifted
  (missing HF_TOKEN / DEEPINFRA_API_KEY) — resolve_provider('auto')
  tests failed on any machine with HF_TOKEN exported. E2E-verified with
  HF_TOKEN/DEEPINFRA_API_KEY set: 42/42 pass.

* test: de-flake 30 timing-sensitive test files for loaded CI runners

Root-cause fixes from the flake audit (session-DB mining + repo sweep):

Event-based sync instead of sleep-sync:
- title_generator: mock sets threading.Event, wait(10) replaces
  sleep(0.3) hoping the daemon thread got scheduled
- docker zombie_reaping / profile_gateway: poll-for-state helpers
  replace fixed 1-3s sleeps (s6 transitions + SIGCHLD reaping are async)
- process_registry tree test: select()-bounded readline replaces an
  unbounded blocking read (parent wedge now fails THIS test with a clear
  message instead of an opaque rc=124 file kill); SIGTERM grace 1s->2s
  (the 1s partition window mid-interpreter-startup is how a child PID
  escaped the live-system guard in CI)

Timeout raises (loaded 8-way-sliced runners see ~5s scheduling floors;
all of these complete in ms-to-1s when healthy so the raises cost
nothing on green runs):
- subprocess/thread waits <= 2s raised to 10-15s across mcp_tool,
  mcp_circuit_breaker, mcp_reconnect_retry_reset, mcp_parked_self_probe,
  mcp_cancelled_error_propagation, registry, clarify_gateway, interrupt,
  voice_cli_integration, docker_environment, session_store_lock_io,
  planned_stop_watcher, cli_interrupt_subagent, thread_scoped_output
  (joins now also assert not is_alive() so stragglers fail loudly)
- wall-clock discrimination ceilings loosened where the guarded hang is
  10x larger: local_background_child_hang 4s->10s, interrupt_cleanup
  setup 5s->20s + pgid-exit 30s->60s, mcp_stability grandchild spinup
  5s->15s, protocol/gil-starvation fast-handler 0.5s->2s,
  iso_certify_seam 1.5s->5s, wait_for_mcp_discovery 0.1s->1s
- narrow assertion windows widened: honcho first-turn wait 0.4..0.65 ->
  0.25..2.0 (property is bounded-not-hung, not an exact wall-clock);
  compression fork-lock TTL 1s->3s (12 refresh chances per lease);
  compression-lock expiry margins symmetric (ttl 0.05->0.5, sleep 1.0)
- telegram hung-DNS bound 1.0->1.4 (fake hang is 1.5s — must stay under)

* fix(tests): repair indentation from de-flake batch edit

* fix(tests): harden env isolation and replace remaining sleep-sync races

The full 42k-test run and complete npm check surfaced three more classes:

- Environment isolation: local ~/.honcho defaultHost and SSH_* variables
  leaked into Python/TUI tests. Pin the default Honcho host in the
  hermetic fixture, isolate the one fallback test from ~/.honcho, and
  blank SSH_* around terminalSetup tests. This flipped 20 false failures
  back to deterministic behavior on developer machines.
- Background-thread sleep-sync: Honcho async writer tests patched
  time.sleep globally, then busy-polled with that same mocked sleep. Under
  full-suite load the poller could starve the writer. Each test now waits
  on an Event emitted by the exact flush/retry transition; 30/30 passed
  under 15-way contention.
- Desktop streaming: the test slept 80ms and assumed a 500ms timer could
  not fire before its assertion. A loaded runner descheduled the test for
  >500ms and both chunks arrived. Producer controls now gate second-chunk
  and completion transitions explicitly.

Also make file-retry observability complete: a self-healed flaky file now
prints BOTH attempts' full output in the FLAKY summary. Two behavioral
runner tests prove pass-on-retry is green+loud+traceback-preserving, while
a deterministic failure remains red.

* refactor(ci): use gh bot pat, better retries

refactor(ci): use retry action for PR label fetch
the retry action now captures stdout as a step output, so it can serve
double duty: retry + output capture for commands like 'gh pr view' whose
result must be consumed by later steps.

Retry action gains:
- 'stdout' output (heredoc-delimited to preserve newlines)
- tee to temp file so stdout still streams to the job log
- step id 'retry' for output reference

Both lint.yml and supply-chain-audit.yml now use the retry action
directly with 'command: gh pr view ...' and read
steps.<id>.outputs.stdout.

ci: use AUTOFIX_BOT_PAT for all gh CLI / GitHub API auth

Replace secrets.GITHUB_TOKEN and github.token with
secrets.AUTOFIX_BOT_PAT across all workflows and composite actions
that use the gh CLI or GitHub API. The PAT has consistent permissions
across fork PRs (where GITHUB_TOKEN is read-only), avoids API rate
limit sharing with the default token, and is already used by
js-autofix.yml for the same reasons.

19 sites swapped across 9 files:
- lint.yml (3): label fetch, comment post/edit, comment update
- supply-chain-audit.yml (5): scan, critical comment, unbounded dep
  comment, label fetch, mcp-catalog comment
- lockfile-diff.yml (1): PR comment post/update
- skills-index-freshness.yml (1): issue creation on degraded probe
- skills-index.yml (2): index build, trigger deploy workflow
- upload_to_pypi.yml (2): release view poll, release upload
- ci.yml (1): timings report
- deploy-site.yml (2): skills index crawl
- detect-changes/action.yml (1): compare API call

---------

Co-authored-by: ethernet <arilotter@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@ap0ught
ap0ught merged commit 3be8255 into main Jul 17, 2026
@ap0ught
ap0ught deleted the sync-upstream-20260717-hermes-agent branch July 17, 2026 21:19
ap0ught pushed a commit that referenced this pull request Aug 8, 2026
The #1 patch failure class in production (state.db mining, 250k-window)
is a re-send of an edit that already landed: 'old_string and new_string
are identical' (299 occurrences) plus a share of hunk-not-found errors
where the new text is already in the file. These errored, sending
models into re-read/re-patch loops.

New tools/fuzzy_match.is_already_applied(content, old, new) — a
conservative check requiring (1) non-trivial new_string (>=8 chars),
(2) EXACT presence of new_string, (3) old_string gone (unless
identical). Wired into three sites:

- patch_replace (replace mode): returns success + no_change: true +
  an explicit note instead of the identical-strings / no-match error.
- V4A validation phase: an already-applied hunk validates as a no-op
  so multi-hunk patches no longer fail wholesale when one hunk landed
  in a prior call.
- V4A apply phase: mirrors the same skip so the two phases agree.

Genuine no-matches (new text absent) and half-applied renames (old
text still present) keep their error behavior — covered by tests.
ap0ught pushed a commit that referenced this pull request Aug 8, 2026
process(action='wait') hitting its window returned status='timeout'
with a terse note — models read it as an error and re-issued identical
waits (process is the #1 exact-duplicate tool call in production: 511
dupes in a 400k-msg window; wait is 57% of all process actions).

The timeout result now carries:
- process_running: true — machine-readable 'this is a status, not a
  failure'
- an explicit note: 'Wait window of Ns elapsed — the process is still
  running. This is not an error. Uptime: Ms.' plus the right next step:
  when notify_on_complete is set, 'you will be notified on exit — do
  more work instead of waiting again'; otherwise a pointer to
  notify_on_complete for next time.
- the clamp note (requested > max) now composes with the status note
  instead of replacing it.

Exited/interrupted results are unchanged.
ap0ught pushed a commit that referenced this pull request Aug 8, 2026
…e-review #1)

revoke_commit_admission() used to invoke the holder-qualified lease
release unconditionally — including while an admitted commit was still
mutating SessionDB — letting a second compressor acquire the durable
lock mid-commit and interleave with the first commit's writes.

The admission_revoked flag store stays lock-free, but the lease-release
decision now coordinates with the fence lock:
- revoke acquires the fence lock non-blocking; on success no commit can
  be in flight (an admitted commit retains the lock until finish_commit)
  and the release runs immediately, still under the lock so a racing
  begin_commit cannot slip between the check and the release.
- on failure the release is deferred: finish_commit() re-checks
  _admission_revoked and performs it AFTER the commit completes (prompt
  even if the worker thread is later parked), and the begin_commit
  refusal path does the same for a revoke that lost the race to a
  transient lock-setup/cancel boundary. All paths are idempotent with
  the worker's own outer cleanup (DB release is holder-qualified).

Invariant encoded + tested: no second compressor can acquire the durable
lock while an admitted commit is still mutating; after a post-revoke
commit finishes the lease is released promptly. Both regressions
(revoke-during-commit deferral, revoke-before-commit immediate release +
refused begin_commit) are sabotage-verified.
ap0ught pushed a commit that referenced this pull request Aug 18, 2026
Addresses both review findings on the remote-gateway download PR:

1. Unbounded buffering (finding #1). fetchBuffer / fetchBufferViaOauthSession
   accumulated the entire response (then copied it again via Buffer.concat)
   before saveGatewayFile even opened the save dialog, so a large gateway file
   could exhaust the native process. Both auth paths now stream: once response
   headers arrive the connect timeout is cleared, the filename is derived, the
   save dialog is shown, and the body is piped to the chosen destination with
   backpressure. A read/write error tears down the stream and unlinks the
   partial file. The byte-moving, data-URL decoding, and filename/path helpers
   are extracted into gateway-file-download.ts so they're unit-testable without
   Electron.

2. No fallback for older gateways (finding NousResearch#2). saveGatewayFile required the new
   /api/fs/download route. Desktop and the remote gateway update independently,
   so a gateway predating this PR 404s. Added a 404-only compatibility fallback
   to the existing capped /api/fs/read-data-url route (bounded, so it only
   serves smaller files — enough to keep older backends working).

Tests: gateway-file-download.test.ts covers streaming, backpressure,
error-cleanup (unlink on write/response error), data-URL decoding, filename
derivation (incl. traversal reduction), and 404 detection;
gateway-file-download-transport.test.ts asserts both transports stream (no
whole-body Buffer.concat) and that the 404 fallback is wired. Both registered
in the desktop platform test list. Server-side /api/fs/download tests
(streaming + sensitive-file reject) already pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ap0ught pushed a commit that referenced this pull request Aug 18, 2026
…-renders (NousResearch#81726)

The scoped find walker wraps transcript text nodes in <mark> elements that
React does not own. Assistant responses stream through markdown-text.tsx,
which rebuilds the markdown DOM on every delta, and a new message is
appended whenever the assistant answers — so a re-render of a changed
region detaches the marks we inserted, dropping the user's highlights while
the bar stays open.

Watch the captured scope with a MutationObserver and re-wrap only when an
unmarked occurrence of the active query actually reappears. The observer is
gated behind a re-entrancy flag while the walker is mutating, coalesced to
one re-apply per microtask, torn down when the bar closes or the query
clears, and restores the active ordinal so a mid-stream re-render doesn't
reset the user's place to match #1. An append that adds no matching text is
a no-op; re-wrapping only fires when highlights genuinely went stale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ap0ught pushed a commit that referenced this pull request Aug 18, 2026
Two independent bugs let a deleted profile reappear / leave orphaned
resources on next launch:

1. hermes_cli/profiles.py's backend-process scanner required argv[0] to
   resolve to an executable literally named "hermes". Electron's
   pool-backend spawn resolves the hermes console-script shim's path and
   execs it via the interpreter directly (python3 /path/to/hermes ...), so
   argv[0] reports as "python3" and the scanner never matched the running
   backend -- delete removed the profile's files but left its live backend
   process running (still bound to a port via uvicorn), which
   accumulates across repeated delete/recreate cycles.
2. The desktop sidebar's ProfileRail only refreshed its cached profile
   list once, on mount, so a delete/create/rename from another surface
   (another window, or the CLI) left a stale ghost entry until something
   unrelated triggered a refetch. Note: a delete via this window's own
   Manage-Profiles view already refreshes the shared $profiles atom
   ProfileRail subscribes to (confirmed by reading refreshProfiles() and
   handleConfirmDelete()) -- this fix only covers the cross-window/cross-
   process staleness gap, not a duplicate of the already-merged
   NousResearch#57329's Manage-Profiles rail-refresh work.

Fix 1: recognize a python-interpreter argv[0] exec'ing a hermes-named
console-script shim via argv[1]. Fix 2: refresh the profile list on window
focus/visibilitychange, matching the existing pattern used elsewhere in
the sidebar (sidebar/index.tsx, use-background-sync.ts, star-map.tsx,
use-gateway-boot.ts all use the same focus+visibilitychange pattern).

## Related work already on main

PR NousResearch#57329 (merged) fixed the *headline* symptom from issue NousResearch#52279
(deleted profile respawns) via a different, non-overlapping mechanism:
routing profile-delete through the primary backend instead of spawning a
fresh pool backend, plus a separate recreation guard in
ensure_hermes_home() (NousResearch#49435, merged) that makes a backend spawned into a
deleted profile's directory raise FileNotFoundError instead of silently
recreating it.

This PR is NOT a duplicate of that fix. Verified: even with both of those
merged, a backend process that survives because of gap #1 above still
holds a bound port via uvicorn -- it just can no longer resurrect the
profile directory. That's real resource-hygiene, not a symptom already
covered. Gap NousResearch#2 touches a different file/component (ProfileRail /
profile-switcher.tsx) than NousResearch#57329's rail-refresh half (which touched the
Manage-Profiles view's own $profiles.ts / index.tsx) and covers a
distinct staleness path (cross-window/cross-process, not same-window
delete-then-refresh).

Tests: tests/hermes_cli/test_profiles.py -- 156 passed (existing +
regression coverage for the argv[0] python-interpreter detection case).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.