Skip to content

fix(hermes): restore gateway core to upstream, clear conformance gate - #4

Merged
ragnos-dev merged 1049 commits into
mainfrom
fix/hermes-upstream-conformance-2026-06-29
Jun 29, 2026
Merged

fix(hermes): restore gateway core to upstream, clear conformance gate#4
ragnos-dev merged 1049 commits into
mainfrom
fix/hermes-upstream-conformance-2026-06-29

Conversation

@ragnos-dev

@ragnos-dev ragnos-dev commented Jun 29, 2026

Copy link
Copy Markdown

Summary

Restores gateway/platforms/base.py and gateway/run.py to the upstream merge-base so the hermes-upstream-conformance gate passes (it was the last blocker on RAGnos just preflight --fast). Two RAGnos-specific behaviors that had been edited into upstream core are relocated or reverted; both stay functional.

1. Own-message reply context -> moved into the Photon adapter (no core edit)

  • Removed reply_to_is_own_message from MessageEvent (base.py) and the branch that consumed it in gateway/run.py.
  • The Photon adapter now prefixes reply_to_text with "your previous message: " for tapbacks on bot messages, so the generic gateway handler emits [Replying to: "your previous message: ..."] - same agent-visible signal, zero core change.

2. Immediate-reconnect -> reverted to upstream's ~30s retry (operator decision: Option A)

  • Reverted the two gateway/run.py reconnect edits (next_retry without the +30 delay, and the early break on _failed_platforms in the idle sleep loop).
  • Fast Photon sidecar-crash recovery is preserved by the adapter's own sidecar-death watcher + the 30s->300s reconnect backoff; only the first retry is delayed up to 30s instead of instant.

Verification

  • git diff <merge-base> -- gateway/run.py gateway/platforms/base.py: empty (both match upstream).
  • hermes_upstream_conformance.py --json: ok: true, violations: [], fork_posture ok.
  • pytest tests/gateway/test_platform_reconnect.py test_platform_reconnect_fd_leak.py test_runner_fatal_adapter.py test_reply_to_injection.py tests/plugins/platforms/photon/test_reactions.py: 59 passed.
  • Updated test_retryable_runtime_error_reconnects_immediately -> test_retryable_runtime_error_queued_with_retry_delay to assert the 30s queueing.

Merging this clears the RAGnos preflight Hermes conformance block.

benbarclay and others added 30 commits June 18, 2026 14:46
…hronos)

Phase 4E (E.1 + E.2). The inbound side of Chronos: NAS POSTs the agent when a
one-shot fires; the agent verifies a NAS-minted JWT and runs the job.

E.1 — plugins/cron/chronos/verify.py:
- verify_nas_fire_token(token, expected_audience, jwks_or_key, issuer): verifies
  signature against the NAS JWKS (RS/ES family; symmetric rejected), aud == this
  agent, exp/nbf, iss, and purpose == "cron_fire" (so a general agent JWT can't
  be replayed against the fire endpoint). Returns claims or None; never raises.
  Crypto delegated to PyJWT[crypto] (already a declared dep) — no hand-rolled
  JWT, no new dependency. No key configured → refuse (never unsigned-decode a
  security boundary).
- get_fire_verifier(): pluggable indirection so the DQ-4 escape hatch
  (direct per-job cron-key) can swap in with no handler change.

E.2 — gateway/platforms/api_server.py:
- POST /api/cron/fire (registered only when _CRON_AVAILABLE). Authenticated by
  the NAS-JWT via get_fire_verifier() — NOT API_SERVER_KEY (NAS holds no API
  key; this is the only inbound that triggers remote job execution, so it gets
  its own purpose-scoped check). Verifier args come from cron.chronos.* config.
  401 on bad/missing/forged token. 400 on missing job_id. On success: 202 +
  fire_due runs in the background (so a long agent turn never trips NAS's HTTP
  timeout); the store CAS claim inside fire_due de-dupes a scheduler retry.

Tests:
- test_chronos_verify (11): REAL RS256 signing — valid→claims, wrong-aud,
  missing/wrong purpose, expired, wrong-iss, tampered-signature (attacker key),
  no-key-refuse, empty-token, JWKS-URL key resolution, get_fire_verifier.
- test_cron_fire_webhook (5): valid→202+fire, invalid→401+no-fire, missing
  token→401, missing job_id→400, and fire path does NOT require API_SERVER_KEY.
api_server regression suites (214) green.

E.3 (NAS endpoints) is a separate cross-repo PR; the wire contract lands next
(docs/chronos-managed-cron-contract.md).
…AS contract

Phase 4F (F.1 + F.2 + F.3, agent side). F.4 is the operator-run live smoke
(needs a NAS deployment); recorded in the PR, not code.

F.1 — on_jobs_changed wiring:
- cron/scheduler.py: _notify_provider_jobs_changed() — resolve the active
  provider, call on_jobs_changed(), swallow errors. Lives in scheduler.py (not
  jobs.py) so the store stays free of provider imports (no import cycle).
- Wired at the consumer surfaces AFTER a successful mutation: the cronjob model
  tool (tools/cronjob_tools.py, create/update/remove/pause/resume) — which the
  `hermes cron` CLI also routes through — and the REST handlers
  (gateway/platforms/api_server.py, same five). Built-in's no-op default = zero
  behavior change on the default path. Sleeping-agent direct jobs.json writes
  (no tool/CLI/REST) are covered by reconcile-on-wake in start().

F.2 — config: cron.chronos.{portal_url,callback_url,expected_audience,
nas_jwks_url}. All non-secret; the agent holds no scheduler creds and the
outbound provision call reuses the existing Nous token (no token key). Additive
deep-merge key, no version literal.

F.3 — docs:
- docs/chronos-managed-cron-contract.md: authoritative agent↔NAS wire contract
  (the three agent-cron endpoints + inbound /api/cron/fire + the 3-hop trust
  model + at-most-once/re-arm semantics). This is what the NAS-side agent builds
  against.
- cron-internals.md: "Managed cron (Chronos) for scale-to-zero" section.
- cli-commands.md: cron.provider accepts chronos + the cron.chronos.* keys.
- User docs name no scheduler vendor (QStash is a NAS-internal detail).

INVARIANT re-verified: zero qstash/upstash hits across plugins/cron, gateway,
hermes_cli, tools, website/docs (the one remaining repo hit is an unrelated
Context7 MCP comment in tools/mcp_tool.py).

Tests: test_jobs_changed_notify (5) — notify calls provider hook, swallows
errors, built-in harmless, tool create/remove notify. Full cron + chronos +
webhook + config + api_server_jobs suites green (504 in the cron+chronos+webhook
run).
…e) (NousResearch#48242)

The gateway half of relay Phase 3. On a MANAGED boot with relay configured and
no secret pinned, the runtime self-provisions its relay credentials IN-PROCESS:
resolve the agent's own Nous access token (resolve_nous_access_token) -> POST
the connector's /relay/provision asserting its own endpoint + route keys ->
set GATEWAY_RELAY_ID/SECRET/DELIVERY_KEY into os.environ so the immediately-
following register_relay_adapter() reads them and dials out authenticated.

No human, no enrollment token, no disk write — the creds live only in process
memory (save_env_value refuses under managed anyway, and keeping the secret off
any volume is the stronger posture). Stateless: process-env creds don't survive
a restart, so a managed container re-provisions every boot; the connector's
rotation window covers a still-connected prior instance. An explicitly-pinned
GATEWAY_RELAY_SECRET is respected (skip). Self-hosted is unchanged: humans keep
using `hermes gateway enroll`.

Endpoint provenance is gateway-asserted (GATEWAY_RELAY_ENDPOINT +
GATEWAY_RELAY_ROUTE_KEYS, env or gateway.relay_* config) — uniform code path
whether the operator sets it (self-hosted) or NAS stamps it (hosted, the only
case NAS knows the public URL). Both absent -> outbound-only provisioning
(credentials, no inbound routes). The connector scopes the asserted endpoint to
the verified tenant, so it stays within the security model.

- gateway/relay/__init__.py: relay_endpoint(), relay_route_keys(),
  _provision_url(), _post_provision(), self_provision_if_managed() (never
  raises — a provision failure logs and boots without relay auth).
- gateway/run.py: call self_provision_if_managed() immediately before
  register_relay_adapter() in the startup path.

Tests: 12 unit (trigger logic, respect-pinned-secret, in-process env wiring,
endpoint+routes vs outbound-only, fail-soft on token/connector failure);
mutation-checked (drop is_managed guard / pinned-secret guard -> tests fail).
Cross-repo live E2E driver lands on the connector side (depends on this).

EXPERIMENTAL: relay auth scheme may change until >=2 Class-1 platforms validate.
…state (NS-501) (NousResearch#48243)

Importing a backup wrote every file from the zip over the target home
wholesale. On a hosted instance this clobbered gateway_state.json with the
source machine's last recorded run/desired state — driving the container-boot
reconciler (container_boot._read_desired_state, which only auto-starts a
gateway whose state is "running") off stale/foreign state and leaving the
gateway stuck "starting", disconnected from the Nous portal.

Add _IMPORT_SKIP_NAMES (gateway_state.json, gateway.pid, cron.pid,
gateway.lock, processes.json) and skip them by basename in run_import, so both
the root profile and named profiles preserve the target's own runtime state.
This mirrors what container_boot._STALE_RUNTIME_FILES already sweeps on every
container boot, and protects against older backups that predate the
backup-side exclusions. The import summary reports which files were preserved.

This is the second half of NS-501 (filed separately as NS-508): the upload
502 was fixed in NousResearch#47663; this fixes the import-breaks-the-instance half.
…SON (NS-501) (NousResearch#47663)

* fix(dashboard): stream file uploads via multipart instead of base64 JSON

The dashboard file manager uploaded files (including backup/restore zip
archives) by reading them client-side with FileReader.readAsDataURL and
POSTing a base64 data URL inside a JSON body to /api/files/upload. For a
large backup this (a) inflates the payload ~33%, (b) buffers the whole
file plus its decoded copy in memory, and (c) reliably trips an upstream
proxy body-size/timeout limit, surfacing as a 502 with the upload
appearing to hang indefinitely (NS-501). Dashboard-only hosted users have
no shell fallback to place the archive, so backup restore was unusable.

Add a streaming multipart endpoint POST /api/files/upload-stream
(UploadFile + Form) that reads the request body in 1 MiB chunks straight
to a sibling temp file, enforces the existing 100 MB size cap as it
streams (413 on overflow, before buffering the whole file), and
atomically renames into place so a partial/aborted/over-limit upload
never clobbers an existing file. The frontend api.uploadFile now sends
multipart/form-data (raw bytes, no base64, browser-set boundary) and
FilesPage passes the File object directly; the dead readAsDataUrl helper
is removed. The legacy base64 JSON endpoint stays for backward compat.

FastAPI's UploadFile/Form require python-multipart, which is NOT pulled in
by fastapi itself, so it is added to the base deps, the [web] extra, and
the tool.dashboard lazy-install set (kept in sync).

Validated: 5 new endpoint tests (roundtrip, multi-chunk >1 MiB,
over-limit 413 without clobbering + no temp-file leak, overwrite=false
conflict, forced-root traversal containment); existing base64 tests still
pass; web typecheck + vite build clean; and a real uvicorn server E2E
(5 MB multipart upload -> HTTP 200 in 0.21s, exact byte match) plus a
30 MB TestClient roundtrip confirm constant-memory streaming end to end.

Reported via beta (NS-501).

* build(deps): regenerate uv.lock for python-multipart (NS-501)

CI ran uv lock --check / uv sync --locked which failed because the
python-multipart dependency add was not reflected in uv.lock. Regenerate
the lockfile (resolves to 0.0.20, matching the [web] extra pin) after
merging current main.
)

Resolves conflicts from the OpenViking churn that merged after NousResearch#32445 was
opened (NousResearch#48042/NousResearch#47662 session-switch + write hardening, NousResearch#47311/NousResearch#47973):

- plugins/memory/openviking/__init__.py: keep both __init__ field groups
  (the PR's _runtime_start_* alongside main's _prefetch_threads/_shutting_down).
- tests/plugins/memory/test_openviking_provider.py: keep BOTH the PR's new
  setup-validation tests and main's session-switch/concurrency tests (disjoint
  additions to the same region).

Two fixes layered while reconciling (contributor work otherwise preserved):

- Restore the merged tenant-header contract (NousResearch#22414/NousResearch#21232). The PR had changed
  _VikingClient defaults to '' and made empty account/user OMIT the tenant
  headers; main's contract is that empty falls back to 'default' and the
  X-OpenViking-Account/User headers are ALWAYS sent (ROOT API keys need them).
  Reverted the constructor to 'account or os.environ.get(..., "default")' and
  updated the two PR tests that asserted the omit-when-empty behavior.

- Close a secret-file TOCTOU in the setup writers. _write_env_vars and
  _write_ovcli_config wrote the api_key/root_api_key file and chmod 0600
  AFTERWARD, leaving a world-readable window on newly-created files. Added
  _precreate_secret_file() to create with 0600 before any secret bytes land.
…python-multipart (NS-501)

Follow-up to NousResearch#47663 (streaming multipart upload), fixing two issues that
landed with it.

1. Temp file leaked on client disconnect. The streaming upload endpoint's
   except chain caught only HTTPException / PermissionError / OSError — all
   Exception subclasses. asyncio.CancelledError, raised when a browser aborts
   a large upload mid-stream (the exact NS-501 scenario), is a BaseException,
   so it bypassed every except clause and reached a finally that only closed
   the file handle and never unlinked the temp file. Every aborted large
   upload orphaned a partial `.{name}.*.upload` file (up to ~100 MB) in the
   target directory. Cleanup now lives in finally, keyed on a `renamed`
   success flag, so the temp file is removed on every non-success exit
   including BaseException paths. Added test_stream_upload_cleans_temp_on_cancellation,
   which fails on the pre-fix code (leaks the temp file) and passes with the fix.

2. python-multipart pinned to ==0.0.27 instead of ==0.0.20. The package was
   already resolved at 0.0.27 transitively (via daytona) before NousResearch#47663; the
   explicit ==0.0.20 pin in the [web] extra and the tool.dashboard lazy-install
   set downgraded it. Bumped both to ==0.0.27 and regenerated with `uv lock`,
   keeping the lockfile coherent. The base dependency stays >=0.0.9,<1.
…ch#48261)

PR infographics are decorative visual hooks for a PR body, not repo
artifacts. The established convention (commit 5772e63, "chore: drop
in-repo infographic/ directory; keep PR-body URLs only", NousResearch#30854) is to
hotlink an externally-hosted image so GitHub camo-proxies it inline,
leaving zero binary footprint in the tree.

Two such assets had been committed anyway and are referenced nowhere in
the codebase:

- docs/assets/ns504-chat-session-reconnect.png (1024-equiv, NS-504 PR
  infographic, added in NousResearch#47674 alongside the ChatPage.tsx fix)
- infographic/kanban-db-corruption-defense/infographic.png (re-added a
  directory NousResearch#30854 had explicitly removed, in NousResearch#30952)

Both are unreferenced decorative infographics, so removing them has no
effect on docs, website, or app builds. Removing the latter also clears
the stray top-level infographic/ directory that NousResearch#30854 had retired.

These blobs remain in history (the commits that introduced them are
already on main and bundled with real code, so they can't be dropped);
this just removes them from the working tree going forward.
…dead constants

Follow-up cleanup on the OpenViking setup path merged in NousResearch#48262:

- _write_ovcli_config now uses utils.atomic_json_write(path, data, mode=0o600)
  instead of the local _precreate_secret_file + write_text + chmod sequence.
  The shared helper (already used by honcho/mem0/supermemory/hindsight) writes
  via temp-file + fchmod(0600) + fsync + os.replace, so the ovcli.conf is
  written atomically (no half-written secret file on crash) and with no
  chmod-after-write TOCTOU window. _precreate_secret_file stays for the .env
  writer path.
- Remove dead _DEFAULT_ACCOUNT/_DEFAULT_USER constants (0 references; the
  empty->'default' tenant fallback lives in the _VikingClient constructor).

Tests: tests/plugins/memory/test_openviking_provider.py + test_memory_setup.py
+ openviking_plugin/test_openviking.py -> 130 passed; ruff clean.
…mic-json-write

refactor(openviking): reuse atomic_json_write for ovcli config; drop dead constants
`hermes update` keeps (won't overwrite) bundled skills the user edited
locally, but only printed a count — "~ N user-modified (kept)" — with no way
to learn which skills, or see what changed. Reverting already existed
(`hermes skills reset <name> [--restore]`); discovery and inspection did not.

Add two CLI commands (zero model-tool footprint), reusing the manifest
origin-hash that sync already maintains:

- `hermes skills list-modified [--json]` — list the bundled skills whose
  on-disk copy diverges from the last-synced origin hash (the exact test the
  sync loop uses to decide what to skip).
- `hermes skills diff <name>` — unified diff between the user's copy and the
  current bundled (stock) version, so the user can confirm what changed
  before reverting.

Both are mirrored as `/skills list-modified` and `/skills diff`. The
`hermes update` notice now points at `hermes skills list-modified`. Core
helpers `list_user_modified_bundled_skills()` and `diff_bundled_skill()` live
in tools/skills_sync.py alongside the existing reset logic.
Exercises the real sync pipeline (no mocked comparison logic): a pristine
synced skill is not flagged; an edited one is listed and diffed (modified +
added files); an unknown skill returns not-ok; and `reset --restore` clears
the modified state so revert and discovery stay consistent.
…iguate diff

Salvage follow-up to the cherry-picked feat/test commits:

- W1: the unpack/install update path in main.py printed the
  '~ N user-modified (kept)' notice without the new
  'hermes skills list-modified' hint that the git-pull path got.
  Mirror the hint to both sites so the count is actionable
  regardless of which update path runs.
- W2: 'hermes skills diff <name>' (bundled-vs-stock) now shares the
  verb with the gateway write-approval 'diff <id>'. The gateway
  handler's docstring + truncation message pointed users to
  '/skills diff <id>' on the CLI, which now resolves a bundled skill
  by that name instead. Point at the pending JSON file and note the
  two diff commands are distinct.
- Add an invariant test asserting every 'user-modified (kept)' notice
  in main.py carries the discovery hint (guards sibling drift).
…ls-list-modified-diff

feat(skills): find & diff user-modified bundled skills (salvage of NousResearch#47802)
The PR added helper-level tests for _trace_key but nothing exercised the
keys through the real hooks. This adds TestTurnTraceIsolation, which drives
on_pre_llm_request / on_post_llm_call across two turns of one gateway
session (task_id == session_id, unique turn_id, api_call_count reset per
turn) and asserts each turn opens its own root trace when the first turn
fails to finalize (tool-only final step). This test fails on the pre-fix
code (only one trace opened, turn 2 absorbed into turn 1) and passes with
the scoping fix.

Also pins the turn_id-over-api_request_id key precedence: the turn-scoped
post_llm_call carries no api_request_id, so it must still resolve to the
same key as the request-scoped hooks or finalization breaks.
… trim diff contract

Cleanup pass on the salvage (behavior-preserving):

- diff_bundled_skill now uses the existing _skill_file_list() helper
  instead of reimplementing the rglob/is_file/relative_to file-set
  enumeration inline (twice).
- Extract _is_tracked_user_modification(origin_hash, user_hash) and use
  it in BOTH the sync loop and list_user_modified_bundled_skills() so the
  'kept user edit' rule can't drift between the two sites.
- _read_text_for_diff -> _read_for_diff returns (bytes, text); the binary
  branch now compares the bytes it already read instead of re-reading
  both files from disk.
- Drop the unused 'user_present' key from diff_bundled_skill's return
  contract (no consumer or test ever read it).
- test_update_modified_notice: drop the brittle '>= 2 sites' count-floor
  so consolidating the two print paths into a shared helper stays a
  welcome refactor; keep the per-site 'count notice => discovery hint'
  invariant (still mutation-tested).
…-diff-cleanup

refactor(skills): dedupe file-listing + share user-modified predicate (follow-up to NousResearch#48286)
The turn- and api-scoped branches each repeated the same
task/session/thread fallback ladder with only the infix differing. Extract
the shared prefix into _scope_prefix so a future scope dimension touches one
ladder instead of three. The legacy branch still returns a bare task_id (not
the task: prefix) for backward compatibility, so it stays separate.

Output key strings are unchanged; a new test pins them across every
task/session/turn/api combination since the keys are matched across hooks
and any drift would silently break trace finalization.
Scoping the trace key by turn_id (the prior commit) fixed cross-turn
collisions but introduced a slow leak: _finish_trace only pops a key when a
turn ends cleanly (final response has content and no tool calls), so any
turn that is interrupted, ends on a tool call, or has empty final content
now leaves its uniquely-keyed entry in _TRACE_STATE forever. Previously the
constant per-session key was overwritten by the next turn, capping growth at
~1 entry per session.

Add an LRU cap (_MAX_TRACE_STATE) enforced by _evict_stale_locked, called
under _STATE_LOCK immediately before each insert. It evicts the
least-recently-updated entries (using the previously-dead last_updated_at
field) and ends their root span so nothing dangles. Regression test drives
50 non-finalizing turns against a cap of 8 and asserts the dict stays bounded
with the most-recent turns surviving.
The prior assertion `all("turn1" in k or "turn2" in k for k in keys)` was
weak on two counts: it passes vacuously when keys is empty (a regression
that lost all state would slip through), and after turn 2 finalizes only
turn 1 lingers, so it only ever inspected turn 1 anyway. Replace it with an
exact check that one key survives, it is turn 1, and turn 2 never merged
into it — the real isolation invariant the test name claims.
…trace-scope-salvage

fix(langfuse): scope trace state by turn/request ids (salvage NousResearch#47945)
Add infinitycrew39@gmail.com -> infinitycrew39 to AUTHOR_MAP so the
contributor audit resolves the two cherry-picked commits from the NousResearch#47945
langfuse trace-scope salvage (merged as NousResearch#48292) to a GitHub handle instead
of flagging them as an unmapped author email.
…map-infinitycrew39

chore(release): map infinitycrew39 author email
…r uv

The Windows installer's Install-Uv spawned the astral uv installer with a
hardcoded bare `powershell -ExecutionPolicy ByPass -c "irm .../uv | iex"`.
That name resolves only to Windows PowerShell, and only when its System32
directory is on PATH. Run under PowerShell 7+ (`pwsh`) — or any session where
`powershell` isn't on PATH — the spawn dies with "The term 'powershell' is not
recognized", and uv installation aborts (the installer then appears stuck).

Add Get-PowerShellHostExe, which prefers the absolute path of the host we're
already running in (PATH-independent), then falls back to powershell/pwsh via
Get-Command, then to the bare name. Install-Uv now invokes that resolved exe.
teknium1 and others added 16 commits June 19, 2026 12:38
Replays the RAGnos delta (5 ragnos/ shim files clean; 2 Photon files) onto the
v2026.6.19 release. The 3 Photon conflict hunks resolved: keep the timeout-aware
send error + record_sent_message; merge the per-call client (event-loop safety)
with the env-configurable PHOTON_SIDECAR_TIMEOUT (90s default); keep both sidecar
imports. adapter.py compiles. Merge + the live iMessage round-trip are
operator-gated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…emetry)

Observe-only by default; RAGNOS_GOVERNANCE_ENFORCE=1 blocks gated tools so they
route through the Hermes Hub. Sprint 2 of the realignment. Additive, in the
RAGnos-owned plugins/ragnos-governance/ surface; upstream core untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cooldown

Closes NousResearch#50185

Two independent gaps let a transient Photon/Spectrum upstream overflow
degrade message delivery and amplify gRPC pressure:

1. _is_retryable_error did not recognise Photon- or Envoy-specific error
   strings ("internal sidecar error", "upstream connect error",
   "reset reason: overflow"), so _send_with_retry fell through to the
   plain-text fallback immediately instead of backing off and retrying.

2. send_typing had no rate gate, so a burst of typing-indicator calls
   during an overflow event kept hitting the upstream gRPC connection and
   widened the failure window.

Fix:
- Add _PHOTON_RETRYABLE_PATTERNS with the three high-specificity Envoy /
  sidecar substrings and override _is_retryable_error on PhotonAdapter to
  check them after delegating to the base-class patterns.  base.py and all
  other adapters are untouched.
- Add a 5 s per-chat cooldown in send_typing backed by _typing_last_sent.
  stop_typing clears the entry so the next start after a completed turn
  fires immediately — only rapid consecutive starts without a stop are
  suppressed.
- Reduce PhotonAdapter._send_with_retry default max_retries from 2 to 1
  (single 2 s back-off check) — enough to confirm whether the Envoy
  circuit-breaker has opened, without adding unnecessary latency.

All changes are scoped to plugins/platforms/photon/adapter.py.
When the Node spectrum-ts sidecar process exited mid-session (crash,
OOM, upstream overflow escalation), _supervise_sidecar returned
silently — readline hit EOF, the log-pump loop broke, and nothing
notified the gateway. _inbound_loop entered an infinite retry loop
against a dead port, _running stayed True, and the adapter remained
in self.adapters with no path to self-recovery short of a manual
gateway restart.

Add a death-detection tail to _supervise_sidecar: after the log-pump
exits (EOF or exception), guard on _inbound_running to distinguish
unexpected death from a deliberate disconnect(). On unexpected exit,
call _set_fatal_error("SIDECAR_CRASHED", retryable=True) followed by
_notify_fatal_error() so the reconnect watcher picks up the platform
within 30 s and retries with exponential backoff (30 s → 300 s cap)
until the sidecar comes back up. All other platforms remain unaffected.

The _inbound_running guard is safe against races: disconnect() sets
_inbound_running = False before _stop_sidecar() cancels the supervisor
task. CancelledError is BaseException, not Exception, so it bypasses
the except clause and propagates normally — the detection block never
runs during a clean shutdown.
…tection

Follow-up for salvaged PR NousResearch#50256. Unit tests for the three behaviors:
retryable classification of Envoy/sidecar overflow strings, per-chat typing
cooldown with stop_typing reset, and the _supervise_sidecar crash-detection
path that raises a retryable fatal (and the clean-shutdown no-op).
…alate

spectrum-ts routes stream telemetry through @photon-ai/otel's createLogger,
which sends severity>=ERROR to console.error and WARN/INFO to console.log.
The two lines the health monitor keys off land on different channels:
log.error("stream persistently failing") -> console.error (caught), but
log.warn("stream interrupted; reconnecting") -> console.log (was missed).

The original interception patched console.error only, so the recovering->
degraded escalation counter never saw the interrupt bursts that are the
primary silent-inbound symptom. Verified live against spectrum-ts 3.1.0 +
@photon-ai/otel: 3 real log.warn('stream interrupted') calls now escalate
to degraded -> process.exit(75) -> adapter reconnect.

Adds a shared classifyStreamLog() fed by both console.error and console.log,
plus a regression test asserting both channels are intercepted.
Update the Photon platform plugin's Node.js sidecar from spectrum-ts
3.1.0 to 7.0.0, which splits the SDK into scoped `@spectrum-ts/*`
packages with `spectrum-ts` as the umbrella re-export.

- Bump exact pin in package.json/package-lock.json to 7.0.0
- Update mixed-attachments patch script to target the new
  `@spectrum-ts/imessage/dist/index.js` path and tab-indented output
- Rewrite test fixture to match v7.x mapper shape (tab-indented,
  `const ... = async` declarations, single-line builder calls) and
  point at `@spectrum-ts/imessage/dist/index.js`
- Update README upgrade guide to document the v5 package split and
  the postinstall patch validation step
- Update comments in cli.py and index.mjs to reference v5/v7 changes
v8 made `richlink` outbound-only; inbound rich links now arrive as
plain `text`. Remove the `getBalloonBundleId`/`toRichlinkMessage`
branches from the iMessage mapper patch and update the fixture,
lockfile, and README accordingly.
Populate `reply_to_message_id`, `reply_to_text`, and
`reply_to_is_own_message` on reaction events so the gateway injects
`[Replying to your previous message: "..."]` when the agent receives
a tapback.

The sidecar now extracts a capped text preview from the hydrated
reaction target (plain text and mixed group messages; null for
attachment/voice-only targets), emitting it as `targetText` in the
NDJSON reaction payload. The Python adapter reads this field and sets
the reply correlation fields on the `MessageEvent`.
…06-28

fix(photon): pull upstream v8 recovery fixes
Restores gateway/platforms/base.py and gateway/run.py to upstream
state so hermes-upstream-conformance passes.

The reply_to_is_own_message field is removed from MessageEvent. Instead
the Photon adapter prefixes reply_to_text with "your previous message: "
when handling tapbacks on bot messages. The generic gateway handler then
emits `[Replying to: "your previous message: ..."]` with no core edits
required.

Tests updated to assert the new reply_to_text encoding.
19 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

🔎 Lint report: fix/hermes-upstream-conformance-2026-06-29 vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 10861 on HEAD, 10139 on base (🆕 +722)

🆕 New issues (535):

Rule Count
unresolved-import 142
unresolved-attribute 123
invalid-argument-type 119
invalid-assignment 49
unsupported-operator 46
not-subscriptable 29
invalid-method-override 6
unresolved-reference 5
invalid-return-type 5
no-matching-overload 3
call-non-callable 2
unused-type-ignore-comment 2
not-iterable 2
unused-awaitable 1
invalid-parameter-default 1
First entries
tests/gateway/test_telegram_voice_v0_regressions.py:67: [invalid-argument-type] invalid-argument-type: Argument to function `GatewaySlashCommandsMixin._handle_voice_command` is incorrect: Expected `MessageEvent`, found `SimpleNamespace`
tests/hermes_cli/test_whatsapp_cloud_setup.py:308: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `str | None` and `str`
tests/gateway/test_multiplex_lifecycle.py:2: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
gateway/run.py:7236: [invalid-argument-type] invalid-argument-type: Argument to bound method `PairingStore.generate_code` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "whatsapp_cloud", ... omitted 19 literals] | set[Unknown]`
gateway/run.py:9685: [unresolved-attribute] unresolved-attribute: Object of type `object` has no attribute `pop`
tests/gateway/test_multiplex_lifecycle.py:16: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `None` in union `dict[str, Any] | None`
hermes_cli/dump.py:260: [invalid-assignment] invalid-assignment: Object of type `Literal["(unknown)"]` is not assignable to `Literal["0.17.0"]`
tests/tui_gateway/test_billing_rpc.py:14: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
hermes_cli/config.py:4957: [unresolved-attribute] unresolved-attribute: Attribute `items` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
plugins/cron/__init__.py:215: [unresolved-reference] unresolved-reference: Name `CronScheduler` used when not defined
tests/gateway/test_cron_fire_webhook.py:12: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
gateway/platforms/whatsapp_common.py:76: [unresolved-attribute] unresolved-attribute: Object of type `Self@_effective_reply_prefix` has no attribute `_reply_prefix`
gateway/slash_commands.py:2998: [invalid-argument-type] invalid-argument-type: Argument to function `query_session_listing` is incorrect: Expected `str | None`, found `Literal["local", "telegram", "discord", "whatsapp", "whatsapp_cloud", ... omitted 19 literals] | set[Unknown]`
tests/gateway/test_raft_adapter.py:8: [unresolved-import] unresolved-import: Cannot resolve imported module `aiohttp.test_utils`
tests/hermes_cli/test_whatsapp_cloud_setup.py:122: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["32"]` and `str | None`
tests/gateway/test_gateway_silence_tokens.py:48: [invalid-assignment] invalid-assignment: Object of type `(_key, _gen) -> Literal[True]` is not assignable to attribute `_is_session_run_current` of type `def _is_session_run_current(self, session_key: str, generation: int) -> bool`
tests/gateway/test_matrix_approval_reaction_fail_closed.py:58: [unresolved-attribute] unresolved-attribute: Unresolved attribute `EventType` on type `ModuleType`
tests/gateway/test_multiplex_credential_isolation.py:8: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
gateway/run.py:15357: [invalid-assignment] invalid-assignment: Object of type `def _event_callback_sync(event_type: str, context: dict[Unknown, Unknown]) -> None` is not assignable to attribute `event_callback` on type `(Any & ~None) | AIAgent`
tests/hermes_state/test_session_archiving.py:3: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/tools/test_discord_tool.py:640: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["discord_admin"]` and `str | list[str] | list[Unknown] | bool`
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
tests/gateway/test_restart_resume_pending.py:1653: [invalid-assignment] invalid-assignment: Object of type `bound method GatewayRunner._release_running_agent_state(session_key: str, *, run_generation: int | None = None) -> bool` is not assignable to attribute `_release_running_agent_state` of type `def _release_running_agent_state(self, session_key: str, *, run_generation: int | None = None) -> bool`
run_agent.py:1274: [unresolved-attribute] unresolved-attribute: Object of type `Self@_max_tokens_param` has no attribute `model`
gateway/platforms/telegram.py:1375: [unresolved-attribute] unresolved-attribute: Attribute `do_api_request` is not defined on `None` in union `Unknown | None`
... and 510 more

✅ Fixed issues (119):

Rule Count
invalid-argument-type 38
unresolved-attribute 34
unsupported-operator 14
unresolved-import 10
invalid-assignment 9
invalid-return-type 3
invalid-method-override 3
unknown-argument 2
call-non-callable 2
unresolved-reference 2
no-matching-overload 1
invalid-parameter-default 1
First entries
tests/hermes_state/test_resolve_resume_session_id.py:34: [unresolved-attribute] unresolved-attribute: Attribute `commit` is not defined on `None` in union `Connection | None`
gateway/channel_directory.py:87: [invalid-argument-type] invalid-argument-type: Argument to function `_build_from_sessions` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown]`
gateway/config.py:581: [invalid-argument-type] invalid-argument-type: Argument to bound method `PlatformRegistry.get` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown]`
gateway/run.py:11622: [invalid-argument-type] invalid-argument-type: Argument to bound method `PlatformRegistry.is_registered` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown]`
tests/cli/test_fast_command.py:484: [invalid-argument-type] invalid-argument-type: Argument to bound method `TestCase.assertIn` is incorrect: Expected `Iterable[Any] | Container[Any]`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 31 union elements`
acp_adapter/server.py:83: [invalid-assignment] invalid-assignment: Object of type `Literal["0.0.0"]` is not assignable to `Literal["0.16.0"]`
gateway/run.py:6283: [invalid-argument-type] invalid-argument-type: Argument to bound method `PairingStore._is_rate_limited` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown]`
gateway/run.py:11499: [invalid-return-type] invalid-return-type: Return type does not match returned value: expected `tuple[str, list[str]]`, found `str`
tests/tools/test_browser_console.py:341: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["record_sessions"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 31 union elements`
hermes_cli/config.py:4659: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 31 union elements`
hermes_cli/config.py:4649: [unresolved-attribute] unresolved-attribute: Attribute `items` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 31 union elements`
agent/auxiliary_client.py:3122: [unknown-argument] unknown-argument: Argument `base_url` does not match any known parameter of function `resolve_provider_client`
gateway/run.py:11263: [invalid-argument-type] invalid-argument-type: Argument to bound method `set.add` is incorrect: Expected `tuple[str, str, str | None]`, found `tuple[Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown], str, str | None]`
gateway/run.py:14215: [invalid-argument-type] invalid-argument-type: Argument to bound method `AIAgent.run_conversation` is incorrect: Expected `str`, found `Any | str | list[dict[str, Any]]`
tools/process_registry.py:1099: [invalid-argument-type] invalid-argument-type: Argument to bound method `ProcessRegistry._reconcile_local_exit` is incorrect: Expected `ProcessSession`, found `ProcessSession | None`
tests/gateway/test_matrix.py:985: [unresolved-import] unresolved-import: Cannot resolve imported module `mautrix`
tests/hermes_cli/test_mcp_reload_confirm_gate.py:33: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 31 union elements`
tests/tools/test_stage2_hook_build_tree_chown.py:92: [no-matching-overload] no-matching-overload: No overload of function `run` matches arguments
hermes_state.py:764: [unresolved-attribute] unresolved-attribute: Attribute `cursor` is not defined on `None` in union `Connection | None`
hermes_state.py:591: [unresolved-attribute] unresolved-attribute: Attribute `rollback` is not defined on `None` in union `Connection | None`
gateway/run.py:5989: [invalid-argument-type] invalid-argument-type: Argument to bound method `PlatformRegistry.create_adapter` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown]`
gateway/run.py:1579: [invalid-return-type] invalid-return-type: Return type does not match returned value: expected `str`, found `Literal["cli", "local", "telegram", "discord", "whatsapp", ... omitted 18 literals] | set[Unknown]`
gateway/authz_mixin.py:196: [invalid-argument-type] invalid-argument-type: Argument to bound method `PlatformRegistry.get` is incorrect: Expected `str`, found `Literal["local", "telegram", "discord", "whatsapp", "slack", ... omitted 17 literals] | set[Unknown]`
skills/red-teaming/godmode/scripts/parseltongue.py:475: [invalid-argument-type] invalid-argument-type: Argument to function `escape` is incorrect: Argument type `Sized` does not satisfy constraints (`str`, `bytes`) of type variable `AnyStr`
tests/agent/test_compression_concurrent_fork.py:85: [unresolved-attribute] unresolved-attribute: Attribute `execute` is not defined on `None` in union `Connection | None`
... and 94 more

Unchanged: 5156 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@github-actions

Copy link
Copy Markdown

🚨 CRITICAL Supply Chain Risk Detected

This PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging.

🚨 CRITICAL: Install-hook file added or modified

These files can execute code during package installation or interpreter startup.

Files:

setup.py

Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting.

@github-actions

Copy link
Copy Markdown

⚠️ MCP catalog security review required

This PR changes the bundled MCP catalog or MCP catalog installer code. MCP entries can define local commands that users later install into mcp_servers, so this needs explicit maintainer review before merge.

A maintainer should verify:

  • any new/changed optional-mcps/**/manifest.yaml command and args are expected,
  • stdio transports do not use shell+egress/exfiltration payloads,
  • git install refs are pinned and bootstrap commands are minimal,
  • requested env vars/secrets match the upstream MCP's documented needs.

After review, add the mcp-catalog-reviewed label and re-run this check.

Reverts the two immediate-reconnect edits in gateway/run.py (next_retry
without the +30 delay, and the early-break on _failed_platforms in the
idle sleep loop) so gateway/run.py matches the upstream merge-base and the
hermes-upstream-conformance gate passes.

Operator decision (Option A): accept upstream's ~30s retry delay. Fast
Photon sidecar-crash recovery is still provided by the adapter's own
sidecar-death watcher (plugins/platforms/photon/adapter.py) plus the
30s->300s reconnect backoff, so recovery is preserved, just not instant
on the first retry.

Updates test_retryable_runtime_error_reconnects_immediately ->
test_retryable_runtime_error_queued_with_retry_delay to assert the 30s
queueing. 59 gateway/photon tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

🚨 CRITICAL Supply Chain Risk Detected

This PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging.

🚨 CRITICAL: Install-hook file added or modified

These files can execute code during package installation or interpreter startup.

Files:

setup.py

Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting.

@github-actions

Copy link
Copy Markdown

⚠️ MCP catalog security review required

This PR changes the bundled MCP catalog or MCP catalog installer code. MCP entries can define local commands that users later install into mcp_servers, so this needs explicit maintainer review before merge.

A maintainer should verify:

  • any new/changed optional-mcps/**/manifest.yaml command and args are expected,
  • stdio transports do not use shell+egress/exfiltration payloads,
  • git install refs are pinned and bootstrap commands are minimal,
  • requested env vars/secrets match the upstream MCP's documented needs.

After review, add the mcp-catalog-reviewed label and re-run this check.

@ragnos-dev ragnos-dev changed the title fix(hermes): move own-message reply context into Photon adapter fix(hermes): restore gateway core to upstream, clear conformance gate Jun 29, 2026
@ragnos-dev ragnos-dev added the mcp-catalog-reviewed Maintainer reviewed MCP catalog changes label Jun 29, 2026
@github-actions

Copy link
Copy Markdown

🚨 CRITICAL Supply Chain Risk Detected

This PR contains a pattern that has been used in real supply chain attacks. A maintainer must review the flagged code carefully before merging.

🚨 CRITICAL: Install-hook file added or modified

These files can execute code during package installation or interpreter startup.

Files:

setup.py

Scanner only fires on high-signal indicators: .pth files, base64+exec/eval combos, subprocess with encoded commands, or install-hook files. Low-signal warnings were removed intentionally — if you're seeing this comment, the finding is worth inspecting.

@ragnos-dev
ragnos-dev merged commit 652f281 into main Jun 29, 2026
35 checks passed
@ragnos-dev
ragnos-dev deleted the fix/hermes-upstream-conformance-2026-06-29 branch June 29, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mcp-catalog-reviewed Maintainer reviewed MCP catalog changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.